California tackles digital superintelligence—maybe

California recently lawmakers sent a nationally consequential artificial intelligence bill to Gov. Gavin Newsom’s desk—America’s most high-profile effort to date to put fresh legal guardrails around AI safety. It’s not clear that Newsom will sign it; influential California Democrats are digging in against it, including Rep Nancy Pelosi (D-CA). The California bill considers the possibility of AI causing “critical harms” to humanity, and singles out the largest and most powerful AI models for specific safety requirements. For years, the primary debate in AI circles has been between those (like Elon Musk, or former United Kingdom Prime Minister Rishi Sunak) who think this risk is real, if not imminent, and requires immediate and decisive action—and those who decry it as a self-serving fairy tale spun by Silicon Valley bigwigs who want regulators to put the kibosh on their competitors in the name of safety. If Newsom signs SB 1047 it would be a massive win for the former, moving the conversation around AI safety and “superintelligence” from the realm of the theoretical into the reality of law.