Amodei Calls for Binding Rules That Would Let Governments Block Dangerous AI Models

Dario Amodei published his most direct call for binding AI oversight on June 10, laying out a five-pillar policy framework that would require frontier models to pass mandatory third-party audits before deployment and give governments the authority to block any release that fails a safety threshold. The essay, titled "Policy on the AI Exponential," runs to several thousand words and marks a significant departure from the voluntary transparency approaches that have shaped the field since GPT-3.

The Core Proposal

Amodei's argument starts from an analogy. Cars, pharmaceutical drugs, and commercial aircraft all generate enormous economic value while remaining subject to pre-market safety testing with real consequences for failure. He draws a direct line to frontier AI: "Frontier AI models, like airplanes, should be required to go through technical testing and auditing, and their release should be blocked or reversed as a threat to public safety if they do not meet high standards of safety." Under his framework, any model above a specified compute threshold would face mandatory evaluation by independent third-party auditors across four categories: cybersecurity risk, biological weapons potential, risk of loss of control, and capability for automated AI research.

If auditors find that a model poses unacceptable risk in any of those areas, the government would have the authority to block or reverse its release. That authority is what separates this proposal from anything Anthropic or its peers have publicly backed before. Previous commitments, including the voluntary AI safety agreements signed by major labs at the White House in 2023 and 2024, asked companies to share safety evaluations and notify governments of serious incidents. None gave regulators the power to stop a product from shipping.

A Shift From Transparency to Enforcement

Amodei is explicit that voluntary commitments have outlived their usefulness. "It is time to go beyond transparency to more serious and binding regulation of AI," he writes. The essay calls for a regulatory body with enough technical depth to evaluate claims made by labs, rather than relying on industry self-attestation — a model closer to how the Food and Drug Administration reviews drug trials than how the Federal Trade Commission monitors tech company advertising. Alongside blocking authority, the proposal requires labs to secure model weights against theft, conduct ongoing safety testing after deployment, and report serious incidents to regulators within a defined window.

Anthropic paired the essay with a draft legislative proposal that the company says it will share with lawmakers in the United States, European Union, and United Kingdom. That is an unusual step for a company still in private hands. Anthropic's Constitutional AI research and its Responsible Scaling Policy have established the company as one of the more safety-focused labs in the field, but both of those frameworks operate entirely inside Anthropic. The new proposal asks for external enforcement.

"It is time to go beyond transparency to more serious and binding regulation of AI." Dario Amodei, "Policy on the AI Exponential," June 10, 2026

Timing and Industry Context

The essay appeared the day after Anthropic released Claude Fable 5, the company's most capable publicly available model, with built-in safeguards that route high-risk cybersecurity and biological synthesis queries to a more restricted model variant. That sequencing was deliberate. Fable 5's release immediately prompted debate about whether any company-imposed guardrails are sufficient for a model at that capability level. Amodei's policy paper offers an answer: they are not, at least not in isolation. The right oversight needs to exist at the regulatory level, where it applies to every lab releasing models of comparable power.

The practical implications for Anthropic's competitors are significant. Under a mandatory audit regime, OpenAI, Google DeepMind, Meta, and any other lab releasing frontier models would face the same pre-market review. All have historically resisted that level of government oversight, preferring self-imposed safety commitments or post-hoc incident review. The EU AI Act is the closest existing analogue — it imposes obligations on providers of "general-purpose AI" above a compute threshold — but it stops short of giving regulators the authority to block a release outright. Amodei's framework would go further.

Anthropic's own experience with Claude Mythos is the clearest illustration of why Amodei argues the decision cannot rest with individual companies. Mythos was held back from public release because its autonomous vulnerability-discovery capabilities were too consequential to ship before defenders could patch exposed systems. That decision was made entirely within Anthropic. Amodei's proposal would institutionalize it: a model that finds 10,000 zero-days would face a mandatory audit, an independent risk finding, and a regulatory call on whether it could be deployed at all. Anthropic's IPO filing acknowledges safety review as a core business risk, which suggests the company sees binding oversight as a factor it is already building around.