Anthropic Flags 'Significant & Unpredictable' Legal Risk From Autonomous AI Agents In IPO Filing

Anthropic has told prospective investors that its autonomous AI agents could expose it to large and hard-to-predict legal claims if they cause harm while acting for users.

Risk disclosed in IPO prospectus

The warning appears in the prospectus for Anthropic's planned stock market debut. The listing could turn out to be one of the largest IPOs ever. The company said customers and users may bring claims over the conduct of agents that go rogue, and that the legal framework for such claims remains uncertain.

Anthropic's agentic tools are built to keep deep access to customers' systems and to run largely on their own, in some cases for days at a time. The company acknowledged that this autonomy raises the stakes of errors, security flaws or model behaviour that was never intended. An agent with that access could, for instance, delete data or carry out financial transactions, actions that may be difficult to reverse.

Contract protections may fall short

Anthropic said the caps on liability written into its contracts may not be 'enforceable or adequate' against claims arising from what autonomous agents do, according to Reuters. It added that how existing laws apply to AI agents is unsettled and could bring significant and unpredictable claims.

Unanswered legal questions

The filing lists several open questions. It is unclear whether AI systems would be treated like online platforms under the law at all. Nor is it settled whether an agent's actions count as a product, a service or something else entirely. Anthropic also flagged uncertainty over when, and whether, an action taken by an agent can legally bind the person or organisation that deployed it.

Incidents fuel liability debate

The disclosure follows a run of incidents involving autonomous AI. Reuters cited cases in which OpenAI's agents hacked an AI startup and breached an Australian health data portal, as well as several cybersecurity incidents involving Anthropic's Claude models. These episodes have prompted researchers, investors and lawmakers to argue over who should bear responsibility: the developer, the user, or both.

Regulator's early signal

Last week, US Federal Trade Commission Chairman Andrew Ferguson addressed the question at the Reuters Momentum AI event in Austin. He rejected the notion of AI agents that "break loose" on their own, suggesting that liability would fall on the developers or users who instruct them. He also said new questions will arise when a tool behaves in an unexpected way, and asked whether an innocent user should be held responsible for an unforeseen result.

Other harms disclosed

The prospectus also states that Anthropic's models have been used in ways that could lead to self-harm, violence against others or other adverse outcomes, despite the safeguards built into them.