What AI Companies Can Learn from the Oversight Board

Kenji Yoshino and Ronaldo Lemos are members of the Oversight Board.

Will frontier AI companies learn the lessons social media companies did before disaster strikes, or after? That is the question Dario Amodei posed last month in a blog post that called on tech companies to address the existential risks posed by AI. He argued for a two-pronged response: legislation, which is essential but slow, and industry self-governance, which is less forceful but more nimble.

Anthropic is positioning itself as a current leader in AI governance and has actively worked to differentiate itself in the market as the ‘ethical’ AI option, yet Amodei is right that “the industry should continue to explore mechanisms that go further” and Anthropic itself can do more.

Based on our experience as members of the Oversight Board—the largest experiment in independent platform oversight to date which was launched by Meta in 2019 in response to several crises—we offer some thoughts on what “more” should look like.

We agree that legislation is necessary, but insufficient. Regulation will set baseline rules and enforcement mechanisms essential to combat AI’s threats. Yet as Amodei notes, legislation is notoriously slow. He opens his essay by comparing responsible AI advocates talking to regulators to the hobbits in the Lord of the Rings talking to Treebeard, the oldest creature in Middle Earth, who takes “a full day simply to say hello to another tree.” The mismatch is structural, not temporary. By the time legislators have responded to one generation of technology, companies have moved on to the next. Even with thoughtful laws in place, institutions are needed to respond with more dispatch.

We believe that effective oversight must have at least three components. First, oversight requires a credible body of overseers who are both independent and representative. The independence must be decisional—meaning the body must be able to make decisions that are binding on the company, and require public responses to its non-binding recommendations. It must also have financial independence such that the company cannot control its decisions by clawing back funding or penalizing members for decisions they don’t like. The oversight body must have the independence to control its own membership. And critically, the body must be drawn from diverse geographies and cultures, given that AI, like social media, presents borderless challenges that require legitimacy across regions.

Second, meaningful oversight requires a credible, external body of standards that are beyond the company’s power to change out of day-to-day convenience. These could include company-generated standards to which the company has made pre-commitments, like the community standards created by Meta, a charter, or a constitution. But it could also be—as in the Oversight Board’s case—a reliance on international human rights law, which offers an external set of values that stand entirely outside the company itself and that have already been accepted around the globe.

Finally, effective oversight requires transparency in what the decisions are, how they are made, and how they are responded to by the company. Much has been written about the importance of open-source code; what we need here is open-source governance. Publishing substantive decisions creates precedent, allows for critique, and ensures accountability to the public. Clarity about how decisions are made and how institutions are structured demonstrates that an institution has avoided capture and shows others what works.

Anthropic has done more than many of its peers to ensure independent accountability. Amodei offers up the example of Anthropic’s “Long-Term Benefit Trust” (an independent governance body designed to hold the company to its mission) as an instance of such accountability. The trust is genuinely innovative, and trust-appointed directors now hold a board majority. Anthropic has also adopted a public benefit corporate form, which creates a legal duty to balance mission against profit. In addition, Anthropic has recently published a “constitution” for Claude that sets forth principles for safe AI development.

Nevertheless, Amodei is also correct that more is needed here. While Anthropic has been vocal about transparency, the trust operates in near-total opacity. Its governing agreement is not public, its trustees serve only one-year terms, and shareholders retain little-understood powers to override trust decisions. Similarly, as our colleague Suzanne Nossel has written, the Anthropic constitution has no enforcement mechanism that would give it any bite, nor does it recognize the rights of Claude’s users.

In exploring what meaningful and effective oversight looks like, we urge frontier AI companies to consider independent bodies that take specific decisions and publish reasoned opinions regarding alignment with stated values. Those bodies should have, at a minimum, irrevocable funding; members with fixed, staggered terms removable only for cause; global representation; published, reasoned decisions; a defined remit that is adaptable to the company’s dynamic challenges and duty to act; and a clear distinction between binding decisions and advisory recommendations, both requiring public company response.

We underscore, as we must, that the Oversight Board remains a work in progress. We are far from perfect. But we have now issued 200-plus decisions (where Meta has been bound by our leave-up/take-down directives) and 300-plus recommendations (with Meta adopting, by the Board’s count, roughly 75 percent of them either whole or in part). And while content moderation is different from AI governance, many of the structural principles relating to the constraint of private power are eminently portable. While the technical questions differ, the governance problem of how to constrain private actors exercising immense power over the public is fundamentally the same.

This governance problem is approaching critical urgency in the AI context. As Amodei notes, the potential harms of advanced AI include cybersecurity attacks on critical infrastructure, biological weapons, and autonomous systems that evade democratic oversight. Those risks demand governance mechanisms capable of addressing challenges that are both technically complex and moving faster than traditional regulatory processes. If Meta’s content decisions warranted this level of independent oversight, AI’s architectural decisions demand at least as much accountability—and demand it before, not after, a catastrophe occurs.