GovAI's Embedded Assessments Paper Turns CEO Safety Pledges Into an Oversight Blueprint

🔍 From pledge to blueprint

On September 21, 2026, the Centre for the Governance of AI published Embedded Assessments for Frontier AI, a paper that turns a fast-moving industry idea into something closer to an oversight architecture.

The authors — Jacob Charnock, Sophie Williams, Zaheed Kara, Markus Anderljung, Alejandro Tlaie Boria, Stephen Casper, Anka Reuel, and Jonas Freund — recommend that frontier AI developers begin hosting embedded assessments now. They identify three particularly important areas for managing risks from internal AI use: internal agent monitoring, internal agent security controls and permissions, and model alignment.

The paper also recommends continuous access and detailed public reporting at least quarterly.

The timing matters.

On September 12, Anthropic CEO Dario Amodei published “We Must Pace the Frontier”, proposing that frontier AI companies give third-party evaluators ongoing, employee-like access to their organizations. His proposal was more concrete than a generic promise of outside auditing: evaluators could receive badges, laptops, office space, access to employees, internal documentation, and risk-assessment systems, while retaining substantial freedom to publish their findings.

OpenAI CEO Sam Altman quickly said OpenAI would make a similar commitment.

GovAI’s contribution is therefore not inventing embedded evaluation. It is asking the harder question:

What rules make an embedded evaluator meaningful once you let one through the door?

🏢 The first embedded evaluator is already moving in

That question stopped being theoretical almost immediately.

On September 18, Anthropic announced an embedded-evaluation partnership with Accenture, led by Accenture’s specialist AI business, Faculty.

The arrangement moves Amodei’s proposal from principle toward implementation. The partnership is intended to support model evaluation, alignment assessment, and safeguard testing, while Anthropic and Accenture each expect to invest at least $1 billion over five years in building evaluation capacity.

But the announcement also exposes the governance problem.

An evaluator can have extraordinary access while still operating through a commercial relationship with the company being evaluated.

That produces a distinction worth keeping clear:

Operational access ≠ institutional independence.

Who pays the evaluator? Who determines its scope? Can the evaluator investigate something the developer would rather exclude? Can it publish an adverse finding? What happens if it discovers an unacceptable risk?

Those questions cannot be answered by giving someone a badge and a laptop.

📋 Seven questions behind one simple word

GovAI’s most useful contribution is converting “embedded evaluation” into seven design questions covering:

• scope;
• information gathering;
• duration;
• timing;
• terms of engagement;
• disclosure; and
• escalation.

That is a much better way to evaluate these programs than asking whether a company “has independent evaluators.”

Two companies could both make that claim while operating radically different oversight regimes.

One evaluator might receive continuous access to internal agent traces, security controls, employees, training processes, and incident records, publish findings independently, and escalate serious concerns.

Another might review a predefined set of systems, under confidentiality restrictions, for a limited period, with findings routed privately to management.

Both are technically “embedded.”

They are not equivalent governance mechanisms.

This is the same specification problem this site has encountered elsewhere. Apollo Research’s Auto-Mode audit showed why continuous agent monitoring requires examining what happens during execution, while California’s auditor registry raised the separate question of who qualifies to perform independent AI assurance.

And as argued in Agentic AI Has Outrun the Governance Playbook, governance designed for systems that predict does not automatically tell us how to supervise systems that act continuously inside an organization with real permissions.

Embedded assessment is one attempt to close that gap.

☢️ Why nuclear inspectors are the more revealing analogy

Amodei explicitly invoked banking supervision, noting that regulators sometimes place supervisors inside financial institutions.

GovAI also examines institutional precedents that help explain why physical and organizational proximity matters.

The clearest is nuclear power.

The NRC’s resident inspector program has stationed inspectors at U.S. nuclear plants for decades. Resident inspectors do not merely arrive periodically to inspect a finished product. They observe an operating institution continuously enough to understand its systems, procedures, personnel, and emerging problems.

That is much closer to the governance challenge frontier AI increasingly presents.

But the analogy also reveals what today’s AI arrangements lack.

NRC inspectors are government regulators backed by statutory authority. Embedded AI evaluators are currently third parties operating largely through contractual arrangements.

Access and authority are not the same thing.

An embedded evaluator may discover a dangerous practice. A regulator can potentially compel corrective action. A private evaluator may only be able to report it.

That difference becomes especially important when assessments uncover problems management does not want to address.

⚠️ Assessment is not enforcement

The distinction creates three increasingly powerful layers of oversight.

Model evaluations ask what a model can do and how it behaves under particular tests.

Embedded assessments can observe how models are trained, monitored, secured, deployed, and used inside the institution operating them.

Regulatory supervision can add something neither evaluation layer inherently possesses: legal authority.

GovAI is primarily trying to strengthen the middle layer.

That matters because many emerging agentic risks are not properties of the model alone. They arise from the combination of the model, its tools, permissions, monitoring systems, deployment environment, and organizational processes.

A pre-deployment benchmark cannot fully observe that system.

An evaluator sitting inside the organization potentially can.

But seeing a problem does not automatically create the power to stop it.

🏛️ California could change that equation

The next step may already be forming.

California is considering whether independent oversight should move beyond voluntary arrangements. Governor Gavin Newsom’s September executive action directed state officials to examine mechanisms including independent verification organizations operating inside frontier AI developers and stronger mechanisms for responding to severe AI risks.

That does not mean California has mandated embedded evaluators.

But it establishes a potentially important progression:

Voluntary industry commitment → operational standards → regulatory mandate

The distinction matters.

If embedded assessment remains voluntary, companies largely determine who enters, what they see, how long they stay, and what happens after they report a problem.

If governments eventually define minimum access, independence, disclosure, and escalation requirements, embedded assessment starts looking less like contracted assurance and more like supervision.

🎯 The real governance question is what happens after access

The embedded-evaluator debate initially sounds like an access problem.

Frontier labs possess information outsiders cannot see, so independent experts need to get inside.

GovAI’s paper shows why that is only the beginning.

The harder questions concern scope, independence, disclosure, and escalation. Anthropic’s Accenture partnership makes those questions immediate rather than hypothetical, while California’s interest raises the possibility that today’s voluntary experiment could become tomorrow’s regulatory architecture.

The important question is therefore no longer simply:

Will frontier AI companies let independent evaluators inside?

At least one major lab is already moving in that direction.

The better question is:

Once evaluators are inside, who determines what they can see — and what happens when they find something the company does not want to hear?

That is the difference between embedded evaluation as assurance and embedded evaluation as oversight.