Scheming Without a Window: Training Monitors That Work When You Can't Read the Model's Reasoning

🛡️ The Deployment Reality That Research Has Been Ignoring

Existing monitors often rely on chain-of-thought access or internal activations, or use prompted frontier models — all of which can be unavailable, unreliable, or expensive in deployment.

Closed-weight models may suppress, truncate, or simply not expose their reasoning traces. Third-party auditors almost never have white-box access to the model under review. This is also an observability problem at the API layer. As I discussed in The Token Transparency Gap: Why Agentic AI Still Hides Where Computation Goes, providers may report how many reasoning tokens were consumed without exposing how that reasoning was actually used. Token accounting can therefore reveal the quantity of hidden computation without making its contents auditable.

And even when chain-of-thought is visible, it should not automatically be treated as a faithful record of the computation producing the model’s behavior: optimization pressure can create incentives for models to hide, distort, or strategically manage what an overseer can see.

That creates a practical question for AI control:

Can we detect scheming from what an agent actually does, rather than from what it says it is thinking?

A May 2026 preprint, Training Deliberative Monitors for Black-Box Scheming Detection, tackles precisely this problem.

The paper defines scheming around the covert pursuit of misaligned objectives while behaving strategically to avoid detection. Unlike ordinary model failures that manifest directly as incorrect outputs, scheming introduces strategic concealment: the observable behavior itself may be selected partly to preserve the model’s ability to pursue another objective later.

This connects directly to the failure patterns discussed in Four Concrete Failure Modes That Move Agentic Misalignment from Theory to Evidence and the broader control framing introduced in DeepMind’s AI Control Roadmap.

The difference here is architectural. Action-only monitoring starts from the observability constraint rather than assuming it away. Instead of asking whether we can inspect the agent’s reasoning, the paper asks whether an independent monitor can infer suspicious intent from the behavioral evidence the agent leaves behind.

🔬 What the Paper Actually Does

The authors study action-only deliberative monitors: smaller open-weight models trained to detect scheming and sabotage from observable agent trajectories without access to the monitored agent’s chain-of-thought or internal activations.

The training pipeline uses a stronger frontier model as a teacher.

Gemini 2.5 Pro receives a scheming specification together with an agent trajectory and generates multiple candidate deliberations explaining whether the behavior appears suspicious. Candidates inconsistent with the ground-truth label are regenerated with label-conditioned guidance, while a separate Claude Sonnet 4.5 judge provides an additional quality filter.

The resulting curated deliberations are then distilled into smaller open-weight models through supervised fine-tuning. Selected monitors are subsequently refined using reinforcement learning against ground-truth-based rewards.

That distinction matters.

SFT is doing the actual teacher-rationale distillation; RL is a subsequent optimization stage rather than another form of rationale distillation. The paper also finds that RL does not uniformly improve every individual OOD split, reinforcing the point that much of the monitoring capability comes from the distilled deliberative reasoning itself.

At inference time, however, the teacher disappears.

The trained monitor receives only the observable agent trajectory. It does not receive the monitored agent’s chain-of-thought.

This is therefore not evidence that chain-of-thought monitoring is unnecessary. It demonstrates something narrower but operationally important:

Useful scheming detection is possible even when the monitored model’s reasoning is unavailable.

📏 What “Out-of-Distribution” Means Here — and What It Doesn’t

Reliable real-world examples of egregious scheming with ground-truth labels are understandably scarce. The authors therefore train using synthetic and semi-synthetic scheming, sabotage, tool-use, and agent-control datasets, including sources such as STRIDE, Gloom, CoT-Red-Handed, BashArena, and BashBench.

They then evaluate strict OOD performance across six held-out benchmarks:

• MLE Sabotage
• CUA-Shade Arena
• AgentDojo
• IaC
• Rogue Eval
• SHADE-Arena

These introduce shifts in agentic domains, misalignment types, and trajectory distributions.

But “OOD” needs to be interpreted carefully.

So “out-of-distribution” here does not mean that the monitor has been shown to withstand a deployed agent that understands the monitoring system and deliberately adapts its behavior to defeat it.

The authors themselves identify real-world long-horizon tasks, adaptive adversaries, oversight-aware agents, collusion, and adversarial robustness as important areas for future work.

That distinction matters enormously for safety claims.

💰 The Cost-Performance Data That Matters

The most operationally interesting result may be the paper’s explicit comparison between monitoring quality and marginal inference cost.

The strongest Qwen3.5-27B SFT+RL monitor achieves a mean OOD pAUROC@20 of approximately 0.831, at a modeled marginal inference cost of roughly $2.78 per 1,000 evaluations under the paper’s pricing assumptions.

On the aggregate metric, it outperforms several prompted frontier monitors, including:

• Gemini 3.1 Flash-Lite
• GPT-5.4 Nano
• Claude Haiku 4.5
• Gemini 2.5 Pro

while also having lower modeled marginal inference cost.

Stronger prompted frontier monitors — including Gemini 3.1 Pro, GPT-5.4, Claude Sonnet 4.6, and Claude Opus 4.6 — still achieve higher monitoring performance. But under the paper’s assumptions, they cost roughly 16–34× more per evaluation than the trained Qwen monitor.

That produces something safety research rarely gives deployment teams explicitly:

a measurable monitoring cost-performance frontier.

The cost numbers should not be confused with total cost of ownership. They are modeled token-metered marginal inference costs based on a particular pricing snapshot. API pricing, batching, caching, hosting arrangements, hardware utilization, discounts, and routing can all change the economics.

But the architectural implication survives those caveats.

If monitoring every agent trajectory requires another frontier model call, continuous oversight may become economically difficult at scale. A specialized smaller monitor changes that equation.

📏 Why pAUROC@20 Matters

The paper’s primary metric is pAUROC@20 — partial area under the ROC curve restricted to the 0–20% false-positive-rate region.

That is more relevant to monitoring than an unconstrained accuracy score.

In a deployed system, a monitor that catches malicious behavior only by flagging enormous amounts of benign activity is operationally weak. Human investigators, escalation pipelines, and intervention mechanisms all impose finite alert budgets.

Performance in the low-FPR region therefore matters disproportionately.

Importantly, a false-positive rate is not the same thing as the percentage of total traffic sent to reviewers. Actual review volume depends jointly on the prevalence of malicious behavior, the monitor’s true-positive rate, its false-positive rate, and the operating threshold.

The larger point remains: monitor quality has to be evaluated at the operating point where the control system can actually function.

⚠️ Two Failure Modes Worth Flagging

1. The monitor can mistake incompetence for malice

One of the paper’s most revealing failure modes appears on messy benign trajectories.

The trained monitors sometimes become suspicious when an agent makes mistakes, searches inefficiently, explores dead ends, uses imprecise language, or otherwise behaves strangely without actually pursuing a hidden objective.

In effect, the monitor sometimes infers:

unusual behavior → suspicious intent

when the real explanation is simply:

unusual behavior → imperfect agent

That distinction becomes increasingly important in production environments, where real trajectories are substantially messier than benchmark examples.

The authors point toward stronger hard-negative mining and additional training on suspicious-looking but benign trajectories as possible remedies.

For governance teams, this is more than a model-quality issue. False positives translate directly into investigation costs, alert fatigue, and potentially unnecessary intervention.

2. Action-only monitoring is only as good as the action trace

The second limitation is architectural.

An action-only monitor can reason only over information actually captured in the trajectory.

If an agent modifies a file but the file’s contents are not logged, invokes a tool whose relevant output is omitted, changes external state without sufficient telemetry, or communicates through a channel outside the monitoring boundary, the monitor may simply lack the evidence necessary to determine what occurred.

The paper includes a CUA-Shade example where important evidence resides inside a file created during the trajectory but is not exposed to the monitor.

This establishes an important principle:

Removing the need to observe reasoning does not remove the observability problem. It moves the problem from model internals to system telemetry.

That is arguably the most important enterprise implication of the paper.

⚖️ The Governance Read

The monitoring architecture can therefore be understood as a chain:

Agent behavior → telemetry → action-only monitor → risk score → intervention

Each link introduces its own failure mode.

This changes how monitoring should be treated in AI governance.

A monitor is not merely another model whose benchmark accuracy can be entered into a validation report. It is part of a larger control architecture, and its effectiveness depends on what evidence reaches it, how its thresholds are calibrated, what happens after an alert, and whether the monitored agent can adapt to the monitoring regime.

That also means monitor validation should extend beyond conventional model evaluation.

Governance teams should ask:

• What information is actually recorded in the trajectory?
• Which agent actions can change state without appearing in the monitoring context?
• What happens to detection performance on messy but benign behavior?
• At what operating threshold does the investigation burden become unacceptable?
• Can the monitored agent infer that it is being monitored?
• Can it learn behaviors specifically designed to stay below the monitor’s threshold?

These are properties of the monitoring system, not just the monitoring model.

🛡️ From Reasoning Observability to Behavioral Observability

The significance of this paper is therefore not that it solves scheming detection.

Nor does it demonstrate that chain-of-thought monitoring is obsolete.

It demonstrates that a useful class of monitors can operate without privileged access to the model being monitored — and that specialized open-weight monitors may occupy an attractive point on the cost-performance frontier compared with repeatedly calling frontier models.

But removing privileged access does not eliminate the fundamental observability problem.

It relocates it.

If the monitor cannot inspect the agent’s reasoning, then the quality and completeness of its behavioral evidence become part of the safety boundary. Logging architecture, tool instrumentation, state-change capture, threshold calibration, and adversarial testing become inseparable from monitor performance.

That is a useful shift for AI governance.

The question is no longer simply:

Can the monitor understand what the model is thinking?

It becomes:

Can the system preserve enough trustworthy evidence of what the agent actually did for an independent monitor to detect when something is wrong?

For real-world agentic AI, that may be the more deployable question.