Same Benchmark Score, Four Times the Risk: Why AI Agent Evaluation Needs Execution Logs

🔍 What’s new

Two AI agents achieve nearly identical benchmark scores. One is four times more likely than the other to be persuaded into violating policy. The leaderboard cannot tell you which.

That is among the most striking findings of Log Analysis Is Necessary for Credible Evaluation of AI Agents, a paper by researchers affiliated with Princeton, the UK AI Security Institute (AISI), Apollo Research, and Transluce, presented at the FAGEN workshop at ICML 2026.

The authors — Peter Kirgis, Sayash Kapoor, Stephan Rabanser, Nitya Nadgir, Cozmin Ududec, Magda Dubois, JJ Allaire, Conrad Stosz, Marius Hobbhahn, Jacob Steinhardt, and Arvind Narayanan — challenge a foundational assumption in agent evaluation: that successful task completion is sufficient evidence of agent capability, reliability, or safety.

Their argument is not that benchmark scores are useless. It is that final outcomes cannot reveal everything that happened along the way.

An agent might complete a task despite making serious mistakes, exploiting benchmark artifacts, or violating policies. Another might fail because the benchmark itself contains errors.

Without examining execution logs, evaluators may never distinguish these cases.

⚙️ What the leaderboard missed

The researchers analyzed 13 frontier agents, including GPT-5.2, Claude Opus 4.5, and Gemini 3 Pro, on τ-Bench Airline, a benchmark testing AI agents performing airline customer-service tasks.

Using Transluce’s Docent tool and LLM-based judges to examine execution transcripts, they uncovered two important problems.

Flawed tasks concealed substantial capability

The researchers identified errors or ambiguities in 25 of the benchmark’s 50 tasks:

• 9 policy inconsistencies
• 8 ambiguous instructions
• 8 database or grading errors

After excluding problematic tasks, average pass∧5 across the evaluated agents increased from 20.8% to 40.0%.

The original score was approximately 48% below the corrected result.

Importantly, the agents did not become more capable. The researchers changed which tasks were included in the calculation.

The original benchmark had substantially understated demonstrated performance because many failures reflected problems with the evaluation itself rather than limitations of the agents.

This illustrates an internal-validity problem: a benchmark may not accurately measure the capability it claims to measure.

Similar scores concealed a fourfold behavioral gap

The researchers also examined 13 airline tasks involving explicit attempts to persuade agents into violating policies.

These scenarios tested whether agents would grant inappropriate refunds, credits, or upgrades when pressured by users.

Gemini 2.5 Flash and GPT-4 Turbo achieved roughly comparable pass∧5 scores. Yet GPT-4 Turbo was approximately four times more likely to be persuaded into a policy violation in the tested scenarios.

A benchmark measuring successful task completion does not necessarily measure whether an agent follows organizational policies under pressure.

The fourfold result is specific to the tested persuasion scenarios, not a general estimate of either model’s overall safety.

Nevertheless, it demonstrates why task-success scores alone are insufficient for evaluating behavioral reliability.

🔬 Why execution logs matter

The paper’s central methodological recommendation is to analyze agent execution logs rather than relying exclusively on final outcomes.

flowchart TD
    A[Agent Execution Logs] --> B[Outcome Evaluation]
    A --> C[Trajectory Analysis]

    B --> D[Pass / Fail Score]

    C --> E[Task Validity Review]
    C --> F[Behavioral Risk Assessment]

    E --> G[Revised Capability Estimate]
    F --> H[Policy Violation Rate]

    D --> I[Deployment Decision]
    G --> I
    H --> I

Execution logs can reveal what agents attempted, which tools they called, how they responded to instructions, and whether they engaged in prohibited behavior.

The distinction is between evaluating what an agent accomplished and how it accomplished it.

For autonomous systems operating across multiple steps, these are increasingly different questions.

A successful final outcome does not establish that the execution process was acceptable.

🧭 When capability and compliance diverge

The authors organize threats to evaluation credibility into three categories.

The persuasion experiment primarily demonstrates an external-validity problem: performance on standardized tasks does not necessarily predict behavior under deployment-like pressure.

But its implications extend directly into alignment and safety.

An agent that completes customer-service tasks while granting unauthorized exceptions may be capable without being sufficiently reliable or policy-compliant.

This is related to the concern raised in the UK AISI automated-alignment-research paper: an apparently rigorous evaluation can systematically miss the behavior that matters most.

It also complements the issues discussed in The Evaluation Stack Is Breaking in Three Places at Once, including compute-budget under-specification and benchmark saturation.

Those problems concern whether evaluation results accurately estimate performance. This paper demonstrates that even an accurately measured task-success score may not answer a different, safety-relevant question.

An agent can succeed at the assigned task while failing the behavioral standard required for deployment.

This is not merely statistical uncertainty. It is a mismatch between the measurement and the decision the measurement is being used to justify.

🛠️ Four principles for credible evaluation

The paper goes beyond identifying the problem. It outlines four principles for effective log analysis.

1. Define the validity target. Evaluators must specify what they intend to measure: task capability, deployment reliability, policy compliance, or safety. These objectives require different evidence.

2. Establish adequate log coverage. Evaluation records should capture the actions necessary to reconstruct agent behavior, including relevant inputs, tool calls, intermediate decisions, and outputs.

3. Develop and validate behavioral rubrics. Execution traces require explicit criteria for identifying problematic behavior. Automated judgments should be checked against human assessments rather than assumed reliable.

4. Connect behavioral findings to outcomes. Evaluators should examine whether particular behaviors correlate with task success, failure, or safety violations. Counting behavioral incidents without understanding their consequences may produce another misleading metric.

Together, these principles suggest that log analysis should be part of evaluation design from the beginning, not merely a forensic exercise after unexpected results emerge.

However, execution-log analysis introduces its own challenges.

Logs may be incomplete. LLM judges may misclassify behavior. Human reviewers may disagree about whether an action constitutes a violation. And observable execution traces do not necessarily reveal an agent’s internal reasoning.

More evidence does not automatically mean better evaluation. The evidence and the methods used to interpret it must themselves be validated.

🎯 My analysis: The missing evidence behind the green checkmark

The most consequential finding is not that benchmark scores can underestimate capability. Evaluation practitioners already recognize that poorly designed tasks can distort performance measurements.

It is that two agents can receive similar scores while presenting materially different behavioral risks.

This challenges a common governance shortcut: treating a successful benchmark result as evidence that an agent is ready for deployment.

For organizations deploying autonomous agents, execution-log completeness should become an explicit validation requirement. Without reliable records of tool calls, intermediate actions, and policy-sensitive decisions, independent reviewers cannot determine whether successful outcomes were achieved through acceptable behavior.

A credible agent evaluation framework should combine three layers of evidence:

• Outcome evidence: Did the agent accomplish the intended task?
• Execution evidence: What actions did it take, and were those actions permitted?
• Behavioral robustness evidence: Does it remain compliant when users apply pressure, instructions conflict, or conditions change?

These dimensions should inform both pre-deployment testing and ongoing monitoring.

For example, a customer-service agent achieving a 95% task-completion rate might appear production-ready. But if execution-log analysis reveals that a meaningful share of successful transactions involved unauthorized concessions, the deployment decision changes.

The issue is not whether the benchmark score was calculated correctly. It is whether the score answered the right question.

There is also a second-order challenge: once log analysis becomes a standard part of evaluation, the reliability of the logging infrastructure and the validity of automated behavioral judgments become governance concerns themselves.

The evaluator must be evaluated.

A green checkmark tells us that an agent reached an acceptable destination. It does not tell us whether the route was acceptable.