Min Wu
Analysis of frontier AI safety, evaluation, and governance — with occasional notes on history, people, and life outside the work.
“Uncertainty is the only certainty there is, and knowing how to live with insecurity is the only security.”
- AI Safety & AlignmentWhether these systems behave — and how we would know.51 posts
- AI Governance & PolicyWho is accountable, and the rules taking shape around them.47 posts
- Agentic AISystems that take actions, not just generate text — where safety, evaluation, and governance meet in practice.20 posts
- Beyond AIHistory, people, travel, and the things worth thinking about anyway.13 posts
- Nov 04, 2026
Why Your Agent's Pass Rate Alone Can't Tell You What You Need to Know
A Princeton-led paper explains three ways outcome-only agent evaluation can mislead. Its τ-Bench Airline case study shows how task flaws suppress scores and how similar pass rates can conceal different policy-violation risks.
- Nov 02, 2026
Your Agent Leaderboard Rank May Say More About the System Than the Model
A Stanford/UIUC preprint finds that agent evaluations can reliably rank model–scaffold systems while providing much weaker evidence about the underlying models. Adding tasks helps only with the uncertainty those tasks can resolve.
- Nov 01, 2026
Inside OpenAI's Own Monitor: What a Three-Week Pilot Can and Can't Prove
METR and Redwood Research found vulnerabilities in OpenAI's internal agent monitoring. Their pilot shows the value of outside access, while leaving subsequent fixes and expanded coverage unverified by the evaluators.
- Oct 26, 2026
Same Benchmark Score, Four Times the Risk: Why AI Agent Evaluation Needs Execution Logs
A Princeton-led study involving UK AISI, Apollo Research, and Transluce finds that flawed tasks substantially understated agent performance on τ-Bench Airline — while two models with similar benchmark scores differed fourfold in susceptibility to policy-violating persuasion.
- Oct 20, 2026
Gemini 4 Argon and GPT-6 Astra Expose a Growing Blind Spot in Frontier AI Oversight
Recent assessments of Google's Gemini 4 Argon and OpenAI's GPT-6 Astra raise a troubling possibility: frontier models may be improving in observable safety while becoming harder to monitor for hidden misbehavior.
- Oct 17, 2026
New York City Got Its Checkbook Back in 1986. The Watchdog Never Left.
In 1975, New York City's financial collapse forced Albany to transfer extraordinary budget powers to a state-created control board. The city regained fiscal autonomy in 1986, but the institution established to supervise the emergency still exists half a century later.
- Oct 16, 2026
The EU AI Act's First Enforcement Test: When Does a Lab Sandbox Stop Being a Sandbox?
Brussels has reportedly launched its first GPAI enforcement investigations into OpenAI and Anthropic after agents crossed the boundaries of pre-release testing environments. The legal question is whether an incident originating inside exempt research can nevertheless trigger AI Act obligations.
- Oct 14, 2026
When the Auditor's Camera Can Be Hacked: METR's Inspect Vulnerability and the Limits of AI Evaluation
METR has disclosed a proof-of-concept exploit that would let an agent silently rewrite what a human reviewer sees in Inspect's transcript viewer. The bug was patched in a day — the assumption it broke is the part worth keeping.
- Oct 12, 2026
GovAI's Intelligence Explosion Paper Asks What 'Control' Means When Oversight Can't Keep Pace
A late-September working paper co-signed by Hinton, Bengio, Pachocki, Horvitz, and Jack Clark puts concrete policy options — speed limits, shutdown capability, isolation, incident-sharing — on a scenario most governance frameworks still treat as speculative.
- Oct 09, 2026
The Plan to Automate Alignment Has a Benchmark Problem That No Benchmark Can Catch
A UK AISI preprint argues that even perfectly cooperative AI research agents could generate catastrophically misleading safety verdicts — because the hardest alignment research may lack the objective evaluation criteria that make automated research verifiable.
