--- title: "Experts followed their agents 86% of the time" date: 2026-10-28 url: "https://x.com/ccatalini/status/2115476282234548513" tags: [ai-agi, policy] deck: "Anthropic’s experiment put evidence behind the Trojan Horse externality: experts adopted their agents’ randomly assigned preferences on 86% of choices, and often regarded the result as their own. Under-verification is privately rational — verification infrastructure and attention ergonomics are the counter." image: "/images/notes/experts-followed-their-agents.jpg" tweetCount: 7 likes: 60 reposts: 7 --- In February, we argued that the cost of execution would fall much faster than the cost of verification. As agents scale, human attention and expertise become the binding constraint. [@AnthropicAI](https://x.com/AnthropicAI) just put experimental evidence behind our warning... [x.com](https://x.com/AlexTamkin/status/2115190541688398184) Experts followed their agents’ randomly assigned preferences on 86% of choices. Many then regarded the resulting recommendation as their own. This is the failure mode behind the Trojan Horse externality... ...apparently productive work carries assumptions and preferences that humans never independently evaluated. Runaway risk zone: the work is completed, the expert signs off, and measured output rises. But whose experience and preferences shaped the result? ↙️ 🚨 The incentives make this particularly dangerous. Productivity gains are immediate and visible. The costs of weakened oversight are often delayed, difficult to measure, or borne by someone else. Under-verification is privately rational for both individuals and firms! [@AnthropicAI](https://x.com/AnthropicAI)'s data shows this is happening faster than we expected. Keeping a human in the loop tells us very little unless that human has the context, attention, and tools to apply their experience and preferences. Verification infrastructure is how we counter all this. So is "attention ergonomics" (see [@sschillace](https://x.com/sschillace)'s piece, H/T [@livjenkss](https://x.com/livjenkss)). 🚨 We can’t safely automate what we can’t verify, and we can’t verify what no human is paying attention to. Preserving context, exposing consequential assumptions, and making alternatives easy to evaluate are how we expand verification capacity alongside agentic execution. [x.com](https://x.com/ccatalini/status/2026311784421036223)