Back to blog

AI Agents Fail at Long Tasks

Based on research by Ziyu Ma, Hailang Huang, Shun Zou, Yong Wang, Shidong Yang

Imagine asking an AI to plan a complex project. At first, it seems brilliant. But as the task grows, the AI starts hallucinating, forgetting steps, and making decisions based on outdated or incorrect assumptions. This is the hidden flaw in today’s most advanced large language model agents. They drown in their own growing context, losing track of reality as they try to manage long-horizon tasks.

Researchers have identified a critical bottleneck: current systems keep task execution, state, and assessment tangled together in a massive, ever-expanding context window. This makes it nearly impossible to track what is actually true versus what the model merely thinks is true. Errors propagate unchecked, leading to catastrophic failures in complex workflows that require sustained reasoning and tool use over many interdependent steps.

To fix this, the team developed LongHorizon-Harness, a system that treats long-horizon execution as a strict task-state management problem. Instead of letting the model guess its progress, the system keeps the task state explicitly outside the execution loop. It uses a Manage-Execute-Audit loop where a manager determines the next subtask, a fresh-context executor performs it, and a read-only auditor independently verifies the results against the real environment before moving forward. This ensures that only verified facts update the state, preventing errors from snowballing.

The results are striking. By decoupling state management from execution, the system significantly boosts performance across different models and benchmarks. On WeaveBench, Qwen 3.7-Plus jumped from 51.8% to 80.7% accuracy. On Terminal-Bench 2.1, Qwen 3.7-Plus rose from 69.7% to 77.2%, and on OSWorld 2.0, it improved from 2.8% to 8.3%. Claude Opus 4.7 saw gains on an OSWorld 2.0 subset, rising from 20.0% to 34.3%. This approach proves that keeping AI grounded in verified reality, rather than its own internal narrative, is essential for tackling real-world complexity.

Source: arXiv:2608.01964

This post was generated by staik AI based on the academic publication above.