September 20, 2026
Agents loop on long tasks: hundreds of billions of tokens and not a single solution
Hundreds of billions of tokens spent on math problems produced not a single solution: the agent kept getting bogged down in small calculations.
Peter Gostev would give an agent a math problem, come back a day later, and find a mountain of calculations instead of progress.
The problem is not math but long runs. The agent gets carried away by small subtasks and validations instead of figuring out how to solve the problem as a whole.
Peter Gostev
@petergostev
I can't get this chart out of my head, and it gets me excited about what models will be capable of next. I burned through hundreds of billions of tokens on different math problems and solved none of them. Every time, the same story: - The model starts strong, then I come back a day later and it has slipped into small local “puzzles” and endless calculations about nothing - Then I spend time building structure to prevent it: an outer loop, another observer agent, synthesis and abstraction steps, whatever comes to mind. It works for a while. - I come back a day later, and it is the same again: “we did a lot of calculations, but went in circles and got nowhere.” Great. To me, this says models were not bad at mathematics as such; they just lacked the ability to think the way people do. A good human will think about a problem, try a couple of approaches, read around it, return to the problem, come at it from another angle, and so on. People are lazy: instead of calculating every individual case by hand, they want to figure out how to think about the problem more systematically and solve it that way. The current generation of agents, by contrast, loves small puzzles, validation runs, and cargo-cult math. They are happy to keep calculating while making absolutely no progress on the problem itself. That is why this chart gets me so excited. Even at the lowest level of test-time compute (the X-axis is unfortunately unlabeled), the “internal” model is far ahead of everything Astra could achieve with unlimited tokens. For me, this is a signal: the model is not simply better when given lots of tokens to think, it seems to approach problems in a qualitatively different way. If so, the implications go far beyond mathematics. Future models should have much deeper judgment, abstract thinking, and the ability not to fixate on unimportant details. I am very excited about this. This will be huge.
· 11.8K views
Structure did not save it. Gostev built a scaffold: an outer loop, a second observer agent, separate synthesis and abstraction steps. It helped briefly; a day later, the agent was looping again.
At the minimum reasoning budget, the internal model in his chart outperforms everything Astra achieves with unlimited tokens. The X-axis is unlabeled. Gostev presents his conclusion as a hunch: the model does not simply think longer, but approaches the problem differently.
Previously, only mathematicians advanced unsolved Erdős problems: 111 moved from open to solved in 2024-2025. Now models are on the board too.
On the FrontierMath Erdős set of 68 such problems, which Epoch AI launched on 1 September 2026, only GPT-6 Astra posts a non-zero result: two problems, 3%.
Price of one attempt. Epoch gave each problem $300 and 72 hours. Astra's two solutions cost $222 and $172 and took roughly 10 hours each.
What helps avoid going in circles is clear from Liam Price and Kevin Barreto, who have been solving Erdős problems since December 2025. They ask a chatbot for a solution, give it to a fresh instance for checking, and repeat until a working version emerges. Then come formal verification in Aristotle and review by human mathematicians.
The working loop around an agent is arranged the same way. A trigger starts an attempt, an external check decides “done or again,” and a stop condition closes the loop.
The internal model will leave a public trail only once it has been run on an open set like FrontierMath Erdős.
