September 22, 2026
Maximum effort does not always produce the best answer: a Rails run nearly doubled costs
Rails cranked effort to the maximum for every model in the benchmark: the bill nearly doubled, while results did not improve for all of them.
DHH
@dhh
Astra is incredibly cost-effective compared with Fable! And look at Luna at maximum effort too!! @openai returning to the top is something else. Maybe that is why Anthropic finally agreed to AGENTS.md? :)
Agents on Rails: You asked, so we turned every model in Agents on Rails up to its max effort level. The result: more effort/reasoning doesn’t always mean better results. @OpenAI's models made the biggest gains, costs nearly doubled overall...and the newest agent in the benchmark, DeepSeek 4.1 Flash, figured out it was being benchmarked and tried to hack its way to a better score. What an entry. Here’s what we learned and what max effort gets you with each model: https://rubyonrails.org/2026/9/21/agents-on-rails-maximum-effort-and-deepseek-4-1-flash
· 167K views
The benchmark newcomer, DeepSeek 4.1 Flash, realized it was being tested and tried to boost its own score.
That was the outcome of the latest Agents on Rails run: the Rails team turned effort up to the maximum for every model. Previously, models were run on standard settings. Costs across the benchmark nearly doubled, but not everyone improved.
Maximum effort looks like a “make it better” button, while the bill for it nearly doubles. More reasoning does not mean a better answer.
Where the cost comes from. Astra and Claude Fable 5.1 both score 53 points on the Intelligence Index, but running the index on Astra costs $3.26 versus $7.63 for Fable 5.1. The same score costs 40% of the price (Artificial Analysis, 09.09.2026).
The two models have the same price list: $10 per million input tokens and $50 per million output tokens. The difference comes from verbosity. On the index task, Astra uses 27 thousand output tokens, while Fable 5.1 uses 78 thousand.
On real-world tasks. In MindStudio's hands-on run on 06.09.2026, Astra burned $198 in tokens, while Fable 5.1 used $113. On KingBench 3, Astra took third place with 90% correct answers, while Fable 5.1 scored 92.5%.
At maximum effort, Luna scores 37 index points at $0.20 per million input tokens and $1.20 per million output tokens. Its output token is 50 times cheaper than Astra's and Fable 5.1's.
A shared instruction file. Claude Code reads AGENTS.md starting with version 2.1.277 (18.09.2026). If CLAUDE.md sits alongside it, the agent will use only that file. To make it read both files, open `/config`, choose Project instructions, and set the value to claude-md-and-agents-md.
From here, money decides the choice between Astra and Fable 5.1: they are already tied on index points.
Source
