October 4, 2026
Benzi reads less code when fixing bugs: half as much as Claude Code in its creators' test
Benzi maps relationships in a project before making changes and queries that map instead of reading the entire source code.
In the authors' Oct 3 run, Benzi with DeepSeek v4-flash solved 391 of 500 SWE-bench Verified tasks with one attempt per task.
The authors solved 78.2% of tasks for $37.33 across the entire run. Each successful fix cost 9.5 cents.
Map the project first. Benzi works as a standalone AI coding agent in VS Code or the terminal. Before responding, it analyzes the source code and maps calls, data flow and class inheritance. For example, the agent can query which functions call a given function instead of reading every file where its name appears.
In a separate comparison, the authors measured how much source code the agents read while fixing bugs. Benzi read half as much as Claude Code. The authors also report that it was 40% faster and cost half as much in this comparison.
| Agent | Lines of source code read | | --- | ---: | | Benzi | 9,125 | | Claude Code | 20,704 | | DeepSeek Harness | 43,598 | | OpenCode | Over 65,000 |
Benzi supports 10 programming languages. For Python, it has a runtime tracer that tracks program execution and refines relationships that cannot be determined from the source code.
Running it in a repository. According to Variant Technologies' Oct 3 instructions, Benzi is free, and the extension and CLI require your own Anthropic or OpenAI-compatible API key. Install the package with `pip install benzi`, then log in with `benzi-login --login you@example.com`.
Set the key with `benzi-login --anthropic-key <key>` or `benzi-login --nonanthropic-key <key>`. Run it from the project folder: `benzi-headless . "what does this repo do?"`. The browser demo works without a key.
The same package installs `benzi-mcp`: after logging in, you can connect the compiler to another agent through MCP without using Benzi's own agent loop.
