October 9, 2026
AI agents code at L3, but the staff-level sudo L7 benchmark tests them on 60 tasks
Surge released the sudo L7 benchmark on Oct 9: 60 tasks, mostly from closed production repositories of real companies. Experts score the results against rubrics that run from functional correctness on.
Lenny Rachitsky
@lennysan
A cool new benchmark from Surge on how close AI has come to staff-engineer level in software development. We're halfway there.
echen @echen
introducing sudo L7, our new benchmark for staff-level software engineering. coding agents are getting very good at being L3s. give them a well-defined ticket and they can write excellent code. but a staff engineer is valuable for everything that happens around the code. what should we build? what already exists? what's going to break in prod? what did the tests miss? should we even be doing this? sudo L7 has 60 tasks, mostly in private production repos from real companies, written by engineers who've actually owned these systems. expert rubrics grade everything from functional correctness to engineering craft, architectural judgment, thought partnership, and unnecessary complexity. the best agents succeed on only ~45% of tasks. coding is only part of the job. for now, we're keeping them at idiot genius L3.
· 20.5K views
An agent given a well-written spec writes excellent code. A staff engineer handles the rest: what to build, what already exists, what will break in production, what the tests missed, and whether the task should be done at all. sudo L7 measures exactly that. By the tweet author's estimate, AI has covered about half the way to staff-engineer level.
Next step: the first model results on sudo L7.
Source