Methodology / LLM Index V0 / July 25, 2026
How to read the model rankings
The rankings use four independent sources: Artificial Analysis, DeepSWE, SWE-Rebench, and Aider. Each tab sorts up to 14 models by the source metric and shows the model's best effort/config for that specific run.
Vibecoding AI Coding Index is a different layer. It is our internal composite of four signals, not a fifth source or hidden tab sorting. Curated metadata adds pricing, provider, context, and release date, but does not move a model in a source ranking.
At a glance
The active tab determines the entire row: score, rank, configuration badge, snapshot date, and freshness. If a source did not measure a model, it is not included in that source's list. A score from a neighboring source is not substituted for the missing value.
Public tab = one source's metric Vibecoding Index = internal weighted composite Curated metadata = reference fields, not a score
Check a row within the selected tab: it has one source, one metric, and one date. You can compare a model's ranks across tabs, but you cannot directly add raw percentages.
What appears in the main table
The server separately collects up to 14 rows from each source. The same model may appear in several tabs with different configurations; only that tab's rows remain in the active tab. AA Index is open by default.
- Score and rank - only from the active source.
- Effort/config - the model's best option for this source.
- Price/task - only the actual price published by the active source. A pricing proxy is not substituted here.
- $/1M output, provider, context, and release - curated metadata. These are reference fields, not components of the active source's score.
The machine-readable ItemList follows the same rule: it contains only AA Index rows, up to 14 models. The union is needed by the interface to switch tabs and is not published as a separate ranking.
How the data works
The first layer is raw observations. This is an append-only log of facts: one model configuration, one source, one metric, and one observation date. A new value creates a new record; the old one remains in history.
The second layer is cooked snapshots. It stores each source's original score separately, the internal Vibecoding Index, and curated metadata. Having these fields in the same record does not turn the metadata or composite into a source.
The third layer builds the public board: it selects each model's best effort/config separately for Artificial Analysis, DeepSWE, SWE-Rebench, and Aider, then trims each ranking to 14 rows. The browser switches ready-made snapshots and does not recalculate the formula.
How we collect sources
Each source has an adapter. The adapter does four things: fetches data, parses a table or API, validates fields, and normalizes the result into a common format. The data then goes into raw observations.
API and JSON are considered first-class sources. HTML tables can also be used, but we consider them more fragile and mark them as scraped. If a source is temporarily down, the site should not go blank: the old snapshot stays in place, the error is logged, and the source goes for review.
V0 sources
The four scoring sources work in two roles. On the public page, they are four independent tabs. In the internal index, the same signals receive the weights listed below; a weight does not change a model's rank within a source tab.
| Source | Tab and metric | Why it is needed | Weight in composite |
|---|---|---|---|
| Artificial Analysis | AA Index: coding index | external assessment of a model's coding strength | 10% |
| DeepSWE | DeepSWE: pass@1 | anchor for the internal composite; repo-agent tasks in a real repository | 60% |
| SWE-Rebench | SWE-Rebench: resolved rate | SWE-like signal with a separate task set | 20% |
| Aider | Aider: percent correct | practical code editing scenario via Aider | 10% |
How we calculate the internal index
The formula below applies only to the Vibecoding AI Coding Index. It does not change the order of the four source tabs. DeepSWE is a required anchor: without it, a model does not receive an internal composite, but may remain in another source's ranking.
DeepSWE 60% SWE-Rebench 20% Artificial Analysis 10% Aider 10%
If DeepSWE is available but one of the additional sources is missing, it is not counted as zero. The available additional sources are normalized together with DeepSWE. This keeps the ranking similar to the DeepSWE ranking, while SWE-Rebench, Artificial Analysis, and Aider refine the internal score but cannot create a composite without DeepSWE.
DeepSWE = 70 Artificial Analysis = 60 Vibecoding Score = (70 * 60 + 60 * 10) / (60 + 10) = 68.6
Why price is separate
Price/task appears only where the active source publishes an actual, comparable task cost. Total run spend without a known number of tasks, a token-based estimate, or a pricing proxy do not appear in this column.
$/1M output is a separate pricing fact from curated metadata. It helps estimate the budget, but does not belong to the source run and does not change either its ranking or the internal index.
Why the internal index needs confidence
Confidence shows the coverage of the internal composite. If a model has a DeepSWE-score but no confirmation from other sources, the index can be calculated, but it has less supporting evidence.
A missing additional source is not counted as zero and does not directly penalize the score. It lowers coverage/confidence. A public tab has a different contract: no measurement - no row, while freshness is shown by the source's date and indicator.
How the site updates
The backend saves new observations and recalculates snapshots. The page server then selects the best config and up to 14 models separately for each source. Each tab has its own date, freshness, and coverage: an Artificial Analysis update does not refresh an old Aider.
The frontend does not calculate the internal index or merge sources. Extended signals, provenance, and change history remain in snapshots and cards; a public tab shows only the data needed to verify a model's rank.
What the index does not promise
Vibecoding Index V0 does not mean that vibecoding.tech itself ran models through a lab benchmark. It aggregates published signals and remains an internal analytical layer, not a measurement source.
A source ranking and composite do not replace testing on your own project, stack, and budget. Start with the tab whose methodology is closest to your task, then test the leaders on your own code.