Benchmarks / GPQA-Diamond
measuredThe router on GPQA-Diamond
One set of 198 graduate-level questions, answered by Enso's router and by single models from other labs, every row a research run.
- dataset
- GPQA
- licence
- CC BY 4.0
- introduced by
- GPQA: A Graduate-Level Google-Proof Q&A Benchmark — Rein, Hou, Stickland, Petty, Pang, Dirani, Michael, Bowman, COLM 2024
- harness
- api.hanzo.ai/v1/research
curl -s "https://api.hanzo.ai/v1/research/runs?org=hanzo&benchmark=enso.gpqa_diamond"prints the table on this page — transcribed from the research API, read when the page opens
What is measured
GPQA-Diamond is the hardest subset of GPQA: 198 multiple-choice questions in biology, physics and chemistry, written by domain PhDs and filtered to the ones that other PhDs get right and skilled non-experts get wrong even with the web open. It is a knowledge benchmark rather than an agentic one — one question, one answer, no tools — which is what makes it useful here: it isolates the choice of model from everything else a system does.
What the router adds is that the choice is per question. Enso reads each one and dispatches it, so a tier is not a model — it is a pool and a willingness to spend. Every row below answers the same 198 questions, which is why a score and what its run cost belong in one table, and why a score quoted without its cost says nothing about the system it came from.
GPQA Diamond · enso.gpqa_diamond, split test · every row a research run, read from api.hanzo.ai when the page opens · sorted by accuracy
| System | Accuracy | Correct | Run cost | Tag, date |
|---|---|---|---|---|
| ensorun | 92.9% | 184 of 198 | $5.58 | frontier · 2026-07-19 |
| enso-ultrarun | 92.9% | 184 of 198 | $20.20 | frontier · 2026-07-19 |
| gpt-5.6-solrun | 91.9% | 182 of 198 | $4.08 | frontier · 2026-07-19 |
| opus-5.5run | 90.9% | 180 of 198 | $3.78 | fill · 2026-10-16 |
| sonnet-5.5run | 90.4% | 179 of 198 | $1.85 | fill · 2026-10-16 |
| gpt-5.5run | 89.9% | 178 of 198 | $11.95 | frontier · 2026-07-19 |
| fable-5.1run | 87.9% | 174 of 198 | $8.23 | fill · 2026-10-16 |
| opus-4.8run | 86.4% | 171 of 198 | $3.66 | frontier · 2026-07-19 |
| haiku-5.5run | 83.3% | 165 of 198 | $0.190 | fill · 2026-10-16 |
| fable-5run | 82.8% | 164 of 198 | $9.87 | frontier · 2026-07-19 |
| claude-5-sonnetrun | 77.8% | 154 of 198 | $5.66 | frontier · 2026-07-19 |
| deepseek-v4-prorun | 74.2% | 147 of 198 | $0.221 | frontier · 2026-07-19 |
| gpt-4orun | 54.5% | 108 of 198 | $1.58 | frontier · 2026-07-19 |
Each system's frontier run, else its v1 run, else its fill run: the three put the same prompts to a model. A run cost is the input and output tokens the run records at the price list's rate for its own model id, the router's included, at list prices, Oct 30, 2026; a dash is a run that records no tokens, a model the list does not price, or a list that could not be read.
What this does not establish
A benchmark score is not a capability. GPQA-Diamond is four-way multiple choice on settled science. It measures what a pool knows and rewards a router that recognises a hard question; it says nothing about whether the answer is reasoned, whether the model holds up over a long autonomous run, or whether it can use a tool. The routing overhead is real work this table does not score either.
A row is a run, as of its date. Every row went through enso-bench's harness at the tag and on the date it shows; a system is the router or a single model as that run met it, not necessarily a preset the catalog serves today. What a row cost is the tokens its run records at today's list price for its own model id, the router's included, so two rows are priced by one rule.
every figure a research run at api.hanzo.ai · the dataset: GPQA under CC BY 4.0