8370–96Index score#213 · provisional, limited data
0/ 8Core tests taken0 of 6 areas · plus 2 anchor tests
−17Versus the frontier at releasebest then: GPT-4Replaced by Mistral 7B v0.3 –Price per million tokensno public API price found
–Context windowtokens it reads at once
Where it sitson the frontier
GPT-4 reached this score 6 months earlier.
Test resultsby area · tick = best by any model
Reasoning & knowledgeevidence
Humanity's Last Examno result Mathsevidence
FrontierMath T1-3no result Codingno evidence yet
SWE-bench Verifiedno result Novel problemsno evidence yet
Long tasksno evidence yet
METR time horizonno result Independent results from the Epoch AI benchmarking hub, best reasoning setting. The core tests are shown here; anchor tests also inform the score and can be shown above.