o1-preview

OpenAIReleased 12 Sep 2024ClosedReasoningDate verified

Compare Announcement

121110–133Index score#137 of 229 models

2/ 8Core tests taken2 of 6 areas · plus 6 anchor tests

−2Versus the frontier at releasebest then: o1-miniReplaced by o3

–Price per million tokensno public API price found

–Context windowtokens it reads at once

Where it sitson the frontier

Among the first models to reach this score.

Test resultsby area · tick = best by any model

Reasoning & knowledgeevidence

GPQA Diamond

50%

Humanity's Last Exam

no result

Mathsevidence

FrontierMath T1-3

no result

Codingevidence

SWE-bench Verified

no result

Terminal-Bench

no result

Agentsno evidence yet

APEX-Agents

no result

Novel problemsevidence

ARC-AGI-2

no result

Long tasksevidence

METR time horizon

20 min

Independent results from the Epoch AI benchmarking hub, best reasoning setting. The core tests are shown here; anchor tests also inform the score and can be shown above.

What others sayrank on each leaderboard

The Model Indexindependent tests, our method#137of 219 · 121

LMArena · Creative writingcreative writing#141of 386 · 1366 Epoch Capabilities Indexoverall capability from many benchmarks#96of 239 · 141.9 LMArena · Textoverall chat quality, judged by people#156of 388 · 1389 LMArena · Mathsmaths questions in chat#159of 373 · 1386 LMArena · Codingcoding questions in chat#174of 383 · 1416

Further left is better. Leaderboards measure different things: LMArena is people voting blind between two answers, the others are test scores. Click any row for the source.