The Model Index
Up to four models

o4-mini vs o3-mini

On the index (GPT-4 = 100), o4-mini scores 141 and o3-mini 131. Side by side on the same third-party tests, with price and context. The link in your address bar shares this exact comparison.

Next to o4-mini
Presets
o4-mini
141126–157
WeightsClosed
ReleasedApr 2025
Price in / out–
Context–
Core tests taken5 of 9
o3-mini
131115–147
WeightsClosed
ReleasedJan 2025
Price in / out–
Context–
Core tests taken3 of 9

o4-mini leads o3-mini by 10 points, ahead on 3 of 3 shared core tests.

Test by testthird-party results

o4-minio3-mini
0%25%50%75%100%REASONING & KNOWLEDGEGPQA Diamond (Epoch AI)GPQA Diamond (Epoch AI)GPQA Diamond (Vals AI)GPQA Diamond (Vals AI)MMLU ProMMLU ProFiction.LiveBenchFiction.LiveBenchSimpleBenchSimpleBenchSimpleQA VerifiedSimpleQA VerifiedMATHSFrontierMath (tiers 1-3)FrontierMath (tiers 1-3)AIME-style maths (OTIS mock)AIME-style maths (OTIS mock)MATH (level 5)MATH (level 5)MATH 500MATH 500MGSMMGSMChess puzzlesChess puzzlesFrontierMath (2025 set)FrontierMath (2025 set)FrontierMath tier 4 (2025 set)FrontierMath tier 4 (2025 set)FrontierMath (tier 4)FrontierMath (tier 4)CODINGAider PolyglotAider PolyglotLiveCodeBenchLiveCodeBenchCadEvalCadEvalGSO-BenchGSO-BenchWeirdMLWeirdMLAGENTSDTBenchDTBenchLMCALMCAMystery game puzzlesMystery game puzzlesPROFESSIONAL WORKCorpFin v2CorpFin v2LegalBenchLegalBenchTaxEval v2TaxEval v2NOVEL PROBLEMSARC-AGI-2ARC-AGI-2ARC-AGI-1ARC-AGI-1
Every result as a table
Testo4-minio3-mini
Reasoning & knowledge
GPQA Diamond (Epoch AI)79.6%77.0%
GPQA Diamond (Vals AI)74.5%75.5%
MMLU Pro80.6%78.7%
Fiction.LiveBench77.8%50.0%
SimpleBench38.7%22.8%
SimpleQA Verified19.6%15.3%
Maths
FrontierMath (tiers 1-3)36.1%18.6%
AIME-style maths (OTIS mock)81.7%76.9%
MATH (level 5)97.8%96.5%
MATH 50094.2%91.8%
MGSM93.4%91.3%
Chess puzzles22.0%15.0%
FrontierMath (2025 set)24.8%12.4%
FrontierMath tier 4 (2025 set)6.3%4.2%
FrontierMath (tier 4)4.9%0.0%
Coding
Aider Polyglot72.0%60.4%
LiveCodeBench82.2%71.5%
CadEval62.0%54.0%
GSO-Bench3.6%1.3%
WeirdML52.6%43.8%
Agents
DTBench77.6%68.8%
LMCA26.5%19.0%
Mystery game puzzles5.0%7.0%
Professional work
CorpFin v259.0%45.3%
LegalBench79.2%71.5%
TaxEval v274.8%69.4%
Novel problems
ARC-AGI-26.1%3.0%
ARC-AGI-158.7%34.5%
Task length · METR38 sec

Tests at least two of these models have taken. Bold rows are the core tests.

This page as a chartDownload PNG