The Model Index
Up to four models

o3 vs Mistral Small 3.1

On the index (GPT-4 = 100), o3 scores 147 and Mistral Small 3.1 108. Side by side on the same third-party tests, with price and context. The link in your address bar shares this exact comparison.

Next to o3
Presets
o3
147131–163
WeightsClosed
ReleasedApr 2025
Price in / out$2.0 / $8.0
Context200K
Core tests taken6 of 9
Mistral Small 3.1
10892–124
WeightsOpen
ReleasedMar 2025
Price in / out$0.35 / $0.56
Context128K
Core tests taken1 of 9

o3 leads Mistral Small 3.1 by 39 points, ahead on 1 of 1 shared core tests, at 8.7× the price.

Test by testthird-party results

o3Mistral Small 3.1
0%25%50%75%100%REASONING & KNOWLEDGEGPQA Diamond (Epoch AI)GPQA Diamond (Epoch AI)GPQA Diamond (Vals AI)GPQA Diamond (Vals AI)MMLU ProMMLU ProMMMU ProMMMU ProMATHSAIME-style maths (OTIS mock)AIME-style maths (OTIS mock)MATH (level 5)MATH (level 5)MATH 500MATH 500MGSMMGSMCODINGLiveCodeBenchLiveCodeBenchAGENTSDTBenchDTBenchPROFESSIONAL WORKCorpFin v2CorpFin v2LegalBenchLegalBenchMortgageTaxMortgageTaxTaxEval v2TaxEval v2
Every result as a table
Testo3Mistral Small 3.1
Reasoning & knowledge
GPQA Diamond (Epoch AI)81.8%47.5%
GPQA Diamond (Vals AI)84.1%44.2%
MMLU Pro85.6%66.0%
MMMU Pro80.4%60.1%
Maths
AIME-style maths (OTIS mock)84.4%5.8%
MATH (level 5)97.8%46.8%
MATH 50094.6%68.4%
MGSM91.8%84.2%
Coding
LiveCodeBench83.9%31.8%
Agents
DTBench84.8%58.6%
Professional work
CorpFin v259.7%44.2%
LegalBench83.8%69.2%
MortgageTax65.7%61.2%
TaxEval v274.6%58.3%
Task length · METR39 sec

Tests at least two of these models have taken. Bold rows are the core tests.

This page as a chartDownload PNG