The Model Index
Up to four models

Llama 3.3 70B vs GPT-4o mini

On the index (GPT-4 = 100), Llama 3.3 70B scores 108 and GPT-4o mini 104. Side by side on the same third-party tests, with price and context. The link in your address bar shares this exact comparison.

Next to Llama 3.3 70B
Presets
Llama 3.3 70B
10892–124
WeightsOpen
ReleasedDec 2024
Price in / out$0.71 / $0.71
Context131K
Core tests taken1 of 9
GPT-4o mini
10487–120
WeightsClosed
ReleasedJul 2024
Price in / out$0.15 / $0.60
Context128K
Core tests taken3 of 9

Llama 3.3 70B and GPT-4o mini are too close to call (4 points apart, within the uncertainty), ahead on 1 of 1 shared core tests, at 2.7× the price.

Test by testthird-party results

Llama 3.3 70BGPT-4o mini
0%25%50%75%100%REASONING & KNOWLEDGEGPQA Diamond (Epoch AI)GPQA Diamond (Epoch AI)MMLU (HELM runs)MMLU (HELM runs)SimpleBenchSimpleBenchMATHSAIME-style maths (OTIS mock)AIME-style maths (OTIS mock)MATH (level 5)MATH (level 5)CODINGWeirdMLWeirdMLAGENTSBALROGBALROGDTBenchDTBenchLMCALMCA
Every result as a table
TestLlama 3.3 70BGPT-4o mini
Reasoning & knowledge
GPQA Diamond (Epoch AI)47.4%37.7%
MMLU (HELM runs)79.1%76.7%
SimpleBench19.9%10.7%
Maths
AIME-style maths (OTIS mock)5.1%6.9%
MATH (level 5)41.6%52.6%
Coding
WeirdML14.4%11.8%
Agents
BALROG23.0%17.4%
DTBench59.5%54.4%
LMCA17.5%10.4%

Tests at least two of these models have taken. Bold rows are the core tests.

This page as a chartDownload PNG