The Model Index
Up to four models

Claude 3.5 Sonnet vs Claude 3.5 Sonnet (Jun 2024)

On the index (GPT-4 = 100), Claude 3.5 Sonnet scores 122 and Claude 3.5 Sonnet (Jun 2024) 113. Side by side on the same third-party tests, with price and context. The link in your address bar shares this exact comparison.

Next to Claude 3.5 Sonnet
Presets
Claude 3.5 Sonnet
122107–138
WeightsClosed
ReleasedOct 2024
Price in / out–
Context–
Core tests taken3 of 9
Claude 3.5 Sonnet (Jun 2024)
11397–129
WeightsClosed
ReleasedJun 2024
Price in / out–
Context–
Core tests taken2 of 9

Claude 3.5 Sonnet leads Claude 3.5 Sonnet (Jun 2024) by 9 points, ahead on 2 of 2 shared core tests.

Test by testthird-party results

Claude 3.5 SonnetClaude 3.5 Sonnet (Jun 2024)
0%25%50%75%100%REASONING & KNOWLEDGEGPQA Diamond (Epoch AI)GPQA Diamond (Epoch AI)MMLU (HELM runs)MMLU (HELM runs)SimpleBenchSimpleBenchVPCTVPCTMATHSAIME-style maths (OTIS mock)AIME-style maths (OTIS mock)MATH (level 5)MATH (level 5)FrontierMath (2025 set)FrontierMath (2025 set)FrontierMath tier 4 (2025 set)FrontierMath tier 4 (2025 set)CODINGWeirdMLWeirdMLAGENTSDTBenchDTBench
Every result as a table
TestClaude 3.5 SonnetClaude 3.5 Sonnet (Jun 2024)
Reasoning & knowledge
GPQA Diamond (Epoch AI)55.3%54.0%
MMLU (HELM runs)87.3%86.5%
SimpleBench41.4%27.5%
VPCT33.0%33.0%
Maths
AIME-style maths (OTIS mock)8.5%6.5%
MATH (level 5)57.0%51.7%
FrontierMath (2025 set)2.1%1.0%
FrontierMath tier 4 (2025 set)0.0%0.0%
Coding
WeirdML40.0%31.0%
Agents
DTBench67.8%67.8%
Task length · METR27 sec24 sec

Tests at least two of these models have taken. Bold rows are the core tests.

This page as a chartDownload PNG