The Model Index
Up to four models

Claude 3.5 Haiku vs GPT-4 Turbo (Apr 2024)

On the index (GPT-4 = 100), Claude 3.5 Haiku scores 109 and GPT-4 Turbo (Apr 2024) 108. Side by side on the same third-party tests, with price and context. The link in your address bar shares this exact comparison.

Next to Claude 3.5 Haiku
Presets
Claude 3.5 Haiku
10993–125
WeightsClosed
ReleasedNov 2024
Price in / out–
Context–
Core tests taken1 of 9
GPT-4 Turbo (Apr 2024)
10892–125
WeightsClosed
ReleasedApr 2024
Price in / out–
Context–
Core tests taken3 of 9

Claude 3.5 Haiku and GPT-4 Turbo (Apr 2024) are too close to call (1 point apart, within the uncertainty), ahead on 0 of 1 shared core tests.

Test by testthird-party results

Claude 3.5 HaikuGPT-4 Turbo (Apr 2024)
0%25%50%75%100%REASONING & KNOWLEDGEGPQA Diamond (Epoch AI)GPQA Diamond (Epoch AI)MMLU (HELM runs)MMLU (HELM runs)MATHSAIME-style maths (OTIS mock)AIME-style maths (OTIS mock)MATH (level 5)MATH (level 5)CODINGWeirdMLWeirdML
Every result as a table
TestClaude 3.5 HaikuGPT-4 Turbo (Apr 2024)
Reasoning & knowledge
GPQA Diamond (Epoch AI)38.1%46.6%
MMLU (HELM runs)74.3%81.3%
Maths
AIME-style maths (OTIS mock)4.3%6.7%
MATH (level 5)46.4%46.7%
Coding
WeirdML30.7%18.0%
Task length · METR22 sec

Tests at least two of these models have taken. Bold rows are the core tests.

This page as a chartDownload PNG