The Model Index
Up to four models

Llama 3.3 70B vs Gemini 1.5 Pro (May 2024)

On the index (GPT-4 = 100), Llama 3.3 70B scores 108 and Gemini 1.5 Pro (May 2024) 107. Side by side on the same third-party tests, with price and context. The link in your address bar shares this exact comparison.

Next to Llama 3.3 70B
Presets
Llama 3.3 70B
10892–124
WeightsOpen
ReleasedDec 2024
Price in / out$0.71 / $0.71
Context131K
Core tests taken1 of 9
Gemini 1.5 Pro (May 2024)
10791–123
WeightsClosed
ReleasedMay 2024
Price in / out–
Context–
Core tests taken1 of 9

Llama 3.3 70B and Gemini 1.5 Pro (May 2024) are too close to call (1 point apart, within the uncertainty), ahead on 1 of 1 shared core tests.

Test by testthird-party results

Llama 3.3 70BGemini 1.5 Pro (May 2024)
0%25%50%75%100%REASONING & KNOWLEDGEGPQA Diamond (Epoch AI)GPQA Diamond (Epoch AI)MATHSAIME-style maths (OTIS mock)AIME-style maths (OTIS mock)MATH (level 5)MATH (level 5)AGENTSDTBenchDTBench
Every result as a table
TestLlama 3.3 70BGemini 1.5 Pro (May 2024)
Reasoning & knowledge
GPQA Diamond (Epoch AI)47.4%45.9%
Maths
AIME-style maths (OTIS mock)5.1%6.8%
MATH (level 5)41.6%40.8%
Agents
DTBench59.5%56.5%

Tests at least two of these models have taken. Bold rows are the core tests.

This page as a chartDownload PNG