The Model Index
Up to four models

Gemini 1.5 Pro (May 2024) vs Qwen2

On the index (GPT-4 = 100), Gemini 1.5 Pro (May 2024) scores 107 and Qwen2 106. Side by side on the same third-party tests, with price and context. The link in your address bar shares this exact comparison.

Next to Gemini 1.5 Pro (May 2024)
Presets
Gemini 1.5 Pro (May 2024)
10791–123
WeightsClosed
ReleasedMay 2024
Price in / out–
Context–
Core tests taken1 of 9
Qwen2
10690–122
WeightsOpen
ReleasedJun 2024
Price in / out–
Context–
Core tests taken2 of 9

Gemini 1.5 Pro (May 2024) and Qwen2 are too close to call (1 point apart, within the uncertainty), ahead on 1 of 1 shared core tests.

Test by testthird-party results

Gemini 1.5 Pro (May 2024)Qwen2
0%25%50%75%100%REASONING & KNOWLEDGEGPQA Diamond (Epoch AI)GPQA Diamond (Epoch AI)MATHSMATH (level 5)MATH (level 5)
Every result as a table
TestGemini 1.5 Pro (May 2024)Qwen2
Reasoning & knowledge
GPQA Diamond (Epoch AI)45.9%40.8%
Maths
MATH (level 5)40.8%39.1%
Task length · METR18 sec

Tests at least two of these models have taken. Bold rows are the core tests.

This page as a chartDownload PNG