Up to four models
Qwen2 vs Gemini 1.5 Flash
On the index (GPT-4 = 100), Qwen2 scores 106 and Gemini 1.5 Flash 101. Side by side on the same third-party tests, with price and context. The link in your address bar shares this exact comparison.
Qwen2 and Gemini 1.5 Flash are too close to call (5 points apart, within the uncertainty), ahead on 1 of 1 shared core tests.
Test by testthird-party results
Every result as a table
| Test | Qwen2 | Gemini 1.5 Flash |
|---|---|---|
| Reasoning & knowledge | ||
| GPQA Diamond (Epoch AI) | 40.8% | 40.4% |
| MMLU (HELM runs) | 82.4% | 77.9% |
| Maths | ||
| MATH (level 5) | 39.1% | 25.1% |
Task length · METR
18 sec
Tests at least two of these models have taken. Bold rows are the core tests.