Up to four models
Gemma 4 vs o1
On the index (GPT-4 = 100), Gemma 4 scores 137 and o1 136. Side by side on the same third-party tests, with price and context. The link in your address bar shares this exact comparison.
Gemma 4 and o1 are too close to call (2 points apart, within the uncertainty), ahead on 0 of 1 shared core tests.
Test by testthird-party results
Every result as a table
| Test | Gemma 4 | o1 |
|---|---|---|
| Reasoning & knowledge | ||
| GPQA Diamond (Epoch AI) | 75.8% | 76.8% |
| SimpleQA Verified | 10.4% | 41.1% |
| Maths | ||
| AIME-style maths (OTIS mock) | 73.3% | 73.3% |
| Coding | ||
| WeirdML | 52.3% | 46.1% |
| Agents | ||
| DTBench | 82.7% | 74.7% |
| LMCA | 39.3% | 22.3% |
Task length · METR31 sec
Tests at least two of these models have taken. Bold rows are the core tests.