The Model Index
Up to four models

Gemini 2.0 Flash (Dec 2024) vs DeepSeek-V3

On the index (GPT-4 = 100), Gemini 2.0 Flash (Dec 2024) scores 122 and DeepSeek-V3 119. Side by side on the same third-party tests, with price and context. The link in your address bar shares this exact comparison.

Next to Gemini 2.0 Flash (Dec 2024)
Presets
Gemini 2.0 Flash (Dec 2024)
122106–139
WeightsClosed
ReleasedDec 2024
Price in / out–
Context–
Core tests taken0 of 9
DeepSeek-V3
119103–135
WeightsOpen
ReleasedDec 2024
Price in / out$0.26 / $1.03
Context164K
Core tests taken2 of 9

Gemini 2.0 Flash (Dec 2024) and DeepSeek-V3 are too close to call (3 points apart, within the uncertainty).

Test by testthird-party results

Gemini 2.0 Flash (Dec 2024)DeepSeek-V3
0%25%50%75%100%REASONING & KNOWLEDGEMMLU (HELM runs)MMLU (HELM runs)SimpleBenchSimpleBenchCODINGAider PolyglotAider Polyglot
Every result as a table
TestGemini 2.0 Flash (Dec 2024)DeepSeek-V3
Reasoning & knowledge
MMLU (HELM runs)79.7%87.2%
SimpleBench31.1%18.9%
Coding
Aider Polyglot38.2%48.4%
Task length · METR28 sec

Tests at least two of these models have taken. Bold rows are the core tests.

This page as a chartDownload PNG