The Model Index
Up to four models

Gemini 1.5 Flash vs Mixtral 8x22B

On the index (GPT-4 = 100), Gemini 1.5 Flash scores 101 and Mixtral 8x22B 99. Side by side on the same third-party tests, with price and context. The link in your address bar shares this exact comparison.

Next to Gemini 1.5 Flash
Presets
Gemini 1.5 Flash
10185–118
WeightsClosed
ReleasedMay 2024
Price in / out–
Context–
Core tests taken1 of 9
Mixtral 8x22B
9983–116
WeightsOpen
ReleasedApr 2024
Price in / out$2.0 / $6.0
Context66K
Core tests taken1 of 9

Gemini 1.5 Flash and Mixtral 8x22B are too close to call (2 points apart, within the uncertainty), ahead on 1 of 1 shared core tests.

Test by testthird-party results

Gemini 1.5 FlashMixtral 8x22B
0%25%50%75%100%REASONING & KNOWLEDGEGPQA Diamond (Epoch AI)GPQA Diamond (Epoch AI)MMLU (HELM runs)MMLU (HELM runs)MATHSMATH (level 5)MATH (level 5)AGENTSDTBenchDTBench
Every result as a table
TestGemini 1.5 FlashMixtral 8x22B
Reasoning & knowledge
GPQA Diamond (Epoch AI)40.4%34.1%
MMLU (HELM runs)77.9%77.8%
Maths
MATH (level 5)25.1%24.2%
Agents
DTBench51.4%55.1%

Tests at least two of these models have taken. Bold rows are the core tests.

This page as a chartDownload PNG