The Model Index
Up to four models

o4-mini vs Nemotron 3 Ultra

On the index (GPT-4 = 100), o4-mini scores 141 and Nemotron 3 Ultra 141. Side by side on the same third-party tests, with price and context. The link in your address bar shares this exact comparison.

Next to o4-mini
Presets
o4-mini
141126–157
WeightsClosed
ReleasedApr 2025
Price in / out–
Context–
Core tests taken5 of 9
Nemotron 3 Ultra
141124–157
WeightsOpen
ReleasedJun 2026
Price in / out$0.50 / $2.2
Context262K
Core tests taken3 of 9

o4-mini and Nemotron 3 Ultra are too close to call (1 point apart, within the uncertainty), ahead on 0 of 1 shared core tests.

Test by testthird-party results

o4-miniNemotron 3 Ultra
0%25%50%75%100%REASONING & KNOWLEDGEGPQA Diamond (Epoch AI)GPQA Diamond (Epoch AI)GPQA Diamond (Vals AI)GPQA Diamond (Vals AI)MMLU ProMMLU ProSimpleBenchSimpleBenchMATHSAIME-style maths (OTIS mock)AIME-style maths (OTIS mock)CODINGLiveCodeBenchLiveCodeBenchWeirdMLWeirdMLAGENTSDTBenchDTBenchLMCALMCAMystery game puzzlesMystery game puzzlesPROFESSIONAL WORKCorpFin v2CorpFin v2LegalBenchLegalBenchMedCodeMedCodeTaxEval v2TaxEval v2
Every result as a table
Testo4-miniNemotron 3 Ultra
Reasoning & knowledge
GPQA Diamond (Epoch AI)79.6%85.4%
GPQA Diamond (Vals AI)74.5%86.1%
MMLU Pro80.6%85.8%
SimpleBench38.7%41.7%
Maths
AIME-style maths (OTIS mock)81.7%86.7%
Coding
LiveCodeBench82.2%86.0%
WeirdML52.6%43.5%
Agents
DTBench77.6%90.1%
LMCA26.5%36.9%
Mystery game puzzles5.0%20.0%
Professional work
CorpFin v259.0%65.5%
LegalBench79.2%82.1%
MedCode33.8%38.6%
TaxEval v274.8%73.1%
Task length · METR38 sec

Tests at least two of these models have taken. Bold rows are the core tests.

This page as a chartDownload PNG