Qwen2

AlibabaReleased 7 Jun 2024OpenDate verified

Compare Announcement

10391–114Index score#180 of 229 models

2/ 8Core tests taken2 of 6 areas · plus 4 anchor tests

−7Versus the frontier at releasebest then: GPT-4oReplaced by Qwen2.5

–Price per million tokensno public API price found

–Context windowtokens it reads at once

Where it sitson the frontier

GPT-4 Turbo reached this score 7 months earlier.

Test resultsby area · tick = best by any model

Reasoning & knowledgeevidence

GPQA Diamond

41%

Humanity's Last Exam

no result

Mathsevidence

FrontierMath T1-3

no result

Codingevidence

SWE-bench Verified

no result

Terminal-Bench

no result

Agentsevidence

APEX-Agents

no result

Novel problemsno evidence yet

ARC-AGI-2

no result

Long tasksevidence

METR time horizon

2.2 min

Independent results from the Epoch AI benchmarking hub, best reasoning setting. The core tests are shown here; anchor tests also inform the score and can be shown above.

What others sayrank on each leaderboard

The Model Indexindependent tests, our method#180of 219 · 103

Epoch Capabilities Indexoverall capability from many benchmarks#153of 239 · 125.3 LMArena · Mathsmaths questions in chat#252of 373 · 1273 LMArena · Creative writingcreative writing#285of 386 · 1223 LMArena · Textoverall chat quality, judged by people#288of 388 · 1262 LMArena · Codingcoding questions in chat#285of 383 · 1297

Further left is better. Leaderboards measure different things: LMArena is people voting blind between two answers, the others are test scores. Click any row for the source.