Llama 3.1
Llama 3.1 is an open-weights AI model from Meta with an index score of 110 (GPT-4 = 100), rank 176 of 217.
11094–127Index score#176 of 217 models
–Price per million tokensno public API price found
–Context windowtokens it reads at once
23 Jul 2024Released2 years 4 months ago
Where it sitson the frontier
GPT-4o reached this score 2 months earlier.
Test resultsby area · tick = best by any model
Reasoning & knowledgeevidence
Mathsevidence
Codingevidence
Agentsevidence
Professional workno evidence yet
Novel problemsno evidence yet
Long tasksno evidence yet
Third-party results only, at the model’s best standard setting. Hover a bar for who ran it.
What others sayrank on 8 leaderboards
The Model Indexthird-party tests, our method#176of 217 · 110
LMArena · Creative writingcreative writing#189of 360 · 1304LMArena · Mathsmaths questions in chat#188of 346 · 1319LMArena · Codingcoding questions in chat#195of 357 · 1376LMArena · Textoverall chat quality, judged by people#199of 362 · 1336LMArena · Instruction followingfollowing detailed instructions#203of 362 · 1315LMArena · Long promptslong, complex requests#209of 340 · 1329Epoch Capabilities Indexoverall capability from many benchmarks#145of 231 · 127.9Best attop three among current models
Not in the top three among current models on any category we track.
Facts and sources
Released23 Jul 2024
Announcementai.meta.com ↗
Release entryLlama 3.1 (405B)
Scored asLlama 3.1-405B
Output speed–
Hugging Facenot linked