Benchmark Leaderboard

MMLU

127 models with MMLU scores, ranked from highest to lowest.

Score Leaderboard

#	Model	Developer	Score
1	Google: Gemini 2.5 Flash	google	88.2
2	MiniMax: MiniMax M2.5	minimax	87.5
3	MiniMax: MiniMax M2.7	minimax	87.5
4	MiniMax: MiniMax M2.1	minimax	87.5
5	OpenAI: GPT-5	openai	87.1
6	DeepSeek: DeepSeek V3.2 Speciale	deepseek	86.3
7	Google: Gemini 2.5 Pro	google	86.2
8	OpenAI: GPT-5.2-Codex	openai	86.0
9	OpenAI: GPT-5 Codex	openai	86.0
10	OpenAI: GPT-5.1-Codex	openai	86.0
11	OpenAI: o3	openai	85.3
12	DeepSeek: R1	deepseek	84.9
13	MoonshotAI: Kimi K2 Thinking	moonshotai	84.8
14	Qwen: Qwen3 235B A22B Thinking 2507	qwen	84.3
15	OpenAI: o1	openai	84.1
16	OpenAI: GPT-5 Mini	openai	83.7
17	Anthropic: Claude 3.7 Sonnet (thinking)	anthropic	83.7
18	Google: Gemini 2.5 Pro Preview 05-06	google	83.7
19	DeepSeek: DeepSeek V3.2	deepseek	83.7
20	DeepSeek: DeepSeek V3.1 Terminus	deepseek	83.6
21	Qwen: Qwen3 VL 235B A22B Instruct	qwen	83.6
22	Qwen: Qwen3 VL 235B A22B Thinking	qwen	83.6
23	OpenAI: o4 Mini	openai	83.2
24	Nous: Hermes 3 405B Instruct	nousresearch	82.9
25	Qwen: Qwen3 235B A22B Instruct 2507	qwen	82.8
26	xAI: Grok 3 Mini	x-ai	82.8
27	MoonshotAI: Kimi K2 0711	moonshotai	82.4
28	Qwen: Qwen3 Next 80B A3B Thinking	qwen	82.4
29	Qwen: Qwen3 Next 80B A3B Instruct	qwen	82.4
30	Qwen: Qwen3 Max Thinking	qwen	82.4
31	MoonshotAI: Kimi K2.5	moonshotai	82.4
32	MoonshotAI: Kimi K2.6	moonshotai	82.4
33	Prime Intellect: INTELLECT-3	prime-intellect	82.2
34	OpenAI: GPT-5.1-Codex-Mini	openai	82.0
35	MiniMax: MiniMax M2	minimax	82.0
36	MoonshotAI: Kimi K2 0905	moonshotai	81.9
37	MiniMax: MiniMax M1	minimax	81.6
38	Z.ai: GLM 4.5 Air	z-ai	81.5
39	NVIDIA: Llama 3.3 Nemotron Super 49B V1.5	nvidia	81.4
40	Kwaipilot: KAT-Coder-Pro V2	kwaipilot	81.3
41	Nous: Hermes 3 70B Instruct	nousresearch	81.1
42	Google: Gemini 2.5 Flash Lite Preview 09-2025	google	80.8
43	OpenAI: gpt-oss-120b	openai	80.8
44	Qwen: Qwen3 VL 30B A3B Thinking	qwen	80.7
45	Mistral Large	mistralai	80.7
46	Qwen: Qwen3 30B A3B Instruct 2507	qwen	80.5
47	Qwen: Qwen3 30B A3B Thinking 2507	qwen	80.5
48	OpenAI: o3 Mini High	openai	80.2
49	Qwen: Qwen3 VL 32B Instruct	qwen	79.8
50	DeepSeek: R1 Distill Llama 70B	deepseek	79.5
51	NVIDIA: Nemotron 3 Nano 30B A3B (free)	nvidia	79.4
52	xAI: Grok Code Fast 1	x-ai	79.3
53	Qwen: Qwen3 30B A3B	qwen	79.2
54	OpenAI: o3 Mini	openai	79.1
55	Z.ai: GLM 4.5V	z-ai	78.8
56	OpenAI: GPT-4.1 Mini	openai	78.1
57	OpenAI: GPT-5.4 Mini	openai	78.1
58	OpenAI: GPT-5 Nano	openai	78.0
59	Baidu: ERNIE 4.5 300B A47B	baidu	77.6
60	Anthropic: Claude Sonnet 4.5	anthropic	77.2
61	Anthropic: Claude Sonnet 4	anthropic	77.2
62	Anthropic: Claude Sonnet 4.6	anthropic	77.2
63	Qwen: QwQ 32B	qwen	76.4
64	AllenAI: Olmo 3 32B Think	allenai	76.3
65	Qwen: Qwen3 235B A22B	qwen	76.2
66	Qwen: Qwen3 Max	qwen	76.2
67	Anthropic: Claude Haiku 4.5	anthropic	76.0
68	NVIDIA: Nemotron Nano 12B 2 VL (free)	nvidia	75.9
69	Perplexity: Sonar Pro	perplexity	75.5
70	Z.ai: GLM 4.6V	z-ai	75.2
71	Upstage: Solar Pro 3	upstage	75.0
72	Qwen: Qwen3 VL 8B Thinking	qwen	74.9
73	Qwen: Qwen3 VL 8B Instruct	qwen	74.9
74	OpenAI: GPT-4o	openai	74.8
75	Xiaomi: MiMo-V2-Flash	xiaomi	74.4
76	xAI: Grok 4.1 Fast	x-ai	74.3
77	xAI: Grok 4 Fast	x-ai	74.3
78	NVIDIA: Nemotron Nano 9B V2 (free)	nvidia	74.2
79	OpenAI: GPT-4o (2024-05-13)	openai	74.0
80	DeepSeek: R1 Distill Qwen 32B	deepseek	73.9
81	NVIDIA: Nemotron Nano 9B V2	nvidia	73.9
82	Qwen: Qwen3 32B	qwen	72.7
83	Qwen: Qwen3 VL 30B A3B Instruct	qwen	72.5
84	Google: Gemini 2.0 Flash Lite	google	72.4
85	Google: Gemini 2.5 Flash Lite	google	72.4
86	Qwen: Qwen2.5 VL 72B Instruct	qwen	72.0
87	OpenAI: gpt-oss-20b	openai	71.8
88	Meta: Llama 3.3 70B Instruct	meta-llama	71.3
89	Cohere: Command A	cohere	71.2
90	Mistral: Devstral Medium	mistralai	70.8
91	Qwen: Qwen3 Coder 30B A3B Instruct	qwen	70.6
92	xAI: Grok 3 Beta	x-ai	70.3
93	Mistral: Pixtral Large 2411	mistralai	70.1
94	Anthropic: Claude Opus 4.7	anthropic	69.6
95	Anthropic: Claude Opus 4.6	anthropic	69.6
96	Anthropic: Claude Opus 4	anthropic	69.6
97	Anthropic: Claude Opus 4.5	anthropic	69.6
98	OpenAI: GPT-4 Turbo	openai	69.4
99	NVIDIA: Llama 3.1 Nemotron 70B Instruct	nvidia	69.0
100	Perplexity: Sonar	perplexity	68.9
101	Mistral Large 2407	mistralai	68.3
102	Mistral: Mistral Medium 3.1	mistralai	68.3
103	Mistral: Mistral Medium 3	mistralai	68.3
104	Mistral: Mistral Small 3	mistralai	68.1
105	Mistral: Devstral Small 1.1	mistralai	67.8
106	Qwen: Qwen3 14B	qwen	67.5
107	Reka Flash 3	rekaai	66.9
108	OpenAI: GPT-4.1 Nano	openai	65.7
109	NVIDIA: Nemotron Nano 12B 2 VL	nvidia	64.9
110	OpenAI: GPT-4o-mini	openai	64.8
111	Qwen: Qwen3 8B	qwen	64.3
112	Qwen2.5 Coder 32B Instruct	qwen	63.5
113	Qwen: Qwen-Turbo	qwen	63.3
114	Mistral: Saba	mistralai	61.1
115	NVIDIA: Nemotron 3 Nano 30B A3B	nvidia	57.9
116	AI21: Jamba Large 1.7	ai21	57.7
117	OpenAI: GPT-5.4 Nano	openai	55.6
118	Google: Gemma 3n 4B	google	48.8
119	Meta: Llama 3 8B Instruct	meta-llama	47.6
120	Meta: Llama 3.1 8B Instruct	meta-llama	47.6
121	Meta: Llama 3.2 11B Vision Instruct	meta-llama	46.4
122	OpenAI: GPT-3.5 Turbo	openai	46.2
123	Cohere: Command R+ (08-2024)	cohere	43.2
124	Mistral: Mixtral 8x7B Instruct	mistralai	38.7
125	Meta: Llama 3.2 3B Instruct	meta-llama	34.7
126	Mistral: Mistral 7B Instruct v0.1	mistralai	24.5
127	Meta: Llama 3.2 1B Instruct	meta-llama	20.0

Value Rankings

Best performance per dollar — score divided by avg price per million tokens.

#	Model	Score	Avg $/M	Value
1	Meta: Llama 3.1 8B Instruct	47.6	$0.03	1360.0
2	Meta: Llama 3 8B Instruct	47.6	$0.04	1360.0
3	Mistral: Mistral Small 3	68.1	$0.07	1047.7
4	Qwen: Qwen3 235B A22B Instruct 2507	82.8	$0.09	968.4
5	OpenAI: gpt-oss-20b	71.8	$0.09	844.7
6	Qwen: Qwen-Turbo	63.3	$0.08	779.1
7	NVIDIA: Nemotron Nano 9B V2	73.9	$0.10	739.0
8	OpenAI: gpt-oss-120b	80.8	$0.11	705.7
9	Google: Gemma 3n 4B	48.8	$0.09	542.2
10	NVIDIA: Nemotron 3 Nano 30B A3B	57.9	$0.12	463.2
11	Qwen: Qwen3 32B	72.7	$0.16	454.4
12	Qwen: Qwen3 14B	67.5	$0.15	450.0
13	Reka Flash 3	66.9	$0.15	446.0
14	Qwen: Qwen3 30B A3B	79.2	$0.18	440.0
15	Qwen: Qwen3 Coder 30B A3B Instruct	70.6	$0.17	415.3
16	Qwen: Qwen3 30B A3B Instruct 2507	80.5	$0.20	412.8
17	Xiaomi: MiMo-V2-Flash	74.4	$0.19	391.6
18	Google: Gemini 2.0 Flash Lite	72.4	$0.19	386.1
19	OpenAI: GPT-5 Nano	78.0	$0.22	346.7
20	Meta: Llama 3.3 70B Instruct	71.3	$0.21	339.5
21	Mistral: Devstral Small 1.1	67.8	$0.20	339.0
22	Qwen: Qwen3 30B A3B Thinking 2507	80.5	$0.24	335.4
23	NVIDIA: Llama 3.3 Nemotron Super 49B V1.5	81.4	$0.25	325.6
24	Google: Gemini 2.5 Flash Lite Preview 09-2025	80.8	$0.25	323.2
25	Qwen: Qwen3 VL 32B Instruct	79.8	$0.26	306.9
26	Google: Gemini 2.5 Flash Lite	72.4	$0.25	289.6
27	Qwen: Qwen3 8B	64.3	$0.22	285.8
28	Nous: Hermes 3 70B Instruct	81.1	$0.30	270.3
29	DeepSeek: DeepSeek V3.2	83.7	$0.32	265.7
30	OpenAI: GPT-4.1 Nano	65.7	$0.25	262.8
31	Qwen: Qwen3 VL 8B Instruct	74.9	$0.29	258.3
32	DeepSeek: R1 Distill Qwen 32B	73.9	$0.29	254.8
33	AllenAI: Olmo 3 32B Think	76.3	$0.33	234.8
34	Qwen: Qwen3 VL 30B A3B Instruct	72.5	$0.33	223.1
35	xAI: Grok 4.1 Fast	74.3	$0.35	212.3
36	xAI: Grok 4 Fast	74.3	$0.35	212.3
37	Qwen: QwQ 32B	76.4	$0.36	209.3
38	xAI: Grok 3 Mini	82.8	$0.40	207.0
39	Upstage: Solar Pro 3	75.0	$0.38	200.0
40	Meta: Llama 3.2 11B Vision Instruct	46.4	$0.24	189.4
41	Qwen: Qwen3 Next 80B A3B Thinking	82.4	$0.44	187.8
42	Meta: Llama 3.2 3B Instruct	34.7	$0.20	177.5
43	Meta: Llama 3.2 1B Instruct	20.0	$0.11	176.2
44	OpenAI: GPT-4o-mini	64.8	$0.38	172.8
45	DeepSeek: DeepSeek V3.1 Terminus	83.6	$0.50	167.2
46	Z.ai: GLM 4.5 Air	81.5	$0.49	166.3
47	Mistral: Mistral 7B Instruct v0.1	24.5	$0.15	163.3
48	NVIDIA: Nemotron Nano 12B 2 VL	64.9	$0.40	162.3
49	Qwen: Qwen3 VL 235B A22B Instruct	83.6	$0.54	154.8
50	Mistral: Saba	61.1	$0.40	152.8
51	Qwen: Qwen2.5 VL 72B Instruct	72.0	$0.50	144.0
52	MiniMax: MiniMax M2.1	87.5	$0.62	141.1
53	Qwen: Qwen3 Next 80B A3B Instruct	82.4	$0.60	138.5
54	MiniMax: MiniMax M2.5	87.5	$0.65	134.6
55	MiniMax: MiniMax M2	82.0	$0.63	130.7
56	Prime Intellect: INTELLECT-3	82.2	$0.65	126.5
57	Z.ai: GLM 4.6V	75.2	$0.60	125.3
58	MiniMax: MiniMax M2.7	87.5	$0.75	116.7
59	Baidu: ERNIE 4.5 300B A47B	77.6	$0.69	112.5
60	Kwaipilot: KAT-Coder-Pro V2	81.3	$0.75	108.4
61	DeepSeek: DeepSeek V3.2 Speciale	86.3	$0.80	107.9
62	DeepSeek: R1 Distill Llama 70B	79.5	$0.75	106.0
63	Qwen: Qwen3 235B A22B Thinking 2507	84.3	$0.82	102.5
64	Qwen: Qwen3 VL 8B Thinking	74.9	$0.74	101.1
65	Qwen: Qwen3 VL 30B A3B Thinking	80.7	$0.84	95.5
66	xAI: Grok Code Fast 1	79.3	$0.85	93.3
67	Nous: Hermes 3 405B Instruct	82.9	$1.00	82.9
68	OpenAI: GPT-4.1 Mini	78.1	$1.00	78.1
69	OpenAI: GPT-5.4 Nano	55.6	$0.72	76.7
70	Qwen2.5 Coder 32B Instruct	63.5	$0.83	76.5
71	OpenAI: GPT-5 Mini	83.7	$1.13	74.4
72	OpenAI: GPT-5.1-Codex-Mini	82.0	$1.13	72.9
73	Mistral: Mixtral 8x7B Instruct	38.7	$0.54	71.7
74	Perplexity: Sonar	68.9	$1.00	68.9
75	MoonshotAI: Kimi K2 0905	81.9	$1.20	68.3
76	MoonshotAI: Kimi K2.5	82.4	$1.22	67.5
77	Qwen: Qwen3 235B A22B	76.2	$1.14	67.0
78	Z.ai: GLM 4.5V	78.8	$1.20	65.7
79	Google: Gemini 2.5 Flash	88.2	$1.40	63.0
80	MiniMax: MiniMax M1	81.6	$1.30	62.8
81	Mistral: Devstral Medium	70.8	$1.20	59.0
82	Qwen: Qwen3 VL 235B A22B Thinking	83.6	$1.43	58.5
83	NVIDIA: Llama 3.1 Nemotron 70B Instruct	69.0	$1.20	57.5
84	MoonshotAI: Kimi K2 0711	82.4	$1.44	57.4
85	Mistral: Mistral Medium 3.1	68.3	$1.20	56.9
86	Mistral: Mistral Medium 3	68.3	$1.20	56.9
87	MoonshotAI: Kimi K2 Thinking	84.8	$1.55	54.7
88	DeepSeek: R1	84.9	$1.60	53.1
89	OpenAI: GPT-3.5 Turbo	46.2	$1.00	46.2
90	Qwen: Qwen3 Max Thinking	82.4	$2.34	35.2
91	Qwen: Qwen3 Max	76.2	$2.34	32.6
92	MoonshotAI: Kimi K2.6	82.4	$2.70	30.5
93	OpenAI: o4 Mini	83.2	$2.75	30.3
94	OpenAI: GPT-5.4 Mini	78.1	$2.63	29.8
95	OpenAI: o3 Mini High	80.2	$2.75	29.2
96	OpenAI: o3 Mini	79.1	$2.75	28.8
97	Anthropic: Claude Haiku 4.5	76.0	$3.00	25.3
98	Mistral Large	80.7	$4.00	20.2
99	Mistral: Pixtral Large 2411	70.1	$4.00	17.5
100	Mistral Large 2407	68.3	$4.00	17.1
101	OpenAI: o3	85.3	$5.00	17.1
102	OpenAI: GPT-5	87.1	$5.63	15.5
103	Google: Gemini 2.5 Pro	86.2	$5.63	15.3
104	OpenAI: GPT-5 Codex	86.0	$5.63	15.3
105	OpenAI: GPT-5.1-Codex	86.0	$5.63	15.3
106	Google: Gemini 2.5 Pro Preview 05-06	83.7	$5.63	14.9
107	OpenAI: GPT-4o	74.8	$6.25	12.0
108	AI21: Jamba Large 1.7	57.7	$5.00	11.5
109	Cohere: Command A	71.2	$6.25	11.4
110	OpenAI: GPT-5.2-Codex	86.0	$7.88	10.9
111	Anthropic: Claude 3.7 Sonnet (thinking)	83.7	$9.00	9.3
112	Anthropic: Claude Sonnet 4.5	77.2	$9.00	8.6
113	Anthropic: Claude Sonnet 4	77.2	$9.00	8.6
114	Anthropic: Claude Sonnet 4.6	77.2	$9.00	8.6
115	Perplexity: Sonar Pro	75.5	$9.00	8.4
116	xAI: Grok 3 Beta	70.3	$9.00	7.8
117	OpenAI: GPT-4o (2024-05-13)	74.0	$10.00	7.4
118	Cohere: Command R+ (08-2024)	43.2	$6.25	6.9
119	Anthropic: Claude Opus 4.7	69.6	$15.00	4.6
120	Anthropic: Claude Opus 4.6	69.6	$15.00	4.6
121	Anthropic: Claude Opus 4.5	69.6	$15.00	4.6
122	OpenAI: GPT-4 Turbo	69.4	$20.00	3.5
123	OpenAI: o1	84.1	$37.50	2.2
124	Anthropic: Claude Opus 4	69.6	$45.00	1.5