Small Local LLMs: Intelligence Over Time

Intelligence over time

Each dot is one open-weight model release. Size ∝ parameters. Line traces the best score achieved to date.

What could I run on 24 GB?

Best MMLU achievable in a given VRAM budget, quarter by quarter.

Today's champion at ≤ 24 GB

Qwen 3.6 27B

89% MMLU

27.8B params · 24 GB @ Q6 · released Apr 2026

Quarter	Best model (≤ 24 GB)	Params	MMLU
2023-Q1	LLaMA 33B	33B	57.8%
2023-Q3	Mistral 7B	7.3B	60.1%
2023-Q4	Mistral 7B	7.3B	60.1%
2024-Q1	Gemma 7B	7B	64.3%
2024-Q2	Phi-3 medium 14B	14B	78%
2024-Q3	Qwen2.5 32B Instruct	32.5B	83.3%
2024-Q4	Phi-4 14B	14B	84.8%
2025-Q1	Phi-4 14B	14B	84.8%
2025-Q2	Qwen3 32B	32.8B	85.1%
2025-Q3	Qwen3 32B	32.8B	85.1%
2026-Q1	GLM-4.7-Flash	31.2B	86%
2026-Q2	Qwen 3.6 27B	27.8B	89%

Milestones

Sep 2023
Mistral-7B matches Llama-2-13B
First widely-used 7B to leap past a 13B on MMLU (60.1 vs 54.8) — the 'smaller can win' era begins.
Apr 2024
Phi-3-mini fits Llama-2-70B on a phone
3.8B parameters hit MMLU 68.8, matching Meta's 70B from nine months earlier at ~20× less VRAM.
Sep 2024
Qwen2.5-7B passes GPT-3.5-class coding
HumanEval 84.8 on a 5 GB laptop model — coding capability once reserved for API-only frontier models.
Dec 2024
Phi-4 14B crosses MMLU 85
A 14B dense model reaches accuracy Meta needed 70B for in April, on the same 9 GB VRAM budget.
Jan 2025
DeepSeek-R1 distills bring o1-class reasoning to 32B
Chain-of-thought reasoning distilled from a 671B teacher into a 32B student runnable on a single 24 GB card.
Aug 2025
gpt-oss-20B rivals proprietary mid-tier
OpenAI-released open weights: 20B MoE (3.6B active) at MMLU 82.5, HumanEval 89.5 — proprietary quality, apache-2.0 license.
Apr 2026
Qwen 3.6 27B: frontier fits in 24 GB
A single consumer GPU now runs a model that would have been GPT-4-class two years earlier.

Compare tiers by:

By size class · MMLU

Running best score for dense models in each parameter tier. Smaller tiers catch up to yesterday's giants.

≤ 3B

56.7%

≤ 8B

74.2%

≤ 14B

84.8%

≤ 32B

89%

Full dataset

47 model releases. Click a column to sort.

Date ↓	Model	Family	Params	VRAM	MMLU	HumanEval	Arena Elo	License
2026-04-21	Qwen 3.6 27B	Qwen 3	27.8B	24 GB	89	94	1395	apache-2.0
2026-03-11	Gemma 4 26B-A4B IT	Gemma	26.5 (3.8a)B	28 GB	87.4	93	1370	apache-2.0
2026-03-11	Gemma 4 31B IT	Gemma	32.7B	29 GB	88.1	93.4	1381	apache-2.0
2026-01-19	GLM-4.7-Flash	GLM	31.2 (12a)B	23 GB	86	92.5	1352	mit
2025-09-09	Qwen3-Next 80B-A3B	Qwen 3	81.3 (3a)B	53 GB	85.8	91.8	1348	apache-2.0
2025-08-04	gpt-oss 20B	gpt-oss	21.5 (3.6a)B	23 GB	82.5	89.5	1315	apache-2.0
2025-08-04	gpt-oss 120B	gpt-oss	120.4 (5.1a)B	51 GB	87.2	92.3	1361	apache-2.0
2025-07-20	GLM-4.5-Air	GLM	110.5 (12a)B	60 GB	84.7	91.5	1341	mit
2025-04-27	Qwen3 14B	Qwen 3	14.8B	12 GB	82.4	88.4	1288	apache-2.0
2025-04-27	Qwen3 32B	Qwen 3	32.8B	20 GB	85.1	90.2	1332	apache-2.0
2025-04-27	Qwen3 30B-A3B	Qwen 3	30.5 (3a)B	32 GB	82.7	89.6	1304	apache-2.0
2025-04-02	Llama 4 Scout 17B-16E	Llama 4	109 (17a)B	60 GB	79.6	79.6	1272	llama-4
2025-03-12	Gemma 3 27B IT	Gemma	27B	17 GB	78.6	87.8	1338	gemma
2025-03-05	QwQ 32B	Qwen	32.8B	20 GB	82.5	89	1314	apache-2.0
2025-01-20	DeepSeek-R1-Distill-Qwen 32B	DeepSeek	32.8B	20 GB	82.9	88.4	1305	mit
2024-12-12	Phi-4 14B	Phi	14B	9 GB	84.8	82.6	1229	mit
2024-11-11	Qwen2.5-Coder 32B	Qwen	32.5B	20 GB	75.1	92.7	—	apache-2.0
2024-11-01	SmolLM2 1.7B	SmolLM	1.7B	1 GB	50.3	23.3	—	apache-2.0
2024-09-25	Llama 3.2 1B	Llama 3	1.2B	1 GB	32.2	—	—	llama-3.2
2024-09-25	Llama 3.2 3B	Llama 3	3.2B	2 GB	63.4	—	1103	llama-3.2
2024-09-19	Qwen2.5 7B Instruct	Qwen	7B	5 GB	74.2	84.8	1189	apache-2.0
2024-09-19	Qwen2.5 14B Instruct	Qwen	14B	9 GB	79.7	83.5	1207	apache-2.0
2024-09-19	Qwen2.5 32B Instruct	Qwen	32.5B	20 GB	83.3	88.4	1257	apache-2.0
2024-09-19	Qwen2.5 72B Instruct	Qwen	72B	42 GB	86.1	86.6	1268	tongyi-qianwen
2024-09-17	Mistral Small 22B	Mistral	22B	13 GB	72	42	1119	mrl
2024-07-23	Llama 3.1 8B Instruct	Llama 3	8B	6 GB	73	72.6	1176	llama-3.1
2024-07-23	Llama 3.1 70B Instruct	Llama 3	70B	40 GB	83.6	80.5	1247	llama-3.1
2024-07-18	Mistral Nemo 12B	Mistral	12.2B	8 GB	68	40.3	1073	apache-2.0
2024-06-27	Gemma 2 9B	Gemma	9B	6 GB	71.3	40.2	1189	gemma
2024-06-27	Gemma 2 27B	Gemma	27B	17 GB	75.2	51.8	1218	gemma
2024-06-06	Qwen2 7B Instruct	Qwen	7B	5 GB	70.3	79.9	1141	apache-2.0
2024-06-06	Qwen2 72B Instruct	Qwen	72B	42 GB	82.3	86	1187	tongyi-qianwen
2024-05-21	Phi-3 medium 14B	Phi	14B	9 GB	78	62.2	—	mit
2024-04-23	Phi-3 mini 3.8B	Phi	3.8B	3 GB	68.8	58.5	1037	mit
2024-04-18	Llama 3 8B Instruct	Llama 3	8B	6 GB	68.4	62.2	1152	llama-3
2024-04-18	Llama 3 70B Instruct	Llama 3	70B	40 GB	82	81.7	1206	llama-3
2024-02-21	Gemma 7B	Gemma	7B	5 GB	64.3	32.3	1037	gemma
2023-12-13	Phi-2 2.7B	Phi	2.7B	2 GB	56.7	47	—	mit
2023-12-11	Mixtral 8x7B	Mistral	46.7 (12.9a)B	28 GB	70.6	40.2	1114	apache-2.0
2023-09-27	Mistral 7B	Mistral	7.3B	5 GB	60.1	30.5	1071	apache-2.0
2023-07-18	Llama 2 7B	Llama 2	7B	5 GB	45.3	12.8	1037	llama-2
2023-07-18	Llama 2 13B	Llama 2	13B	8 GB	54.8	18.3	1063	llama-2
2023-07-18	Llama 2 70B	Llama 2	70B	40 GB	68.9	29.9	1093	llama-2
2023-02-24	LLaMA 7B	LLaMA	7B	14 GB	35.1	10.5	—	llama-1
2023-02-24	LLaMA 13B	LLaMA	13B	26 GB	46.9	15.8	—	llama-1
2023-02-24	LLaMA 33B	LLaMA	33B	20 GB	57.8	21.7	—	llama-1
2023-02-24	LLaMA 65B	LLaMA	65B	38 GB	63.4	23.7	—	llama-1

Intelligence over time

What could I run on 24 GB?

Milestones

Size class comparison

By size class · MMLU

Full dataset