Catalog · updated 2026-09-01
Local LLM VRAM requirements, model by model
86 architectures, 26 of them mixture-of-experts designs that occupy the memory of a large model while running at the speed of a small one. Every figure below is derived from the model's own configuration file rather than a rule of thumb, which is why two models of the same size can want very different amounts of memory.
Most downloaded
Weights shown at Q4_K_M, cache at an 8k window.
| Model | Family | Params | Weights at Q4 | Cache at 8k | Context | Downloads |
|---|---|---|---|---|---|---|
| Qwen3 0.6B | Qwen | 0.75B | 0.4 GB | 0.88 GB | 40k | 22.7M |
| Qwen3 8B | Qwen | 8.19B | 4.6 GB | 1.13 GB | 40k | 13.6M |
| Qwen3.5 9B | Qwen | 9.65B | 5.4 GB | 0.30 GB | 256k | 12.6M |
| Qwen2.5 7B Instruct | Qwen | 7.62B | 4.3 GB | 0.44 GB | 32k | 10.6M |
| Gemma 4 31B Instruct | Gemma | 31.27B | 17.6 GB | 3.28 GB | 256k | 8.3M |
| Gemma 4 26B A4B Instruct | Gemma | 25.81B (3.74B) | 14.5 GB | 0.82 GB | 256k | 8.2M |
| Qwen2.5 VL 7B Instruct | Qwen | 8.29B | 4.7 GB | 0.44 GB | 125k | 8.0M |
| Qwen2.5 1.5B Instruct | Qwen | 1.54B | 0.9 GB | 0.22 GB | 32k | 7.7M |
| Qwen2.5 3B Instruct | Qwen | 3.09B | 1.7 GB | 0.28 GB | 32k | 7.6M |
| Qwen3.5 4B | Qwen | 4.66B | 2.6 GB | 0.30 GB | 256k | 7.4M |
| Llama 3.2 1B Instruct | Llama | 1.24B | 0.7 GB | 0.25 GB | 128k | 6.7M |
| GPT OSS 20B | GPT-OSS | 20.91B (4.18B) | 11.3 GB | 0.19 GB | 128k | 6.5M |
Under 3B
Runs on anything, including a phone or a laptop with no dedicated GPU. 15 models in this range.
Qwen3 0.6B
0.75B · 0.4 GB at Q4
Qwen2.5 1.5B Instruct
1.54B · 0.9 GB at Q4
Llama 3.2 1B Instruct
1.24B · 0.7 GB at Q4
Qwen2.5 0.5B Instruct
0.49B · 0.3 GB at Q4
Qwen3 1.7B
2.03B · 1.1 GB at Q4
Gemma 3 1B Instruct
1B · 0.6 GB at Q4
Qwen3.5 2B
2.27B · 1.3 GB at Q4
Qwen3.5 0.8B
0.87B · 0.5 GB at Q4
OLMo 2 0425 1B
1.48B · 0.8 GB at Q4
DeepSeek R1 Distill Qwen 1.5B
1.78B · 1.0 GB at Q4
LFM2.5 1.2B Instruct
1.17B · 0.7 GB at Q4
SmolLM2 1.7B
1.71B · 1.0 GB at Q4
LFM2.5 2.6B
2.7B · 1.5 GB at Q4
Gemma 2 2B Instruct
2.61B · 1.5 GB at Q4
LFM2.5 350M
0.35B · 0.2 GB at Q4
3B to 9B
The sweet spot for an 8 GB card. Good general chat, weak at hard reasoning. 24 models in this range.
Qwen3 8B
8.19B · 4.6 GB at Q4
Qwen2.5 7B Instruct
7.62B · 4.3 GB at Q4
Qwen2.5 VL 7B Instruct
8.29B · 4.7 GB at Q4
Qwen2.5 3B Instruct
3.09B · 1.7 GB at Q4
Qwen3.5 4B
4.66B · 2.6 GB at Q4
Qwen3 4B
4.02B · 2.3 GB at Q4
Llama 3.1 8B Instruct
8.03B · 4.5 GB at Q4
Gemma 4 E4B Instruct
8B · 4.5 GB at Q4
Gemma 4 E2B Instruct
5.12B · 2.9 GB at Q4
Mistral 7B Instruct V0.3
7.25B · 4.1 GB at Q4
Qwen2.5 Coder 7B Instruct
7.62B · 4.3 GB at Q4
Gemma 3 4B Instruct
4.3B · 2.4 GB at Q4
Llama 3.2 3B Instruct
3.21B · 1.8 GB at Q4
Mistral 7B Instruct V0.2
7.24B · 4.1 GB at Q4
Granite 4.1 8B
8.79B · 4.9 GB at Q4
DeepSeek Coder 7B Instruct V1.5
6.91B · 3.9 GB at Q4
Phi 3 Mini 4k Instruct
3.82B · 2.1 GB at Q4
Phi 4 Mini Instruct
3.84B · 2.2 GB at Q4
and 6 more in this range.
9B to 32B
Needs 12 to 24 GB. This is where local models start feeling genuinely useful. 21 models in this range.
Qwen3.5 9B
9.65B · 5.4 GB at Q4
Gemma 4 31B Instruct
31.27B · 17.6 GB at Q4
Gemma 4 26B A4B Instruct
25.81B · 3.74B active · 14.5 GB at Q4
GPT OSS 20B
20.91B · 4.18B active · 11.3 GB at Q4
Gemma 4 12B Instruct
11.96B · 6.7 GB at Q4
Qwen2.5 14B Instruct
14.77B · 8.3 GB at Q4
Qwen3.5 27B
27.78B · 15.6 GB at Q4
Qwen3 30B A3B
30.53B · 3.34B active · 17.2 GB at Q4
GLM 4.7 Flash
31.22B · 3.66B active · 17.6 GB at Q4
Qwen3 14B
14.77B · 8.3 GB at Q4
Qwen2.5 Coder 14B Instruct
14.77B · 8.3 GB at Q4
Gemma 3 12B Instruct
12.19B · 6.9 GB at Q4
DeepSeek Coder V2 Lite Instruct
15.71B · 2.74B active · 8.8 GB at Q4
Gemma 2 9B Instruct
9.24B · 5.2 GB at Q4
Phi 4
14.66B · 8.2 GB at Q4
Qwen3 Coder 30B A3B Instruct
30.53B · 3.34B active · 17.2 GB at Q4
Gemma 3 27B Instruct
27.43B · 15.4 GB at Q4
Mistral Nemo Instruct 2407
12.25B · 6.9 GB at Q4
and 3 more in this range.
32B to 80B
A 24 GB card at low precision, or unified memory. Slow but strong. 11 models in this range.
Qwen3 32B
32.76B · 18.4 GB at Q4
Qwen3.5 35B A3B
35.95B · 2.9B active · 20.2 GB at Q4
Qwen2.5 32B Instruct
32.76B · 18.4 GB at Q4
Qwen2.5 Coder 32B Instruct
32.76B · 18.4 GB at Q4
Llama 3.3 70B Instruct
70.55B · 39.7 GB at Q4
Qwen3 Coder Next
79.67B · 3.19B active · 44.8 GB at Q4
Qwen2.5 72B Instruct
72.71B · 40.9 GB at Q4
Mixtral 8x7B Instruct V0.1
46.7B · 12.88B active · 26.3 GB at Q4
Qwen2.5 VL 72B Instruct
73.41B · 41.3 GB at Q4
Llama 3 3 Nemotron Super 49B V1
49.87B · 28.0 GB at Q4
Phi 3.5 MoE Instruct
41.87B · 6.64B active · 23.5 GB at Q4
Over 80B
Multi-GPU, a big Mac, or a server. Mixture-of-experts models are the exception. 15 models in this range.
GPT OSS 120B
116.83B · 5.7B active · 59.0 GB at Q4
DeepSeek R1
684.53B · 38.57B active · 384.9 GB at Q4
DeepSeek V3.2
685.4B · 38.57B active · 385.4 GB at Q4
GLM 5.2
753.33B · 51.62B active · 423.6 GB at Q4
MiniMax M2.7
228.69B · 10.98B active · 128.6 GB at Q4
Qwen3.5 122B A10B
125.09B · 8.17B active · 70.3 GB at Q4
Kimi K2.6
1026.88B · 39.06B active · 577.4 GB at Q4
MiniMax M2.5
228.7B · 10.98B active · 128.6 GB at Q4
Qwen3 235B A22B
235.09B · 22.14B active · 132.2 GB at Q4
Qwen3 Next 80B A3B Instruct
81.32B · 3.19B active · 45.7 GB at Q4
MiniMax M3
427.04B · 25.86B active · 240.1 GB at Q4
Kimi K2 Instruct
1026.41B · 39.06B active · 577.1 GB at Q4
GLM 4.5 Air
110.47B · 13.4B active · 62.1 GB at Q4
GLM 4.5
358.34B · 33.56B active · 201.5 GB at Q4
GLM 5.1
753.86B · 35.91B active · 423.9 GB at Q4
By family
Gemma
11 models
Mistral
6 models
DeepSeek
5 models
Llama
5 models