Local LLM VRAM Calculator: Will It Run on Your GPU?

Your GPU or machine

Common picks

Context window

KV cache precision

Number of GPUs

Operating system

System RAM

System RAM speed

RAM only matters when a model does not fit the card, and its speed then sets how fast the part in RAM runs. Several GPUs are modeled as a layer split: memory adds up and bandwidth does not.

What runs on the GeForce RTX 3060 12GB

12 GB, 360 GB/s, 8k context. 57 of 86 models fit entirely in memory.

Model	Params	Verdict	Best fit	Memory used	Speed	Max context
Gemma 4 12B Instruct Gemma	11.96B	Just fits	Q6_K	10.84 GB	25.4 t/s	8k
Qwen3.5 9B Qwen	9.65B	Runs great	Q8_0	9.25 GB	30.0 t/s	64k
Qwen3 14B Qwen	14.77B	Just fits	Q5_K_M	10.96 GB	25.2 t/s	8k
Gemma 3 12B Instruct Gemma	12.19B	Just fits	Q6_K	10.52 GB	26.2 t/s	16k
Gemma 4 E4B Instruct Gemma	8B	Runs great	Q8_0	8.42 GB	32.9 t/s	128k
Granite 4.1 8B Granite	8.79B	Just fits	Q8_0	10.35 GB	26.6 t/s	8k
Qwen2.5 14B Instruct Qwen	14.77B	Runs great	Q4_K_M	9.81 GB	28.3 t/s	8k
Qwen3 8B Qwen	8.19B	Runs great	Q8_0	9.02 GB	30.8 t/s	16k
Qwen2.5 Coder 14B Instruct Qwen	14.77B	Runs great	Q4_K_M	9.81 GB	28.3 t/s	8k
Qwen2.5 VL 7B Instruct Qwen	8.29B	Runs great	Q8_0	8.49 GB	32.7 t/s	32k
Phi 4 Phi	14.66B	Runs great	Q4_K_M	9.96 GB	27.8 t/s	8k
Mistral Nemo Instruct 2407 Mistral	12.25B	Just fits	Q6_K	10.53 GB	26.2 t/s	8k
Qwen3.5 27B Qwen	27.78B	Fits only at 2 to 3 bits	IQ3_XXS!	10.53 GB	26.3 t/s	n/a
Qwen2.5 7B Instruct Qwen	7.62B	Runs great	Q8_0	7.83 GB	35.6 t/s	32k
Llama 3.1 8B Instruct Llama	8.03B	Runs great	Q8_0	8.83 GB	31.4 t/s	16k
Qwen2.5 Coder 7B Instruct Qwen	7.62B	Runs great	Q8_0	7.83 GB	35.6 t/s	32k
Gemma 2 9B Instruct Gemma	9.24B	Runs great	Q6_K	9.42 GB	29.3 t/s	8k
OLMo 3 7B Instruct OLMo	7.3B	Runs great	Q8_0	9.72 GB	28.4 t/s	16k
Internlm3 8B Instruct InternLM	8.8B	Runs great	Q8_0	8.97 GB	30.9 t/s	32k
Mistral 7B Instruct V0.3 Mistral	7.25B	Runs great	Q8_0	8.45 GB	32.9 t/s	16k
DeepSeek Coder V2 Lite Instruct DeepSeek	15.71B (2.74B active)	Just fits	Q5_K_M	10.84 GB	75.7 t/s	16k
Gemma 4 E2B Instruct Gemma	5.12B	Runs great	Q8_0	5.44 GB	51.7 t/s	128k
Qwen3.5 4B Qwen	4.66B	Runs great	Q8_0	5.27 GB	54.0 t/s	128k
Mistral 7B Instruct V0.2 Mistral	7.24B	Runs great	Q8_0	8.44 GB	33.0 t/s	16k
DeepSeek Coder 7B Instruct V1.5 DeepSeek	6.91B	Just fits	Q8_0	10.58 GB	26.0 t/s	4k
Mistral Small 24B Instruct 2501 Mistral	23.57B	Fits only at 2 to 3 bits	IQ3_XXS!	9.85 GB	28.2 t/s	n/a
DeepSeek Coder 6.7B Instruct DeepSeek	6.74B	Just fits	Q8_0	10.94 GB	25.1 t/s	8k
LFM2.5 8B A1B Liquid	8.47B (1.57B active)	Runs great	Q8_0	8.81 GB	93.3 t/s	64k
Qwen3 4B Qwen	4.02B	Runs great	Q8_0	5.46 GB	51.9 t/s	32k
Codestral 22B V0.1 Mistral	22.25B	Fits only at 2 to 3 bits	IQ3_XXS!	10.08 GB	27.6 t/s	n/a
Gemma 3 4B Instruct Gemma	4.3B	Runs great	Q8_0	4.91 GB	58.2 t/s	128k
Phi 4 Mini Instruct Phi	3.84B	Runs great	Q8_0	5.17 GB	55.2 t/s	32k
Granite 4.1 3B Granite	3.4B	Runs great	Q8_0	4.34 GB	66.4 t/s	64k
Qwen3 32B Qwen	32.76B	Fits only at 2 to 3 bits	IQ2_XXS!	10.11 GB	27.4 t/s	n/a
Qwen2.5 3B Instruct Qwen	3.09B	Runs great	Q8_0	3.68 GB	79.3 t/s	32k
Phi 3 Mini 4k Instruct Phi	3.82B	Runs great	Q8_0	4.8 GB	59.8 t/s	4k
Llama 3.2 3B Instruct Llama	3.21B	Runs great	Q8_0	4.42 GB	65.4 t/s	64k
Granite 4.1 30B Granite	28.87B	Fits only at 2 to 3 bits	IQ2_XXS!	9.33 GB	29.7 t/s	n/a
Qwen2.5 32B Instruct Qwen	32.76B	Fits only at 2 to 3 bits	IQ2_XXS!	10.11 GB	27.4 t/s	n/a
Qwen2.5 Coder 32B Instruct Qwen	32.76B	Fits only at 2 to 3 bits	IQ2_XXS!	10.11 GB	27.4 t/s	n/a
LFM2.5 2.6B Liquid	2.7B	Runs great	Q8_0	3.14 GB	94.7 t/s	128k
Qwen3.5 2B Qwen	2.27B	Runs great	Q8_0	2.7 GB	112 t/s	256k
Gemma 3 27B Instruct Gemma	27.43B	Fits only at 2 to 3 bits	IQ2_XXS!	8.11 GB	34.6 t/s	n/a
Gemma 2 2B Instruct Gemma	2.61B	Runs great	Q8_0	3.54 GB	83.0 t/s	8k
Starcoder2 3B StarCoder	3.03B	Runs great	Q8_0	3.34 GB	89.3 t/s	16k
Qwen3 1.7B Qwen	2.03B	Runs great	Q8_0	3.22 GB	91.8 t/s	32k
DeepSeek R1 Distill Qwen 1.5B Qwen	1.78B	Runs great	Q8_0	2.07 GB	151 t/s	128k
Qwen2.5 1.5B Instruct Qwen	1.54B	Runs great	Q8_0	2.06 GB	152 t/s	32k
SmolLM2 1.7B SmolLM	1.71B	Runs great	Q8_0	3.53 GB	83.0 t/s	8k
OLMo 2 0425 1B OLMo	1.48B	Runs great	Q8_0	2.6 GB	117 t/s	4k
Llama 3.2 1B Instruct Llama	1.24B	Runs great	Q8_0	1.82 GB	179 t/s	128k
LFM2.5 1.2B Instruct Liquid	1.17B	Runs great	Q8_0	1.46 GB	237 t/s	64k
Gemma 3 1B Instruct Gemma	1B	Runs great	Q8_0	1.35 GB	255 t/s	32k
Qwen3.5 0.8B Qwen	0.87B	Runs great	Q8_0	1.28 GB	272 t/s	256k
Qwen3 0.6B Qwen	0.75B	Runs great	Q8_0	1.92 GB	164 t/s	32k
Qwen2.5 0.5B Instruct Qwen	0.49B	Runs great	Q8_0	0.88 GB	458 t/s	32k
LFM2.5 350M Liquid	0.35B	Runs great	Q8_0	0.68 GB	709 t/s	64k
GPT OSS 20B GPT-OSS	20.91B (4.18B active)	Partial offload experts in RAM	MXFP4	11.55 GB	65.7 t/s	n/a
Qwen3.5 35B A3B Qwen	35.95B (2.9B active)	Partial offload experts in RAM	Q4_K_M	20.48 GB	55.7 t/s	n/a
GLM 4.7 Flash GLM	31.22B (3.66B active)	Partial offload experts in RAM	Q4_K_M	18.13 GB	37.6 t/s	n/a
Gemma 4 26B A4B Instruct Gemma	25.81B (3.74B active)	Partial offload experts in RAM	Q4_K_M	15.7 GB	37.6 t/s	n/a
Qwen3 30B A3B Qwen	30.53B (3.34B active)	Partial offload experts in RAM	Q4_K_M	18.08 GB	36.3 t/s	n/a
Qwen3 Coder 30B A3B Instruct Qwen	30.53B (3.34B active)	Partial offload experts in RAM	Q4_K_M	18.08 GB	36.3 t/s	n/a
Qwen3 Coder Next Qwen	79.67B (3.19B active)	Partial offload experts in RAM	Q4_K_M	45.22 GB	33.2 t/s	n/a
Qwen3 Next 80B A3B Instruct Qwen	81.32B (3.19B active)	Partial offload experts in RAM	Q4_K_M	46.15 GB	32.7 t/s	n/a
GPT OSS 120B GPT-OSS	116.83B (5.7B active)	Partial offload experts in RAM	MXFP4	59.4 GB	16.9 t/s	n/a
Phi 3.5 MoE Instruct Phi	41.87B (6.64B active)	Partial offload experts in RAM	Q4_K_M	24.87 GB	13.1 t/s	n/a
Qwen3.5 122B A10B Qwen	125.09B (8.17B active)	Partial offload experts in RAM	Q4_K_M	70.61 GB	12.6 t/s	n/a
GLM 4.5 Air GLM	110.47B (13.4B active)	Partial offload experts in RAM	Q4_K_M	63.61 GB	6.9 t/s	n/a
MiniMax M2.5 MiniMax	228.7B (10.98B active)	Partial offload experts in RAM	Q4_K_M	130.56 GB	6.6 t/s	n/a
MiniMax M2.7 MiniMax	228.69B (10.98B active)	Partial offload experts in RAM	Q4_K_M	130.55 GB	6.6 t/s	n/a
Mixtral 8x7B Instruct V0.1 Mistral	46.7B (12.88B active)	Partial offload experts in RAM	Q4_K_M	27.59 GB	6.1 t/s	n/a
MiniMax M3 MiniMax	427.04B (25.86B active)	Partial offload experts in RAM	Q4_K_M	240.84 GB	3.5 t/s	n/a
Qwen3 235B A22B Qwen	235.09B (22.14B active)	Partial offload experts in RAM	Q4_K_M	133.71 GB	3.4 t/s	n/a
Gemma 4 31B Instruct Gemma	31.27B	Partial offload 30/60 layers	n/a	21.31 GB	2.6 t/s	n/a
Kimi K2.6 Kimi	1026.88B (39.06B active)	Partial offload experts in RAM	Q4_K_M	577.78 GB	2.2 t/s	n/a
Kimi K2 Instruct Kimi	1026.41B (39.06B active)	Partial offload experts in RAM	Q4_K_M	577.52 GB	2.2 t/s	n/a
DeepSeek V3.2 DeepSeek	685.4B (38.57B active)	Partial offload experts in RAM	Q4_K_M	385.91 GB	2.2 t/s	n/a
DeepSeek R1 DeepSeek	684.53B (38.57B active)	Partial offload experts in RAM	Q4_K_M	385.42 GB	2.2 t/s	n/a
GLM 5.1 GLM	753.86B (35.91B active)	Partial offload experts in RAM	Q4_K_M	424.51 GB	2.1 t/s	n/a
GLM 4.5 GLM	358.34B (33.56B active)	Partial offload 4/92 layers	n/a	204.37 GB	1.4 t/s	n/a
GLM 5.2 GLM	753.33B (51.62B active)	Partial offload 1/78 layers	n/a	424.21 GB	1.0 t/s	n/a
Qwen2.5 VL 72B Instruct Qwen	73.41B	Partial offload 19/80 layers	n/a	43.62 GB	0.9 t/s	n/a
Qwen2.5 72B Instruct Qwen	72.71B	Partial offload 19/80 layers	n/a	43.22 GB	0.9 t/s	n/a
Llama 3.3 70B Instruct Llama	70.55B	Partial offload 20/80 layers	n/a	42.12 GB	0.9 t/s	n/a
Llama 3 3 Nemotron Super 49B V1 Llama	49.87B	Partial offload 17/80 layers	n/a	47.99 GB	0.8 t/s	n/a

Speeds are single-stream estimates. Mixture-of-experts speeds are good to about 30%, and non-NVIDIA backends use efficiency factors we have not measured. What we measured and what we did not.

The filters need JavaScript. The table above is the result for the GeForce RTX 3060 12GB, and every model and every device also has a static page with the same numbers: browse all models or all hardware.

Asking it the other way round

If you have not bought the hardware yet, start from the model instead.

What do I need to buy?

Pick the model and how well you want it to run. We work back to the cheapest card or machine that gets there, used parts included.

Can my card run it?

A direct answer for popular pairings, with the settings we would use and what to run instead when the answer is no.

Compare two models

Side by side on the same device. The gap usually shows up in the context column rather than the weights.

Which GPU should I buy?

Memory decides what loads, bandwidth decides how fast it answers, and they are separate purchases.

What the numbers mean

Weights are only part of it

A 4-bit 8B model is about 4.8 GB of weights. Loaded with a 32k window a classic design wants closer to 11 GB, because the attention cache grows with every token in the conversation.

Same-size models can differ by many gigabytes

Grouped-query, sliding-window and linear attention cut the cache by 4 to 30 times. We read each model's real layout rather than assuming, which is why our answers sometimes disagree with rules of thumb.

Speed follows bandwidth, not TFLOPS

Generating a token means reading every active weight from memory. That makes memory bandwidth the ceiling, which is why a fast card with slow memory disappoints.

Not fitting is not the end

Runtimes split a model between GPU and CPU, and mixture-of-experts models can keep their experts in RAM. We say how much lands on the card and what that does to the speed, instead of stopping at "no".

Start with a guide

Plain explanations built from the same engine as the calculator, so their numbers match it.

How much VRAM do you need for a local LLM?

An 8B model at 4-bit needs about 6 GB, not 8. The formula behind that number, and why it moves with the model and the context window.

Best GPU for local LLMs by budget: what to buy at each price

Which graphics card to buy for running models at home, ranked by memory and bandwidth rather than by gaming scores, with the strongest model each one runs.

KV cache explained: why a long context window eats your VRAM

The KV cache is the memory nobody budgets for. What it holds, why it grows with every token, and the four designs that keep it small.

What can you run with 12 GB of VRAM? 14B models, 20B experts and long context

12 GB is the first tier where local models feel roomy: 14B models at Q5, a 20B mixture-of-experts at the edge, and a used card that doubles the speed.

All guides

Common questions

How much VRAM do I need to run a local AI model?

Roughly 0.6 GB per billion parameters at 4 bits for the weights, plus the KV cache, plus about 1 GB of runtime overhead. An 8B model at Q4_K_M needs about 5.63 GB at an 8k context window, so an 8 GB card fits it with room to spare and a 6 GB card is a squeeze.

What happens if a model does not fit in VRAM?

Runtimes such as llama.cpp and Ollama put as many layers as fit on the GPU and run the rest on the CPU from system RAM. It still works, but speed drops sharply, because every token waits for the slow part. On our own RTX 3050, moving the last 5 of 28 layers to the card was worth more than the first 12 together.

Does context length change how much VRAM I need?

Yes, and it is the part people forget. The KV cache grows with every token in the window. On a classic model a 32k window can cost more memory than the weights, while models with sliding-window or linear attention pay a fraction of that. The calculator reads each model's real attention design.

How accurate are the estimates?

We checked them on a real machine. Against Qwen2.5 7B on an RTX 3050 6 GB our memory estimate is off by 2.1% on average and our speed estimate by 2.5%. That is one machine and one model, and the validation page says what it does and does not show.

Is this free, and does it store anything about me?

It is free and needs no account. The calculator runs in your browser and your choices never leave it. The site carries advertising and Amazon affiliate links, and the privacy page explains what those services collect and how to decline.

Will that local AI model run on your GPU?