How much VRAM do I need to run a local AI model?

Roughly 0.6 GB per billion parameters at 4-bit, plus the KV cache, plus about 1 GB of overhead. An 8B model at Q4_K_M needs about 6.4 GB at 8k context, so an 8 GB card fits it and a 6 GB card does not.

What happens if a model does not fit in VRAM?

Runtimes such as llama.cpp and Ollama put as many layers as fit on the GPU and run the rest on the CPU. It still works, but speed drops sharply because system RAM is roughly ten times slower than GPU memory.

Does context length change how much VRAM I need?

Yes, and it is the part people forget. The KV cache grows linearly with context. On some models a 32k window costs more memory than the weights themselves, while models with grouped-query attention pay a fraction of that.

Local AI, checked before you download

Will that model run on your machine?

Pick your hardware. We work out which models fit, which quantisation to download, how much room your context window leaves, and roughly how fast it will answer. No sign-up, nothing leaves your browser.

Your GPU or machine

Common picks

Context window

Operating system

System RAM

Results

Model	Params	Verdict	Best fit	VRAM used	Speed	Max context
Loading the catalogue…

The calculator needs JavaScript. Every model and every device also has a static page with the same numbers: browse all models or all hardware.

Asking it the other way round

If you have not bought the hardware yet, start from the model instead.

What do I need to buy?

Pick the model and how well you want it to run. We work back to the cheapest card or machine that gets there, used parts included.

Can my card run it?

A direct yes or no for popular pairings, with what to run instead when the answer is no.

Compare two models

Side by side on the same device. The gap usually shows up in the context column rather than the weights.

Which GPU should I buy?

Memory decides what loads, bandwidth decides how fast it answers, and they are separate purchases.

What the numbers mean

Weights are only part of it

A 4-bit 8B model is 4.8 GB of weights. Loaded with a 32k window it wants closer to 11 GB, because the attention cache grows with every token in the conversation.

Two same-size models can differ by 12 GB

Grouped-query attention cuts the cache by four to eight times. We read each model's real KV-head count rather than assuming, which is why our answers sometimes disagree with rules of thumb.

Speed follows bandwidth, not TFLOPS

Generating a token means reading every active weight from memory. That makes memory bandwidth the ceiling, which is why a fast card with slow memory disappoints.

Not fitting is not the end

Runtimes split a model between GPU and CPU. We tell you how many layers land on the card and what that does to the speed, instead of stopping at "no".