How we calculate VRAM and speed for local AI models
Everything on this site is arithmetic on published specifications, checked against one real machine. Here is every formula and every constant, including which ones we measured and which we have not been able to.
Where the numbers come from
Model architectures are read from each model's own config.json on Hugging Face,
and parameter counts from the safetensors index published beside the weights. Nothing about a
model's shape is typed in by hand, so we cannot get a layer count wrong, and the site follows
the models instead of lagging behind them. The catalog holds 86 architectures and was
last refreshed on 2026-09-01. A model whose attention layout we cannot establish from
its configuration is left out, because a memory figure we cannot stand behind is worse than none.
Hardware specifications come from vendor datasheets: memory capacity, peak memory bandwidth and FP16 tensor throughput, for 131 profiles. Chips with unified memory are expanded into one profile per memory option, because a 36 GB machine and a 128 GB machine with the same chip are different machines for this purpose. Prices live in a separate file, reviewed by hand and dated 2026-09-01, because a price goes stale and a bandwidth does not.
Memory
Three terms, added.
Weights. Parameters times the bits per weight of the quantization, divided by eight. We use values measured on real files, not nominal ones: a Q4_K_M file carries block scales and keeps some tensors at higher precision, so it lands at 4.83 bits per weight and not 4. Models released in a native low-precision format, such as gpt-oss in MXFP4, are the exception. Their files are the same size whatever the quantization is called, so for those we use the file size read from the repository.
KV cache. Two tensors, times the layers that keep a cache, times the KV heads, the head dimension, the context length and the element size. Models do not keep a cache the same way in every layer, and we read the kind of each layer from the configuration. Full attention grows with the context. Sliding-window layers stop growing at their window, which is why Gemma carries a cache a fraction of the size you would guess. Linear layers, in the hybrid models, keep a small fixed state instead. Compressed attention, in DeepSeek, stores one small vector per layer. Our first catalog treated all of these as full attention and overstated some caches by 4 times, which is why the layout is now per layer and covered by tests.
Overhead. A flat allocation for the runtime and its graph, a buffer that scales a little with hidden size, and the logits buffer. We assume flash attention, which keeps the scratch memory flat as the window grows and is the default in current runtimes. On a discrete card the input embedding table stays in system RAM, so it is not counted against VRAM.
From the total we subtract what the operating system has already taken: 0.8 GB on a Windows display adapter, 0.3 GB on Linux with a desktop, 0.1 GB headless. On unified memory we apply the share the operating system lets the GPU wire down, which on macOS is between 67 and 75 percent depending on how much is installed.
Speed
Generating one token means reading every active weight, plus the cache, out of memory. That makes single-stream generation bandwidth-bound, so the estimate is the device's peak bandwidth times an efficiency factor, divided by the bytes read per token.
Two of the efficiency factors are measured. On our RTX 3050 the CUDA factor is 0.79, and on the same machine's CPU the factor is 0.63, both from the runs on the validation page. The others are estimates we have not been able to check on hardware: 0.67 for ROCm, 0.73 for Metal and 0.61 for Vulkan. We say so wherever a result on those backends is shown, and the plan is to replace each with a measurement as machines become available.
When a model does not fit we work out how many whole layers the card holds, then add the time each half takes: the bytes on the GPU over its bandwidth, plus the bytes in system RAM over the RAM's. Against our measured sweep this lands within 2.5% on average.
A mixture-of-experts model reads only its active parameters, which is why a 30B model with 3B
active is much faster than a dense 30B. It does not reach the bandwidth a dense model does,
because it gathers small expert matrices from all over memory. We multiply the GPU's bandwidth
by 0.58 for these models, a factor set from published llama.cpp figures and not
from our own runs. We have not measured a MoE model ourselves, and those speeds should be read as good to
about 30%. When a MoE model does not fit, we also consider keeping its experts in system RAM and
everything dense on the GPU, which is what llama.cpp's --cpu-moe does, and use whichever is faster.
Prompt processing is the one compute-bound part, estimated from FP16 tensor throughput at a much lower efficiency factor. It is a rougher number than generation speed, and on Apple chips in particular it is the weakest estimate we publish.
How we rank and recommend
The "best model for your card" lists are not sorted by size. Each pairing gets a score: the model's capability, taken as its dense-equivalent parameter count (for a mixture-of-experts model, between its active and total counts and closer to the total), times a quality factor for the quantization, times a speed factor that stops mattering once the model is faster than you can read, times small adjustments for how recent the model is and how widely it is downloaded. Those last two are blunt, and we keep them mild on purpose. Pairings that only fit at a quantization below the quality floor are marked as compromised and never lead a recommendation. Models under 3B parameters and models built for one job, such as code-only or vision-only ones, are not offered as general picks.
This is a judgment, and we state it as one. Which model is smarter is a question our arithmetic does not answer, and a different weighting would change the order. What it does answer is which pairings are sound, and the table shows its working in every row.
Image, video and audio models
These use a separate engine, because almost nothing above applies to them. A diffusion model has no KV cache, it is several networks and not one, and its speed is set by compute and not by memory bandwidth. Reusing the language-model math would produce figures that look plausible and are wrong.
Components. Each network in a pipeline is measured separately from its safetensors headers or file sizes, and every model page says which applied. Memory. Peak depends on how the components are arranged: all resident, the text encoder on the processor, one component on the GPU at a time, or weights streamed layer by layer. On top of the weights sit the activations, which scale with the number of latent tokens, and the decode pass that turns the latent back into pixels, a real spike and a common cause of an out-of-memory error at the very end of a run.
Time. For a diffusion transformer the linear layers do two operations per parameter per token, and attention adds a term that grows with the square of the sequence. That term is a footnote on one image and dominant on a video with 100 frames; omitting it made our first Wan estimate six times too fast. For a UNet we use a constant calibrated against iteration rates people report. For audio the useful figure is the real-time factor, and the estimate separates models that emit tokens one at a time from models that generate an utterance at once. We have not timed any of these ourselves. Treat the times as good to a factor of two, and the memory as the stronger claim.
What we will not tell you
- Exact tokens per second. Real throughput depends on your runtime, driver, batch size, whether flash attention is on, and what else uses the GPU. Treat our figures as the right order of magnitude and the right ranking.
- How good a model is. We measure whether it fits and how fast it runs. Whether it answers well is a different question.
- Image or audio quality. We say what a model costs and how long it takes, not whether you will like the output.
- LoRAs, ControlNets and upscalers. Each adds memory and time on top of what we report. Treat our figures as the floor for a bare pipeline.
Known limits
- Bandwidth figures are theoretical peaks. The efficiency factors absorb the gap to what a device sustains but do not model it per card.
- Multi-GPU setups are not in the calculator. The multi-GPU guide explains the arithmetic and gives estimates for the common cases.
- Quantization quality scores are a coarse ranking drawn from published perplexity comparisons, not a measurement we performed.
- Dense FP16 tensor throughput with FP16 accumulate, the path llama.cpp and vLLM actually take. Not shader FP16, which is far lower on NVIDIA.
Corrections
If a number here disagrees with what your machine does, we want to know. A systematic gap usually means a term in the model is wrong, and fixing it helps everyone. Tell us what you measured, with the model, the quantization, the runtime and the context length. Changes to the engine are recorded in the update log.