How we calculate

Where the numbers come from

Model architectures are read from each model's own config.json on Hugging Face, and parameter counts from the safetensors index published alongside the weights. Nothing is typed in by hand, which means we cannot get a layer count wrong, and it means the site follows the models rather than lagging behind them. The catalogue currently holds 133 architectures and was last refreshed on 2026-09-01.

Hardware specifications come from vendor datasheets: memory capacity, peak memory bandwidth, and FP16 tensor throughput. 131 profiles are covered. Unified-memory chips are expanded into one profile per memory configuration, because a 36 GB machine and a 128 GB machine with the same chip are not the same machine for this purpose.

Memory

Three terms, added.

Weights. Parameters multiplied by the measured bits per weight of the quantisation, divided by eight. We use measured values, not nominal ones: a Q4_K_M file carries block scales and keeps some tensors at higher precision, so it lands at 4.83 bits per weight rather than 4.

KV cache. Two tensors, times layers, times KV heads, times head dimension, times context length, times the element size. The KV-head count is the term that separates models of otherwise identical size, and we read it per model rather than assuming. Two variations are handled separately: sliding-window attention, where most layers stop accumulating past a fixed distance, and latent cache compression, where keys and values are projected into one small vector per layer.

Overhead. A fixed graph allocation, a compute buffer that scales with context and hidden size, the logits buffer, and the driver's own reservation. Together this is typically 0.6 to 1.5 GB.

From the total we subtract what the operating system has already taken: 0.8 GB on a Windows display adapter, 0.3 GB on Linux with a desktop, 0.1 GB headless. On unified memory we apply the share the OS will actually let the GPU wire down, which on macOS is between 67 and 75 per cent depending on how much is installed.

Speed

Generating one token means reading every active weight, plus the cache, out of memory. That makes single-stream generation bandwidth-bound, not compute-bound, so the estimate is the device's peak bandwidth times an efficiency factor, divided by the bytes read per token. Efficiency is 0.82 on CUDA, 0.78 on Metal, 0.72 on ROCm, 0.65 on Vulkan and 0.62 on CPU.

On a mixture-of-experts model only the active parameters are read, which is why a 30B model with 3B active generates at roughly the speed of a 3B while occupying the memory of a 30B.

When a model does not fit we work out how many whole layers the card holds, then blend the two bandwidths by the time each half takes. Prompt processing is the one compute-bound part, estimated from FP16 tensor throughput at a much lower efficiency factor.

Image, video and audio models

These use a completely separate engine, because almost nothing above applies to them. A diffusion model has no KV cache, it is several networks rather than one, and its speed is set by compute rather than by memory bandwidth. Reusing the language-model maths here would produce figures that look plausible and are wrong.

Components. Each network in a pipeline is measured separately, straight from its safetensors headers, which state every tensor's shape and precision. Where a repository is gated the headers are refused but the file listing is not, so the figures come from total bytes and the published precision. Where the weights ship in a format with no readable header, the figures are stated by us. Every model page says which of the three applied to it.

Memory. Peak depends on how the components are arranged: all resident, the text encoder run on the processor, one component on the GPU at a time, or weights streamed layer by layer. Which arrangement helps most is a property of the model, not a rule. On top of the weights sit the activations, which scale with the number of latent tokens, and the decode pass that turns the latent back into pixels, which is a real spike and a common cause of an out-of-memory error at the very end of a run.

Latent tokens. The resolution divided by the VAE's compression and the patch size, times the compressed frame count. This one number drives both the memory and the time. The VAE compression factors are declared per model rather than read from the configs, because the configs express them too inconsistently to parse safely, and getting one wrong on a video model would be off by a factor of sixteen.

Time. For a diffusion transformer, the linear layers do two operations per parameter per token, and attention adds a term that grows with the square of the sequence. That second term is a footnote on a single image and dominant on a hundred-frame video; omitting it made our first estimate for Wan six times too fast. For a UNet the relationship between parameters and work is architecture-specific, so that path uses a constant calibrated against the iteration rates people report, and is an estimate in a way the transformer path is not.

Calibration points, checked against widely reported figures on an RTX 4090: Stable Diffusion XL at 1024 square in about four seconds, Stable Diffusion 1.5 at 512 in about one, FLUX.1 dev at 28 steps in the twenties, LTX-Video in the twenties, and Wan 2.2 at 720p in minutes rather than seconds. These are in the test suite, so a change that breaks them fails before it ships.

What we will not tell you

• Exact tokens per second. Real throughput depends on your runtime, driver version, batch size, whether flash attention is on, and what else is using the GPU. Treat our figures as the right order of magnitude and the right ranking, not a benchmark.
• How good a model is. We measure whether it fits and how fast it runs. Whether it answers well is a different question and we do not pretend to answer it.
• Non-transformer architectures. State-space and hybrid models do not keep a conventional KV cache, so our cache maths does not describe them. Rather than publish a plausible wrong number we leave them out of the catalogue.
• Image quality. We say what a diffusion model costs and how long it takes, not whether you will like what it produces.
• LoRAs, ControlNets and upscalers. Each adds memory and time on top of what we report. Treat our figures as the floor for a bare pipeline.

Known limits

• Bandwidth figures are theoretical peaks. Sustained bandwidth is typically 75 to 85 per cent of peak, which the efficiency factors absorb but do not model per device.
• Multi-GPU setups are not modelled. Splitting a model across two cards adds interconnect costs we would only be guessing at.
• Quantisation quality percentages are a coarse ranking drawn from published perplexity comparisons, not a measurement we performed.
• Dense FP16 tensor throughput with FP16 accumulate, the path llama.cpp and vLLM actually take. Not shader FP16, which is far lower on NVIDIA.

Corrections

If a number here disagrees with what your machine actually does, we want to know — a systematic gap usually means a term in the model is wrong, and that is worth fixing for everyone. Tell us what you measured, with the model, the quantisation, the runtime and the context length.