What we tested

A calculator is easy to build and easy to get wrong, and the usual way to find out is to run the thing it describes. So we did. The test machine is a desktop with an NVIDIA GeForce RTX 3050 (6 GB), an Intel Xeon CPU E3-1245 v5 @ 3.50GHz and 32 GB of DDR4-2133 in dual channel, running Windows with the desktop active, which is how most people really use a card. The runtime is Ollama 0.35.0 and the model is Qwen2.5 7B Instruct at Q4_K_M. Each point is the median of 3 warm runs of 96 generated tokens at temperature 0, after one discarded run to load the model.

Two sweeps came out of it. In the first we let Ollama place the model by itself at five context sizes. In the second we forced the number of layers on the GPU from none to all, at a fixed 4k window, which is the cleanest way to see what offload really costs.

Memory: measured against estimated

The estimate is the whole footprint, in system RAM and VRAM together, since that is what Ollama reports. The cache is the part that moves with the window, so this is also a test of the cache formula.

Context	Measured	Estimated	Error	On the GPU
2,048	4.66 GiB	4.78 GiB	+2.6%	84%
4,096	4.77 GiB	4.89 GiB	+2.5%	84%
8,192	5.01 GiB	5.11 GiB	+2.0%	79%
16,384	5.41 GiB	5.55 GiB	+2.5%	73%
32,768	6.35 GiB	6.42 GiB	+1.1%	61%

The worst case is 2.6% and the average 2.1%, and every estimate is a little above the measurement, which is the safe side to be on. The last column is a finding of its own. Ollama never put the whole model on the card in this sweep, even at the smallest window where everything would have fit, and the share it kept there fell as the window grew. Our Ollama guide shows what forcing the placement does.

Speed: measured against estimated

The estimate is the formula the calculator uses for a partly offloaded model: the bytes read per token, split between the GPU and system RAM in proportion to the layers on each, and each half divided by its own bandwidth and efficiency. Here it is against the forced sweep.

Layers on the GPU	Measured	Estimated	Error
0 of 29	4.8 t/s	4.8 t/s	+0.1%
6 of 29	5.4 t/s	5.8 t/s	+7.7%
12 of 29	7.2 t/s	7.3 t/s	+0.9%
17 of 29	9.6 t/s	9.4 t/s	-2.6%
23 of 29	14.7 t/s	14.2 t/s	-3.4%
29 of 29	29.5 t/s	29.5 t/s	-0.3%

The worst case is 7.7% and the average 2.5%. The curve it traces is the useful part. Moving the first dozen layers to the card bought almost nothing, and moving the last few bought most of the speed, because every token waits for whatever is slowest. A model that is a little too big for the card is the worst place to be, and the calculator says so.

What these tests do not show

One machine and one model is a narrow base, and we would rather say so than let a good-looking table imply more. The numbers support the CUDA and system-RAM constants we use, and the memory arithmetic for a standard grouped-query transformer. They say nothing yet about AMD, Intel or Apple hardware, whose efficiency figures are still estimates. They do not cover mixture-of-experts models, whose speed we derate by a factor taken from published llama.cpp results and not from our own runs. And they do not cover image, video or audio models, whose times are estimates good to about a factor of two. Each of those pages says so where the figure appears.

What changed because of the measurements

The first version of the engine was wrong in three ways that a spreadsheet would never have shown us. It overestimated memory by 8% to 18%, because it added overhead that grows with the window when flash attention keeps it flat. It claimed that a 7B model did not fit a 6 GB card when it fit with more than half a gigabyte to spare, because it counted the embedding table as resident when it stays in system RAM. And it mixed binary and decimal gigabytes in the speed formula. All three are fixed, and the efficiency constants are now the values we measured, 0.79 on CUDA and 0.63 on the CPU. A test in our suite fails if a later change pushes the engine away from these measurements.

Check it yourself

The raw measurements are in a JSON file, with every run and every reading. The script that produced them, bench-ollama.mjs, takes a model name and writes the same file, so anyone with Node, Ollama and a GPU can repeat it. If your numbers disagree with ours by more than the errors above, that is worth knowing: send us the model, the quantization, the runtime and the result, and we will look at it.

Measured on 2026-10-05. Further machines will be added here as we run them, and the update log records every change to the engine.