What can you run with 6 GB of VRAM? Tested on an RTX 3050

6 GB is the smallest amount of VRAM that still gets you somewhere useful, and it is where we do our own testing, on an RTX 3050 6 GB. That makes this the one tier page where almost every claim has been run on the real card, so we can say where the margins are rather than guess.

What fits

The rule at 6 GB is simple: a model has to be 4 bits and 9 billion parameters or smaller, with a window of 8k or less. Qwen2.5 7B at Q4_K_M took 4.42 GB of the card in our run, with every layer on the GPU, and left 0.63 GB free. That is the whole story of this tier: it fits, and you can see the wall.

The table below is what the engine recommends for a card like ours. It ranks pairings by how good the result is and not by size, and it skips the quantizations that damage a model.

Pick	Model	Download	Memory	Speed
Best all-rounder	Qwen3.5 9B	IQ4_XS	4.98 GB	27.1 t/s
Best for code	Qwen2.5 Coder 7B Instruct	Q4_K_M	4.81 GB	28 t/s
Largest that still runs well	Qwen3.5 9B	IQ4_XS	4.98 GB	27.1 t/s

Model	Parameters	Download	Memory	Speed	Longest context
Qwen3.5 9B	9.65B	IQ4_XS	4.98 GB	27.1 t/s	8k
Gemma 4 E4B Instruct	8B	Q4_K_M	5.00 GB	26.6 t/s	16k
Qwen2.5 VL 7B Instruct	8.29B	Q4_K_M	5.18 GB	25.8 t/s	8k
Qwen2.5 7B Instruct	7.62B	Q4_K_M	4.81 GB	28 t/s	8k
Gemma 4 E2B Instruct	5.12B	Q6_K	4.29 GB	31.2 t/s	128k
Qwen3.5 4B	4.66B	Q6_K	4.21 GB	32 t/s	32k
Llama 3.1 8B Instruct	8.03B	IQ4_XS	5.12 GB	26.2 t/s	8k
Qwen2.5 Coder 7B Instruct	7.62B	Q4_K_M	4.81 GB	28 t/s	8k
Qwen3 8B	8.19B	Q3_K_M	4.98 GB	27 t/s	8k
Internlm3 8B Instruct	8.8B	IQ4_XS	4.87 GB	27.7 t/s	8k
Mistral 7B Instruct V0.3	7.25B	IQ4_XS	4.93 GB	27.3 t/s	8k
Qwen3 4B	4.02B	Q6_K	4.55 GB	29.5 t/s	8k

Two entries deserve a note. Qwen3.5 9B only fits at IQ4_XS, which is the smallest 4-bit format that keeps its quality, and it leaves almost nothing for the cache. And the small Gemma 4 E2B is the one model here that can use a long window, because its attention keeps the cache small. If you plan to feed a model long documents, a small model with a large window beats a 9B that has to stay under 8k.

The two choices that matter

Quantization is the first. Going from Q4_K_M to IQ4_XS saves about half a gigabyte on a 9B model, and that is the difference between loading and not loading. Dropping to 3 bits saves more and costs quality you will notice, so we would take a smaller model at 4 bits before a larger one at 3. This is how Qwen3.5 9B looks across the ladder at an 8k window:

Quantization	Weights	KV cache	Total	Quality
Q8_0	8.54 GB	0.30 GB	9.25 GB	99%
Q6_K	6.59 GB	0.30 GB	7.30 GB	98%
Q5_K_M	5.72 GB	0.30 GB	6.42 GB	96%
Q4_K_M	4.85 GB	0.30 GB	5.56 GB	93%
IQ4_XS	4.27 GB	0.30 GB	4.98 GB	91%
Q3_K_M	3.93 GB	0.30 GB	4.63 GB	86%
IQ3_XXS	3.08 GB	0.30 GB	3.78 GB	79%

Context is the second. At 6 GB the cache competes with the weights for the same space, so a longer window directly shrinks what can load. And this is where runtimes bite: on our card Ollama kept 16% of the model in system RAM at a 4k window, though it fit entirely, and generation was about a third slower than it needed to be. The Ollama guide shows the setting that fixes it. If you run llama.cpp directly, -ngl all and a short -c do the same job.

What does not fit

A 12B to 14B model at 4 bits needs about 9 GB, so a third of it lands in system RAM. Our offload sweep shows what that does: with 12 of 28 layers on the card a 7B ran at 7.2 tokens per second, against 29.5 with all of them there. The speed loss is out of proportion to the share you move, because every token waits for the slow part. Treat anything that needs offload as a batch tool, not a chat partner.

The large mixture-of-experts models are the one case where offload behaves, since only a few experts are read per token. Even so, a 30B-class MoE on 6 GB needs 32 GB or more of fast system RAM to be pleasant, and anything larger needs 64 GB. The MoE guide explains where that line sits.

Which cards have 6 GB

Only a few cards sit at exactly 6 GB, and they differ more than the number suggests. The list below is sorted by memory bandwidth, so the fastest generators come first.

Device	Memory	Bandwidth	Price
GeForce GTX 1660 SUPER Desktop GPU, used	6 GB	336 GB/s	$110
GeForce RTX 3050 6GB Desktop GPU	6 GB	168 GB/s	$150

The cheap ones are the older GTX 1660 Super and the RTX 3050 6 GB. The GTX 1660 Super has twice the bandwidth of the 3050, so it generates faster on the same models, though it is an older architecture with less software attention. Anything with 8 GB is a clear step up, and the 8 GB guide shows how much.

Our advice at this tier

Treat 6 GB as a 7B machine and enjoy it as one. Pick a recent 7B to 9B model at Q4_K_M or IQ4_XS, keep the window near 4k to 8k, make sure the runtime puts every layer on the card, and stop there. If you find yourself fighting the memory every session, the upgrade is a used 12 GB card, which opens up 14B models, and the GPU buying guide covers which.