By amount of VRAM
What can you run with 8 GB of VRAM? Models, quantizations and limits
8 GB is where most people start, because it is what the mainstream cards ship with. It runs the models most people want to try, and it is also where you first hit the wall. This page sorts out which of your problems are memory and which are something else.
What runs well
An 8 GB card has roughly 7.2 GB usable once the desktop takes its share, and that sets the size of everything. A 7B to 9B model at Q5 or Q6 fits with a window of 8k and a little over, and generates at a pace that keeps up with reading. The pairings below are the strongest the engine finds on an RTX 4060, ranked by quality of result and not by size.
| Pick | Model | Download | Memory | Speed |
|---|---|---|---|---|
| Best all-rounder | Qwen3.5 9B | Q5_K_M | 6.42 GB | 33.3 t/s |
| Best for code | Qwen2.5 Coder 7B Instruct | Q6_K | 6.23 GB | 34.3 t/s |
| Largest that still runs well | Mistral Nemo Instruct 2407 | Q3_K_M | 6.96 GB | 30.7 t/s |
| Model | Parameters | Download | Memory | Speed | Longest context |
|---|---|---|---|---|---|
| Qwen3.5 9B | 9.65B | Q5_K_M | 6.42 GB | 33.3 t/s | 32k |
| Gemma 4 E4B Instruct | 8B | Q6_K | 6.61 GB | 32 t/s | 32k |
| Qwen2.5 VL 7B Instruct | 8.29B | Q6_K | 6.74 GB | 31.5 t/s | 16k |
| Qwen3 8B | 8.19B | Q5_K_M | 6.54 GB | 32.6 t/s | 8k |
| Qwen2.5 7B Instruct | 7.62B | Q6_K | 6.23 GB | 34.3 t/s | 16k |
| Llama 3.1 8B Instruct | 8.03B | Q6_K | 7.14 GB | 29.7 t/s | 8k |
| Gemma 4 12B Instruct | 11.96B | Q3_K_M | 7.16 GB | 29.6 t/s | 8k |
| Granite 4.1 8B | 8.79B | Q4_K_M | 6.60 GB | 32.3 t/s | 8k |
| Qwen2.5 Coder 7B Instruct | 7.62B | Q6_K | 6.23 GB | 34.3 t/s | 16k |
| Internlm3 8B Instruct | 8.8B | Q6_K | 7.10 GB | 29.9 t/s | 8k |
| Mistral 7B Instruct V0.3 | 7.25B | Q6_K | 6.84 GB | 31.1 t/s | 8k |
| Gemma 4 E2B Instruct | 5.12B | Q8_0 | 5.44 GB | 39.1 t/s | 128k |
Qwen3.5 9B takes the top spot for a reason that has nothing to do with its parameter count. Most of its layers use linear attention, which keeps a small fixed state in place of a growing cache, so only one layer in four pays for context. That is why it fits a 32k window beside Q5 weights, where an older 8B design would need several more gigabytes.
| Context | KV cache | Total memory |
|---|---|---|
| 4k | 0.17 GB | 6.30 GB |
| 8k | 0.30 GB | 6.42 GB |
| 16k | 0.55 GB | 6.67 GB |
| 32k | 1.05 GB | 7.17 GB |
| 64k | 2.05 GB | 8.17 GB |
| 128k | 4.05 GB | 10.2 GB |
If your use is long documents or a coding agent that reads a lot of files, the choice of model matters more than the choice of quantization. For short chats, any of the 7B to 9B models in the list will do.
Where 8 GB runs out
Context is the first limit. An 8 GB card has no room to spare, and a classic 8B with grouped-query attention holds 4.00 GB of cache at 32k, which is more than the card has left after the weights. The calculator tells you the longest window for each pairing, and the KV cache guide explains the designs that avoid it.
Model size is the second. Anything above about 10B has to go to 3-bit to fit, and at that point you pay in quality what you saved in memory. We would rather run a 9B at Q6 than a 12B at Q3, and the engine marks the squeezed pairings as compromised so they do not rise to the top of a recommendation list. The case where this changes is a mixture-of-experts model with enough system RAM, which the MoE guide covers.
Image and video generation are the third. SDXL runs on 8 GB, Flux needs a quantized variant, and most video models need much more. See the low-VRAM image guide for what fits.
8 GB is not one thing
A pair of cards sold as 8 GB can differ by more than a factor of two in speed, because generation follows memory bandwidth and bandwidth is the spec a box does not print. The table shows an 8B model on the 8 GB cards people actually own.
| Device | Memory | Bandwidth | Qwen3 8B | Qwen3.5 9B |
|---|---|---|---|---|
| GeForce RTX 4060 | 8 GB | 272 GB/s | 32.6 t/s | 33.3 t/s |
| GeForce RTX 4060 Ti 8GB | 8 GB | 288 GB/s | 34.5 t/s | 35.2 t/s |
| GeForce RTX 5060 | 8 GB | 448 GB/s | 53.7 t/s | 54.8 t/s |
| GeForce RTX 3060 Ti | 8 GB | 448 GB/s | 53.7 t/s | 54.8 t/s |
| GeForce RTX 3070 | 8 GB | 448 GB/s | 53.7 t/s | 54.8 t/s |
| GeForce RTX 3070 Ti | 8 GB | 608 GB/s | 72.9 t/s | 74.4 t/s |
An RTX 4060 has 272 GB/s and an RTX 3070 Ti has 608, and that is why the older, cheaper card answers about 2.2 times as fast. Both load the same models, and one of them will feel noticeably slower to talk to. When you are buying, look at the bandwidth column, and when you are troubleshooting a slow card, check it before blaming your settings.
Which cards have 8 GB
| Device | Memory | Bandwidth | Price |
|---|---|---|---|
| GeForce RTX 3070 Ti Desktop GPU, used | 8 GB | 608 GB/s | $300 |
| Arc A750 Desktop GPU, used | 8 GB | 512 GB/s | $200 |
| GeForce RTX 5060 Ti 8GB Desktop GPU | 8 GB | 448 GB/s | $380 |
| GeForce RTX 5060 Desktop GPU | 8 GB | 448 GB/s | $300 |
| GeForce RTX 3070 Desktop GPU, used | 8 GB | 448 GB/s | $260 |
| GeForce RTX 3060 Ti Desktop GPU, used | 8 GB | 448 GB/s | $230 |
| GeForce RTX 4060 Ti 8GB Desktop GPU | 8 GB | 288 GB/s | $350 |
| GeForce RTX 4060 Desktop GPU | 8 GB | 272 GB/s | $280 |
| GeForce RTX 4070 Laptop Laptop GPU | 8 GB | 256 GB/s | n/a |
| GeForce RTX 4060 Laptop Laptop GPU | 8 GB | 256 GB/s | n/a |
| GeForce RTX 3050 8GB Desktop GPU, used | 8 GB | 224 GB/s | $170 |
| Jetson Orin Nano Super 8GB Unified memory | 8 GB | 102 GB/s | $249 |
New 8 GB cards from the last two generations are a poor buy for this purpose, since the narrow bus keeps their speed low and the memory is the thing you will want more of. The older, wider cards on the used market are the better value, and above them the jump to 12 GB is the upgrade we recommend, since it moves you into 14B models and a real context window. Our 12 GB guide shows what that unlocks.
Our advice at this tier
Pick a hybrid-attention 9B model if you care about context and a good 8B if you do not, run at Q5 or Q6, and keep the window at what you actually use. Make sure your runtime puts every layer on the card, because a split model on a narrow-bus card is the slowest combination there is. And if you have not bought the card yet, buy a wider one, or spend a little more on 12 GB.
Questions people ask
What is the best local LLM for 8 GB of VRAM?
A recent 8B to 9B model at Q5 or Q6, such as Qwen3.5 9B, Qwen3 8B or Llama 3.1 8B, is the best fit. It leaves a gigabyte or two for a context window of 8k or more, and answers at a comfortable speed.
Can 8 GB run a 13B or 14B model?
Only at 3-bit precision, which costs quality, or with part of the model in system RAM, which costs speed. Most people are better served by a 9B model at a higher quantization than a 14B squeezed in.
Why is my 8 GB card slower than a friend's 8 GB card?
Memory bandwidth. Cards with the same VRAM range from 272 GB/s to over 600 GB/s, and generation speed follows bandwidth, so a wide-bus card answers more than twice as fast as a narrow-bus one.
Can I run a 32k context window in 8 GB?
With the right model, yes. A hybrid-attention model such as Qwen3.5 9B keeps its cache small enough to fit 32k alongside Q5 weights, while a classic 8B model needs another 4 GB or so at that length.