Best GPU for local LLMs by budget: what to buy at each price

Gaming reviews rank graphics cards by frame rates, and almost none of that carries over. For local models only two numbers matter: how much memory the card has, which decides what loads at all, and how fast that memory can be read, which decides how quickly the model answers. Everything below follows from those two, and the prices are indicative ones that we review by hand.

What to look at, in order

Start with memory. A model that does not fit in VRAM either does not run or runs at a fraction of the speed, as the offload sweep we measured shows, so capacity sets the ceiling on what you can do. 12 GB is the practical floor for a card you intend to keep. It runs 8B to 14B models at good quality with room for a real context window, and below that you spend your time shrinking things.

Then bandwidth. Cards with the same memory can generate at very different speeds, because decoding is limited by how fast the weights are read, not by the compute rating that gaming charts feature. A 12 GB RTX 3060 has 360 GB/s and a 12 GB RTX 4070 has 504, and on an 8B model that is about 30.8 against 43.1 tokens per second. The bandwidth guide explains why and how to estimate it from a spec sheet.

After those two come software support and practical limits: the card has to physically fit your case, your power supply has to feed it, and your runtime has to drive it. NVIDIA works everywhere. AMD and Intel work in llama.cpp and Ollama, with more variation between card generations and operating systems.

Picks by budget

This table is built from the same engine as the calculator. For each device it picks the strongest general chat model that fits in memory at a sound quantization, and shows the speed we expect. Cards are listed within each bracket, best first.

Under $250

Device	Memory	Best all-round model	Speed
GeForce RTX 2060 12GB $160 used, 336 GB/s	12 GB	Gemma 4 12B Instruct at Q6_K	23.7 t/s
GeForce RTX 3060 12GB $200 used, 360 GB/s	12 GB	Gemma 4 12B Instruct at Q6_K	25.4 t/s
Radeon RX 6700 XT $250 used, 384 GB/s	12 GB	Gemma 4 12B Instruct at Q6_K	22.9 t/s
GeForce GTX 1080 Ti $150 used, 484 GB/s	11 GB	Qwen3.5 9B at Q8_0	40.3 t/s
GeForce RTX 2080 Ti $250 used, 616 GB/s	11 GB	Qwen3.5 9B at Q8_0	51.3 t/s

$250 to $500

Device	Memory	Best all-round model	Speed
Arc A770 16GB $280 used, 560 GB/s	16 GB	Qwen3.5 27B at IQ4_XS	23.1 t/s
Radeon RX 6800 $330 used, 512 GB/s	16 GB	Qwen3.5 27B at IQ4_XS	23.2 t/s
Radeon RX 6900 XT $400 used, 512 GB/s	16 GB	Qwen3.5 27B at IQ4_XS	23.2 t/s
GeForce RTX 5060 Ti 16GB $430, 448 GB/s	16 GB	Qwen3.5 27B at IQ4_XS	24 t/s
Radeon RX 7800 XT $480, 624 GB/s	16 GB	Qwen3.5 27B at IQ4_XS	28.3 t/s

$500 to $900

Device	Memory	Best all-round model	Speed
GeForce RTX 3090 $700 used, 936 GB/s	24 GB	Qwen3.5 27B at Q6_K	33 t/s
GeForce RTX 3090 Ti $800 used, 1008 GB/s	24 GB	Qwen3.5 27B at Q6_K	35.5 t/s
Radeon RX 7900 XTX $850, 960 GB/s	24 GB	Qwen3.5 27B at Q6_K	28.7 t/s
Radeon RX 7900 XT $650, 800 GB/s	20 GB	Qwen3.5 27B at Q5_K_M	27.4 t/s
Radeon RX 7900 GRE $520 used, 576 GB/s	16 GB	Qwen3.5 27B at IQ4_XS	26.1 t/s

$900 to $1,600

Device	Memory	Best all-round model	Speed
RTX A5000 $1,200 used, 768 GB/s	24 GB	Qwen3.5 27B at Q6_K	27.1 t/s
GeForce RTX 4080 SUPER $950, 736 GB/s	16 GB	Qwen3.5 27B at IQ4_XS	39.4 t/s
GeForce RTX 5080 $1,050, 960 GB/s	16 GB	Qwen3.5 27B at IQ4_XS	51.3 t/s

$1,600 to $3,000

Device	Memory	Best all-round model	Speed
GeForce RTX 5090 $2,200, 1792 GB/s	32 GB	Gemma 4 31B Instruct at Q6_K	48.5 t/s
GeForce RTX 4090 $1,900, 1008 GB/s	24 GB	Qwen3.5 27B at Q6_K	35.5 t/s

Two readings of it are worth spelling out. The used 24 GB cards are the surprise: an RTX 3090 costs less than a new 16 GB card and holds a 27B model at 6 bits with room left for context. And in the lowest brackets the choice is really between memory and age. We would take a 12 GB RTX 3060 over any 8 GB card, new or used, because it lets the next generation of models load.

Cards we would think twice about

The 16 GB RTX 4060 Ti is the awkward one. It has the memory to load 27B models, but a 128-bit bus gives it only 288 GB/s, so it generates them at roughly half the speed of cards in its price range with wider memory. It is a fine card for a model that must fit and a poor one if you expect speed.

8 GB cards of any age are a trap for anyone who wants to grow. They run 7B and 8B models well and then run out, and the models that matter next year are larger. The 8 GB tier page shows what actually works on one. Datacenter cards sold cheaply on the used market, such as the Tesla P40, have a lot of memory and no video output or cooling, and need work to make them behave in a desktop. They suit a patient tinkerer more than a first build.

If you are weighing a second-hand card, our used GPU guide covers what to check. If you are weighing a whole Mac against a PC with a card, the comparison goes through the trade-offs and shows where unified memory wins.

How we would decide

Pick the largest model you want to run daily, read the memory the calculator says it needs at your window size, and buy the cheapest card that clears it with a gigabyte to spare. Then check that its bandwidth gives a speed you will enjoy. That order matters: memory is a yes or no, speed is a matter of degree. The buying tool does exactly this against the price table above.

Prices move, and used markets move faster than anything. Treat every figure here as a place to start a search, and if a card is within 15% of a bracket line, look at the next one up.