Buying a used GPU for local AI: what to check before you pay

For running models at home, the used market has the best deals in computing and the worst failure stories. A used 24 GB card costs less than a new 16 GB one, and the same sellers will happily sell you a card with a dying fan or a memory fault. This guide covers which used cards are worth the trouble, which ones are traps, and how to check one before the return window shuts.

The cards worth buying

The RTX 3090 is the one. It has 24 GB of memory and 936 GB/s of bandwidth, which means it loads a 27B model at 6 bits with room for context and generates it at about 33.0 tokens per second. No new NVIDIA card under about $1,500 offers both numbers, and the one AMD card that does, the RX 7900 XTX, comes with the software caveats we cover in the buying guide. The 3090 Ti is faster by a few percent for a higher price, and we would not pay it.

At 12 GB, the RTX 3060 is the budget answer, and it runs 8B to 14B models comfortably. A 3080 12 GB or 3080 Ti have far more bandwidth for a little more money and the same capacity. The RTX 2080 Ti with 11 GB has strong bandwidth, and it is an older architecture that is slowly losing software attention, so we would pick a 3060 12 GB over it for longevity.

Workstation cards deserve a mention. An RTX A5000 has 24 GB and a modest power draw, and the used price is higher than a 3090 for lower bandwidth, so it only makes sense in a case or a power budget where a 3090 will not fit. Cards sold as "workstation" often have blower coolers and a single power connector, which suits a cramped build.

The traps

Tesla P40 cards come up in every cheap-VRAM thread. They have 24 GB for little money, and they also have 346 GB/s, no video output, passive cooling that needs a fan you add yourself, and an architecture that has lost most of its software support. Generation speed on a 70B-class model is a fraction of what a 3090 gives. They suit a patient tinkerer with a spare server, and not a first build.

Be careful with anything older than Turing. Pascal cards such as the GTX 1080 Ti still run models in llama.cpp, but NVIDIA has been moving them out of current driver and CUDA releases, so check the support status of the exact generation before you buy. And skip any 8 GB card at a discount unless it is for something else, because the tier of models you can run on 8 GB is the one most likely to get cramped soon, as the 8 GB guide shows.

Finally, be wary of listings that are too good. A 3090 at half the market price is a scam or a card with a known fault, and the money you save is not worth the exposure.

How to test one

Buy from a marketplace with buyer protection and a return window, and plan the whole test to fit inside it. Then work through it in order:

1. Look at the card for physical damage, bent fins, missing screws and discolored connectors, and spin each fan by hand.
2. Install it and note the idle temperature and fan noise. A used card that is loud at idle usually needs new fans.
3. Run a sustained GPU load for at least 30 minutes with a monitoring tool open and watch the core and the memory temperature. On a 3090 the memory junction runs hottest, and a sustained reading above about 100 degrees Celsius means the thermal pads need replacing, which is a repair many owners do themselves.
4. Run a memory test, such as the one in OCCT, and look for any errors.
5. Load the biggest model the card should hold, set a long context, and generate for a few minutes. If it crashes, artifacts or throttles hard, send it back.

If the card passes, think about repasting it and replacing the pads on the memory side. It is cheap and it often drops the memory temperature by 10 degrees or more.

Weighing it against new

Used saves money and carries risk. A new card gives you a warranty and a known state, and a used one with a return window and a proper test gives you much more memory for the same price. If a failed card would ruin your month, buy new. If you can absorb a bad pick and send it back, the 3090 is hard to beat. Compare the options for the model you care about in the buying tool, and read the GPU buying guide for how used cards compare with new ones at each price.

One last practical point. A card that draws 350 W needs a power supply that can feed it and a case that can cool it, and the cost of that shows up in your electricity bill too, which we cover in the running cost guide.