What can you run with 48 GB of VRAM? 70B models, one card or two

This is the tier people save for, because it is where a 70B model runs entirely on a graphics card. At 4 bits a 70B needs about 42.1 GB, which does not fit 24 or 32 GB and fits 48 with room for a modest window. Everything about this tier is a question of how you get there and what that room is worth.

What fits

Usable memory on one 48 GB card is about 47.2 GB. The 70B and 72B dense models fit at Q4_K_M. The 27B to 32B models fit at Q8 with plenty of window to spare, and large mixture-of-experts models such as Qwen3 Next 80B A3B fit at Q4. These are the strongest pairings on an RTX A6000.

Pick	Model	Download	Memory	Speed
Best all-rounder	Qwen3.5 27B	Q8_0	27.3 GB	21 t/s
Best for code	Qwen3 32B	Q8_0	34.1 GB	16.8 t/s
Best for reasoning	Qwen3.5 27B	Q8_0	27.3 GB	21 t/s
Largest that still runs well	Qwen3 Next 80B A3B Instruct	Q4_K_M	46.1 GB	160.2 t/s

Model	Parameters	Download	Memory	Speed	Longest context
Qwen3.5 27B	27.78B	Q8_0	27.3 GB	21 t/s	256k
Gemma 4 31B Instruct	31.27B	Q8_0	34.7 GB	16.5 t/s	32k
Qwen3 32B	32.76B	Q8_0	34.1 GB	16.8 t/s	32k
Granite 4.1 30B	28.87B	Q8_0	31.0 GB	18.5 t/s	64k
Qwen2.5 32B Instruct	32.76B	Q8_0	34.1 GB	16.8 t/s	32k
Gemma 3 27B Instruct	27.43B	Q8_0	28.7 GB	20 t/s	128k
Qwen2.5 Coder 32B Instruct	32.76B	Q8_0	34.1 GB	16.8 t/s	32k
Qwen3.5 35B A3B	35.95B (2.9B active)	Q8_0	35.6 GB	107.6 t/s	256k
Gemma 4 26B A4B Instruct	25.81B (3.74B active)	Q8_0	26.7 GB	72.5 t/s	256k
Qwen3 Coder Next	79.67B (3.19B active)	Q4_K_M	45.2 GB	160.2 t/s	64k
GLM 4.7 Flash	31.22B (3.66B active)	Q8_0	31.3 GB	82 t/s	128k
Llama 3.3 70B Instruct	70.55B	Q4_K_M	42.1 GB	13.6 t/s	16k

A 70B model is a different experience from a 30B. It knows more, it follows long instructions more reliably, and it holds a conversation together better over many turns. The cost is speed: on an RTX A6000 with 768 GB/s our engine estimates about 13.6 tokens per second, which is slower than comfortable reading and fine for careful work. A pair of used 3090s does better on paper, with about 16.6 tokens per second on the same model, because each card reads its own half at 936 GB/s.

The window is the limit

At this size the cache is what runs out first. A 70B at Q4_K_M leaves a few gigabytes, and this is how they are spent:

Context	KV cache	Total memory
4k	1.25 GB	40.9 GB
8k	2.50 GB	42.1 GB
16k	5.00 GB	44.6 GB
32k	10.0 GB	49.6 GB

On 48 GB you can run the 70B with a window up to about 16k and then it spills. If you want 32k or more, you have three choices: store the cache at 8 bits to halve it, drop to a smaller 4-bit variant such as IQ4_XS, or use a model with a lighter attention design. The KV cache guide explains the options.

One card or two

The usual routes to 48 GB are a workstation card, such as the RTX A6000 or the newer RTX 6000 Ada, and two used 24 GB cards. They solve the same problem differently.

The single card is simple. It is one slot and about 300 W, there is nothing to configure, and it works in any runtime. It is also several times the price of the pair. The pair is cheaper, and each card reads memory faster than the workstation card does, but it brings the costs we described in the multi-GPU guide: 700 W of graphics power, two thick cards with room between them, and a layer split that gives you the memory and none of the speedup. For a hobbyist the pair wins on price. For a workstation that runs all day, the single card wins on everything else.

A third route is a Mac with 64 GB or more of unified memory, which holds the same models at lower speed and a small fraction of the power draw. See the Mac and PC comparison.

What about the biggest MoE models

The mixture-of-experts models above 100B do not fit whole, and the expert split makes them usable. gpt-oss-120b ships in a native 59 GiB file, so roughly a fifth of its experts live in system RAM, and with 64 GB of RAM our engine estimates about 49.1 tokens per second on one 48 GB card. That is faster than a dense 70B on the same card, and it is a good reason to have the RAM. Models at 200B and above are beyond 48 GB unless you have a lot of fast system memory.

Which devices have 48 GB

A few workstation cards have 48 GB on a single board, and several Apple and AMD machines reach it through shared memory. For the shared-memory ones, remember that the GPU can use only part of the total, so a 48 GB Mac behaves like a card with about 36 GB.

Device	Memory	Bandwidth	Price
RTX 6000 Ada Generation Workstation GPU	48 GB	960 GB/s	$6,800
Radeon PRO W7900 Workstation GPU	48 GB	864 GB/s	$3,500
RTX A6000 Workstation GPU, used	48 GB	768 GB/s	$3,500
Apple M4 Max 48GB Unified memory	48 GB	546 GB/s	$3,699
Apple M3 Max 48GB Unified memory	48 GB	400 GB/s	$3,299
Apple M4 Pro 48GB Unified memory	48 GB	273 GB/s	$1,799

The workstation cards in this list cost several times what a pair of used 24 GB cards does, and in return they are one board, one slot and about 300 W. Whether that is worth it depends on how many hours a day the machine runs and whether you would rather pay once or tinker.

Our advice at this tier

Decide whether you want a 70B because you need it or because it is the biggest thing available. If you have a task that a 32B cannot do, the 70B class is worth the hardware. If you do not, a 24 GB card with a good 32B and a fast MoE model serves most people better, at a fraction of the cost. When you do go to 48 GB, keep the window near 16k, store the cache at 8 bits if you need more, and let the quantization stay at Q4_K_M.