Qwen · 32B to 80B

Qwen2.5 32B Instruct

32.76 billion parameters across 64 layers. At four-bit precision the weights alone come to 18.42 GB, before any conversation is loaded. It uses grouped-query attention with 8 KV heads for 40 query heads, which cuts the cache to 20% of what full attention would need.

General chat

Parameters32.76B

Layers64

KV heads8 / 40

Native context32k

Vocabulary152k

About Qwen2.5 32B Instruct

The 32B of the Qwen2.5 generation, from September 2024. A strong general model that was the default recommendation for 24 GB cards before Qwen3.

Qwen2.5 32B Instruct is a dense model with 32.76B parameters, built from 64 layers with a hidden size of 5120. Its repository was published on 2024-09-17. The license listed on Hugging Face is apache-2.0. It uses grouped-query attention with 8 KV heads for 40 query heads, which cuts the cache to 20% of what full attention would need.

Memory needed, by quantization

At an 8k context window. Quality is our rough ranking of how much the compression costs you: anything at or above Q5 is hard to tell apart from the original in normal use.

Quantization	File size	KV cache	Total VRAM	Quality	What it costs you
Q8_0	32.42 GB	2 GB	34.09 GB	99%	Lossless in practice. Use it when the memory is there.
Q6_K	25.02 GB	2 GB	26.86 GB	98%	Very close to Q8 for two thirds of the size.
Q5_K_M	21.7 GB	2 GB	23.62 GB	96%	The quality-first choice when Q6 will not fit.
Q4_K_M	18.42 GB	2 GB	20.42 GB	93%	The default. Best size-to-quality ratio for local use.
IQ4_XS	16.21 GB	2 GB	18.26 GB	91%	Importance-matrix 4-bit. Q4_K_S quality, smaller file.
Q3_K_M	14.91 GB	2 GB	17 GB	86%	Degradation starts to show. A way to fit one size up.
IQ3_XXS	11.67 GB	2 GB	13.83 GB	79%	Aggressive. Only worth it on very large models.

What a longer conversation costs

Same model at Q4_K_M, only the context window changes. The cache grows in a straight line with every token in the window.

Context	KV cache	Total VRAM	Fits in 8 GB	Fits in 12 GB	Fits in 24 GB
4k	1 GB	19.42 GB	no	no	yes
8k	2 GB	20.42 GB	no	no	yes
16k	4 GB	22.42 GB	no	no	yes
32k	8 GB	26.42 GB	no	no	no

The "fits" columns allow for the roughly 0.8 GB Windows keeps for the desktop.

Which hardware runs Qwen2.5 32B Instruct

37 of 118 consumer devices run it at a quantization worth using, at an 8k context window. Another 32 can load it only by compressing the weights far enough to damage the model, marked with a warning below.

Device	Memory	Verdict	Quantization	Used	Speed
RTX PRO 6000 Blackwell NVIDIA	96 GB	Runs great	Q8_0	34.09 GB	39.2 t/s
RTX 6000 Ada Generation NVIDIA	48 GB	Runs well	Q8_0	34.09 GB	21 t/s
GeForce RTX 5090 NVIDIA	32 GB	Runs great	Q6_K	26.86 GB	49.9 t/s
RTX A6000 NVIDIA	48 GB	Runs well	Q8_0	34.09 GB	16.8 t/s
Apple M3 Ultra 96GB Apple	96 GB	Runs well	Q8_0	34.83 GB	16.5 t/s
Apple M3 Ultra 256GB Apple	256 GB	Runs well	Q8_0	34.83 GB	16.5 t/s
Apple M3 Ultra 512GB Apple	512 GB	Runs well	Q8_0	34.83 GB	16.5 t/s
Apple M1 Ultra 64GB Apple	64 GB	Runs well	Q8_0	34.83 GB	16.2 t/s
Apple M1 Ultra 128GB Apple	128 GB	Runs well	Q8_0	34.83 GB	16.2 t/s
Apple M2 Ultra 64GB Apple	64 GB	Runs well	Q8_0	34.83 GB	16.2 t/s
Apple M2 Ultra 128GB Apple	128 GB	Runs well	Q8_0	34.83 GB	16.2 t/s
Apple M2 Ultra 192GB Apple	192 GB	Runs well	Q8_0	34.83 GB	16.2 t/s
Radeon PRO W7900 AMD	48 GB	Runs well	Q8_0	34.09 GB	16 t/s
GeForce RTX 4090 NVIDIA	24 GB	Runs great	Q4_K_M	20.42 GB	37.1 t/s
GeForce RTX 3090 Ti NVIDIA	24 GB	Runs great	Q4_K_M	20.42 GB	37.1 t/s
GeForce RTX 3090 NVIDIA	24 GB	Runs great	Q4_K_M	20.42 GB	34.5 t/s
GeForce RTX 5090 Laptop NVIDIA	24 GB	Runs great	Q4_K_M	20.42 GB	33 t/s
Radeon RX 7900 XTX AMD	24 GB	Runs great	Q4_K_M	20.42 GB	30 t/s
RTX A5000 NVIDIA	24 GB	Runs great	Q4_K_M	20.42 GB	28.3 t/s
Radeon RX 7900 XT AMD	20 GB	Just fits	IQ4_XS	18.26 GB	28 t/s
Apple M4 Max 48GB Apple	48 GB	Runs well	Q8_0	34.83 GB	11 t/s
Apple M4 Max 64GB Apple	64 GB	Runs well	Q8_0	34.83 GB	11 t/s
Apple M4 Max 128GB Apple	128 GB	Runs well	Q8_0	34.83 GB	11 t/s
Apple M1 Max 64GB Apple	64 GB	Runs well	Q6_K	27.43 GB	10.3 t/s
Apple M2 Max 64GB Apple	64 GB	Runs well	Q6_K	27.43 GB	10.3 t/s

Show 35 more devices

Apple M2 Max 96GB Apple	96 GB	Runs well	Q6_K	27.43 GB	10.3 t/s
Apple M3 Max 48GB Apple	48 GB	Runs well	Q6_K	27.43 GB	10.3 t/s
Apple M3 Max 64GB Apple	64 GB	Runs well	Q6_K	27.43 GB	10.3 t/s
Apple M3 Max 96GB Apple	96 GB	Runs well	Q6_K	27.43 GB	10.3 t/s
Apple M3 Max 128GB Apple	128 GB	Runs well	Q6_K	27.43 GB	10.3 t/s
NVIDIA DGX Spark 128GB NVIDIA	128 GB	Runs well	Q4_K_M	20.86 GB	10.1 t/s
Apple M4 Pro 48GB Apple	48 GB	Runs well	IQ4_XS	18.62 GB	10.4 t/s
Apple M4 Pro 64GB Apple	64 GB	Runs well	IQ4_XS	18.62 GB	10.4 t/s
GeForce RTX 5080 NVIDIA	16 GB	Runs great	IQ3_XXS !	13.83 GB	52.7 t/s
GeForce RTX 5070 Ti NVIDIA	16 GB	Runs great	IQ3_XXS !	13.83 GB	49.2 t/s
GeForce RTX 5080 Laptop NVIDIA	16 GB	Runs great	IQ3_XXS !	13.83 GB	42.2 t/s
GeForce RTX 4080 SUPER NVIDIA	16 GB	Runs great	IQ3_XXS !	13.83 GB	40.4 t/s
GeForce RTX 4080 NVIDIA	16 GB	Runs great	IQ3_XXS !	13.83 GB	39.4 t/s
GeForce RTX 4070 Ti SUPER NVIDIA	16 GB	Runs great	IQ3_XXS !	13.83 GB	36.9 t/s
GeForce RTX 4090 Laptop NVIDIA	16 GB	Runs great	IQ3_XXS !	13.83 GB	31.6 t/s
Radeon RX 9070 XT AMD	16 GB	Runs great	IQ3_XXS !	13.83 GB	30.1 t/s
Radeon RX 9070 AMD	16 GB	Runs great	IQ3_XXS !	13.83 GB	30.1 t/s
Radeon RX 7800 XT AMD	16 GB	Runs great	IQ3_XXS !	13.83 GB	29.1 t/s
Radeon RX 7900 GRE AMD	16 GB	Runs great	IQ3_XXS !	13.83 GB	26.8 t/s
GeForce RTX 5060 Ti 16GB NVIDIA	16 GB	Runs well	IQ3_XXS !	13.83 GB	24.6 t/s
RTX A4000 NVIDIA	16 GB	Runs well	IQ3_XXS !	13.83 GB	24.6 t/s
Radeon RX 6900 XT AMD	16 GB	Runs well	IQ3_XXS !	13.83 GB	23.9 t/s
Radeon RX 6800 AMD	16 GB	Runs well	IQ3_XXS !	13.83 GB	23.9 t/s
Arc A770 16GB Intel	16 GB	Runs well	IQ3_XXS !	13.83 GB	23.8 t/s
GeForce RTX 4060 Ti 16GB NVIDIA	16 GB	Runs well	IQ3_XXS !	13.83 GB	15.8 t/s
Jetson AGX Orin 64GB NVIDIA	64 GB	Fits, but slow	Q8_0	34.86 GB	4.5 t/s
Radeon RX 7600 XT AMD	16 GB	Runs well	IQ3_XXS !	13.83 GB	13.4 t/s
Ryzen AI Max+ 395 64GB AMD	64 GB	Fits, but slow	Q8_0	34.86 GB	4.3 t/s
Ryzen AI Max+ 395 96GB AMD	96 GB	Fits, but slow	Q8_0	34.86 GB	4.3 t/s
Ryzen AI Max+ 395 128GB AMD	128 GB	Fits, but slow	Q8_0	34.86 GB	4.3 t/s
GeForce RTX 3080 Ti NVIDIA	12 GB	Runs great	IQ2_XXS !	10.11 GB	69.4 t/s
GeForce RTX 3080 12GB NVIDIA	12 GB	Runs great	IQ2_XXS !	10.11 GB	69.4 t/s
GeForce RTX 5070 NVIDIA	12 GB	Runs great	IQ2_XXS !	10.11 GB	51.1 t/s
GeForce RTX 2080 Ti NVIDIA	11 GB	Just fits	IQ2_XXS !	10.11 GB	46.9 t/s
GeForce RTX 4070 Ti NVIDIA	12 GB	Runs great	IQ2_XXS !	10.11 GB	38.3 t/s

How it compares

Other models in the same family or of a similar size, on a 12 GB card and a 24 GB card, each at the best quantization that fits.

Model	Weights at 4 bits	GeForce RTX 3060 12GB	GeForce RTX 4090
Qwen2.5 32B Instruct	18.4 GB	does not fit well	37.1 t/s at Q4_K_M
Qwen3 32B	18.4 GB	does not fit well	37.1 t/s at Q4_K_M
Qwen2.5 Coder 32B Instruct	18.4 GB	does not fit well	37.1 t/s at Q4_K_M
Qwen3 30B A3B	17.2 GB	36.3 t/s at Q4_K_M	146.3 t/s at Q5_K_M
Gemma 4 31B Instruct	17.6 GB	does not fit well	35.5 t/s at Q4_K_M
Gemma 4 26B A4B Instruct	14.5 GB	37.6 t/s at Q4_K_M	117 t/s at Q6_K

What to buy to run Qwen2.5 32B Instruct

The cheapest hardware that runs it at a quantization worth using and a speed you would not resent, at an 8k context window.

Cheapest that works

Radeon RX 7900 XT

$650

IQ4_XS · uses 18.26 GB of its 20 GB · about 28 tokens/s

Best value

GeForce RTX 3090

$700 used

Q4_K_M · about 34.5 tokens/s · 936 GB/s

Device	Price	Memory	Runs it at	Speed
Radeon RX 7900 XT AMD	$650	20 GB	IQ4_XS	28 t/s	Check price
GeForce RTX 3090 NVIDIA	$700 used	24 GB	Q4_K_M	34.5 t/s	Check price
GeForce RTX 3090 Ti NVIDIA	$800 used	24 GB	Q4_K_M	37.1 t/s	Check price
Apple M1 Max 32GB Apple · whole machine	$1,400	32 GB	IQ4_XS	15.3 t/s	Check price
Apple M1 Max 64GB Apple · whole machine	$1,900	64 GB	IQ4_XS	15.3 t/s	Check price
GeForce RTX 5090 NVIDIA	$2,200	32 GB	Q6_K	49.9 t/s	Check price
Apple M1 Ultra 128GB Apple · whole machine	$3,400	128 GB	Q8_0	16.2 t/s	Check price
Apple M3 Ultra 256GB Apple · whole machine	$5,599	256 GB	Q8_0	16.5 t/s	Check price

Indicative prices reviewed 2026-09-01; used prices are marketplace typical. Options that cost more than a cheaper one with no more memory and no more speed are hidden. Price links are Amazon affiliate links. Change the standard, the context window or the budget in the buying tool.

These numbers also cover

Fine-tunes share their base model's architecture, so they need exactly the same memory. If you are looking for one of these, the figures above apply unchanged.

deepseek-ai/DeepSeek-R1-Distill-Qwen-32B
Qwen/Qwen2.5-Coder-32B-Instruct
Qwen/Qwen2.5-32B

Where it sits in the catalog

Source: Qwen/Qwen2.5-32B-Instruct on Hugging Face. Downloaded 2.0M times in the last month. Published 2024-09-17. Architecture figures are read from the repository's own configuration file, so they move when the model does.

Qwen3 32B

32.76B · 40k context

Qwen2.5 Coder 32B Instruct

32.76B · 32k context

Qwen3 30B A3B

30.53B (3.34B active) · 40k context

Qwen3 Coder 30B A3B Instruct

30.53B (3.34B active) · 40k context

Qwen3.5 35B A3B

35.95B (2.9B active) · 256k context

Qwen3.5 27B

27.78B · 256k context

Qwen3 14B

14.77B · 40k context

Qwen2.5 14B Instruct

14.77B · 32k context

Questions people ask about Qwen2.5 32B Instruct

How much VRAM does Qwen2.5 32B Instruct need?

At Q4_K_M with an 8k context window, Qwen2.5 32B Instruct needs about 20.42 GB: 17.98 GB of weights, 2 GB of KV cache and 0.44 GB of overhead. At the highest-quality file it needs about 34.09 GB.

What is the cheapest GPU that runs Qwen2.5 32B Instruct?

The smallest consumer device in our catalog that fits it is the Radeon RX 7900 XT with 20 GB, running it at IQ4_XS at roughly 28 tokens per second.

How much does a longer context window add for Qwen2.5 32B Instruct?

On the memory side, a 32k window adds 8.0 GB of KV cache on top of the weights, and the cache grows in a straight line with every token. The longest window the model supports natively is 32k tokens.