Intel Core Ultra 9 288V 32GB: What Local LLMs It Can Run

The short answer

Assuming an 8k context window and default settings, these are the models worth downloading first.

Best all-rounder

GPT OSS 20B

MXFP4 · 11.84 GB · about 18.8 tokens/s

Best for code

DeepSeek Coder V2 Lite Instruct

Q8_0 · 16.12 GB · about 15.4 tokens/s

Largest that still runs well

Phi 3.5 MoE Instruct

41.87B parameters · IQ4_XS · about 10.5 tokens/s

Fastest useful answer

LFM2.5 8B A1B

about 27.2 tokens/s · 8.81 GB

See how fast it feels

Intel Core Ultra 9 288V 32GB running GPT OSS 20B at MXFP4

Wait for the first word1.3 s

Then writes at18.8 tok/s

Whole answer11 s

YouWhy does my model use more memory when the conversation gets longer?

Model

Simulated from our estimate at a 512-token question, not a recording. Assumes nothing else is competing for the GPU.

Where it sits

The GPU can use about 22.4 GB of this machine's 32 GB by default, so for planning it behaves like a 16 GB card with different bandwidth. The 16 GB guide explains what that tier runs and where it runs out, and the bandwidth guide explains why 136 GB/s decides how fast it answers.

We do not track a price for this device, so it does not appear in the budget tools.

Every model, scored on this device

Each row uses the highest-quality quantization that both fits and stays conversational. Speed is a single-stream estimate at 8k context.

Model	Params	Verdict	Download	Memory used	Speed	Max context
GPT OSS 20B GPT-OSS	20.91B (4.18B active)	Runs well	MXFP4	11.84 GB	18.8 t/s	128k
Gemma 4 26B A4B Instruct Gemma	25.81B (3.74B active)	Runs well	Q6_K	20.9 GB	12.2 t/s	16k
Qwen3.5 9B Qwen	9.65B	Runs well	Q6_K	8.07 GB	11.2 t/s	256k
Gemma 4 E4B Instruct Gemma	8B	Runs well	Q6_K	6.61 GB	12.4 t/s	128k
DeepSeek Coder V2 Lite Instruct DeepSeek	15.71B (2.74B active)	Runs well	Q8_0	16.12 GB	15.4 t/s	128k
Qwen2.5 VL 7B Instruct Qwen	8.29B	Runs well	Q6_K	7.16 GB	12.2 t/s	64k
Gemma 4 12B Instruct Gemma	11.96B	Runs well	IQ4_XS	7.63 GB	10.7 t/s	64k
Phi 3.5 MoE Instruct Phi	41.87B (6.64B active)	Just fits	IQ4_XS	22.12 GB	10.5 t/s	8k
Qwen3 8B Qwen	8.19B	Runs well	Q6_K	7.79 GB	11.2 t/s	32k
Granite 4.1 8B Granite	8.79B	Runs well	Q5_K_M	7.48 GB	10.9 t/s	64k
Gemma 4 E2B Instruct Gemma	5.12B	Runs well	Q8_0	5.44 GB	15.1 t/s	128k
Qwen3.5 4B Qwen	4.66B	Runs well	Q8_0	5.27 GB	15.7 t/s	256k
LFM2.5 8B A1B Liquid	8.47B (1.57B active)	Runs great	Q8_0	8.81 GB	27.2 t/s	64k
Llama 3.1 8B Instruct Llama	8.03B	Runs well	Q6_K	7.54 GB	11.5 t/s	64k
Qwen2.5 7B Instruct Qwen	7.62B	Runs well	Q8_0	8.37 GB	10.4 t/s	32k
Gemma 3 12B Instruct Gemma	12.19B	Runs well	Q4_K_M	8.07 GB	10.1 t/s	128k
Mistral 7B Instruct V0.3 Mistral	7.25B	Runs well	Q6_K	6.94 GB	12 t/s	32k
Qwen2.5 Coder 7B Instruct Qwen	7.62B	Runs well	Q8_0	8.37 GB	10.4 t/s	32k
Qwen3 14B Qwen	14.77B	Runs well	Q3_K_M	8.41 GB	10.1 t/s	32k
Internlm3 8B Instruct InternLM	8.8B	Runs well	Q6_K	7.5 GB	11.5 t/s	32k
Mistral Nemo Instruct 2407 Mistral	12.25B	Fits, but slow	Q4_K_M	8.58 GB	10 t/s	64k
Qwen2.5 14B Instruct Qwen	14.77B	Fits, but slow	Q8_0	16.56 GB	5 t/s	32k
Gemma 3 4B Instruct Gemma	4.3B	Runs well	Q8_0	4.91 GB	17 t/s	128k
Qwen2.5 Coder 14B Instruct Qwen	14.77B	Fits, but slow	Q8_0	16.56 GB	5 t/s	32k
OLMo 3 7B Instruct OLMo	7.3B	Fits, but slow	Q6_K	8.48 GB	10 t/s	64k
Mistral 7B Instruct V0.2 Mistral	7.24B	Runs well	Q6_K	6.93 GB	12 t/s	32k
Qwen3 4B Qwen	4.02B	Runs well	Q8_0	5.46 GB	15.1 t/s	32k
Phi 4 Phi	14.66B	Fits, but slow	Q8_0	16.51 GB	5 t/s	16k
Gemma 2 9B Instruct Gemma	9.24B	Runs well	Q4_K_M	7.55 GB	10.8 t/s	8k
Granite 4.1 3B Granite	3.4B	Runs well	Q8_0	4.34 GB	19.4 t/s	128k

Show the other 56 models

Qwen2.5 3B Instruct	3.09B	Runs well	Q8_0	3.68 GB	23.1 t/s	32k
Phi 4 Mini Instruct	3.84B	Runs well	Q8_0	5.17 GB	16.1 t/s	128k
Llama 3.2 3B Instruct	3.21B	Runs well	Q8_0	4.42 GB	19.1 t/s	128k
Phi 3 Mini 4k Instruct	3.82B	Runs well	Q8_0	4.9 GB	17.4 t/s	4k
LFM2.5 2.6B	2.7B	Runs great	Q8_0	3.14 GB	27.6 t/s	128k
Qwen3 Next 80B A3B Instruct	81.32B	Fits only at 2 to 3 bits	IQ2_XXS	20.1 GB	43.9 t/s	64k
Qwen3.5 2B	2.27B	Runs great	Q8_0	2.7 GB	32.8 t/s	256k
DeepSeek Coder 7B Instruct V1.5	6.91B	Runs well	Q4_K_M	8.04 GB	10.4 t/s	4k
DeepSeek Coder 6.7B Instruct	6.74B	Runs well	Q4_K_M	8.2 GB	10 t/s	16k
Gemma 2 2B Instruct	2.61B	Runs well	Q8_0	3.54 GB	24.2 t/s	8k
Starcoder2 3B	3.03B	Runs great	Q8_0	3.49 GB	26 t/s	16k
Qwen3 1.7B	2.03B	Runs great	Q8_0	3.22 GB	26.8 t/s	32k
DeepSeek R1 Distill Qwen 1.5B	1.78B	Runs great	Q8_0	2.3 GB	44.2 t/s	128k
Qwen2.5 1.5B Instruct	1.54B	Runs great	Q8_0	2.06 GB	44.3 t/s	32k
SmolLM2 1.7B	1.71B	Runs well	Q8_0	3.53 GB	24.2 t/s	8k
OLMo 2 0425 1B	1.48B	Runs great	Q8_0	2.8 GB	34.2 t/s	4k
Llama 3.2 1B Instruct	1.24B	Runs great	Q8_0	1.82 GB	52.3 t/s	128k
LFM2.5 1.2B Instruct	1.17B	Runs great	Q8_0	1.59 GB	69.1 t/s	64k
Gemma 3 1B Instruct	1B	Runs great	Q8_0	1.35 GB	74.4 t/s	32k
Qwen3.5 0.8B	0.87B	Runs great	Q8_0	1.28 GB	79.4 t/s	256k
Qwen3 0.6B	0.75B	Runs great	Q8_0	1.92 GB	47.8 t/s	32k
Qwen2.5 0.5B Instruct	0.49B	Runs great	Q8_0	0.88 GB	133.5 t/s	32k
LFM2.5 350M	0.35B	Runs great	Q8_0	0.74 GB	206.8 t/s	64k
Qwen3 Coder Next	79.67B	Fits only at 2 to 3 bits	n/a	28.98 GB	32.2 t/s	n/a
Qwen3.5 35B A3B	35.95B	Just fits	n/a	28.01 GB	18.7 t/s	n/a
GLM 4.7 Flash	31.22B	Just fits	n/a	24.59 GB	14.1 t/s	n/a
Qwen3 30B A3B	30.53B	Just fits	n/a	24.4 GB	13.7 t/s	n/a
Qwen3 Coder 30B A3B Instruct	30.53B	Just fits	n/a	24.4 GB	13.7 t/s	n/a
GLM 4.5 Air	110.47B	Fits only at 2 to 3 bits	n/a	28.34 GB	9.7 t/s	n/a
Mixtral 8x7B Instruct V0.1	46.7B	Just fits	n/a	27.66 GB	5.5 t/s	n/a
Codestral 22B V0.1	22.25B	Just fits	n/a	24.24 GB	3.3 t/s	n/a
Granite 4.1 30B	28.87B	Just fits	n/a	24.45 GB	3.2 t/s	n/a
Mistral Small 24B Instruct 2501	23.57B	Just fits	n/a	25.01 GB	3.2 t/s	n/a
Qwen3 32B	32.76B	Just fits	n/a	27.46 GB	2.9 t/s	n/a
Qwen2.5 32B Instruct	32.76B	Just fits	n/a	27.46 GB	2.9 t/s	n/a
Qwen3.5 27B	27.78B	Just fits	n/a	28.57 GB	2.9 t/s	n/a
Qwen2.5 Coder 32B Instruct	32.76B	Just fits	n/a	27.46 GB	2.9 t/s	n/a
Qwen2.5 72B Instruct	72.71B	Fits only at 2 to 3 bits	n/a	28.94 GB	2.8 t/s	n/a
Llama 3.3 70B Instruct	70.55B	Fits only at 2 to 3 bits	n/a	28.17 GB	2.8 t/s	n/a
Gemma 4 31B Instruct	31.27B	Just fits	n/a	27.61 GB	2.8 t/s	n/a
Qwen2.5 VL 72B Instruct	73.41B	Fits only at 2 to 3 bits	n/a	29.19 GB	2.7 t/s	n/a
Gemma 3 27B Instruct	27.43B	Just fits	n/a	28.68 GB	2.7 t/s	n/a
Llama 3 3 Nemotron Super 49B V1	49.87B	CPU only	n/a	48.58 GB	n/a	n/a
Kimi K2.6	1026.88B	Will not run	n/a	578.44 GB	n/a	n/a
Kimi K2 Instruct	1026.41B	Will not run	n/a	578.18 GB	n/a	n/a
GLM 5.1	753.86B	Will not run	n/a	425.04 GB	n/a	n/a
GLM 5.2	753.33B	Will not run	n/a	424.75 GB	n/a	n/a
DeepSeek V3.2	685.4B	Will not run	n/a	386.43 GB	n/a	n/a
DeepSeek R1	684.53B	Will not run	n/a	385.95 GB	n/a	n/a
MiniMax M3	427.04B	Will not run	n/a	241.53 GB	n/a	n/a
GLM 4.5	358.34B	Will not run	n/a	204.8 GB	n/a	n/a
Qwen3 235B A22B	235.09B	Will not run	n/a	134.06 GB	n/a	n/a
MiniMax M2.5	228.7B	Will not run	n/a	130.91 GB	n/a	n/a
MiniMax M2.7	228.69B	Will not run	n/a	130.9 GB	n/a	n/a
Qwen3.5 122B A10B	125.09B	Will not run	n/a	71.04 GB	n/a	n/a
GPT OSS 120B	116.83B	Will not run	n/a	59.68 GB	n/a	n/a

What it will not run

33 of the 86 architectures we track are out of reach here, even at 2-bit precision.

If you want more headroom

The next steps up in memory, in order. More memory changes which models load at all; more bandwidth changes how fast they answer, so the two are worth weighing separately.

Device	Memory	Bandwidth	Models that fit
RTX 6000 Ada Generation	48 GB	960 GB/s	72	Check price
Radeon PRO W7900	48 GB	864 GB/s	72	Check price
RTX A6000	48 GB	768 GB/s	72	Check price
Ryzen AI Max+ 395 64GB	64 GB	256 GB/s	66	Check price

Price links go to an Amazon search for the model name and are affiliate links: if you buy through one, we earn a commission at no cost to you. We do not take payment for placement, and the ordering above is by memory capacity alone.

Other devices with 32 GB

Same capacity, different speed. Once a model fits, bandwidth is what separates these.

GeForce RTX 5090

1792 GB/s · NVIDIA

Apple M1 Max 32GB

400 GB/s · Apple

Apple M2 Max 32GB

400 GB/s · Apple

Ryzen AI Max+ 395 32GB

256 GB/s · AMD

Jetson AGX Orin 32GB

204.8 GB/s · NVIDIA

Apple M1 Pro 32GB

200 GB/s · Apple

Questions people ask about an Intel Core Ultra 9 288V 32GB

How much VRAM does an Intel Core Ultra 9 288V 32GB have?

The Intel Core Ultra 9 288V 32GB has 32 GB of unified memory shared with the CPU, of which roughly 22.4 GB is available to a model once the operating system takes its share.

Can an Intel Core Ultra 9 288V 32GB run a 70B model?

Yes, once the macOS GPU memory limit is raised. At IQ3_XXS it needs 28.98 GB and runs at around 32.2 tokens per second.

What is the best local model for an Intel Core Ultra 9 288V 32GB?

GPT OSS 20B at MXFP4 is the strongest all-round fit: 11.84 GB used, around 18.8 tokens per second at 8k context.