Unified memory · NVIDIA
What AI models can an NVIDIA DGX Spark 128GB run?
128 GB is server territory. The constraint stops being memory and becomes bandwidth: everything fits, and what varies is how long you wait for it. Memory is shared with the CPU, so the practical ceiling is lower than the sticker number: macOS hands roughly three quarters of it to the GPU by default.
The short answer
Assuming an 8k context window and default settings, these are the models worth downloading first.
Best all-rounder
Laguna S 2.1
Q6_K · 90.66 GB · about 37.4 tokens/s
Best for code
Laguna S 2.1
Q6_K · 90.66 GB · about 37.4 tokens/s
Largest that still runs well
DeepSeek V4 Flash DSpark
165.27B parameters · Q4_K_M · about 19.5 tokens/s
Fastest useful answer
PowerMoE 3B
about 163.3 tokens/s · 4.53 GB
See how fast it feels
NVIDIA DGX Spark 128GB running Laguna S 2.1 at Q6_K
YouWhy does my model use more memory when the conversation gets longer?
Because of the KV cache. Every token you send leaves behind a key and a value vector in each layer of the model, and those stay in memory for as long as the conversation lasts.
The weights are a fixed cost: load a 4-bit 8B model and that is about 4.8 GB, whether you write one word or ten thousand. The cache is the part that grows, and it grows in a straight line with the number of tokens in the window.
How steeply depends on the model's attention design. With grouped-query attention, several query heads share one key-value pair, which cuts the cache by that ratio. Without it, every head keeps its own, and a long context can cost more memory than the weights themselves.
If you are short on memory, the first thing to try is lowering the context window in your runtime, not the quantisation.
Reading your question
Simulated from our estimate at a 512-token question, not a recording. Assumes nothing else is competing for the GPU.
Every model, scored on this device
Each row uses the highest-quality quantisation that both fits and stays conversational. Speed is a single-stream estimate at 8k context.
| Model | Params | Verdict | Download | Memory used | Speed | Max context |
|---|---|---|---|---|---|---|
| Laguna S 2.1 Laguna | 117.56B (7.71B active) | Runs great | Q6_K | 90.66 GB | 37.4 t/s | 128k |
| GPT OSS 120B GPT-OSS | 116.83B (5.7B active) | Runs great | Q6_K | 90.01 GB | 51.3 t/s | 128k |
| Qwen3.5 122B A10B Qwen | 125.09B (8.17B active) | Runs great | Q5_K_M | 84.4 GB | 36.3 t/s | 64k |
| GLM 4.5 Air GLM | 110.47B (13.4B active) | Runs well | Q6_K | 86.65 GB | 19.2 t/s | 32k |
| DeepSeek V4 Flash DSpark DeepSeek | 165.27B (20.36B active) | Runs well | Q4_K_M | 93.79 GB | 19.5 t/s | 64k |
| Qwen3 Next 80B A3B Instruct Qwen | 81.32B (3.19B active) | Runs great | Q8_0 | 81.94 GB | 57.3 t/s | 128k |
| Qwen3 Coder Next Qwen | 79.67B (3.19B active) | Runs great | Q8_0 | 80.31 GB | 57.3 t/s | 128k |
| Phi 3.5 MoE Instruct Phi | 41.87B (6.64B active) | Runs great | Q8_0 | 43.28 GB | 29.6 t/s | 128k |
| Qwen3.5 35B A3B Qwen | 35.95B (2.9B active) | Runs great | Q8_0 | 36.93 GB | 64.1 t/s | 256k |
| Mixtral 8x7B Instruct V0.1 Mistral | 46.7B (12.88B active) | Runs well | Q8_0 | 48.06 GB | 16.3 t/s | 32k |
| Qwen3 235B A22B Qwen | 235.09B (22.14B active) | Runs well | IQ3_XXS ! | 86.07 GB | 23.9 t/s | 32k |
| GLM 4.7 Flash GLM | 31.22B (3.66B active) | Runs great | Q8_0 | 32.03 GB | 55.5 t/s | 128k |
| MiniMax M2.7 MiniMax | 228.69B (10.98B active) | Runs great | IQ3_XXS ! | 84.19 GB | 38.3 t/s | 32k |
| MiniMax M2.5 MiniMax | 228.7B (10.98B active) | Runs great | IQ3_XXS ! | 84.2 GB | 38.3 t/s | 32k |
| Qwen3 30B A3B Qwen | 30.53B (3.34B active) | Runs great | Q8_0 | 31.69 GB | 55.2 t/s | 32k |
| Command A Plus 05 2026 Bf16 Command | 218.75B (18.52B active) | Runs great | IQ3_XXS ! | 79.28 GB | 31.5 t/s | 128k |
| DeepSeek V4 Flash 0731 Spark DeepSeek | 60.31B (20.36B active) | Runs well | Q8_0 | 60.54 GB | 11.1 t/s | 1M |
| GPT OSS 20B GPT-OSS | 20.91B (4.18B active) | Runs great | Q8_0 | 21.47 GB | 54 t/s | 128k |
| DeepSeek Coder V2 Lite Instruct DeepSeek | 15.71B (2.74B active) | Runs great | Q8_0 | 16.51 GB | 75.9 t/s | 128k |
| Qwen1.5 MoE A2.7B Qwen | 14.32B (2.69B active) | Runs great | Q8_0 | 16.4 GB | 53.8 t/s | 8k |
| Gemma 4 26B A4B Instruct Gemma | 25.81B | Runs well | Q6_K | 20.72 GB | 11.2 t/s | 256k |
| Mistral Small 24B Instruct 2501 Mistral | 23.57B | Runs well | Q6_K | 20.16 GB | 11.6 t/s | 32k |
| Gemma 4 31B Instruct Gemma | 31.27B | Runs well | Q5_K_M | 22.58 GB | 10.3 t/s | 256k |
| OTel 2.0 LLM 31B Instruct Other | 32.11B | Runs well | Q5_K_M | 23.14 GB | 10.1 t/s | 256k |
| Gemma 3 27B Instruct Gemma | 27.43B | Runs well | Q6_K | 22.96 GB | 10.2 t/s | 128k |
| Codestral 22B V0.1 Mistral | 22.25B | Runs well | Q6_K | 19.72 GB | 11.9 t/s | 32k |
| Qwen3.5 27B Qwen | 27.78B | Runs well | Q5_K_M | 21.32 GB | 11 t/s | 256k |
| Granite 4.1 30B Granite | 28.87B | Runs well | Q5_K_M | 21.97 GB | 10.6 t/s | 128k |
| Gemma 4 12B Instruct Gemma | 11.96B | Runs well | Q8_0 | 13.05 GB | 18.3 t/s | 256k |
| vLLM Translategemma 12B Instruct Gemma | 13.19B | Runs well | Q8_0 | 14.26 GB | 16.7 t/s | 128k |
| Qwen3 32B Qwen | 32.76B | Runs well | Q4_K_M | 21.33 GB | 11 t/s | 32k |
| Qwen2.5 32B Instruct Qwen | 32.76B | Runs well | Q4_K_M | 21.33 GB | 11 t/s | 32k |
| Gemma 3 12B Instruct Gemma | 12.19B | Runs well | Q8_0 | 13.71 GB | 17.4 t/s | 128k |
| Mistral Nemo Instruct 2407 Mistral | 12.25B | Runs well | Q8_0 | 14.29 GB | 16.7 t/s | 128k |
| Qwen3 14B Qwen | 14.77B | Runs well | Q8_0 | 16.78 GB | 14.1 t/s | 32k |
| Qwen2.5 14B Instruct Qwen | 14.77B | Runs well | Q8_0 | 17.03 GB | 13.9 t/s | 32k |
| Phi 4 Phi | 14.66B | Runs well | Q8_0 | 16.98 GB | 13.9 t/s | 16k |
| Qwen3.5 9B Qwen | 9.65B | Runs well | Q8_0 | 11.4 GB | 21.2 t/s | 256k |
| Gemma 2 9B Instruct Gemma | 9.24B | Runs well | Q8_0 | 11.28 GB | 21.4 t/s | 8k |
| GLM 4.5 GLM | 358.34B (33.56B active) | Runs well | IQ2_XXS ! | 89.72 GB | 20.5 t/s | 16k |
| Granite 4.1 8B Granite | 8.79B | Runs well | Q8_0 | 10.8 GB | 22.5 t/s | 128k |
| Fanar 1 9B Instruct Fanar | 8.78B | Runs well | Q8_0 | 10.82 GB | 22.4 t/s | 4k |
| Internlm3 8B Instruct InternLM | 8.8B | Runs well | Q8_0 | 9.93 GB | 24.6 t/s | 32k |
| MiMo V2.5 MiMo | 310.78B (16.05B active) | Runs great | IQ2_XXS ! | 75.4 GB | 57.9 t/s | 256k |
| MiMo V2 Flash MiMo | 309.79B (16.05B active) | Runs great | IQ2_XXS ! | 75.16 GB | 57.9 t/s | 256k |
| LFM2.5 8B A1B Liquid | 8.47B (1.57B active) | Runs great | Q8_0 | 9.48 GB | 116.1 t/s | 64k |
| DeepSeek V4 Flash 0731 DeepSeek | 304.18B (20.36B active) | Runs great | IQ2_XXS ! | 73.81 GB | 45.7 t/s | 256k |
| Qwen2.5 VL 7B Instruct Qwen | 8.29B | Runs great | Q8_0 | 9.46 GB | 25.9 t/s | 64k |
| DeepSeek V4 Flash DeepSeek | 290.94B (20.36B active) | Runs great | IQ2_XXS ! | 70.63 GB | 45.7 t/s | 256k |
| Qwen3 8B Qwen | 8.19B | Runs well | Q8_0 | 10.08 GB | 24.3 t/s | 32k |
| Granite 3.0 8B Instruct Granite | 8.17B | Runs well | Q8_0 | 10.18 GB | 24 t/s | 4k |
| T Lite Instruct 2.1 T-Lite | 8.19B | Runs well | Q8_0 | 10.08 GB | 24.3 t/s | 32k |
| GPT NeoX 20B GPT-NeoX | 20.74B | Runs well | Q5_K_M | 22.96 GB | 10.2 t/s | 2k |
| Llama 3.1 8B Instruct Llama | 8.03B | Runs great | Q8_0 | 9.8 GB | 25 t/s | 128k |
| Apertus 8B Instruct 2509 Apertus | 8.05B | Runs well | Q8_0 | 9.82 GB | 25 t/s | 64k |
| Llama 3 Taiwan 8B Instruct Llama | 8.03B | Runs great | Q8_0 | 9.8 GB | 25 t/s | 8k |
| Gemma 4 E4B Instruct Gemma | 8B | Runs great | Q8_0 | 8.72 GB | 28.1 t/s | 128k |
| Qwen2.5 7B Instruct Qwen | 7.62B | Runs great | Q8_0 | 8.8 GB | 28.1 t/s | 32k |
| OLMo 2 1124 13B Instruct OLMo | 13.72B | Runs well | Q8_0 | 20.74 GB | 11.3 t/s | 4k |
| Qwen1.5 7B Qwen | 7.72B | Runs well | Q8_0 | 12.49 GB | 19.2 t/s | 32k |
| OLMo 3 7B Instruct OLMo | 7.3B | Runs well | Q8_0 | 10.07 GB | 24.3 t/s | 64k |
| Mistral 7B Instruct V0.3 Mistral | 7.25B | Runs great | Q8_0 | 9.02 GB | 27.4 t/s | 32k |
| Mistral 7B Instruct V0.2 Mistral | 7.24B | Runs great | Q8_0 | 9.01 GB | 27.4 t/s | 32k |
| Falcon 7B Falcon | 7.22B | Runs well | Q8_0 | 12.46 GB | 19.3 t/s | 8k |
| DeepSeek Coder 7B Instruct V1.5 DeepSeek | 6.91B | Runs well | Q8_0 | 11.44 GB | 21.1 t/s | 4k |
| OLMoE 1B 7B 0125 Instruct OLMo | 6.92B (1.28B active) | Runs great | Q8_0 | 8.57 GB | 98.8 t/s | 4k |
| CodeLlama 7B Llama | 6.74B | Runs well | Q8_0 | 11.52 GB | 21 t/s | 16k |
| DeepSeek Coder 6.7B Instruct DeepSeek | 6.74B | Runs well | Q8_0 | 11.52 GB | 21 t/s | 16k |
| Llama 3.3 70B Instruct Llama | 70.55B | Fits, but slow | Q8_0 | 73.41 GB | 3.1 t/s | 64k |
| Qwen2.5 VL 72B Instruct Qwen | 73.41B | Fits, but slow | Q8_0 | 76.24 GB | 3 t/s | 64k |
| Qwen2.5 72B Instruct Qwen | 72.71B | Fits, but slow | Q8_0 | 75.55 GB | 3 t/s | 32k |
| Llama 3 3 Nemotron Super 49B V1 Llama | 49.87B | Fits, but slow | Q8_0 | 70.45 GB | 3.2 t/s | 16k |
| Gemma 4 E2B Instruct Gemma | 5.12B | Runs great | Q8_0 | 5.78 GB | 44 t/s | 128k |
| Qwen3.5 4B Qwen | 4.66B | Runs great | Q8_0 | 6.37 GB | 39.9 t/s | 256k |
| Agents A1 4B Other | 4.54B | Runs great | Q8_0 | 6.25 GB | 40.8 t/s | 256k |
| Step 3.5 Flash Step | 199.38B | Fits, but slow | Q3_K_M | 92.31 GB | 2.4 t/s | 64k |
| Gemma 3 4B Instruct Gemma | 4.3B | Runs great | Q8_0 | 5.31 GB | 49.2 t/s | 128k |
| Phi 3 Vision 128k Instruct Phi | 4.15B | Runs great | Q8_0 | 7.89 GB | 31.5 t/s | 128k |
| Qwen3 4B Qwen | 4.02B | Runs great | Q8_0 | 5.86 GB | 43.9 t/s | 32k |
| Phi 4 Mini Instruct Phi | 3.84B | Runs great | Q8_0 | 5.59 GB | 46.6 t/s | 128k |
| Phi 3 Mini 4k Instruct Phi | 3.82B | Runs great | Q8_0 | 5.32 GB | 49.4 t/s | 4k |
| PowerLM 3B PowerLM | 3.51B | Runs great | Q8_0 | 7.03 GB | 35.6 t/s | 4k |
| Granite 4.1 3B Granite | 3.4B | Runs great | Q8_0 | 4.75 GB | 56.1 t/s | 128k |
| PowerMoE 3B PowerLM | 3.37B (0.88B active) | Runs great | Q8_0 | 4.53 GB | 163.3 t/s | 4k |
| Llama 3.2 3B Instruct Llama | 3.21B | Runs great | Q8_0 | 4.84 GB | 55.3 t/s | 128k |
| Qwen2.5 3B Instruct Qwen | 3.09B | Runs great | Q8_0 | 4.07 GB | 67 t/s | 32k |
| SmolLM3 3B Base SmolLM | 3.08B | Runs great | Q8_0 | 4.34 GB | 62 t/s | 64k |
| Starcoder2 3B StarCoder | 3.03B | Runs great | Q8_0 | 3.9 GB | 71.9 t/s | 16k |
| Phi 2 Phi | 2.78B | Runs great | Q8_0 | 6.01 GB | 42.6 t/s | 2k |
| LFM2.5 2.6B Liquid | 2.7B | Runs great | Q8_0 | 3.87 GB | 71.3 t/s | 128k |
Showing the 90 best results of 133. The remaining 43 need more memory than this device has, at any quantisation.
What it will not run
12 of the 133 architectures we track are out of reach here, even at two-bit precision.
Other devices with 128 GB
Same capacity, different speed. Once a model fits, bandwidth is what separates these.