Data center GPU · AMD
What AI models can an Instinct MI300X run?
192 GB is server territory. The constraint stops being memory and becomes bandwidth: everything fits, and what varies is how long you wait for it. At 5300 GB/s there is enough bandwidth to keep generation responsive for anything that fits.
The short answer
Assuming an 8k context window and default settings, these are the models worth downloading first.
See how fast it feels
Instinct MI300X running MiniMax M2.7 at Q6_K
YouWhy does my model use more memory when the conversation gets longer?
Reading your question
Simulated from our estimate at a 512-token question, not a recording. Assumes nothing else is competing for the GPU.
Where it sits
With 192 GB it belongs to the 48 GB tier. The 48 GB guide explains what that tier runs and where it runs out, and the bandwidth guide explains why 5300 GB/s decides how fast it answers.
We do not track a price for this device, so it does not appear in the budget tools.
Every model, scored on this device
Each row uses the highest-quality quantization that both fits and stays conversational. Speed is a single-stream estimate at 8k context.
| Model | Params | Verdict | Download | Memory used | Speed | Max context |
|---|---|---|---|---|---|---|
| MiniMax M2.7 MiniMax | 228.69B (10.98B active) | Runs great | Q6_K | 176.49 GB | 186.2 t/s | 64k |
| MiniMax M2.5 MiniMax | 228.7B (10.98B active) | Runs great | Q6_K | 176.5 GB | 186.2 t/s | 64k |
| Qwen3 235B A22B Qwen | 235.09B (22.14B active) | Runs great | Q6_K | 180.93 GB | 104.6 t/s | 32k |
| Qwen3.5 122B A10B Qwen | 125.09B (8.17B active) | Runs great | Q8_0 | 123.73 GB | 229.3 t/s | 256k |
| GPT OSS 120B GPT-OSS | 116.83B (5.7B active) | Runs great | MXFP4 | 59.4 GB | 608.6 t/s | 128k |
| GLM 4.5 GLM | 358.34B (33.56B active) | Runs great | IQ4_XS | 180.23 GB | 98.7 t/s | 32k |
| Llama 3.3 70B Instruct Llama | 70.55B | Runs great | Q8_0 | 71.81 GB | 46.4 t/s | 128k |
| Qwen2.5 VL 72B Instruct Qwen | 73.41B | Runs great | Q8_0 | 74.45 GB | 44.7 t/s | 64k |
| Qwen2.5 72B Instruct Qwen | 72.71B | Runs great | Q8_0 | 73.76 GB | 45.2 t/s | 32k |
| Gemma 4 31B Instruct Gemma | 31.27B | Runs great | Q8_0 | 34.67 GB | 96.6 t/s | 256k |
| GLM 4.5 Air GLM | 110.47B (13.4B active) | Runs great | Q8_0 | 110.54 GB | 131.2 t/s | 128k |
| Qwen3 32B Qwen | 32.76B | Runs great | Q8_0 | 34.09 GB | 98.3 t/s | 32k |
| Qwen3.5 27B Qwen | 27.78B | Runs great | Q8_0 | 27.32 GB | 123.1 t/s | 256k |
| Llama 3 3 Nemotron Super 49B V1 Llama | 49.87B | Runs great | Q8_0 | 68.85 GB | 48.4 t/s | 32k |
| Qwen3 Coder Next Qwen | 79.67B (3.19B active) | Runs great | Q8_0 | 79.13 GB | 563.5 t/s | 256k |
| Granite 4.1 30B Granite | 28.87B | Runs great | Q8_0 | 30.97 GB | 108.2 t/s | 128k |
| Qwen2.5 32B Instruct Qwen | 32.76B | Runs great | Q8_0 | 34.09 GB | 98.3 t/s | 32k |
| Qwen2.5 Coder 32B Instruct Qwen | 32.76B | Runs great | Q8_0 | 34.09 GB | 98.3 t/s | 32k |
| Qwen3 Next 80B A3B Instruct Qwen | 81.32B (3.19B active) | Runs great | Q8_0 | 80.76 GB | 563.5 t/s | 256k |
| Gemma 3 27B Instruct Gemma | 27.43B | Runs great | Q8_0 | 28.68 GB | 117.2 t/s | 128k |
| Qwen3.5 35B A3B Qwen | 35.95B (2.9B active) | Runs great | Q8_0 | 35.63 GB | 629.9 t/s | 256k |
| Gemma 4 26B A4B Instruct Gemma | 25.81B (3.74B active) | Runs great | Q8_0 | 26.72 GB | 424.3 t/s | 256k |
| GLM 4.7 Flash GLM | 31.22B (3.66B active) | Runs great | Q8_0 | 31.33 GB | 479.8 t/s | 128k |
| Mixtral 8x7B Instruct V0.1 Mistral | 46.7B (12.88B active) | Runs great | Q8_0 | 47.49 GB | 139.9 t/s | 32k |
| Qwen3 30B A3B Qwen | 30.53B (3.34B active) | Runs great | Q8_0 | 30.99 GB | 477 t/s | 32k |
| Mistral Small 24B Instruct 2501 Mistral | 23.57B | Runs great | Q8_0 | 24.35 GB | 138.3 t/s | 32k |
| Phi 3.5 MoE Instruct Phi | 41.87B (6.64B active) | Runs great | Q8_0 | 42.71 GB | 254.1 t/s | 128k |
| Gemma 4 12B Instruct Gemma | 11.96B | Runs great | Q8_0 | 13.55 GB | 251.5 t/s | 256k |
| Qwen3 14B Qwen | 14.77B | Runs great | Q8_0 | 15.54 GB | 219.1 t/s | 32k |
| GPT OSS 20B GPT-OSS | 20.91B (4.18B active) | Runs great | MXFP4 | 11.55 GB | 803.2 t/s | 128k |
Show the other 56 models
| Qwen3 Coder 30B A3B Instruct | 30.53B | Runs great | Q8_0 | 30.99 GB | 477 t/s | 32k |
| Qwen2.5 14B Instruct | 14.77B | Runs great | Q8_0 | 15.79 GB | 215.5 t/s | 32k |
| Qwen2.5 Coder 14B Instruct | 14.77B | Runs great | Q8_0 | 15.79 GB | 215.5 t/s | 32k |
| Codestral 22B V0.1 | 22.25B | Runs great | Q8_0 | 24.04 GB | 140.3 t/s | 32k |
| Qwen3.5 9B | 9.65B | Runs great | Q8_0 | 9.25 GB | 374.1 t/s | 256k |
| Phi 4 | 14.66B | Runs great | Q8_0 | 16 GB | 212.5 t/s | 16k |
| Gemma 3 12B Instruct | 12.19B | Runs great | Q8_0 | 13.27 GB | 256.9 t/s | 128k |
| GLM 5.2 | 753.33B | Fits only at 2 to 3 bits | IQ2_XXS | 181.59 GB | 147 t/s | 64k |
| Gemma 4 E4B Instruct | 8B | Runs great | Q8_0 | 8.42 GB | 410.3 t/s | 128k |
| Granite 4.1 8B | 8.79B | Runs great | Q8_0 | 10.35 GB | 332.4 t/s | 128k |
| Qwen3 8B | 8.19B | Runs great | Q8_0 | 9.02 GB | 383.9 t/s | 32k |
| Mistral Nemo Instruct 2407 | 12.25B | Runs great | Q8_0 | 13.15 GB | 260.2 t/s | 128k |
| Qwen2.5 VL 7B Instruct | 8.29B | Runs great | Q8_0 | 8.49 GB | 408.2 t/s | 64k |
| DeepSeek V3.2 | 685.4B | Fits only at 2 to 3 bits | IQ2_XXS | 165.19 GB | 196.3 t/s | 128k |
| DeepSeek R1 | 684.53B | Fits only at 2 to 3 bits | IQ2_XXS | 164.98 GB | 196.3 t/s | 128k |
| Qwen2.5 7B Instruct | 7.62B | Runs great | Q8_0 | 7.83 GB | 444.6 t/s | 32k |
| Llama 3.1 8B Instruct | 8.03B | Runs great | Q8_0 | 8.83 GB | 392.5 t/s | 128k |
| GLM 5.1 | 753.86B | Fits only at 2 to 3 bits | IQ2_XXS | 181.72 GB | 206.6 t/s | 64k |
| DeepSeek Coder V2 Lite Instruct | 15.71B | Runs great | Q8_0 | 15.91 GB | 658.6 t/s | 128k |
| Gemma 2 9B Instruct | 9.24B | Runs great | Q8_0 | 11.5 GB | 297.6 t/s | 8k |
| Qwen2.5 Coder 7B Instruct | 7.62B | Runs great | Q8_0 | 7.83 GB | 444.6 t/s | 32k |
| OLMo 3 7B Instruct | 7.3B | Runs great | Q8_0 | 9.72 GB | 355 t/s | 64k |
| Internlm3 8B Instruct | 8.8B | Runs great | Q8_0 | 8.97 GB | 386.3 t/s | 32k |
| Mistral 7B Instruct V0.3 | 7.25B | Runs great | Q8_0 | 8.45 GB | 411.3 t/s | 32k |
| Gemma 4 E2B Instruct | 5.12B | Runs great | Q8_0 | 5.44 GB | 646 t/s | 128k |
| Qwen3.5 4B | 4.66B | Runs great | Q8_0 | 5.27 GB | 673.7 t/s | 256k |
| Mistral 7B Instruct V0.2 | 7.24B | Runs great | Q8_0 | 8.44 GB | 411.6 t/s | 32k |
| DeepSeek Coder 7B Instruct V1.5 | 6.91B | Runs great | Q8_0 | 10.58 GB | 325.1 t/s | 4k |
| DeepSeek Coder 6.7B Instruct | 6.74B | Runs great | Q8_0 | 10.94 GB | 313.8 t/s | 16k |
| LFM2.5 8B A1B | 8.47B | Runs great | Q8_0 | 8.81 GB | 1164.4 t/s | 64k |
| Qwen3 4B | 4.02B | Runs great | Q8_0 | 5.46 GB | 648.1 t/s | 32k |
| Gemma 3 4B Instruct | 4.3B | Runs great | Q8_0 | 4.91 GB | 726.5 t/s | 128k |
| Phi 4 Mini Instruct | 3.84B | Runs great | Q8_0 | 5.17 GB | 689 t/s | 128k |
| Granite 4.1 3B | 3.4B | Runs great | Q8_0 | 4.34 GB | 829 t/s | 128k |
| Qwen2.5 3B Instruct | 3.09B | Runs great | Q8_0 | 3.68 GB | 990.5 t/s | 32k |
| Phi 3 Mini 4k Instruct | 3.82B | Runs great | Q8_0 | 4.8 GB | 746.2 t/s | 4k |
| Llama 3.2 3B Instruct | 3.21B | Runs great | Q8_0 | 4.42 GB | 816.3 t/s | 128k |
| LFM2.5 2.6B | 2.7B | Runs great | Q8_0 | 3.14 GB | 1182.5 t/s | 128k |
| Qwen3.5 2B | 2.27B | Runs great | Q8_0 | 2.7 GB | 1402.4 t/s | 256k |
| Gemma 2 2B Instruct | 2.61B | Runs great | Q8_0 | 3.54 GB | 1036.1 t/s | 8k |
| Starcoder2 3B | 3.03B | Runs great | Q8_0 | 3.34 GB | 1115 t/s | 16k |
| Qwen3 1.7B | 2.03B | Runs great | Q8_0 | 3.22 GB | 1146.8 t/s | 32k |
| DeepSeek R1 Distill Qwen 1.5B | 1.78B | Runs great | Q8_0 | 2.07 GB | 1890.7 t/s | 128k |
| Qwen2.5 1.5B Instruct | 1.54B | Runs great | Q8_0 | 2.06 GB | 1897.8 t/s | 32k |
| SmolLM2 1.7B | 1.71B | Runs great | Q8_0 | 3.53 GB | 1036 t/s | 8k |
| OLMo 2 0425 1B | 1.48B | Runs great | Q8_0 | 2.6 GB | 1462.6 t/s | 4k |
| Llama 3.2 1B Instruct | 1.24B | Runs great | Q8_0 | 1.82 GB | 2239.1 t/s | 128k |
| LFM2.5 1.2B Instruct | 1.17B | Runs great | Q8_0 | 1.46 GB | 2956.3 t/s | 64k |
| Gemma 3 1B Instruct | 1B | Runs great | Q8_0 | 1.35 GB | 3183.5 t/s | 32k |
| Qwen3.5 0.8B | 0.87B | Runs great | Q8_0 | 1.28 GB | 3399.6 t/s | 256k |
| Qwen3 0.6B | 0.75B | Runs great | Q8_0 | 1.92 GB | 2045 t/s | 32k |
| Qwen2.5 0.5B Instruct | 0.49B | Runs great | Q8_0 | 0.88 GB | 5715.5 t/s | 32k |
| LFM2.5 350M | 0.35B | Runs great | Q8_0 | 0.68 GB | 8850.2 t/s | 64k |
| MiniMax M3 | 427.04B | Partial offload | n/a | 240.84 GB | 17 t/s | n/a |
| Kimi K2.6 | 1026.88B | Partial offload | n/a | 577.78 GB | 3.6 t/s | n/a |
| Kimi K2 Instruct | 1026.41B | Partial offload | n/a | 577.52 GB | 3.6 t/s | n/a |
What it will not run
0 of the 86 architectures we track are out of reach here, even at 2-bit precision. Another 3 run by splitting the model between the card and system RAM, which works but drops generation to single digits for most dense models.
Other devices with 192 GB
Same capacity, different speed. Once a model fits, bandwidth is what separates these.
Questions people ask about an Instinct MI300X
How much VRAM does an Instinct MI300X have?
The Instinct MI300X has 192 GB of VRAM, of which roughly 191.2 GB is available to a model once the operating system takes its share.
Can an Instinct MI300X run a 70B model?
Yes. At Q8_0 a 70B model fits in 71.81 GB and generates around 46.4 tokens per second.
What is the best local model for an Instinct MI300X?
MiniMax M2.7 at Q6_K is the strongest all-round fit: 176.49 GB used, around 186.2 tokens per second at 8k context.