How much VRAM does an H100 PCIe 80GB have?

The H100 PCIe 80GB has 80 GB of VRAM, of which roughly 79.2 GB is available to a model once the operating system takes its share.

Can an H100 PCIe 80GB run a 70B model?

Yes. At Q8_0 a 70B model fits in 76.24 GB and generates around 21.8 tokens per second.

What is the best local model for an H100 PCIe 80GB?

Laguna S 2.1 at Q5_K_M is the strongest all-round fit: 78.75 GB used, around 315.3 tokens per second at 8k context.

H100 PCIe 80GB: which AI models it runs

The short answer

Assuming an 8k context window and default settings, these are the models worth downloading first.

Best all-rounder

Laguna S 2.1

Q5_K_M · 78.75 GB · about 315.3 tokens/s

Best for code

Laguna S 2.1

Q5_K_M · 78.75 GB · about 315.3 tokens/s

Largest that still runs well

DeepSeek V4 Flash DSpark

165.27B parameters · Q3_K_M · about 176.8 tokens/s

Fastest useful answer

PowerMoE 3B

about 1196.4 tokens/s · 4.53 GB

See how fast it feels

H100 PCIe 80GB running Laguna S 2.1 at Q5_K_M

Wait for the first word27 ms

Then writes at315.3 tok/s

Whole answer576 ms

YouWhy does my model use more memory when the conversation gets longer?

Model

Because of the KV cache. Every token you send leaves behind a key and a value vector in each layer of the model, and those stay in memory for as long as the conversation lasts.

The weights are a fixed cost: load a 4-bit 8B model and that is about 4.8 GB, whether you write one word or ten thousand. The cache is the part that grows, and it grows in a straight line with the number of tokens in the window.

How steeply depends on the model's attention design. With grouped-query attention, several query heads share one key-value pair, which cuts the cache by that ratio. Without it, every head keeps its own, and a long context can cost more memory than the weights themselves.

If you are short on memory, the first thing to try is lowering the context window in your runtime, not the quantisation.

Simulated from our estimate at a 512-token question, not a recording. Assumes nothing else is competing for the GPU.

Every model, scored on this device

Each row uses the highest-quality quantisation that both fits and stays conversational. Speed is a single-stream estimate at 8k context.

Model	Params	Verdict	Download	Memory used	Speed	Max context
Laguna S 2.1 Laguna	117.56B (7.71B active)	Just fits	Q5_K_M	78.75 GB	315.3 t/s	16k
GPT OSS 120B GPT-OSS	116.83B (5.7B active)	Runs great	Q5_K_M	78.17 GB	433.3 t/s	32k
GLM 4.5 Air GLM	110.47B (13.4B active)	Runs great	Q5_K_M	75.46 GB	159 t/s	16k
Qwen3 Next 80B A3B Instruct Qwen	81.32B (3.19B active)	Runs great	Q6_K	63.58 GB	514.7 t/s	128k
Qwen2.5 VL 72B Instruct Qwen	73.41B	Runs well	Q8_0	76.24 GB	21.8 t/s	8k
Qwen3 Coder Next Qwen	79.67B (3.19B active)	Runs great	Q6_K	62.32 GB	514.7 t/s	128k
Qwen2.5 72B Instruct Qwen	72.71B	Runs well	Q8_0	75.55 GB	22 t/s	16k
Qwen3.5 122B A10B Qwen	125.09B (8.17B active)	Runs great	Q4_K_M	71.88 GB	306.9 t/s	64k
Llama 3.3 70B Instruct Llama	70.55B	Runs well	Q8_0	73.41 GB	22.7 t/s	16k
DeepSeek V4 Flash 0731 Spark DeepSeek	60.31B (20.36B active)	Runs great	Q8_0	60.54 GB	81.4 t/s	256k
Llama 3 3 Nemotron Super 49B V1 Llama	49.87B	Runs well	Q8_0	70.45 GB	23.6 t/s	8k
Mixtral 8x7B Instruct V0.1 Mistral	46.7B (12.88B active)	Runs great	Q8_0	48.06 GB	119.3 t/s	32k
DeepSeek V4 Flash DSpark DeepSeek	165.27B (20.36B active)	Runs great	Q3_K_M	76.09 GB	176.8 t/s	64k
Phi 3.5 MoE Instruct Phi	41.87B (6.64B active)	Runs great	Q8_0	43.28 GB	216.6 t/s	128k
Qwen3.5 35B A3B Qwen	35.95B (2.9B active)	Runs great	Q8_0	36.93 GB	469.3 t/s	256k
Qwen3 32B Qwen	32.76B	Runs great	Q8_0	35.33 GB	47.7 t/s	32k
Qwen2.5 32B Instruct Qwen	32.76B	Runs great	Q8_0	35.33 GB	47.7 t/s	32k
OTel 2.0 LLM 31B Instruct Other	32.11B	Runs great	Q8_0	33.64 GB	50.1 t/s	256k
Gemma 4 31B Instruct Gemma	31.27B	Runs great	Q8_0	32.81 GB	51.4 t/s	256k
GLM 4.7 Flash GLM	31.22B (3.66B active)	Runs great	Q8_0	32.03 GB	406.5 t/s	128k
Qwen3 30B A3B Qwen	30.53B (3.34B active)	Runs great	Q8_0	31.69 GB	404.4 t/s	32k
Granite 4.1 30B Granite	28.87B	Runs great	Q8_0	31.42 GB	53.7 t/s	128k
Step 3.5 Flash Step	199.38B	Runs well	IQ3_XXS !	72.58 GB	22.9 t/s	128k
Qwen3.5 27B Qwen	27.78B	Runs great	Q8_0	30.4 GB	55.6 t/s	128k
Gemma 3 27B Instruct Gemma	27.43B	Runs great	Q8_0	29.16 GB	58.1 t/s	128k
Gemma 4 26B A4B Instruct Gemma	25.81B	Runs great	Q8_0	26.55 GB	63.6 t/s	256k
Mistral Small 24B Instruct 2501 Mistral	23.57B	Runs great	Q8_0	25.49 GB	66.7 t/s	32k
Codestral 22B V0.1 Mistral	22.25B	Runs great	Q8_0	24.74 GB	69 t/s	32k
GPT OSS 20B GPT-OSS	20.91B (4.18B active)	Runs great	Q8_0	21.47 GB	395.9 t/s	128k
GPT NeoX 20B GPT-NeoX	20.74B	Runs great	Q8_0	29.75 GB	57 t/s	2k
DeepSeek Coder V2 Lite Instruct DeepSeek	15.71B (2.74B active)	Runs great	Q8_0	16.51 GB	556.2 t/s	128k
Qwen2.5 14B Instruct Qwen	14.77B	Runs great	Q8_0	17.03 GB	101.8 t/s	32k
Qwen3 14B Qwen	14.77B	Runs great	Q8_0	16.78 GB	103.4 t/s	32k
Phi 4 Phi	14.66B	Runs great	Q8_0	16.98 GB	102.1 t/s	16k
Qwen1.5 MoE A2.7B Qwen	14.32B (2.69B active)	Runs great	Q8_0	16.4 GB	394.1 t/s	8k
OLMo 2 1124 13B Instruct OLMo	13.72B	Runs great	Q8_0	20.74 GB	82.7 t/s	4k
vLLM Translategemma 12B Instruct Gemma	13.19B	Runs great	Q8_0	14.26 GB	122.1 t/s	128k
Mistral Nemo Instruct 2407 Mistral	12.25B	Runs great	Q8_0	14.29 GB	122.6 t/s	128k
Gemma 3 12B Instruct Gemma	12.19B	Runs great	Q8_0	13.71 GB	127.4 t/s	128k
Gemma 4 12B Instruct Gemma	11.96B	Runs great	Q8_0	13.05 GB	134.3 t/s	256k
Qwen3.5 9B Qwen	9.65B	Runs great	Q8_0	11.4 GB	155.5 t/s	256k
Gemma 2 9B Instruct Gemma	9.24B	Runs great	Q8_0	11.28 GB	156.9 t/s	8k
Granite 4.1 8B Granite	8.79B	Runs great	Q8_0	10.8 GB	164.9 t/s	128k
Fanar 1 9B Instruct Fanar	8.78B	Runs great	Q8_0	10.82 GB	164 t/s	4k
Internlm3 8B Instruct InternLM	8.8B	Runs great	Q8_0	9.93 GB	180.6 t/s	32k
MiMo V2.5 MiMo	310.78B (16.05B active)	Runs great	IQ2_XXS !	75.4 GB	424.1 t/s	128k
MiMo V2 Flash MiMo	309.79B (16.05B active)	Runs great	IQ2_XXS !	75.16 GB	424.1 t/s	128k
LFM2.5 8B A1B Liquid	8.47B (1.57B active)	Runs great	Q8_0	9.48 GB	850.4 t/s	64k
DeepSeek V4 Flash 0731 DeepSeek	304.18B (20.36B active)	Runs great	IQ2_XXS !	73.81 GB	335.2 t/s	128k
Qwen2.5 VL 7B Instruct Qwen	8.29B	Runs great	Q8_0	9.46 GB	189.8 t/s	64k
DeepSeek V4 Flash DeepSeek	290.94B (20.36B active)	Runs great	IQ2_XXS !	70.63 GB	335.2 t/s	256k
Qwen3 8B Qwen	8.19B	Runs great	Q8_0	10.08 GB	177.7 t/s	32k
Granite 3.0 8B Instruct Granite	8.17B	Runs great	Q8_0	10.18 GB	175.7 t/s	4k
T Lite Instruct 2.1 T-Lite	8.19B	Runs great	Q8_0	10.08 GB	177.7 t/s	32k
Llama 3.1 8B Instruct Llama	8.03B	Runs great	Q8_0	9.8 GB	183.3 t/s	128k
Apertus 8B Instruct 2509 Apertus	8.05B	Runs great	Q8_0	9.82 GB	182.9 t/s	64k
Llama 3 Taiwan 8B Instruct Llama	8.03B	Runs great	Q8_0	9.8 GB	183.3 t/s	8k
Gemma 4 E4B Instruct Gemma	8B	Runs great	Q8_0	8.72 GB	206.1 t/s	128k
Qwen1.5 7B Qwen	7.72B	Runs great	Q8_0	12.49 GB	140.9 t/s	32k
Qwen2.5 7B Instruct Qwen	7.62B	Runs great	Q8_0	8.8 GB	205.6 t/s	32k
Qwen3 235B A22B Qwen	235.09B (22.14B active)	Runs great	IQ2_XXS !	58.7 GB	241.9 t/s	32k
MiniMax M2.7 MiniMax	228.69B (10.98B active)	Runs great	IQ2_XXS !	57.57 GB	358.8 t/s	64k
MiniMax M2.5 MiniMax	228.7B (10.98B active)	Runs great	IQ2_XXS !	57.57 GB	358.8 t/s	64k
OLMo 3 7B Instruct OLMo	7.3B	Runs great	Q8_0	10.07 GB	177.8 t/s	64k
Command A Plus 05 2026 Bf16 Command	218.75B (18.52B active)	Runs great	IQ2_XXS !	53.81 GB	331.9 t/s	128k
Mistral 7B Instruct V0.3 Mistral	7.25B	Runs great	Q8_0	9.02 GB	200.6 t/s	32k
Mistral 7B Instruct V0.2 Mistral	7.24B	Runs great	Q8_0	9.01 GB	200.9 t/s	32k
Falcon 7B Falcon	7.22B	Runs great	Q8_0	12.46 GB	141.6 t/s	8k
DeepSeek Coder 7B Instruct V1.5 DeepSeek	6.91B	Runs great	Q8_0	11.44 GB	154.9 t/s	4k
OLMoE 1B 7B 0125 Instruct OLMo	6.92B (1.28B active)	Runs great	Q8_0	8.57 GB	723.6 t/s	4k
CodeLlama 7B Llama	6.74B	Runs great	Q8_0	11.52 GB	153.7 t/s	16k
DeepSeek Coder 6.7B Instruct DeepSeek	6.74B	Runs great	Q8_0	11.52 GB	153.7 t/s	16k
Gemma 4 E2B Instruct Gemma	5.12B	Runs great	Q8_0	5.78 GB	322.6 t/s	128k
Qwen3.5 4B Qwen	4.66B	Runs great	Q8_0	6.37 GB	292.3 t/s	256k
Agents A1 4B Other	4.54B	Runs great	Q8_0	6.25 GB	298.6 t/s	256k
Gemma 3 4B Instruct Gemma	4.3B	Runs great	Q8_0	5.31 GB	360.3 t/s	128k
Phi 3 Vision 128k Instruct Phi	4.15B	Runs great	Q8_0	7.89 GB	230.8 t/s	128k
Qwen3 4B Qwen	4.02B	Runs great	Q8_0	5.86 GB	321.4 t/s	32k
Phi 4 Mini Instruct Phi	3.84B	Runs great	Q8_0	5.59 GB	341.7 t/s	128k
Phi 3 Mini 4k Instruct Phi	3.82B	Runs great	Q8_0	5.32 GB	362.1 t/s	4k
PowerLM 3B PowerLM	3.51B	Runs great	Q8_0	7.03 GB	260.9 t/s	4k
Granite 4.1 3B Granite	3.4B	Runs great	Q8_0	4.75 GB	411.1 t/s	128k
PowerMoE 3B PowerLM	3.37B (0.88B active)	Runs great	Q8_0	4.53 GB	1196.4 t/s	4k
Llama 3.2 3B Instruct Llama	3.21B	Runs great	Q8_0	4.84 GB	404.8 t/s	128k
Qwen2.5 3B Instruct Qwen	3.09B	Runs great	Q8_0	4.07 GB	491.2 t/s	32k
SmolLM3 3B Base SmolLM	3.08B	Runs great	Q8_0	4.34 GB	454.3 t/s	64k
Starcoder2 3B StarCoder	3.03B	Runs great	Q8_0	3.9 GB	526.4 t/s	16k
Phi 2 Phi	2.78B	Runs great	Q8_0	6.01 GB	312.3 t/s	2k
LFM2.5 2.6B Liquid	2.7B	Runs great	Q8_0	3.87 GB	522.2 t/s	128k
Gemma 2 2B Instruct Gemma	2.61B	Runs great	Q8_0	3.73 GB	548.7 t/s	8k

Showing the 90 best results of 133. The remaining 43 need more memory than this device has, at any quantisation.

What it will not run

0 of the 133 architectures we track are out of reach here, even at two-bit precision. Another 13 run by splitting layers between the card and system RAM, which works but drops generation to single digits.

If you want more headroom

The next steps up in memory, in order. More memory changes which models load at all; more bandwidth changes how fast they answer, so the two are worth weighing separately.

Device	Memory	Bandwidth	Models that fit
RTX PRO 6000 Blackwell	96 GB	1792 GB/s	121	Check price
Ryzen AI Max+ 395 96GB	96 GB	256 GB/s	117	Check price
NVIDIA DGX Spark 128GB	128 GB	273 GB/s	121	Check price
Ryzen AI Max+ 395 128GB	128 GB	256 GB/s	121	Check price

Price links go to an Amazon search for the model name and are affiliate links: if you buy through one, we earn a commission at no cost to you. We do not take payment for placement, and the ordering above is by memory capacity alone.