LocalLLMChecker

Desktop GPU · NVIDIA

What AI models can a GeForce RTX 3050 6GB run?

With 6 GB to work with, the question is not which model is best but which ones fit at all. The answer is: small ones, and they are better than they used to be. Bandwidth is the weak point at 168 GB/s. Models fit, then generate slowly, because every token means reading the whole active model out of memory.

This is the card behind our own measurements. It draws 70 W and needs no power connector, so it drops into an old office PC, and that is most of its appeal. Its 96-bit memory bus is the limit: 168 GB/s keeps a 7B model near 30 tokens per second, which is fine for chat and slow for anything bigger.

Memory6 GB
Bandwidth168 GB/s
Usable for a model5.2 GB
Runtime backendCUDA
ArchitectureAmpere
Power70 W

The short answer

Assuming an 8k context window and default settings, these are the models worth downloading first.

Best all-rounder

Qwen3.5 9B

IQ4_XS · 4.98 GB · about 27.1 tokens/s

Best for code

Qwen2.5 Coder 7B Instruct

Q4_K_M · 4.81 GB · about 28 tokens/s

Largest that still runs well

Qwen3.5 9B

9.65B parameters · IQ4_XS · about 27.1 tokens/s

Fastest useful answer

LFM2.5 8B A1B

about 73.4 tokens/s · 5.19 GB

See how fast it feels

GeForce RTX 3050 6GB running Qwen3.5 9B at IQ4_XS

Wait for the first word929 ms
Then writes at27.1 tok/s
Whole answer7.3 s

YouWhy does my model use more memory when the conversation gets longer?

Model

Simulated from our estimate at a 512-token question, not a recording. Assumes nothing else is competing for the GPU.

Where it sits

With 6 GB it belongs to the 6 GB tier. The 6 GB guide explains what that tier runs and where it runs out, and the bandwidth guide explains why 168 GB/s decides how fast it answers.

Our indicative price for this card is $150 new, reviewed 2026-09-01. That is 4.0 GB of memory and 112 GB/s of bandwidth for every $100. Prices move, so treat these as a place to start a search and not a quote.

Devices in a similar price band, fastest memory first:

DevicePriceMemoryBandwidthBest all-round pick
GeForce GTX 1080 Ti $150 used 11 GB 484 GB/s Qwen3.5 9B, 40.3 t/s
GeForce RTX 2060 12GB $160 used 12 GB 336 GB/s Gemma 4 12B Instruct, 23.7 t/s
GeForce RTX 3050 8GB $170 used 8 GB 224 GB/s Qwen3.5 9B, 27.4 t/s

Can an RTX 3050 6GB run it?

A direct answer for the models people ask about most, each with the settings we would use.

Qwen2.5 7B InstructLlama 3.1 8B InstructQwen3 8BQwen3.5 9BGemma 4 12B InstructQwen3 14BPhi 4GPT OSS 20BQwen3 30B A3BGPT OSS 120B

Every model, scored on this device

Each row uses the highest-quality quantization that both fits and stays conversational. Speed is a single-stream estimate at 8k context.

Model Params Verdict Download Memory used Speed Max context
Qwen3.5 9B
Qwen 		9.65B Just fits IQ4_XS 4.98 GB 27.1 t/s 8k
Gemma 4 E4B Instruct
Gemma 		8B Just fits Q4_K_M 5 GB 26.6 t/s 16k
Qwen2.5 VL 7B Instruct
Qwen 		8.29B Just fits Q4_K_M 5.18 GB 25.8 t/s 8k
Qwen2.5 7B Instruct
Qwen 		7.62B Just fits Q4_K_M 4.81 GB 28 t/s 8k
Gemma 4 E2B Instruct
Gemma 		5.12B Just fits Q6_K 4.29 GB 31.2 t/s 128k
Qwen3.5 4B
Qwen 		4.66B Just fits Q6_K 4.21 GB 32 t/s 32k
Llama 3.1 8B Instruct
Llama 		8.03B Just fits IQ4_XS 5.12 GB 26.2 t/s 8k
Qwen2.5 Coder 7B Instruct
Qwen 		7.62B Just fits Q4_K_M 4.81 GB 28 t/s 8k
Qwen3 8B
Qwen 		8.19B Just fits Q3_K_M 4.98 GB 27 t/s 8k
Internlm3 8B Instruct
InternLM 		8.8B Just fits IQ4_XS 4.87 GB 27.7 t/s 8k
Mistral 7B Instruct V0.3
Mistral 		7.25B Just fits IQ4_XS 4.93 GB 27.3 t/s 8k
Qwen3 4B
Qwen 		4.02B Just fits Q6_K 4.55 GB 29.5 t/s 8k
Gemma 3 4B Instruct
Gemma 		4.3B Just fits Q8_0 4.91 GB 27.2 t/s 16k
Mistral 7B Instruct V0.2
Mistral 		7.24B Just fits IQ4_XS 4.92 GB 27.4 t/s 8k
LFM2.5 8B A1B
Liquid 		8.47B (1.57B active) Just fits Q4_K_M 5.19 GB 73.4 t/s 8k
Granite 4.1 8B
Granite 		8.79B Fits only at 2 to 3 bits IQ3_XXS ! 4.79 GB 28.2 t/s 8k
Phi 4 Mini Instruct
Phi 		3.84B Just fits Q8_0 5.17 GB 25.8 t/s 8k
Granite 4.1 3B
Granite 		3.4B Just fits Q8_0 4.34 GB 31 t/s 16k
Qwen2.5 3B Instruct
Qwen 		3.09B Runs great Q8_0 3.68 GB 37 t/s 32k
Phi 3 Mini 4k Instruct
Phi 		3.82B Just fits Q8_0 4.8 GB 27.9 t/s 4k
Llama 3.2 3B Instruct
Llama 		3.21B Just fits Q8_0 4.42 GB 30.5 t/s 8k
LFM2.5 2.6B
Liquid 		2.7B Runs great Q8_0 3.14 GB 44.2 t/s 128k
Qwen3.5 2B
Qwen 		2.27B Runs great Q8_0 2.7 GB 52.4 t/s 128k
Gemma 2 2B Instruct
Gemma 		2.61B Runs great Q8_0 3.54 GB 38.7 t/s 8k
Starcoder2 3B
StarCoder 		3.03B Runs great Q8_0 3.34 GB 41.7 t/s 16k
Qwen3 1.7B
Qwen 		2.03B Runs great Q8_0 3.22 GB 42.9 t/s 16k
Gemma 4 12B Instruct
Gemma 		11.96B Fits only at 2 to 3 bits IQ2_XXS ! 4.58 GB 29.6 t/s 8k
Qwen3 14B
Qwen 		14.77B Fits only at 2 to 3 bits IQ2_XXS ! 5.05 GB 26.8 t/s 8k
Gemma 3 12B Instruct
Gemma 		12.19B Fits only at 2 to 3 bits IQ2_XXS ! 4.13 GB 33.1 t/s 16k
DeepSeek R1 Distill Qwen 1.5B
Qwen 		1.78B Runs great Q8_0 2.07 GB 70.7 t/s 64k
Show the other 56 models
Qwen2.5 1.5B Instruct 1.54B Runs great Q8_0 2.06 GB 70.9 t/s 32k
Mistral Nemo Instruct 2407 12.25B Fits only at 2 to 3 bits IQ2_XXS 4.47 GB 30.7 t/s 8k
SmolLM2 1.7B 1.71B Runs great Q8_0 3.53 GB 38.7 t/s 8k
OLMo 2 0425 1B 1.48B Runs great Q8_0 2.6 GB 54.7 t/s 4k
Gemma 2 9B Instruct 9.24B Fits only at 2 to 3 bits IQ2_XXS 4.57 GB 29.5 t/s 8k
Llama 3.2 1B Instruct 1.24B Runs great Q8_0 1.82 GB 83.7 t/s 64k
OLMo 3 7B Instruct 7.3B Fits only at 2 to 3 bits IQ2_XXS 4.56 GB 29.8 t/s 8k
LFM2.5 1.2B Instruct 1.17B Runs great Q8_0 1.46 GB 110.5 t/s 64k
Gemma 3 1B Instruct 1B Runs great Q8_0 1.35 GB 119 t/s 32k
Qwen3.5 0.8B 0.87B Runs great Q8_0 1.28 GB 127.1 t/s 256k
Qwen3 0.6B 0.75B Runs great Q8_0 1.92 GB 76.4 t/s 32k
Qwen2.5 0.5B Instruct 0.49B Runs great Q8_0 0.88 GB 213.6 t/s 32k
LFM2.5 350M 0.35B Runs great Q8_0 0.68 GB 330.8 t/s 64k
DeepSeek Coder V2 Lite Instruct 15.71B Partial offload n/a 9.29 GB 32.7 t/s n/a
Qwen3.5 35B A3B 35.95B Partial offload n/a 20.48 GB 32.2 t/s n/a
Qwen3 Coder Next 79.67B Partial offload n/a 45.22 GB 24.1 t/s n/a
Qwen3 Next 80B A3B Instruct 81.32B Partial offload n/a 46.15 GB 23.9 t/s n/a
GPT OSS 20B 20.91B Partial offload n/a 11.55 GB 21.9 t/s n/a
GLM 4.7 Flash 31.22B Partial offload n/a 18.13 GB 21 t/s n/a
Qwen3 30B A3B 30.53B Partial offload n/a 18.08 GB 20 t/s n/a
Qwen3 Coder 30B A3B Instruct 30.53B Partial offload n/a 18.08 GB 20 t/s n/a
Gemma 4 26B A4B Instruct 25.81B Partial offload n/a 15.7 GB 18.7 t/s n/a
GPT OSS 120B 116.83B Partial offload n/a 59.4 GB 13.7 t/s n/a
Qwen3.5 122B A10B 125.09B Partial offload n/a 62.22 GB 11.1 t/s n/a
Phi 3.5 MoE Instruct 41.87B Partial offload n/a 22.06 GB 10 t/s n/a
DeepSeek Coder 7B Instruct V1.5 6.91B Partial offload n/a 7.81 GB 7.8 t/s n/a
DeepSeek Coder 6.7B Instruct 6.74B Partial offload n/a 8.12 GB 7.1 t/s n/a
MiniMax M2.5 228.7B Partial offload n/a 130.56 GB 5.4 t/s n/a
MiniMax M2.7 228.69B Partial offload n/a 130.55 GB 5.4 t/s n/a
Qwen2.5 14B Instruct 14.77B Partial offload n/a 9.81 GB 5.1 t/s n/a
Phi 4 14.66B Partial offload n/a 9.96 GB 5.1 t/s n/a
Qwen2.5 Coder 14B Instruct 14.77B Partial offload n/a 9.81 GB 5.1 t/s n/a
Mixtral 8x7B Instruct V0.1 46.7B Partial offload n/a 27.59 GB 4.5 t/s n/a
GLM 4.5 Air 110.47B Partial offload n/a 63.61 GB 3.5 t/s n/a
Mistral Small 24B Instruct 2501 23.57B Partial offload n/a 14.57 GB 2.8 t/s n/a
Codestral 22B V0.1 22.25B Partial offload n/a 14.62 GB 2.8 t/s n/a
Qwen3.5 27B 27.78B Partial offload n/a 15.99 GB 2.5 t/s n/a
Gemma 3 27B Instruct 27.43B Partial offload n/a 16.96 GB 2.3 t/s n/a
Qwen3 235B A22B 235.09B Partial offload n/a 133.71 GB 2.2 t/s n/a
MiniMax M3 427.04B Partial offload n/a 240.84 GB 2 t/s n/a
Granite 4.1 30B 28.87B Partial offload n/a 18.64 GB 2 t/s n/a
Qwen3 32B 32.76B Partial offload n/a 20.42 GB 1.8 t/s n/a
Qwen2.5 32B Instruct 32.76B Partial offload n/a 20.42 GB 1.8 t/s n/a
Qwen2.5 Coder 32B Instruct 32.76B Partial offload n/a 20.42 GB 1.8 t/s n/a
Gemma 4 31B Instruct 31.27B Partial offload n/a 21.31 GB 1.7 t/s n/a
GLM 4.5 358.34B Partial offload n/a 204.37 GB 1.4 t/s n/a
Qwen2.5 VL 72B Instruct 73.41B Partial offload n/a 43.62 GB 0.8 t/s n/a
Qwen2.5 72B Instruct 72.71B Partial offload n/a 43.22 GB 0.8 t/s n/a
Llama 3.3 70B Instruct 70.55B Partial offload n/a 42.12 GB 0.8 t/s n/a
Llama 3 3 Nemotron Super 49B V1 49.87B Partial offload n/a 47.99 GB 0.7 t/s n/a
GLM 5.1 753.86B Will not run n/a 424.51 GB 1.4 t/s n/a
Kimi K2.6 1026.88B Will not run n/a 577.78 GB 1.3 t/s n/a
Kimi K2 Instruct 1026.41B Will not run n/a 577.52 GB 1.3 t/s n/a
DeepSeek V3.2 685.4B Will not run n/a 385.91 GB 1.3 t/s n/a
DeepSeek R1 684.53B Will not run n/a 385.42 GB 1.3 t/s n/a
GLM 5.2 753.33B Will not run n/a 424.21 GB 1 t/s n/a

What it will not run

6 of the 86 architectures we track are out of reach here, even at 2-bit precision. Another 37 run by splitting the model between the card and system RAM, which works but drops generation to single digits for most dense models.

If you want more headroom

The next steps up in memory, in order. More memory changes which models load at all; more bandwidth changes how fast they answer, so the two are worth weighing separately.

DeviceMemoryBandwidthModels that fit
GeForce RTX 3070 Ti 8 GB 608 GB/s 49 Check price
Arc A750 8 GB 512 GB/s 49 Check price
GeForce RTX 5060 Ti 8GB 8 GB 448 GB/s 49 Check price
GeForce RTX 5060 8 GB 448 GB/s 49 Check price

Price links go to an Amazon search for the model name and are affiliate links: if you buy through one, we earn a commission at no cost to you. We do not take payment for placement, and the ordering above is by memory capacity alone.

Other devices with 6 GB

Same capacity, different speed. Once a model fits, bandwidth is what separates these.

GeForce GTX 1660 SUPER

336 GB/s · NVIDIA

Questions people ask about an RTX 3050 6GB

How much VRAM does a GeForce RTX 3050 6GB have?

The GeForce RTX 3050 6GB has 6 GB of VRAM, of which roughly 5.2 GB is available to a model once the operating system takes its share.

Can a GeForce RTX 3050 6GB run a 70B model?

Only partially. About 48 of 48 layers fit on the card and the rest run on the CPU, which brings generation down to roughly 24.1 tokens per second.

What is the best local model for a GeForce RTX 3050 6GB?

Qwen3.5 9B at IQ4_XS is the strongest all-round fit: 4.98 GB used, around 27.1 tokens per second at 8k context.