Best local LLM for coding: what fits your GPU, and how much context you need

Coding is the use where a local model earns its keep fastest, because the code never has to leave your machine, and also the one where a small mistake in sizing hurts most. A coding assistant reads a lot of text before it writes a line. That makes the context window and the speed of reading it as important as the size of the model, and both are decided by memory.

What coding asks of a model

Three things differ from chat. The prompt is long: a few files, a diff and your question can run to 10,000 tokens before the model writes anything. Reading that is compute-bound, so the card's compute matters here more than for conversation, and a slow prompt reader makes every request feel stalled. The answer has to be exact, which favors a higher quantization over a bigger model squeezed small. And the same model is called many times, by an editor for completions and by an agent for steps, so speed compounds.

That is why the pairings below are ranked by what actually fits at a sound quantization with an 8k window, and why we say window first and model second when we advise anyone.

The strongest code model for each amount of VRAM

These are the code-tuned models in our catalog, ranked the same way as everywhere else on the site: fit first, then quality, then speed.

Device	Memory	Best code model	Memory used	Speed
GeForce RTX 3050 6GB	6 GB	Qwen2.5 Coder 7B Instruct at Q4_K_M	4.81 GB	28 t/s
GeForce RTX 4060	8 GB	Qwen2.5 Coder 7B Instruct at Q6_K	6.23 GB	34.3 t/s
GeForce RTX 3060 12GB	12 GB	Qwen2.5 Coder 14B Instruct at Q4_K_M	9.81 GB	28.3 t/s
GeForce RTX 5060 Ti 16GB	16 GB	Qwen2.5 Coder 14B Instruct at Q6_K	12.63 GB	27.1 t/s
GeForce RTX 3090	24 GB	Qwen3 32B at Q4_K_M	20.42 GB	34.5 t/s
GeForce RTX 5090	32 GB	Qwen3 32B at Q6_K	26.86 GB	49.9 t/s
RTX A6000	48 GB	Qwen3 32B at Q8_0	34.09 GB	16.8 t/s

Two things are worth saying about the table. The catalog's code-tuned models are a subset of what exists, so a general model such as Qwen3 32B can be a better coder than the code-tuned pick on a card where it fits, and the hardware pages list both. And mixture-of-experts coders are the exception to everything above: Qwen3 Coder 30B A3B has the architecture of Qwen3 30B A3B, so on a 24 GB card it runs at about 136 tokens per second, and on a 12 GB card with enough system RAM it runs with its experts offloaded, as the MoE guide explains.

What a repository costs in memory

The window is the part people forget. A coding model that fits at 8k may not fit at 32k, and the difference is the KV cache. This is what the common coders need at both sizes:

Model	Weights	Cache at 8k	Total at 8k	Cache at 32k	Total at 32k
Qwen2.5 Coder 14B Instruct 14.77B	8.30 GB	1.50 GB	9.81 GB	6.00 GB	14.3 GB
Qwen2.5 Coder 32B Instruct 32.76B	18.4 GB	2.00 GB	20.4 GB	8.00 GB	26.4 GB
Qwen3 30B A3B 30.53B, 3.34B active	17.2 GB	0.75 GB	18.1 GB	3.00 GB	20.3 GB

On a 24 GB card the 32B coder reaches about 16k before it runs out, which is enough for a handful of files and not for a whole repository. Three things stretch it. Store the cache at 8 bits, which halves it for a small quality cost, as the KV cache guide describes. Pick the MoE coder, whose cache is smaller for its capability. Or send less: most editors let you choose which files go into the prompt, and a well-chosen 8k beats a careless 32k.

Setting it up

Run a local server and point your editor at it. Ollama listens on port 11434 once it is running, and llama.cpp's llama-server on port 8080 by default. Both offer an OpenAI-compatible endpoint under /v1, which most editor extensions accept as a custom base URL. Set the context length to what you need, since a server that defaults to 4k will silently drop the start of a long file, and check with ollama ps that the whole model is on the GPU. Our Ollama guide covers both settings.

For autocomplete inside the editor, use a small model of 1B to 7B. Completions have to arrive in a fraction of a second, and a model that speed-reads a short prefix beats a larger one that thinks. For chat and for agents, run the biggest model that fits at the window you need, and keep both loaded if memory allows.

How we would choose

Start from your memory. With 6 to 8 GB, run a code-tuned 7B at Q5 or Q6 and keep the window at 8k. With 12 to 16 GB, take a 14B coder at Q5 or Q6, or a MoE coder with experts in RAM if you have 32 GB of it. With 24 GB, a 32B coder at Q4 with an 8-bit cache gives you the best answers a single card can, and the MoE coder gives you the best speed. Above that, the question is mostly how much context you want, and the 24 GB guide and 48 GB guide show what is possible.