Can an Apple M4 16GB run Gemma 4 31B Instruct?

Q: Can an Apple M4 16GB run Gemma 4 31B Instruct?

Yes, with a setting. It needs 11.2 GB at IQ2_XXS and macOS allows 10.72 GB by default. Raise the limit and it runs at about 7.6 tokens per second.

In practice

It loads, and we would not call that running it. The only quantization that fits the Apple M4 16GB is IQ2_XXS, which compresses the weights far enough that Gemma 4 31B Instruct loses a visible amount of its ability, so we do not recommend it. At Q4_K_M it needs 21.28 GB against the 10.72 GB available.

A smaller model at a proper quantization is a better answer than a larger one squeezed in, and the next section lists the ones that fit.

The numbers

Gemma 4 31B Instruct has 31.27 billion parameters across 60 layers. The Apple M4 16GB has 16 GB of unified memory at 120 GB/s, of which about 10.72 GB is left for a model once the operating system has taken its share.

Quantization	Total needed	Fits	Speed
Q8_0	34.64 GB	no	n/a
Q6_K	27.58 GB	no	n/a
Q5_K_M	24.41 GB	no	n/a
Q4_K_M	21.28 GB	no	n/a
IQ4_XS	19.17 GB	no	n/a
Q3_K_M	17.93 GB	no	n/a
IQ3_XXS degraded	14.84 GB	no	n/a
Q2_K degraded	15.9 GB	no	n/a
IQ2_XXS degraded	11.2 GB	with raised limit	7.6 t/s

Nothing on this ladder fits entirely, including the 2-bit levels we do not recommend.

If you would rather not change the limit

A smaller model from the same family is the straightforward answer. These fit on an Apple M4 16GB at a quantization worth using:

Gemma 3 12B Instruct, 12.19B at Q4_K_M, 8.04 GB, about 10.6 tokens/s
Gemma 4 12B Instruct, 11.96B at Q4_K_M, 8.41 GB, about 10.2 tokens/s
Gemma 2 9B Instruct, 9.24B at Q5_K_M, 8.45 GB, about 10.1 tokens/s

Models of this size are what a second GPU or a large-memory machine is for. The multi-GPU guide shows what 2 cards add, and the Mac and PC comparison shows where unified memory holds a model that no single card can.

What else an Apple M4 16GB runs well

For comparison, these are the other popular models that fit this device at a sound quantization, best pairings first.

Model	Quantization	Memory	Speed
Qwen3.5 9B	Q6_K	8.04 GB	11.8 t/s
Gemma 4 12B Instruct	Q4_K_M	8.41 GB	10.2 t/s
Qwen3 8B	Q6_K	7.76 GB	11.8 t/s
Qwen3 14B	IQ4_XS	8.97 GB	10.0 t/s

The same model on nearby hardware

Memory decides whether Gemma 4 31B Instruct loads and bandwidth decides how quickly it answers. Here is what a step down, a card with the same memory and a step up do with it.

Device	Memory	Bandwidth	Result
GeForce RTX 4070	12 GB	504 GB/s	partial, 2.7 t/s
GeForce RTX 5080	16 GB	960 GB/s	does not fit
GeForce RTX 4090	24 GB	1008 GB/s	Q4_K_M, 35.5 t/s

See how fast it feels

Apple M4 16GB running Gemma 4 31B Instruct at IQ2_XXS

Wait for the first word23 s

Then writes at7.6 tok/s

Whole answer46 s

YouWhy does my model use more memory when the conversation gets longer?

Model

Simulated from our estimate at a 512-token question, not a recording. Assumes nothing else is competing for the GPU.

Questions people ask

How much VRAM does Gemma 4 31B Instruct need on Apple M4 16GB?

At Q4_K_M with an 8k context window it needs 21.28 GB in total: 17.58 GB of weights, 3.28 GB of KV cache and 0.42 GB of runtime overhead. The Apple M4 16GB makes about 10.72 GB available once the operating system has taken its share.

Can I run Gemma 4 31B Instruct on Apple M4 16GB anyway?

Yes, after raising the macOS GPU memory limit, which one sysctl command does. Without it the model needs 11.2 GB and the default limit allows 10.72 GB.

Keep going

Everything the Apple M4 16GB runs, the full breakdown for Gemma 4 31B Instruct, how the numbers are made in the methodology, or a different pairing in the calculator.