Running Flux and SDXL on a low-VRAM card: 6, 8 and 12 GB

Most advice about image generation assumes a 24 GB card. Most people have 6 to 12 GB, and a good share of the setups that look impossible on paper work fine with a few settings changed. The trick is knowing which setting does what, and what each costs you, so that you do not turn on everything and then wonder why it is slow.

What each tier can do

The table below uses the same engine as the image calculator. For each card it tries the highest precision and the least offload that fits, and shows the estimated time for one image at the model's native size. SDXL and its Turbo variant are the easy case. Z-Image Turbo is a newer 6B model that is built to be fast. The Flux variants are the stress test.

Device	Memory	Stable Diffusion XL	SDXL Turbo	Z-Image Turbo	FLUX.1 schnell	FLUX.1 dev
GeForce RTX 3050 6GB	6 GB	46 s GGUF Q8_0, encoder on CPU	400 ms GGUF Q8_0	1 min 3 s GGUF Q4_K_M, encoder on CPU	5 min 45 s BF16, sequential offload	40 min 14 s BF16, sequential offload
GeForce RTX 4060	8 GB	20 s GGUF Q8_0	200 ms GGUF Q8_0	29 s FP8, encoder on CPU	2 min 38 s BF16, sequential offload	18 min 28 s BF16, sequential offload
GeForce RTX 3060 12GB	12 GB	24 s BF16	300 ms BF16	33 s GGUF Q5_K_M	33 s GGUF Q5_K_M, encoder on CPU	3 min 52 s GGUF Q5_K_M, encoder on CPU
GeForce RTX 4060 Ti 16GB	16 GB	14 s BF16	200 ms BF16	19 s GGUF Q8_0	19 s GGUF Q5_K_M	2 min 8 s GGUF Q5_K_M

The summary we would give from it. With 6 GB, stay with SDXL and its relatives, and treat anything bigger as a patient experiment. With 8 GB, SDXL is fast, and Flux is possible with a quantized denoiser and the text encoder offloaded. With 12 GB, Flux at 8 or 5 bits is a real option, though schnell is the one you will want if you generate more than a few images. With 16 GB, most things run, and the question becomes speed. The exact numbers are estimates and the pattern is the part to trust.

The settings, by tool

In diffusers, three calls do most of the work. Model offload keeps one component on the GPU at a time and is the first one to try. Sequential offload streams the denoiser layer by layer from system RAM, so it fits almost anything and is many times slower. VAE tiling decodes the image in pieces so the last step does not blow the memory.

pipe.enable_model_cpu_offload()      # one component on the GPU at a time
# pipe.enable_sequential_cpu_offload()  # last resort, much slower
pipe.vae.enable_tiling()             # decode large images in tiles

In ComfyUI the same behaviors are mostly automatic, and the flags --lowvram and --novram push the memory management harder when the default is not enough. For Flux, load a GGUF or FP8 version of the denoiser, and use a separate loader for the T5 encoder so you can pick a smaller one. In AUTOMATIC1111 and its forks, --medvram and --lowvram do the equivalent, and --medvram-sdxl applies the middle setting to SDXL only, which is the one most people with 8 GB want.

Precision is where the biggest saving sits. FP8 halves a denoiser and is fast on recent NVIDIA cards, and an older card either converts on the fly or loses the speedup. GGUF quantizations of Flux, from Q8 down to Q4, work on any card with a loader that reads them, and Q8 is nearly indistinguishable from the original. Here is what the components of Flux add up to at each precision, so you can see which of them to shrink:

Part of FLUX.1 dev	Parameters	BF16	FP8	4-bit
transformer	11.90B	22.2 GB	11.1 GB	6.69 GB
text encoder	0.12B	0.23 GB	0.11 GB	0.07 GB
text encoder 2	4.76B	8.87 GB	4.43 GB	2.68 GB
vae	0.08B	0.16 GB	0.16 GB	0.16 GB
All parts	16.87B	31.4 GB	15.8 GB	9.60 GB

What you give up

Each saving has a cost, and they are not equal. Dropping the denoiser to Q8 or FP8 costs almost nothing visible. Q4 and NF4 soften fine texture, such as hair, text and skin, and some prompts lose crispness. Offloading the text encoder to the CPU costs a few seconds and nothing in quality. Sequential offload costs minutes. Smaller output sizes cost resolution and not much else, and you can upscale afterwards with a small model that needs far less memory than generating large in the first place.

LoRAs and ControlNets sit on top of the base model. A LoRA adds from under half a gigabyte up to over a gigabyte, depending on its rank, and a ControlNet is its own network on the card. On 8 GB, add one extra at a time and keep an eye on the decode step.

A path that works

Start with the model that fits without tricks and learn its settings. If you want Flux and have 12 GB, take schnell with an 8-bit denoiser and the encoder offloaded. If you have 8 GB, take the 4-bit version, tiled decode and one image at a time, and accept a minute or two. If you have 6 GB, run SDXL and enjoy the speed, and consider whether the models you actually want exist at that size. Then check the arrangement against the image calculator before a long download, since it tells you which precision and offload mode the numbers assume.