Choosing hardware
What it costs to run a local LLM: electricity, heat and noise
Nobody buys a graphics card for local models because the electricity is cheap, and almost nobody regrets the bill either. It is worth knowing what it is, because the two numbers that people quote, the card's rated power and the headline price per kilowatt-hour, are both the wrong ones to multiply together.
What a card draws while it generates
A card's rated board power is the ceiling, and how close generation gets to it depends on the card. On our RTX 3050, which is rated at 70 W, the card's own sensor read 69 W when the whole model sat on the GPU, so right at the limit. When part of the model was in system RAM, the card spent time waiting for the CPU and the draw fell to between 26 and 60 W. Larger cards are usually reported to sit below their rating while generating, around 250 to 300 W for a 350 W RTX 3090 and 300 to 350 W for a 450 W RTX 4090, because token generation leaves the compute units partly idle. Treat those two as typical figures, not ours.
The rest of the machine adds a base load of 50 to 100 W for a desktop, depending on the processor and drives. Apple Silicon is the outlier: a whole Mac under generation load draws well under what a desktop card alone does, and that is a large part of its appeal for an always-on assistant.
A calculator for your own case
Set your card, how many hours you generate each day, and what you pay per kilowatt-hour. The default price is around the US residential average, and the idle hours matter more than they seem.
$0 a month, 0 kWh, or $0 for every hour you generate.
Draw while generating is often below a card's rated power, because token generation waits on memory and leaves the compute units partly idle. The figures are typical, rounded values for planning, and our RTX 3050 number comes from the card's own power sensor during the benchmark. A plug-in power meter beats any table.
For a mid-range card used 2 hours a day and left on idle for 6 more, the bill is a few dollars a month, and usually the idle part is as large as the active part. That is the lever people miss. A machine that is left on to be ready at any moment draws its idle power all day, and in many setups sleeping or switching off between sessions saves more than any tuning of the model.
Three typical profiles
The calculator is the right tool for your case, and these three are a sense of scale. They assume a mid-range card drawing 130 W while generating, 70 W for the rest of the machine, 40 W idle, and $0.17 per kWh.
| Profile | Generating | Idle | Per month |
|---|---|---|---|
| Occasional chat | 1 hour a day | 4 hours a day | $1.84 |
| Daily use and coding | 3 hours a day | 8 hours a day | $4.69 |
| A home server left on all day | 4 hours a day | 20 hours a day | $8.16 |
The gap between the first and third rows is mostly idle time, not generation. Sleeping the machine between sessions moves the third row down to the second, and it is the only change on this page that does not cost you any speed.
Heat, noise and the case
Power ends up as heat in your room. A 300 W card running for 2 hours is the heat of a small space heater for the same time, welcome in winter and a nuisance in summer. A card under sustained load also spins its fans, and the noise depends more on the cooler design than on the rating. If the machine sits on your desk, the quietest choice is a card with a big cooler running at a lower power limit.
That brings up the cheapest trick available. Token generation is memory-bound, so reducing the power limit of a card to 70 or 80% costs a few percent of speed and removes a noticeable amount of heat and noise. On NVIDIA cards nvidia-smi -pl sets the limit, and most vendor tools expose the same control. Try it with a benchmark of your own before you trust our number or anyone else's.
Local against an API
On electricity alone a hosted model is cheaper per answer than a local one, because the provider runs the hardware flat out for many users and you run yours for one. The reasons to run locally are different: privacy, working offline, no per-request limits, the freedom to try any model, and the fact that many readers already own a gaming card. If you are deciding whether to buy a card for this, count the card's price over its years of use and not the bill, since the card is the real cost. The buying guide and used GPU guide cover that side.
One last number worth knowing. An hour of non-stop generation on a 300 W card, with the rest of the machine, costs about 6 cents at US prices, and a typical 500-token answer takes well under a hundredth of that. If a "cost per answer" you read looks like more than a fraction of a cent, check what it is multiplying.
Questions people ask
How much electricity does a local LLM use?
A desktop with a mid-range GPU draws roughly 200 to 250 watts while generating and under 50 watts idle. At 2 hours of generation a day and $0.17 per kWh that is a few dollars a month, and the idle hours can cost as much as the active ones.
Does a bigger model use more power?
Only a little. Generation is limited by memory, so the card sits at a similar draw whichever model it runs. A larger model takes longer per answer, so the energy per answer rises, and the draw per second hardly moves.
Is running a model locally cheaper than paying for an API?
Not on electricity alone, because hosted models are cheap per answer and you also pay for the hardware. Local wins when you use it heavily, when privacy decides it, or when the card was bought for another reason.
Can I reduce a GPU's power without losing much speed?
Yes. Because token generation is memory-bound, capping the power limit to 70 or 80% usually costs a few percent of speed and cuts heat and noise noticeably.