VRAM Calculator
LLM VRAM Requirements Calculator
Work out real VRAM requirements for local LLMs: weights by quantization, KV cache with the production correction, fragmentation, and the honest production minimum — not just the on-paper number.
Seeds size + architecture; every field can be overridden.
MoE: total params — all experts load.
Architecture & overheads (from the model card)
2 = FP16 cache, 1 = FP8/INT8 KV.
~3 vLLM-class, ~1 llama.cpp.
Weights
40.2 GB
KV cache ×1.3
1.7 GB
Frag + overheads
15.4 GB
On paper
57.4 GB
Itemized budget — the number most calculators stop at.
Production minimum
77.5 GB
What a live server actually needs under real traffic.
“On paper only” = the itemized budget fits but the production minimum does not — the zone where a model loads fine and then falls over under real traffic.
llama-server -m ./model.gguf -c 4096 -ngl 99 --flash-attnTL;DR
A 70B model at Q4_K_M is ~40 GB of weights — but the honest budget at 4K context is 57 GB on paper and ~77 GB in production, because KV-cache overcorrection, fragmentation, and runtime overhead are real. Most VRAM calculators stop at the first number; deployments fail at the second. Set your model, quant, and context above — the calculator shows both, plus a ready-to-copy serving command.
What fits where (from the Local LLMs guide's hardware tiers)
Updated Jul 25, 2026| VRAM | Comfortable fit |
|---|---|
| 8 GB | Phi-4-Mini, Gemma 3 (1B–4B) |
| 16 GB | Phi-4, Gemma 3 12B, Llama 4 Scout at Q4 |
| 24 GB | Llama 4 Maverick at Q4, Qwen2.5-Coder-32B |
| 32 GB | 70B models at Q4 (tight), Llama 4 Maverick at Q8 |
| 64 GB unified | Qwen3.5-122B at Q4, Mixtral 8x22B |
| 128 GB+ | DeepSeek-R1 at Q4, Kimi K2.5 at Q2–Q4 |
How it works
weights = params × bytes/param (Q4_K_M ≈ 0.57 B/param, calibrated to the guide's 70B → ~40 GB example). KV = 2 × layers × kv_heads × head_dim × context × batch × kv_bytes, then × 1.3 for the production correction (alignment, padding, prefix caching). Fragmentation ≈ 20% of (weights + KV), calibrated to the guide's itemized 70B budget (~10 GB); runtime overhead defaults to ~3 GB.
Production minimum = the itemized budget × 1.35, calibrated to the same worked example (≈59 GB itemized vs ~80 GB stated real minimum) — it absorbs the measured 20–38% KV-cache utilization reality and the headroom a live server needs. These are planning estimates, not guarantees; architecture fields accept your model card's real values.
FAQ
- How much VRAM does a 70B model need?
- More than the file size suggests. At Q4_K_M a 70B model's weights are ~40 GB, but the itemized budget — weights + KV cache (with the ×1.3 production correction) + activations + runtime overhead + allocator fragmentation — lands around 57 GB at 4K context, and the realistic production minimum is ~77 GB. At 32K context it climbs past 120 GB. That gap between "fits on paper" and "survives real traffic" is why 70B deployments OOM on 48 GB cards that the naive math said were fine.
- How is KV cache size calculated?
- KV_cache = 2 × n_layers × n_kv_heads × d_head × seq_len × batch_size × bytes_per_element. The 2 is the K and V tensors; n_kv_heads is the grouped-query (GQA) head count, not the attention head count — modern 70B-class models use 8 KV heads, which is why their caches are far smaller than older MHA models. In production, multiply the formula by ~1.3 for alignment, padding, and prefix-caching overhead.
- Why does my model run out of memory when the math says it fits?
- Because the clean formula is the floor, not the budget. Fragmentation alone wastes double-digit gigabytes at 70B scale, CUDA context takes ~3 GB before a single token, and vLLM research cited in our Local LLMs guide found only ~20–38% of allocated KV-cache memory is actually used in production — the rest disappears into fragmentation and over-allocation. This calculator shows both numbers: the on-paper budget and the production minimum.
- Does quantizing the KV cache reduce VRAM?
- Yes — the KV cache scales linearly with bytes per element, so FP8/INT8 KV halves it versus FP16, independently of weight quantization. At long contexts the KV cache rivals or exceeds the weights themselves, which makes KV quantization the highest-leverage lever for long-context serving. Set 'KV bytes/elem' to 1 in the calculator to see the effect.
- How much VRAM do MoE models need?
- Budget for TOTAL parameters, not active parameters. A mixture-of-experts model activates a few billion parameters per token, but every expert must be resident in VRAM — a '35B-A3B' model computes like 3B and loads like 35B. Enter the total parameter count in this calculator.