Local LLM VRAM Calculator

How is required VRAM calculated?

This version covers standard Dense/GQA text models fully loaded onto one NVIDIA GPU. It approximates weight memory with the GGUF file size, then adds KV cache and the selected runtime reserve. File metadata, alignment and runtime buffers mean disk size and GPU allocation need not match exactly. The reserve is a planning assumption, not a measured constant.

F16 KV cache = 2 × layers × KV heads × head dimension × context tokens × 2 bytes

The first 2 accounts for K and V storage. Context includes both input and output tokens. Doubling context doubles KV cache under this formula. GQA shares KV heads across query heads, so use KV heads, not total attention heads. Where head_dim is explicitly specified, as in Qwen3, use that value rather than deriving it from hidden_size.

Weight quantization and cache precision are separate. This tool fixes KV cache at F16. Quantizing the cache can reduce memory, but block scales, alignment and quality effects prevent an exact estimate from nominal bits alone. Q4_K_M weights also include scale data and tensors at different precisions, so their file size is not simply parameter count multiplied by 4 bits.

Sliding-window models such as Gemma 3 may use less cache than full-context storage, depending on layer layout and runtime allocation. However, llama.cpp also has a full-size SWA cache option, so savings cannot be assumed. MLA, linear attention, hybrid architectures, multiple GPUs, images and audio are outside this formula's scope.

For MoE originals such as DeepSeek-V3 and R1, active parameters per token do not determine the memory needed to load all weights. Consider total weights and placement. llama.cpp can keep some expert weights on the CPU. DeepSeek-R1-Distill models, by contrast, are standard Dense models. Offloading and throughput are separate questions; this estimate does not predict execution speed.

Sources & verification scope

Sources checked on 2026-11-04. Five variants were selected from text model families in Ollama's popularity-sorted library whose public artifacts and structures could be checked. This is not a ranking of the current top five models. Qwen structure comes from official config.json files; Llama 3.2 structure comes from GGUF metadata published by Ollama. Meta's original config could not be read directly because login was required.

• Ollama: context and GPU loading ↗
• Ollama: KV cache, Flash Attention and parallel requests ↗
• llama.cpp: loading, cache and SWA options ↗
• llama.cpp: quantization formats and mixed tensors ↗
• DeepSeek-R1: original and Dense Distill models ↗
• Official Ollama library · sorted by popularity ↗

Calculated estimates, official artifact data and measurements are distinguished. Untested hardware and models are never presented as measured. Not measured means no execution record; unavailable means no verified distribution tag; unsupported means the architecture is outside the calculation scope.