--- title: "LLM VRAM Calculator: Will This Model Fit on Your GPU?" url: "https://www.geeklord.com/tools/llm-vram-calculator/" author: "Shobhit Kumar Prabhakar" date: "2026-11-05" modified: "2026-11-05" reading_time_minutes: 4 word_count: 693 schema_type: "WebPage" --- # LLM VRAM Calculator: Will This Model Fit on Your GPU? Find out in seconds whether a local AI model will run on your graphics card or Mac. Pick your hardware, the model size and the quantization (Q8_0, Q4_K_M, IQ2 and so on). The calculator estimates the file size, KV cache, total memory, a rough speed in tokens per second, and the best quant that fits fully on your GPU. Everything runs in your browser and nothing is sent anywhere. Your hardware GTX 1650 (4 GB)GTX 1660 Super (6 GB)RTX 2060 (6 GB)RTX 2060 (12 GB)RTX 3050 (6 GB)RTX 3050 (8 GB)RTX 3060 (12 GB)RTX 3060 Ti (8 GB)RTX 3070 (8 GB)RTX 3080 (10 GB)RTX 3080 (12 GB)RTX 3090 (24 GB)RTX 4060 (8 GB)RTX 4060 Ti (8 GB)RTX 4060 Ti (16 GB)RTX 4070 (12 GB)RTX 4070 Super (12 GB)RTX 4070 Ti Super (16 GB)RTX 4080 / 4080 Super (16 GB)RTX 4090 (24 GB)RTX 5060 (8 GB)RTX 5060 Ti (16 GB)RTX 5070 (12 GB)RTX 5070 Ti (16 GB)RTX 5080 (16 GB)RTX 5090 (32 GB) RTX 3050 Laptop (4 GB)RTX 4050 Laptop (6 GB)RTX 4060 Laptop (8 GB)RTX 4070 Laptop (8 GB)RTX 4080 Laptop (12 GB)RTX 4090 Laptop (16 GB) RTX A6000 (48 GB)RTX 6000 Ada (48 GB)A100 (80 GB)H100 (80 GB) RX 6600 (8 GB)RX 6700 XT (12 GB)RX 6800 (16 GB)RX 7600 (8 GB)RX 7600 XT (16 GB)RX 7800 XT (16 GB)RX 7900 XT (20 GB)RX 7900 XTX (24 GB)RX 9070 XT (16 GB) Arc A770 (16 GB)Arc B580 (12 GB) Mac, M1/M2/M3 base (8 GB)Mac, M-series base (16 GB)Mac, M-series base (24 GB)Mac, M3 Pro (36 GB)Mac, M4 Pro (48 GB)Mac, M4 Max (64 GB)Mac, M4 Max (128 GB)Mac, M2 Ultra (192 GB) CPU only (no GPU)Custom GPU VRAM (GB) System RAM 8 GB16 GB32 GB64 GB128 GB Model size 1B3B4B7B8B12B14B20B (gpt-oss-20b)24B27B32B30B MoE (3B active)70B120B MoE (5B active)Custom Total params (B) Active params (B) Quant F16 / BF16 (16)Q8_0 (8.5)Q6_K (6.6)Q5_K_M (5.7)Q4_K_M (4.85)Q4_K_S (4.6)MXFP4 (4.25)IQ4_XS (4.25)Q3_K_M (3.9)Q3_K_S (3.4)Q2_K (3.2)IQ2_XXS (2.4) Context length 8K tokens KV cache F16 (default)Q8_0 (about half)Q4_0 (about a quarter) Model file 0 weights only KV cache + runtime 0 at chosen context Total needed 0 of available Est. speed 0 tokens per second, rough Checking ### Every quant for this model on your hardware | Quant | File | Total | Result | | --- | --- | --- | --- | **Copy link to this setup ## How the calculator works - Model file** = parameters × bits per weight ÷ 8. An 8B model at Q4_K_M (4.85 bits) is about 4.9 GB. This is within a few percent of real GGUF files. - **KV cache + runtime** is a mid-range estimate that grows with context length and model size, plus about 0.5 GB for buffers. Models with grouped-query attention (most modern ones) use less, older designs use more. Quantizing the KV cache to Q8_0 roughly halves it. - **Fits on GPU** means the total stays under about 95% of VRAM. On a Mac, the GPU can use roughly 75% of unified memory by default. - **Speed** is estimated from memory bandwidth: every token reads the active weights once, so tokens per second ≈ usable bandwidth ÷ active model size. Layers spilled to system RAM run at CPU memory speed, which is why a partial fit is so much slower. Prompt processing, batching and drivers all change real numbers, so treat speed as a ballpark. - **MoE models** need memory for all parameters but read only the active ones per token, so they are much faster than their size suggests. ## Tips if your model does not fit 1. Drop one quant step (Q5_K_M to Q4_K_M, or Q4_K_M to IQ4_XS). Each step saves roughly 10 to 20%. 2. Reduce the context length. Many chats never need more than 8K tokens. 3. Quantize the KV cache to Q8_0. The quality cost is small and it frees a lot of memory at long context. 4. Pick a smaller model, or an MoE model that runs fast even when part of it sits in system RAM. 5. Avoid going below Q3 on small models. A smaller model at Q4 usually beats a bigger one at Q2. **New to Q4_K_M, IQ4_XS and friends?** Our illustrated guide explains what every quant name means, what breaks first, and which file to download. [Read the guide](https://www.geeklord.com/gguf-quantization-explained/) Estimates only. Real memory use depends on the model's architecture, the runtime (llama.cpp, Ollama, LM Studio, MLX), drivers and other apps using your GPU. Bandwidth figures are manufacturer specs. Leave some headroom. --- *Published on GeekLord.com - Original URL: https://www.geeklord.com/tools/llm-vram-calculator/*