Find out in seconds whether a local AI model will run on your graphics card or Mac. Pick your hardware, the model size and the quantization (Q8_0, Q4_K_M, IQ2 and so on). The calculator estimates the file size, KV cache, total memory, a rough speed in tokens per second, and the best quant that fits fully on your GPU. Everything runs in your browser and nothing is sent anywhere.
Model file
0
weights only
KV cache + runtime
0
at chosen context
Total needed
0
of available
Est. speed
0
tokens per second, rough
Checking
Every quant for this model on your hardware
| Quant | File | Total | Result |
|---|
How the calculator works
- Model file = parameters × bits per weight ÷ 8. An 8B model at Q4_K_M (4.85 bits) is about 4.9 GB. This is within a few percent of real GGUF files.
- KV cache + runtime is a mid-range estimate that grows with context length and model size, plus about 0.5 GB for buffers. Models with grouped-query attention (most modern ones) use less, older designs use more. Quantizing the KV cache to Q8_0 roughly halves it.
- Fits on GPU means the total stays under about 95% of VRAM. On a Mac, the GPU can use roughly 75% of unified memory by default.
- Speed is estimated from memory bandwidth: every token reads the active weights once, so tokens per second ≈ usable bandwidth ÷ active model size. Layers spilled to system RAM run at CPU memory speed, which is why a partial fit is so much slower. Prompt processing, batching and drivers all change real numbers, so treat speed as a ballpark.
- MoE models need memory for all parameters but read only the active ones per token, so they are much faster than their size suggests.
Tips if your model does not fit
- Drop one quant step (Q5_K_M to Q4_K_M, or Q4_K_M to IQ4_XS). Each step saves roughly 10 to 20%.
- Reduce the context length. Many chats never need more than 8K tokens.
- Quantize the KV cache to Q8_0. The quality cost is small and it frees a lot of memory at long context.
- Pick a smaller model, or an MoE model that runs fast even when part of it sits in system RAM.
- Avoid going below Q3 on small models. A smaller model at Q4 usually beats a bigger one at Q2.
New to Q4_K_M, IQ4_XS and friends? Our illustrated guide explains what every quant name means, what breaks first, and which file to download.
Read the guideEstimates only. Real memory use depends on the model's architecture, the runtime (llama.cpp, Ollama, LM Studio, MLX), drivers and other apps using your GPU. Bandwidth figures are manufacturer specs. Leave some headroom.