VRAM
Will it fit? Ask the calculator.
Weights + KV cache + runtime overhead, estimated before you spend a dollar. Based on the community heuristic (≈0.58GB per billion params at Q4_K_M).
Quantization is the first lever
Dropping FP16 → Q4_K_M cuts memory to ~29% with minor quality loss. Q8 is near-lossless at ~52%.
Context eats VRAM silently
Long chats and big RAG prompts grow the KV cache. Double the context and watch the number climb.
Leave headroom
Target ≤85% of your VRAM. OS, browser and monitor all borrow from the same card.
Some links are affiliate links. They never change our rankings or the math — they help keep AIVerseWeb independent.