Hardware calculator for AI models
How much GPU memory you need to run an open AI model on your own server: model size, quantization, context. Which GPU fits.
VRAM needed—
Fits—
On a CPU: you need the same amount of RAM, but responses will be several times slower.
Pick hardware for my taskHow it is calculated
- Weights: parameters × bytes per parameter (2 for FP16, 1 for 8-bit, about 0.56 for 4-bit).
- Context: KV cache memory grows with context length and the number of concurrent requests.
- Plus about 10% and 1 GB of engine overhead.
This is a rough estimate (±30%): exact numbers depend on the model architecture, the engine (llama.cpp, vLLM, Ollama) and settings. Before buying hardware I will test the model on your tasks.
Need a model for your task?
An open model can run on your own server: data stays in-house, there is no per-request fee, and the model can be fine-tuned on your documents.
- SelectThe model and size for your task and hardware budget
- DeployOn your server or in a closed network, with an API
- Fine-tuneOn your data, or connect a knowledge base
- IntegrateInto your CRM, ERP, bot, website or team chat