Buying guide
Find hardware for my AI workload
Work backwards from the model you want to run, not from the biggest number on a spec sheet. This guide walks through the three decisions that actually determine what you need: memory, accelerator class, and the system around it.
How much memory you actually need
Model weights have to fit in memory before anything else matters. The rough rule is that memory scales with parameter count, and quantisation roughly halves or quarters the footprint at a small quality cost. Context length, batching and the KV cache then eat into whatever is left.
| Model size | Precision | Memory (estimate) | Typical fit |
|---|---|---|---|
| 1B–3B | FP16 | ~2–6 GB | Small assistants, embeddings, classification. |
| 7B–8B | 4-bit quantised | ~5–7 GB | Comfortable on a single consumer GPU. |
| 13B–14B | 4-bit quantised | ~9–12 GB | Upper end of 12 GB cards, with little context headroom. |
| 30B–34B | 4-bit quantised | ~20–24 GB | A 24 GB accelerator is the practical floor. |
| 70B | 4-bit quantised | ~40–48 GB | Two 24 GB cards, or a single 48 GB accelerator. |
| 70B | 8-bit | ~75–80 GB | Needs multiple accelerators or a large-memory workstation. |
These are estimates, not promises
Choosing between accelerator classes
Dedicated AI accelerators
Built for sustained inference. Highest memory bandwidth per rupee for pure model serving, but they need a host machine with free slots, power and cooling.
Choose this if the machine already exists and you are adding capacity.
Workstations
A complete, validated machine. You pay a premium over components, but you get a single box that boots and runs, with warranty support across the whole system.
Choose this if you want to unbox and start, or if you need a single vendor for support.
Consumer-class GPUs
Excellent value at 12–24 GB. Fine for a single developer or a small internal tool; watch power draw and cooling in a dense build.
Choose this if you are prototyping and want the lowest entry cost.
Edge and mini devices
Unified memory in a small, quiet footprint. Bandwidth is lower than a discrete accelerator, so throughput suffers on large models.
Choose this if noise, size or power draw matter more than raw tokens per second.
Power, cooling and system balance
- Size the PSU for the spikes, not the averageAdd the accelerator’s rated draw to the rest of the system and leave meaningful headroom. Transient spikes are higher than steady-state numbers.
- Cooling is part of the specBlower-style cards behave differently from open-air cards in a multi-GPU chassis. Pick cooling that matches the enclosure you actually have.
- Bandwidth matters as much as capacityTwo cards with the same memory can differ substantially in bandwidth, and bandwidth drives tokens per second on memory-bound inference.
- Do not forget storageModel weights are large. Fast NVMe storage makes load times tolerable, and enough capacity lets you keep several models available.
Next step
Shortlist from the catalogue
Once you know your memory target and class, filter the catalogue by those constraints and compare the candidates side by side.
Still unsure? Tell us what you are trying to run and we will point you at the two or three listings that fit.