GPU hardware for training and inference.

Start from your model's memory requirement (parameters × 1 GB, plus 20% working room) — not clock speed or a marketing FLOPS number. Pick the cheapest hardware that clears that number. If you're training rather than just running a model, budget 3–4x the memory of the raw weights for gradients and optimizer state. Budget and lead time narrow the choice from there: built-to-order enterprise hardware can take weeks to months.

Inference just runs a fixed model to produce output — it needs the model's weights plus a working buffer, and often fits on one card. Training or fine-tuning updates the weights, which means also holding gradients and optimizer state in memory — commonly 3–4x more, which is why training hardware recommendations lean toward multi-GPU servers even for models that would fit a single card for inference alone.

A model's weights have to fit entirely in GPU memory before the GPU can run any of it — there's no partial fallback to system RAM. If a model needs more memory than a card has, that card is off the table regardless of how fast or expensive it is. This is why every product on this page leads with memory, not clock speed.

Watts measure electricity draw, not compute capability — a higher-watt GPU usually means more raw speed, but also a bigger power bill and more heat to remove from the room. Desktop and workstation cards use ordinary air cooling; multi-GPU servers need datacenter-grade airflow or liquid cooling; full racks like the GB300 NVL72 require a facility built for it, not an office closet.

Buying based on a marketing FLOPS number instead of checking memory fit first. Sizing for inference and then discovering training needs 3–4x more memory. Ordering rack-scale hardware without confirming the facility has the power and cooling to support it. Assuming several desktop cards plugged into one PC over ordinary PCIe is a "cluster" — it isn't, and it performs far worse than real NVLink-connected hardware. Not accounting for built-to-order lead times, which can run months for enterprise-scale systems.