Scope first: this page is a procurement checklist for GPU servers in on-premise or private environments, to help buyers organise conditions and risks. It is not a public LLM deployment, model-training managed service or 7×24 hosted ops service, and it is not a careers page. We do not promise inference performance figures or list specific GPU model stock.

Write the workload before choosing cards

Do not decide the card model first; write the workload down:

  • Purpose: single-machine inference, fine-tuning, or multi-machine training; requirements for bandwidth, memory and interconnect differ substantially.
  • VRAM and batch: model size, acceptable batch and latency targets decide memory capacity and throughput needs.
  • Headroom: how many machines in the short term, and whether a second phase of network and rack capacity is reserved.

If the workload is unclear, fill these in first; model and configuration can follow once the boundaries are clear.

Host, PCIe／power connectors and memory: items to align

When choosing the host and GPU configuration, check item by item:

  • PCIe lanes and generation: card count, lane count and PCIe generation must match; otherwise slots run at half speed.
  • Power connectors and power budget: GPU power connectors (such as a high-power 16-pin type), host PSU and redundancy headroom must align; a high-power card cannot rely on slot support alone.
  • System memory: capacity and bandwidth must match model and data loading; avoid enough VRAM but system memory as the bottleneck.
  • Chassis and cooling headroom: chassis space, fan layout and exhaust direction must sustain multi-card long-load operation.
  • Storage bandwidth: local or backend storage read／write bandwidth matches checkpoint and data loading.

These are alignment items, not performance guarantees; actual results depend on workload, firmware and cooling.

How network and optics connect to existing guides

Multi-machine deployment goes back to east-west traffic and interconnect design: see the AI／GPU high-speed networking primer. DAC, AOC and optic selection is in the compatible optics selection guide and compatible optics procurement page. For single-machine use, general Ethernet outward suffices; once multi-machine interconnect is added, fabric and lossless tuning must be assessed separately.

Rack power and airflow: questions to confirm with the operator

Before gear leaves for the room, confirm with the room／rack operator:

  • available rack power budget and single／dual feed arrangement;
  • supply and return air temperature, hot／cold aisle layout and airflow direction;
  • floor or rack load capacity and installation space for multi-card hosts;
  • if liquid cooling is involved, whether liquid interfaces and conditions are in place (detail in the rack power and cooling primer).

Insufficient power and cooling are the commonest barriers to bringing high-density hosts online; these conditions are not within a device supplier’s unilateral promise.

Extra risks with used or refurbished GPU hosts

Used or refurbished hosts can be considered, but check more than general servers:

  • Firmware and compatibility: whether GPU, motherboard and PCIe firmware versions match;
  • Per-card and per-port testing: no sample check only; display output, compute load and network ports must be actually verified;
  • Licence and support: some feature licences or OEM support may not transfer; verify current vendor terms;
  • Ageing condition: service record of PSU, fans and cooling parts; refurbished／used generally carry 3–6 months’ warranty, per the quote.

FAQ

Q: Will you deploy and host models long-term? A: This page covers equipment procurement matching and a checklist; we do not offer a public LLM deployment or 7×24 hosted ops service.

Q: Can you guarantee the inference speed of a given card? A: No. Inference performance depends on model, batch, driver, firmware, cooling and network; we provide condition alignment and inspection process, not a specific performance figure.

Q: What GPU stock do you have now? A: This page does not list stock or model lists; send workload and quantity, and we reply by actual availability.

Q: How long is the used GPU host warranty? A: Refurbished／used generally 3–6 months; new gear under OEM warranty; specifics follow the quote and terms of service.

Next step

Note: this page is a procurement checklist and risk advisory; it does not guarantee any GPU model, inference performance or availability. Actual results depend on workload, topology, firmware, cooling and room conditions; procurement conditions, warranty and liability follow the quote and the terms of service.