SEMICONDUCTOR DESIGN
11 · Power & cooling

A rack now draws as much as a neighbourhood

AI racks have gone from ~40 kW to hundreds of kilowatts in three generations. At those levels, the 48–54 V busbars inside today's racks would need thousands of amps, so the industry is moving to 800 V DC distribution, and cooling is moving from cold plates toward channels etched into the silicon.

Presets: GB300 NVL72 (~120–140 kW) · Vera Rubin NVL72 (~180–230 kW, est.) · Kyber NVL576 (~600 kW)

—current at 54 V DC
—current at 800 V DC
—resistive loss at 800 V vs 54 V, same conductor

Rack power by generation

kW per rack, midpoint of reported range · NVIDIA does not publish Rubin rack power
Standard · OCP

Mt Diablo: ±400 V and 800 V

Microsoft started the Diablo 400 specification with Google and Meta (draft v0.5.2 in May 2025, then v0.7.0). It allows ±400 V bipolar or 800 V distribution for 100 kW to 1 MW racks. A native-800 V "Mt Diablo 2.0" and a solid-state-transformer spec (v0.3, July 2026) are in progress with NVIDIA.

NVIDIA roadmap

Sidecars, then row power

Vera Rubin NVL72 still takes 415–480 V AC and converts to 50 V DC in the rack. A hybrid MGX power rack follows in 2H 2026 and a row-level power centre in 2027, ahead of ~600 kW Kyber racks for Rubin Ultra (some reports say 2028).

Cooling · Microsoft 2025

Coolant inside the chip

Microsoft and Corintis etched microfluidic channels into the back of the die. In lab tests on a GPU they report up to 3× the heat removal of cold plates and a 65% lower peak temperature rise. Stacked HBM, SoIC and backside power all concentrate heat, so cooling is becoming a chip-design constraint.

Current I = P / V. Resistive loss scales with I², so at the same conductor 800 V carries the same power with (54/800)² ≈ 0.46% of the loss. Real systems also change conductor sizes and add conversion stages; this is a first-order comparison.

DEEP DIVEThe power path of an AI data centre

Electricity passes through several conversions between the grid and a transistor, and each loses a little energy as heat.

  1. Grid to site: high-voltage transmission (hundreds of kV) is stepped down at a substation to medium voltage (typically 10–35 kV).
  2. Site distribution: transformers step down to low-voltage AC (415–480 V); uninterruptible power supplies and generators provide backup.
  3. Rack: power shelves convert AC to 48–54 V DC on the busbar. The emerging alternative is 800 V DC (or ±400 V) distributed to the rack from solid-state transformers or row-level power centres.
  4. Board: intermediate converters step 48–54 V down to around 12 V or directly to the processor supply.
  5. Package: voltage-regulator modules beside or under the package deliver about 0.7–0.9 V at more than 1,000 A.

With five or more stages each 95–98% efficient, several percent of all energy is lost before it reaches the chip — which is why fewer, higher-voltage stages are attractive.

PUEMeasuring facility efficiency

PUE = total facility energy / IT equipment energy

A power usage effectiveness of 1.0 would mean every watt goes to computing. Efficient hyperscale sites report around 1.1 (Google reports a fleet average near 1.09–1.10); older enterprise data centres are often 1.5 or higher. Liquid cooling helps because pumps use far less energy than moving the equivalent heat with air, and warm-water cooling allows free cooling without chillers in many climates.

LIQUID COOLINGHow a liquid-cooled rack works

Cold platesCopper plates with microchannels sit on each GPU, CPU and switch; coolant flows through them in parallel.
Coolant distribution unitA CDU exchanges heat between the rack's clean secondary loop and the facility water loop, and controls flow and temperature.
Facility loopWarm water (often 30–45 °C) is cooled by dry coolers or cooling towers; higher return temperatures make heat reuse possible.
Residual airMemory, storage and power supplies may still be air-cooled, typically 10–20% of the heat in current racks.

SCALEFrom racks to gigawatt campuses

A 72-GPU rack draws on the order of 120–230 kW depending on generation, and planned Kyber racks around 600 kW. A campus with tens of thousands of such racks needs hundreds of megawatts to gigawatts — comparable to a large power station. Grid connection time, transformer supply and local generation (gas turbines, nuclear agreements, renewables with storage) have become the limiting factors for new AI capacity in many regions, alongside chips.

EFFICIENCYEnergy per useful result

What matters is not watts but energy per token generated or per training run. It improves through better chips (lower precision, more on-chip memory), better software (the techniques on the GPU optimization page) and higher utilisation. An idle or poorly batched accelerator still draws a large fraction of its peak power, so scheduling and batching are energy measures as much as performance measures.