Step by step, this page works out whether a rack's heat leaves by air or by water, how many times power is cut on its way from the grid to the chip, what PUE fails to measure, why rack distribution voltage is going up, how much electricity and memory 10,000 accelerators consume, and how the cost of one million tokens splits between power and capital.
① Air vs. liquid cooling: what carries a rack's heat away
Specific heat fixed at 1,005 (air) and 4,186 (water) J/(kg·K)
Air mass flow
Air volume flow
Per minute
CFM
Air velocity at rack face
Water mass flow ≈ L/s
Water flow
Volume flow ratio (air ÷ water)
Pipe velocity
The air-cooling limit depends on how air is supplied (raised-floor tiles, hot-aisle containment, or in-row coolers) and on the allowed temperature rise, so sources disagree. The figure therefore does not draw the limit as a single line. It shades the text's reading, "floor-tile supply alone tops out at roughly 30 kW per rack," as a soft band, and draws two numbers from the 2021 ASHRAE TC 9.9 white paper as reference lines. Reaching 40–50 kW takes supplementary cooling, such as the in-row coolers or rear-door heat exchangers the same paper names as alternatives.
Check against the solutions ()
The "Solution" column gives the values from the solution to Exercise 1 in Session 7, Self-study appendix A, and from Section 3 (the quick exercise and the air-cooling limit paragraph). Computed values are rounded to the digits the solution reports before comparison.
Every watt of loss turns into heat that must be cooled again. Better distribution efficiency therefore pays twice: it cuts the electricity bill and cuts the cooling load by the same amount. The heat load of the whole facility equals its grid input. All the power that comes in leaves as heat.
Check against the solutions ()
The "Solution" column gives the values from the solution to Exercise 2 in Session 7, Self-study appendix A (overall efficiency, loss table, design implication), and from Section 2, "Losses multiply at each stage." Computed values are rounded to the digits the solution reports before comparison.
Item
Computed
Solution
③ The PUE trap: same work, worse metric
PUE = (IT + cooling + distribution) / IT · overhead = fixed part + variable part × (ITafter / ITbefore)
Overhead
ITCoolingDistribution
PUE before
PUE after
Total power
Total power reduction
The denominator of PUE is IT power. Make IT more efficient and the denominator shrinks, so the metric gets worse. PUE does not just ignore IT efficiency; it penalizes improving it. It also contains no measure of work done, so at the same PUE a facility producing ten times the tokens looks no different from one that does not.
Check against the solutions ()
The "Solution" column gives the values from the solution to Exercise 3 in Session 7, Self-study appendix A, and its table. Computed values are rounded to the digits the solution reports before comparison.
Item
Computed
Solution
④ Rack distribution voltage: 54 V and 800 V
I = P / V · same conductor: I²R loss ratio = (I1 / I2)² = (V2 / V1)²
54 V is the rack bus voltage in NVIDIA documents; 48 V is the OCP Open Rack V3 nominal value
800 V DC, which NVIDIA says it will move to from 2027
V₁ current
V₂ current
Current ratio
I²R loss ratio (same conductor)
In a May 2025 technical blog post, NVIDIA judged that 48 V-class rack distribution hits its limit once a rack passes 200 kW, and said it would move from 2027 to 800 V DC distribution supporting IT racks of 1 MW and more. According to the same post, feeding a single 1 MW rack at 54 V would take up to 200 kg of copper busbar alone. The post also claims up to 5% better end-to-end power efficiency and up to 30% lower total cost of ownership. These are NVIDIA's claims, not independently verified values. Real designs use a different conductor size at each voltage, so the factor of 219 is a comparison that holds only "if the same conductor were used."
Check against the solutions ()
The "Solution" column gives the values from the end of the solution to Exercise 4 in Session 7, Self-study appendix A ("doing the same calculation at rack level"). This figure covers rack distribution only, so the chip IR drop in the first part of Exercise 4 (0.1 V, 0.9 V, 50 W) is not checked. Computed values are rounded to the digits the solution reports before comparison.
Item
Computed
Solution
⑤ Power and memory of a 10,000-accelerator cluster
IT = accelerators × kW per chip × (1 + overhead), or (accelerators ÷ 8) × system kW · facility = IT × PUE · annual = facility × 8,760 h × utilization · required flow = facility power / (cp ΔT)
IT power boundary
Utilization 90% is assumed
PUE
1.54 is the weighted average across facilities responding to the Uptime 2025 survey, 1.48 is the average in the same survey for facilities built around 2020 or later, and 1.09 is Google's trailing twelve-month average. One is a weighted average of survey answers; the other is a measurement inside a boundary one company set for itself. Sample and boundary both differ.
Memory per chip
Power
Water-source cooling (Q = m cp ΔT)
HBM
Rack layout (4 m² per rack)
8 per rack
72 per rack
"1 kW per accelerator + 30% overhead" is an assumption for practice. The DGX boundaries use the vendor-stated maximum system power, so no 30% overhead is added, and they come out above the assumed 13 MW. If the water stays cold and steady all year, chiller load falls. In exchange come intake and discharge permits and the ecological effect of warmer discharge water.
Check against the solutions ()
The "Solution" column gives the values from the solution to Exercise 5 in Session 7, Self-study appendix A ((a)–(d), "comparing with real systems," and the paragraph applying Session 3 Exercise 1), the water-source cooling paragraph of Section 3, pitfall 2 of Section 4, and the first paragraph of Section 6. Computed values are rounded to the digits the solution reports before comparison.
Item
Computed
Solution
⑥ Power cost and capital cost per million tokens
tok/s ≲ B / model bytes × Bsz × η · power cost = node power × PUE × time × electricity price · capital cost = node price / (years × 8,760 × utilization) × time
Node power
PowerCapital (depreciation)
Effective throughput
Time for 1M tokens
Energy (facility level)
Power cost
Capital cost
Capital ÷ power
Capital cost per calendar hour
Per operating hour (÷ utilization)
The dominant cost is a fixed cost per hour, so utilization is cost. Who bears that risk is set by contract. A customer renting GPU cloud capacity on a take-or-pay commitment pays for the committed capacity whether it is used or not. The cost of idle GPUs then falls on the customer, not the provider. Power works as a constraint rather than a cost. Without a grid connection, accelerators can be bought but not installed.
Check against the solutions ()
The "Solution" column gives the values from the solution to Exercise 6 in Session 7, Self-study appendix A (setup paragraph, table by batch size, share table, conclusion 1). Computed values are rounded to the digits the solution reports before comparison.
Item
Computed
Solution
What this calculation leaves out
Fan power scales with the cube of airflow, and fan noise roughly with the fifth to sixth power of air velocity. The text states only the proportionality, so this figure does not put numbers on the effect.
Even with direct-to-chip liquid cooling, components other than the chip still need air cooling.
DGX system power is the maximum the vendor states for the system boundary. That is why it exceeds the assumed 13 MW.
PUE 1.54 is the weighted average across facilities responding to the Uptime 2025 survey, 1.48 is the average in the same survey for facilities built around 2020 or later, and 1.09 is Google's measurement inside a boundary it set for itself. Before subtracting two values with different samples and boundaries and reading the gap as an efficiency difference, check those differences first.
Throughput is the Session 2 decode upper bound multiplied by bandwidth utilization. Real service cost adds labor, networking, storage, unused capacity and safety margin, and API prices add margin and recovery of development cost. This calculation gives a floor, not a price.