Cooling system for a single Nvidia Blackwell Ultra NVL72 rack costs a staggering $50,000 — set to increase to $56,000 with next-generation NVL144 racks

The total value of liquid cooling components in Nvidia's GB300 NVL72 rack-scale AI system is $49,860, according to a recently released report from Morgan Stanley that was obtained by @Jukanlosreve. The cost of the liquid cooling system for the company's next-generation Vera Rubin NVL144 solution is set to increase by a further 17% to $55,710, as next-generation Rubin GPUs and NVLink switches will get more power hungry, the report claims.

The bill of materials (BOM) cost of the cooling system for Nvidia's NVL72 'Oberon' rack — that includes 18 compute trays (each consuming at least 6.6kW* of power, but requiring cooling for 6.2kW*) and 9 switch trays — totals $49,860.

Cooling contents of a compute tray are valued at around $2,260; hence, cooling components for all 18 compute trays cost $40,680. By contrast, a cooling system for an NVSwitch tray costs $1,020, so all cooling components for switch trays are priced at around $9,180. As expected, the most expensive cooling components for AI servers are tailor-made high-performance cold plates for CPUs and GPUs that cost $300 per unit, as well as for NVSwitch ASICs that cost $200 per unit.

Thermals will grow further in the Vera Rubin NVL144 platform that will rely on hotter Vera CPUs, Rubin GPUs (up to 1,800W per unit), and next-generation NVSwitch 6.0 ASICs, bringing the cost of the rack-scale cooling system to $55,710, up 17% compared to the cooling system for the GB300, according to Morgan Stanley.

Morgan Stanley expects the cost of the compute tray's cooling system to increase by 18%, to $2,660 per tray, bringing the total per-rack compute cooling to $47,880. The number of trays remains the same, but each one now includes higher-capacity cold plates that cost $400 per unit. By contrast, the switch tray cooling system is expected to get cheaper: $870 per tray and $7,830 per rack.

As CPUs and GPUs gain performance, they also gain power consumption, so they require more and more advanced cooling solutions. The transition from GB200 NVL72 to GB300 NVL72 (+20% for the cooling system) and from GB300 NVL72 to Vera Rubin NVL144 (+17%) marks this upward trend very noticeably.

Starting with Rubin Ultra GPUs, Nvidia is set to transition to four compute and 16 HBM4E chiplets per package, thus increasing TDP to 3,600W, which will require new cold plates or even the use of immersion or embedded cooling, at least for some use cases. Nvidia itself is preparing an all-new liquid-cooled NVL576 'Kyber' rack-scale solution with 144 GPU packages that will double performance compared to Vera Rubin NVL144 (which carries 72 GPU packages), but at the cost of much higher thermal draw. Exotic cold plates for Nvidia's Rubin Ultra GPUs and other enhancements to their cooling system will likely increase the cost of NVL576's cooling system. How much will it cost is something that remains to be seen, but high-performance cold-plates capable of removing 3.6kW of thermal energy from a GPU package will certainly cost significantly more than $400 per unit.

*Each Blackwell Ultra data center GPU consumes 1,400W, one Grace CPUs consumes 300W, SOCAMM memory consumes 200W per socket. Liquid cooling is used for two CPUs and eight GPUs per tray, whereas memory is equipped with heat spreaders.

Follow Tom's Hardware on Google News, or add us as a preferred source, to get our latest news, analysis, & reviews in your feeds.