Nvidia has stopped the development of GB200 NVL36*2 (dual-rack 72 GPU version) 🤔

Analyst Ming-Chi Kuo stated, 'Nvidia has discontinued development of GB200 NVL36*2 (dual rack 72 GPU version). The long-term AI and Nvidia trends continue to be strong, but frequent product plan changes may ease short-term trade sentiment' - Medium

In the future, unless there are customized requirements, Nvidia will only provide the single rack version of GB200 NVL72 (NVL72) and will discontinue the dual rack version (NVL36*2). The single rack GB200 NVL36 (NVL36) will maintain the initial development and shipping plans.

This development will not impact the long-term positive trends of AI and Nvidia. However, in the short term, some market participants may begin to question Nvidia's and the supply chain's execution capabilities.
Nvidia's recent frequent changes to the AI server product roadmap reflect their efforts to improve supply chain execution, competitive advantage, and balance customer demand under limited resources (discontinuation of NVL36*2 development is one example). While this shows Nvidia taking a more practical approach to product planning, this transition may confuse some market participants regarding supply chain changes.

Due to the low visibility of Blackwell Server's 2025 product shipping configuration (a few months ago, the market generally believed there were only NVL36, NVL72, NVL36*2), some suppliers such as assembly and cooling solution providers may have a significant impact on the outlook for 2025.
Comparison of the two 72 GPU models: Reasons for selecting NVL72 and reasons for removing NVL36 *2

Development resources are limited. The initial plan was to develop three GB200 projects (NVL36, NVL72, NVL36*2) simultaneously. The development drops starting in mid-November will focus on NVL72 and NVL36*2 (NVL36 theoretically ready for mass production) with both final quality assurance (QA) scheduled to be completed by mid-March 2025. However, due to uncertainties in NVL36 development, developing both 72 GPU versions (NVL72 and NVL36*2) simultaneously is even more challenging.
NVL72 saves space in data centers. If the challenges of sidecar cooling design are appropriately addressed, NVL72, requiring one less rack than NVL36*2, improves space efficiency in data centers.
NVL72 offers improved inference efficiency. NVL72 and NVL36*2 benefit from parallelizable software design, resulting in almost no difference in AI LLM training outcomes. However, in inference processes that are difficult to parallelize or not parallelizable (such as autoregressive models), NVL72 tends to outperform NVL36*2.
Main preferences from customers. Clients like Microsoft prefer NVL72 over NVL36*2.

Fulfillment of public commitments. NVIDIA has consistently promoted the single-rack NVL72 to the public. Prioritizing NVL72 development over NVL36*2 to fulfill such commitments in the presence of resource constraints.
The development of NVL72 is facing unprecedented technical challenges, and the production schedule remains uncertain.

The biggest challenge in NVL72 development is mainly due to the 132 kW Thermal Design Power (TDP) requirement. As a result of this requirement, NVL72 has become the server with the highest power consumption in history. Nvidia and its supply chain require more time to solve unprecedented technical issues.
It is important to note that TDP refers to the average power consumption during continuous operation. If the peak power consumption due to inadequate design (referred to as Electrical Design Power (EDP) at Nvidia) exceeds TDP, two or more sidecars may be necessary. This complicates cooling design, makes manufacturing more challenging, and results in losing the space-saving benefits of NVL72 datacenters.
Another design challenge of sidecars is maintaining stable control within 5-10°C of approaching temperatures, and relaxing this criterion may affect system stability.
It is worth noting that the high power consumption challenges mentioned above are related not only to sidecars but also to all components and system designs.
According to my latest supply chain survey, mass production of NVL72 may be delayed until the second half of 2025 (in contrast to Nvidia's optimistic target of the first half of 2025).