High Bandwidth Memory, The Cutting-Edge Technology Is Driving A Resurgence in The Memory Chip Sector

■ What is HBM? Why is HBM crucial in the AI era?

High Bandwidth Memory (HBM) is an emerging type of computer memory that is designed to provide both high-bandwidth and low power consumption.

In the past, the development of the memory industry was far behind that of processors. Traditional DRAMs have limited bandwidth, which can bottleneck the performance of processors, graphic cards, and other devices. Processors of all kinds have been over-provisioned on computing and under-provisioned on memory bandwidth. The most advanced GPUs are literally starving for memory capacity to drive up the utilization.

Besides, the development of AI requires GPUs to have higher computing power, forcing DRAM to be updated. And then HBM came into being. By increasing bandwidth and expanding memory capacity, larger models and more parameters can be kept closer to the core processors, thereby reducing the latency caused by storage shortage.

Nvidia's H200 launch became a direct catalyst for HBM. H200 fully proves that even without adding more CUDA cores or overclocking, just adding more HBM, and even keeping the existing Hopper architecture unchanged, NVIDIA can still achieve performance improvements equivalent to a generational upgrade of its architecture.

■ HBM is in serious short supply.

From a cost perspective, the average selling price of HBM is at least three times that of DRAM. Driven by ChatGPT and limited by insufficient production capacity, the price of HBM has been rising. HBM3 is now five times more expensive than the highest-performance DRAM. Since HBM is in short supply, major manufacturers are expanding production.

■ Who are the main players?

■ The traditional memory market is also recovering.

With original manufacturers actively reducing production and controlling supply, the traditional market is also recovering.

TrendForce reports indicated a universal price increase for both DRAM and NAND Flash starting in the fourth quarter. DRAM prices, for instance, are projected to see a quarterly surge of about 3-8%.

■ Risk disclosure

Although memory chip prices have stabilized recently, they remain at historically low levels. Whether this upward momentum can be sustained will hinge on the suppliers' steadfastness in maintaining production cuts and the degree of resurgence in actual demand, with the general-purpose server market being a critical determinant.

Besides, the memory industry relies on high capital expenditures and has experienced overcapacity in the past two years. The recent earnings reports showed that major companies' current margins are still recovering.