AI Infra Demand Shift - Inference Market Booming For AMD?

As demand for AI services increases, there is a growing need for infrastructure supporting inference workloads.

As we can see that Nvidia has better revenue opportunity in AI training than AI inferencing, the recent stock price decline could signal a shift. The AI infrastructure market is witnessing a significant shift in focus from the training of massive AI models to optimizing their inference—i.e., the process of applying trained models in real-world scenarios.

As the demand for AI-powered services continues to rise, there is a growing need for infrastructure that can support the rapid and efficient processing of data in real-time. This transition is driving substantial investments in specialized hardware and data center architectures designed to handle inference workloads at scale.

While AI training has historically been the most resource-intensive phase of AI development, requiring vast computational power and data, the emphasis is now shifting toward inference. Inference involves using pre-trained AI models to make predictions or decisions based on new data, and it is crucial for real-time applications like recommendation engines, autonomous vehicles, and digital assistants.

In general, GPUs are the preferred hardware for AI training due to their parallel processing capabilities. However, for smaller or less complex inference tasks, energy-intensive GPUs may be overkill, leading to unnecessary power consumption and higher costs.

Investment bank TD Cowen said Nvidia is not the only semiconductor company that will benefit from generative AI.

"AMD is well positioned to grow meaningfully with the AI total addressable market, especially initially on the inference side," TD Cowen analysts said in a report. Other potential beneficiaries for increased inference workloads include chipmakers Broadcom and Advanced Micro Devices.

Oppenheimer has a similar take on the AI inference market. "We are now entering an era in which inferencing infrastructure for generative AI has to be scaled to support every workload across centralized cloud, edge compute, and IoT (Internet of Things) devices," Oppenheimer analysts said in a report.

AMD Ryzen AI process inference models have come at the right time to take advantage of the demand for smaller workload where inference can be started and scale.

"AI inferencing presents a larger monetization opportunity than AI training, we believe, as AI models are put into action, making predictions based on new inputs," Oppenheimer said. "This happens repeatedly across myriad devices and applications, including from voice assistants on smartphones to autonomous vehicles, and health care systems."

Meanwhile, data center operators stand to benefit from AI investments by tech companies and large corporations.

If we looked at how AMD have also participate in the decline brought on by Nvidia, there is a need for AMD to differentiate itself from the semiconductor pack.

Having fallen behind rival Nvidia in the AI hardware market, AMD is accelerating its AI chip development in a bid to close the gap.

While AMD is certainly vying for a slice of the data center training segment, considering Nvidia’s dominance in the space, it is increasingly positioning its range of AI accelerators as a powerful inference solution.

The upcoming MI350 range, which is expected to be available in 2025, will reportedly be 35 times better in inference than AMD’s current generation of AI chips.

On-device AI could further accelerate the shift to novel hardware designs that prioritize cost- and energy-efficiency.

I am confident that AMD could take advantage of its accelerators that can increasingly be found in laptops and home computers as AMD seeks to gain an advantage in the nascent market for AI-optimized PCs.

Appreciate if you could share your thoughts in the comment section whether you think AMD could recover with the demand for AI chips on end user computing devices.

Disclaimer: The analysis and result presented does not recommend or suggest any investing in the said stock. This is purely for Analysis.