Google TPU And Broadcom

Google faced a challenge in the mid-twenty-teens when they needed to scale their AI ambitions but found Nvidia GPUs expensive and hard to find in large quantities. As a result, they created their own custom silicon, the TPU, which is an AI accelerator designed specifically for computationally expensive tasks. TPU v1 was only for internal use, but versions 2-4 are available on Google Cloud, and v5e has been released with cloud customers in mind.

TPUs are slower than GPUs per chip, but much cheaper to buy and run at scale, so Google focuses on throwing many of them at a task. While TPU v4 maxed out at 4k chips, v5e can handle "tens of thousands," which allows for larger models to be trained and deployed. The performance per chip of v5e is about the same or even worse than v4, depending on the workload, but it is far more inexpensive and efficient. Google charges 63% less per chip to rent v5e compared with v4, resulting in significant cost savings for customers over Nvidia H100.

One advantage that Google has over Nvidia is that they have always designed TPUs with pods and entire data centers in mind, whereas Nvidia is new to that. TPU v5e has popular open-source AI frameworks integrated as services, following AWS's lead. However, Google still needs to do more to match Nvidia's software stack if they want to compete.

Broadcom supplies the chips, internal networking "fabric," and networking hardware surrounding the TPUs. Broadcom also has a large software segment, which may get larger pending approval of the VMWare acquisition. The sales of Broadcom's products to Google, including the unannounced TPU v5 hardware, are a tailwind for the company. However, they make a lot of products for weak smartphone and PC markets, and their semiconductor segment is flat YoY without AI.

Overall, TPU v5e has the potential to disrupt Nvidia's market dominance, but Google needs to match Nvidia's software stack. The success of TPU will also benefit Broadcom, but their balance sheet issues and dependence on Google pose some risks. Currently, it is recommended to remain neutral and watch how these companies develop in the future.