A really high-quality interview with a Former $NVIDIA (NVDA....

A really high-quality interview with a Former $NVIDIA(NVDA.US)$ Director on $NVDA's moat and the AI semi-industry. A must read:

1. $NVDA is not just a GPU. The real moat is in writing custom CUDA code to make the models run faster, using both the GPUs and the networking, and combining both. According to him, $NVDA can reduce computation time by almost 50% compared to its competitors. This edge was formed when $NVDA bought Mellanox, a networking company.

2. The reason why $Alphabet-A(GOOGL.US)$ doesn't release TPUs to the public is because outside engineers mostly only know how to code CUDA, and $GOOGL TPUs do not use CUDA.

3. For competitors, it's not just about making a better GPU than $NVDA but about improving the whole ecosystem with software code and networking, which is extremely hard to do.

4. $NVDA has optimized its GPUs and networking so that some computation is done directly on networking devices and then moved to the GPUs, making it much more efficient and less power-consuming. The expert says that when you use both $NVDA GPUs and networking, you get a 20-25% computation time improvement vs. only using GPUs.

5. Because $NVDA has both GPUs and networking with Mellanox, this has also increased their margins significantly. Clients prefer buying both GPUs and networking together.

6. In his view, the way to compete with $NVDA is to target inference workloads like Grok is doing. He also thinks $Advanced Micro Devices(AMD.US)$ has demand right now because $NVDA doesn't have enough supply.

7. He thinks we will gradually move to 90% of the consumption around inference compared to training.