GPU Battle: NVIDIA vs AMD

When AMD launched MI300X at its Advancing AI event, during the presentation Lisa Su, the CEO of AMD, and her colleagues showcased the accelerator’s prowess by comparing it with NVIDIA H100’s inference performance using Llama 2. At the presentation, a single server of AMD which consists of eight MI300X, performs 1.6 times faster than the server of an H100.

But NVIDIA wasn’t happy with the comparison and debunked it. According to a blog post by NVIDIA, contrary to AMD’s presentation, the company contended that the H100 GPU, when benchmarked appropriately with optimised software, outpaces the MI300X by a substantial margin.

NVIDIA countered this by alleging that AMD failed to incorporate its optimisations in the comparison with TensorRT-LLM. NVIDIA’s response involved pitting a single H100 against eight-way H100 GPUs while running the Llama 2 70B chat model. $Advanced Micro Devices (AMD.US)$ $NVIDIA (NVDA.US)$