The case to move on from NVDA and AMD (long term)

There are times when I read about GPU makers here and I'm wondering if any of you have ever tried renting GPU by the hour or week or month for AI purposes in the past 2 years? If so have you ever calculated how much you're being charged per FLOP or in general the direction of the price of raw compute? As AI becomes mainstream and inference demand grows, what do you think happens to the basic laws of supply and demand?

I was in the data center business (aka HPC) until end of last year and I can tell you that pricing for Nvidia A100 rental had compressed by 70% over the last 2 years. The only reason you're seeing all these great GPU sales is because of 2 things: AI compute demand exploded, and greedy mofo distributors have been hording GPUs - yeah you heard me.

I used to sell container loads of servers equipped with various GPUs like the A100, and then H100 purchased from the spot market (we do not have the volume to deal with Nvidia directly), and I've seen warehouses full of these GPUs sitting there waiting for the highest bidder. The only way Nvidia (or AMD for that matter) distributors get their discounts and priority is by placing huge orders. While you hear about big news of TSLA or AMZN buying a few billions worth of GPUs from NVDA, significant chunk of their GPU business is distributed by companies you've never heard of and made into complete systems via companies like SMCI.