How Apple defines on-device AI and future trends — from the perspective of analyzing supported models of Apple Intelligence

$Apple(AAPL.US)$ The iPhone 15 with the A16 chip cannot support Apple Intelligence, but M1-equipped models can. Therefore, it can be concluded that the key to supporting the current Apple Intelligence on-device model is the DRAM spec, rather than computing power (TOPS).

The AI computing power of the M1 is about 11 TOPS, lower than the 17 TOPS of the A16. The A16 has 6GB of DRAM, lower than the 8GB of the M1. Therefore, current Apple Intelligence on-device AI LLM require about 2GB or less of DRAM.

The demand for DRAM can be verified in another way. Apple Intelligence uses an on-device 3B LLM (which should be FP16, as the M1’s NPU/ANE supports FP16 well). After compression (using a mixed 2-bit and 4-bit configuration), approximately 0.7-1.5GB of DRAM needs to be reserved at any time to run the Apple Intelligence on-device LLM.

From the above analysis, the following further thoughts and conclusions can be drawn:

1. To enhance existing on-device applications with AI, at least a on-device 3B LLM must be deployed, and based on that, the DRAM spec is determined by the compression method.
2. $Microsoft(MSFT.US)$ believes the key specification for an AI PC is 40 TOPS of computing power. However, for Apple, integrated with cloud AI (Private Cloud Compute), 11 TOPS of on-device computing power is sufficient to start providing on-device AI applications.
3. Consumers may find purchasing Microsoft’s AI PC confusing (calculating whether it reaches 40 TOPS before purchase), whereas Apple directly tells consumers which models can support Apple Intelligence. Regardless of whether on-device AI applications meet consumer needs, Apple has a clear selling advantage from the start.
4. In the future, Apple Intelligence’s on-device AI will certainly upgrade (most likely to a 7B LLM), which will require larger DRAM to operate. It is worth watching whether Apple will use this strategy to differentiate between high- and low-end models.
5. Whether the user experience is as good as Apple claims still needs to be observed ( $Alphabet-A(GOOGL.US)$ ’s Gemini, for example, ever made exaggerated claims in its promotions).
6. Samsung S24’s AI capabilities are limited, and Microsoft’s AI PC still confuses consumers. Apple has successfully defined on-device AI (at least consumers are already aware of the rich AI features and selling points of Apple’s AI devices), which will accelerate competitors’ imitation and catch-up, thereby driving rapid growth in the on-device AI-related industries.