One Research document From Apple’s developers, which has been shared with anyone interested in reading the technical details, suggests that Apple has solved the challenges of local processing of AI queries in the Pocket, but there’s more.
This will benefit everyone
The good news is that the company’s researchers have found a way to render LLM models natively on iPhones as well, effectively on older models as well. How did Apple solve this? By using the storage chips found on iPhones, which are typically 128GB and above, in stark contrast to many iPhones that have 4GB and perhaps 6GB of RAM.
Apple just upgraded its latest 15 Pro models to 8GB of RAM, while many Android devices have 12GB and more. So the company is not a fan of stuffing its products with RAM for one reason or another. This is still controversial, however, especially since the MacBook Pro still starts with 8GB of RAM.
Much better performance
Roughly speaking, this technology involves recycling previously used data (“windowing”), which reduces the need for the mobile phone to return to information in Random Access Memory (RAM). The second technology is called “row and column pooling” and is “designed specifically for the sequential data access capabilities of flash memory, which increases the size of the chunks of data read from flash memory.
Both methods make it possible to run LLM models with up to twice the amount of RAM that a mobile phone can use And It greatly improves performance:
“Combining these approaches allows AI models to run on up to twice the available memory of an iPhone, according to the paper. This means a 4 to 5 times speedup on CPUs, and 20 to 25 times better performance on GPUs.” .
Apple’s AI framework is also called Ajax, which we reported on in July:
Through the framework internally called “Ajax” (since 2022), the company is building its own “Apple GPT” AI. We’ve long wondered when Siri will get a proper, much-needed boost, so it may seem like it will happen next year – perhaps with iOS 18?
In fact, Apple has already leveraged Ajax to improve AI tasks related to Search, Siri, and Maps. “Ajax is now being used to create large language models that will serve as the basis for an internal tool similar to ChatGPT,” reveals Gorman, who has spoken to people with knowledge of Apple’s secret plans.
Although Apple hasn’t talked about it, we find it almost unbelievable that iOS 18 will launch with Siri. without LLM, then we’ll see if they run a local one. It has already been leaked that Samsung is equipping the new Galaxy S24 series with a new Galaxy AI device. The new models are expected to appear on January 17.
“Web specialist. Lifelong zombie maven. Coffee ninja. Hipster-friendly analyst.”
