Apple’s Improved AI Models Fail To Deliver Strong Performance

Apple has unveiled updates to the AI models behind its Apple Intelligence features across iOS, macOS, and other platforms. However, the company’s own benchmarks reveal that these models lag behind older offerings from competitors like OpenAI.

In a blog post Monday, Apple stated that human evaluators found the text quality produced by its new “Apple On-Device” model — which operates offline on devices like the iPhone — to be on par with, but not superior to, similarly sized models from Google and Alibaba. Meanwhile, testers ranked Apple’s more powerful “Apple Server” model, which runs in Apple’s data centers, behind OpenAI’s year-old GPT-4o.

In a separate image analysis test, human raters favored Meta’s Llama 4 Scout model over Apple Server, according to Apple — a surprising result given that Llama 4 Scout often trails behind top models from Google, Anthropic, and OpenAI in various benchmarks.

Apple’s AI Struggles

The benchmark results support claims that Apple’s AI research division has struggled to keep pace with competitors in the highly competitive AI landscape. In recent years, Apple’s AI efforts have been seen as underwhelming, with a long-promised Siri upgrade still indefinitely delayed. Some customers have even filed lawsuits, alleging that Apple promoted AI features that have yet to materialize.

The Apple On-Device model, which has about 3 billion parameters, powers functions like summarization and text analysis. (Parameters are a measure of a model’s ability to solve problems—more parameters typically mean better performance.) As of Monday, third-party developers can access the model through Apple’s Foundation Models framework.

Apple claims that both Apple On-Device and Apple Server models offer enhanced tool usage and greater efficiency over previous versions, with support for approximately 15 languages. This improvement is partly due to a broader training dataset that now includes images, PDFs, documents, manuscripts, infographics, tables, and charts.

Read the origional article on: TechCrunch

Read more: Tech Tip: These Applications Are Now Able To Sidestep Apple’s In-App Payment System