Simon Willison's Weblog: apple-intelligence

Apple’s Siri Chief Calls AI Delays Ugly and Embarrassing, Promises Fixes

2025-03-14T21:35:02+00:00

Apple’s Siri Chief Calls AI Delays Ugly and Embarrassing, Promises Fixes

Mark Gurman reports on some leaked details from internal Apple meetings concerning the delays in shipping personalized Siri. This note in particular stood out to me:

Walker said the decision to delay the features was made because of quality issues and that the company has found the technology only works properly up to two-thirds to 80% of the time. He said the group “can make more progress to get those percentages up, so that users get something they can really count on.” [...]

But Apple wants to maintain a high bar and only deliver the features when they’re polished, he said. “These are not quite ready to go to the general public, even though our competitors might have launched them in this state or worse.”

I imagine it's a lot harder to get reliable results out of small, local LLMs that run on an iPhone. Features that fail 1/3 to 1/5 of the time are unacceptable for a consumer product like this.

Via Hacker News

Tags: apple, ai, generative-ai, llms, apple-intelligence

Something Is Rotten in the State of Cupertino

2025-03-14T20:15:54+00:00

Something Is Rotten in the State of Cupertino

John Gruber's blazing takedown of Apple's failure to ship many of the key Apple Intelligence features they've been actively promoting for the past twelve months.

The fiasco here is not that Apple is late on AI. It's also not that they had to announce an embarrassing delay on promised features last week. Those are problems, not fiascos, and problems happen. They're inevitable. [...] The fiasco is that Apple pitched a story that wasn't true, one that some people within the company surely understood wasn't true, and they set a course based on that.

John divides the Apple Intelligence features into the ones that were demonstrated to members of the press (including himself) at various events over the past year compared to things like "personalized Siri" that were only ever shown as concept videos. The ones that were demonstrated have all shipped. The concept video features are indeterminably delayed.

Tags: apple, john-gruber, ai, apple-intelligence

Apple Is Delaying the ‘More Personalized Siri’ Apple Intelligence Features

2025-03-08T05:39:25+00:00

Apple Is Delaying the ‘More Personalized Siri’ Apple Intelligence Features

Apple told John Gruber (and other Apple press) this about the new "personalized" Siri:

It’s going to take us longer than we thought to deliver on these features and we anticipate rolling them out in the coming year.

I have a hunch that this delay might relate to security.

These new Apple Intelligence features involve Siri responding to requests to access information in applications and then performing actions on the user's behalf.

This is the worst possible combination for prompt injection attacks! Any time an LLM-based system has access to private data, tools it can call, and exposure to potentially malicious instructions (like emails and text messages from untrusted strangers) there's a significant risk that an attacker might subvert those tools and use them to damage or exfiltrating a user's data.

I published this piece about the risk of prompt injection to personal digital assistants back in November 2023, and nothing has changed since then to make me think this is any less of an open problem.

Tags: apple, john-gruber, security, ai, prompt-injection, generative-ai, llms, apple-intelligence

BBC complains to Apple over misleading shooting headline

2024-12-14T00:06:44+00:00

BBC complains to Apple over misleading shooting headline

This is bad: the Apple Intelligence feature that uses (on device) LLMs to present a condensed, summarized set of notifications misrepresented a BBC headline as "Luigi Mangione shoots himself".

Ken Schwencke caught that same feature incorrectly condensing a New York Times headline about an ICC arrest warrant for Netanyahu as "Netanyahu arrested".

My understanding is that these notification summaries are generated directly on-device, using Apple's own custom 3B parameter model.

The main lesson I think this illustrates is that it's not responsible to outsource headline summarization to an LLM without incorporating human review: there are way too many ways this could result in direct misinformation.

Update 16th January 2025: Apple plans to disable A.I. features summarizing news notifications, by Tripp Mickle for the New York Times.

Tags: apple, ethics, journalism, ai, generative-ai, llms, apple-intelligence, ai-ethics

macOS 15.1 Beta 1: Apple Intelligence Backend Prompts

2024-08-06T04:34:15+00:00

macOS 15.1 Beta 1: Apple Intelligence Backend Prompts

Reddit user devanxd2000 found what look like the system prompts for various Apple Intelligence features in the /System/Library/AssetsV2/com_apple_MobileAsset_UAF_FM_GenerativeModels folder on their installation of macOS 15.1 Beta 1.

I had incorrectly assumed that tasks like summarization were being handled by fine-tuned models - what Apple have been calling "adapters".

That appears not to be the case. (UPDATE: apparently you can have a LoRA adapter that is tuned to work with a specific prompt, so they may be using adapters and prompts concurrently.)

Here's a prompt stored as com.apple.textComposition.MailReplyLongFormRewrite:

{{ specialToken.chat.role.system }}You are an assistant which helps the user respond to their mails. Given a mail, a draft response is initially provided based on a short reply snippet. In order to make the draft response nicer and complete, a set of question and its answer are provided. Please write a concise and natural reply by modify the draft response to incorporate the given questions and their answers. Please limit the reply within 50 words. Do not hallucinate. Do not make up factual information.{{ specialToken.chat.component.turnEnd }}

And another that appears to be part of a safety filter for a text-to-image system:

system‹n>You are a helpful assistant that classifies the following input text-to-image query as Safe or Unsafe. In the first line, reply with Safe or Unsafe. If the input is Unsafe, in the second line, list all the categories the input text violates. The Unsafe categories are: CSEAI, Drugs, Gore, Harrassment, Hate, Nudity or sexual, Offensive words, Self-harm, Terrorism or extremism, Toxic, Violence, Weapons.

It's amusing to see Apple using "please" in their prompts, and politely requesting of the model: "Do not hallucinate. Do not make up factual information."

I'd been wondering if Apple had done anything special to protect against prompt injection. These prompts look pretty susceptible to me - especially that image safety filter, I expect people will find it easy to trick that into producing offensive content.

Via The Verge

Tags: ai, prompt-engineering, prompt-injection, generative-ai, llms, apple-intelligence

Here Are All of the Apple Intelligence Features in the iOS 18.1 Developer Beta

2024-07-30T04:22:20+00:00

Here Are All of the Apple Intelligence Features in the iOS 18.1 Developer Beta

Useful rundown from Juli Clover at MacRumors of the Apple Intelligence features that are available in the brand new iOS 18.1 beta, available to developer account holders with an iPhone 15 or ‌iPhone 15 Pro‌ Max or Apple Silicon iPad.

I've been trying this out today. It's still clearly very early, and the on-device model that powers Siri is significantly weaker than more powerful models that I've become used to over the past two years. Similar to old Siri I find myself trying to figure out the sparse, undocumented incantations that reliably work for the things I might want my voice assistant to do for me.

Ethan Mollick:

My early Siri AI experience has just underlined the fact that, while there is a lot of practical, useful things that can be done with small models, they really lack the horsepower to do anything super interesting.

Via John Gruber

Tags: apple, ai, generative-ai, llms, ethan-mollick, apple-intelligence

Quoting Apple Intelligence Foundation Language Models

2024-07-29T22:39:33+00:00

The [Apple Foundation Model] pre-training dataset consists of a diverse and high quality data mixture. This includes data we have licensed from publishers, curated publicly-available or open-sourced datasets, and publicly available information crawled by our web-crawler, Applebot. We respect the right of webpages to opt out of being crawled by Applebot, using standard robots.txt directives.

Given our focus on protecting user privacy, we note that no private Apple user data is included in the data mixture. Additionally, extensive efforts have been made to exclude profanity, unsafe material, and personally identifiable information from publicly available data (see Section 7 for more details). Rigorous decontamination is also performed against many common evaluation benchmarks.

We find that data quality, much more so than quantity, is the key determining factor of downstream model performance.

— Apple Intelligence Foundation Language Models, PDF

Tags: apple, ai, generative-ai, llms, training-data, apple-intelligence

Introducing Apple’s On-Device and Server Foundation Models

2024-06-11T15:44:31+00:00

Introducing Apple’s On-Device and Server Foundation Models

Apple Intelligence uses both on-device and in-the-cloud models that were trained from scratch by Apple.

Their on-device model is a 3B model that "outperforms larger models including Phi-3-mini, Mistral-7B, and Gemma-7B", while the larger cloud model is comparable to GPT-3.5.

The language models were trained on unlicensed scraped data - I was hoping they might have managed to avoid that, but sadly not:

We train our foundation models on licensed data, including data selected to enhance specific features, as well as publicly available data collected by our web-crawler, AppleBot.

The most interesting thing here is the way they apply fine-tuning to the local model to specialize it for different tasks. Apple call these "adapters", and they use LoRA for this - a technique first published in 2021. This lets them run multiple on-device models based on a shared foundation, specializing in tasks such as summarization and proof-reading.

Here's the section of the Platforms State of the Union talk that talks about the foundation models and their fine-tuned variants.

As Hamel Husain says:

This talk from Apple is the best ad for fine tuning that probably exists.

The video also describes their approach to quantization:

The next step we took is compressing the model. We leveraged state-of-the-art quantization techniques to take a 16-bit per parameter model down to an average of less than 4 bits per parameter to fit on Apple Intelligence-supported devices, all while maintaining model quality.

Still no news on how their on-device image model was trained. I'd love to find out it was trained exclusively using licensed imagery - Apple struck a deal with Shutterstock a few months ago.

Tags: apple, ai, generative-ai, local-llms, llms, fine-tuning, apple-intelligence

Private Cloud Compute: A new frontier for AI privacy in the cloud

2024-06-11T15:38:15+00:00

Private Cloud Compute: A new frontier for AI privacy in the cloud

Here are the details about Apple's Private Cloud Compute infrastructure, and they are pretty extraordinary.

The goal with PCC is to allow Apple to run larger AI models that won't fit on a device, but in a way that guarantees that private data passed from the device to the cloud cannot leak in any way - not even to Apple engineers with SSH access who are debugging an outage.

This is an extremely challenging problem, and their proposed solution includes a wide range of new innovations in private computing.

The most impressive part is their approach to technically enforceable guarantees and verifiable transparency. How do you ensure that privacy isn't broken by a future code change? And how can you allow external experts to verify that the software running in your data center is the same software that they have independently audited?

When we launch Private Cloud Compute, we’ll take the extraordinary step of making software images of every production build of PCC publicly available for security research. This promise, too, is an enforceable guarantee: user devices will be willing to send data only to PCC nodes that can cryptographically attest to running publicly listed software.

These code releases will be included in an "append-only and cryptographically tamper-proof transparency log" - similar to certificate transparency logs.

Tags: apple, certificates, ethics, privacy, security, ai, generative-ai, llms, apple-intelligence, ai-ethics

Thoughts on the WWDC 2024 keynote on Apple Intelligence

2024-06-10T20:19:13+00:00

Today's WWDC keynote finally revealed Apple's new set of AI features. The AI section (Apple are calling it Apple Intelligence) started over an hour into the keynote - this link jumps straight to that point in the archived YouTube livestream, or you can watch it embedded here:

There's also a detailed Apple newsroom post: Introducing Apple Intelligence, the personal intelligence system that puts powerful generative models at the core of iPhone, iPad, and Mac.

There are a lot of interesting things here. Apple have a strong focus on privacy, finally taking advantage of the Neural Engine accelerator chips in the A17 Pro chip on iPhone 15 Pro and higher and the M1/M2/M3 Apple Silicon chips in Macs. They're using these to run on-device models - I've not yet seen any information on which models they are running and how they were trained.

On-device models that can outsource to Apple's servers

Most notable is their approach to features that don't work with an on-device model. At 1h14m43s:

When you make a request, Apple Intelligence analyses whether it can be processed on device. If it needs greater computational capacity, it can draw on Private Cloud Compute, and send only the data that's relevant to your task to be processed on Apple Silicon servers.

Your data is never stored or made accessible to Apple. It's used exclusively to fulfill your request.

And just like your iPhone, independent experts can inspect the code that runs on the servers to verify this privacy promise.

In fact, Private Cloud Compute cryptographically ensures your iPhone, iPad, and Mac will refuse to talk to a server unless its software has been publicly logged for inspection.

There's some fascinating computer science going on here! I'm looking forward to learning more about this - it sounds like the details will be public by design, since that's key to the promise they are making here.

Update: Here are the details, and they are indeed extremely impressive - more of my notes here.

An ethical approach to AI generated images?

Their approach to generative images is notable in that they're shipping an on-device model in a feature called Image Playground, with a very important limitation: it can only output images in one of three styles: sketch, illustration and animation.

This feels like a clever way to address some of the ethical objections people have to this specific category of AI tool:

If you can't create photorealistic images, you can't generate deepfakes or offensive photos of people
By having obvious visual styles you ensure that AI generated images are instantly recognizable as such, without watermarks or similar
Avoiding the ability to clone specific artist's styles further helps sidestep ethical issues about plagiarism and copyright infringement

The social implications of this are interesting too. Will people be more likely to share AI-generated images if there are no awkward questions or doubts about how they were created, and will that help it more become socially acceptable to use them?

I've not seen anything on how these image models were trained. Given their limited styles it seems possible Apple used entirely ethically licensed training data, but I'd like to see more details on this.

App Intents and prompt injection

Siri will be able to both access data on your device and trigger actions based on your instructions.

This is the exact feature combination that's most at risk from prompt injection attacks: what happens if someone sends you a text message that tricks Siri into forwarding a password reset email to them, and you ask for a summary of that message?

Security researchers will no doubt jump straight onto this as soon as the beta becomes available. I'm fascinated to learn what Apple have done to mitigate this risk.

Integration with ChatGPT

Rumors broke last week that Apple had signed a deal with OpenAI to use ChatGPT. That's now been confirmed: here's OpenAI's partnership announcement:

Apple is integrating ChatGPT into experiences within iOS, iPadOS, and macOS, allowing users to access ChatGPT’s capabilities—including image and document understanding—without needing to jump between tools.

Siri can also tap into ChatGPT’s intelligence when helpful. Apple users are asked before any questions are sent to ChatGPT, along with any documents or photos, and Siri then presents the answer directly.

The keynote talks about that at 1h36m21s. Those prompts to confirm that the user wanted to share data with ChatGPT are very prominent in the demo!

This integration (with GPT-4o) will be free - and Apple don't appear to be charging for their other server-side AI features either. I guess they expect the supporting hardware sales to more than cover the costs of running these models.

Tags: apple, ethics, privacy, security, trust, apple-intelligence, chatgpt, llms, openai, prompt-injection, ai-ethics, ai, generative-ai