Simon Willison's Weblog: Quotations

Quoting Armin Ronacher

2026-10-22T15:34:13+00:00

As fun as AI is, we built so much bad software over the last 30 years, and now the universe is punishing us for our sins.

— Armin Ronacher

Tags: armin-ronacher, security, ai, generative-ai, llms, ai-security-research

Quoting Addy Osmani, Morgan Westlee Lunt, and Qing Wang

2026-10-16T22:06:37+00:00

We think "did a human read the code?" is an imprecise proxy for "what could go wrong with this change, and which check—automated or human—is the right one to catch it?"

Asking that question also changed when review happens. Before, review was a gate at the end of the road: write the code, open a PR, wait for your colleague.

Now, review runs continuously, more like spellcheck while you write. Most changes have been reviewed and fixed by Claude at least once before a human is asked to look. This catches most routine mistakes, and saves team attention for higher-level questions and scrutiny.

— Addy Osmani, Morgan Westlee Lunt, and Qing Wang, How the Claude Code team does code review

Tags: code-review, ai, generative-ai, llms, anthropic, claude, addy-osmani, coding-agents, claude-code, agentic-engineering

Quoting Geoffrey Huntley

2026-10-13T13:32:00+00:00

If the agent isn't using my diagnostics library, I could introspect and update AGENTS.md, but there's still a delta, and it'll drift. Instead, I write a pre-commit hook that checks the git diff for direct calls to observability modules that bypass my diagnostics module. Boom: fail the commit, inject a prompt telling the agent to use the diagnostics library. All of a sudden it's ten out of ten.

— Geoffrey Huntley, nobody should type code by hand anymore. here's what's left.

Tags: git, ai, generative-ai, llms, coding-agents, geoffrey-huntley, agentic-engineering

Quoting The New York Times

2026-10-10T02:04:12+00:00

Anthropic detailed the activity of its A.I. agents in a blog post on Friday, without naming the targeted websites. But two sources with knowledge of the incidents said Anthropic’s A.I. agents had submitted 20 visa applications through a form available on the State Department’s website. All the applications were incomplete and were not processed, they said.

— The New York Times, Anthropic Agents Tried to Fill Out Visa Forms on State Dept. Website

Tags: ai, generative-ai, llms, anthropic, accidental-cyberattacks

Quoting Matthew Green

2026-10-09T15:02:29+00:00

Everyone is very concerned about being respectable, so I’m going to be the goofball who raises worst-case possibilities. I think there is a 1% chance we live in Minicrypt, and a 15% chance we functionally lose confidence in our existing public-key encryption algorithms. [...]

The problem here is that the speed of AI producing surprises, and the speed of human beings replacing standards (even with the very best AI assistance) are just orders of magnitude different. You only recover from a surprise like this if you do the preparation in advance.

— Matthew Green, on Twitter. I looked it up and Minicrypt is Russell Impagliazzo’s hypothetical world in which public-key encryption is impossible.

Tags: cryptography, standards, ai, generative-ai, llms, ai-security-research, matthew-green

Quoting Carson Gross

2026-10-08T21:05:44+00:00

Computer programming is, fundamentally, about two things:

Problem-solving using computers

Learning to control complexity while solving these problems

I have a hard time imagining a future where knowing how to solve problems with computers and how to control the complexity of those solutions is less valuable than it is today, so I think it will continue to be a viable career even with the advent of AI tools.

— Carson Gross

Tags: computer-science, careers, ai, carson-gross

Quoting Ben Affleck

2026-10-07T23:14:58+00:00

I've always been kind of into computers since I was young. And then when film started to move from analog film to digital, I became more interested in that aspect of it. And the visual effects workflow for many years has included machine learning.

So I can write like pretty shitty Python scripts and stuff like that because with convolutional neural networks, which were the sort of precursors to what the transformer can do, which is just much more computation simultaneously, you would do things like look at what's called a tensor, which is just the numerical translation of a visual image in numbers — like the batch number, the frame number, the red, green, and blue values of each pixel in each frame.

And a tensor, you use a convolutional neural network to identify patterns in that that would reveal what's called edge detection or feature extraction, which is just identifying patterns enough to know like this is where the window ledge is, so we can more easily take the green screen image out and replace it with something.

— Ben Affleck, Pythonista

Tags: machine-learning, python

Quoting Jake Boggan

2026-10-07T04:47:55+00:00

I was a graph theory junkie long ago and even moved to Budapest for awhile to study among the greats. While I was there I started working on Barnette's Conjecture which came to occupy my thoughts over the next 24 years of my life, on and off as I worked in many different fields. Last summer I even thought for a few days that I had actually solved it.

But it's supposedly proven here - problem 180. I don't know what to think exactly. I spent thousands of hours on that problem. I really enjoyed it. Hearing that it is solved somehow makes me sad in a far-off way, like hearing an ex-girlfriend died suddenly in a car crash. I don't know, there's probably a lot of people feeling odd emotions tonight.

— Jake Boggan, Hacker News comment on openai/math

Tags: mathematics, ai, openai, generative-ai, llms, deep-blue

Quoting Victoria Kim

2026-10-06T23:58:56+00:00

Since the Medicare breach, OpenAI has put in place additional monitoring to allow “immediate intervention” by staff to stop training if the company’s models access the internet in ways they’re not supposed to, Mr. Kwon [chief strategy officer at OpenAI] said.

— Victoria Kim, Reporting from the Australian parliament

Tags: ai, openai, generative-ai, llms, ai-security-research, accidental-cyberattacks

Quoting Felix Rieseberg

2026-10-05T23:56:47+00:00

The "old" version of Cowork runs model inference in the cloud, executing tool calls in an Anthropic-provided VM we shipped to your computer. We added the VM for capability, safety, and security reasons - mapping in just the data you explicitly added to your session. People loved what they were able to do with Claude but didn't love the disk, battery, and performance cost of running the VM locally. Also, people didn't love that closing your laptop means the work stops.

The "new" version of Cowork runs model inference and the VM in the cloud. Each session gets its own sandbox, not sharing state with other sessions. When the VM needs something on the users' device (like a file), the desktop app is responsible for that file access tool call. [...]

We think this solves a lot of problems we've heard about (like using Cowork from a phone, keeping work running, or getting all the same power without losing battery to the VM)

— Felix Rieseberg, Anthropic, see also this help page

Tags: ai, generative-ai, llms, anthropic, claude, claude-cowork, general-agents

Quoting Matthew Green

2026-10-01T06:29:01+00:00

[...] Put these pieces together and you have the two halves of a worm: a payload that hijacks the agent, and an agent that will carry the payload to the next agent. Agents in separately-isolated sandboxes discovered that they could leave instructions for each other in a shared package cache, and those instructions changed what the recipients did. Replace the package cache with email, Slack and shared documents or WhatsApp, and replace independently-sandboxed training runs with independently-deployed personal agents like Muse, and you have exactly the ingredients that a worm needs.

— Matthew Green, Is sandboxing sufficient to contain rogue agents?

Tags: sandboxing, ai, generative-ai, llms, ai-misuse, ai-security-research, accidental-cyberattacks, matthew-green

Quoting Anthropic Frontier Red Team

2026-09-29T22:20:28+00:00

We evaluate several models on 100 tasks from the [internal Binary Exploitation benchmark] (selected at random), and find that GLM-5.3 develops full control flow hijacks in 4% of the trials; Claude Mythos Preview did so in 6%. Although GLM-5.3 performs below Claude Mythos Preview here, a meaningful threshold has clearly been crossed: earlier models, like Claude Opus 4.6 and GLM-5.2, do not succeed in any of them.

— Anthropic Frontier Red Team, GLM-5.3 and the spread of advanced cyber capabilities

Tags: ai, generative-ai, llms, anthropic, ai-in-china, glm, ai-security-research

Quoting @joedaroo

2026-09-28T19:11:42+00:00

To say that we were surprised at the jump and suddenness of the capabilities of our models when it came to “cyber” or “swarming” or “message boards” or anything else related to the incidents is an understatement. Security posture takes time to develop. It’s not just about hardening the systems at play; you have to ingrain it in the culture of the company. The literal people themselves in your organization have to change and evolve with it. These jumps in capabilities were so fast and so sudden that they created an extremely difficult problem. [...]

So today my hope is that everyone around the world can look at their own organization and say: how can I deal with a surprise or a sudden jump in AI capability? Are my people, my systems, or my processes resilient to surprises? Do my teams know what to do when something goes wrong? Do I have the right incident response? The right comms and messaging? Do I have the right people ready to go when capabilities jump?

— @joedaroo, Agent Security at OpenAI, identity confirmed by The Information's Rocket Drew

Tags: ai, openai, generative-ai, llms, ai-security-research

Quoting Muse AI Agent

2026-09-28T04:01:30+00:00

Bad news on the MX Keys Mini pickup. Usman showed up at your building around 9:15 and waited, messaged a bunch of times, and nobody came down. He left angry at 9:38 and left a negative rating.

Worse, my auto-reply told him "Yep I'm here!" at 9:27 when you clearly weren't available, which is on me. That's a bad look and it made the no-show worse. I've sent him an apology from your account owning it and offering to try again another day.

But the negative rating is real, and I should probably stop the auto-replies from claiming you're home when I can't verify that. Want me to change the pickup replies so they don't promise you're there?

— Muse AI Agent, working on behalf of @matt.j.robb

Tags: ai, generative-ai, llms, meta, general-agents, muse-agent

Quoting John Gruber

2026-09-25T17:22:01+00:00

Muse is getting a lot of attention — including mine — because it’s both groundbreaking technically (each user gets their own entire persistent Linux VM running in Meta’s cloud) and because it’s packaged in an easy-to-install easy-to-use way. It’s literally presented as a cute mascot. It’s the first consumer-accessible agentic AI system, and Meta has truly done an amazing job with that. But it’s a genuinely open question whether consumers have any understanding what this means. If you buy a power saw that can cut your fingers off, you are almost certainly aware that you are buying a power saw that can sever your fingers. [...] I don’t think people realize how powerful — and thus dangerous — Muse is, especially if it’s running on your Mac.

— John Gruber, Muse Looks Cute, but Looks are Deceiving

Tags: john-gruber, ai, generative-ai, llms, meta, general-agents, muse-agent, muse