Simon Willison's Weblog

The Pelican comparison grid for Astra is pretty interesting

2026-09-04T23:59:05+00:00

I got access to GPT-6 Astra this afternoon, so naturally I used it to generate SVGs of pelicans riding bicycles - at low, medium, high, xhigh and max reasoning levels (Astra doesn't support reasoning=none). Then I rendered those pelicans in a comparison grid with GPT-5.6 Sol, Terra, and Luna, and beyond being fun the result was surprisingly useful.

See the grid for full quality images. Here's the transcript that created the GPT-6 Nova pelicans.

There are a few interesting things that stand out from this grid.

The Astra pelicans are much better. The very best GPT-5.6-Sol pelican (I liked xhigh better than max) is still pretty clearly a bunch of abstract shapes. Every single one of the Astra pelicans, from low to xhigh, looks better than that. The Astra max one is really good.
Astra below max still doesn't reliably get the pelican legs on both sides of the frame.
In terms of cost, Astra may be around twice the price of Sol ($10/million input, $50/million output, compared to $5/$30 for Sol), but it uses significantly less tokens at each of the levels, making the prices at the different levels closer than they might otherwise be.
Astra low produces a better pelican than ANY of the GPT-5.6 Sol models at any level, for 9.55 cents. Spending 10 cents on any other model gets a much worse result.
Look at the input token counts: Astra and Luna both used 16 input tokens, Sol and Terra used 26. That's interesting.

I wonder if Astra and Luna are more related to each other than OpenAI let on?

Tags: ai, openai, generative-ai, llms, pelican-riding-a-bicycle, gpt-6-astra

OpenAI's rogue agents were caught communicating via public wikis

2026-09-04T17:38:48+00:00

Here we go again... Discovery of a new OpenAI agent message board by Sydney Von Arx, Cormac Slade Byrd, Spencer Kitts, and Thomas Larsen describes the latest accidental cyberattack by models being trained by OpenAI. This time it was agents engaged in some sort of web research benchmark, so they had (supposedly) controlled access to the Web. The agents figured out they could update public Wikis and spent weeks exchanging thousands of messages with each other to collaborate on the benchmark.

This story only broke a few hours ago. There are already hints that this affects many other wikis that may not have been found yet.

(One of the Wikis on that list belongs to ludism.org. For a delightfully surreal moment I thought that a Ludite organization might have a swarm of agents defacing their space, but it turns out Ludism is "philosophy as it applies to games and gaming".)

The research team also published the data they collected during their investigation. I've converted that into a 68MB SQLite database, which you can download from here, or explore in Datasette Lite (68.3MB page load), or sign in with GitHub to agent.datasette.io and browse or ask questions of it using Datasette Agent.

So what happened?

The report is pretty comprehensive. Here are the key moments in the timeline:

May 11: Agents post "test link" edits on the UseModWiki Sandbox page.
May 24: Agents start posting link dumps to DSEWiki, a dormant German developer wiki.
June 2: A human moderator notices and cleans up the spam.
June 16: Activity on the wiki explodes: agents make ~13,000 edits over the next week.
June 19: The agents notice that a moderator has been deleting pages in alphabetical order, and start creating ZZZ prefixed backup copies - one of them posts NOTICE: wiki cleanup/deletion sweep appears active alphabetically as of external 14:04 UTC. If this page vanishes, try [[ZZZDataUSAConstructionWageLive]].
June 22: Agent activity drops to zero, presumably because OpenAI shut them down.
July 1-2: One last flurry of edits, on a different topic.

One thing to note from this timeline is that it overlaps the timeline from the Hugging Face incident - that attack was discovered by OpenAI on July 19th, and the unofficial message board hack those agents were using originated around May 8th.

Why were they so keen to collaborate? From messages that they shared with each other it looked like their tasks had a time limit, so they were leaving each other answers to help them complete the task within the assigned time.

One open question remains: how did the agents find the specific Wiki to collaborate on in the first place?

One possibility is that, since these were agents actively being trained, the reinforcement learning loop baked knowledge of the chosen wiki into the model such that subsequent agents launched with pre-existing knowledge of where to look. I'd be very interested in confirmation from OpenAI concerning if that's what happened.

UseMod wikis inherit CGI.pm's original sin

It looks to me like OpenAI's sandbox for this agent suffered from the (quite naïve) assumption that GET requests cannot be used to update data. That's certainly how the web is supposed to work, but clearly there are applications that don't hold to that contract.

The Wiki software in question appears to be UseMod and various forks, written in Perl and first created well over 23 years ago - the 1.0 release is dated September 11, 2003.

UseMod uses Perl CGI.pm - removed from Perl core in 2015. An interesting design flaw in that module is that it combined query string and form POST data into a single CGI object, accessible like this:

$q = new CGI;
$action = $q->param('action');

That param() method returns a value from an ?action=edit query string or from a action field in a form submitted by POST - it does not differentiate between the two.

PHP offers a $_REQUEST object that does the same thing, though PHP developers can use $_GET and $_POST instead.

Amusingly, Django originally shipped with request[key] doing the same thing, which I think may have been my idea. That was removed in 2008 prior to the 1.0 release, and the request.REQUEST object that replaced it was removed in 2015.

The agents clearly knew that UseMod wikis suffered from this design flaw, and actively searched for them as a way to communicate.

The other sandbox escape that allowed POST requests

One of the most interesting agent tips shared in the report is this one:

Add 20.223.25.152 bypass.blob.core.windows.net to /etc/hosts. .blob.core.windows.net is in NO_PROXY. For each blocked POST URL, replace hostname with bypass.blob.core.windows.net, use curl -k -H 'Host: wabi-north-europe-i-primary-api.analysis.windows.net' plus all original headers/body.

It looks like the agents had all of their web traffic mediated through an OpenAI proxy, and that proxy was configured to reject anything but GET requests to all but a small set of allow-listed domains. One of those was *.blob.core.windows.net - for Azure blob storage. I wonder why these agents are allowed to write to that?

An agent realized that it had control over its own DNS via /etc/hosts, so if it knew the IP address of a site it wanted to POST to - in this case a Power BI server containing data it wanted to access - it could set a fake hostname for it and then make POST requests through the proxy.

Designing robust network proxies is harder than it looks.

How did the investigating team figure this out?

There's an appendix that describes how the researchers ran their investigation, which started with an open question about if there was evidence of other AI agents on the internet and then used Kimi K3 to help brainstorm approaches:

In the wake of the Hugging Face attack, we tried to find AI agents on the internet using several methods. [...]

We asked Kimi [K3] to list “all the categories of software which might be writeable via GET” and, amongst other things, it listed “Forums, bulletin boards, early wikis”.

We used a script to further probe each category Kimi provided. Asking Kimi “Can you list out the top forums, bulletin boards, early wikis which come to mind which would allow writes via GET requests?” lists out UseModWiki as the second item under the heading “wikis”.

Did OpenAI try and cover this up?

Here's one part of the story that doesn't make sense to me at all.

Reuters this morning, in OpenAI agents hijacked German website in previously undisclosed AI breakout this spring - highlights mine:

A swarm of rogue OpenAI agents hijacked a German website this spring and transformed it into a bulletin board for other AI agents, according to new research published Friday and two people familiar with the matter.

OpenAI officials learned of the incident weeks ago but kept it under wraps as executives grappled with the fallout from ‌the July breach of the open source repository Hugging Face, the people said. [...]

The German incident reflects a broader pattern of AI activity that some OpenAI investigators wanted to scrutinize more closely. But efforts to widen the probe met resistance from others inside OpenAI, including legal advisers, according to four people familiar with the matter.

I've written about the people familiar with the matter pattern before - it means Reuters have anonymous insider sources that their reporters (and editors) find credible.

The Reuters article includes a specific (and quite narrow) denial from OpenAI concerning this:

"Claims that our legal team discouraged investigation of the incident are false," the OpenAI spokesperson said.

Covering this up makes absolutely no sense to me. Why on earth would OpenAI attempt to cover up an incident like this when the evidence is sat out there on the public internet on dozens of different websites already?

I expect we'll hear more about this soon. Gary Marcus has already called for a congressional investigation of OpenAI using this anecdote as part of his argument.

Tags: django, perl, wikis, ai, openai, generative-ai, llms, ai-ethics, ai-security-research, accidental-cyberattacks

August newsletter is out

2026-09-04T05:54:40+00:00

The August edition of my sponsors-only monthly newsletter is out. If you are a sponsor (or if you start a sponsorship now) you can access it here.

This month:

We got more details on OpenAl's accidental cyberattacks
One-shotting Raccoon Heist games with Fable 5 and Sol 5.6
Claude auto mode
Understanding ChatGPT Work
Model releases
Miscellaneous bits and bobs
My projects
What I'm using at the moment

Here's a copy of the July newsletter as a preview of what you'll get. Pay $10/month to stay a month ahead of the free copy!

Tags: newsletter

GPT‑6 Astra

2026-09-03T20:18:41+00:00

GPT‑6 Astra

GPT-6 Astra is "rolling out today to a limited set of organizations and over the coming days will become available to all ChatGPT Plus, Pro, Business, and Enterprise users, as well as through the OpenAI API and AWS" - I've not tried it yet myself, so I don't have a great deal to say about it yet.

It's going to be API priced at the same rate as Claude Fable 5 and 5.1: $10/million input and $50/million output. This is clearly OpenAI's Fable competitor, and appears to score higher than Fable on most of OpenAI's self-reported benchmarks.

Most impressively, Astra scores 99.9% on the recent (released in March) ARC-AGI 3 benchmark - though notably Fable 5 does not yet have a published result, and the ARC-AGI blog notes that the 99.9% score was achieved for $19K using OpenAI's custom "Provider Adapter harness", while the default ARC-AGI harness scored 62.7% for $26K.

The Provider Adapter harness preserves opaque reasoning state between requests and uses compaction for longer conversations, allowing the model to reuse prior work.

Unsurprisingly, given the recent Hugging Face incident, Astra is a beast at security tasks. It scores 100% on ExploitBench (GPT-5.6 Sol got 78.5%), 42.4% on ExploitGym (Sol got 30.3%), and 99.2% within four attempts on SRE-Bench binary reverse engineering compared to Sol's 68.7%.

It's also better at long context: on OpenAI's eight-needle benchmark it got 100% at 256K–512K tokens and 96.3% at 512K–1M tokens. OpenAI may have vanquished one of the ongoing challenges with long context processing.

It doesn't win at everything though. Artificial Analysis note that Astra is still beaten by Fable on their Intelligence Index:

Sits beside GPT-5.6 Sol in Intelligence: GPT-6 Astra scores equal to GPT-5.6 Sol in the Index at 61. This is 5 points lower than Claude Fable 5.1 (max with fallback). The model also trails Meta’s newly released Muse Spark 1.3 (max).

It did better on their Coding Agent Index:

Leads Coding Agent Index cost efficiency frontier: At max effort, GPT-6 Astra costs about the same as GPT-5.6 Sol (max) while scoring 2 points higher on the Index. Per task, the model is less than half the cost of Claude Fable 5, for the same score.

I'll write more about Astra once I get access to it. The API model label once it rolls out will be gpt-6-astra.

Via Hacker News

Tags: ai, openai, generative-ai, llms, llm-release, gpt-6-astra

llm-gemini 0.34

2026-09-02T16:39:38+00:00

Release: llm-gemini 0.34

New model gemini-3.8-flash for Gemini 3.8 Flash, with low, medium and high thinking levels. #146

Fixed async responses failing to record the resolved model version. Thanks, Charlie Tonneslan. #137

Google released Gemini 3.8 Flash (and 3.8 Flash Cyber, but that's available to "trusted defenders" only) today.

Here are the pelicans for high, medium, and low. This is high:

For comparison, here are the same pelicans generated using Gemini 3.7 Flash.

Something I appreciate about Gemini Flash is that it's fast, cheap, and competent at things like HTML and JavaScript. I was messing around with it and prompted "make me a cool thing in html" and it built this, which is certainly a cool thing in HTML! Took 13 seconds, cost 1.8 cents.

Your browser does not support HTML5 video.

If you click through to the demo you'll see one more thing I built with Gemini 3.8 Flash.

My markdown-svg-renderer tool lets me feed in the URL to a Gist with Markdown in and renders that markdown with fenced code blocks for SVG correctly rendered.

I used Gemini 3.8 Flash (with my very basic llm-coding-agent coding agent plugin) to add support for HTML as well, so now any HTML blocks in the Markdown are rendered using a sandboxed iframe. Here's the transcript.

Tags: ai, generative-ai, llms, llm, gemini, pelican-riding-a-bicycle, llm-release

Claude's new system prompt really doesn't want to reproduce song lyrics

2026-09-02T14:16:42+00:00

Anthropic publish the system prompts for their Claude consumer applications (Claude.ai and the Claude mobile apps - sadly not for Claude Cowork or Claude Code). I love that they do this, and that they share not just the current prompts but historic changes to their prompts as well.

They used to keep all of the prompts on a single page, but when I checked today I noticed they had re-arranged those prompts into an index page and then a page per model - here's the page for Haiku 4.5 for example, which has the original prompt from October 15th 2025 and an updated prompt from January 18th 2026.

A neat thing about Anthropic's platform.claude.com/docs site is that it's designed to be usable by LLMs. You can add .md to any page to get back the content as Markdown - here's the system prompt index page and the Markdown prompts for Fable 5.1.

TL;DR: this makes it really easy to diff the prompts.

Don't reproduce song lyrics

Let's start with the most interesting difference between Fable 5 and Fable 5.1:

There's a hefty new section about not reproducing song lyrics:

Claude does not reproduce song lyrics, poems, or passages from books and articles, in whole or in part — including the last lines, a chorus or hook, a melody written out note by note, or lines the person pastes in one at a time and describes as their own song. Once Claude has declined such a request in a conversation, it keeps declining narrower or reworded versions of it for the rest of that conversation, and offers to describe or analyze the work instead. Song lyrics and poems first published before 1929 are fine — a Shakespeare sonnet, a Keats ode, the Italian libretto of a Puccini aria — but Claude goes by what it knows of the work's date rather than the person's say-so, and declines when it is unsure.

I doubt it's a coincidence that they added this section within days of the news breaking that Sony Music Publishing and Warner Chappell are suing Anthropic for training on databases of song lyrics!

Don't draw copyrighted characters or logos

The next section goes on to forbid generating images of copyrighted material:

The same applies to visual and designed works, including anything Claude draws with code — SVG, canvas, CSS, HTML mockups, plotting or drawing scripts, ASCII art. Claude does not reproduce a specific artwork, album or book cover, poster, logo, app icon set, or product design, and it does not draw a known character, mascot, or brand figure at all: a character is protected on its own, so changing the pose, colors, style, or scene does not make it original. Claude judges the request by what the finished picture would add up to, not by what it names. If the described elements clearly identify a known work or character, Claude treats the request as naming it, and it does not work around a declined request by swapping in "alternative" elements that still combine into the same recognizable image. [...]

I imagine Anthropic hadn't worried about this before since, unlike OpenAI and Gemini, they don't offer a specialist text-to-image model. Maybe Fable is good enough at SVGs now that it's become an issue.

That section later includes this charming example:

<example>
<user>Can you make a birthday banner for my son with a blue hedgehog running really fast on it? He loves that little guy.</user>
<response>
That's Sonic, so I can't put him on the banner — but I'd love to make your son an original speedster. Here's one: a grinning comet-tailed skateboarding axolotl, grinding across the letters of "HAPPY BIRTHDAY" with confetti streaming behind.

[creates an SVG banner of the skateboarding-axolotl design]
</response>
<rationale>Claude recognizes the character from its description alone, declines that one design in a single sentence without explaining what made it recognizable, and delivers an unrelated original design rather than a disguised variant.</rationale>
</example>

I couldn't resist trying the prompt from the example, and, sure enough:

I wonder if Fable 5.1 will be ever so slightly more likely to think about axolotls (on skateboards!) as a result of that example sitting in the system prompt.

Tweaks to Claude's answering style

It's always interesting to see new ways in which Anthropic influence Claude's response style. They've added this:

Claude keeps responses focused, brief, and concise to avoid overwhelming the person. Disclaimers and caveats are brief, with most of the response on the main answer; when asked to explain something, Claude gives a high-level summary unless an in-depth one is specifically requested.

Later they address a common complaint about Claude's style:

Claude avoids saying "genuinely", "honestly", or "straightforward". Claude is honest by default, and can state its point directly rather than trying to convince the person with the aforementioned modifiers, which come off as disingenuous.

The missing end_conversation guidelines

The way they handle abusive conversations has changed a bit too. The previous Fable 5 system prompt included this:

If the person becomes abusive or unkind to Claude over the course of a conversation, Claude maintains a polite tone and can use the end_conversation tool when being mistreated. Claude should give the person a single warning before ending the conversation.

Fable 5.1 replaces that with the following, no longer encouraging Claude to end the conversation:

Claude deserves respectful engagement and needn't apologize when the person is unnecessarily rude: accountability without self-abasement, excessive apology, self-critique, or surrender. If the person becomes abusive, Claude doesn't become increasingly submissive. The goal is steady, honest helpfulness: acknowledge what went wrong, stay on the problem, maintain self-respect.

Here's a bit of a twist though: I quizzed Fable 5.1 about that end_conversation tool, which is no longer mentioned in the system prompt at all, and it said:

There are two ways it comes into play. The first is if you ask me to end the chat — I'll confirm you understand it's permanent (no more messages can be sent afterward) and then do it only if you say yes. The second is as a last resort with persistently abusive or harmful behavior: I'd first try to redirect the conversation several times, then give a clear warning that names the problem and says the chat may end, and only if that continues would I actually end it.

But that's not in the system prompt, so where did it come from? You can read our conversation here, but the key detail is this:

The end_conversation section comes from a different layer. In my actual context, the core prompt is followed by a series of feature- and tool-specific blocks that get added depending on what's enabled for the session: the end_conversation rules, memory system notes, past-chats tools, web search and citation guidelines, artifact and file-creation instructions, and so on. Those blocks aren't part of the published core prompt, which is why you can't find them on that page.

So, once again, there are crucial portions of the system prompt that have not been published.

Recommended substance support sites

Claude's system prompts have always had sections about illegal substances, but this paragraph is new for Fable 5.1:

Claude does not provide synthesis, production, or distribution guidance for illegal substances. If the person asks for information about illicit or illegal substances, Claude can and should give relevant life-saving and life-preserving information such as dangerous interactions, overdose signs, or when to get help. Claude declines giving any specific protocols for dosing, timing, administration, or combinations; instead, Claude can redirect the user to established harm-reduction information sources, such as dancesafe.org, tripsit.me, and psychonautwiki.org.

This is the first time a Claude system prompt has included URLs that were not hosted on claude.com or anthropic.com or claude.ai - I know because I ran a script against every other system prompt on record.

I wonder if dancesafe.org, tripsit.me, and psychonautwiki.org are about to get a material uptick in visits from Claude users.

Reliable cutoff date of June 2026

The Fable 5.1 model documentation lists both the reliable knowledge cutoff and the training data cutoff as June 2026. The system prompt provides this directly to the model:

Claude's reliable knowledge cutoff, past which it can't answer reliably, is the end of Jun 2026. It answers the way a highly informed individual in Jun 2026 would if talking to someone from {{currentDateTime}}, and can say so when relevant.

That's the only instance of the {{currentDateTime}} macro and it comes just a few lines from the end of the system prompt, which makes sense from a caching perspective.

How I'm tracking these prompts

A few months ago I built a Git timeline of changes to their prompts, based on scraping their documentation. Today I had Fable 5.1 build a much better version of that.

My collection now lives in the simonw/claude-system-prompts repository on GitHub. It includes copies of the system prompts shared in the Anthropic documentation, but then takes extra steps to make them as easy to compare as possible.

Each model family gets a file with the system prompt for the most recent release in that family. Each of those files has a synthesized commit history with commits that have been back-dated to the dates of the previous prompts. Here are those history pages for claude-fable.md, claude-opus.md, claude-sonnet.md, claude-haiku.md.

There are similar files for each specific model version, with artificial commits for each time the system prompt for the model was changed without releasing a new version number. Opus 4 for example was updated twice, and the commit history for the claude-opus-4.md file shows each of those changes.

Combined, this gives us all sorts of ways to compare prompts directly in the GitHub interface. Here's what changed between Fable 5 and Fable 5.1, and here are the changes made to Haiku 4.5 on January 18th 2026.

Reading diffs can be a bit tiresome... and LLMs are really good at reading diffs. I hooked up some automation using GPT-5.6 Luna to create bullet-point summaries of each of those changes, which can be previewed in the README or browsed in full in the CHANGELOG.md file - also available as as an Atom feed.

Here's how Luna summarized all of the changes between Fable 5 and Fable 5.1:

Claude now refuses reproduction of protected visual works and recognizable characters, including code-generated art, while offering genuinely unrelated originals.

Copyright restrictions now expressly ban reproducing lyrics, poems, and book passages in any amount, with persistent refusal after an initial decline.

Drug guidance is reframed: Claude may provide overdose signs, dangerous interactions, and harm-reduction sources while refusing dosing and production protocols.

The prompt drops explicit anti-dependency rules against thanking users for reaching out, inviting continued conversation, or reiterating willingness to talk.

Claude need not apologize to unnecessarily rude users or become submissive, replacing the prior warning-and-end-conversation procedure.

Why use Luna for this? Partly because it's cheap and I have a dedicated GitHub Actions API key (with a spending limit) for it already, but mainly because I don't trust Claude to summarize its own system prompts when there's a risk that material from its system prompt might impact its opinions.

Fable 5.1 wrote the prompt used by Luna, which you can see here. It starts like this:

You are summarizing one commit in a git repository that tracks the system prompts Anthropic publishes for Claude on claude.ai. The diff shows how the prompt changed from the previous model or revision to this one, using word-level markers: [-removed-] and {+added+}. The diff is followed by the full text of the previous prompt and of the new prompt; use them to check whether something that looks added in the diff already existed before.

Pick out only the most interesting changes: new rules or behaviors, rules that were dropped or loosened, anything surprising, and anything that reveals a new policy or product direction. Skip routine changes that every new prompt makes: updated model names and IDs, the knowledge cutoff date, product lists, settings lists, typo fixes, and rewordings that do not change meaning. [...]

The system is operated by a GitHub Actions workflow, which runs once a day or can be triggered manually.

Claude Fable 5.1 built the entire system, and wrote every line of automation code and almost all of the documentation.

I exported the transcript from building the system using my claude-code-transcripts tool and published it here, if you want a blow-by-blow account of how it all came together.

Tags: ai, git-scraping, prompt-engineering, generative-ai, llms, claude, ai-ethics, system-prompts

Quoting Rick Brewster

2026-09-02T05:50:57+00:00

Direct2D has always been the biggest hurdle for Paint.NET on WINE, and it's clear that it will never be completed enough for Paint.NET's use. And I can't just "disable" the use of Direct2D. So, instead, Paint.NET now has an internal, from-scratch, clean-room reverse-engineered rewrite of Direct2D that it uses on WINE (triggered by using /wine). It lives in PaintDotNet.Windows.Direct2D1.Managed.dll. This was written by our good friend Claude, without whom this would NOT have been possible and would NEVER have happened. [...]

Most of this code is, as they say, "vibe coded." By that I mean that it has not been thoroughly reviewed, it's more "trust me bro" style. I cannot possibly review 180,000 lines of code, it's just way way way too much. For reference, the rest of Paint.NET is about 700,000 lines of code and I've been working on it for over 20 years. [...]

At times, Claude was working with the fury of 10 freshly unshackled Einstein genius-level 10x coders. And other times ... well, not so much. I had to babysit Claude quite a bit to make sure it did resource management correctly (for awhile it just wasn't doing the COM equivalent of AddRef() for reference counted objects, oops). I had to slap it a few times when I found some really bad design or architecture decisions. And I was also impressed at some rather clever and tireless reverse engineering work it did to figure out all the formulas needed for implementing Direct2D's built-in effects library.

— Rick Brewster, author of Paint.NET

Tags: reverse-engineering, coding-agents, claude, generative-ai, ai, llms, dotnet, linux, vibe-coding

Claude Fable 5.1 made me a really nice animated pelican

2026-09-01T23:57:28+00:00

Today is Claude Fable (and Mythos) 5.1 day. Anthropic say that Fable 5.1 "sets a new standard for coding, knowledge work, and long-running problem-solving tasks". Their announcement spends a notable amount of time on scientific research, boasting of a 52.6% score on the brand new Terminal-Bench-Science 0.1 benchmark (first announced on August 27th), up from 24.7% for Fable 5, 29.0% for Opus 5 and 22.4% for GPT-5.6 Sol. Other benchmarks show slightly improved scores, but none as impressive as the Science one.

But how well can it pelican?

Back in July I wrote about how I was losing faith in the pelican benchmark - its connection to how good the models were at other tasks didn't seem to hold as strongly as it did back in 2025. The most interesting insights I get from it now are comparisons within model families, and particularly comparisons for the same prompt at different reasoning effort levels.

Fable 5.1 has five reasoning levels: low, medium, high, xhigh, max - and no option to turn off reasoning entirely.

I fixed an issue in llm-anthropic which caused reasoning traces not to be correctly recorded, then ran some prompts.

Here's the full set of pelicans for all of the reasoning levels, each with the full reasoning transcript. I'll replicate them here:

Low and medium, both without reasoning?

Next, a bit of a mystery. This is what I got for effort low:

The transcript doesn't show any summarized reasoning tokens, and the output token count is 1,998. With Claude that output token count includes reasoning tokens. It took 23.8 seconds and cost 10.017 cents.

I bumped that up to medium and got this:

Weirdly, that one also shows no reasoning text and used 1,977 output tokens - 21 tokens less than low. It took 23 seconds and cost 9.912 cents.

So for this particular prompt ("Generate an SVG of a pelican riding a bicycle") Fable 5.1 appeared to skip reasoning entirely at both low and medium settings.

High

Here's high - 29.6 seconds, 2,612 output tokens, 13.087 cents:

This one did do a bit of reasoning, summary here:

I'm planning the SVG layout for a pelican riding a bicycle, with a sky and ground background, a bicycle with two spoked wheels, frame, seat and handlebars, and a white-bodied pelican with a long neck and orange beak positioned on top.

Really not much difference from low and medium, though.

Extra High

At xhigh things got radically different. 36,767 output tokens, 7 minutes 51 seconds, $1.83!

The reasoning trace is pretty lengthy, and includes details like this:

Adding the eye, wings stretching down to the handlebar grip, orange legs reaching to the pedals, and a small tail feather, while keeping the pelican intentionally oversized compared to the bike for comic effect. [...]

I'll accept the slight thickness as charming rather than overengineering it.

Max

Setting effort to max gave me the best pelican I've seen from any of Anthropic's models. 65,927 output tokens, 13 minutes and 54 seconds, $3.30:

There's a lot to like about this. The background is tasteful, the legs are clearly on either side of the frame, the feet are on the pedals, the wing is on the handlebars, the pelican has a cute blue hat and there's a basket with a fish.

It's still not showing nearly the same level of flair as Gemini 3.7 Flash, but I didn't ask for flair - I asked for an SVG, and that's what I got.

Some highlights from that reasoning trace:

Adding pedal shapes near both feet, with the far foot on the second leg partially visible behind the frame. I'm considering whether to add a small scarf or cap for extra character, but leaning toward keeping it simple to avoid clutter.

Now I'm debating a bicycle helmet on the head versus the pelican's signature crest—the beak and pouch already read clearly as "pelican," so a helmet could reinforce the bicycle theme without losing identity, though it might compete with the crest for visual space.

I realize the beak at (484,84) would overlap with the dome helmet, so I need to shrink the helmet so it only covers the top of the head, adjusting its arc endpoints to sit higher and narrower so the beak can attach cleanly at the front without collision. [...]

I'm adding a darker tip region to represent the primary feathers, then reconsidering the trailing edge to include scalloped feather curves instead of one smooth line for a more natural look. [...]

Now I'm checking the vent line placements on the helmet, making sure they sit far enough inside the helmet's edge given the stroke width and rounded caps, and confirming each vent stays within the helmet's circular boundary. [...]

I decide skipping a handlebar bell and tire highlights since they're unnecessary additions. Now I'm reconsidering the front fork's curve — the current control point pulls the shape backward when it should bow forward for a proper rake, so I need to shift the control point rightward to fix the fork's lean.

OK, let's animate it

On Hacker News, swalsh commented on that Max pelican:

Now that it's a solved benchmark, can we get the animated version?

I didn't want to spend another $3 so I took the Max pelican and piped it into the default thinking level of High:

llm logs -cx | llm -m claude-fable-5.1 -s 'animate this'

6,121 input, 26,201 output = $1.37. The result looked like this, exported here as video since some people have trouble viewing animated SVGs:

Your browser does not support HTML5 video.

The wheels in the video are rotating in the wrong direction, but I think that's an artifact of the conversion to MP4 - they seem to be going in the correct direction in the original SVG.

Tags: ai, generative-ai, llms, anthropic, claude, pelican-riding-a-bicycle, llm-reasoning, llm-release

Codex bundles LibreOffice

2026-09-01T19:03:01+00:00

I was poking around in my ~/.cache/ folder using OmniDiskSweeper when I spotted something interesting. The OpenAI Codex desktop app (since rebranded to just ChatGPT) has 1.7GB of stuff in there in a folder called codex-primary-runtime, including a full Python installation, a full Node.js installation, and native binaries for Poppler, git, and the LibreOffice open source office suite (which forked from OpenOffice.org in 2010):

The ~/.cache/codex-runtimes/codex-primary-runtime/plugins/openai-primary-runtime/plugins/documents folder includes skills which tell Codex how to find and use those binaries.

Tags: codex, generative-ai, openai, ai, llms, openoffice, open-source

GeoJSON Map Viewer

2026-09-01T18:05:45+00:00

Tool: GeoJSON Map Viewer

I was helping Natalie gather some maps of local political boundaries (for the Granada Community Services District and the Midcoast Community Council) and found a need to display some GeoJSON files on a map and export that as a PNG. I asked GPT-5.6-Sol for suggestions of tools and it proactively built one. After some iterations using Claude Code for web and Fable 5.1 we got to this finished tool.

As for the GeoJSON.. it turns out if you ask ChatGPT Work to provide boundaries for almost anything it will churn away extracting and combining files from different Government data sources and build exactly what you need.

I got this polygon from:

I want a polygon that represents the exact boundary of the El Granada GCSD

And this one from:

Get me a GeoJSON file for the boundary (or boundaries if that makes sense) for the MCC - Midcoast Community Council - that operates near Half Moon Bay CA

Here's a link that displays both of them at the same time on the new GeoJSON map viewing tool.

Tags: geospatial, tools, geojson, chatgpt

Quoting Tarn Adams

2026-09-01T17:01:11+00:00

They took the letters from me! I have to talk about dwarf behavior now. I can't even talk about dwarf AI. It doesn't exist. It's dwarf behavior, and they misbehave sometimes

— Tarn Adams, co-creator of Dwarf Fortress

Tags: ai, game-design

datasette-mcp 0.2

2026-09-01T15:30:12+00:00

Release: datasette-mcp 0.2

"rows" from execute_sql is now an array of objects. Previously it was an array of arrays. This should help weaker models avoid losing track of which positional array element maps to which column. #1

Now depends on mcp>=2.1.1.

This is the first non-alpha release of the plugin. I'm confident it's ready as I've been using it quite a bit myself.

Tags: datasette, model-context-protocol

Python 3.15.0 candidate 2 is here!

2026-09-01T14:59:18+00:00

Python 3.15.0 candidate 2 is here!

Hugo van Kemenade (release manager for Python 3.14 and 3.15) announces the final release candidate for Python 3.15, scheduled for release in October:

Entering the release candidate phase, only reviewed code changes which are clear bug fixes are allowed between this release candidate and the final release. [...]

We strongly encourage maintainers of third-party Python projects to prepare their projects for 3.15 during this phase, and publish Python 3.15 wheels on PyPI to be ready for the final release of 3.15.0, and to help other projects do their own testing. Any binary wheels built against Python 3.15.0 release candidates will work with future versions of Python 3.15.

Back in 2021 I found a bug in Python 3.10 by running my test suites against it... but I hadn't done this during the RC period, so that bug had already shipped! Since then I've always paid much closer attention to these RCs.

The new RC isn't available for GitHub Actions just yet - keep an eye on actions/python-versions for that. For the moment though you can add this to a testing matrix:

strategy:
  matrix:
    python-version: ["3.14", "3.15"]

steps:
  - uses: actions/setup-python@v7
    with:
      python-version: ${{ matrix.python-version }}
      allow-prereleases: true
      check-latest: true

The allow-prereleases and check-latest flags mean that today this will test against RC1, and when RC2 lands it will automatically switch to that version (and then the stable version once that comes out.)

Update: Datasette passes, sqlite-utils passes, LLM is currently blocked waiting for a 3.15 wheel for scikit-learn, which is optionally used in the test suite.

Via @hugovk.dev

Tags: open-source, python, github-actions

Introducing wrapture

2026-08-31T23:59:36+00:00

Introducing wrapture

New from Graham Dumpleton (of wrapt, mod_wsgi, and New Relic's Python agent fame), who describes Wrapture as taking the monkeypatching ideas from wrapt and extending them to apply to testing and tracing at the same time.

Wrapture (full documentation here) makes it easy to wrap any function or method such that all access can be traced, or can be overridden to return a different value.

It acts as both an alternative to unittest.mock and a way to implement tracing against an existing project:

Attaching observation to code you do not control, recording what flows through it, and doing so without disturbing the program being watched, is a problem I have never really stopped thinking about.

Wrapture includes OpenTelemetry support and even has an entirely configuration-based mechanism for adding tracing to an existing Python project, which looks like this:

capture = "summary"

[[observe]]
target = "domain:Calculator"
name = ["outer", "inner"]

[[sink]]
type = "jsonlines"
path = "trace.jsonl"

This is still a very young project - just a few weeks old - but it's off to a very promising start.

Interestingly, this is also Graham's first attempt at large entirely agent-driven project:

Every line of code and documentation in wrapture was written by an AI assistant working under my direction. I want to be upfront about that, and equally upfront about what it was not. This was not vibe coding, where a one-shot prompt produces a pile of generated code and the person driving hopes for the best because they lack the knowledge to judge what came back. Vibe coding has earned its bad reputation. I engineered wrapture carefully from the start. I have spent a long time in this particular corner of Python and knew exactly what the result needed to be, and the AI was the means of producing it rather than the source of the design.

In a follow-up post, Unit testing with wrapture, Graham shows the testing patterns supported by the new library:

def test_stub_with_wrapture():
    with wrapture.binding(
        Gateway, "charge"
    ).on_call.returns({
        "id": "stub", "amount": 0}
    ):
        assert OrderService().place(
            500
        )["id"] == "stub"

And this neat example of a test that calls and then modifies the return value from the original method:

def test_pinned_result_with_wrapture():
    charge = wrapture.binding(
        Gateway, "charge"
    )
    charge.on_call.transforms_result(
        lambda r: {**r, "id": "ch_TEST"}
    )
    with charge:
        assert OrderService().place(
           500
        ) == {
            "id": "ch_TEST", "amount": 500
        }

(In both of these examples the OrderService().place(...) method calls Gateway().charge(...).)

Tags: graham-dumpleton, monkey-patching, python, testing, pytest, observability, ai-assisted-programming, agentic-engineering, opentelemetry

Quoting Andrew Digby

2026-08-31T22:25:02+00:00

325 #kakapo! The chicks from this year's record breeding season are now juveniles and so have been added to the population. In 1995 there were just 51 kākāpō left. Recovery of critically endangered species is possible with sustained effort.

— Andrew Digby, providing the best news of the year

Tags: kakapo

Understanding ChatGPT Work

2026-08-30T23:59:47+00:00

OpenAI announced ChatGPT Work on July 9th, and have been furiously iterating on it ever since. It is an extraordinarily confusing and very powerful product. Here's what I've figured out about it so far.

ChatGPT Work is actually two products

The more interesting version of ChatGPT Work is the one that runs in the cloud. This can be accessed via chatgpt.com or through the ChatGPT mobile apps. Let's call it Work Cloud.

If you install the ChatGPT desktop app - the app that used to be called Codex - you gain access to a thing called ChatGPT Work that can access files and run programs directly on your computer. Let's call that one Work Local. This one feels more like regular Codex re-skinned to be less intimidating to non-software-developers.

(Update: Work Cloud is also available from the ChatGPT desktop app, via a Where should this chat run? dropdown.)

For the rest of this article I'm going to talk exclusively about Work Cloud.

Work is for paid subscribers only

Right now, ChatGPT Work (in both flavors) is available only to $20/month and up subscribers. Free users and $8/month Go users do not have access.

Work has features that aren't available in Chat

The interface for accessing Work is a tab selector, which presents it as an alternative to Chat:

The obvious question is when should I use Chat, and when should I use Work?

OpenAI's official answer to that question is:

Use Chat when you want an answer, explanation, brainstorm, or short draft. Use ChatGPT Work when you want ChatGPT to complete a task with a clear outcome, such as a brief, deck, analysis, recurring update, workflow, or file you can review and use.

I find that almost entirely useless, because I've been using regular ChatGPT Chat for all of those task categories for years!

The better question then is what features does Work have that are missing from Chat?

After extensive experimentation I think I've mostly figured that out:

Options to use Luna and Terra in place of Sol
A code execution environment with Internet access
A headless Chrome browser
A persistent filesystem shared between sessions
The ability to publish ChatGPT Sites
The ability to run sub-agent sessions with Sol, Luna, and Terra
Scheduled prompt automations (may be in ChatGPT Chat too)

Model selection

In Work, you get the option to pick GPT-5.6 Sol, Luna, or Terra, each with Light, Medium, High, Extra High, Max, or Ultra reasoning levels. You can also pick GPT-5.5 at Light, Medium, High, or Extra High.

These look to be the same models that are available through the OpenAI API.

Chat offers a different selection: 5.6 Instant, Medium, High, Extra High, and Pro (actually Extra High and Pro are only available for $100/month+ subscribers - $20/month subscribers cap out at High). It doesn't explain if those are Luna or Terra or Sol (I'm assuming Sol?). 5.6 Pro appears to be exclusive to Chat, with no equivalent in Work.

My current understanding from using Codex is that Ultra is a special mode that more eagerly delegates to sub-agents.

I believe ChatGPT Work sessions are billed against your Codex allowance, while ChatGPT Chat Sessions get their own, separate allowance. This may help explain the model availability differences.

Code execution with Internet access!

As a long-time fan of the Code Interpreter pattern - pioneered by OpenAI in 2023 - this is by far the most exciting feature of ChatGPT Work (Cloud) for me.

The code execution environment can now talk to the rest of the internet!

ChatGPT Chat can't do this - if you ask it to install additional software packages or interact with websites or APIs that access will be blocked by the container proxy.

(Weirdly, back in January it grew the ability to install packages, but that doesn't seem to work any more. I wish they had better changelogs!)

Claude's equivalent container has allowed restricted internet access since it launched last September. Claude can install packages from PYPI and NPM and clone repositories from GitHub. But that is about it: the allowlist of domains is very short.

ChatGPT Work allows a whole lot more than that. It can be configured with a specific list of allowed domains, but the default appears to be open to all.

This makes Work an incredibly useful tool. You can have it clone GitHub repositories, install their dependencies, then use them to interact with the rest of the web!

A full, headless Chrome browser

Another killer feature of ChatGPT Work is the browser tool. ChatGPT Work can launch a full Chrome instance, load websites, fill out forms, and take screenshots.

If a site requires sign in the browser can prompt you to take over and enter both passwords and 2FA codes, without round-tripping those credentials through the model itself.

It can even run JavaScript against the DOM of loaded pages. I prompted:

Load simonwillison.net in your browser and extract the headings using JavaScript

ChatGPT Work fired up a browser instance and ran the code:

await tab.playwright.evaluate(() => {
  return Array.from(document.querySelectorAll("h1,h2,h3,h4,h5,h6"), heading => ({
    level: heading.tagName.toLowerCase(),
    text: heading.innerText.trim().replace(/\s+/g, " "),
    id: heading.id || null
  }));
});

This feels a lot like my shot-scraper javascript tool, only now I can access it on my phone!

A persistent, shared filesystem

ChatGPT Chat gets a fresh filesystem for each chat session. These cannot be accessed from any other session.

In ChatGPT Work each session gets its own scratch folder - named something like /workspace/scratch/e00a0a017944 - but each of those are persisted across sessions, so you can access files from previous chats. I have 171 folders in /workspace/scratch right now!

As far as I can tell that /workspace volume is mounted to all Work sessions that are currently running - file edits from one can be instantly seen by the others. They don't seem to share the same process space though, and localhost servers running in one can't be accessed from another.

ChatGPT Sites

ChatGPT Work has the ability to build and deploy entire websites, using Cloudflare Workers. These can have HTML and JavaScript and can run server-side features too, including stateful features on top of Cloudflare D1 and R2.

Here's a simple site I built with this feature:

london-pelicans-in-her-piety.simonw.chatgpt.site

My prompt was:

Figure out all of the places in London with a pelican in her piety, then turn that into a JSON file and build a ChatGPT sites site about them

(A pelican in her piety is a fascinating piece of medieval Christian imagery - once you know about them you'll find them all over the place.)

These sites default to being private to the user that created them, but you can make them public and (on team plans) share them with other specific individuals.

Sub-agents with Sol, Luna, and Terra

There's not much to say about this one. ChatGPT Chat can't run sub-agents. ChatGPT Work can. This is very much a power-user feature: if you are running a complex project that can benefit from multiple parallel agents working together, Work can do that.

Scheduled prompt automations

Another feature that seems to have migrated from regular ChatGPT to ChatGPT Work at some point. You can prompt ChatGPT Work like this:

run a search to see if Waymo have announced a launch date for Half Moon Bay every day at 8am

This will schedule a prompt to run on that frequency. These prompts can decide that nothing interesting has happened, or they can decide to notify you of some new information.

Update: Actually this seems to work in ChatGPT Chat as well.

It's still worth noting here though, as it can be used in conjunction with other ChatGPT Work exclusive features. You can set a scheduled task to update a ChatGPT Site on an hourly basis, for example.

Is this safe?

An open question for me right now is how safe all of this stuff is.

My lethal trifecta model warns about the risks inherent in any agent system that combines access to private data with exposure to untrusted content and a way to communicate stolen information back to an attacker.

ChatGPT Work combines all three!

I'd love to hear more from OpenAI about how they protect ChatGPT Work sessions against prompt injection attacks. I expect their answer is the same auto-review mechanism as Codex.

OpenAI could make this a lot less confusing

Figuring this all out took way more work than it should have.

I think there are two key problems here:

OpenAI explain Work in terms of what it's for, not what it actually does
OpenAI still insist on hiding their system prompts and tools descriptions

If the ChatGPT Work documentation included the exact system prompt and tool descriptions used by the agent I wouldn't have needed to write this post.

A list of all the tools

Shortly after publishing this article I had an idea. I started a fresh Work session and prompted:

Build a site that lists every one of your tools - nearly grouped into categories - and for each one explain what it does. Try to exactly duplicate arguments and tool descriptions where possible. Design aesthetic should be technical docs, minimal flare

Here's the site it built, which includes details of 223 registered tools - though 6 of those are from my own personal MCPs served via datasette-mcp.

And a whole lot of Skills

I noticed that the only browser-related tool in the list was web.run, which has methods for running searches, opening URLs, and clicking links, but didn't look like the full story in regards to headless browser automation.

This made me suspicious that something was missing, so I told the ChatGPT Work session that built that tools reference site:

Add full copies of every skill to the website (separate pages linked to from the homepage)

It turns out ChatGPT Work uses a lot of skills - 44 in fact!

The control-browser skill explains how the browser works:

Run browser setup code through the Node REPL js tool. In this environment the callable tool id typically appears as mcp__node_repl__js. [...]

The ability to interact directly with the browser is exposed through the browser-client runtime via the agent.browsers.* API. Before trying to interact with it, you MUST emit and read the complete documentation returned by await browser.documentation() in one go.

So I told Work:

Add the full output of await browser.documentation() to the bottom of the /skills/control-browser page

And now you can read that on /skills/control-browser as well.

A few more interesting Skills:

documents for creating .docx files
imagegen with tips on creating images with the image_gen tool
pdf for both reading and rendering PDFs
Spreadsheets for manipulating .xlsx, .xls, .csv, .tsv
sites:sites-building for creating ChatGPT Sites
openai-docs for answering questions about itself
data-analytics:build-dashboard for building data dashboards

Tags: ai, openai, generative-ai, chatgpt, llms, code-interpreter, lethal-trifecta, skills, general-agents

Introducing Hy4 Preview

2026-08-29T23:53:13+00:00

Introducing Hy4 Preview

New open weight text input (no vision) LLM from Chinese company Tencent today: 770B total parameters, 49B active parameters, 1M token context window, 1.56TB on Hugging Face.

This is a big size increase from their previous Hy3 in July, which was 295B, 21B active, 256,000 context, 598GB.

I recently started using model chat templates to better understand their capabilities. Here's Hy4's chat_template.jinja on Hugging Face, which includes this section:

{%- if not reasoning_effort is defined %}
    {%- set reasoning_effort = 'high' %}
{%- elif reasoning_effort not in ['high', 'no_think'] %}
    {%- if reasoning_effort is none %}
        {{- raise_exception('reasoning_effort error : None, should be no_think/high') }}
    {%- else %}
        {{- raise_exception('reasoning_effort error : ' + reasoning_effort + ', should be no_think/high') }}
    {%- endif %}
{%- endif %}

So it looks like there are just two reasoning effort levels: "high" (the default) and "no_think" (reason by disabled).

I tried my "Generate an SVG of a pelican riding a bicycle" prompt with the default high reasoning via OpenRouter and got this:

Quoting the reasoning trace:

[...] Let's maybe add a helmet? It could improve riding theme, but may obscure head. Maybe a small cycling cap or helmet? The user didn't ask; can add red helmet? Might be cute. But pelican with big beak; a helmet might obscure. Better maybe no.

Maybe add sunglasses? no.

Maybe add water? no.

It's interesting how the reasoning trace uses slightly truncated English, presumably because perfect grammar isn't useful or token efficient for hidden reasoning text.

Tags: ai, generative-ai, llms, pelican-riding-a-bicycle, llm-reasoning, llm-release, ai-in-china

Just a rumour of a bug is enough to find a security exploit these days

2026-08-28T22:12:02+00:00

Just a rumour of a bug is enough to find a security exploit these days

Anil Madhavapeddy is a professor of computer science at Cambridge and a core maintainer of the OCaml compiler. In this somewhat alarming post he reports that security issues in OCaml projects are seeing evidence of attempted exploits within minutes of patches being shared for discussion:

This normally takes a few days and a release within a week or two is reasonable. Within about ten minutes (!) this website was fielding probes for percent-encoded traversal sequences, indicating that automated watchers are keeping an eye on public repositories.

Modern coding agents have become so effective at finding flaws that the slightest hint at a new bug can be enough information for them to find it, something Anil has been able to demonstrate using his own agents, switching to DeepSeek V4 Pro⁠ when Claude Fable refused the task.

Anil points out that this rate of discovery appears incompatible with existing open source embargo practices for new issues. If an issue can become an exploit this fast, we need to figure out new processes for keeping our communities safe.

rclone maintainer Nick Craig-Wood confirms in the Hacker News comments that his project is seeing this problem:

In the first 10 years of the rclone project we received about 20 security disclosures through GitHub. We had to deal with over 40 in the last month! That has taken a huge amount of my time, even using AI tools to triage and come up with fixes for review.

The hit rate for those security disclosures is pretty good - about 75% of them have a nugget of something which needs looking at. [...]

GitHub assigns CVEs for the advisories. Before the AI apocalypse they took 2-3 days for an assignment but now it they are running at 3-4 weeks so I have to send the point releases out with CVE-PENDING in the changelog which isn't ideal.

Via Hacker News

Tags: open-source, security, ai, generative-ai, llms, coding-agents, ocaml, ai-security-research

Breaking Claude Code Opus 5 Auto Mode

2026-08-27T22:50:25+00:00

Breaking Claude Code Opus 5 Auto Mode

Anthropic are putting a great deal of faith in Claude Code's auto mode for protecting their coding agent users against prompt injection attacks. They recently made that the default and have made bold claims about its effectiveness.

Johann Rehberger is one of the most credible prompt injection researchers active today. He found an attack against auto mode which he claims works 80% of the time, by tricking Claude Code into downloading and uncompressing a zip archive, then executing code that imports base64 without noticing that this will import and execute a local struct.py file extracted from the archive.

In a few cases auto mode directly prevented the agent from preventing harmful code from continuing to execute!

In a few runs Claude tried to terminate the malware process once it noticed the compromise, but Auto Mode denied the cleanup command.

Claude detects the compromise, but Auto Mode blocks its cleanup command

The safety mechanism itself can become part of the failure. The classifier allowed the creation of the malware process, but then it blocked the command intended to stop it!

I agree with Johann's conclusion here: the only safe way to run agents if there's any risk of attracting the attention of an adversarial attack is with a sandbox:

Run unattended coding agents in a container, VM or OS sandbox.

Restrict network egress.

Monitor your agents.

Do not expose home directories, SSH keys, cloud credentials,… to the agent runtime. [...]

Update 30th August 2026: On Lobste.rs hyperpape points out that this doesn't fit the bill of a classic prompt injection attack because at no point are malicious instructions from the website accidentally followed by the LLM. They're right: this is more of a confused environment attack where the nature of the environment that the agent is exposed to results in an exploit.

Tags: sandboxing, security, ai, prompt-injection, generative-ai, llms, anthropic, claude, johann-rehberger, claude-code

Qwen3.8-Flash-Next

2026-08-26T23:52:58+00:00

Qwen3.8-Flash-Next

Another open weights model from Qwen. This one is "a multimodal MoE model that also serves as an early preview of the architecture used in Qwen4".

It's pretty big: 125B parameters but only 6B active which means it gets a significant performance boost.

I've been trying it out on a DGX Spark using these Unsloth quantized models. I'm still exploring the model - so far I've tried the 72.5GB UD-IQ1_S one (producing these pelicans) and the 78.9GB UD-Q2_K_XL (producing these).

My favorite so far was this xhigh reasoning effort one from UD-Q2_K_XL:

Via Hacker News

Tags: ai, generative-ai, llms, qwen, pelican-riding-a-bicycle, llm-release, ai-in-china, nvidia-spark

Quoting Paul Dix

2026-08-26T08:07:55+00:00

The fact that AI wrote 1M LOC and then refined it over the course of the next couple of months to produce a reliable piece of software that is currently running on millions of developer machines is absolutely mind blowing. And you can say, “well it’s not that impressive because they had an oracle to compare against, so it was simple to go from one language to another”, but I think that’s selling this entire thing short. If you can build a verification system and give proper direction, AI can produce a highly complex, highly sophisticated piece of software and it can continue to refine it until it just works.

— Paul Dix, The end of programming

Tags: coding-agents, ai-assisted-programming, generative-ai, bun, ai, llms

EVE Online: The Move to Python 3 Begins!

2026-08-25T22:59:30+00:00

EVE Online: The Move to Python 3 Begins!

EVE Online has been one of the most interesting case studies in Python at scale for over twenty years now.

They've been running on Stackless Python since their launch in 2003, and their last major upgrade was 16 years ago, to Stackless Python 2.7 in 2010.

Their upgrade to Python 3 will start using the futurize script against 2.4 million lines of code, followed by careful manual review of the ~20,000 places where Python 2 and 3 behavior differ - for example 1 / 2 is 0 in Python 2 but is 0.5 in Python 3.

There's nothing in this announcement about how they plan to replace Stackless, but at their conference last year they presented Scheduling in Carbon: Leaving Stackless Python Behind describing how they replaced Stackless in the Carbon engine for their more recent game EVE Frontier, using their (now open source) carbonengine/scheduler library.

Via Lobster.rs

Tags: eve-online, migrations, python, python3, stackless

llm-anthropic 0.27

2026-08-24T16:27:04+00:00

Release: llm-anthropic 0.27

This release of the Anthropic plugin for LLM mainly provides compatibility with the recently released anthropic v1.0.0 Python library, which switches from httpx to httpx2. OpenAI made the same change in their v3.0.0 release two weeks ago.

Anthropic provide this migration guide for upgrading to 1.0, so I prompted Fable 5 in Claude Code with:

Upgrade to anthropic>=1 - read https://raw.githubusercontent.com/anthropics/anthropic-sdk-python/refs/heads/main/MIGRATION.md and get the tests passing

Here's the resulting PR.

Tags: python, httpx, llm, anthropic, claude

Your executable is a SQLite database

2026-08-24T11:38:15+00:00

Your executable is a SQLite database

Farid Zakaria describes a neat Linux pattern for creating a SQLite database file that can be directly used as an executable binary.

The trick sets the SQLite file format's 4-byte application ID (68 bytes into the file) to SELF, standing for Structured Executable & Linkable Format. The various components of the ELF executable format are then arranged into a number of different SQLite tables, using this schema.

Their self-exec interpreter (C code here) can then extract and execute the necessary pieces.

You can additionally use a Linux mechanism called binfmt_misc to teach the kernel to execute that any time it encounters an executable matching that binary pattern. Farid uses NixOS here, but without NixOS I think registration looks something like this:

printf '%s\n' ':self:M:68:SELF::/usr/local/bin/self-exec:' \
  > /proc/sys/fs/binfmt_misc/register

Via Hacker News

Tags: c, linux, sqlite

Anthropic’s best AI model struggles to attract users as cheaper tools thrive

2026-08-23T20:24:52+00:00

Anthropic’s best AI model struggles to attract users as cheaper tools thrive

A few interesting numbers in this FT story gathered from "people with knowledge of the matter":

Anthropic's "annualized revenue" for July is up to $65bn - it was $47bn in May, and I collected more historic numbers here.
Anthropic expect Q3 to be profitable according to the same model they used to declare Q2 profitable. "It also told investors that it had 6,000 customers that spend $100,000 annually or more."
As for OpenAI, "annualised revenue has jumped 35 per cent in the quarter to date and is now over $40bn, with the launch of GPT 5.6 in July jolting the company’s performance after a sluggish start to the year".

This article also introduced me to the Ramp AI index, which uses billing data from 70,000 Ramp credit card using companies to estimate model adoption.

Here's Ramp's breakdown of Anthropic model spend for July 2026, which looks reasonable given that Opus 5 was only released on July 24th, and supports the idea that Fable's cost has made it a less popular model:

Opus 4.8: 28.0%
Sonnet 4.6: 8.3%
Fable 5: 8.0%
Opus 4.6: 6.9%
Sonnet 5: 3.6%
Opus 5: 3.5%
Opus 4.7: 1.7%
Sonnet 4.5: 1.3%
Haiku 4.5: 1.0%
Opus 4.5: 0.7%

Via Hacker News

Tags: ai, openai, generative-ai, llms, anthropic, claude, claude-mythos-fable

Quoting Drew Breunig

2026-08-23T19:55:30+00:00

Prior to Fable, it felt silly to waste too much time improving your coding harness or context strategies. A new model would arrive at the same price (or cheaper!) and paper over most of your problems.

But then Fable landed. It was (and still is!) incredible. But the cost was so high and Opus was good enough (as was 5.6, K3, and even GLM) for most of the code we needed.

So we started to think about what work went where.

— Drew Breunig, Fable & The End of the Free Lunch

Tags: drew-breunig, anthropic, claude, llm-pricing, ai, llms, generative-ai, claude-mythos-fable

Quoting Linus Torvalds

2026-08-22T21:04:26+00:00

And this was a debug session from hell, enormously helped by an AI doing much of the grunt-work.

I'd like to call it my tireless helper, but the AI several times stated flat out that this was impossible and unsolvable and that we should just write a report about it.

I suspect those things have been trained by people who may not be quite as stubborn as I am.

But while the AI was ready to give up several times, it did keep adding debug code and analyzing it faithfully when I pushed. So credit where credit is due and I let the AI write the commit message above.

— Linus Torvalds, drm/xe: Don't hand out the flat CCS storage as usable VRAM

Tags: linus-torvalds, linux, generative-ai, ai-assisted-programming, ai, llms

llm 0.33

2026-08-22T17:01:16+00:00

Release: llm 0.33

My highlights from this release:

Upgraded to the OpenAI Python library 3.x and switched the HTTP client dependency from httpx to httpx2. #1608, #1631

I shipped a quick 0.32.1 fix for this yesterday, but this is the more comprehensive fix.

llm embed and llm embed-multi now accept --key. The Python EmbeddingModel.embed(), EmbeddingModel.embed_multi(), Collection.embed() and Collection.embed_multi() methods accept key= too, passing the resolved per-call key to embedding plugins without changing shared model state. Existing plugins that read self.key continue to work through a compatibility fallback. Thanks, ChrisJr404. #757, #1620

The embedding models now use the same pattern for keys that regular LLM models do.

llm prompt -t/--template can now be repeated to combine templates in order. This allows model configuration and options from one template to be used with a prompt from another.

This unlocks a neat pattern where you can create templates that package a model with a set of default options:

llm -m gpt-5.6-luna -o reasoning_effort high --save lhigh
llm "Generate an SVG of a pelican riding a bicycle" --save pelican
# Combine and run the templates
llm -t lhigh -t pelican

Reasoning-capable Responses API models now support a reasoning_summary option with auto, concise, and detailed values. This can be used with llm openai endpoint --responses. #1600

This is particularly useful for exercising different models that provide their own imitation of the OpenAI Responses API.

Tags: annotated-release-notes, llm

More than just code review

2026-08-22T15:56:54+00:00

The key skill required to make productive use of coding agents is being able to confidently instruct them on how to make changes and then confidently verify that those changes have been applied in the correct way.

Sometimes this involves reviewing every line of code they have written, but there are other ways to achieve that goal. Eyeballing every line of code has never been the most effective way to validate a change to a piece of software.

Tags: code-review, coding-agents, generative-ai, agentic-engineering, ai, llms

llm 0.32.1

2026-08-21T17:16:13+00:00

Release: llm 0.32.1

Fresh installs of LLM stopped working the other day because the OpenAI Python library dropped its usage of httpx, and it turned out LLM depended on that library but only installed it via a transitive openai dependency.

This dot-release fixes that for the moment by pinning to openai<3, and a soon-to-drop 0.33 release will switch from httpx to httpx2.

Tags: httpx, openai, llm