Simon Willison's Weblog: realtime

Introducing gpt-realtime

2025-09-01T17:34:55+00:00

Released a few days ago (August 28th), gpt-realtime is OpenAI's new "most advanced speech-to-speech model". It looks like this is a replacement for the older gpt-4o-realtime-preview model that was released last October.

This is a slightly confusing release. The previous realtime model was clearly described as a variant of GPT-4o, sharing the same October 2023 training cut-off date as that model.

I had expected that gpt-realtime might be a GPT-5 relative, but its training date is still October 2023 whereas GPT-5 is September 2024.

gpt-realtime also shares the relatively low 32,000 context token and 4,096 maximum output token limits of gpt-4o-realtime-preview.

The only reference I found to GPT-5 in the documentation for the new model was a note saying "Ambiguity and conflicting instructions degrade performance, similar to GPT-5."

The usage tips for gpt-realtime have a few surprises:

Iterate relentlessly. Small wording changes can make or break behavior.

Example: Swapping “inaudible” → “unintelligible” improved noisy input handling. [...]

Convert non-text rules to text: The model responds better to clearly written text.

Example: Instead of writing, "IF x > 3 THEN ESCALATE", write, "IF MORE THAN THREE FAILURES THEN ESCALATE."

There are a whole lot more prompting tips in the new Realtime Prompting Guide.

OpenAI list several key improvements to gpt-realtime including the ability to configure it with a list of MCP servers, "better instruction following" and the ability to send it images.

My biggest confusion came from the pricing page, which lists separate pricing for using the Realtime API with gpt-realtime and GPT-4o mini. This suggests to me that the old gpt-4o-mini-realtime-preview model is still available, despite it no longer being listed on the OpenAI models page.

gpt-4o-mini-realtime-preview is a lot cheaper:

Model	Token Type	Input	Cached Input	Output
gpt-realtime	Text	$4.00	$0.40	$16.00
	Audio	$32.00	$0.40	$64.00
	Image	$5.00	$0.50	-
gpt-4o-mini-realtime-preview	Text	$0.60	$0.30	$2.40
gpt-4o-mini-realtime-preview	Audio	$10.00	$0.30	$20.00

The mini model also has a much longer 128,000 token context window.

Update: Turns out that was a mistake in the documentation, that mini model has a 16,000 token context size.

Update 2: OpenAI's Peter Bakkum clarifies:

There are different voice models in API and ChatGPT, but they share some recent improvements. The voices are also different.

gpt-realtime has a mix of data specific enough to itself that its not really 4o or 5

Tags: audio, realtime, ai, openai, generative-ai, llms, llm-pricing, multi-modal-output, llm-release

twitter-to-sqlite 0.6, with track and follow

2019-10-06T04:54:33+00:00

twitter-to-sqlite 0.6, with track and follow

I shipped a new release of my twitter-to-sqlite command-line tool this evening. It now includes experimental features for subscribing to the Twitter streaming API: you can track keywords or follow users and matching Tweets will be written to a SQLite database in real-time as they come in through the API. Since Datasette supports mutable databases now you can run Datasette against the database and run queries against the tweets as they are inserted into the tables.

Tags: projects, realtime, twitter, dogsheep

Realtime Election Tweets

2010-05-06T21:20:00+00:00

Realtime Election Tweets

Jay Caines-Gooby’s realtime election tweet service, using Node.js, nginx and WebSocket with a Flash fallback.

Tags: comet, flash, javascript, nodejs, realtime, twitter, websockets, recovered, election, jay-caines-gooby

Quoting Neil Fraser

2009-08-14T10:38:57+00:00

Last night I woke up at 2am and realized that there was a fundamental problem with cursor preservation in today’s real-time collaborative applications [...] MobWrite now has what I believe to be the most advanced cursor preservation algorithm available.

— Neil Fraser

Tags: collaboration, mobwrite, neil-fraser, realtime

The Pushbutton Web: Realtime Becomes Real

2009-07-24T18:30:01+00:00

The Pushbutton Web: Realtime Becomes Real

Anil Dash is excited by the potential for PubSubHubBub and Webhooks to make near-real-time scalable event publishing accessible to regular web developers. So am I.

Tags: anil-dash, pubsubhubbub, pushbutton, realtime, realtimeweb, webhooks

EtherPad

2009-07-24T00:35:25+00:00

EtherPad

Outstanding implementation of an online real-time collaborative text editor—basically SubEthaEdit in your browser. I can see myself using this a lot.

Tags: appjet, comet, etherpad, javascript, realtime, subethaedit

pubsubhubbub

2009-04-20T18:49:45+00:00

pubsubhubbub

From Brad Fitzpatrick, a simple but clever way of using web hooks (HTTP callbacks) to inform subscribers that an Atom feed has updated in almost real-time—solving the constant polling problem and making it easier for small sites to offer publish-subscribe APIs. Any Atom feed can delegate subscriber updates to a “hub” server. An example hub server implementation is provided running on App Engine.

Tags: atom, brad-fitzpatrick, google-app-engine, pubsub, pubsubhubbub, python, realtime, webhooks

Giant, Python Powered Robots

2009-01-28T10:35:44+00:00

Giant, Python Powered Robots

“They each weigh about 11 tonnes and have a 17 meter reach. The control system is written in Python, with small sections of C which run in hard-real-time to guarantee safety.”

Tags: awesome, python, realtime, robots

Simple Update Protocol: Update

2008-12-18T23:33:46+00:00

Simple Update Protocol: Update

Already implemented by more than five services, each of which now have near-real-time updates in to the FriendFeed syndication engine.

Tags: friendfeed, realtime, sup, syndication

FriendFeed launch a real-time API

2008-10-22T14:18:56+00:00

FriendFeed launch a real-time API

This is huge: JSONP plus long polling Comet, with “everything since X” tokens to ensure you don’t miss anything. This is the first open Comet API I’ve seen anywhere. Combine this with FriendFeed’s regular API (which allows arbitrary message posting) and you’ve got a really powerful tool for hackers who want to experiment with Comet without rigging up their own infrastructure.

Tags: apis, comet, friendfeed, javascript, json, jsonp, realtime