Simon Willison's Weblog: kellan-elliott-mccrea

Quoting Kellan Elliott-McCrea

2026-02-25T03:30:32+00:00

It’s also reasonable for people who entered technology in the last couple of decades because it was good job, or because they enjoyed coding to look at this moment with a real feeling of loss. That feeling of loss though can be hard to understand emotionally for people my age who entered tech because we were addicted to feeling of agency it gave us. The web was objectively awful as a technology, and genuinely amazing, and nobody got into it because programming in Perl was somehow aesthetically delightful.

— Kellan Elliott-McCrea, Code has always been the easy part

Tags: perl, generative-ai, kellan-elliott-mccrea, agentic-engineering, ai, llms, deep-blue

Quoting Kellan Elliott-McCrea

2025-03-02T14:16:32+00:00

Regarding the recent blog post, I think a simpler explanation is that hallucinating a non-existent library is a such an inhuman error it throws people. A human making such an error would be almost unforgivably careless.

— Kellan Elliott-McCrea

Tags: ai-assisted-programming, generative-ai, kellan-elliott-mccrea, ai, llms

A Link Blog in the Year 2024

2024-06-09T00:10:45+00:00

A Link Blog in the Year 2024

Kellan Elliott-McCrea has started a new link blog:

Like many people I’ve been dealing with the collapses of the various systems I relied on for information over the previous decades. After 17 of using Twitter daily and 24 years of using Google daily neither really works anymore. And particular with the collapse of the social spaces many of us grew up with, I feel called back to earlier forms of the Internet, like blogs, and in particular, starting a link blog.

I've been leaning way more into link blogging over the last few months, especially now my own link blog supports markdown. This means I'm posting longer entries, somewhat inspired by Daring Fireball (my own favourite link blog to read).

Link blogging is a pleasantly low-pressure way of writing online. Found something interesting? Post a link to it, with a sentence or two about why it's worth checking out.

I'd love to see more people embrace this form of personal publishing.

Via @kellan

Tags: blogging, kellan-elliott-mccrea

Quoting Kellan Elliott-McCrea

2024-06-09T00:08:12+00:00

Much like Gen X is sometimes the forgotten generation (or at least we feel that way), the generation of us who grew up with an internet that seemed an unalloyed good fall awkwardly into the middle between those who didn’t grow up with it, and those for whom there has always been the whiff of brimstone, greed, and ruin around the place.

— Kellan Elliott-McCrea

Tags: kellan-elliott-mccrea

Quoting Kellan Elliott-McCrea

2023-02-27T20:12:10+00:00

Just a reminder, the way you evaluate yourself as a leader is how much both the individuals and teams in your organization grow in their capacity to achieve hard goals. Everything else is a distraction.

— Kellan Elliott-McCrea

Tags: kellan-elliott-mccrea, management, leadership

Quoting Kellan Elliott-McCrea

2023-01-24T05:26:47+00:00

Large teams spend more time dealing with coordination and are more likely to reach for architecture and abstractions that they hope will reduce coordination costs, aka if I architect this well enough I don’t have to speak to my colleagues. Microservices, event buses, and schema free databases are all examples of attempts to architect our way around coordination. A decade in we’ve learned that these patterns raise the cost of reasoning about a system, during onboarding, during design, and during incidents and outages.

— Kellan Elliott-McCrea

Tags: kellan-elliott-mccrea, software-architecture, microservices

Building Layoffs on a Healthy Foundation

2022-09-01T18:11:57+00:00

Building Layoffs on a Healthy Foundation

Kellan provides some valuable guidance for running layoffs in as humane a way as possible.

Via @kellan

Tags: kellan-elliott-mccrea, management

Dropbox: Sharing our Engineering Career Framework with the world

2021-07-13T23:31:48+00:00

Dropbox: Sharing our Engineering Career Framework with the world

Dropbox have published their engineering career framework, with detailed descriptions of the different levels of the engineering (as opposed to management) career track and what is expected for each one. I’m fascinated by how different companies handle the challenge of keeping career progression working for engineers without pushing them into people management, and this as a particularly detailed and well thought-out implementation of that.

Via @kellan

Tags: kellan-elliott-mccrea, careers, dropbox

Friday wins and a case study in ritual design

2019-06-08T18:14:57+00:00

Friday wins and a case study in ritual design

“Culture is what you celebrate. Rituals are the tools you use to shape culture.”

Tags: kellan-elliott-mccrea, management

Questions for a new technology

2019-02-06T04:10:08+00:00

Questions for a new technology

Kellan poses 8 questions which should be asked of any technology that is being proposed for inclusion in an existing tech stack. I’m particularly fond of “Will this solution kill and eat the solution that it replaces?”. My rule of thumb these days is that new technology either needs to make something possible that isn’t possible at all with the existing stack, or it needs to represent at least a 3X productivity improvement in order to compensate for the switching and retraining costs across a large team.

Tags: kellan-elliott-mccrea

Quoting Kellan Elliott-McCrea

2018-03-10T01:11:19+00:00

I’m still a novice to the healthcare space, but if I walked away with a single insight, it’s that the problems of the US healthcare system are very tractable. The high cost and mixed results are unique to our system. There are incumbents fighting fiercely to maintain the status quo, but no more so than in other industries that technology has overturned. The regulatory environment is complex, but again not uniquely so. There are industries where one has to dig to find the problems that technology is well suited to solve, but US healthcare, an industry that communicates via fax, is not one of them.

— Kellan Elliott-McCrea

Tags: kellan-elliott-mccrea, healthcare

Tip: Flickr standard photo response as slideshow

2011-01-25T03:51:00+00:00

Tip: Flickr standard photo response as slideshow

Neat trick—you can construct a URL to Flickr’s slideshow widget that includes the results of any API method, including the all-powerful flickr.photos.search. It’s a shame you can’t embed the resulting slideshow in an iframe.

Tags: apis, flickr, kellan-elliott-mccrea, widgets, recovered

Quoting Kellan Elliott-McCrea

2010-05-18T18:21:00+00:00

With Flickr you can get out, via the API, every single piece of information you put into the system. [...] Asking people to accept anything else is sharecropping. It’s a bad deal. Flickr helped pioneer “Web 2.0″, and personal data ownership is a key piece of that vision. Just because the wider public hasn’t caught on yet to all the nuances around data access, data privacy, data ownership, and data fidelity, doesn’t mean you shouldn’t be embarrassed to be failing to deliver a quality product.

— Kellan Elliott-McCrea

Tags: data, flickr, kellan-elliott-mccrea, sharecropping, web20, recovered

Quoting Kellan Elliott-McCrea

2010-01-25T08:11:02+00:00

If you’re ever debugging a problem and you see the number 42-mumble-mumble-mumble-7295 you’ve run out of 32-bit storage. If you see 2-mumble-mumble-mumble-647 (2147483647) you’ve run out of signed 32-bit storage. 167-mumble-mumble-15 (16777215) you’ve run out of 24-bits and 65-mumble-mumble-35 (65535) you’ve run out of 16-bits of integers.

— Kellan Elliott-McCrea

Tags: kellan-elliott-mccrea, mysql, integers

Quoting Kellan Elliott-McCrea

2009-04-12T16:00:41+00:00

We’re using the same trick on flic.kr to avoid having to maintain a look up database, though we’re using base 58.

— Kellan Elliott-McCrea

Tags: flickr, revcanonical, urls, base58, kellan-elliott-mccrea

rev=canonical bookmarklet and designing shorter URLs

2009-04-11T17:37:55+00:00

I've watched the proliferation of URL shortening services over the past year with a certain amount of dismay. I care about the health of the web and try to ensure that URLs I am responsible will last for as long as possible, and I think it's very unlikely that all of these new services will still be around in twenty years time. Last month I suggested that the Internet Archive start mirroring redirect databases, and last week I was pleased to hear that Archiveteam, a different organisation, had already started crawling.

The most recent discussion was kicked off by Joshua Schachter and Dave Winer, and a solution has emerged driven by some lightning fast hacking by Kellan Elliott-McCrea. The idea is simple: sites get to chose their preferred source of shortened URLs (including self-hosted solutions) and specify it from individual pages using <link rev="canonical" href="... shorter URL here ...">.

By hosting their own shorteners, the reliability should match that of the host site - and the amount of damage caused by a major shortener going missing can be dramatically reduced.

I've been experimenting with this new pattern today. Here are a few small contributions to the wider discussion.

A URL shortening bookmarklet

Kellan's rev=canonical service exposes rev=canonical links using a server-side script running on App Engine. An obvious next step is to distil that logic in to a bookmarklet. I decided to combine the rev=canonical logic with my json-tinyurl web service (also on App Engine), which allows browsers to lookup or create TinyURLs using a cross-domain JSONP request. The resulting bookmarklet will display the site's rev=canonical link if it exists, or create and display a TinyURL link otherwise:

Bookmarklet: Shorten (drag to your browser toolbar)

You can also grab the uncompressed source code.

Designing short URLs

I've also implemented rev=canonical on this site. I ended up buying a new domain for this, since simonwillison.net is both difficult to spell and 17 characters long. I ended up going with swtiny.eu - 9 characters, and keeping tiny in the domain helps people guess the nature of the site from just the URLs it generates. Be warned: the DNS doesn't appear to have finished resolving yet.

For the path component, I turned to a variant of base 62 encoding. Decimal integers are represented using 10 digits (0-9), but base 62 uses those digits plus the letters of the alphabet in both lower and upper case. A 13 character integer such as 7250397214971 compresses down to just 8 characters (CDeIPpOD) using base62. My baseconv.py module implements base62, among others. I considered using base 57 by excluding o, O, 0, 1 and l as being too easily confused but decided against it.

This site has three key types of content: entries, blogmarks and quotations. Each one is a separate Django model, and hence each has its own underlying database table and individual ID sequence. Since the IDs overlap, I need a way of separating out the shortened URLs for each content type.

I decided to spend a byte on namespacing my shortened URLs. A prefix of E means an entry, Q means a quotation and B means a blogmark. For example:

http://swtiny.eu/EZ8: Entry with ID 1584
http://swtiny.eu/BBEQ: Blogmark with ID 4108
http://swtiny.eu/QE5: Quotation with ID 279

By using upper case letters for the prefixes, I can later define custom paths starting with a lower case letter. I also have another 23 upper case prefix letters reserved in case I need them.

I asked on Twitter and consensus opinion was that a 301 permanent redirect was the right thing to do (as opposed to a 302), both for SEO reasons and because the content will never exist at the shorter URL.

Implementation using Django and nginx

I run all of my Django sites using Apache and mod_wsgi, proxied behind nginx. Each site gets an Apache running on a high port, and nginx deals with virtual host configuration (proxying each domain to a different Apache backend) and static file serving. I didn't want to set up a full Django site just to run swtiny.eu, especially since my existing blog engine was required in order to resolve the shortened URLs.

Instead, I implemented the shortened URL direction as just another view within my existing site: http://simonwillison.net/shorter/EZ8. I then configured nginx to invisibly requests to swtiny.eu through to that URL. The correct incantation took a while to figure out, so here's the relevant section of my nginx.conf:

server {
    listen 80;
    server_name www.swtiny.eu swtiny.eu;
    location / {
        rewrite (.*) /shorter$1 break;
        proxy_pass http://simonwillison.net;
        proxy_redirect off;
    }
}

proxy_redirect off is needed to prevent nginx from replacing simonwillison.net in the resulting location header with swtiny.eu. My Django view code is relatively shonky, but if you're interested you can find it here.

The nice thing about this approach is that it makes it trivial to add custom URL shortening domains to other projects - a quick view function and a few lines of nginx configuration are all that is needed.

Update: The bookmarklet now supports the rev attribute on A elements as well - thanks for the suggestion, Jeremy.

Tags: bookmarklets, dave-winer, django, joshua-schachter, kellan-elliott-mccrea, projects, python, revcanonical, tinyurl, urls

Streams, affordances, Facebook, and rounding errors

2009-03-19T14:02:31+00:00

Streams, affordances, Facebook, and rounding errors

I asked Kellan about scaling activity streams the other day. Here he suggests the best technique is not to promise a perfect stream (like Twitter does)—Facebook used to get away with 80% loss of update messages, but their new redesign has changed the contract with their users.

Tags: activitystreams, facebook, kellan-elliott-mccrea, scaling, twitter

Quoting Kellan Elliott-McCrea

2008-09-29T15:29:34+00:00

The only down side is everyone I’ve talked to at Freebase seems pretty solid on this being their proprietary secret sauce, because a good, fast scalable open source tuple store might actually jump start a real semantic (small-S) web after all these years.

— Kellan Elliott-McCrea

Tags: kellan-elliott-mccrea, freebase, open-source, proprietary, graphd, semanticweb

Twitter, or Architecture Will Not Save You

2008-05-29T01:16:04+00:00

Twitter, or Architecture Will Not Save You

Kellan is not an armchair architect. He also doesn’t mention Rails once. Well worth reading.

Tags: architecture, kellan-elliott-mccrea, rails, twitter

Flickr Place IDs

2008-01-19T07:34:38+00:00

Flickr Place IDs

flickr.places.find, flickr.places.resolvePlaceURL and flickr.places.resolvePlaceID combine to provide a really useful, lightweight not-quite-a-geocoder API. It's a shame you can't search for places by providing a latitude/longitude point yet.

Tags: api, flickr, flickrplaces, geocoding, geospatial, kellan-elliott-mccrea

Crowdvine, iCalico, Pathable, a Study in Collusion

2007-07-12T23:09:27+00:00

Crowdvine, iCalico, Pathable, a Study in Collusion

Stitching sites together around a single user database using subdomains and simple signed cookies.

Tags: collusion, crowdvine, foocamp, icalico, kellan-elliott-mccrea, pathable, signedcookies, sso

PHP Library Tips

2003-08-13T18:11:58+00:00

Kellan Elliott-McCrea (author of the popular Magpie RSS parser): A Few Tips for Writing Useful Libraries in PHP. Kellan makes the interesting observation that PHP encourages a culture in which most development occurs in the context of either full applications or C extensions, with few people devoting themselves to releasing libraries.

Tags: kellan-elliott-mccrea, php, rss