Simon Willison’s Weblog

Does SQLAlchemy depend on MySQLdb?

No. SQLAlchemy can talk to all sorts of different DB-API compliant backends, including MySQL Connector/J (Jython only), MySQL Connector/Python, mysql-python (the MySQLdb module) and OurSQL—plus backends for many other databases. See the full list here:

Django (web framework): Why did theonion.com stop using Drupal?

They wrote about their reasons in detail in a post to the Django sub-reddit a while ago: http://www.reddit.com/r/django/c...

What is the best way to learn about setting up server software for Python based web apps?

I’m a big fan of Fabric for automated deployment scripts. Start by reading this tutorial: http://morethanseven.net/2009/07...

In what circumstances should one use “magic quotes” in PHP?

Absolutely never. Magic quotes was a badly designed feature, and PHP has been trying to escape its legacy for years. If you are constructing SQL strings using string concatenation you’re asking for trouble—use prepared statements or a library that interpolates and correctly escapes variables for you.

Why do some people disable JavaScript in their browser?

For security reasons.

Which major companies are using Solr for search?

The Guardian newspaper uses Solr for its Open Platform Content API. http://www.guardian.co.uk/open-p...

Which Solr app for Django is better: Haystack or django-solr-search (solango)?

I’d go with Haystack—while it supports multiple backends, I get the feeling Solr is the principle backend it was developed for. It’s extremely well documented in my opinion, and the SearchQuerySet API it gives you makes running low-level queries really easy if the higher level class-based view it provides don’t do quite what you want.

What is the largest production deployment of CouchDB for online use?

The BBC have a pretty big CouchDB cluster, which they use mostly as a replicated key-value store. It’s used by their new identity platform which includes customisation features for iPlayer.

What is the highest traffic website built on top of Django?

My best guess would be Disqus. Instagram are pretty enormous these days as well.

What is the history of the Django web framework? Why has it been described as “developed in a newsroom”?

I was there!

Getting married and going travelling

It’s been a busy month. On Saturday the 5th of June I married the wonderful Natalie Downe in a beautiful ceremony at Roedean School in Brighton. The reception had owls, cheese, a ferret, a golden eagle, amazing Turkish food, Jewish chair dancing and lovely guests. It was the happiest day of my life.

Comprehensive notes from my three hour Redis tutorial

Last week I presented two talks at the inaugural NoSQL Europe conference in London. The first was presented with Matthew Wall and covered the ways in which we have been exploring NoSQL at the Guardian. The second was a three hour workshop on Redis, my favourite piece of software to have the NoSQL label applied to it.

WildlifeNearYou talk at £5 app, and being Wired (not Tired)

Two quick updates about WildlifeNearYou. First up, I gave a talk about the site at £5 app, my favourite Brighton evening event which celebrates side projects and the joy of Making Stuff. I talked about the site’s genesis on a fort, crowdsourcing photo ratings, how we use Freebase and DBpedia and how integrating with Flickr’s machine tags gave us a powerful location API for free. Here’s the video of the talk, courtesy of Ian Oszvald:

Some questions about the “blocking” of HTML5

Some background reading. I was planning to fill in answers as they arrive, but I screwed up the moderation of the comments and got flooded with detailed responses—I strongly recommend reading the comments.

WildlifeNearYou: It began on a fort...

Back in October 2008, myself and 11 others set out on the first /dev/fort expedition. The idea was simple: gather a dozen geeks, rent a fort, take food and laptops and see what we could build in a week.

Crowdsourced document analysis and MP expenses

As you may have heard, the UK government released a fresh batch of MP expenses documents a week ago on Thursday. I spent that week working with a small team at Guardian HQ to prepare for the release. Here’s what we built:

Node.js is genuinely exciting

I gave a talk on Friday at Full Frontal, a new one day JavaScript conference in my home town of Brighton. I ended up throwing away my intended topic (JSONP, APIs and cross-domain security) three days before the event in favour of a technology which first crossed my radar less than two weeks ago.

Why I like Redis

I’ve been getting a lot of useful work done with Redis recently.

This shouldn’t be the image of Hack Day

I love hack days. I was working in the vicinity of Chad Dickerson when he organised the first internal Yahoo! Hack Day back in 2005, and I’ve since participated in hack day events at Yahoo!, Global Radio and the Guardian. I’ve also been to every one of Yahoo!’s Open Hack Day events in London. They’re fantastic, and the team that organises them should be applauded.

Django ponies: Proposals for Django 1.2

I’ve decided to step up my involvement in Django development in the run-up to Django 1.2, so I’m currently going through several years worth of accumulated pony requests figuring out which ones are worth advocating for. I’m also ensuring I have the code to back them up—my innocent AutoEscaping proposal a few years ago resulted in an enormous amount of work by Malcolm and I don’t think he’d appreciate a repeat performance.

Hack Day tools for non-developers

We’re about to run our second internal hack day at the Guardian. The first was an enormous amount of fun and the second one looks set to be even more productive.

Teaching users to be secure is a shared responsibility

Ryan Janssen: Why an OAuth iframe is a Great Idea.

Facebook Usernames and OpenID

Today’s launch of Facebook Usernames provides an obvious and exciting opportunity for Facebook to become an OpenID provider. Facebook have clearly demonstrated their interest in becoming the key online identity for their users, and the new usernames feature is their acknowledgement that URL-based identities are an important component of that, no doubt driven in part by Twitter making usernames trendy again.

djng—a Django powered microframework

djng is nearly two weeks old now, so it’s about time I wrote a bit about the project.

rev=canonical bookmarklet and designing shorter URLs

I’ve watched the proliferation of URL shortening services over the past year with a certain amount of dismay. I care about the health of the web and try to ensure that URLs I am responsible will last for as long as possible, and I think it’s very unlikely that all of these new services will still be around in twenty years time. Last month I suggested that the Internet Archive start mirroring redirect databases, and last week I was pleased to hear that Archiveteam, a different organisation, had already started crawling.

List of SxSW 2009 panels with “social” in the title

• A Hard Sell? Social Media & Your Boss
• Can Social Media End Racism?
• Digital Urbanites: How To Become Part of the New Social Capital
• The Future Of Social Networks
• How Social Networks Are Killing the Revolution
• Making Whuffie: Raising Social Capital in Online Communities
• The Mix at Six Hosted by Social Media Group
• Mobile Social SXSW BBQ
• My Boss Doesn’t Get It: Championing Social Media to the Man
• PBS’ Interactive Social Media & Online Video Studio
• The Search for a More Social Web
• Security for the Social Set
• Social Engineering: Scam Your Way Into Anything or From Anybody
• Social Gamers: Away From the Keyboard
• Social Media For Social Good
• Social Media Marketing
• Social Media Marketing: An Hour a Day
• Social Media Nonprofit ROI Poetry Slam
• Social Media: If You Liked it, Then You Should Have Put a Digg on it...
• Social Networking in Health: e-Patients, Data & Privacy
• Social Patterns and Antipatterns For the Win
• Suxorz ’09: The Ten Worst Social Media Campaigns
• Twitter for Marketers: Is It Still Social Media?
• Using GPS & Location to Enhance Social Networking
• Using the New Digital Social Media to Accelerate Sustainability

A few notes on the Guardian Open Platform

This morning we launched the Guardian Open Platform at a well attended event in our new offices in Kings Place. This is one of the main projects I’ve been helping out with since joining the Guardian last year, and it’s fantastic to finally have it out in the open.

Pragmatism, purity and JSON content types

I started a conversation about this on Twitter the other day, but Twitter is a horrible place to have an archived discussion so I’m going to try again here.

Rate limiting with memcached

On Monday, several high profile “celebrity” Twitter accounts started spouting nonsense, the victims of stolen passwords. Wired has the full story—someone ran a dictionary attack against a Twitter staff member, discovered their password and used Twitter’s admin tools to reset the passwords on the accounts they wanted to steal.

DjangoCon and PyCon UK

September is a big month for conferences. DjangoCon was a weekend ago in Mountain View (forcing me to miss both d.Construct and BarCamp Brighton), PyCon UK was this weekend in Birmingham, I’m writing this from @media Ajax and BarCamp London 5 is coming up over another weekend at the end of this month. As always, I’ve been posting details of upcoming talks and notes and materials from previous ones on my talks page.