Simon Willison's Weblog: workflow

What's the best way to keep track of changes to a project you're not directly contributing to on github?

2014-01-23T17:48:00+00:00

My answer to What's the best way to keep track of changes to a project you're not directly contributing to on github? on Quora

This is what GitHub's "watch" feature is for: https://help.github.com/articles...

Tags: github, open-source, project-management, quora, workflow

What are optimal workflows for deploying one's web application?

2012-01-14T12:40:00+00:00

My answer to What are optimal workflows for deploying one's web application? on Quora

The absolute first step is to automate your deployments. It's absolutely crucial that deploying the site is a single command. I've found Fabric (an automation tool written in Python) works extremely well for this - Capistrano is a popular alternative that uses Ruby instead.

A simple policy that can help is this: never run a command on your remote server without putting it in a Fabric file first. This way your deploy and setups will always be repeatable, with almost no additional effort on your part.

At Lanyrd our deployment process has evolved significantly over the past 18 months - our most recent setup, designed by our engineer Tom Insam, is pretty much my ideal situation. Here's (roughly) how it works.

Our servers run on EC2 and are configured with Puppet. The Puppet roles are assigned as EC2 tags, which means we can start a new EC2 instance, assign tags to it (e.g. application-server, redis, cron-runner) and Puppet will do the rest of the setup.

When we deploy, we specify an environment (e.g. development, staging or live) and a git branch or tag (default is master). Setting up our staging environment is trivial because of our Puppet-powered server configs - this also means makes it a no-brainer to turn it off when we're not using it (e.g. over the weekend) to save on EC2 server costs.

We also have Jenkins set up to run our test suite every time someone commits to our git repository. Finally, we have a convenient "deploy master to live" button (we try to keep master shippable at all time, developing potentially site breaking features in a feature branch).

There's a fair bet more to it than the above - we have the ability to roll back to a previous version by flipping a symlink should we break the site, and there's some extra stuff to handle pushing static assets to S3 and running database migrations. Overall it's an enormous improvement on my previous hacked-together fabric scripts, which were themselves a huge improvement over ad-hoc deploys via mucking around with SSH.

Tags: webapps, quora, workflow

What does an ideal Django workflow setup look like?

2010-10-22T11:26:00+00:00

My answer to What does an ideal Django workflow setup look like? on Quora

Short answer: virtualenv, pip, south for migrations, fabric for deployment.

Long answer: I haven't seen the perfect tutorial covering all of the above yet, which is a shame, but here are some good links:

http://morethanseven.net/2009/07...
http://www.saltycrane.com/blog/2...
http://www.clemesha.org/blog/mod...
http://www.caktusgroup.com/blog/...

Tags: django, quora, workflow

Getting from point A to B (the right way)

2007-10-11T14:30:00+00:00

My answer to Getting from point A to B (the right way) on Ask MetaFilter

If your laptop is relatively recent it might have hardware support for virtualization (Intel Core Duo chips do, for example). If so, it's worth looking in to using VMWare or Parallels to run a virtual linux server locally on your machine. You'll need a fair amount of RAM for this as well - 2 GB minimum probably.

I do this and it's fantastic as a development tool. I've got an Ubuntu virtual server which means I can install pretty much anything I want to with an apt-get - then I mount it to my "real" laptop over Samba so I can edit files with a local text editor. Best of both worlds.

I can't recommend using source control enough - it's not that tricky to get started with and it means you'll never be afraid of changing code again (since you can always roll back). I'd go for Subversion - newer systems are more trendy, but for a single person getting started with version control subversion will probably be easiest to pick up. This book should help: http://www.pragprog.com/titles/svn2

Tags: ask-metafilter, development, mysql, php, platform, web, workflow, work, efficiency, optimization