Simon Willison's Weblog: productivity

Quoting Daniel Lemire

2025-12-06T14:40:46+00:00

If you work slowly, you will be more likely to stick with your slightly obsolete work. You know that professor who spent seven years preparing lecture notes twenty years ago? He is not going to throw them away and start again, as that would be a new seven-year project. So he will keep teaching using aging lecture notes until he retires and someone finally updates the course.

— Daniel Lemire, Why speed matters

Tags: productivity

Quoting Aaron Boodman

2025-10-28T02:08:57+00:00

Claude doesn't make me much faster on the work that I am an expert on. Maybe 15-20% depending on the day.

It's the work that I don't know how to do and would have to research. Or the grunge work I don't even want to do. On this it is hard to even put a number on. Many of the projects I do with Claude day to day I just wouldn't have done at all pre-Claude.

Infinity% improvement in productivity on those.

— Aaron Boodman

Tags: ai-assisted-programming, claude, generative-ai, ai, llms, aaron-boodman, productivity

Quoting Kate Niederhoffer, Gabriella Rosen Kellerman, Angela Lee, Alex Liebscher, Kristina Rapuano and Jeffrey T. Hancock

2025-09-22T23:21:49+00:00

We define workslop as AI generated work content that masquerades as good work, but lacks the substance to meaningfully advance a given task.

Here’s how this happens. As AI tools become more accessible, workers are increasingly able to quickly produce polished output: well-formatted slides, long, structured reports, seemingly articulate summaries of academic papers by non-experts, and usable code. But while some employees are using this ability to polish good work, others use it to create content that is actually unhelpful, incomplete, or missing crucial context about the project at hand. The insidious effect of workslop is that it shifts the burden of the work downstream, requiring the receiver to interpret, correct, or redo the work. In other words, it transfers the effort from creator to receiver.

— Kate Niederhoffer, Gabriella Rosen Kellerman, Angela Lee, Alex Liebscher, Kristina Rapuano and Jeffrey T. Hancock, Harvard Business Review

Tags: ai-misuse, productivity, ai-ethics, generative-ai, ai, llms, definitions

Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity

2025-07-12T18:12:23+00:00

Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity

METR - for Model Evaluation & Threat Research - are a non-profit research institute founded by Beth Barnes, a former alignment researcher at OpenAI (see Wikipedia). They've previously contributed to system cards for OpenAI and Anthropic, but this new research represents a slightly different direction for them:

We conduct a randomized controlled trial (RCT) to understand how early-2025 AI tools affect the productivity of experienced open-source developers working on their own repositories. Surprisingly, we find that when developers use AI tools, they take 19% longer than without—AI makes them slower.

The full paper (PDF) has a lot of details that are missing from the linked summary.

METR recruited 16 experienced open source developers for their study, with varying levels of exposure to LLM tools. They then assigned them tasks from their own open source projects, randomly assigning whether AI was allowed or not allowed for each of those tasks.

They found a surprising difference between developer estimates and actual completion times:

After completing the study, developers estimate that allowing AI reduced completion time by 20%. Surprisingly, we find that allowing AI actually increases completion time by 19%—AI tooling slowed developers down.

I shared my initial intuition about this paper on Hacker News the other day:

My personal theory is that getting a significant productivity boost from LLM assistance and AI tools has a much steeper learning curve than most people expect.

This study had 16 participants, with a mix of previous exposure to AI tools - 56% of them had never used Cursor before, and the study was mainly about Cursor.

They then had those 16 participants work on issues (about 15 each), where each issue was randomly assigned a "you can use AI" v.s. "you can't use AI" rule.

So each developer worked on a mix of AI-tasks and no-AI-tasks during the study.

A quarter of the participants saw increased performance, 3/4 saw reduced performance.

One of the top performers for AI was also someone with the most previous Cursor experience. The paper acknowledges that here:

However, we see positive speedup for the one developer who has more than 50 hours of Cursor experience, so it's plausible that there is a high skill ceiling for using Cursor, such that developers with significant experience see positive speedup.

My intuition here is that this study mainly demonstrated that the learning curve on AI-assisted development is high enough that asking developers to bake it into their existing workflows reduces their performance while they climb that learing curve.

I got an insightful reply there from Nate Rush, one of the authors of the study, which included these notes:

Some prior studies that find speedup do so with developers that have similar (or less!) experience with the tools they use. In other words, the "steep learning curve" theory doesn't differentially explain our results vs. other results.

Prior to the study, 90+% of developers had reasonable experience prompting LLMs. Before we found slowdown, this was the only concern that most external reviewers had about experience was about prompting -- as prompting was considered the primary skill. In general, the standard wisdom was/is Cursor is very easy to pick up if you're used to VSCode, which most developers used prior to the study.

Imagine all these developers had a TON of AI experience. One thing this might do is make them worse programmers when not using AI (relatable, at least for me), which in turn would raise the speedup we find (but not because AI was better, but just because with AI is much worse). In other words, we're sorta in between a rock and a hard place here -- it's just plain hard to figure out what the right baseline should be!

We shared information on developer prior experience with expert forecasters. Even with this information, forecasters were still dramatically over-optimistic about speedup.

As you say, it's totally possible that there is a long-tail of skills to using these tools -- things you only pick up and realize after hundreds of hours of usage. Our study doesn't really speak to this. I'd be excited for future literature to explore this more.

In general, these results being surprising makes it easy to read the paper, find one factor that resonates, and conclude "ah, this one factor probably just explains slowdown." My guess: there is no one factor -- there's a bunch of factors that contribute to this result -- at least 5 seem likely, and at least 9 we can't rule out (see the factors table on page 11).

Here's their table of the most likely factors:

I think Nate's right that jumping straight to a conclusion about a single factor is a shallow and unproductive way to think about this report.

That said, I can't resist the temptation to do exactly that! The factor that stands out most to me is that these developers were all working in repositories they have a deep understanding of already, presumably on non-trivial issues since any trivial issues are likely to have been resolved in the past.

I think this is a really interesting paper. Measuring developer productivity is notoriously difficult. I hope this paper inspires more work with a similar level of detail to analyzing how professional programmers spend their time:

To compare how developers spend their time with and without AI assistance, we manually label a subset of 128 screen recordings with fine-grained activity labels, totaling 143 hours of video.

Via Hacker News

Tags: open-source, productivity, ai, generative-ai, llms, ai-assisted-programming, paper-review

Using LLMs and Cursor to become a finisher

2025-01-04T20:56:39+00:00

Using LLMs and Cursor to become a finisher

Zohaib Rauf describes a pattern I've seen quite a few examples of now: engineers who moved into management but now find themselves able to ship working code again (at least for their side projects) thanks to the productivity boost they get from leaning on LLMs.

Zohaib also provides a very useful detailed example of how they use a combination of ChatGPT and Cursor to work on projects, by starting with a spec created through collaboration with o1, then saving that as a SPEC.md Markdown file and adding that to Cursor's context in order to work on the actual implementation.

Via Hacker News

Tags: productivity, ai, generative-ai, chatgpt, llms, ai-assisted-programming, o1, cursor

Supercharging Developer Productivity with ChatGPT and Claude with Simon Willison

2024-09-17T16:21:22+00:00

Supercharging Developer Productivity with ChatGPT and Claude with Simon Willison

I'm the guest for the latest episode of the TWIML AI podcast - This Week in Machine Learning & AI, hosted by Sam Charrington.

We mainly talked about how I use LLM tooling for my own work - Claude, ChatGPT, Code Interpreter, Claude Artifacts, LLM and GitHub Copilot - plus a bit about my experiments with local models.

Via @twimlai

Tags: podcasts, productivity, ai, generative-ai, chatgpt, claude, podcast-appearances

Quoting Evan Hahn

2024-07-01T20:42:43+00:00

When presented with a difficult task, I ask myself: “what if I didn’t do this at all?”. Most of the time, this is a stupid question, and I have to do the thing. But ~5% of the time, I realize that I can completely skip some work.

— Evan Hahn

Tags: productivity, programming

Shell History Is Your Best Productivity Tool

2024-04-10T23:17:59+00:00

Shell History Is Your Best Productivity Tool

Martin Heinz drops a wealth of knowledge about ways to configure zsh (the default shell on macOS these days) to get better utility from your shell history.

Via lobste.rs

Tags: productivity, shell, zsh

Advanced Topics in Reminders and To Do Lists

2024-03-15T02:38:55+00:00

Advanced Topics in Reminders and To Do Lists

Fred Benenson’s advanced guide to the Apple Reminders ecosystem. I live my life by Reminders—I particularly like that you can set them with Siri, so “Hey Siri, remind me to check the chickens made it to bed at 7pm every evening” sets up a recurring reminder without having to fiddle around in the UI. Fred has some useful tips here I hadn’t seen before.

Tags: productivity

Tom Scott, and the formidable power of escalating streaks

2024-01-02T20:32:08+00:00

Ten years ago yesterday, Tom Scott posted this video to YouTube about "Special Crossings For Horses In Britain". It was the first in his Things You Might Not Know series, but more importantly it was the start of a streak.

Tom maintained a streak of posting a video approximately once a week for the next ten years.

Yesterday, he ended that streak with After ten years, it's time to stop making videos. He's not done with YouTube, but he's no longer holding himself to that intimidating weekly schedule.

I strongly recommend watching his final video. There's a moment when you realize what he's up to in it which is quite delightful.

I've known Tom for a long time. I made an appearance in the 11th "Things You Might Not Know" video, A Zeppelin, A Cat, and The World's First In-Flight Radio Message, two weeks into his streak (he was doing one a day at first), filmed at our leaving-the-UK-for-the-USA party in January 2014.

Watching from afar has been somewhat surreal. I didn't watch every video, but every now and then I'd see that Tom was flying with the Red Arrows, or visiting yet another nuclear reactor site, or overcoming his fear of rollercoasters. And then I'd notice that he'd picked up another million subscribers.

Hanging out with Tom was fun because he would inevitably be recognised by someone. 6.3 million subscribers is a lot of people!

Tom's success on YouTube comes down to a whole bunch of different factors. He was already a talented public speaker, a skilled researcher, had a brief stint as TV presenter and deep understanding of the viral internet.

Experienced YouTubers will tell you that frequency is key to success on that platform. YouTube's audience (and maybe their opaque algorithm) rewards consistency: publishing regularly is a crucial part of building an audience.

Tom is also incredibly conscientious about the content he produces. Take a look at his corrections and clarifications page to see how much effort he puts into getting things right: 25 detailed corrections across over 500 videos. See also his recent video Every mistake I've made since 2014.

His most significant correction became a whole new video clarifying how London fire brigades handled uninsured buildings in the 18th century, backed by two weeks of paid research by an archives and heritage research consultant. His commitment to accessibility is inspiring as well.

And then there was the streak.

Escalating streaks

The best way to get really good at anything is to do that thing on a regular basis, thoughtfully, and with the goal of doing it slightly better every time.

Tom's streak publishing a video to YouTube once a week for ten years is the single best illustration I've ever seen of that principle in action.

His initial videos were interesting, educational and had his signature enthusiastic energy, but they weren't exactly high budget affairs.

As he iterated on the format, he started to figure out what worked. His scripts got tighter, his research deeper and he started working with professionals to improve his production values.

He also learned to use his growing audience to gain access to a dizzying array of fascinating locations, experts and experiences.

The amount of work he invested in this project is staggering. The research, logistics, travel, writing, filming, editing and community management involved are hard for me to even comprehend.

The end result is something truly extraordinary. What a legacy! That final video has over 42,000 comments already, overwhelmingly thankful and positive.

Streaks can be insidious

In Tom's closing video he says:

So now it’s time to take a breather. I can’t keep this up. This is my dream job, and I have a lot of fun doing it. I know I’m incredibly lucky. But a dream job is still a job. And it’s a job that keeps getting bigger and more complicated and I am so tired! There’s nothing in my life right now except work. I did get close to burning out, but fortunately I always knew when to step back from the brink.

Streaks are a powerful psychological tool. Once Tom got to nine years, there was no way he wasn't going to push through to ten. I'm glad for his sake that in hitting that final milestone he's finally able to take a break!

My own experience with streaks

I've found great benefit from streaks myself. I'm on day 1,826 (that's 5 years yesterday) of a Duolingo streak, primarily learning Spanish. It's kind of working - from an investment of less than 15 minutes a day I'm now able to understand ~90% of news articles written in that language.

There are certainly more effective ways to learn a language, but I've tried different approaches in the past and nothing ever stuck for me to the point that I made real progress.

It turns out the streak mechanism was exactly what I needed. That tiny piece of effort, repeated every day over multiple years, really does add up.

I'm also 172 entries into my streak of publishing weeknotes - not-quite-weekly (more at-least-monthly) posts about what I've been doing, which I use mainly as an accountability tool to keep myself on track despite working independently without any form of boss.

A few years ago I started a website about tiny museums I have been to. I used streak pressure to bootstrap the site: I added a museum once a day for a hundred days, digging through old photos and memories.

My streaks are noway near the same league as Tom's. That's why I introduced the term escalating streaks earlier in this post - to emphasize that the true magic comes when you mindfully improve with every iteration.

I did however notice that by the end of my 100 day museum streak I was writing significantly higher quality articles than when I first started.

Flexibility and forgiveness is crucial

Streaks have multiple dangers. At one extreme, they can take over your life, forcing you to leave home behind and spend a decade traveling the world making increasingly brilliant YouTube videos.

The other challenge is what happens when you accidentally break them.

In the past, I've tried my hand at strict streaks... and then found that 100 days in I miss a day, and suddenly I'm reset to zero and I lose all motivation to continue.

The solution here is to build in some flexibility. I started a new streak recently to reply to at least one email every day, to encourage me to spend more time in my inbox. My goal for this is four out of seven days, so I can miss three days a week and still keep the streak going.

Duolingo has a "streak freeze" mechanism which can be used to forgive the occasional mishap, which I'm happy to take advantage of.

Initially I felt like this was "cheating", but it really isn't. Streaks are a powerful motivational tool if you figure out the best way to apply them.

The Tom Scott Streak

Three of my biggest inspirations in life are these:

The movie Into the Spider-Verse, demonstrating what happens when a group of creative people get together, rewrite the rules and elevate the quality bar for an entire industry.
Tom Holland's "Umbrella" performance on Lip Sync Battle, showing what happens when someone takes an opportunity and executes it with such skill, enthusiasm and panache that people are still talking about it six years later.
Ray Bandar's Basement Full of Skulls, a 60-year project resulting in 7,000+ meticulously preserved animal skulls, leading me to ask "what's MY basement full of skulls going to be?"

Today I'm adding a fourth thing to that list: the Tom Scott Streak.

Tags: inspiring, productivity, tom-scott, youtube, streaks, duolingo

Quoting Matt Bateman

2023-04-05T23:50:35+00:00

[On AI-assisted programming] I feel like I got a small army of competent hackers to both do my bidding and to teach me as I go. It's just pure delight and magic.

It's riding a bike downhill and playing with legos and having a great coach and finishing a project all at once.

— Matt Bateman

Tags: productivity, llms, ai, generative-ai, ai-assisted-programming

Quoting Robin Sloan

2023-02-05T17:59:41+00:00

When you start a creative project but don’t finish, the experience drags you down. Worst of all is when you never decisively abandon a project, instead allowing it to fade into forgetfulness. The fades add up; they become a gloomy haze that whispers, you’re not the kind of person who DOES things.

When you start and finish, by contrast — and it can be a project of any scope: a 24-hour comic, a one-page short story, truly anything — it is powerful fuel that goes straight back into the tank. When a project is finished, it exits the realm of “this is gonna be great” and becomes instead something you (and perhaps others) can actually evaluate. Even if that evaluation is disastrous, it is also, I will insist, thrilling and productive. A project finished is the pump of a piston, preparing the engine for the next one.

— Robin Sloan

Tags: productivity, robin-sloan

Coping strategies for the serial project hoarder

2022-11-26T15:47:02+00:00

I gave a talk at DjangoCon US 2022 in San Diego last month about productivity on personal projects, titled "Massively increase your productivity on personal projects with comprehensive documentation and automated tests".

The alternative title for the talk was Coping strategies for the serial project hoarder.

I'm maintaining a lot of different projects at the moment. Somewhat unintuitively, the way I'm handling this is by scaling down techniques that I've seen working for large engineering teams spread out across multiple continents.

The key trick is to ensure that every project has comprehensive documentation and automated tests. This scales my productivity horizontally, by freeing me up from needing to remember all of the details of all of the different projects I'm working on at the same time.

You can watch the talk on YouTube (25 minutes). Alternatively, I've included a detailed annotated version of the slides and notes below.

This was the title I originally submitted to the conference. But I realized a better title was probably...

Coping strategies for the serial project hoarder

This video is a neat representation of my approach to personal projects: I always have a few on the go, but I can never resist the temptation to add even more.

My PyPI profile (which is only five years old) lists 185 Python packages that I've released. Technically I'm actively maintaining all of them, in that if someone reports a bug I'll push out a fix. Many of them receive new releases at least once a year.

Aside: I took this screenshot using shot-scraper with a little bit of extra JavaScript to hide a notification bar at the top of the page:

shot-scraper 'https://pypi.org/user/simonw/' \
--javascript "
    document.body.style.paddingTop = 0;
    document.querySelector(
        '#sticky-notifications'
    ).style.display = 'none';
  " --height 1000

How can one individual maintain 185 projects?

Surprisingly, I'm using techniques that I've scaled down from working at a company with hundreds of engineers.

I spent seven years at Eventbrite, during which time the engineering team grew to span three different continents. We had major engineering centers in San Francisco, Nashville, Mendoza in Argentina and Madrid in Spain.

Consider timezones: engineers in Madrid and engineers in San Francisco had almost no overlap in their working hours. Good asynchronous communication was essential.

Over time, I noticed that the teams that were most effective at this scale were the teams that had a strong culture of documentation and automated testing.

As I started to work on my own array of smaller personal projects, I found that the same discipline that worked for large teams somehow sped me up, when intuitively I would have expected it to slow me down.

I wrote an extended description of this in The Perfect Commit.

I've started structuring the majority of my work in terms of what I think of as "the perfect commit" - a commit that combines implementation, tests, documentation and a link to an issue thread.

As software engineers, it's important to note that our job generally isn't to write new software: it's to make changes to existing software.

As such, the commit is our unit of work. It's worth us paying attention to how we can make our commits as useful as possible.

Here's a recent example from one of my projects, Datasette.

It's a single commit which bundles together the implementation, some related documentation improvements and the tests that show it works. And it links back to an issue thread from the commit message.

Let's talk about each component in turn.

There's not much to be said about the implementation: your commit should change something!

It should only change one thing, but what that actually means varies on a case by case basis.

It should be a single change that can be documented, tested and explained independently of other changes.

(Being able to cleanly revert it is a useful property too.)

The goals of the tests that accompany a commit are to prove that the new implementation works.

If you apply the implementation the new tests should pass. If you revert it the tests should fail.

I often use git stash to try this out.

If you tell people they need to write tests for every single change they'll often push back that this is too much of a burden, and will harm their productivity.

But I find that the incremental cost of adding a test to an existing test suite keeps getting lower over time.

The hard bit of testing is getting a testing framework setup in the first place - with a test runner, and fixtures, and objects under test and suchlike.

Once that's in place, adding new tests becomes really easy.

So my personal rule is that every new project starts with a test. It doesn't really matter what that test does - what matters is that you can run pytest to run the tests, and you have an obvious place to start building more of them.

I maintain three cookiecutter templates to help with this, for the three kinds of projects I most frequently create:

simonw/python-lib for Python libraries
simonw/click-app for command line tools
simonw/datasette-plugin for Datasette plugins

Each of these templates creates a project with a setup.py file, a README, a test suite and GitHub Actions workflows to run those tests and ship tagged releases to PyPI.

I have a trick for running cookiecutter as part of creating a brand new repository on GitHub. I described that in Dynamic content for GitHub repository templates using cookiecutter and GitHub Actions.

This is a hill that I will die on: your documentation must live in the same repository as your code!

You often see projects keep their documentation somewhere else, like in a wiki.

Inevitably it goes out of date. And my experience is that if your documentation is out of date people will lose trust in it, which means they'll stop reading it and stop contributing to it.

The gold standard of documentation has to be that it's reliably up to date with the code.

The only way you can do that is if the documentation and code are in the same repository.

This gives you versioned snapshots of the documentation that exactly match the code at that time.

More importantly, it means you can enforce it through code review. You can say in a PR "this is great, but don't forget to update this paragraph on this page of the documentation to reflect the change you're making".

If you do this you can finally get documentation that people learn to trust over time.

Another trick I like to use is something I call documentation unit tests.

The idea here is to use unit tests to enforce that concepts introspected from your code are at least mentioned in your documentation.

I wrote more about that in Documentation unit tests.

Here's an example. Datasette has a test that scans through each of the Datasette plugin hooks and checks that there is a heading for each one in the documentation.

The test itself is pretty simple: it uses pytest parametrization to look through every introspected plugin hook name, and for each one checks that it has a matching heading in the documentation.

The final component of my perfect commit is this: every commit must link to an issue thread.

I'll usually have these open in advance but sometimes I'll open an issue thread just so I can close it with a commit a few seconds later!

Here's the issue for the commit I showed earlier. It has 11 comments, and every single one of those comments is by me.

I have literally thousands of issues on GitHub that look like this: issue threads that are effectively me talking to myself about the changes that I'm making.

It turns out this a fantastic form of additional documentation.

What goes in an issue?

Background: the reasons for the change. In six months time you'll want to know why you did this.
State of play before-hand: embed existing code, link to existing docs. I like to start my issues with "I'm going to change this code right here" - that way if I come back the next day I don't have to repeat that little piece of research.
Links to things! Documentation, inspiration, clues found on StackOverflow. The idea is to capture all of the loose information floating around that topic.
Code snippets illustrating potential designs and false-starts.
Decisions. What did you consider? What did you decide? As programmers we make decisions constantly, all day, about everything. That work doesn't have to be invisible. Writing them down also avoids having to re-litigate them several months later when you've forgotten your original reasoning.
Screenshots - of everything! Animated screenshots even better. I even take screenshots of things like the AWS console to remind me what I did there.
When you close it: a link to the updated documentation and demo

The reason I love issues is that they're a form of documentation that I think of as temporal documentation.

Regular documentation comes with a big commitment: you have to keep it up to date in the future.

Issue comments skip that commitment entirely. They're displayed with a timestamp, in the context of the work you were doing at the time.

No-one will be upset or confused if you fail to keep them updated to match future changes.

So it's a commitment-free form of documentation, which I for one find incredibly liberating.

I think of this approach as issue driven development.

Everything you are doing is issue-first, and from that you drive the rest of the development process.

This is how it relates back to maintaining 185 projects at the same time.

With issue driven development you don't have to remember anything about any of these projects at all.

I've had issues where I did a bunch of design work in issue comments, then dropped it, then came back 12 months later and implemented that design - without having to rethink it.

I've had projects where I forgot that the project existed entirely! But I've found it again, and there's been an open issue, and I've been able to pick up work again.

It's a way of working where you treat it like every project is going to be maintained by someone else, and it's the classic cliche here that the somebody else is you in the future.

It horizontally scales you and lets you tackle way more interesting problems.

Programmers always complain when you interrupt them - there's this idea of "flow state" and that interrupting a programmer for a moment costs them half an hour in getting back up to speed.

This fixes that! It's much easier to get back to what you are doing if you have an issue thread that records where you've got to.

Issue driven development is my key productivity hack for taking on much more ambitious projects in much larger quantities.

Another way to think about this is to compare it to laboratory notebooks.

Here's a page from one by Leonardo da Vinci.

Great scientists and great engineers have always kept detailed notes.

We can use GitHub issues as a really quick and easy way to do the same thing!

Another thing I like to use these for is deep research tasks.

Here's an example, from when I was trying to figure out how to run my Python web application in an AWS Lambda function:

Figure out how to deploy Datasette to AWS Lambda using function URLs and Mangum

This took me 65 comments over the course of a few days... but by the end of that thread I'd figured out how to do it!

Here's the follow-up, with another 77 comments, in which I figure out how to serve an AWS Lambda function with a Function URL from a custom subdomain.

I will never have to figure this out ever again! That's a huge win.

https://github.com/simonw/public-notes is a public repository where I keep some of these issue threads, transferred from my private notes repos using this trick.

The last thing I want to encourage you to do is this: if you do project, tell people what it is you did!

This counts for both personal and work projects. It's so easy to skip this step.

Once you've shipped a feature or built a project, it's so tempting to skip the step of spending half an hour or more writing about the work you have done.

But you are missing out on so much of the value of your work if you don't give other people a chance to understand what you did.

I wrote more about this here: What to blog about.

For projects with releases, release notes are a really good way to do this.

I like using GitHub releases for this - they're quick and easy to write, and I have automation setup for my projects such that creating release notes in GitHub triggers a build and release to PyPI.

I've done over 1,000 releases in this way. Having them automated is crucial, and having automation makes it really easy to ship releases more often.

Please make sure your release notes have dates on them. I need to know when your change went out, because if it's only a week old it's unlikely people will have upgraded to it yet, whereas a change from five years ago is probably safe to depend on.

I wrote more about writing better release notes here.

This is a mental trick which works really well for me. "No project of mine is finished until I've told people about it in some way" is a really useful habit to form.

Twitter threads are (or were) a great low-effort way to write about a project. Build a quick thread with some links and images, and maybe even a video.

Get a little unit about your project out into the world, and then you can stop thinking about it.

(I'm trying to do this on Mastodon now instead.)

Even better: get a blog! Having your own corner of the internet to write about the work that you are doing is a small investment that will pay off many times over.

("Nobody blogs anymore" I said in the talk... Phil Gyford disagrees with that meme so much that he launched a new blog directory to show how wrong it is.)

The enemy of projects, especially personal projects, is guilt.

The more projects you have, the more guilty you feel about working on any one of them - because you're not working on the others, and those projects haven't yet achieved their goals.

You have to overcome guilt if you're going to work on 185 projects at once!

This is the most important tip: avoid side projects with user accounts.

If you build something that people can sign into, that's not a side-project, it's an unpaid job. It's a very big responsibility, avoid at all costs!

Almost all of my projects right now are open source things that people can run on their own machines, because that's about as far away from user accounts as I can get.

I still have a responsibility for shipping security updates and things like that, but at least I'm not holding onto other people's data for them.

I feel like if your project is tested and documented, you have nothing to feel guilty about.

You have put a thing out into the world, and it has tests to show that it works, and it has documentation that explains what it is.

This means I can step back and say that it's OK for me to work on other things. That thing there is a unit that makes sense to people.

That's what I tell myself anyway! It's OK to have 185 projects provided they all have documentation and they all have tests.

Do that and the guilt just disappears. You can live guilt free!

You can follow me on Mastodon at @simon@simonwillison.net or on GitHub at github.com/simonw. Or subscribe to my blog at simonwillison.net!

From the Q&A:

You've tweeted about using GitHub Projects. Could you talk about that?
- GitHub Projects V2 is the perfect TODO list for me, because it lets me bring together issues from different repositories. I use a project called "Everything" on a daily basis (it's my browser default window) - I add issues to it that I plan to work on, including personal TODO list items as well as issues from my various public and private repositories. It's kind of like a cross between Trello and Airtable and I absolutely love it.
How did you move notes from the private to the public repo?
- GitHub doesn't let you do this. But there's a trick I use involving a temp repo which I switch between public and private to help transfer notes. More in this TIL.
Question about the perfect commit: do you commit your failing tests?
- I don't: I try to keep the commits that land on my main branch always passing. I'll sometimes write the failing test before the implementation and then commit them together. For larger projects I'll work in a branch and then squash-merge the final result into a perfect commit to main later on.

Tags: djangocon, documentation, productivity, my-talks, testing, annotated-talks, github-issues

Quoting @gamemakerstk

2020-08-13T15:12:40+00:00

We’re generally only impressed by things we can’t do - things that are beyond our own skill set. So, by definition, we aren’t going to be that impressed by the things we create. The end user, however, is perfectly able to find your work impressive.

— @gamemakerstk

Tags: productivity

Quoting Tobi Lutke

2019-12-26T19:06:35+00:00

For creative work, you can't cheat. My believe is that there are 5 creative hours in everyone's day. All I ask of people at Shopify is that 4 of those are channeled into the company.

— Tobi Lutke

Tags: productivity, management, tobias-lutke

Weeknotes: Niche Museums, Kepler, Trees and Streaks

2019-10-28T22:42:10+00:00

Niche Museums

Every now and then someone will ask “so when are you going to build Museums Near Me then?”, based on my obsession with niche museums and websites like www.owlsnearme.com.

For my Strategic Communications course at Stanford last week I had to perform a midterm presentation - a six minute talk to convince my audience of something, accompanied by slides and a handout.

I chose “you should seek out and explore tiny museums” as my topic, and used it as an excuse to finally start the website!

www.niche-museums.com is the result. It’s a small but growing collection of niche museums (17 so far, mostly in the San Francisco Bay Area) complete with the all important blue “Use my location” button to see museums near you.

Naturally I built it on Datasette. I’ll be writing more about the implementation (and releasing the underlying code) soon. I also built a new plugin for it, datasette-haversine.

Mapping museums against Starbucks

I needed a way to emphasize quite how many tiny museums there are in the USA. I decided to do this with a visualization.

It turns out there are 15,891 branches of Starbucks in the USA… and at least 30,132 museums!

I made these maps using a couple of sources.

All The Places is a crowdsourced scraper project which aims to build scrapers for every company that has a “store locator” area of their website. Starbucks has a store locator and All The Places have a scraper for it, so you can download GeoJSON of every Starbucks. I wrote a quick script to import that GeoJSON into Datasette using sqlite-utils.

The Institute of Museum and Library Services is an independent agency of the federal government that supports museums and libraries across the country. They publish a dataset of Museums in the USA as a set of CSV files. I used csvs-to-sqlite to load those into Datasette, than ran a union query to combine the three files together.

So I have Datasette instances (with a CSV export feature) for both Starbucks and USA museums, with altitudes and longitudes for each.

Now how to turn that into a map?

I turned to my new favourite GIS tool, Kepler. Kepler is an open source GIS visualization tool released by Uber, based on WebGL. It’s astonishingly powerful and can be used directly in your browser by clicking the “Get Started” button on their website (which I assumed would take you to installation instructions, but no, it loads up the entire tool in your browser).

You can import millions of points of data into Kepler and it will visualize them for you directly. I used a Datasette query to export the CSVs, then loaded in my Starbucks CSV, exported an image, loaded in the Museums CSV as a separate colour and exported a second image. The whole project ended up taking about 15 minutes. Kepler is a great addition to the toolbelt!

Animating the PG&E outages

My PG&E outages scraper continues to record a snapshot of the PG&E outage map JSON every ten minutes. I’m posting updates to a thread on Twitter, but discovering Kepler inspired me to look at more sophisticated visualization options.

This tutorial by Giuseppe Macrì tipped me off the the fact that you can use Kepler to animate points against timestamps!

Here’s the result: a video animation showing how PG&E’s outages have evolved since the 5th of October:

Here's a video animation of PG&E's outages from October 5th up until just a few minutes ago pic.twitter.com/50K3BrROZR
- Simon Willison (@simonw) October 28, 2019

Hayes Valley Trees

The city announced plans to cut down 27 ficus trees in our neighborhood in San Francisco. I’ve been working with Natalie to help a small group of citizens organize an appeal, and this weekend I helped run a survey of the affected trees (recording their exact locations in a CSV file) and then built www.hayes-valley-trees.com (source on GitHub) to link to from fliers attached to each affected tree.

It started out as a Datasette (running on Glitch) but since it’s only 27 data points I ended up freezing the data in a static JSON file to avoid having to tolerate any cold start times. The site is deployed as static assets on Zeit Now using their handy GitHub continuous deployment tool.

Streaks

It turns out I’m very motivated by streaks: I’m at 342 days for Duolingo Spanish and 603 days for an Apple Watch move streak. Could I apply this to other things in my life?

I asked on Twitter and was recommended the Streaks iOS app. It’s beautiful! I’m now tracking streaks for guitar practice, Duolingo, checking email, checking Slack, reading some books and adding a new museum to www.niche-museums.com (if I add one a day I can get from 17 museums today to 382 in a year!)

It seems to be working pretty well so far. I particularly like their iPhone widget.

Tags: museums, productivity, projects, visualization, weeknotes, baked-data, streaks, duolingo

Seeking the Productive Life: Some Details of My Personal Infrastructure

2019-02-22T21:46:13+00:00

Seeking the Productive Life: Some Details of My Personal Infrastructure

Stephen Wolfram’s 15,000 word epic about his personal approach to productivity, developed over the past thirty years. This is a fascinating document—I found myself thinking “surely there can’t be more information than this” and then spotting that the scrollbar wasn’t even a third done yet. Very hard to summarize: it turns out if you’re the work-from-home CEO of your own privately held 800 person company you can construct some very opinionated habits.

Via Hacker News

Tags: productivity

Have you ever experienced a boost in productivity by switching to a different programming language?

2014-03-19T18:38:00+00:00

My answer to Have you ever experienced a boost in productivity by switching to a different programming language? on Quora

Switching from PHP to Python (over a decade ago now) dramatically improved my productivity as a programmer.

Tags: productivity, programming, programming-languages, quora

What are some tips for improving public speaking skills quickly?

2014-03-07T11:57:00+00:00

My answer to What are some tips for improving public speaking skills quickly? on Quora

Practice your talk, out loud, in private, as many times as possible before you deliver it. There's no better way of ensuring you know your material and that you can deliver it at a sensible pace without freezing up.

Make sure you have a full written copy of your talk available to you when you deliver it. You should aim to give your talk from memory (prompted by your slides), but I find that having the notes nearby ready to fall back gives me much more confidence. The audience will not mind in the slightest if you pause for a moment to check your notes - they probably won't even notice.

Tags: productivity, speaking, quora

What should be considered when deciding to do a marathon?

2013-12-03T12:16:00+00:00

My answer to What should be considered when deciding to do a marathon? on Quora

Running a marathon is easier then you think.

First, get to a point where you can run 5K. Couch to 5K is a popular program for this. Barring major health issues, anyone can get to that point.

Next, do 10K. After a few times running a 5K this shouldn't be too big a stretch.

Guess what: a marathon is 42km. You're nearly a quarter of the way there!

Now, sign up for a half marathon. That's only two 10Ks back to back. If you've been running 10K once a week you should be able to do this without much trouble.

The bad news is that a marathon is way more than just two half marathons... but it's totally possible. Follow a well documented training scheme and you'll make it. It will be one of the hardest things you've ever done, but it's absolutely possible to do it.

I managed a marathon after less than six months of training, starting from zero fitness. I trained 3-4 times a week: 2 short runs, a medium run and a long run on a Sunday.

Go for it!

Tags: productivity, running, quora

How do I overcome my fear of public speaking (of people just "switching off", or simply getting up and leaving the room)?

2013-11-24T18:14:00+00:00

My answer to How do I overcome my fear of public speaking (of people just "switching off", or simply getting up and leaving the room)? on Quora

Look for opportunities to give "lightning talks" - 5 minute talks given as part of a series of talks. These are excellent for beginner speakers as they help force you to get to the point as quickly as possible, and you only have to survive for five minutes! They are good for the audience too as if they don't enjoy our talk they only have to sit politely for a couple of minutes before the next talk comes along.

Tags: productivity, speaking, quora

Is there a website or app that helps you track what you have completed each day?

2013-11-08T09:49:00+00:00

My answer to Is there a website or app that helps you track what you have completed each day? on Quora

I've tried a few solutions for this. Surprisingly the one that has stuck for me is Evernote - I keep a different document for each week (I tried a document per day but that was annoying to update, and meant I didn't look at my older notes as often) and each day I add a new header at the top of the document for that day. Being able to link through to other notes from my day summaries is useful too.

Tags: productivity, webapps, quora, motivation

I want to write a short summary for every article I read online for future use. What is the best tool to do this?

2013-08-13T18:27:00+00:00

My answer to I want to write a short summary for every article I read online for future use. What is the best tool to do this? on Quora

http://pinboard.in is well suited to this. It's a bookmarking service (like the old Delicious) with a bookmarklet that lets you quickly annotate and add tags to a link, privately or in public. For an extra fee the site will archive copies of the pages you are linking to as well in case they vanish in the future.

Tags: productivity, quora, lifehacks

What are some recommended efficient apps for personal productivity (e.g., setting & receiving reminders for completing tasks)?

2013-07-16T11:18:00+00:00

My answer to What are some recommended efficient apps for personal productivity (e.g., setting & receiving reminders for completing tasks)? on Quora

Things is excellent (at least if you are a Mac/iPhone person) - intuitive, powerful and with flawless syncing. Only catch is it's a tad expensive considering you have to buy the iPhone and Mac apps separately.

Tags: productivity, quora

What are some productive things to do for 15 minutes a day?

2013-01-15T13:25:00+00:00

My answer to What are some productive things to do for 15 minutes a day? on Quora

Learn a foreign language - using DuoLingo on the iPhone, or with podcasts such as Coffee Break Spanish.

Tags: productivity, quora, duolingo

What are some creative programs to use for presentations?

2012-09-23T16:29:00+00:00

My answer to What are some creative programs to use for presentations? on Quora

Here's a trick I've used with success in the past: set up your Mac to have 9 virtual desktops, then arrange your "slides" on each desktop using a combination of applications. I've done this with a title slide in keynote on the first desktop, a text editor with some sample code on the second, a terminal prompt set up for live coding on the third, a browser showing a demo on the fourth and so on. Learn the keyboard commands to switch between desktops and off you go.

This works great, looks impressive, is easy to set up and amuses the audience. There is a BIG risk though: if your computer crashes you'll have a heck of a time setting it all up again! Thankfully that hasn't happened to me yet, but I always have a text file and a set of bookmarks in place so I can muddle through if things go wrong.

Tags: productivity, speaking, quora

What are some things I can do if I'm lying awake and unable to sleep for an hour?

2011-12-26T13:31:00+00:00

My answer to What are some things I can do if I'm lying awake and unable to sleep for an hour? on Quora

Listen to a podcast. If you're lying at rest in bed you'll still get at least some of the benefits of sleeping, and you might find that listening to the podcast helps take your mind off things and sends you to sleep.

Tags: productivity, quora, sleep

Quoting Auren Hoffman

2009-06-24T11:00:47+00:00

Software engineers today are about 200-400% more productive than software engineers were 10 years ago because of open source software, better programming tools, common libraries, easier access to information, better education, and other factors. This means that one engineer today can do what 3-5 people did in 1999!

— Auren Hoffman

Tags: engineers, productivity, aurenhoffman, open-source

Quoting Matt Mullenweg

2007-09-07T22:42:22+00:00

Spend 10 minutes collecting everything you need to work on a problem, and unplug the internet for 2 hours. You'll finish in 30 minutes.

— Matt Mullenweg

Tags: matt-mullenweg, productivity

Geek | Manager

2007-02-19T10:15:24+00:00

Geek | Manager

Meri Williams is one of the most productive people I know. This is her new blog on being a manager and a geek at the same time, with plenty of productivity advice.

Tags: geek, manager, meriwilliams, productivity