Simon Willison's Weblog: hosting

SSH has no Host header

2026-01-22T23:57:50+00:00

exe.dev is a new hosting service that, for $20/month, gives you up to 25 VMs "that share 2 CPUs and 8GB RAM". Everything happens over SSH, including creating new VMs. Once configured you can sign into your exe.dev VMs like this:

ssh simon.exe.dev

Here's the clever bit: when you run the above command exe.dev signs you into your VM of that name... but they don't assign every VM its own IP address and SSH has no equivalent of the Host header, so how does their load balancer know which of your VMs to forward you on to?

The answer is that while they don't assign a unique IP to every VM they do have enough IPs that they can ensure each of your VMs has an IP that is unique to your account.

If I create two VMs they will each resolve to a separate IP address, each of which is shared with many other users. The underlying infrastructure then identifies my user account from my SSH public key and can determine which underlying VM to forward my SSH traffic to.

Via lobste.rs

Tags: dns, hosting, ssh

Where is all of the fediverse?

2024-01-12T18:54:15+00:00

Where is all of the fediverse?

Neat piece of independent research by Ben Cox, who used the /api/v1/instance/peers Mastodon API endpoint to get a list of “peers” (instances his instance knows about), then used their DNS records to figure out which hosting provider they were running on.

Next Ben combined that with active users from the /nodeinfo/2.0 API on each instance to figure out the number of users on each of those major hosting providers.

Cloudflare and Fastly were heavily represented, but it turns out you can unveil the underlying IP for most instances by triggering an HTTP Signature exchange with them and logging the result.

Ben’s conclusion: Hertzner and OVH are responsible for hosting a sizable portion of the fediverse as it exists today.

Via lobste.rs

Tags: dns, hosting, mastodon, fediverse

Stringing together several free tiers to host an application with zero cost using fly.io, Litestream and Cloudflare

2022-10-07T17:47:34+00:00

Stringing together several free tiers to host an application with zero cost using fly.io, Litestream and Cloudflare

Alexander Dahl provides a detailed description (and code) for his current preferred free hosting solution for small sites: SQLite (and a Go application) running on Fly’s free tier, with the database replicated up to Cloudflare’s R2 object storage (again on a free tier) by Litestream.

Tags: hosting, sqlite, cloudflare, fly, litestream

Zeit Now v1 to sunset soon: no new deployments from 1st May, total shutdown 7th August

2020-04-04T05:32:02+00:00

Zeit Now v1 to sunset soon: no new deployments from 1st May, total shutdown 7th August

I posted a thread on Twitter with some thoughts. Zeit Now v1 remains the best hosting platform I’ve ever used given my particular tastes. They’ve handled the shutdown very responsibly, but I’m sad to see it go.

Tags: hosting, zeit-now, datasette

datasette-publish-fly

2020-03-19T03:40:17+00:00

datasette-publish-fly

Fly is a neat new Docker hosting provider with a very tempting pricing model: Just $2.67/month for their smallest always-on instance, and they give each user $10/month in free credit. datasette-publish-fly is the first plugin I’ve written using the publish_subcommand plugin hook, which allows extra hosting providers to be added as publish targets. Install the plugin and you can run “datasette publish fly data.db” to deploy SQLite databases to your Fly account.

Via @simonw

Tags: hosting, projects, docker, datasette, fly

Running Datasette on Glitch

2019-04-23T04:08:53+00:00

Update 28th May 2025: Sadly Glitch is shutting down user project hosting, so this tutorial is no longer relevant.

The worst part of any software project is setting up a development environment. It’s by far the biggest barrier for anyone trying to get started learning to code. I’ve been a developer for more than twenty years and I still feel the pain any time I want to do something new.

Glitch is the most promising attempt I’ve ever seen at tackling this problem. It provides an entirely browser-based development environment that allows you to edit code, see the results instantly and view and remix the source code of other people’s projects.

It’s developed into a really fun, super-creative community and a fantastic resource for people looking to get started in the ever-evolving world of software development.

This evening I decided to get Datasette running on it. I’m really impressed with how well it works, and I think Glitch provides an excellent environment for experimenting with Datasette and related tools.

TLDR version: visit https://glitch.com/edit/#!/remix/datasette-csvs right now, drag-and-drop in a CSV file and watch it get served by Datasette on Glitch just a few seconds later.

Running Python on Glitch

The Glitch documentation is all about Node.js and JavaScript, but they actually have very solid Python support as well.

Every Glitch project runs in a container that includes Python 2.7.12 and Python 3.5.2, and you can use pip install --user or pip3 install --user to install Python dependencies.

The key to running non-JavaScript projects on Glitch is the glitch.json file format. You can use this to specify an install script, which sets up your container, and a start script, which starts your application running. Glitch will route HTTP traffic to port 3000, so your application server needs to listen on that port.

This means the most basic Glitch project to run Datasette looks like this:

https://datasette-basic.glitch.me/ (view source)

It contains a single glitch.json file:

{
    "install": "pip3 install --user datasette",
    "start": "datasette -p 3000"
}

This installs Datasette using pip3, then runs it on port 3000.

Since there’s no actual data to serve, this is a pretty boring demo. The most interesting page is this one, which shows the installed versions of the software:

https://datasette-basic.glitch.me/-/versions

Something more interesting: datasette-csvs

Let’s build one with some actual data.

My csvs-to-sqlite tool converts CSV files into a SQLite database. Since it’s also written in Python we can run it against CSV files as part of the Glitch install script.

Glitch provides a special directory called .data/ which can be used as a persistent file storage space that won’t be cleared in between restarts. The following "install" script installs datasette and csvs-to-sqlite, then runs the latter to create a SQLite database from all available CSV files:

{
    "install":  "pip3 install --user datasette csvs-to-sqlite && csvs-to-sqlite *.csv .data/csv-data.db",
    "start": "datasette .data/csv-data.db -p 3000"
}

Now we can simply drag and drop CSV files into the root of the Glitch project and they will be automatically converted into a SQLite database and served using Datasette!

We need a couple of extra details. Firstly, we want Datasette to automatically re-build the database file any time a new CSV file is added or an existing CSV file is changed. We can do that by adding a "watch" block to glitch.json:

"watch": {
    "install": {
        "include": [
            "\\.csv$"
        ]
    }
}

This ensures that our "install" script will run again any time a CSV file changes.

Let’s tone down the rate at which the scripts execute, by using throttle to set the polling interval to once a second:

"throttle": 1000

The above almost worked, but I started seeing errors if I changed the number of columns in a CSV file, since doing so clashed with the schema that had already been created in the database.

My solution was to add code to the install script that would delete the SQLite database file before attempting to recreate it - using the rm ... || true idiom to prevent Glitch from failing the installation if the file it attempted to remove did not already exist.

My final glitch.json file looks like this:

{
  "install": "pip3 install --user datasette csvs-to-sqlite && rm .data/csv-data.db || true && csvs-to-sqlite *.csv .data/csv-data.db",
  "start": "datasette .data/csv-data.db -p 3000 -m metadata.json",
  "watch": {
    "install": {
      "include": [
        "\\.csv$"
      ]
    },
    "restart": {
      "include": [
        "^metadata.json$"
      ]
    },
    "throttle": 1000
  }
}

I also set it up to use Datasette’s metadata.json format, and automatically restart the server any time the contents of that file changes.

https://datasette-csvs.glitch.me/ (view source) shows the results, running against a simple example.csv file I created.

Remixing!

Here’s where things get really fun: Glitch projects support “remixing”, whereby anyone can click a link to create their own editable copy of a project.

Remixing works even if you aren’t logged in to Glitch! Anonymous projects expire after five days, so be sure to sign in with GitHub or Facebook if you want to keep yours around.

Try it out now: Visit https://glitch.com/edit/#!/remix/datasette-csvs to create your own remix of my project. Then drag a new CSV file directly into the editor and within a few seconds Datasette on Glitch will be up and running against a converted copy of your file!

Limitations

The Glitch help center article What technical restrictions are in place? describes their limits. Most importantly, projects are limited to 4,000 requests an hour - and there’s currently no way to increase that limit. They also limit projects to 200MB of disk space - easily enough to get started exploring some interesting CSV files with Datasette.

Next steps

I’m delighted at how easy this was to setup, and how much power the ability to remix these Datasette demos provides. I’m tempted to start creating remixable Glitch demos that illustrate other aspects of Datasette’s functionality such as plugins or full-text search.

Glitch is an exceptionally cool piece of software. I look forward to seeing their Python support continue to evolve.

Tags: glitch, hosting, projects, datasette

What's the cheapest or free stack solution to deploy and experiment with a realtime application in 2016?

2016-08-16T09:02:00+00:00

My answer to What's the cheapest or free stack solution to deploy and experiment with a realtime application in 2016? on Quora

Heroku have a good free tier, and comprehensive support for deploying both Python and Node.js. If you are mainly interested in realtime I would suggest starting out with Node.js on Heroku. Depending on the complexity of your project you might even be able to use raw Node.js without adding something like Express.

Getting Started on Heroku with Node.js

Tags: hosting, programming, webapps, web-development, websockets, quora, heroku

What is the best service for web hosting and buying a domain? Is it better to have both under one provider?

2013-09-22T15:12:00+00:00

My answer to What is the best service for web hosting and buying a domain? Is it better to have both under one provider? on Quora

No, it's not better to have both under the same provider. Good web hosts do not necessarily make good DNS hosts and vice versa.

Tags: domains, hosting, web-development, quora

What is the difference between Windows and Linux for web hosting, in other words, what are the pros and cons of each, each’s limitations, performance development environment and deployment between Windows and Linux?

2013-08-05T12:24:00+00:00

My answer to What is the difference between Windows and Linux for web hosting, in other words, what are the pros and cons of each, each’s limitations, performance development environment and deployment between Windows and Linux? on Quora

Any and every operation you perform on a Linux server can be trivially automated by copying the commands you ran in to a text file. I haven't managed a Windows server in years and I hear PowerShell is pretty great these days but an OS based around a GUI is always going to be harder to automate than one based around a command line.

Linux has a much stronger complement of high quality open source software - and new open source server software (stuff like node.js) usually becomes available for Linux first.

If you don't know anything about Linux but are comfortable with Windows, you'll find it easier to manage a Windows server in the short term.

Tags: hosting, linux, web-development, windows, quora, ecommerce

The First Few Weeks - ep.io

2011-01-13T04:25:00+00:00

The First Few Weeks - ep.io

Another take on managed Python Django/WSGI hosting, from Andrew Godwin and Ben Firshman.

Tags: andrew-godwin, ben-firshman, django, hosting, python, wsgi, recovered

Hello from Gondor

2011-01-13T04:24:00+00:00

Hello from Gondor

“Effortless production Django hosting” from the Eldarion team.

Tags: django, hosting, recovered, eldarion

Quoting Jeffrey Zeldman

2010-01-24T13:40:19+00:00

A suggestion for a business. Sooner or later, some hosting company is going to figure out that it can provide a service and make a killing (as it were) by offering ten-, twenty-, and hundred-year packets of posthumous hosting. A hundred years is not eternity, but you are not Shakespeare, and it’s a start.

— Jeffrey Zeldman

Tags: death, hosting, jeffrey-zeldman

Setup mod_wsgi for Django and Shared Hosting

2008-03-26T14:42:02+00:00

Setup mod_wsgi for Django and Shared Hosting

Tutorial by David Cramer; attached are useful comments from mod_wsgi author Graham Dumpleton.

Tags: david-cramer, django, graham-dumpleton, hosting, modwsgi, python, wsgi

Djangofriendly

2008-03-26T08:47:13+00:00

Djangofriendly

Ryan Berg’s attractive new site collecting ratings and reviews for web hosts that support Django. I’m still happily hosted on a bytemark VPS, which isn’t currently listed on the site.

Via Ryan Berg

Tags: bytemark, django, hosting, python, ryan-berg, vps

Version 2.0 of mod_wsgi is now available

2008-03-21T13:23:11+00:00

Version 2.0 of mod_wsgi is now available

Includes features that should make Python (and Django) on shared hosting much easier: a non-root user can touch their WSGI script file to restart just their application’s daemon processes when they make changes and Python virtual environments are supported to allow different versions of packages without interference.

Tags: django, hosting, modwsgi, python, sharedhosting, wsgi

Troubleshooting Memory Usage

2008-01-23T18:52:37+00:00

Troubleshooting Memory Usage

Useful for getting the most out of a VPS.

Tags: hosting, memory, vps

AppJet: Instant Web Programming

2007-12-15T15:37:46+00:00

AppJet: Instant Web Programming

Another attempt at simple server-side JavaScript application hosting. Worth checking out for the impressive syntax highlighting code editor, which even matches braces.

Tags: appjet, hosting, javascript, syntaxhighlighting, y-combinator

Django GridContainer

2007-09-22T12:01:40+00:00

Django GridContainer

Media Temple’s virtualized Django hosting is now accepting applications for beta testers.

Tags: django, gridcontainer, hosting, mediatemple, virtualization

GoPHP5.org

2007-07-08T10:08:50+00:00

GoPHP5.org

A campaign to encourage a mass switchover from PHP 4 to PHP 5 on February 8th 2008, by co-ordinating both hosting companies and PHP projects.

Tags: hosting, php, php5

Web hosting landscape and mod_wsgi

2007-07-02T15:47:59+00:00

Web hosting landscape and mod_wsgi

Graham Dumpleton explains how mod_wsgi’s daemon mode should provide secure Python deployment for commodity hosting providers.

Tags: graham-dumpleton, hosting, modwsgi, python, wsgi

Security Breach

2007-06-08T08:16:37+00:00

Security Breach

A statement from Dreamhost.

Tags: dreamhost, hosting, security

Massive Dreamhost hack, WordPress not to blame

2007-06-06T09:38:00+00:00

On mezzoblue, Dave Shea reports that someone had modified every index.php and index.html file on his site to include spam links at the bottom of the page, hidden inside a <u style="display: none;">. Dozens of other people in his comments reported the same thing happening to their sites.

At first, it looked like the common thread was WordPress hosted on Dreamhost. Initial commenters were all running WordPress (Dave has it installed for other domains on his hosting account even though he doesn't use it for mezzoblue itself) and there was a vulnerability in WordPress 2.0.7 which was fixed back in January but would still affect people who hadn't yet upgraded. I posted a link suggesting that WordPress users in particular should check their sites.

I apologise to the WordPress team for even suggesting that their product had something to do with this. Here's an e-mail Dreamhost sent out to some of their customers last night:

We have detected what appears to be the exploit of a number of accounts belonging to DreamHost customers, and it appears that your account was one of those affected.

We're still working to determine how this occurred, but it appears that a 3rd party found a way to obtain the password information associated with approximately 3,500 separate FTP accounts and has used that information to append data to the index files of customer sites using automated scripts (primarily for search engine optimization purposes).

Our records indicate that only roughly 20% of the accounts accessed - less than 0.15% of the total accounts that we host - actually had any changes made to them. Most accounts were untouched.

Scary stuff.

Tags: dreamhost, hosting, security, wordpress

Unsettling

2007-06-05T21:16:58+00:00

Unsettling

Sounds like there might be a massive scripted hack going on against out of date WordPress installs on Dreamhost. Check your site. See also discussion in the comments attached to this post.

Tags: dave-shea, dreamhost, hosting, php, security, spam, wordpress

Nginx vs. Lighttpd for a small VPS

2007-01-10T11:31:22+00:00

Nginx vs. Lighttpd for a small VPS

My VPS is still running nginx with no problems at all.

Tags: hosting, lighttpd, nginx, vps

WebFaction blog: BIG holiday present!

2006-12-22T23:44:51+00:00

WebFaction blog: BIG holiday present!

WebFaction offer Django/Rails/TurboGears hosting for $7.50/month, allowing one long-running process and 40MB of RAM for their basic plan.

Via A comment on Jon Udell's "Django gymnastics".

Tags: django, hosting, webfaction