Simon Willison's Weblog: tutorials

Weeknotes: Self-hosted language models with LLM plugins, a new Datasette tutorial, a dozen package releases, a dozen TILs

2023-07-16T05:55:54+00:00

A lot of stuff to cover from the past two and a half weeks.

LLM and self-hosted language model plugins

My biggest project was the new version of my LLM tool for interacting with Large Language Models. LLM now accepts plugins for adding alternative language models to the tool, meaning it's now applicable to more than just the OpenAI collection.

I figured out quite a few of the details of this while offline on a camping trip up in the Northern California redwoods, which forced the issue on figuring out how to work with LLMs that I could host on my own computer because I didn't have a connection to access the OpenAI APIs.

Comprehensive documentation is sorely lacking in the world of generative AI. I've decided to push back against that for LLM, so I spent a bunch of time working on an extremely comprehensive tutorial for writing a plugin that adds a new language model to the tool:

Writing a plugin to support a new model

As part of researching this tutorial I finally figured out how to build a Python package using just a pyproject.toml file, with no setup.py or setup.cfg or anything else like that. I wrote that up in detail in Python packages with pyproject.toml and nothing else, and I've started using that pattern for all of my new Python packages.

LLM also now includes a Python API for interacting with models, which provides an abstraction that works the same for the OpenAI models and for other models (including self-hosted models) installed via plugins. Here's the documentation for that - it ends up looking like this:

import llm

model = llm.get_model("gpt-3.5-turbo")
model.key = 'YOUR_API_KEY_HERE'
response = model.prompt("Five surprising names for a pet pelican")
for chunk in response:
    print(chunk, end="")

To use another model, just swap its name in for gpt-3.5-turbo. The self-hosted models provided by the llm-gpt4all plugin work the same way:

pip install llm-gpt4all

Then:

import llm

model = llm.get_model("ggml-vicuna-7b-1")
response = model.prompt("Five surprising names for a pet pelican")
# You can do this instead of looping through the chunks:
print(response.text())

I've released three plugins so far:

llm-gpt4all with 17 self-hosted models from the GPT4All project.
llm-palm with Google's PaLM 2 language model, via their API.
llm-mpt30b providing the 19GB MPT-30B model, using TheBloke/mpt-30B-GGML.

I'm looking forward to someone else following the tutorial and releasing their own plugin!

A new tutorial: Data analysis with SQLite and Python

I presented this as a 2hr45m tutorial at PyCon a few months ago. The video is now available, and I like to try to turn these kinds of things into more permanent documentation.

The Datasette website has a growing collection of tutorials, and I decided to make that the final home for this one too.

Data analysis with SQLite and Python now has the full 2hr45m video plus an improved version of the handout I used for the talk. The written material there there should also be valuable for people who don't want to spend nearly three hours watching the video!

As part of putting that page together I solved a problem I've been wanting to figure out for a long time: I figured out a way to build a custom Jinja block tag that looks like this:

{% markdown %}
# This will be rendered as markdown

- Bulleted
- List
{% endmarkdown %}

I released that in datasette-render-markdown 2.2. I also wrote up a TIL on Custom Jinja template tags with attributes describing the pattern I used.

One bonus feature for that tutorial: I decided to drop in a nested table of contents, automatically derived from the HTML headers on the page.

I wrote the code for this entirely using the new ChatGPT Code Interpreter, which can write Python based on your description and, crucially, execute it and see if it works.

Here's my ChatGPT transcript showing how I built the feature.

I've been using ChatGPT Code Interpreter for a few months now, and I'm completely hooked: I think it's the most interesting thing in the whole AI space at the moment.

I participated in a Code Interpreter Latent Space episode to talk about it, which ended up drawing 17,000 listeners on Twitter Spaces and is now also available as a podcast episode, neatly edited together by swyx.

Symbex --check and --rexec

Symbex is my Python CLI tool for quickly finding Python functions and classes and outputting either the full code or just the signature of the matching symbol. I first wrote about that here.

symbex 1.1 adds two new features.

symbex --function --undocumented --check

This new --check mode is designed to run in Continuous Integration environments. If it finds any symbols matching the filters (in this case functions that are missing their docstring) it returns a non-zero exit code, which will fail the CI step.

It's an imitation of black . --check - the idea is that Symbex can now be used to enforce code quality issues like docstrings and the presence of type annotations.

The other new feature is --rexec. This is an extension of the existing --replace feature, which lets you find a symbol in your code and replace its body with new code.

--rexec takes a shell expression. The body of the matching symbol will be piped into that command, and its output will be used as the replacement.

Which means you can do things like this:

symbex my_function \
  --rexec "llm --system 'add type hints and a docstring'"

This will find def my_function() and its body, pass that through llm (using the gpt-3.5-turbo default model, but you can specify -m gpt-4 or any other model to use something else), and then take the output and update the file in-place with the new implementation.

As a demo, I ran it against this:

def my_function(a, b):
    return a + b + 3

And got back:

def my_function(a: int, b: int) -> int:
    """
    Returns the sum of two integers (a and b) plus 3.

    Parameters:
    a (int): The first integer.
    b (int): The second integer.

    Returns:
    int: The sum of a and b plus 3.
    """
    return a + b + 3

Obviously this is fraught with danger, and you should only run this against code that has already been committed to Git and hence can be easily recovered... but it's a really fun trick!

ttok --encode --decode

ttok is my CLI tool for counting tokens, as used by LLM models such as GPT-4. ttok 0.2 adds a requested feature to help make tokens easier to understand, best illustrated by this demo:

ttok Hello world
# Outputs 2 - the number of tokens
ttok Hello world --encode
# Outputs 9906 1917 - the encoded tokens
ttok 9906 1917 --decode
# Outputs Hello world - decoding the tokens back again
ttok Hello world --encode --tokens
# Outputs [b'Hello', b' world']

Being able to easily see the encoded tokens including whitespace (the b' world' part) is particularly useful for understanding how the tokens all fit together.

I wrote more about GPT tokenization in understanding GPT tokenizers.

TIL this week

Using tree-sitter with Python - 2023-07-14
Auto-formatting YAML files with yamlfmt - 2023-07-13
Quickly testing code in a different Python version using pyenv - 2023-07-10
Using git-filter-repo to set commit dates to author dates - 2023-07-10
Using OpenAI functions and their Python library for data extraction - 2023-07-10
Python packages with pyproject.toml and nothing else - 2023-07-08
Syntax highlighted code examples in Datasette - 2023-07-02
Custom Jinja template tags with attributes - 2023-07-02
Local wildcard DNS on macOS with dnsmasq - 2023-06-30
A Discord bot to expand issue links to a private GitHub repository - 2023-06-30
Bulk editing status in GitHub Projects - 2023-06-29
CLI tools hidden in the Python standard library - 2023-06-29

Releases this week

symbex 1.1 - 2023-07-16
Find the Python code for specified symbols
llm-mpt30b 0.1 - 2023-07-12
LLM plugin adding support for the MPT-30B language model
llm-markov 0.1 - 2023-07-12
Plugin for LLM adding a Markov chain generating model
llm-gpt4all 0.1 - 2023-07-12
Plugin for LLM adding support for the GPT4All collection of models
llm-palm 0.1 - 2023-07-12
Plugin for LLM adding support for Google's PaLM 2 model
llm 0.5 - 2023-07-12
Access large language models from the command-line
ttok 0.2 - 2023-07-10
Count and truncate text based on tokens
strip-tags 0.5.1 - 2023-07-09
CLI tool for stripping tags from HTML
pocket-to-sqlite 0.2.3 - 2023-07-09
Create a SQLite database containing data from your Pocket account
datasette-render-markdown 2.2 - 2023-07-02
Datasette plugin for rendering Markdown
asgi-proxy-lib 0.1a0 - 2023-07-01
An ASGI function for proxying to a backend over HTTP
datasette-upload-csvs 0.8.3 - 2023-06-28
Datasette plugin for uploading CSV files and converting them to database tables

Tags: plugins, projects, tutorials, ai, datasette, weeknotes, sqlite-utils, generative-ai, local-llms, llms, symbex, llm

Weeknotes: Parquet in Datasette Lite, various talks, more LLM hacking

2023-06-04T21:14:27+00:00

I've fallen a bit behind on my weeknotes. Here's a catchup for the last few weeks.

Parquet in Datasette Lite

Datasette Lite is my build of Datasette (a server-side Python web application) which runs entirely in the browser using WebAssembly and Pyodide. I recently added the ability to directly load Parquet files over HTTP.

This required an upgrade to the underlying version of Pyodide, in order to use the WebAssembly compiled version of the fastparquet library. That upgrade was blocked by a AttributeError: module 'os' has no attribute 'link' error, but Roman Yurchak showed me a workaround which unblocked me.

So now the following works:

https://lite.datasette.io/?parquet=https://github.com/Teradata/kylo/blob/master/samples/sample-data/parquet/userdata1.parquet

This will work with any URL to a Parquet file that is served with open CORS headers - files on GitHub (or in a GitHub Gist) get these headers automatically.

Also new in Datasette Lite: the ?memory=1 query string option, which starts Datasette Lite without loading any default demo databases. I added this to help me construct this demo for my new datasette-sqlite-url-lite plugin:

https://lite.datasette.io/?memory=1&install=datasette-sqlite-url-lite#/_memory?sql=select+'url_valid()'+as+fn%2C+url_valid(%3Aurl)+as+result%0Aunion+all%0Aselect+'url_scheme()'%2C+url_scheme(%3Aurl)%0Aunion+all%0Aselect+'url_host()'%2C+url_host(%3Aurl)%0Aunion+all%0Aselect+'url_path()'%2C+url_path(%3Aurl)%0Aunion+all%0Aselect+'url_fragment()'%2C+url_fragment(%3Aurl)%3B&url=https%3A%2F%2Fwww.sqlite.org%2Fvtab.html%23usage

datasette-sqlite-url-lite - mostly written by GPT-4

datasette-sqlite-url is a really neat plugin by Alex Garcia which adds custom SQL functions to SQLite that allow you to parse URLs and extract their components.

There's just one catch: the extension itself is written in C, and there isn't yet a version of it compiled for WebAssembly to work in Datasette Lite.

I wanted to use some of the functions in it, so I decided to see if I could get a Pure Python alternative of it working. But this was a very low stakes project, so I decided to see if I could get GPT-4 to do essentially all of the work for me.

I prompted it like this - copying and pasting the examples directly from Alex's documentation:

Write Python code to register the following SQLite custom functions:

select url_valid('https://sqlite.org'); -- 1
select url_scheme('https://www.sqlite.org/vtab.html#usage'); -- 'https'
select url_host('https://www.sqlite.org/vtab.html#usage'); -- 'www.sqlite.org'
select url_path('https://www.sqlite.org/vtab.html#usage'); -- '/vtab.html'
select url_fragment('https://www.sqlite.org/vtab.html#usage'); -- 'usage'

The code it produced was almost exactly what I needed.

I wanted some tests too, so I prompted:

Write a suite of pytest tests for this

This gave me the tests I needed - with one error in the way they called SQLite, but still doing 90% of the work for me.

Here's the full ChatGPT conversation and the resulting code I checked into the repo.

Various talks

Videos for three of my recent talks are now available on YouTube:

Big Opportunities in Small Data is the keynote I gave at Citus Con: An Event for Postgres 2023 - talking about Datasette, SQLite and some tricks I would love to see the PostgreSQL community adopt from the explorations I've been doing around small data.
The Data Enthusiast's Toolkit is an hour long interview with Rizel Scarlett about both Datasette and my career to date. Frustratingly I had about 10 minutes of terrible microphone audio in the middle, but the conversation itself was really great.
Data analysis with SQLite and Python is a video from PyCon of the full 2hr45m tutorial I gave there last month. The handout notes for that are available online too.

I also spotted that the Changelog put up a video Just getting in to AI for development? Start here with an extract from our podcast episode LLMs break the internet.

Entries this week

Releases this week

datasette-sqlite-url-lite 0.1 - 2023-05-26
A pure Python alternative to sqlite-url ready to be used in Datasette Lite
sqlite-utils 3.32.1 - 2023-05-21
Python CLI utility and library for manipulating SQLite databases
strip-tags 0.3 - 2023-05-19
CLI tool for stripping tags from HTML
ttok 0.1 - 2023-05-18
Count and truncate text based on tokens
llm 0.3 - 2023-05-17
Access large language models from the command-line

TIL this week

Testing the Access-Control-Max-Age CORS header - 2023-05-25
Comparing two training datasets using sqlite-utils - 2023-05-23
mlc-chat - RedPajama-INCITE-Chat-3B on macOS - 2023-05-22
hexdump and hexdump -C - 2023-05-22
Exploring Baseline with Datasette Lite - 2023-05-12

Tags: projects, speaking, tutorials, datasette, parquet, weeknotes, datasette-lite, llms

Exploration de données avec Datasette

2023-05-27T00:36:52+00:00

Exploration de données avec Datasette

One of the great delights of open source development is seeing people run workshops on your project, even more so when they’re in a language other than English! Romain Clement presented this French workshop for the Python Grenoble meetup on 25th May 2023, using GitHub Codespaces as the environment. It’s pretty comprehensive, including a 300,000+ row example table which illustrates Datasette plugins such as datasette-cluster-map and datasette-leaflet-geojson.

Tags: tutorials, datasette, github-codespaces, leaflet

Using Datasette in GitHub Codespaces

2023-02-24T00:40:11+00:00

Using Datasette in GitHub Codespaces

A new Datasette tutorial showing how it can be run inside GitHub Codespaces—GitHub’s browser-based development environments—in order to explore and analyze data. I’ve been using Codespaces to run tutorials recently and it’s absolutely fantastic, because it puts every tutorial attendee on a level playing field with respect to their development environments.

Tags: github, tutorials, datasette, github-codespaces

Cleaning data with sqlite-utils and Datasette

2022-07-31T19:57:51+00:00

Cleaning data with sqlite-utils and Datasette

I wrote a new tutorial for the Datasette website, showing how to use sqlite-utils to import a CSV file, clean up the resulting schema, fix date formats and extract some of the columns into a separate table. It’s accompanied by a ten minute video originally recorded for the HYTRADBOI conference.

Via @simonw

Tags: documentation, tutorials, datasette, sqlite-utils

JMeter Result Analysis using Datasette

2021-02-01T04:42:19+00:00

JMeter Result Analysis using Datasette

NaveenKumar Namachivayam wrote a detailed tutorial on using Datasette (on Windows) and csvs-to-sqlite to analyze the results of JMeter performance test runs and then publish them online using Vercel.

Via @QAInsights

Tags: tutorials, datasette

Making Datasets Fly with Datasette and Fly

2020-03-26T23:56:46+00:00

Making Datasets Fly with Datasette and Fly

It’s always exciting to see a Datasette tutorial that wasn’t written by me! This one is great—it shows how to load Central Park Squirrel Census data into a SQLite database, explore it with Datasette and then publish it to the Fly hosting platform using datasette-publish-fly and datasette-cluster-map.

Via @flydotio

Tags: tutorials, datasette, fly

What are the best free tutorials on HTML5?

2014-01-24T13:16:00+00:00

My answer to What are the best free tutorials on HTML5? on Quora

Dive Into HTML5 and HTML5 Rocks are both outstanding.

Tags: html5, tutorials, web-development, quora

What are the best books/tutorials to begin learning about memcached?

2013-04-08T14:51:00+00:00

My answer to What are the best books/tutorials to begin learning about memcached? on Quora

There isn't really enough of memcached to justify a whole book - it's a pretty straight-forward API.

It's always interesting hearing about advanced usage patterns for it though. Again, these don't necessarily justify a book but they are frequently presented at conferences.

Here's one video that may be relevant: High Performance Caching with Rails - a session at GoRuCo 2012 by Matt Duncan

Here's our full collection of 36 slides and video from talks about memcached: Conference coverage about Memcached on Lanyrd

Tags: books, memcached, programming, tutorials, web-development, quora

maps from scratch

2009-03-15T13:20:23+00:00

maps from scratch

An idea whose time has come: using EC2 AMIs for tutorial sessions to give everyone a pre-configured environment.

Tags: cloud-computing, ec2, mapping, michal-migurski, tutorials

Django snippets: Orderable inlines using drag and drop with jQuery UI

2008-09-13T12:19:02+00:00

Django snippets: Orderable inlines using drag and drop with jQuery UI

Code example from my PyCon tutorial on customising the Django admin interface.

Tags: django, django-admin, dragndrop, jquery, pyconuk, pyconuk2008, python, snippets, sortable, speaking, my-talks, tutorials

Learning jQuery

2007-08-17T11:50:45+00:00

Learning jQuery

An entire year’s worth of jQuery tutorials, split in to beginner, intermediate and advanced.

Tags: javascript, jquery, learningjquery, tutorials