Simon Willison’s Weblog

shot-scraper 1.9. New release of my shot-scraper CLI tool for taking screenshots and scraping websites with JavaScript from the terminal.

• The shot-scraper har command has a new -x/--extract option which extracts all of the resources loaded by the page out to a set of files. This location can be controlled by the -o dir/ option. #184
• Fixed the shot-scraper accessibility command for compatibility with the latest Playwright. #185

The new shot-scraper har -x https://simonwillison.net/ command is really neat. The inspiration was the digital forensics expedition I went on to figure out why Rob Pike got spammed. You can now perform a version of that investigation like this:

cd /tmp
shot-scraper har --wait 10000 'https://theaidigest.org/village?day=265' -x

Then dig around in the resulting JSON files in the /tmp/theaidigest-org-village folder.