Simon Willison's Weblog: freebase

Using Freebase Gridworks to Create Linked Data

2010-08-23T20:11:00+00:00

Using Freebase Gridworks to Create Linked Data

A very handy tutorial from data.gov.uk’s Jeni Tennison.

Tags: data, datagovuk, freebase, gridworks, recovered, jenitennison

WildlifeNearYou talk at £5 app, and being Wired (not Tired)

2010-04-11T20:42:11+00:00

Two quick updates about WildlifeNearYou. First up, I gave a talk about the site at £5 app, my favourite Brighton evening event which celebrates side projects and the joy of Making Stuff. I talked about the site's genesis on a fort, crowdsourcing photo ratings, how we use Freebase and DBpedia and how integrating with Flickr's machine tags gave us a powerful location API for free. Here's the video of the talk, courtesy of Ian Oszvald:

£5 App #22 WildLifeNearYou by Simon Willison and Natalie Downe from IanProCastsCoUk on Vimeo.

Secondly, I'm excited to note that WildlifeNearYou spin-off OwlsNearYou.com is featured in UK Wired magazine's Wired / Tired / Expired column... and we're Wired!

Tags: api, crowdsourcing, fivepoundapp, flickr, freebase, natalie-downe, owlsnearyou, my-talks, wildlifenearyou, wired

Preview: Freebase Gridworks

2010-03-27T18:43:42+00:00

Preview: Freebase Gridworks

If my experience with government datasets has taught me anything, it’s that most datasets are collected by human beings (probably using Excel) and human beings are inconsistent. The first step in any data related project inevitably involves cleaning up the data. The Freebase team must run up against this all the time, and it looks like they’re tackling the problem head-on. Freebase Gridworks is just a screencast preview at the moment but an open source release is promised “within a month”—and the tool looks absolutely fantastic. DabbleDB-style data refactoring of spreadsheet data, running on your desktop but with the UI served in a browser. Full undo, a JavaScript-based expression language, powerful faceting and the ability to “reconcile” data against Freebase types (matching up country names, for example). I can’t wait to get my hands on this.

Via Jon Udell

Tags: cleanup, dabbledb, data, freebase, gridworks, javascript, open-data

Hack Day tools for non-developers

2009-07-28T14:23:53+00:00

We're about to run our second internal hack day at the Guardian. The first was an enormous amount of fun and the second one looks set to be even more productive.

There's only one rule at hack day: build something you can demonstrate at the end of the event (Powerpoint slides don't count). Importantly though, our hack days are not restricted to just our development team: anyone from the technology department can get involved, and we extend the invitation to other parts of the organisation as well. At the Guardian, this includes journalists.

For our first hack day, I put together a list of "tools for non-developers" - sites, services and software that could be used for hacking without programming knowledge as a pre-requisite. I'm now updating that list with recommendations from elsewhere. Here's the list so far:

Freebase

Originally a kind of structured version of Wikipedia, Freebase changed its focus last year towards being a "social database about things you know and love". In other words, it's the most powerful OCD-enabler in the history of the world. Create your own "Base" on any subject you like, set up your own types and start gathering together topics from the millions already available in Freebase - or add your own. Examples include the Battlestar Galactica base, the Tall Ships base and the fabulous Database base. If you are a developer the tools in the Make Things with Freebase section are top notch.

Dabble DB

Dabble is a weird combination of a spreadsheet, an online database and a set of visualisation tools. Watch the 8 minute demo to get an idea of how powerful this is - you can start off by loading in an existing spreadsheet and take it from there. You'll need to sign up for the free 30 day trial.

Google Docs

You can always build a hack in Excel, but Google Spreadsheets is surprisingly powerful and means that you can collaborate with others on your hack (including developers, who can use the Google Docs API to get at the data in your spreadsheet). Check out the following tutorials, which describe ways of using Google Spreadsheets to scrape in data from other webpages and output it in interesting formats:

There's also a simple way to create a form that submits data in to a Google Spreadsheet.

Yahoo! Pipes

Visual tools for combining, filtering and modifying RSS feeds. Combine with the large number of full-content feeds on guardian.co.uk for all sorts of interesting possibilities. Here's a tutorial that incorporates Google Docs as well.

Google My Maps

Google provide a really neat interface for adding your own points, lines and areas to a Google Map. Outputs KML, a handy file format for carting geographic data around between different tools.

If you already have a KML or GeoRSS feed URL from somewhere (e.g. the output of a Yahoo! Pipe), you can paste it directly in to the Google Maps search box to see the points rendered on a map.

Google SketchUp

A simple to use 3D drawing package that lets you create 3D models of real-world buildings and then import them in to Google Earth.

OpenStreetMap

Try your hand at some open source cartography on OpenStreetMap, the geographic world's answer to Wikipedia. If you have the equipment you can contribute GPS traces, otherwise there's a clever online editor that will let you trace out roads from satellite photos - or you could just make sure your favourite pub is included on the map. The export tools can provide vector or static maps, and if you export as SVG you can further edit your map in Illustrator or Inkscape.

CloudMade Maps

Commercial tools built on top of OpenStreetMap, the most exciting of which allows you to create your own map theme by setting your preferred colours and line widths for various types of map feature.

Many Eyes

IBM Research's suite of data visualisation tools, with a wiki-style collaboration platform for publishing data and creating visualisations.

Dapper

Dapper provides a powerful tool for screen scraping websites, without needing to write any code. Output formats include RSS, iCalendar and Google Maps.

TiddlyWiki

TiddlyWiki is a complete wiki in a single HTML file, which you can save locally and use as a notebook, collaboration tool and much more. There's a large ecosystem of plugins and macros which can be used to extend it with new features - see TiddlyVault for an index.

WolframAlpha

The "computational knowledge engine" with the hubristic search-based interface, potentially useful as a source of data and a tool for processing and visualising that data.

Tumblr

Useful as both an input and an output for feeds processed using other tools, and with a smart bookmarklet for collecting bits and pieces from around the web.

The UCSB Toy Chest

An outstanding list of tools that people "without programming skills (but with basic computer and Internet literacy) can use to create interesting projects", compiled by the English department at UC Santa Barbara.

Your help needed

There must be dozens, if not hundreds of useful tools missing from the above. Tell me in the comments and I'll add them to the list.

Tags: freebase, google, google-maps, guardian, hackday, mapping, nondevelopers, openstreetmap, pipes, sketchup, tools, yahoo-pipes, computer-literacy

Freebase Sets

2008-12-13T09:26:26+00:00

Freebase Sets

Give it some topics and it will tell you what they have in common and show further topics matching the same rules. Kind of like the old Google Labs sets tool but this one shows its workings.

Via The Freebase Blog

Tags: freebase, freebasesets, sets

Introducing Acre

2008-11-07T23:23:28+00:00

Introducing Acre

I’m losing track of all the server-side JavaScript hosted web application platforms now. Here’s the Freebase contribution to the genre, complete with IDE, templating language and strong integration with Freebase itself.

Tags: acre, freebase, javascript, serversidejavascript

Freebase Hack Day

2008-10-24T00:06:06+00:00

Freebase Hack Day

I’m finding Freebase increasingly interesting at the moment, and their public hack day on the 8th November in San Francisco looks like it could be a lot of fun. They’ll be previewing Acre, a new server-side JavaScript application platform targeted at building Freebase powered applications. Hit “view source” at the bottom of the hack day site to see what an Acre app looks like.

Tags: acre, events, freebase, hackday, javascript, san-francisco

Quoting Kellan Elliott-McCrea

2008-09-29T15:29:34+00:00

The only down side is everyone I’ve talked to at Freebase seems pretty solid on this being their proprietary secret sauce, because a good, fast scalable open source tuple store might actually jump start a real semantic (small-S) web after all these years.

— Kellan Elliott-McCrea

Tags: freebase, graphd, kellan-elliott-mccrea, open-source, proprietary, semanticweb

A Brief Tour of Graphd

2008-09-29T11:32:25+00:00

A Brief Tour of Graphd

The secret sauce behind Freebase—a custom written graph server that models everything as a typed, versioned relationship and can churn through over 3,000 simple queries a second on a single AMD64 core.

Tags: databases, freebase, grahd, graphserver

freebase-suggest

2008-09-24T23:58:22+00:00

freebase-suggest

A jQuery plugin that performs auto-completion against the Freebase JSONP API, and allows the results to be limited to specific categories or subsets.

Via Alf Eaton

Tags: autocomplete, freebase, freebasesuggest, javascript, jquery, jsonp, metadata

Freebase developer documentation

2007-09-03T02:38:01+00:00

Freebase developer documentation

The JSON API and particularly the query language are fascinating.

Tags: api, freebase, json

Freebase

2007-09-03T02:35:04+00:00

Freebase

Out of closed beta, although you still need an invite code to contribute. I hope they drop the JavaScript requirement for viewing content on the site.

Tags: freebase, inaccessible, javascript