Simon Willison's Weblog: coldfusion

A custom template system from the mid-2000s era

2025-07-01T19:57:46+00:00

Using LLMs for code archaeology is pretty fun.

I stumbled across this blog entry from 2003 today, in which I had gotten briefly excited about ColdFusion and implemented an experimental PHP template engine that used XML tags to achieve a similar effect:

<h1>%title%</h1>
<sql id="recent">
select title
from entries 
order by added desc
limit 0, %limit%
</sql>
<ul>
  <output sql="recent">
    <li>%title%</li>
  </output>
</ul>

I'd completely forgotten about this, and in scanning through the PHP it looked like it had extra features that I hadn't described in the post.

So... I fed my 22 year old TemplateParser.class.php file into Claude and prompted:

Write detailed markdown documentation for this template language

Here's the resulting documentation. It's pretty good, but the highlight was the Claude transcript which concluded:

This appears to be a custom template system from the mid-2000s era, designed to separate presentation logic from PHP code while maintaining database connectivity for dynamic content generation.

Mid-2000s era indeed!

Tags: coldfusion, php, projects, ai, generative-ai, llms, ai-assisted-programming, claude

Weeknotes: datasette-template-sql

2019-11-18T22:29:29+00:00

Last week I talked about wanting to take ona a larger Datasette project, and listed some candidates. I ended up pushing a big project that I hadn't listed there: the upgrade of Datasette to Python 3.8, which meant dropping support for Python 3.5 (thanks to incompatible dependencies).

Since Glitch now runs Python 3.7.5 my biggest reason for supporting 3.5 was gone, so I decided to make the upgrade.

Datasette 0.31 was the first version to drop support. Datasette 0.32 is the first to take advantage of it: I switched Datasette's template rendering over to use Jinja's async template support, which requires Python 3.6+.

This has exciting implications for the extra_template_vars plugin hook, which allows plugins to add extra variables (and functions) to the template scope.

Plugin authors can now add custom template functions that are defined with async def ... and make await calls within the body of the function. When the template is rendered, Jinja will automatically await those function calls.

I released a new plugin that takes advantage of this capability: datasette-template-sql. It lets you embed additional SQL queries directly in a custom Datasette template. For example:

{% for article in sql(
    "select headline, date, summary from articles order by date desc limit 5",
    "news"
) %}
    <h3>{{ article.headline }}</h2>
    <p class="date">{{ article.date }}</p>
    <p>{{ article.summary }}</p>
{% endfor %}

This new sql() function takes a SQL query and the optional name of the database to run the query against (in case you have more than one database file attached to your Datasette instance).

I'm really excited about this capability. I quipped about it on Twitter:

If you're thinking "aren't you just reinventing PHP at this point?" I'm actually more going for ColdFusion
- Simon Willison (@simonw) November 15, 2019

This is the great thing about having a plugin system: even if I'm not convinced this is the right technical direction for the core Datasette project, I can still expose this kind of feature in a plugin that people can opt into if they want to have this ability.

The official Datasette website is going to make extensive use of this plugin. I have an early prototype of that up and running now, which inspired me to release datasette-render-markdown 0.2 with a custom template function for rendering markdown directly:

{{ render_markdown(value) }}

Aside from the work on Python 3, my Datasette time this week has mostly involved ongoing refactors of both the query execution code and the core TableView.data() method. Hopefully these will unblock a flurry of interesting new functionality in the not too distant future.

Niche museums

This week's new museums on www.niche-museums.com.

California State Capitol Dioramas in Sacramento
Zeppelin Museum in Friedrsichshafen
Dai Loy Museum in Locke (Sacramento Delta)
Wallace Collection in London
Cookin' in San Francisco
Ramen Museum in Yokohama
Cable Car Museum in San Francisco

Tags: coldfusion, projects, sql, datasette, weeknotes

What was programming use Myspace?

2010-10-13T12:46:00+00:00

My answer to What was programming use Myspace? on Quora

Originally ColdFusion, but then a small army of consultants from Microsoft helped rewrite it in .NET.

Tags: coldfusion, myspace, quora

Quoting Aral Balkan

2009-01-08T18:10:31+00:00

The simple truth is that in the age of Web 2.0/3.0, in the era of cloud and utility computing, the application server is a commodity. A commercial, proprietary app server simply cannot survive in this environment anywhere outside the lethargic, soft-padded walls of the enterprise.

— Aral Balkan

Tags: appservers, aral-balkan, coldfusion, commoditisation, enterprise, open-source

New PHP experiment, inspired by ColdFusion

2003-07-17T09:58:53+00:00

I've been reading up on ColdFusion MX recently, and I have to admit it looks like a really nice piece of technology. I'd previously written ColdFusion off as being too simplistic and primitive, but having seen how much its capable of I'm reconsidering my position.

If you've never seen ColdFusion before, it's a server side scripting language/application server from Macromedia (who obtained it when they bought Allaire). MX is the latest version of the language, which sees it completely rewritten in Java to allow it to integrate with Java application servers and existing Java servlet applications. This is shrewd marketing on the part of Macromedia, and I've already seen them advertising it as something to make your existing Java web deployments easier to customise.

The thing that put me off ColdFusion originally is that it is a tag-based scripting language, specifically designed to make it easy for HTML developers with little or no previous programming experience to pick up. Here's an example of a chunk of CF syntax:


<cfinclude template="some-other-file.cfm">

<cfif isdefined("form.name")>
 <cfoutput>
Hi there, #form.name#
 </cfoutput>
</cfif>

The thing that most surprised me about ColdFusion is that although the above syntax looks pretty verbose, it is actually designed in a way that means you can do an awful lot with very little code. The above in PHP would look like this:


<?php
include('some-other-file.php');
if (isset($form['name'])) {
    print "Hi there, $form['name']";
}
?>

It's less typing, but only by a bit. Where CF gets really clever though is with its handling of SQL. Here's the code to run a SQL query and loop through outputting the results to a simple table:


<cfquery name="users" datasource="myDSN">
select name, email from users order by name
</cfquery>
<table>
<cfoutput query="users">
<tr>
 <td>#name#</td><td>#email#</td>
</tr>
</cfoutput>
</table>

Here's the same in PHP, assuming $db contains a connection to a MySQL database.


<?php
$result = mysql_query("select name, email from users order by name", $db);
echo '<table>';
while ($row = mysql_fetch_assoc($result)) {
    echo "<tr>
  <td>$row['name']</td><td>{$row['email']}</td>
</tr>";
}
echo '</table>';
?>

The ColdFusion example achieves the same effect but with less complicated code.

ColdFusion is not without its problems: it comes with a price tag, and it encourages mixing application logic with presentation code (although as with PHP this can be avoided through discipline and careful application design). Never the less, I found it interesting enough that last night I spent a few hours putting together a PHP implementation of a couple of ColdFusion concepts. As with many of the experiments I post here this is very much experimental code - I won't be supporting it and I would not recommend using it for anything more than casual experimentation.

This demo renders this example Recent Entries page.

Here's the Recent Entries page PHP source code:

<?php
// Set up $db as a connection to MySQL ...
include('TemplateParser.class.php');

$parser = new TemplateParser($db, array(
    'title' => 'Recent Entries', 
    'limit' => 10
));

echo $parser->parse(implode('', file('templates/recent.xml')));
?>

It renders this recent.xml template:

<template id="recent">
  <doctype type="xhtml1strict"/>
  <html>
    <head>
      <meta http-equiv="Content-Type" content="text/html; charset=utf-8"/>
      <title>%title%</title>
      <style type="text/css">
        body {
          font-family: georgia;
          margin: 2em;
        }
      </style>
    </head>
    <body>
      <h1>%title%</h1>
      <sql id="recent">
        select title
        from entries 
        order by added desc
        limit 0, %limit%
      </sql>
      <ul>
        <output sql="recent">
          <li>%title%</li>
        </output>
      </ul>
    </body>
  </html>
</template>

And here's the TemplateParser.class.php class that does all the work:

<?php

class TemplateParser {
    var $collect = false; // If true, tags are "collected" to $buffer rather than being added to $output
    var $buffer = '';
    var $output = '';
    var $parser;
    var $sqlid;
    var $outputsql;
    var $sql = array();   // 2D array of extracted SQL
    // -
    var $db;
    var $replacements = array(); // 2D array of replacement to make on cdata
    var $templatedir = 'templates/';
    // -
    var $doctypes = array(
        'html4strict' => '<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">',
        'html4trans'  => '<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">',
        'xhtml1strict' => '<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">',
    );
    function TemplateParser($db, $replacements = array()) {
        $this->db = $db;
        $this->replacements = $replacements;
    }
    function parse($data) {
        $this->parser = xml_parser_create();
        // Set XML parser to take the case of tags in to account
        xml_parser_set_option($this->parser, XML_OPTION_CASE_FOLDING, false);
        // Set XML parser callback functions
        xml_set_object($this->parser, $this);
        xml_set_element_handler($this->parser, 'tag_open', 'tag_close');
        xml_set_character_data_handler($this->parser, 'cdata');
        if (!xml_parse($this->parser, $data)) {
            die(sprintf('XML error: %s at line %d',
                xml_error_string(xml_get_error_code($this->parser)),
                xml_get_current_line_number($this->parser)));
        }
        xml_parser_free($this->parser);
        return trim($this->output);
    }
    function tag_open($parser, $tag, $attr) {
        $singletag = false;
        switch ($tag) {
            case 'doctype':
                $this->handle_doctype($attr);
                return;
            case 'include':
                if (!isset($attr['template'])) {
                    die('include tag has no template attribute');
                }
                $path = $this->templatedir.$attr['template'].'.xml';
                if (!file_exists($path)) {
                    die("Template $path does not exist");
                }
                $temp = new TemplateParser($this->db, $attr);
                $this->output .= $temp->parse(implode('', file($path)));
                return;
            case 'ignore-start':
                $this->collect = true; // Although we have no intention of using it
                return;
            case 'ignore-stop':
                $this->buffer = '';
                $this->collect = false;
                return;
            case 'sql':
                if (!isset($attr['id'])) {
                    die('sql tag has no id attribute');
                }
                $this->sqlid = $attr['id'];
                $this->collect = true;
                return;
            case 'output':
                if (!isset($attr['sql'])) {
                    die('output tag has no sql attribute');
                }
                $this->outputsql = $attr['sql'];
                $this->collect = true;
                return;
            case 'template':
                return;
            case 'img':  // These are all HTML elements with no end tag
            case 'br':
            case 'link':
            case 'meta':
            case 'hr':
                $singletag = true;
        }
        // Now either collect or add the tag to output
        $xml = '<'.$tag.$this->makeattr($attr);
        if ($singletag) {
            $xml .= ' />';
        } else {
            $xml .= '>';
        }
        if ($this->collect) {
            $this->buffer .= $xml;
        } else {
            $this->output .= $xml;
        }
    }
    function tag_close($parser, $tag) {
        switch ($tag) {
            case 'sql':
                $this->sql[$this->sqlid] = $this->convert($this->buffer);
                $this->buffer = '';
                $this->collect = false;
                return;
            case 'output':
                // This is the tricky one. Run the SQL query, loop through it
                // and run convert on the saved block multiple times replacing with
                // the results of the query
                // First clear the buffer
                $block = $this->buffer;
                $this->buffer = '';
                $this->collect = false;
                // Now run the query
                $sql = $this->sql[$this->outputsql];
                $result = mysql_query($sql, $this->db);
                if (!$result) {
                    die("SQL query failed: '$sql'");
                }
                $i = 0;
                while ($row = mysql_fetch_assoc($result)) {
                    $row['#'] = ++$i; // '%#%' is now available as a counter var
                    $this->output .= $this->convert($block, $row);
                }    
                return;
            case 'template': // These are template elements with no end tag
            case 'include':
            case 'ignore-start':
            case 'ignore-stop':
            case 'doctype':
            case 'img':  // These are all HTML elements with no end tag
            case 'br':
            case 'link':
            case 'meta':
            case 'hr':
                return;
        }
        // Add the end tag to the buffer or output
        $xml = '</'.$tag.'>';
        if ($this->collect) {
            $this->buffer .= $xml;
        } else {
            $this->output .= $xml;
        }
    }
    function cdata($parser, $data) {
        if ($this->collect) {
            $this->buffer .= $data;
            return;
        }
        $this->output .= $this->convert($data);
    }
    function convert($text, $replacements = false) {
        if (!$replacements) {
            $replacements = $this->replacements;
        }
        if (strpos($text, '%') === false) {
            return $text; // Nothing to replace
        }
        foreach ($replacements as $from => $to) {
            $text = str_replace('%'.$from.'%', $to, $text);
        }
        return $text;
    }
    function makeattr($attr) {
        $return = '';
        foreach ($attr as $name => $value) {
            $return .= $name.'="'.$value.'"';
        }
        if ($return != '') {
            $return = ' '.$return;
        }
        return $return;
    }
    function handle_doctype($attr) {
        if (!isset($attr['type'])) {
            die('doctype requires type attribute');
        }
        if (!in_array($attr['type'], array_keys($this->doctypes))) {
            die('doctype requires valid type attribute');
        }
        $this->output .= $this->doctypes[$attr['type']];
    }
}

?>

With a bit more development, something like this could be a useful tool for quick-and-dirty PHP scripts that simply pull data out of MySQL and display it as HTML. I still prefer PHP, but ColdFusion has a lot of good ideas which are well worth knowing about.

Update July 1st 2025: I stumbled across this long-forgotten experiment today and decided to have Claude write me some documentation for it, since this blog entry didn't describe all of its features. Here's the result (and the Claude transcript). Amusingly, Claude described the code like this:

This appears to be a custom template system from the mid-2000s era, designed to separate presentation logic from PHP code while maintaining database connectivity for dynamic content generation.

Tags: coldfusion, php, projects, sql, templating, xml