http://simonwillison.net/2009-09-21T23:09:04+00:00Simon Willison2009-09-21T23:09:04+00:002009-09-21T23:09:04+00:00https://simonwillison.net/2009/Sep/21/cloudcrowd/#atom-tag

<p><strong><a href="http://github.com/documentcloud/cloud-crowd">cloud-crowd</a></strong></p> New parallel processing worker/job queue system with a strikingly elegant architecture. The central server is an HTTP server that manages job requests, which are farmed out to a number of node HTTP servers which fork off worker processes to do the work. All communication is webhook-style JSON, and the servers are implemented in Sinatra and Thin using a tiny amount of code. The web-based monitoring interface is simply beautiful, using canvas to display graphs showing the system’s overall activity. <p>Tags: <a href="https://simonwillison.net/tags/canvas">canvas</a>, <a href="https://simonwillison.net/tags/cloudcrowd">cloudcrowd</a>, <a href="https://simonwillison.net/tags/http">http</a>, <a href="https://simonwillison.net/tags/json">json</a>, <a href="https://simonwillison.net/tags/message-queues">message-queues</a>, <a href="https://simonwillison.net/tags/ruby">ruby</a>, <a href="https://simonwillison.net/tags/sinatra">sinatra</a>, <a href="https://simonwillison.net/tags/thin">thin</a>, <a href="https://simonwillison.net/tags/webhooks">webhooks</a>, <a href="https://simonwillison.net/tags/workers">workers</a></p>