Parsing the SXSW Twitter Stream for Fun

Over the weekend I decided to toss together a simple twitter stream monitoring app that would capture SXSW tweets. I wanted to build on some of what I learned while hacking together the stuff for code2009 and it was also an excuse to play with node.js and a few other things. I figured I would put together a quick post with all the different parts and as I have time I'll pull together some of the more important sections into larger posts. Be warned that the site may stop functioning since it was only a few hours work and mostly put together with bailing wire and gum.

Server side:

• node.js – Used to process the long polling requests
• ngnix – Front for multiple nodejs instances and serving some static files
• underscore.js
• rabbitmq – Used to monitor the realtime feed
• Redis – Used as a datastore for everything
• ruby – Used to glue a bunch of things together, tons of gems used
• Resque – Used to handle the screen captures

Browser side:

• jQuery
• jQuery flot
• jQuery sparklines

Hosting:

• AWS EC2 – Used for processing browser snapshots
• AWS Cloudfront – Used for storing CSS, etc
• Rackspace Cloud Servers – Used to run the node servers
• Linode – Used to run the main web server

Bookmark to: