One of the first things I needed to do while building the GeeQE iPhone application was process the CC data dump from Stack Overflow. The dump contains XML files representing tables from Stack Overflow with the largest file being posts.xml weighing in at 1.2G as of September. I decided it would be pretty easy to use Ruby to parse the XML and load the data into MySQL so I went about finding the right parser for the job.
If you haven't processed large amounts of XML before one thing to realize is that you don't want to use a DOM parser because it is going to load the entire XML structure into memory. What you want is a SAX parser that can work on the XML stream as it comes in. With this in mind I started looking around and quickly found an older benchmark post that gave me an educated guess that the LibXML library was going to be the fastest parser for Ruby. After figuring out how to use it I decided to also give a couple other libraries a shot to see how they stacked up, the other two I looked at were REXML and Nokogiri.
I recently won a programming contest that netted me a gift card for ThinkGeek and not knowing what else to do I strolled the site looking for something interesting to use the gift card on. Eventually I ran into the RFID Experimentation Kit they have and decided that was what I needed. I have been wanting to play around with RFID for a while and this kit turned out to be pretty nice for tinkering.
Yaml based configuration file. See the project page for a complete list of options.
Ability to transfer segments via copy, ftp, scp and s3.
Added the ability to do variable bitrate streams.
Added re-streaming support.
Added logging to a file and better debug output.
The variable bitrate streams where done by using pipes. I have done a large amount of testing via ITT Systems and it seems to work fairly well. I am able to stream a live HD video source into 3 different bitrates on a fairly old PC. Here are a couple clips I created to show the progressive enhancement in action, you probably want to switch to the HD version of the video and watch it full screen to get the bet view:
The configuration file will allow for any number of encoding options or transfer options and they can be put together in a number of different ways. Here are a couple examples of both, see the example configuration files for more.
As a final note on changes, you are no longer able to use the segmenter without the script now really. If you want to do that you should use the original version of the segmenter source.
Please note that there is still some work to be done on the script to be complete. If I have time my next enhancement will be to add encryption and I will probably try to test builds on other distributions (maybe attempt to create segmenter binaries).
If you are looking for something you can buy out of the box it appears that Akamai is doing iPhone video streaming now. I believe that the following solution using Amazon S3 and Cloudfront is probably as good as what Akamai can offer but it may be a better choice if you don't want to have to maintain the configuration.
I put together a quick diagram of the process of transferring the video stream from source to final destination that will hopefully help people understand the full picture before jumping into the details:
In my attempt to stream live video to my iPhone I ran into an issue with the USB QuickCam I have. Instead of fighting the problem I decided to turn to the iSight camera on a macbook but I quickly found that there isn't a great way to get the video stream to another machine. The best way that I have found to stream from the iSight to a second machine is to use a combination of QuickTime Broadcaster and VLC. In this post I detail how I got it set up and working.
With the release of the iPhone OS 3 update came the ability to do live streaming. There are a few types of streaming and each requires a certain encoding and segmentation. I've put together a cheat sheet on how I went about building a static stream using FFMpeg and an example segmenter that someone has posted. I'm not covering windowed streams in this post but if you are thinking about implementing a windowed stream the following will help you make a step in that direction.
Before getting started it is best to read over the Apple documentation on HTTP live streaming. Start out with the iPhone streaming media overview. This document covers the basics of how the streaming works and has some nice diagrams.
If you want even more information after reading the overview you can take a look at the HTTP Live streaming draft proposal that was submitted to the IETF by Apple. It covers the streaming protocol in complete detail and has examples of the stream file format for reference.
Once you are ready to start grab a decent quality video clip to use. If you don't have one handy I found a nice list of downloadable HD clips in various formats for testing.
For people who can a complete re-install is probably best. One reason for that is the inclusion of ext4 in Fedora11. You won't get the benefit of ext4 unless you do a fresh install or upgrade from ext3 to ext4. If you read the upgrade guide that Fedora produces it recommends not doing an upgrade.
Over time I've gathered up some nice tips I think would help get someone started developing applications for the BlackBerry so I decided to pull some of the more interesting ones together into the following list.
While developing ideas for iPhone applications I've played around with just using Interface Builder to stub things out. This works reasonably well but I know how to use Interface Builder so that makes a difference. If you are designing an application and want to stick with graphical tools only you are in luck because Yahoo has produced a set of wireframing SVG stencils for the iPhone that are very nice.
For some reason the old link I had that contained display dimensions for BlackBerry devices is resulting in a 404 now. When you are doing BlackBerry development it is nice to be able to go to one page that just shows you the screen sizes instead of having to look at each device page so I don't know why they removed this page. After looking around a little I couldn't find a consolidated list that has just screen sizes so I figured I would put one together that replicates the old link: