IONCANNON » Streaming Video

iPad Streaming Video and More

carson — Tue, 06 Apr 2010 11:01:33 +0000

I've updated the configuration examples in the open source segmenter project to reflect Apple's recommended stream bitrates for iPad video streaming, added a few fixes and a few new features. If you are interested in streaming video on the iPad, iPhone or iPod Touch and haven't done so yet you it may help to start with my post on windowed streaming on for the iPhone, then read about iPhone HTTP streaming with FFMpeg and the open source segmenter and finally check out the iPad, iPhone, and iPod Touch live video streaming project page.

Here is a demo of the iPad streaming video created with the segmenter (I tried to show the progressive upgrade happening but it happens very quickly since the iPad is on WIFI, I also show that you can scrub without any issues and if you look in the background you can see the server log displaying entries as the segments are downloaded):

If you want to view the demo yourself I've created a demo for the iPad, iPhone, and iPod Touch. Note that those are two links, one for the iPad version and one for the iPhone/iPod Touch version. The main difference is the size of the video.

I used the open source video Big Buck Bunny (the 1920×1080 ogg version) for the above demos.

If you are interested in more details on what changed read on or skip to the bottom if you want to see what I'll be working towards in future versions of the segmenter.

Over the past few weeks I learned that it is important that each segment starts with an IDR frame. To accomplish this I thought setting the gop so that the segment time % (gop size / frame rate) = 0 would work but I haven't completely convinced myself of this yet. An example of would be a segment size of 10 seconds % (300 gop size / 30 frame rate) = 0. This should insure that each segment starts with the correct i-frame and while I have looked over the resulting segments with a hex editor and believe it works I still get errors when using certain tools on the individual segments that makes me think it isn't working like I think it should. Either way I have included updates to the gop size in the example configuration files. The gop size is controlled by the -g option for FFMpeg and in the examples I have set it so that for a 10 second segment with the given frame rate the gop size makes sure each segment starts correctly. If you want to know more you can dig into the resulting segments and use this forum post on how to extract an i-frame, this list of mpeg headers to verify that each has the correct i-frame at the start.

The Apple streaming tech notes where informative in a few other ways as well. The tech notes contain the supported h264 profiles for each device. For the iPhone/iPod Touch baseline level 3.0 is supported while the iPad supports baseline level 3.1. You can find out more about h264 encoding levels with FFMpeg and also review FFMpeg x264 encoding guide, FFMpeg option mappings and information on encoding with the x264 codec for more information.

Another bit of information disclosed in the Apple tech notes is the existence of a media stream validation tool. This tool can be downloaded from the Apple iPhone developer site and has to be run on OS X. Apple recommends that you use this tool to validate any streams that you create.

One other note that I have recently ran across is that Apple seems to be rejecting native apps that have video streams without a fall back audio stream. The correct way of generating the fallback 64k audio only stream is something that is lacking in the current version of the segmenter but I hope to fix that soon. It won't matter for those using the HTML5 video tag however.

If you are using HTML5 video tags it is important not to use my older posts as a guide for the iPad since I use a now outdated example. Those examples still work on the iPhone but they do not work on the iPad. Here is an updated example that will work on both the iPhone and iPad:


  
    Video Test

Finally I am currently using the latest version of FFMpeg to do tests and it seems to have become more reliable for me. The current version in git seems to be pretty good at producing usable videos from a variety of source videos.

The following is a quick summary of the recent changes I have made:

Applied an audio patch for the segmenter from Scott Kidder. This patch skips any video processing if there is no video stream.
Modified sample configs with newer FFMpeg string, gop sizes, frame rates and other parameters to match the Apple recomended values for iPhone/iPod Touch with aspect ratios of 4:3 and 6:9 on cell or wifi and iPad with aspect ratios of 4:3 and 6:9 on cell or wifi
Added configuration sanity checks
Made gems not required if not using those features that need them
Added fix for deprecation warning from newest version of libavformat
Made termination work better when killed with SIGINT/ctrl-c
Fixed some minor issues with index file format

The following are enhancements I'm planning on making:

I plan on creating a howto on getting everything set up on EC2.
Dig more into verification that each segment starts with the correct I-frame.
I'm going to work on refactoring everything so that instead of creating files and then transferring them it streams them all. This makes more sense with doing encryption as just another part of the pipeline.
Add ability to do encryption. I have a proof of concept working but I decided I need to get everything set up in one continuous stream before integrating encryption into the mix.
I had thoughtabout using the ruby daemon utils to wrap the segmenter but I decided it probably isn't necessary at this point.

Streaming Video Between QuickTime Broadcaster and VLC

carson — Sat, 04 Jul 2009 00:19:58 +0000

In my attempt to stream live video to my iPhone I ran into an issue with the USB QuickCam I have. Instead of fighting the problem I decided to turn to the iSight camera on a macbook but I quickly found that there isn't a great way to get the video stream to another machine. The best way that I have found to stream from the iSight to a second machine is to use a combination of QuickTime Broadcaster and VLC. In this post I detail how I got it set up and working.

Step 1: Install QuickTime Broadcaster

The first step is to install QuickTime Broadcaster. It doesn't have a lot of requirements so it should work on most macs. The QuickTime Broadcaster will be the source of the video and the destination of the video in my case is a second machine with VLC installed on it although you could run VLC on the source machine if you wanted to.

An important note at this point is to make sure the destination machine doesn't have a firewall turned on or if it does that it isn't blocking ports 5432 and 5434.

Step 2: Configure QuickTime Broadcaster for Streaming

Navigate to the Applications folder and launch QuickTime broadcaster. Click on the network tab and select "Custom" from the "Preset" list. For "Transmission" select the "Manual Unicast" option. The "Address" field contains the IP for your target machine, the one that you are streaming to. When you are finished, your settings should look something like this:

Now the configuration is ready and it is time to create an export for use in VLC. To export go to File -> Export -> SDP menu option. For this example I'm calling the file stream01.sdp.

You should transfer this file to the machine you are going to run VLC on at this point. If you are interested you can actually view the contents of the SDP file. Here is the one generated for me from the above:

v=0

o=- 0 3702455757 IN IP4 127.0.0.0

s=QuickTime

t=0 0

a=range:npt=now-

a=isma-compliance:2,2.0,2

m=audio 5432 RTP/AVP 96

c=IN IP4 192.168.132.100

b=AS:125

a=rtpmap:96 mpeg4-generic/44100/2

a=fmtp:96 profile-level-id=15;mode=AAC-hbr;sizelength=13;indexlength=3;indexdeltalength=3;config=1210

a=mpeg4-esid:101

m=video 5434 RTP/AVP 97

c=IN IP4 192.168.132.100

b=AS:1372

a=rtpmap:97 H264/90000

a=fmtp:97 packetization-mode=1;profile-level-id=4D401E;sprop-parameter-sets=J01AHqkYFAe2ANQYBBrbCte98BA=,KN4JF6A=

a=mpeg4-esid:201

a=cliprect:0,0,480,640

a=framesize:97 640-480

Step 3: Play the Video Stream with VLC

Once the SDP file is on the destination machine you can start VLC and open it. In VLC go to File -> Open File then select the SDP file that was exported earlier.

VLC is now ready to stream from QuickTime Broadcaster. In QuickTime Broadcaster click the Broadcast button and then in VLC click the play button. At this point you should start seeing the stream in VLC. The order you start broadcasting between QuickTime Broadcaster and VLC doesn't seem to matter. There were a few times that I had VLC start for a second then freeze but I was using a wireless network so I'll blame that for the freezes. In general it worked well once it was all set up.

You can start VLC from the command line and point it directly to the stream file if you want. To do that use this command:

vlc stream01.sdp

or if you want fancy logging:

vlc -vvv –color –file-logging –logfile vlc-log.html –logmode html stream01.sdp

Step 4: VLC Streaming

This is actually what I would call re-streaming since my need is to get the iSight stream to FFMpeg. In the perfect world I would be able to stream directly from QuickTime Broadcaster to FFMpeg but the limitations of QuickTime Broadcaster don't allow that to happen, at least from what I could find.

To re-stream the iSight stream follow these steps:

Go to Media -> Streaming

Enter the SDP file in the file window and hit the Stream button.
After you click the Stream button a window will pop up where you have the option to select a Destination:

Select "Display locally" if you want to see the video as it is re-streamed.

Select HTTP from the "New destination" drop down list and click the Add button. I have picked HTTP here because it is what works the best for me but there are other options you can use if you want to try different streaming protocols. I was able to leave the defaults as they were for the HTTP stream.
Select a transcoding profile. Here I've selected the "Video – Theora + Vorbis (OGG)" profile for streaming. It seems to work the best for me:

Click the Stream button at the bottom to start streaming.
Test the stream with something like mplayer or ffplay. Because I'm using HTTP to do the stream I can run the following command to test:

mplayer http://192.168.132.100:8080/

Just as the playback can be done from the command line the re-streaming can too:

vlc stream01.sdp –sout '#transcode{vcodec=theo,vb=800,scale=1,acodec=vorb,ab=128,channels=2,samplerate=44100}:duplicate{dst=std{access=http,mux=ogg,dst=0.0.0.0:8080},dst=display}'

If you want to drop the local display:

vlc stream01.sdp –sout '#transcode{vcodec=theo,vb=800,scale=1,acodec=vorb,ab=128,channels=2,samplerate=44100}:duplicate{dst=std{access=http,mux=ogg,dst=0.0.0.0:8080}}'