IONCANNON » ffmpeg

Compiling WebM into FFMpeg for Windows

carson — Tue, 13 Jul 2010 10:30:55 +0000

I have updated this post with a newer version of the VP8 patches to FFMpeg and support for libvorbis instead of the built in vorbis support.

Google has released source for the VP8 codec as the WebM project. The WebM project will be an open alternative in the HTML5 video tag codec space and being backed by Google, who will use it for YouTube, will give it a fighting chance. WebM is starting to be integrated into FFMpeg but there are still some patches that need to be applied. While there are some guides to how to build FFMpeg for Linux with VP8 WebM support I wanted to get it compiled for Windows.

First off this won't be done using a Windows install but instead using Virtual Box with a fresh install of Ubuntu 10.04 desktop. It is just faster and easier that way. So go ahead and grab Virtual Box and get Ubuntu installed on it before you continue.

I used this cross compiling X264 using Ubuntu as a base for part of my script. The entire script follows but if you want to make sure you get a good copy you can download it:

mkdir ffmpeg
cd ffmpeg

BASEDIR=`pwd`
MINGBASE=$BASEDIR/win32-webm

sudo apt-get install pkg-config yasm subversion cvs git-core gcc-mingw32

cat > mingw32-setup.sh << EOF
#!/bin/sh
export CC=i586-mingw32msvc-gcc
export CXX=i586-mingw32msvc-g++
export CPP=i586-mingw32msvc-cpp
export AR=i586-mingw32msvc-ar
export RANLIB=i586-mingw32msvc-ranlib
export ADD2LINE=i586-mingw32msvc-addr2line
export AS=i586-mingw32msvc-as
export LD=i586-mingw32msvc-ld
export NM=i586-mingw32msvc-nm
export STRIP=i586-mingw32msvc-strip

export PATH="/usr/i586-mingw32msvc/bin:$PATH"
export PKG_CONFIG_PATH="$MINGBASE/lib/pkgconfig/"
exec "\$@"
EOF
chmod +x mingw32-setup.sh

mkdir $MINGBASE $MINGBASE/src $MINGBASE/lib $MINGBASE/include $MINGBASE/share $MINGBASE/bin

cd $MINGBASE/src

wget -qO - http://downloads.xiph.org/releases/ogg/libogg-1.2.0.tar.gz | tar xzf -
cd libogg-1.2.0
$BASEDIR/mingw32-setup.sh ./configure --enable-static --disable-shared --host=mingw32
make
DESTDIR=$MINGBASE make install prefix=
cd ..

wget -qO - http://downloads.xiph.org/releases/vorbis/libvorbis-1.3.1.tar.gz | tar xzf -
cd libvorbis-1.3.1
LDFLAGS="-L$MINGBASE/lib" CFLAGS="-I$MINGBASE/include" $BASEDIR/mingw32-setup.sh ./configure --enable-static --disable-shared --host=mingw32
make
DESTDIR=$MINGBASE make install prefix=
cd ..

wget -qO - ftp://sourceware.org/pub/pthreads-win32/pthreads-w32-2-8-0-release.tar.gz | tar xzf -
cd pthreads-w32-2-8-0-release
make GC-static CROSS=i586-mingw32msvc-
cp libpthreadGC2.a $MINGBASE/lib
cp *.h $MINGBASE/include
cd ..

wget -qO - http://zlib.net/zlib-1.2.5.tar.gz | tar xzf -
cd zlib-1.2.5/
$BASEDIR/mingw32-setup.sh ./configure
sed -i"" -e 's/-lc//' Makefile
make
DESTDIR=$MINGBASE make install prefix=
cd ..

git clone git://review.webmproject.org/libvpx.git
cd libvpx/
git checkout fd0d7ff4c155b94d3f322addc7b66234b6908cc6
cat > /tmp/libvpx.x.patch << EOP
diff -rupN libvpx.orig/configure libvpx/configure
--- libvpx.orig/configure	2010-07-11 14:49:18.703230816 -0400
+++ libvpx/configure	2010-07-11 14:50:33.747300030 -0400
@@ -436,9 +436,9 @@ process_detect() {
         }
     fi
     check_header stdio.h || die "Unable to invoke compiler: \${CC} \${CFLAGS}"
-    check_ld < /tmp/ffmpeg-0.6_libvpx-0.9.1.diff
wget -qO - http://www.ffmpeg.org/releases/ffmpeg-0.6.tar.gz | tar xzf -
cd ffmpeg-0.6
patch -p0 < /tmp/ffmpeg-0.6_libvpx-0.9.1.diff

LDFLAGS="-L$MINGBASE/lib" CFLAGS="-I$MINGBASE/include" ./configure --target-os=mingw32 --cross-prefix=i586-mingw32msvc- --arch=x86 --prefix=$MINGBASE --enable-memalign-hack --enable-gpl --enable-avisynth --enable-version3 --enable-postproc --enable-libvpx --enable-runtime-cpudetect --disable-network --disable-devices --disable-encoder=vorbis --enable-libvorbis
make

cp ffmpeg.exe $BASEDIR

Just transfer that script to the user you create when you installed Ubuntu and run it. You will need to provide your password once so that the system can install a few tools but after that everything is done automatically. At the end you will have exe files in the directory you ran the script from.

Transfer the exe files that are created to a windows machine and then transcode a video like this:

ffmpeg.exe -i Big_Buck_Bunny_1080p.avi -acodec libvorbis -ac 2 -ab 128k -ar 44100 -s 720×480 Big_Buck_Bunny_1080p.webm

Please note that you need to include the audio codec information above to correctly encode the video. If you don't include rate values you will get an error telling you the libvorbis can't be initialized.

When the transcoding is done you will have a WebM video. Grab your closest HTML5 video player like VideoJS and you are ready to go.

For a demo video encoded using FFMpeg created with the above script check out this WebM demo video. You will need to use one of the following browsers to view the demo (all of these are as of July 13 2010, support will eventually be added to the main build):

Chromium dev channel build
Firefox 4 beta
Opera
IE 9 when the user has the Direct Show DLLs installed

I have tested the demo video on each of the above, except for IE9 that doesn't have WebM support as of yet, and it seems to work pretty well.

How to Create iPad Formatted Videos Using HandBrake or FFMpeg

carson — Tue, 13 Apr 2010 10:56:19 +0000

This is for those who may want to load a video onto their iPad with iTunes that isn't in the correct format. I needed to do this because I was trying to put an iTunes University video on my iPad a couple days ago and iTunes complained that it wasn't in the correct format for the iPad. I'm not sure exactly how that could be but I decided to take the opportunity to see if I could use HandBrake to easily convert it to an iPad friendly format. There is currently no pre-loaded iPad configuration for HandBrake like there is for the iPhone and iPod Touch so I created a few profiles that can easily be imported into HandBrake to output different sizes for the iPad:

iPad-4×3.plist – A 4×3 aspect ratio at 640×480
iPad-16×9.plist – A 16×9 aspect ratio at 640×360
iPad-Full.plist – A 4×3 aspect ratio at 1024×576

Note that when you import videos using iTunes the iPad puts them in their own Videos app unlike the iPhone where they show up under the iPod app. You will want to find the Videos icon if you don't already know where it is:

After verifying that the above 4×3 version worked for the iTunes University video I went about testing it on a couple other video formats. I tested the Big Buck Bunny video that I also used for my post on iPad video streaming in both 640×360 and 1024×576 output formats. Both resolutions looked great. I also tried converting a DVD. If you decide to convert a DVD you will probably want to turn on de-interlacing in HandBrake. You do that by first selecting the "Picture Settings" option:

Then the filters tab and then select the type of de-interlace you want (fast, slow, slowest):

If you want to use FFMpeg to do all this you can. The following is a slightly modified version of the streaming command I'm using that will output a high bitrate version of the input video:

ffmpeg -y -i input.avi -acodec aac -ar 48000 -ab 128k -ac 2 -s 1024×768 -vcodec libx264 -b 1200k -flags +loop+mv4 -cmp 256 -partitions +parti4x4+partp8x8+partb8x8 -subq 7 -trellis 1 -refs 5 -coder 0 -me_range 16 -keyint_min 25 -sc_threshold 40 -i_qfactor 0.71 -bt 1200k -maxrate 1200k -bufsize 1200k -rc_eq 'blurCplx^(1-qComp)' -qcomp 0.6 -qmin 10 -qmax 51 -qdiff 4 -level 30 -aspect 16:9 -r 30 -g 90 -async 2 output.mp4

iPhone HTTP Streaming with FFMpeg and an Open Source Segmenter

carson — Sun, 28 Jun 2009 23:43:45 +0000

With the release of the iPhone OS 3 update came the ability to do live streaming. There are a few types of streaming and each requires a certain encoding and segmentation. I've put together a cheat sheet on how I went about building a static stream using FFMpeg and an example segmenter that someone has posted. I'm not covering windowed streams in this post but if you are thinking about implementing a windowed stream the following will help you make a step in that direction.

Before getting started it is best to read over the Apple documentation on HTTP live streaming. Start out with the iPhone streaming media overview. This document covers the basics of how the streaming works and has some nice diagrams.

If you want even more information after reading the overview you can take a look at the HTTP Live streaming draft proposal that was submitted to the IETF by Apple. It covers the streaming protocol in complete detail and has examples of the stream file format for reference.

Once you are ready to start grab a decent quality video clip to use. If you don't have one handy I found a nice list of downloadable HD clips in various formats for testing.

Step 1: Grab the latest version of FFMpeg

You may be able to get away with anything after FFMpeg 0.5 but you might as well pull down a more recent version. The FFMpeg download page has instructions on getting the latest version. I pulled the version I used out of git.

I used the following command to configure FFMpeg:


configure --enable-gpl --enable-nonfree --enable-pthreads --enable-libfaac --enable-libfaad --enable-libmp3lame --enable-libx264

One of the main things to note is the --enable-libx264 flag.

Step 2: Encode your video for the iPhone

Once you have a working version of FFMpeg it is time to create an X264 encoded stream that will work with the iPhone. There are a few things to note before diving in:

The supported bitrates for streaming are: 100 Kbps to 1.6 Mbps
The suggested bitrates for streaming are*:
- Low – 96 Kbps video, 64 Kbps audio
- Medium – 256 Kbps video, 64 Kbps audio
- High – 800 Kbps video, 64 Kbps audio
The iPhone screen size is: 480×320

* See step 7 for more information on what I think are better bitrates.

Taking all that into account someone on the iPhone developer forums suggested the following and it works well for me:

ffmpeg -i  -f mpegts -acodec libmp3lame -ar 48000 -ab 64k -s 320×240 -vcodec libx264 -b 96k -flags +loop -cmp +chroma -partitions +parti4x4+partp8x8+partb8x8 -subq 5 -trellis 1 -refs 1 -coder 0 -me_range 16 -keyint_min 25 -sc_threshold 40 -i_qfactor 0.71 -bt 200k -maxrate 96k -bufsize 96k -rc_eq 'blurCplx^(1-qComp)' -qcomp 0.6 -qmin 10 -qmax 51 -qdiff 4 -level 30 -aspect 320:240 -g 30 -async 2 

If you want some more detail on some of these commands check out the X264 encoding guide and in general the FFMpeg documentation to see what all the flags mean.

Note that I have the bitrate set to 96k in the above example. That can be changed to fit your needs. Use the script that I have created later in the post or just make sure you change the -b, -maxrate, and -bufsize values.

Step 3: Download and build the segmenter

Now you have a complete video but you don't want to toss the entire thing up or you wouldn't be reading about HTTP streaming. What you need is a way to segment the video stream into smaller chunks. You can download Apple's segmenter (see the overview above for more information on where to find it) or you can download one created by the forum user corp186.

There is an SVN repository set up for the segmenter source. It is only a couple files and it is easy to build. The trouble you may run into is that the Makefile that it comes with won't build the binary correctly. Don't worry it just takes some extra link flags to make it work. The following is what I needed in the Makefile to get it to build on my system:

all:

        gcc -Wall -g segmenter.c -o segmenter -lavformat -lavcodec -lavutil -lbz2 -lm -lz -lfaac -lmp3lame -lx264 -lfaad
clean:

        rm segmenter

After compiling the segmenter you are ready to create your first HTTP streaming content.

The format of the segmenter command is:

segmenter     

Following is an example used to create a stream from a video file created with the above FFMpeg command split into 10 second intervals:

segmenter sample_low.ts 10 sample_low stream_low.m3u8 http://www.ioncannon.net/

Step 4: Prepare the HTTP server

At this point you should have a set of files that represent the stream and a stream definition file. Those files can be uploaded to a web server at this point but there is another important step to take that ensures they will be download correctly and that is setting up mime types. There are two mime types that are important for the streaming content:

.m3u8 application/x-mpegURL

.ts video/MP2T

If you are using Apache you would want to add the following to your httpd.conf file:

AddType application/x-mpegURL .m3u8

AddType video/MP2T .ts

If you are using lighttpd you would want to put this in your configuration file (if you have other mime types defined make sure you just add these and don't set them):

mimetype.assign = ( ".m3u8" => "application/x-mpegURL", ".ts" => "video/MP2T" )

Step 5: Test the stream

The video is encoded for the iPhone, segmented for streaming, and the server is configured. The only thing left to do is test the stream and the fastest way to do that is to use the new HTML5 video tag. Here is an example of how to set it up:

<html>

  <head>

    <title>Video Test</title>

    <meta name="viewport" content="width=320; initial-scale=1.0; maximum-scale=1.0; user-scalable=0;"/>

  </head>

  <body style="background-color:#FFFFFF; ">

    <center>

      width='150' height='150' src="stream-128k.m3u8" />

    </center>

  </body>

</html>

If everything has been done correctly you should see the video.

If you want to test the stream out in an application then download the MoviePlayer iPhone demo application from the iPhone developer site. Build and run it in the simulator or put it on an actual phone and then type the URL in for the server you uploaded your stream to.

That is all there is to building a single static HTTP stream. A good number of steps but if you have some experience using FFMpeg it isn't too hard to set up. The only pitfalls I ran into revolve around trying to segment the stream without the segmeter code. I don't know enough about how the segmentation works to know why this is so difficult to do but I believe it could have something to do with synchronization points in the stream. Of course when you stray from the path the stream just doesn't work and you get a generic error message so that is just my best guess. I'll also guess that Apple may tighten up the player over time and make it work better with miss-formatted streams.

Step 6: Automating the stream encoding and segmentation

Here is a little script I put together that first encodes an input file and then segments it into 10 second chunks:

#!/bin/sh
BR=800k
ffmpeg -i $1 -f mpegts -acodec libmp3lame -ar 48000 -ab 64k -s 320×240 -vcodec libx264 -b $BR -flags +loop -cmp +chroma -partitions +parti4x4+partp8x8+partb8x8 -subq 5 -trellis 1 -refs 1 -coder 0 -me_range 16 -keyint_min 25 -sc_threshold 40 -i_qfactor 0.71 -bt 200k -maxrate $BR -bufsize $BR -rc_eq 'blurCplx^(1-qComp)' -qcomp 0.6 -qmin 10 -qmax 51 -qdiff 4 -level 30 -aspect 320:240 -g 30 -async 2 sample_$BR_pre.ts
segmenter sample_$BR_pre.ts 10 sample_$BR stream-$BR.m3u8 http://www.ioncannon.net/
rm -f sample_$BR_pre.ts

The script could use some work but it does a good enough job for testing.

Step 7: Create a variable rate HTTP stream

Once you have creating a single stream down you need to try out creating a variable bitrate stream. There isn't much to it, just create different bitrate encoded streams and link to their stream definition files in a separate stream definition file. Here is an example:

#EXTM3U

#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=96000

http://192.168.132.15/ipv/stream-96k.m3u8

#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=256000

http://192.168.132.15/ipv/stream-256k.m3u8

#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=800000
http://192.168.132.15/ipv/stream-800k.m3u8

I gave the above a try using both the iPhone's 3G connection and a WIFI connection. The following log shows the two different attempts (first 3G then WIFI):


6.x.x.x ioncannon.net - [20:49:13] "GET /varpl.m3u8 HTTP/1.1" 304 0 "-" "..."
6.x.x.x ioncannon.net - [20:49:14] "GET /varpl.m3u8 HTTP/1.1" 206 288 "-" "..."
6.x.x.x ioncannon.net - [20:49:15] "GET /varpl.m3u8 HTTP/1.1" 200 288 "-" "..."
6.x.x.x ioncannon.net - [20:49:16] "GET /stream-96k.m3u8 HTTP/1.1" 200 719 "-" "..."
6.x.x.x ioncannon.net - [20:49:18] "GET /s_96k-00001.ts HTTP/1.1" 200 334828 "-" "..."
6.x.x.x ioncannon.net - [20:49:21] "GET /s_96k-00002.ts HTTP/1.1" 200 377880 "-" "..."
6.x.x.x ioncannon.net - [20:49:30] "GET /s_96k-00003.ts HTTP/1.1" 200 383520 "-" "..."
6.x.x.x ioncannon.net - [20:49:32] "GET /stream-256k.m3u8 HTTP/1.1" 200 730 "-" "..."
6.x.x.x ioncannon.net - [20:49:39] "GET /s_256k-00003.ts HTTP/1.1" 200 716844 "-" "..."
6.x.x.x ioncannon.net - [20:49:49] "GET /s_256k-00004.ts HTTP/1.1" 200 705564 "-" "..."
6.x.x.x ioncannon.net - [20:49:57] "GET /stream-96k.m3u8 HTTP/1.1" 200 719 "-" "..."
6.x.x.x ioncannon.net - [20:49:59] "GET /s_96k-00004.ts HTTP/1.1" 200 368668 "-" "..."
6.x.x.x ioncannon.net - [20:50:03] "GET /s_96k-00005.ts HTTP/1.1" 200 371300 "-" "..."
6.x.x.x ioncannon.net - [20:50:13] "GET /s_96k-00006.ts HTTP/1.1" 200 398936 "-" "..."
6.x.x.x ioncannon.net - [20:50:16] "GET /stream-256k.m3u8 HTTP/1.1" 200 730 "-" "..."
6.x.x.x ioncannon.net - [20:50:22] "GET /s_256k-00006.ts HTTP/1.1" 200 758016 "-" "..."
6.x.x.x ioncannon.net - [20:50:36] "GET /s_256k-00007.ts HTTP/1.1" 200 737524 "-" "..."
6.x.x.x ioncannon.net - [20:50:40] "GET /s_256k-00008.ts HTTP/1.1" 200 773244 "-" "..."
6.x.x.x ioncannon.net - [20:50:46] "GET /s_256k-00009.ts HTTP/1.1" 200 717032 "-" "..."
6.x.x.x ioncannon.net - [20:50:57] "GET /s_256k-00010.ts HTTP/1.1" 200 768920 "-" "..."
6.x.x.x ioncannon.net - [20:51:06] "GET /s_256k-00011.ts HTTP/1.1" 200 611000 "-" "..."

1.x.x.x ioncannon.net - [20:52:23] "GET /varpl.m3u8 HTTP/1.1" 304 0 "-" "..."
1.x.x.x ioncannon.net - [20:52:24] "GET /varpl.m3u8 HTTP/1.1" 206 288 "-" "..."
1.x.x.x ioncannon.net - [20:52:25] "GET /varpl.m3u8 HTTP/1.1" 200 288 "-" "..."
1.x.x.x ioncannon.net - [20:52:25] "GET /stream-96k.m3u8 HTTP/1.1" 200 719 "-" "..."
1.x.x.x ioncannon.net - [20:52:26] "GET /s_96k-00001.ts HTTP/1.1" 200 334828 "-" "..."
1.x.x.x ioncannon.net - [20:52:27] "GET /s_96k-00002.ts HTTP/1.1" 200 377880 "-" "..."
1.x.x.x ioncannon.net - [20:52:28] "GET /stream-800k.m3u8 HTTP/1.1" 200 730 "-" "..."
1.x.x.x ioncannon.net - [20:52:31] "GET /s_800k-00002.ts HTTP/1.1" 200 1774156 "-" "..."
1.x.x.x ioncannon.net - [20:52:34] "GET /s_800k-00003.ts HTTP/1.1" 200 1916096 "-" "..."
1.x.x.x ioncannon.net - [20:52:38] "GET /s_800k-00004.ts HTTP/1.1" 200 1831872 "-" "..."
1.x.x.x ioncannon.net - [20:52:41] "GET /s_800k-00005.ts HTTP/1.1" 200 1831496 "-" "..."
1.x.x.x ioncannon.net - [20:52:46] "GET /s_800k-00006.ts HTTP/1.1" 200 1967608 "-" "..."
1.x.x.x ioncannon.net - [20:52:50] "GET /s_800k-00007.ts HTTP/1.1" 200 1676208 "-" "..."
1.x.x.x ioncannon.net - [20:52:54] "GET /s_800k-00008.ts HTTP/1.1" 200 2094132 "-" "..."
1.x.x.x ioncannon.net - [20:52:58] "GET /s_800k-00009.ts HTTP/1.1" 200 1860260 "-" "..."
1.x.x.x ioncannon.net - [20:53:08] "GET /s_800k-00010.ts HTTP/1.1" 200 2008404 "-" "..."
1.x.x.x ioncannon.net - [20:53:19] "GET /s_800k-00011.ts HTTP/1.1" 200 1400224 "-" "..."

Notice that there is a decent bit of indecisiveness on the part of what stream to pick when using 3G. For my test it actually caused the player to pause while it switched from the 256k stream back to the 96k stream. The stream on the WIFI connection starts out low but then jumps right to the highest quality and stays there. Overall it seems like the variable rate streaming works decently and again Apple may be able to tweak it down the road to get even better results.

The bitrate jump between 96k and 256k is probably too large even though that is what Apple seems to recommend. I believe with some testing a better set of bitrates could be found. The video quality of the 256k bitrate looks pretty good so I would say that 96k, 128k and 384k would potentially be a better choice.

Some parting notes

If you are interested in how the segmenter works you can find out more on how to use libavformat at the following resources: an older libavformat tutorial, some sample libavformat code, How to Write a Video Player in Less Than 1000 Lines, and more sample libavformat code.

The next step for this is to do a windowed live stream. I've done a little experimenting so far and with a modified segmeter I can generate a live stream. I will need to heavily modify the segmeter to get a live windowed stream so it may take a little while to get it done. My intent of course will be to combine the modifications with something fun like S3 and cloudfront since I believe that would be a sweat combination.