iPhone HTTP Streaming with FFMpeg and an Open Source Segmenter

With the release of the iPhone OS 3 update came the ability to do live streaming. There are a few types of streaming and each requires a certain encoding and segmentation. I've put together a cheat sheet on how I went about building a static stream using FFMpeg and an example segmenter that someone has posted. I'm not covering windowed streams in this post but if you are thinking about implementing a windowed stream the following will help you make a step in that direction.

Before getting started it is best to read over the Apple documentation on HTTP live streaming. Start out with the iPhone streaming media overview. This document covers the basics of how the streaming works and has some nice diagrams.

If you want even more information after reading the overview you can take a look at the HTTP Live streaming draft proposal that was submitted to the IETF by Apple. It covers the streaming protocol in complete detail and has examples of the stream file format for reference.

Once you are ready to start grab a decent quality video clip to use. If you don't have one handy I found a nice list of downloadable HD clips in various formats for testing.

Step 1: Grab the latest version of FFMpeg

You may be able to get away with anything after FFMpeg 0.5 but you might as well pull down a more recent version. The FFMpeg download page has instructions on getting the latest version. I pulled the version I used out of git.

I used the following command to configure FFMpeg:

configure --enable-gpl --enable-nonfree --enable-pthreads --enable-libfaac --enable-libfaad --enable-libmp3lame --enable-libx264

One of the main things to note is the --enable-libx264 flag.

Step 2: Encode your video for the iPhone

Once you have a working version of FFMpeg it is time to create an X264 encoded stream that will work with the iPhone. There are a few things to note before diving in:

  1. The supported bitrates for streaming are: 100 Kbps to 1.6 Mbps
  2. The suggested bitrates for streaming are*:
    • Low – 96 Kbps video, 64 Kbps audio
    • Medium – 256 Kbps video, 64 Kbps audio
    • High – 800 Kbps video, 64 Kbps audio
  3. The iPhone screen size is: 480×320

* See step 7 for more information on what I think are better bitrates.

Taking all that into account someone on the iPhone developer forums suggested the following and it works well for me:

ffmpeg -i <in file> -f mpegts -acodec libmp3lame -ar 48000 -ab 64k -s 320x240 -vcodec libx264 -b 96k -flags +loop -cmp +chroma -partitions +parti4x4+partp8x8+partb8x8 -subq 5 -trellis 1 -refs 1 -coder 0 -me_range 16 -keyint_min 25 -sc_threshold 40 -i_qfactor 0.71 -bt 200k -maxrate 96k -bufsize 96k -rc_eq 'blurCplx^(1-qComp)' -qcomp 0.6 -qmin 10 -qmax 51 -qdiff 4 -level 30 -aspect 320:240 -g 30 -async 2 <output file>

If you want some more detail on some of these commands check out the X264 encoding guide and in general the FFMpeg documentation to see what all the flags mean.

Note that I have the bitrate set to 96k in the above example. That can be changed to fit your needs. Use the script that I have created later in the post or just make sure you change the -b, -maxrate, and -bufsize values.

Step 3: Download and build the segmenter

Now you have a complete video but you don't want to toss the entire thing up or you wouldn't be reading about HTTP streaming. What you need is a way to segment the video stream into smaller chunks. You can download Apple's segmenter (see the overview above for more information on where to find it) or you can download one created by the forum user corp186.

There is an SVN repository set up for the segmenter source. It is only a couple files and it is easy to build. The trouble you may run into is that the Makefile that it comes with won't build the binary correctly. Don't worry it just takes some extra link flags to make it work. The following is what I needed in the Makefile to get it to build on my system:

all:
        gcc -Wall -g segmenter.c -o segmenter -lavformat -lavcodec -lavutil -lbz2 -lm -lz -lfaac -lmp3lame -lx264 -lfaad

clean:
        rm segmenter

After compiling the segmenter you are ready to create your first HTTP streaming content.

The format of the segmenter command is:

segmenter <input MPEG-TS file> <segment duration in seconds> <output MPEG-TS file prefix> <output m3u8 index file> <http prefix>

Following is an example used to create a stream from a video file created with the above FFMpeg command split into 10 second intervals:

segmenter sample_low.ts 10 sample_low stream_low.m3u8 http://www.ioncannon.net/

Step 4: Prepare the HTTP server

At this point you should have a set of files that represent the stream and a stream definition file. Those files can be uploaded to a web server at this point but there is another important step to take that ensures they will be download correctly and that is setting up mime types. There are two mime types that are important for the streaming content:

.m3u8 application/x-mpegURL
.ts video/MP2T

If you are using Apache you would want to add the following to your httpd.conf file:

AddType application/x-mpegURL .m3u8
AddType video/MP2T .ts

If you are using lighttpd you would want to put this in your configuration file (if you have other mime types defined make sure you just add these and don't set them):

mimetype.assign = ( ".m3u8" => "application/x-mpegURL", ".ts" => "video/MP2T" )

Step 5: Test the stream

The video is encoded for the iPhone, segmented for streaming, and the server is configured. The only thing left to do is test the stream and the fastest way to do that is to use the new HTML5 video tag. Here is an example of how to set it up:

<html>
  <head>
    <title>Video Test</title>
    <meta name="viewport" content="width=320; initial-scale=1.0; maximum-scale=1.0; user-scalable=0;"/>
  </head>
  <body style="background-color:#FFFFFF; ">
    <center>
      <video width='150' height='150' src="stream-128k.m3u8" />
    </center>
  </body>
</html>

If everything has been done correctly you should see the video.

If you want to test the stream out in an application then download the MoviePlayer iPhone demo application from the iPhone developer site. Build and run it in the simulator or put it on an actual phone and then type the URL in for the server you uploaded your stream to.

That is all there is to building a single static HTTP stream. A good number of steps but if you have some experience using FFMpeg it isn't too hard to set up. The only pitfalls I ran into revolve around trying to segment the stream without the segmeter code. I don't know enough about how the segmentation works to know why this is so difficult to do but I believe it could have something to do with synchronization points in the stream. Of course when you stray from the path the stream just doesn't work and you get a generic error message so that is just my best guess. I'll also guess that Apple may tighten up the player over time and make it work better with miss-formatted streams.

Step 6: Automating the stream encoding and segmentation

Here is a little script I put together that first encodes an input file and then segments it into 10 second chunks:

#!/bin/sh

BR=800k

ffmpeg -i $1 -f mpegts -acodec libmp3lame -ar 48000 -ab 64k -s 320x240 -vcodec libx264 -b $BR -flags +loop -cmp +chroma -partitions +parti4x4+partp8x8+partb8x8 -subq 5 -trellis 1 -refs 1 -coder 0 -me_range 16 -keyint_min 25 -sc_threshold 40 -i_qfactor 0.71 -bt 200k -maxrate $BR -bufsize $BR -rc_eq 'blurCplx^(1-qComp)' -qcomp 0.6 -qmin 10 -qmax 51 -qdiff 4 -level 30 -aspect 320:240 -g 30 -async 2 sample_$BR_pre.ts

segmenter sample_$BR_pre.ts 10 sample_$BR stream-$BR.m3u8 http://www.ioncannon.net/

rm -f sample_$BR_pre.ts

The script could use some work but it does a good enough job for testing.

Step 7: Create a variable rate HTTP stream

Once you have creating a single stream down you need to try out creating a variable bitrate stream. There isn't much to it, just create different bitrate encoded streams and link to their stream definition files in a separate stream definition file. Here is an example:

#EXTM3U
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=96000
http://192.168.132.15/ipv/stream-96k.m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=256000
http://192.168.132.15/ipv/stream-256k.m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=800000
http://192.168.132.15/ipv/stream-800k.m3u8

I gave the above a try using both the iPhone's 3G connection and a WIFI connection. The following log shows the two different attempts (first 3G then WIFI):

6.x.x.x ioncannon.net - [20:49:13] "GET /varpl.m3u8 HTTP/1.1" 304 0 "-" "..."
6.x.x.x ioncannon.net - [20:49:14] "GET /varpl.m3u8 HTTP/1.1" 206 288 "-" "..."
6.x.x.x ioncannon.net - [20:49:15] "GET /varpl.m3u8 HTTP/1.1" 200 288 "-" "..."
6.x.x.x ioncannon.net - [20:49:16] "GET /stream-96k.m3u8 HTTP/1.1" 200 719 "-" "..."
6.x.x.x ioncannon.net - [20:49:18] "GET /s_96k-00001.ts HTTP/1.1" 200 334828 "-" "..."
6.x.x.x ioncannon.net - [20:49:21] "GET /s_96k-00002.ts HTTP/1.1" 200 377880 "-" "..."
6.x.x.x ioncannon.net - [20:49:30] "GET /s_96k-00003.ts HTTP/1.1" 200 383520 "-" "..."
6.x.x.x ioncannon.net - [20:49:32] "GET /stream-256k.m3u8 HTTP/1.1" 200 730 "-" "..."
6.x.x.x ioncannon.net - [20:49:39] "GET /s_256k-00003.ts HTTP/1.1" 200 716844 "-" "..."
6.x.x.x ioncannon.net - [20:49:49] "GET /s_256k-00004.ts HTTP/1.1" 200 705564 "-" "..."
6.x.x.x ioncannon.net - [20:49:57] "GET /stream-96k.m3u8 HTTP/1.1" 200 719 "-" "..."
6.x.x.x ioncannon.net - [20:49:59] "GET /s_96k-00004.ts HTTP/1.1" 200 368668 "-" "..."
6.x.x.x ioncannon.net - [20:50:03] "GET /s_96k-00005.ts HTTP/1.1" 200 371300 "-" "..."
6.x.x.x ioncannon.net - [20:50:13] "GET /s_96k-00006.ts HTTP/1.1" 200 398936 "-" "..."
6.x.x.x ioncannon.net - [20:50:16] "GET /stream-256k.m3u8 HTTP/1.1" 200 730 "-" "..."
6.x.x.x ioncannon.net - [20:50:22] "GET /s_256k-00006.ts HTTP/1.1" 200 758016 "-" "..."
6.x.x.x ioncannon.net - [20:50:36] "GET /s_256k-00007.ts HTTP/1.1" 200 737524 "-" "..."
6.x.x.x ioncannon.net - [20:50:40] "GET /s_256k-00008.ts HTTP/1.1" 200 773244 "-" "..."
6.x.x.x ioncannon.net - [20:50:46] "GET /s_256k-00009.ts HTTP/1.1" 200 717032 "-" "..."
6.x.x.x ioncannon.net - [20:50:57] "GET /s_256k-00010.ts HTTP/1.1" 200 768920 "-" "..."
6.x.x.x ioncannon.net - [20:51:06] "GET /s_256k-00011.ts HTTP/1.1" 200 611000 "-" "..."

1.x.x.x ioncannon.net - [20:52:23] "GET /varpl.m3u8 HTTP/1.1" 304 0 "-" "..."
1.x.x.x ioncannon.net - [20:52:24] "GET /varpl.m3u8 HTTP/1.1" 206 288 "-" "..."
1.x.x.x ioncannon.net - [20:52:25] "GET /varpl.m3u8 HTTP/1.1" 200 288 "-" "..."
1.x.x.x ioncannon.net - [20:52:25] "GET /stream-96k.m3u8 HTTP/1.1" 200 719 "-" "..."
1.x.x.x ioncannon.net - [20:52:26] "GET /s_96k-00001.ts HTTP/1.1" 200 334828 "-" "..."
1.x.x.x ioncannon.net - [20:52:27] "GET /s_96k-00002.ts HTTP/1.1" 200 377880 "-" "..."
1.x.x.x ioncannon.net - [20:52:28] "GET /stream-800k.m3u8 HTTP/1.1" 200 730 "-" "..."
1.x.x.x ioncannon.net - [20:52:31] "GET /s_800k-00002.ts HTTP/1.1" 200 1774156 "-" "..."
1.x.x.x ioncannon.net - [20:52:34] "GET /s_800k-00003.ts HTTP/1.1" 200 1916096 "-" "..."
1.x.x.x ioncannon.net - [20:52:38] "GET /s_800k-00004.ts HTTP/1.1" 200 1831872 "-" "..."
1.x.x.x ioncannon.net - [20:52:41] "GET /s_800k-00005.ts HTTP/1.1" 200 1831496 "-" "..."
1.x.x.x ioncannon.net - [20:52:46] "GET /s_800k-00006.ts HTTP/1.1" 200 1967608 "-" "..."
1.x.x.x ioncannon.net - [20:52:50] "GET /s_800k-00007.ts HTTP/1.1" 200 1676208 "-" "..."
1.x.x.x ioncannon.net - [20:52:54] "GET /s_800k-00008.ts HTTP/1.1" 200 2094132 "-" "..."
1.x.x.x ioncannon.net - [20:52:58] "GET /s_800k-00009.ts HTTP/1.1" 200 1860260 "-" "..."
1.x.x.x ioncannon.net - [20:53:08] "GET /s_800k-00010.ts HTTP/1.1" 200 2008404 "-" "..."
1.x.x.x ioncannon.net - [20:53:19] "GET /s_800k-00011.ts HTTP/1.1" 200 1400224 "-" "..."

Notice that there is a decent bit of indecisiveness on the part of what stream to pick when using 3G. For my test it actually caused the player to pause while it switched from the 256k stream back to the 96k stream. The stream on the WIFI connection starts out low but then jumps right to the highest quality and stays there. Overall it seems like the variable rate streaming works decently and again Apple may be able to tweak it down the road to get even better results.

The bitrate jump between 96k and 256k is probably too large even though that is what Apple seems to recommend. I believe with some testing a better set of bitrates could be found. The video quality of the 256k bitrate looks pretty good so I would say that 96k, 128k and 384k would potentially be a better choice.

Some parting notes

If you are interested in how the segmenter works you can find out more on how to use libavformat at the following resources: an older libavformat tutorial, some sample libavformat code, How to Write a Video Player in Less Than 1000 Lines, and more sample libavformat code.

The next step for this is to do a windowed live stream. I've done a little experimenting so far and with a modified segmeter I can generate a live stream. I will need to heavily modify the segmeter to get a live windowed stream so it may take a little while to get it done. My intent of course will be to combine the modifications with something fun like S3 and cloudfront since I believe that would be a sweat combination.

198 thoughts on “iPhone HTTP Streaming with FFMpeg and an Open Source Segmenter

  1. Vicky

    Hi All,
    Thanks for the wonderful job. It works very well.
    I met a lot of troubles when compiling the FFmpeg. Finnally, I resolved the problems by using a old version – ffmpeg-5.0. People who is struggling with FFmpeg can try it.

    I'm just wondering how to get the streaming log. Can anybody tell me?

  2. Wei

    Hi,
    Ignore the last comment. I made a stupid mistake. Some place in the segmenter.c was changed occasionally.
    Sorry.

  3. ram

    Can I create the mpeg2ts chunks directly without creating a .ts file and the using the segmenter to create chunks.

    Thanks,

    Ram.

  4. Scott Kidder

    I've made some minor changes to the segmenter.c source file to allow audio-only segmenting. I've used it extensively with FFMPEG-encoded AAC audio files. Here are the patches:

    — segmenter.c 2009-07-29 17:33:15.000000000 -0700
    +++ segmenter.c.audio 2010-02-25 09:44:04.462813091 -0800
    @@ -280,13 +280,15 @@

    dump_format(oc, 0, output_prefix, 1);

    – codec = avcodec_find_decoder(video_st->codec->codec_id);
    – if (!codec) {
    + if (video_index >=0) {
    + codec = avcodec_find_decoder(video_st->codec->codec_id);
    + if (!codec) {
    fprintf(stderr, "Could not find video decoder, key frames will not be honored\n");
    – }
    + }

    – if (avcodec_open(video_st->codec, codec) codec, codec) codec);
    + if (video_index >= 0) {
    + avcodec_close(video_st->codec);
    + }

    for(i = 0; i nb_streams; i++) {
    av_freep(&oc->streams[i]->codec);

  5. Mark

    Hey Scott,

    I can't get your patch to work. Is your modified segmenter available elsewhere?

    I need to encode AAC audio as well.

    Thanks!

    Mark

  6. skiria

    When I try to segment the .ts after encode with ffmpeg segmenter produce this:
    dts < PCR, TS is invalid

    How can I solve it? What's the problem?
    Thanks

  7. carson Post author

    For everyone having "dts < PRC, TS is invalid" issues it looks like it has been fixed in recent versions of FFMpeg. Check here for the discussion of the issue:

    http://archives.free.net.ph/thread/20090721.235201.0f49abe2.en.html#20090721.235201.0f49abe2
    http://archives.free.net.ph/message/20090721.152453.b727c066.en.html

    Here is the patch if anyone is interested:

    https://roundup.ffmpeg.org/issue1279

    Finally, I verified that the patch is in the ffmpeg-0.5.1 version. It looks like even more work has been done to mpegts in the current git version.

  8. skiria

    I have the current ffmpeg version (ffmpeg 0.5.1) and it encodes ok. But "dts <- PCR, TS is invalid" appears when I try to segment the ts file encoded with ffmpeg. So, I supose that the problems is in segmenter not in ffmpeg.

    Thanks carson for your message but this solution doesn't works on my computer.

    Thanks in advance

  9. carson Post author

    @skiria The segmenter uses the same libavformat code as FFMpeg so it wouldn't be any different between the two. I haven't seen the warning since upgrading so I would guess it is a problem with the input stream. From my experience the message is also just a warning and doesn't seem to cause any problems with the resulting encoding although I could imagine it might for long running video streams. FFMpeg is pretty nice but it can't convert everything. You may try converting the input into another format first and then using the segmenter.

  10. thanatos

    The "dts <- PCR, TS is invalid" bug is still there in 0.5.1. I've tried with many different types of input files, but still getting it. The error is both outputed by ffmpeg and the open source segmenter. The Apple segmenter complains about finding "pes but no timecode".

    I'm also having problems that previous posts mention.

    The resulting files (from both open source segmenter and Apple segmenter) play on the iPhone, but after seeking in the movie, the video runs at wrong speed (it is fast forwarding) while the audio plays fine. Other times when seeking the video starts flickering and stutering so that lip sync is lost.

    I there any other tool than ffmpeg that can be used? Would an older revision of ffmpeg solve this?

  11. Pingback: Life is Short » Apple’s HTTP Live Streaming: A Nightmare?

  12. bgit

    How can I test the segmented files on the computer? I generate a index file with the locations of the segments, like this:
    #EXTINF:10,
    video–1.ts
    #EXTINF:10,
    video-2.ts
    (I save the index file and the segments on the same folder)

    And I test whith a html file with video tag, with the source of video–> src="video-index.m3u8"

    But when I try to test a html file on safari anything is played. What can be wrong? Maybe on the locations? I also get PCR, Ts invalid but the .ts files can be played on vlc.

    Thanks in advance

  13. skiria

    I stil have PCR, ts not valid error in 0.5.1 version. When I try to play the ts segments in vlc, audio is out and video jumps. I try to test on Safari and it doesn't play anything.

    Anybody have the solution? How can i solve this?

  14. Ryan Lu

    For ffmpeg compile,
    ./configure –enable-gpl –enable-nonfree –enable-pthreads –enable-libfaac –enable-libfaad –enable-libmp3lame –enable-libx264 –extra-cflags=-I/opt/include/ –extra-ldflags=-L/opt/lib/
    Some may encounter the faac/faad/x264/lame error during configure, just download faad2.7 faac1.1.28 x264 and lame (all can be gotten from sourceforge.net) and configure it with –prefix=/opt, then configure it again.

    ————————–
    For segmenter compile,
    gcc -Wall -g segmenter.c -o segmenter -lavformat -lavcodec -lavutil -L../../ffmpeg/libavcodec/ -L../../ffmpeg/libavformat/ -L../../ffmpeg/libavutil/ -lbz2 -lm -lz -lfaac -lmp3lame -lx264 -lfaad -L/opt/lib -lpthread
    Where the ../../ffmpeg/ must be the lastest ffmpeg, otherwith you will suffer dts warning still.
    —————————————-
    For encoding
    ./ffmpeg -y -i 1mbps_70s.mp4 -f mpegts -acodec libmp3lame -ar 48000 -ab 128k -vcodec libx264 -s 380×480 -b 1000k -flags +loop -cmp +chroma -subq 5 -trellis 1 -refs 1 -coder 0 -me_range 16 -keyint_min 25 -sc_threshold 40 -i_qfactor 0.71 -bt 1500k -maxrate 1000k -bufsize 1000k -rc_eq 'blurCplx^(1-qComp)' -qcomp 0.6 -qmin 10 -qmax 51 -qdiff 4 -level 30 -g 30 -aspect 380:480 -async 2 1mbps_70s.ts

    Some may encounter pts<pcr warning, this is due to you're using outdated ffmpeg. Just check out the newest snapshot via git or svn (svn may suffer problem is you're inside NAT, so use git instead).
    ———————–
    For segment
    segmenter ../../1mbps_70s.ts 10 1mbps_70 1mbps_70.m3u8 http://x.x.x.x

    Now, you can play the segmented ts in VLC.

  15. crazyfool2100

    I prefer to use HandBrakeCLI on Ubuntu 10.04 rather than ffmpeg. Ive had less errors and problems converting videos for the iphone with that software. Here is what I have used to successfully convert an flv file to a ts file segmented for streaming:

    # Convert FLV to MP4
    HandBrakeCLI -i input.flv -o output.mp4 -e x264 -q 0.589999973773956 -a 1 -E faac -B 64 -R 48 -6 dpl2 -f mp4 -b 96 -X 320 -x level=30:cabac=0:ref=2:mixed-refs:analyse=all:me=umh:no-fast-pskip=1:subq=5

    # Convert MP4 to MPEGTS
    ffmpeg -i output.mp4 -f mpegts -vcodec copy -acodec copy -vbsf h264_mp4toannexb output.ts

    # Segment output.ts
    segmenter output.ts 10 output output.m3u8 http://domain.com/video/

    Works great. Just make sure to compile the segmenter with the flag -lpthread if you get any pthread errors.

  16. thanatos

    I've got better results now when using the latest ffmpeg from git as suggested in earlier posts.

    If I play one bitrate at a time it works flawlessly now.

    But if I use the same streams together to get adaptive streaming I still get the "fast forwarding" effect some times.

    I guess the TS support in ffmpeg is not 100% fixed yet.

  17. madhu

    Hi Carson,
    Thank you so much for the post.
    And I have few doubts can you please help me.
    I used your cmd as it is like below

    ffmpeg -i -f mpegts -acodec libmp3lame -ar 48000 -ab 64k -s 320×240 -vcodec libx264 -b 96k -flags +loop -cmp +chroma -partitions +parti4×4+partp8×8+partb8×8 -subq 5 -trellis 1 -refs 1 -coder 0 -me_range 16 -keyint_min 25 -sc_threshold 40 -i_qfactor 0.71 -bt 200k -maxrate 96k -bufsize 96k -rc_eq 'blurCplx^(1-qComp)' -qcomp 0.6 -qmin 10 -qmax 51 -qdiff 4 -level 30 -aspect 320:240 -g 30 -async 2

    It thrown "Incorrect frame size" error. Then I changed "-s 320×240" to "-s 480×320".
    After this it thrown "Invalid value '+parti4×4+partp8×8+partb8×8' for option 'partitions'". So I removed "-partitions +parti4×4+partp8×8+partb8×8" as well then it started converting.
    Do I have to do any thing else to get rid of these errors??

    And One more thing After converting my 3min 12sec video to 96k bit video I got 5.7MB file size, am I in the right track??

    Thanks,
    Madhu.

  18. Pavan

    Sorry for the double post, my error message disappeared from the previous post because it had tags in it – so I've replaced the brackets with parentheses…
    Has anyone had this problem… no matter what I input into the segmenter, I get the following message:
    "Usage: live_segmenter (segment length) (output location) (filename prefix) (encoding profile)"
    I believe I've linked all headers and shared libraries correctly. I am using the most up to date versions of ffmpeg and codecs… But I can't figure out what I've done wrong. I'm on centos 5.4… maybe that could be a problem? If anyone has experienced this or has some ideas, please let me know.

  19. Farish

    Hi,

    I have accomplished the conversion to MPEG2 TS amd tjem segmentation using the open source segmenter u have mentioned. I am trying to use an Apache webserver but I dont have an iphone or Mac available.

    When I open the url (http://localhost/sample_segment.m3u8) using VLC media player or Gnome Mplayer, the segments play but there is noticeable delay when one segment is over and the next begins. I think this is because there is no buffering of next segments beforehand.

    Is there a solution which allows to play the segments without the glitches?

  20. Farish

    I checked the log of Apache web server and found out that the HTTP GET requests arrived at about 10 or more seconds apart (I m running the server locally on the same computer which has the segmented video files). so the next fragments are not requested quickly enough by VLC.

    PS: i m using a segment duration of 10 secs.

    my next problem is that both ffmpeg and segmenter were running fine before but due to a problem, i had to reinstall ffmpeg (i installed the latest version) and now when i run the segmenter, it gives the error of dts < pts, TS is invalid.

    has someone been able to solve this problem? which version of ffmpeg doesnt give this error? i do get the segments but except for the first few segments, there is no sound in the others.

  21. Ben

    The Apple HTTP streaming tools don't seem to be available on the developer website anymore. Does anyone know where to find them?

    The segmenter svn in this post is still there, but the Makefile doesn't work for me (even with the suggested revisions).

  22. Mikko Myllyniemi

    Hi,

    Carson:
    Thanks for the tutorial.

    Madhu:
    As mentioned earlier message replace '×' with basic 'x' with partitions and frame size (320×240). It seems that there has been used some special font with example command.

  23. Gerard Aguilar

    http://tinyurl.com/28afpko
    Apple also mentions the audio only with still image option in section 10. Could you please let me know if you are aware of exactly how this can be done with ffmpeg using a HE-AAC file and a jpeg image. I assume some mpeg.ts stream is created with perhaps a single IFrame for video and an aac audio ES stream or a specially designed container format for the image, but can't seem to figure out how to create such a stream that would be compliant with the iPhone. My application is to embed coverart into an audio stream between every song. Any suggestions? Any help appreciated. Thanks

  24. Haehchen

    @Clint J
    @Peter S.
    @other windows

    If you complie the segmenter on windows you get this error:
    On Windows I'm seeing two m3u8 files, one called stream.m3u8 and one .stream.m3u8, and the .ts file cleanup doesn't take place when adding a windowing size.

    You must patch the return value on the write_index_file function. On windows the rename of the temporary m3u8 will fail. so replace it:

    remove(index);
    if (rename(tmp_index, index)) {
    fprintf(stderr, "Could not rename %s to %s\n", tmp_index, index);
    return -1;
    }

    return 0;

    i am not a c freak but this helped me. by the way i compiled it with mingw32 and msys. i will do some tests and create a ready zip file with a static (i hope) binary.

    PS: you should use this segmenter.c (http://projects.vdr-developer.org/git/?p=istreamdev.git;a=tree;f=segmenter). they cleaned up some lines.

  25. Loic Bertron

    First of All, Thanks a lot for your amazing work on HTTP Live Streaming for iPhone !
    But i have a little question for you. Actually, I'm encoding 3 different bitrates from a live RTMP Feed and sometimes Ffmpeg just crashed ! I already modify a little bit the process to loop and continue the show, but everytime i relauch the process, it's rewriting over the actual playlist, so clients can't continue the live you know. They have to relauch their players !
    I took a look to your code, but i'm not a developper and i can't understant how to bypass your function in Ruby who's rewriting the playlist.
    I have 2 idea :
    – When i launch the process, check if the main playlist exists. if it exist, just continue incrementing from the last index in that file. But i don't know how to do that using the Queue in Ruby.
    – When i launch the process, start from the beginning but append the new file in the playlist.
    Do you have an idea how to that ?

    Sorry for my poor English, i'm trying to improve it but it's still bad !
    Thank you again for your fantastic job with the iPhone. I hope you will be able to help mi with that.

  26. Timbo

    I'm getting the "fast forward" effect on all my videos. Audio plays normally, but the video speed is roughly 2x faster than it should be.

    I've tried lowering the encoding bitrate/fps/etc but no joy. Tried reducing the segment lengths but that hasn't helped either.

    Using ffmpeg SVN revision 23684.
    Using assembla's segmenter SVN revision 50.

    Any tips are much appreciated!

  27. James

    I'm still having not much joy with the segmenter. I followed (as I'm sure many others did) the guide here verbatim to install the latest x264, libvpx and ffmpeg:
    http://ubuntuforums.org/showthread.php?t=786095

    resulting in:
    ffmpeg_4:SVN-r23876-1_amd64.deb
    x264_2:0.100.1659+git57b2e56-1_amd64.deb
    libvpx_201006290029-git-1_amd64.deb

    To compile the segmenter against the latest ffmpeg i did the following:
    gcc -Wall -g segmenter.c -o segmenter -lavformat -lavcodec -lavutil -L../ffmpeg/libavcodec/ -L../ffmpeg/libavformat/ -L../ffmpeg/libavutil/ -lbz2 -lm -lz -lfaac -lmp3lame -lx264 -lfaad -L/opt/lib -lpthread -lxvidcore -lvpx -lopencore-amrnb -lopencore-amrwb -logg -lvorbis -lvorbisenc -ltheora -ltheoraenc -ltheoradec

    I compiled a sample vid using a command I poached from this thread somewhere:
    ffmpeg -i trailer_1080p.ogg -f mpegts -acodec libmp3lame -ar 48000 -ab 1000k -s 320×240 -vcodec libx264 -b 96k -flags +loop -cmp +chroma -subq 5 -trellis 1 -refs 1 -coder 0 -me_range 16 -keyint_min 25 -sc_threshold 40 -i_qfactor 0.71 -bt 200k -maxrate 96k -bufsize 96k -rc_eq 'blurCplx^(1-qComp)' -qcomp 0.6 -qmin 10 -qmax 51 -qdiff 4 -level 30 -aspect 320:240 -g 30 -async 2 onewfile.ts

    I have tried running the original segmenter source, plus the audio enabled one mentioned on this thread and the one hosted on google code and they all have the same problem:

    ./segmenter ../encodes/onewfile.ts 10 onewfile onewfile.m3u8 http://somewebsite.com

    mpegts @ 0x253e170] max_analyze_duration reachedOutput #0, mpegts, to 'file':
    Stream #0.0: Video: libx264, yuv420p, 480×320, q=2-31, 90k tbn, 25 tbc
    Stream #0.1: Audio: libmp3lame, 48000 Hz, 2 channels, 64 kb/s
    [mpegts @ 0x2551490] muxrate 1 bps, pcr every 5 pkts, sdt every 200, pat/pmt every 40 pkts
    [mpegts @ 0x253e170] invalid dts/pts combination
    Last message repeated 812 times
    [NULL @ 0x2552910] missing picture in access unit

    The result is the output above, plus only two segments have been generated when I would have expected 4 (the video is 32 seconds long). Playing the vid in vlc results in sped up playback of the video, but the audio plays at correct speed.

    The encoded ts plus the segmented ts files (and playlist) are available here if anyone is interested:
    http://www.metacdn.org/sample.zip

    Any suggestions would be most appreciated.

  28. Ofer Kalisky

    Hi Carson,

    I am trying to create a TS with FFMPEG (from some other TS) that doesn't begin in timestamp 0.0 but rather takes the timestamp of the original TS. I tried the ffmpeg "copyts" option with no avail.
    You wrote: "One of those reasons is that it is said to not produce timestamps but there is actually a way to do that so you can offset the timestamps in the resulting stream so they match the offset". How did you manage to do that?

    Thanks, Ofer.

  29. Natsu

    Has anybody tried viewing live streaming on a web-browser. It runs fine on the iPhone as well as on the Safari on my mac. But it doesn't play on Safari on a PC (or any other browsers on my mac). Is there a solution out there for this? And i'm using the HTML 5 element to display the stream on the browser. (actually it displays the text contents of the m3u8 file on the safari on a PC, which is really weird! :\ )

  30. Natsu

    And could this have anything to do with the fact that I'm doing live streaming which is updating the m3u8 index file every now and then with the list of the newly created TS file segments?

  31. André van Toly

    Running in the same bugs as some of the latest posts with the segmenter. I believe it is related to one of the avcodec or avformat libraries included with the latest checkouts of ffmpeg. When I download the sample.zip from @James his post I can make valid segments with an older version of ffmpeg ("FFmpeg version SVN-r21627"). It results in 4 segments like you would expect.

    However, when I use a more recent checkout of ffmpeg ("FFmpeg version SVN-r23561") like on one of our servers it results in the same problems @James mentions: two segments, with the last segment being substantially larger then the others and audio out of sync.

    Maybe others can verify this? Point to a solution?

  32. JLee

    Thanks for all the excellent help at this site!
    I've got the segmenter etc. working (on Mac OS 10.6 — it's really annoying that mediafilesegmenter seems to have disappeared!), and I can create a stream from a file. I'm doing VOD, and I want to be able to do seeking as fast and flexibly as possible — emulating what you can do with rtmp as far as I can. Viewing the result in Safari, using the video tag, I get very strange behaviour whereby after a while of scrolling around the player seems to lose track of where it is in the video. Hard to describe more clearly, maybe you can see it yourself at
    http://ddm.caad.ed.ac.uk/staff/jlee/streamqt.html
    I do not get this behaviour on the iPhone, but also this behaviour is NOT what I get in desktop Safari viewing the Apple bip/bop streams, e.g.
    http://devimages.apple.com/iphone/samples/bipbopgear1.htm
    (which all seem to seek under javascript control or scroll manually just as one would expect).

    I can't see anything different about what I have, but it seems it must be something about the encoding of the segments.
    I guess this is an issue of rather narrow interest, but I'd be grateful if anyone has any thoughts on it …
    Thanks! – John.

  33. John

    @James – I've been running into the same problem as you. I'm a bit fuzzy on mpeg transport streams, unfortunately, but from what I can piece together, I suspect it's a bug in the generation of the original transport stream.
    If you look at the issue (171) that motivated the dts/pts warning, it has a reference to the TS standard stating that a stream should only have a DTS if it's different than the PTS. Looking at the ts I'm getting out of ffmpeg, I've always got DTS==PTS, which triggers that warning.

    In the short term, I hacked up libavformat not to blow away pts and dts when that happens, but that's clearly not the right fix.

  34. John

    Huh – this turns out to be useful:
    #define AVFMT_FLAG_IGNDTS 0x0008 ///< Ignore DTS on frames that contain both DTS & PTS

  35. Liu Yue

    Great job!
    i want to live streaming the video from my Logitech Quickcam usb camera.
    but your materials are from media files.
    i have install video4linux and i can see the video with vlc.
    can it work?

    thank you!

  36. Rudy

    Hello,

    thank you for that good job!

    I still run into one problem, at the last ts file, the video stops, audio keeps playing. When I count the created files and multiply by the segemnt size in seconds, there is always less playingtime in total than the original file had. Anyone experiencing the same, anyone can give me a clue??

    Thanks for your help,
    Rudy

  37. Jamie

    I've been looking into making this work again, and I'm still having problems.
    I am using latest svn versions of ffmpeg and x264.

    Segmenter no longer complains about my transport stream, but the video run twice as fast as the audio ;(
    I've tried playing the transport stream directly generated by ffmpeg, and everything is in sync. So I suspect a problem somewhere in the segmenter.

    With the lastest libavformat and libx264, the segmenter code will no longer compile, so it may need some updating.

  38. Dirk

    There is a patch for the new vlc1.1.2:
    http://techblog.unwiredappeal.com/2010/03/vlc-http-live-streaming-module-patch.html
    So no segmenter needed anymore.
    vlc does the segmentation.
    I take a http stream from my dreambox, and transcode with vlc to a .ts file with segmentation (to watch tv over the internet)

    /usr/bin/vlc http://ip_dreambox:8001/1:0:1:6D66:437:1:C00000:0:0:0: –intf=rc –sout='#transcode{threads=2,width=340,height=240,fps=25,vcodec=h264,ab=24,
    vb=256,channels=1,samplerate=8000,venc=x264{aud,profile=baseline,level=30,
    keyint=30,bframes=0,ref=1,nocabac},channels=1,acodec=mp4a,ab=16}:
    duplicate{dst=std{access=livehttp{seglen=10,delsegs=true,numsegs=5,index=/srv/www/htdocs/stream/sample.m3u8,index-url=http://xxx.dyndns.org:8080/stream/sample-########.ts},mux=ts{use-key-frames},dst=/srv/www/htdocs/stream/sample-########.ts}}'

  39. René

    Hi, first of all many thanks for the awesome work been done.

    But i have the same issues as Jamie.
    If i play the generated files generated by ffmpeg via vlc the file plays absolutely in sync. After segmenting i just get nearly half of the expected segments.

    What i see is that the tbr value changes between the two steps. So, if the output of ffmpeg is 25 tbr, after segmenting it is going to 50 tbr.

    Seems like a segmenter / libavformat issue. Is there already a solution for that ?

    I used the latest ffmpeg / x264 versions from last Saturday.

    Thanks for any help on that issue.

  40. carson Post author

    @Dirk I took a look at the VLC module but it looks like it is fairly limited at this point. Hopefully it will continue to be worked on.

  41. pulczynski

    @René – Ok, found something. live_segmenter.c works exactly the same (I was looking for some simple mistake in code). Difference is how ffmpeg analyzes stream vs disk file. If you do something live


    cat test.ts | ./segmenter - 10 prefix /dev/null whatever

    you will get proper tbr but unseekable files.

    For me it's some kind of ffmpeg analyzes video – container information vs codec information.

    Was you video interlaced BEFORE converting to .ts?

  42. pulczynski

    @René – this simple patch fixes your problem. It seems that your container is 25fps but h264 stream is not hinted properly (defaulting to 50fps).


    Index: segmenter.c
    ===================================================================
    --- segmenter.c (revision 7)
    +++ segmenter.c (working copy)
    @@ -295,6 +295,7 @@
    case CODEC_TYPE_VIDEO:
    video_index = i;
    ic->streams[i]->discard = AVDISCARD_NONE;
    + ic->streams[i]->codec->time_base.den = ic->streams[i]->r_frame_rate.num;
    video_st = add_output_stream(oc, ic->streams[i]);
    break;
    case CODEC_TYPE_AUDIO:

  43. René

    @pulczynski – Thanks for your replies. Sorry for my late answer. Right now i am using 2 different versions of FFmpeg, one for encoding, the other for segmenting. But i will try your soulution. Will be much better if it works. Thank you very much.

  44. XeM

    @Haehchen – I downloaded binary from location you mentioned above on Google. But, as I run segmenter on ts file I get following.
    [NULL @ 005bdbe0]missing picture in access unit. I've tried this on more than one files.
    Hope you can give some direction.

Leave a Reply

Your email address will not be published. Required fields are marked *