iPhone Windowed HTTP Live Streaming Server

After some more work I have enhanced the HTTP segmenter and uploading script from my iPhone streaming using AWS S3 and Cloudfront post. I added a number of features and tried to pull together some of the ideas from the comments. I'll go over some of the features here and there is a full list of configuration options on the HTTP Live Video Streaming server project page and the source is available at the github repo.

So the major changes I have added are:

  • Yaml based configuration file. See the project page for a complete list of options.
  • Ability to transfer segments via copy, ftp, scp and s3.
  • Added the ability to do variable bitrate streams.
  • Added re-streaming support.
  • Added logging to a file and better debug output.

The variable bitrate streams where done by using pipes. I haven't done a large amount of testing but it seems to work fairly well. I am able to stream a live HD video source into 3 different bitrates on a fairly old PC. Here are a couple clips I created to show the progressive enhancement in action, you probably want to switch to the HD version of the video and watch it full screen to get the bet view:


The configuration file will allow for any number of encoding options or transfer options and they can be put together in a number of different ways. Here are a couple examples of both, see the example configuration files for more.

An encoder example:

ep_128k:
  ffmpeg_command: "ffmpeg -er 4 -y -i %s -f mpegts -acodec libmp3lame -ar 48000 -ab 64k -s 320x240 -vcodec libx264 -b 128k -flags +loop -cmp +chroma -partitions +parti4x4+partp8x8+partb8x8 -subq 5 -trellis 1 -refs 1 -coder 0 -me_range 16 -keyint_min 25 -sc_threshold 40 -i_qfactor 0.71 -bt 128k -maxrate 128k -bufsize 128k -rc_eq 'blurCplx^(1-qComp)' -qcomp 0.6 -qmin 10 -qmax 51 -qdiff 4 -level 30 -aspect 320:240 -g 30 -async 2 - | %s %s %s %s %s"
  bandwidth: 128000

Transfer configuration example:

ftp_dev:
  transfer_type: 'ftp'
  remote_host: '192.168.1.1'
  user_name: 'user'
  password: 'pass'
  directory: 'html/streamingvideo'

As a final note on changes, you are no longer able to use the segmenter without the script now really. If you want to do that you should use the original version of the segmenter source.

Please note that there is still some work to be done on the script to be complete. If I have time my next enhancement will be to add encryption and I will probably try to test builds on other distributions (maybe attempt to create segmenter binaries).

63 thoughts on “iPhone Windowed HTTP Live Streaming Server

  1. Sean Goheen

    This is awesome, it is so much better then the hacked version (Tried to add ftp support, didn't work out.) I have been trying to get to work but, it's giving me an error, any help would be appreciated.

    This is on a Mac Pro G5, with all of the prerequisites installed through MacPorts. It gives the same error when it is started by the ruby script.


    $ ./live_segmenter

    dyld: Library not loaded: libopennet.dylib
    Referenced from: /opt/local/lib/libconfig.0.2.3.dylib
    Reason: image not found
    Trace/BPT trap

  2. Markus

    Hi Carson,

    works fine for me on my 1st tests…

    2 questions:

    It is possible for scp-uploads to connect to non-standard ssh ports?

    Logging per stdout and per file at the same time?

    thx for the great work here and greetings from berlin :)

  3. JW

    Carson, first of all thanks for the great work. The new features are awesome.

    I got the stream running using a AVI file but I'm having trouble sending a stream through using VLC.

    From the command line:
    cvlc [mms stream url] --sout "#transcode{vcodec=h264,venc=x264,scale=1,width=320,height=240,acodec=mp3,ab=64,channels=2,samplerate=48000}:std{access=file,mux=ts,dst=-}" | ./http_streamer.rb config-multirate.yml

    The reason I'm using VLC is because ffmpeg can't take a mms stream. But since I'm using it just as a pass through the command is probably could be simpler…

    Another stupid question: When sending the stream through a pipe called '-' do you have to first issue the mkfifo command of "mkfifo -" /end stupid question ;)

  4. Eno

    Anyone tested the segmenter with FME 3.0 with MainConcept AAC Encoder plugin that creates base line 3.0/acc mpg4 feed ?!!

  5. Mait

    hey,

    sorry for the newbie question but how do get pass these compilation errors:

    live_segmenter.c: In function 'add_output_stream':
    live_segmenter.c:45: warning: implicit declaration of function 'exit'
    live_segmenter.c:45: warning: incompatible implicit declaration of built-in function 'exit'
    live_segmenter.c:58: error: 'AVCodecContext' has no member named 'ticks_per_frame'
    live_segmenter.c:61: error: 'AVCodecContext' has no member named 'ticks_per_frame'
    live_segmenter.c:71: error: 'AVCodecContext' has no member named 'channel_layout'
    live_segmenter.c:71: error: 'AVCodecContext' has no member named 'channel_layout'
    live_segmenter.c: In function 'main':
    live_segmenter.c:124: warning: implicit declaration of function 'atoi'
    live_segmenter.c:130: warning: implicit declaration of function 'malloc'
    live_segmenter.c:130: warning: incompatible implicit declaration of built-in function 'malloc'
    live_segmenter.c:134: warning: incompatible implicit declaration of built-in function 'exit'
    live_segmenter.c:145: warning: incompatible implicit declaration of built-in function 'exit'
    live_segmenter.c:153: warning: incompatible implicit declaration of built-in function 'exit'
    live_segmenter.c:159: warning: incompatible implicit declaration of built-in function 'exit'
    live_segmenter.c:166: warning: incompatible implicit declaration of built-in function 'exit'
    live_segmenter.c:169: warning: implicit declaration of function 'avformat_alloc_context'
    live_segmenter.c:169: warning: initialization makes pointer from integer without a cast
    live_segmenter.c:173: warning: incompatible implicit declaration of built-in function 'exit'
    live_segmenter.c:207: warning: incompatible implicit declaration of built-in function 'exit'
    live_segmenter.c:228: warning: incompatible implicit declaration of built-in function 'exit'
    live_segmenter.c:234: warning: incompatible implicit declaration of built-in function 'exit'

    thanks!
    Mait

  6. more fiya

    @Mait..

    You either do not have all libraries installed or your -Include/-Library path does not contain the location of the needed libs..

  7. Mait

    thanks, i got it compiled now!

    another question:
    how could is use an RTSP resource as an input? i understand it should be somehow via VLC but how exactly?

    regards,
    Mait

  8. Ron

    trying to run simple make and have all ffmpeg, faad, faac libs installed, but getting hung up on -lconfig …

    Which lib am I missing?

  9. Ron

    pulling my hair out here. keep getting snagged on just trying to compile your sources correctly.

    I've got x264,faad,faac,mp3lame,and latest ffmpeg all compiled — as far as the segmenter source "as easy as running make in the root" — gives me:

    cc -Wall -g live_segmenter.c -o live_segmenter -lavformat -lavcodec -lavutil -lbz2 -lm -lz -lfaac -lmp3lame -lx264 -lfaad
    Undefined symbols:
    "_av_free_packet", referenced from:
    _main in ccV7L3nn.o
    _main in ccV7L3nn.o
    _main in ccV7L3nn.o
    _main in ccV7L3nn.o
    ld: symbol(s) not found
    collect2: ld returned 1 exit status
    make: *** [all] Error 1

    what am i doing wrong?

  10. Martin

    Does anyone know how to compile live_segmenter.c statically, so it can be moved to other computers, without having to install all the libraries?

    I have been trying on my Intel Mac 10.5, but can't get it to work.

    Anyone who can give some pointers?

    Thanks
    /Martin

  11. Markus

    @ more fiya

    thx for the response … but the method
    Net::SSH.start( 'host', 1234, 'user', 'passwd' ) do |session|
    dosent work…
    error-message Error running transfer: wrong number of arguments (4 for 3)

    after some hour of testing i find a method

    1. edit the /etc/ssh/ssh_config file
    Port
    2. edit http_streamer.rb

    ...
    require 'net/scp'
    require 'net/sftp' #new
    ...
    when 'scp'
    if transfer_config.has_key?('password')
    Net::SCP.upload!(transfer_config['remote_host'], transfer_config['user_name'], source_file, transfer_config['directory'] + '/' + destination_file, :password => transfer_config['password'])
    else
    Net::SCP.upload!(transfer_config['remote_host'], transfer_config['user_name'], source_file, transfer_config['directory'] + '/' + destination_file)
    end
    when 'sftp' #new
    Net::SFTP.start(transfer_config['remote_host'], transfer_config['user_name'], :password => transfer_config['password']) do |sftp| #new
    sftp.upload!(source_file, transfer_config['directory'] + '/' + destination_file) #new
    end
    when 's3'
    ...

    3. edit your config.yml file

    scp_dev to sftp_dev

    or create a new section sftp_dev

    sftp_dev:
    transfer_type: 'sftp'
    remote_host: 'xxx.xxx.xxx.xxx'
    user_name: 'user'
    password: 'xxxxx!'
    directory: '/remotepath'

    and it work

  12. JP

    @Martin & all

    Having the same issue as you – libopennet isn't being found:
    $> make
    gcc -Wall -I/opt/local/include -L/opt/local/lib -g live_segmenter.c -o live_segmenter -lavformat -lavcodec -lavutil -lbz2 -lm -lz -lfaac -lmp3lame -lx264 -lfaad -lconfig
    $> ./live_segmenter
    dyld: Library not loaded: libopennet.dylib
    Referenced from: /opt/local/lib/libconfig.0.2.3.dylib
    Reason: image not found
    Trace/BPT trap

    Would very much appreciate some help, dying to try out this new tech! (Using Snow Leopard, standard install of macports for libraries)

  13. Martin

    JP,

    Leave out -lconfig and compile?

    I did, and it works fine without it.

    I don't know what it's for.

  14. JP

    Martin,

    Thanks for your quick reply – leaving out lconfig allows it to run properly (I'll test it further to see if the outputs are correct, but that'll take a while). Good tip!

    I'm also running find to see what libopennet files are floating about – looks like this is it:

    /opt/local/lib/libopennet.0.9.9.dylib
    /opt/local/lib/libopennet.0.dylib
    /opt/local/lib/libopennet.a
    /opt/local/lib/libopennet.dylib
    /opt/local/var/macports/distfiles/libopennet
    /opt/local/var/macports/distfiles/libopennet/libopennet-0.9.9.tar.gz
    /opt/local/var/macports/receipts/libopennet
    /opt/local/var/macports/software/libopennet
    /opt/local/var/macports/software/libopennet/0.9.9_0/opt/local/lib/libopennet.0.9.9.dylib
    /opt/local/var/macports/software/libopennet/0.9.9_0/opt/local/lib/libopennet.0.dylib
    /opt/local/var/macports/software/libopennet/0.9.9_0/opt/local/lib/libopennet.a
    /opt/local/var/macports/software/libopennet/0.9.9_0/opt/local/lib/libopennet.dylib
    /opt/local/var/macports/sources/rsync.macports.org/release/ports/net/libopennet

    Everything seems to be in order too:
    ls -l /opt/local/lib/libopennet.dylib
    lrwxr-xr-x 1 root admin 22 30 Aug 22:09 /opt/local/lib/libopennet.dylib -> libopennet.0.9.9.dylib

    Strange behaviour ne?

    Cheers again, JP

  15. More Fiya

    @JP

    Try setting your library path.. then try compiling..

    Copy and paste the below..

    export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/local/lib

  16. freewind

    @More Fiya,

    Could we use VLC and bypass ffmpeg?
    I mean, VLC could transcode the RTSP stream, why should we transcode the outut of VLC again with ffmpeg? why not use VLC to transcode the stream to our desired format then pipe to the Segmenter? You know transcode the stream again and again will cost a lot of CPU resource.

  17. john p

    hello,

    i'm not sure if this question is appropriate here. i got everything works the way they should using the sample clips your links provide. however, when i transcode our clips (they are of .mxf format) i got the following errors;

    Output #0, mpegts, to 'clip1.ts':
    Stream #0.0: Video: libx264, yuv420p, 320×240 [PAR 1:1 DAR 4:3], q=10-51, 800 kb/s, 90k tbn, 29.97 tbc
    Stream #0.1: Audio: aac, 48000 Hz, 4 channels, s16, 64 kb/s
    Stream mapping:
    Stream #0.0 -> #0.0
    Stream #0.1 -> #0.1
    Press [q] to stop encoding
    [mpeg2video @ 0x1004600]intra matrix invalid, ignoring
    Resampling with input channels greater than 2 unsupported.
    Can not resample 4 channels @ 48000 Hz to 4 channels @ 48000 Hz

    how can i fix this? or, if you know of some forums i can check, please give the pointers (i'm googling too but think you folks may know off hand)

    tia,

    johnp

  18. Martin

    I'm still looking for pointers on how to compile live_segmenter.c (statically?), so that it can't be moved to other machines or used in an application, without the need for the associated libraries?

    Thanks
    /Martin

  19. More Fiya

    @freewind..

    Take a look on the VLC forums.. I believe there may be development in the works to make VLC more segmentation friendly.. There are users there who also have the same questions you have.. The answers given in response are better than any answer I can give.

  20. john p

    … just wondering if you folks have tried with clips larger than 10G (yes, Gigs)? the transcoding part seems ok, but when the segmenter kicks in, it fails with the following error:

    Output #0, mpegts, to 'vict2008_800':
    Stream #0.0: Video: libx264, yuv420p, 320×240, q=2-31, 90k tbn, 29.97 tbc
    Stream #0.1: Audio: aac, 0 channels, s16
    [mpegts @ 0xa6ec360]sample rate not set
    Could not write mpegts header to first output file

    maybe some of you have run into solutions for similar problem, please give some pointers..

    great many thanks,
    johnp

  21. More Fiya

    @john p..

    if you try a 1gig file what happens?

    just a thought.. maybe try compiling with large file support..

    gcc -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64

  22. john p

    @More Fiya

    i think the threshold is around 1gig… some folks in the other forums suggest transcoding with audio rate at 22.5khz. i tried that and it doesn't really work. i will try your suggestion and see if it works. will keep you folks posted when i found a solution…

    cheers,
    johnp

  23. Stijn Tintel

    Hello, first of all I would like to thank you for your work on this. Unfortunately I too am having an issue with it…

    Got compilation sorted out rather easily, although both Debian and Ubuntu both lack some packages to get this to compile. Compiled out of the box on Gentoo, but don't have the V4L hardware in it.

    Now, I am using /dev/video0 (BT878) as an input device. To get this to work, I had to add '-f video4linux2 -s 320×240', to the encoding profile I am using, right before the '-i %s' option. Now, I see that ffmpeg starts grabbing frames from the video device, but after 20-40 frames, the output is no longer updated.

    I put strace in front of the segmenter_binary, which showed me that the segmenter just hangs after the 4th read. Looks like ffmpeg is writing to a pipe, but the segmenter stops reading from it – although I am not getting a SIGPIPE or anything useful. It just hangs and ffmpeg slowly starts filling memory (I suspect because the segmenter no longer reads from the pipe).

    Any ideas what could be wrong, and how I could solve it? Ran my tests on Ubuntu 9.04, with .deb's that I created from the Ubuntu Jaunty source package (needed to have libx264 support). Because this wasn't ffmpeg >= 0.5, I also tried with the source package from Ubuntu Karmic which is 0.5-svn20090706, but I always ran into the same problem.

    Right now I am updating the system to the latest Ubuntu testing, in the hopes that it would solve anything, but I don't think it will help. Any suggestions would be very appreciated.

  24. Mait

    @More Fiya

    for this VLC command
    vlc rtsp://192.168.1.1/file.wmv –sout '#transcode{vcodec=theo,vb=800,scale=1,acodec=vorb,ab=128,channels=2,samplerate=44100}:duplicate{dst=std{access=http,mux=ogg,dst=0.0.0.0:8080}}',
    what should i use as a input_location and source_command ?

    regards,
    Mait

  25. Aidan

    Hi Carson,

    I want to congratulate you on a great post.
    I have an issue I hope you can help me with.
    I have a problem when trying to restream from VLC. I am using the simple restream configuration and the ts and m3u8 files are being created. The problem comes when `i try to view the stream on an iPod Touch. It just sits of 'Loading Movie File' and no video ever plays.

    Any ideas on what I am doing wrong?

    Thanks for all the great work,
    Aidan

  26. Brian Blood

    Been playing around with these pieces.
    It would be really great if someone who has gotten this to work/understands it better than me, would post a command line call to VLC that does all the right transcoding/capturing/etc. to pull from the built-in iSight on a MacBook Pro and send the result to a udp destination.
    TIA.

  27. Robert

    Hi,

    Can anyone compile the segmenter to a windows .exe to make it run on windows 2003 server? Or can you give detailed step-by-step instructions how to do it including what tools to use. I am only a web programmer and don't know how to do C and compile it. Haven't done it yet. Please help!

    Thanks,
    Robert.

  28. Rene

    Hi,

    first of all, thank you for very nice documentation for the segmenting. But i have one problem getting the http_streamer working.

    I'm working on an debian machine and when i try to prcess i get the following error message:
    sh: line 3: ftp_dev:: command not found
    sh: line 4: scp_dev:: command not found
    sh: line 5: copy_dev:: command not found

    I can't understand why. Is it an ruby issue ? In my understanding the require settings are looking for all commands used in the script.

    Is there anyone who can help me ?

    Thanks,
    René

  29. Kerem

    It seems like there is a problem with index file creation. Consider this scenario: Say, I have a very long video. I set 10 segments to be in the index file and each segment will be 10 seconds long. And I have a very fast encoder computer, which can encode and copy a segment in 1 second. I start encoding and at the same time, open the stream on my iPhone. After 10 seconds of encoding, the first segments will start to be removed from the index file, because my computer encodes, copies the file and updates index file every second. Say, After 20 seconds, my phone redownloads the index file, and in this scenario, the oldest segment in that file will be 11th one. But it is still playing segment 2 and it has to download segment 3 as the next segment. Instead of segment 3, it downloads segment 11 and I lose the segments between 3 and 11.

    So, when you have a fast encoder computer, the computer updates the index file much faster than the phone downloads segments. The updating of the index file (and maybe encoding) should pause when the encoder is too fast. Say, I encoded 50 seconds worth of data in 5 seconds, I should pause for like 45 seconds in order to have the phone catch up with me.

    I hope I could make the problem clear. Any ideas about the solution? :)

  30. Ashley Taylor

    Hi there folks,

    I'm having a few problems compiling the segmenter. I am using Debian Lenny (64bit) and all of the packages are installed – however I am getting the following errors when I try and compile live_segmenter.c:
    http://pastebin.com/m3f35ea36

    A few friends tell me it's because the versions I am using are either too old or too new. The libraries I am using are straight from apt (stable) (Debian package manager).

    Does anyone know of any solutions to this issue?

    Regards,

    Ashley Taylor

  31. yulian

    Hey Mait,
    how you resolve your first Problem?
    I have exact the same but i can not find any help…

    gcc -Wall -g live_segmenter.c -o live_segmenter -lavformat -lavcodec -lavutil -lm -lmp3lame -lfaad -lfaac -lbz2 -lpthread -lz -lx264
    live_segmenter.c: In function ‘add_output_stream’:
    live_segmenter.c:45: warning: implicit declaration of function ‘exit’
    live_segmenter.c:45: warning: incompatible implicit declaration of built-in function ‘exit’
    live_segmenter.c:58: error: ‘AVCodecContext’ has no member named ‘ticks_per_frame’
    live_segmenter.c:61: error: ‘AVCodecContext’ has no member named ‘ticks_per_frame’
    live_segmenter.c:71: error: ‘AVCodecContext’ has no member named ‘channel_layout’
    live_segmenter.c:71: error: ‘AVCodecContext’ has no member named ‘channel_layout’
    live_segmenter.c: In function ‘main’:
    live_segmenter.c:124: warning: implicit declaration of function ‘atoi’
    live_segmenter.c:130: warning: implicit declaration of function ‘malloc’
    live_segmenter.c:130: warning: incompatible implicit declaration of built-in function ‘malloc’
    live_segmenter.c:134: warning: incompatible implicit declaration of built-in function ‘exit’
    live_segmenter.c:145: warning: incompatible implicit declaration of built-in function ‘exit’
    live_segmenter.c:153: warning: incompatible implicit declaration of built-in function ‘exit’
    live_segmenter.c:159: warning: incompatible implicit declaration of built-in function ‘exit’
    live_segmenter.c:166: warning: incompatible implicit declaration of built-in function ‘exit’
    live_segmenter.c:169: warning: implicit declaration of function ‘avformat_alloc_context’
    live_segmenter.c:169: warning: initialization makes pointer from integer without a cast
    live_segmenter.c:173: warning: incompatible implicit declaration of built-in function ‘exit’
    live_segmenter.c:207: warning: incompatible implicit declaration of built-in function ‘exit’
    live_segmenter.c:228: warning: incompatible implicit declaration of built-in function ‘exit’
    live_segmenter.c:234: warning: incompatible implicit declaration of built-in function ‘exit’
    make: *** [all] Fehler 1

  32. fapestniegd

    What version of ffmpeg svn (specifically) did segmenter.c compile against? I get errrors with the latest svn and the 0.5 release, and the one that ships with lenny (which I was sure wouldn't work, but tried anyway)

    (and I've ensurede my -I and $LD_LIBRARY_PATH point to /usr/local/include and /usr/local/lib (respectively) where ffmpeg installed from source…)

  33. Bruce Bertrand

    Why does the bandwidth calculation in the encoding profile only take into account the video bandwidth?

    For example, in the example profile in this post, you have a video bitrate of 128k and an audio bitrate of 64k, for a total of 192k. But the "bandwidth" setting is 128000.

    This is also the case in the example config files.

  34. trebla

    ..just wondering if anyone has seen this symptom: for the same 120 seconds mpeg-ts file, generated by ffmpeg, the number of segments generated by the open source segmenter is 10 whereas the number of segments generated by the apple segmenter is 12. I also tried it on a longer clip, 5880 seconds. 470 segments generated by the open source segmenter vs 588 segments generated by the apple segmenter. The duration of each segment in all cases is set to 10 seconds. Also there is a message, e.g. [NULL@0xfea980]missing picture in access unit, at the end of the open source segmenter command execution. The message is apparently spit out by ffmpeg libavcodec/h264_parser.c. Is this an issue with ffmpeg or an issue with the segmenter? Any pointers would be greatly appreciated. Thanks.

  35. xargle

    Chaps, great work on this. Apple have recently published a spec on recommended live streaming formats :

    TN2224

    Also the streams generated need to pass the streamvalidator, currently the bitrates of the generated streams are way off.

    Has anyone got improved FFMPEG parameters to produce files that pass?

  36. carson Post author

    @xargle Thanks for bringing that up. I've been trying to find some time to go back and apply changes that I've been given as well as tweak some stuff in the segmenter. The validation tool will give me extra reason to do so.

  37. Pingback: iPad Streaming Video and More

  38. Estelle

    Great stuff. Thank you so much. I need to use a UDP MpegTS stream as input to the segmenter. Does ffmpeg handle live streams as input. I thought it did. But, I can find no information on how to do this. Do you or any of your readers know about this?

  39. carson Post author

    You can use whatever you want to feed the ffmpeg processes. The examples all use cat to do so but you could maybe use something like netcat instead. I haven't tried it myself but I'll stick it on my list to test out.

  40. Stanislav

    Very interesting project.
    Has anyone decide problem with rollover in write_frame() after 26 hours?

Leave a Reply

Your email address will not be published. Required fields are marked *