After some more work I have enhanced the HTTP segmenter and uploading script from my iPhone streaming using AWS S3 and Cloudfront post. I added a number of features and tried to pull together some of the ideas from the comments. I'll go over some of the features here and there is a full list of configuration options on the HTTP Live Video Streaming server project page and the source is available at the github repo.
So the major changes I have added are:
- Yaml based configuration file. See the project page for a complete list of options.
- Ability to transfer segments via copy, ftp, scp and s3.
- Added the ability to do variable bitrate streams.
- Added re-streaming support.
- Added logging to a file and better debug output.
The variable bitrate streams where done by using pipes. I have done a large amount of testing via ITT Systems and it seems to work fairly well. I am able to stream a live HD video source into 3 different bitrates on a fairly old PC. Here are a couple clips I created to show the progressive enhancement in action, you probably want to switch to the HD version of the video and watch it full screen to get the bet view:
The configuration file will allow for any number of encoding options or transfer options and they can be put together in a number of different ways. Here are a couple examples of both, see the example configuration files for more.
An encoder example:
ep_128k: ffmpeg_command: "ffmpeg -er 4 -y -i %s -f mpegts -acodec libmp3lame -ar 48000 -ab 64k -s 320x240 -vcodec libx264 -b 128k -flags +loop -cmp +chroma -partitions +parti4x4+partp8x8+partb8x8 -subq 5 -trellis 1 -refs 1 -coder 0 -me_range 16 -keyint_min 25 -sc_threshold 40 -i_qfactor 0.71 -bt 128k -maxrate 128k -bufsize 128k -rc_eq 'blurCplx^(1-qComp)' -qcomp 0.6 -qmin 10 -qmax 51 -qdiff 4 -level 30 -aspect 320:240 -g 30 -async 2 - | %s %s %s %s %s" bandwidth: 128000
Transfer configuration example:
ftp_dev: transfer_type: 'ftp' remote_host: '192.168.1.1' user_name: 'user' password: 'pass' directory: 'html/streamingvideo'
As a final note on changes, you are no longer able to use the segmenter without the script now really. If you want to do that you should use the original version of the segmenter source.
Please note that there is still some work to be done on the script to be complete. If I have time my next enhancement will be to add encryption and I will probably try to test builds on other distributions (maybe attempt to create segmenter binaries).
This is awesome, it is so much better then the hacked version (Tried to add ftp support, didn't work out.) I have been trying to get to work but, it's giving me an error, any help would be appreciated.
This is on a Mac Pro G5, with all of the prerequisites installed through MacPorts. It gives the same error when it is started by the ruby script.
$ ./live_segmenter
dyld: Library not loaded: libopennet.dylib
Referenced from: /opt/local/lib/libconfig.0.2.3.dylib
Reason: image not found
Trace/BPT trap
Hi Carson,
works fine for me on my 1st tests…
2 questions:
It is possible for scp-uploads to connect to non-standard ssh ports?
Logging per stdout and per file at the same time?
thx for the great work here and greetings from berlin :)
Carson, first of all thanks for the great work. The new features are awesome.
I got the stream running using a AVI file but I'm having trouble sending a stream through using VLC.
From the command line:
cvlc [mms stream url] --sout "#transcode{vcodec=h264,venc=x264,scale=1,width=320,height=240,acodec=mp3,ab=64,channels=2,samplerate=48000}:std{access=file,mux=ts,dst=-}" | ./http_streamer.rb config-multirate.ymlThe reason I'm using VLC is because ffmpeg can't take a mms stream. But since I'm using it just as a pass through the command is probably could be simpler…
Another stupid question: When sending the stream through a pipe called '-' do you have to first issue the mkfifo command of "mkfifo -" /end stupid question ;)
Anyone tested the segmenter with FME 3.0 with MainConcept AAC Encoder plugin that creates base line 3.0/acc mpg4 feed ?!!
hey,
sorry for the newbie question but how do get pass these compilation errors:
live_segmenter.c: In function 'add_output_stream':
live_segmenter.c:45: warning: implicit declaration of function 'exit'
live_segmenter.c:45: warning: incompatible implicit declaration of built-in function 'exit'
live_segmenter.c:58: error: 'AVCodecContext' has no member named 'ticks_per_frame'
live_segmenter.c:61: error: 'AVCodecContext' has no member named 'ticks_per_frame'
live_segmenter.c:71: error: 'AVCodecContext' has no member named 'channel_layout'
live_segmenter.c:71: error: 'AVCodecContext' has no member named 'channel_layout'
live_segmenter.c: In function 'main':
live_segmenter.c:124: warning: implicit declaration of function 'atoi'
live_segmenter.c:130: warning: implicit declaration of function 'malloc'
live_segmenter.c:130: warning: incompatible implicit declaration of built-in function 'malloc'
live_segmenter.c:134: warning: incompatible implicit declaration of built-in function 'exit'
live_segmenter.c:145: warning: incompatible implicit declaration of built-in function 'exit'
live_segmenter.c:153: warning: incompatible implicit declaration of built-in function 'exit'
live_segmenter.c:159: warning: incompatible implicit declaration of built-in function 'exit'
live_segmenter.c:166: warning: incompatible implicit declaration of built-in function 'exit'
live_segmenter.c:169: warning: implicit declaration of function 'avformat_alloc_context'
live_segmenter.c:169: warning: initialization makes pointer from integer without a cast
live_segmenter.c:173: warning: incompatible implicit declaration of built-in function 'exit'
live_segmenter.c:207: warning: incompatible implicit declaration of built-in function 'exit'
live_segmenter.c:228: warning: incompatible implicit declaration of built-in function 'exit'
live_segmenter.c:234: warning: incompatible implicit declaration of built-in function 'exit'
thanks!
Mait
@Mait..
You either do not have all libraries installed or your -Include/-Library path does not contain the location of the needed libs..
@Markus..
Re: non-standard ports..
http://net-ssh.rubyforge.org/ssh/v1/chapter-2.html
@Sean..
What is the ftp error message you are receiving?
thanks, i got it compiled now!
another question:
how could is use an RTSP resource as an input? i understand it should be somehow via VLC but how exactly?
regards,
Mait
trying to run simple make and have all ffmpeg, faad, faac libs installed, but getting hung up on -lconfig …
Which lib am I missing?
pulling my hair out here. keep getting snagged on just trying to compile your sources correctly.
I've got x264,faad,faac,mp3lame,and latest ffmpeg all compiled — as far as the segmenter source "as easy as running make in the root" — gives me:
cc -Wall -g live_segmenter.c -o live_segmenter -lavformat -lavcodec -lavutil -lbz2 -lm -lz -lfaac -lmp3lame -lx264 -lfaad
Undefined symbols:
"_av_free_packet", referenced from:
_main in ccV7L3nn.o
_main in ccV7L3nn.o
_main in ccV7L3nn.o
_main in ccV7L3nn.o
ld: symbol(s) not found
collect2: ld returned 1 exit status
make: *** [all] Error 1
what am i doing wrong?
@Maitt
Modify as needed..
vlc rtsp://192.168.1.1/file.wmv –sout '#transcode{vcodec=theo,vb=800,scale=1,acodec=vorb,ab=128,channels=2,samplerate=44100}:duplicate{dst=std{access=http,mux=ogg,dst=0.0.0.0:8080}}'
@Ron
Here you go..
http://www.hyperrealm.com/libconfig/
Does anyone know how to compile live_segmenter.c statically, so it can be moved to other computers, without having to install all the libraries?
I have been trying on my Intel Mac 10.5, but can't get it to work.
Anyone who can give some pointers?
Thanks
/Martin
@ more fiya
thx for the response … but the method
Net::SSH.start( 'host', 1234, 'user', 'passwd' ) do |session|dosent work…
error-message
Error running transfer: wrong number of arguments (4 for 3)after some hour of testing i find a method
1. edit the /etc/ssh/ssh_config file
Port2. edit http_streamer.rb
...
require 'net/scp'
require 'net/sftp' #new
...
when 'scp'
if transfer_config.has_key?('password')
Net::SCP.upload!(transfer_config['remote_host'], transfer_config['user_name'], source_file, transfer_config['directory'] + '/' + destination_file, :password => transfer_config['password'])
else
Net::SCP.upload!(transfer_config['remote_host'], transfer_config['user_name'], source_file, transfer_config['directory'] + '/' + destination_file)
end
when 'sftp' #new
Net::SFTP.start(transfer_config['remote_host'], transfer_config['user_name'], :password => transfer_config['password']) do |sftp| #new
sftp.upload!(source_file, transfer_config['directory'] + '/' + destination_file) #new
end
when 's3'
...
3. edit your config.yml file
scp_dev to sftp_dev
or create a new section sftp_dev
sftp_dev:
transfer_type: 'sftp'
remote_host: 'xxx.xxx.xxx.xxx'
user_name: 'user'
password: 'xxxxx!'
directory: '/remotepath'
and it work
@Martin & all
Having the same issue as you – libopennet isn't being found:
$> makegcc -Wall -I/opt/local/include -L/opt/local/lib -g live_segmenter.c -o live_segmenter -lavformat -lavcodec -lavutil -lbz2 -lm -lz -lfaac -lmp3lame -lx264 -lfaad -lconfig
$> ./live_segmenter
dyld: Library not loaded: libopennet.dylib
Referenced from: /opt/local/lib/libconfig.0.2.3.dylib
Reason: image not found
Trace/BPT trap
Would very much appreciate some help, dying to try out this new tech! (Using Snow Leopard, standard install of macports for libraries)
JP,
Leave out -lconfig and compile?
I did, and it works fine without it.
I don't know what it's for.
JP..
try this..
find / -iname "*libopennet*"
respond with output..
Martin,
Thanks for your quick reply – leaving out lconfig allows it to run properly (I'll test it further to see if the outputs are correct, but that'll take a while). Good tip!
I'm also running find to see what libopennet files are floating about – looks like this is it:
/opt/local/lib/libopennet.0.9.9.dylib/opt/local/lib/libopennet.0.dylib
/opt/local/lib/libopennet.a
/opt/local/lib/libopennet.dylib
/opt/local/var/macports/distfiles/libopennet
/opt/local/var/macports/distfiles/libopennet/libopennet-0.9.9.tar.gz
/opt/local/var/macports/receipts/libopennet
/opt/local/var/macports/software/libopennet
/opt/local/var/macports/software/libopennet/0.9.9_0/opt/local/lib/libopennet.0.9.9.dylib
/opt/local/var/macports/software/libopennet/0.9.9_0/opt/local/lib/libopennet.0.dylib
/opt/local/var/macports/software/libopennet/0.9.9_0/opt/local/lib/libopennet.a
/opt/local/var/macports/software/libopennet/0.9.9_0/opt/local/lib/libopennet.dylib
/opt/local/var/macports/sources/rsync.macports.org/release/ports/net/libopennet
Everything seems to be in order too:
ls -l /opt/local/lib/libopennet.dyliblrwxr-xr-x 1 root admin 22 30 Aug 22:09 /opt/local/lib/libopennet.dylib -> libopennet.0.9.9.dylib
Strange behaviour ne?
Cheers again, JP
@JP
Try setting your library path.. then try compiling..
Copy and paste the below..
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/local/lib
@More Fiya,
Could we use VLC and bypass ffmpeg?
I mean, VLC could transcode the RTSP stream, why should we transcode the outut of VLC again with ffmpeg? why not use VLC to transcode the stream to our desired format then pipe to the Segmenter? You know transcode the stream again and again will cost a lot of CPU resource.
hello,
i'm not sure if this question is appropriate here. i got everything works the way they should using the sample clips your links provide. however, when i transcode our clips (they are of .mxf format) i got the following errors;
Output #0, mpegts, to 'clip1.ts':
Stream #0.0: Video: libx264, yuv420p, 320×240 [PAR 1:1 DAR 4:3], q=10-51, 800 kb/s, 90k tbn, 29.97 tbc
Stream #0.1: Audio: aac, 48000 Hz, 4 channels, s16, 64 kb/s
Stream mapping:
Stream #0.0 -> #0.0
Stream #0.1 -> #0.1
Press [q] to stop encoding
[mpeg2video @ 0x1004600]intra matrix invalid, ignoring
Resampling with input channels greater than 2 unsupported.
Can not resample 4 channels @ 48000 Hz to 4 channels @ 48000 Hz
how can i fix this? or, if you know of some forums i can check, please give the pointers (i'm googling too but think you folks may know off hand)
tia,
johnp
found a solution for this, just in case anyone interested. since iphone only takes mp3 or stereo audio, for all clips that carry higher number of audio channels (i.e. 2), we have to downsampling the clips. here is how:
http://muzso.hu/2009/02/25/downsampling-multichannel-audio-5.1-into-stereo-2-channels-with-ffmpeg
cheers
johnp
I'm still looking for pointers on how to compile live_segmenter.c (statically?), so that it can't be moved to other machines or used in an application, without the need for the associated libraries?
Thanks
/Martin
@Martin..
Google the below keywords..
gcc static linking
@freewind..
Take a look on the VLC forums.. I believe there may be development in the works to make VLC more segmentation friendly.. There are users there who also have the same questions you have.. The answers given in response are better than any answer I can give.
… just wondering if you folks have tried with clips larger than 10G (yes, Gigs)? the transcoding part seems ok, but when the segmenter kicks in, it fails with the following error:
Output #0, mpegts, to 'vict2008_800':
Stream #0.0: Video: libx264, yuv420p, 320×240, q=2-31, 90k tbn, 29.97 tbc
Stream #0.1: Audio: aac, 0 channels, s16
[mpegts @ 0xa6ec360]sample rate not set
Could not write mpegts header to first output file
maybe some of you have run into solutions for similar problem, please give some pointers..
great many thanks,
johnp
@john p..
if you try a 1gig file what happens?
just a thought.. maybe try compiling with large file support..
gcc -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64
@More Fiya
i think the threshold is around 1gig… some folks in the other forums suggest transcoding with audio rate at 22.5khz. i tried that and it doesn't really work. i will try your suggestion and see if it works. will keep you folks posted when i found a solution…
cheers,
johnp
Hello, first of all I would like to thank you for your work on this. Unfortunately I too am having an issue with it…
Got compilation sorted out rather easily, although both Debian and Ubuntu both lack some packages to get this to compile. Compiled out of the box on Gentoo, but don't have the V4L hardware in it.
Now, I am using /dev/video0 (BT878) as an input device. To get this to work, I had to add '-f video4linux2 -s 320×240', to the encoding profile I am using, right before the '-i %s' option. Now, I see that ffmpeg starts grabbing frames from the video device, but after 20-40 frames, the output is no longer updated.
I put strace in front of the segmenter_binary, which showed me that the segmenter just hangs after the 4th read. Looks like ffmpeg is writing to a pipe, but the segmenter stops reading from it – although I am not getting a SIGPIPE or anything useful. It just hangs and ffmpeg slowly starts filling memory (I suspect because the segmenter no longer reads from the pipe).
Any ideas what could be wrong, and how I could solve it? Ran my tests on Ubuntu 9.04, with .deb's that I created from the Ubuntu Jaunty source package (needed to have libx264 support). Because this wasn't ffmpeg >= 0.5, I also tried with the source package from Ubuntu Karmic which is 0.5-svn20090706, but I always ran into the same problem.
Right now I am updating the system to the latest Ubuntu testing, in the hopes that it would solve anything, but I don't think it will help. Any suggestions would be very appreciated.
@More Fiya
for this VLC command
vlc rtsp://192.168.1.1/file.wmv –sout '#transcode{vcodec=theo,vb=800,scale=1,acodec=vorb,ab=128,channels=2,samplerate=44100}:duplicate{dst=std{access=http,mux=ogg,dst=0.0.0.0:8080}}',what should i use as a input_location and source_command ?
regards,
Mait
Hi Carson,
I want to congratulate you on a great post.
I have an issue I hope you can help me with.
I have a problem when trying to restream from VLC. I am using the simple restream configuration and the ts and m3u8 files are being created. The problem comes when `i try to view the stream on an iPod Touch. It just sits of 'Loading Movie File' and no video ever plays.
Any ideas on what I am doing wrong?
Thanks for all the great work,
Aidan
According to http://developer.apple.com/iphone/library/documentation/NetworkingInternet/Conceptual/StreamingMediaGuide/FrequentlyAskedQuestions/FrequentlyAskedQuestions.html the output should have a 3 seconds keyframe.
Should this be set in the livesegmenter.c code?
ie.:
output_codec_context->gop_size = (input_stream->time_base)*3;
?
Been playing around with these pieces.
It would be really great if someone who has gotten this to work/understands it better than me, would post a command line call to VLC that does all the right transcoding/capturing/etc. to pull from the built-in iSight on a MacBook Pro and send the result to a udp destination.
TIA.
Hi,
Can anyone compile the segmenter to a windows .exe to make it run on windows 2003 server? Or can you give detailed step-by-step instructions how to do it including what tools to use. I am only a web programmer and don't know how to do C and compile it. Haven't done it yet. Please help!
Thanks,
Robert.
Hi,
first of all, thank you for very nice documentation for the segmenting. But i have one problem getting the http_streamer working.
I'm working on an debian machine and when i try to prcess i get the following error message:
sh: line 3: ftp_dev:: command not found
sh: line 4: scp_dev:: command not found
sh: line 5: copy_dev:: command not found
I can't understand why. Is it an ruby issue ? In my understanding the require settings are looking for all commands used in the script.
Is there anyone who can help me ?
Thanks,
René
It seems like there is a problem with index file creation. Consider this scenario: Say, I have a very long video. I set 10 segments to be in the index file and each segment will be 10 seconds long. And I have a very fast encoder computer, which can encode and copy a segment in 1 second. I start encoding and at the same time, open the stream on my iPhone. After 10 seconds of encoding, the first segments will start to be removed from the index file, because my computer encodes, copies the file and updates index file every second. Say, After 20 seconds, my phone redownloads the index file, and in this scenario, the oldest segment in that file will be 11th one. But it is still playing segment 2 and it has to download segment 3 as the next segment. Instead of segment 3, it downloads segment 11 and I lose the segments between 3 and 11.
So, when you have a fast encoder computer, the computer updates the index file much faster than the phone downloads segments. The updating of the index file (and maybe encoding) should pause when the encoder is too fast. Say, I encoded 50 seconds worth of data in 5 seconds, I should pause for like 45 seconds in order to have the phone catch up with me.
I hope I could make the problem clear. Any ideas about the solution? :)
Hi there folks,
I'm having a few problems compiling the segmenter. I am using Debian Lenny (64bit) and all of the packages are installed – however I am getting the following errors when I try and compile live_segmenter.c:
http://pastebin.com/m3f35ea36
A few friends tell me it's because the versions I am using are either too old or too new. The libraries I am using are straight from apt (stable) (Debian package manager).
Does anyone know of any solutions to this issue?
Regards,
Ashley Taylor
Hey Mait,
how you resolve your first Problem?
I have exact the same but i can not find any help…
gcc -Wall -g live_segmenter.c -o live_segmenter -lavformat -lavcodec -lavutil -lm -lmp3lame -lfaad -lfaac -lbz2 -lpthread -lz -lx264
live_segmenter.c: In function ‘add_output_stream’:
live_segmenter.c:45: warning: implicit declaration of function ‘exit’
live_segmenter.c:45: warning: incompatible implicit declaration of built-in function ‘exit’
live_segmenter.c:58: error: ‘AVCodecContext’ has no member named ‘ticks_per_frame’
live_segmenter.c:61: error: ‘AVCodecContext’ has no member named ‘ticks_per_frame’
live_segmenter.c:71: error: ‘AVCodecContext’ has no member named ‘channel_layout’
live_segmenter.c:71: error: ‘AVCodecContext’ has no member named ‘channel_layout’
live_segmenter.c: In function ‘main’:
live_segmenter.c:124: warning: implicit declaration of function ‘atoi’
live_segmenter.c:130: warning: implicit declaration of function ‘malloc’
live_segmenter.c:130: warning: incompatible implicit declaration of built-in function ‘malloc’
live_segmenter.c:134: warning: incompatible implicit declaration of built-in function ‘exit’
live_segmenter.c:145: warning: incompatible implicit declaration of built-in function ‘exit’
live_segmenter.c:153: warning: incompatible implicit declaration of built-in function ‘exit’
live_segmenter.c:159: warning: incompatible implicit declaration of built-in function ‘exit’
live_segmenter.c:166: warning: incompatible implicit declaration of built-in function ‘exit’
live_segmenter.c:169: warning: implicit declaration of function ‘avformat_alloc_context’
live_segmenter.c:169: warning: initialization makes pointer from integer without a cast
live_segmenter.c:173: warning: incompatible implicit declaration of built-in function ‘exit’
live_segmenter.c:207: warning: incompatible implicit declaration of built-in function ‘exit’
live_segmenter.c:228: warning: incompatible implicit declaration of built-in function ‘exit’
live_segmenter.c:234: warning: incompatible implicit declaration of built-in function ‘exit’
make: *** [all] Fehler 1
@ Kerem: Insert the ffmpeg parameter "-re" that encodes the video at native frame rate
What version of ffmpeg svn (specifically) did segmenter.c compile against? I get errrors with the latest svn and the 0.5 release, and the one that ships with lenny (which I was sure wouldn't work, but tried anyway)
(and I've ensurede my -I and $LD_LIBRARY_PATH point to /usr/local/include and /usr/local/lib (respectively) where ffmpeg installed from source…)
Why does the bandwidth calculation in the encoding profile only take into account the video bandwidth?
For example, in the example profile in this post, you have a video bitrate of 128k and an audio bitrate of 64k, for a total of 192k. But the "bandwidth" setting is 128000.
This is also the case in the example config files.
..just wondering if anyone has seen this symptom: for the same 120 seconds mpeg-ts file, generated by ffmpeg, the number of segments generated by the open source segmenter is 10 whereas the number of segments generated by the apple segmenter is 12. I also tried it on a longer clip, 5880 seconds. 470 segments generated by the open source segmenter vs 588 segments generated by the apple segmenter. The duration of each segment in all cases is set to 10 seconds. Also there is a message, e.g. [NULL@0xfea980]missing picture in access unit, at the end of the open source segmenter command execution. The message is apparently spit out by ffmpeg libavcodec/h264_parser.c. Is this an issue with ffmpeg or an issue with the segmenter? Any pointers would be greatly appreciated. Thanks.
Thanks to the tools, information and work done here I put together a patch to provide HTTP Live Streaming module for VLC. I created this because I was having alot of issues getting ffmpeg to mux the output from VLC correctly.
If there is any interest you can read the instructions and download the patch here:
http://techblog.unwiredappeal.com/2010/03/vlc-http-live-streaming-module-patch.html
Chaps, great work on this. Apple have recently published a spec on recommended live streaming formats :
TN2224
Also the streams generated need to pass the streamvalidator, currently the bitrates of the generated streams are way off.
Has anyone got improved FFMPEG parameters to produce files that pass?
@xargle Thanks for bringing that up. I've been trying to find some time to go back and apply changes that I've been given as well as tweak some stuff in the segmenter. The validation tool will give me extra reason to do so.
Pingback: iPad Streaming Video and More
Great stuff. Thank you so much. I need to use a UDP MpegTS stream as input to the segmenter. Does ffmpeg handle live streams as input. I thought it did. But, I can find no information on how to do this. Do you or any of your readers know about this?
You can use whatever you want to feed the ffmpeg processes. The examples all use cat to do so but you could maybe use something like netcat instead. I haven't tried it myself but I'll stick it on my list to test out.
Very interesting project.
Has anyone decide problem with rollover in write_frame() after 26 hours?