fatal error: runtime: bsdthread_register error

This error is caused when using a Golang binary that has been compiled with an older version of the Golang compiler, you can find out more in the MacOS12BSDThreadRegisterIssue Golang wiki.

You can run the command helper by hand to verify that it is generating the error:

docker-credential-acr-darwin list

If you use the instructions provided by Microsoft to install the ACR Docker Credential Helper binary then you have installed a binary that is compiled with a fairly old version of Golang. The fix for the error is to recompile the ACR Docker Credential Helper using a newer version of Golang.

To recompile using a newer version you will first need to clone the helper github repo:

git clone https://github.com/Azure/acr-docker-credential-helper

Then you will need to edit the Dockerfile so that the Golang version used is a supported version on macOS 12, I found that version 1.15 worked and the helper still compiled:

FROM golang:1.15-alpine
RUN apk update && apk add make bash zip
ADD . /build-root
WORKDIR /build-root
CMD make

After modifying the Dockerfile you can build the binary by running the build.sh script. This script will build the helper for multiple platforms and put the results in the artifacts directory. The artifacts/docker-credential-acr-darwin-amd64.tar.gz file is the one with the new macOS binaries. The two binaries will need to be extracted from that file and put in the /usr/local/bin/ directory.

You can verify that the update has fixed the issue by running the helper by hand again:

docker-credential-acr-darwin list

To get started make sure you have the mail gem installed:

gem install mail

Next there are two main points to remember before diving into the examples:

• The sort order of the mime types is important, they need to come in the order listed in the examples or you will end up with the plain text version of the email on the watch
• You need to include something in the full featured HTML section that can't render on the watch or you will see the full featured HTML on the watch. See the article above for some pointers but generally the watch isn't going to fetch an image from the web so that should do it and is what I have in the following images.

First a simple example that will show plain text for mail clients that don’t support HTML at all, normal HTML for full featured clients and a subset of HTML for the Apple watch.

require 'mail'
mail = Mail.new do
  to      'user@something.com'
  from    'person@company.com'
  subject 'Watch mail example'
end
#
# The order supplied here matters
#
mail.body.set_sort_order [ "text/plain", "text/watch-html", "text/html" ]
#
# The order here doesn't matter
#
text_part = Mail::Part.new do
  body 'This is plain text'
end
mail.text_part = text_part
watch_part = Mail::Part.new do
  content_type 'text/watch-html; charset=UTF-8'
  body '<b>This is HTML for the Apple watch</b>'
end
mail.add_part watch_part
#
# If this part has something in it that can't display on the watch then
# the watch part will display. Keep that in mind if you want to force the
# watch part to display. Here the link out to an image will force the
# fallback to happen.
#
html_part = Mail::Part.new do
  content_type 'text/html; charset=UTF-8'
  body '<h1>This is HTML</h1><img src="http://images.company.com/someimage.jpg"/>'
end
mail.html_part = html_part
mail.deliver

Here is an example that includes an image that will display on the watch. It is important that in this case the image comes first in the sort order.

require 'mail'
mail = Mail.new do
  to      'user@something.com'
  from    'person@company.com'
  subject 'Watch mail example with image'
end
#
# The order supplied here matters
#
mail.body.set_sort_order [ "image/png", "text/plain", "text/watch-html", "text/html" ]
#
# The order here doesn't matter but you will need to
# reference the image later.
#
mail.attachments['test.png'] = File.read('/tmp/test.png')
image_cid = mail.parts.first.url
  text_part = Mail::Part.new do
  body 'This is plain text'
end
mail.text_part = text_part
  watch_part = Mail::Part.new do
  content_type 'text/watch-html; charset=UTF-8'
  body '<b>This is HTML for the watch</b> <br/> <img src="' + image_cid + '"/>'
end
mail.add_part watch_part
  html_part = Mail::Part.new do
  content_type 'text/html; charset=UTF-8'
  body '<h1>This is HTML</h1><img src="http://images.company.com/someimage.jpg"/>'
end
mail.html_part = html_part
mail.deliver

Before diving into the technical details I think it is worth noting a few things. There are a few hardware based iBeacons already available in various “beta” states, to name a few: KST’s iBeacon, coin for arduino and ReadBearLab’s iBeacon. These dedicated devices are going to have a power and most likely a price advantage over the Raspberry Pi so that is something to keep in mind. The setup I used cost around $70 for example and that is around the cost of the KST device but more than the others and the more that are made the more the price will probably decrease. The main advantage of the Raspberry Pi is flexibility and included in that $70 is a wifi dongle that the other devices don’t have.

My configuration for this post:

• Raspberry Pi Model B. The Model A would work as well but currently you wouldn’t be able to run both wifi and bluetooth at the same time. You also need power, storage and maybe a case.
• I’ve used both the IOGEAR Bluetooth 4.0 USB Micro Adapter (GBU521) and the Cirago Bluetooth 4.0 USB Mini Adapter (BTA8000) adapters successfully.
• For wifi I’ve been using the Edimax device.

Assuming you have your Pi hardware ready the first step is to install the Raspbian distro. I tested on the 2013-09-25-wheezy-raspbian version. Make sure it boots and then run the following commands as root to get the dependancies ready:

apt-get update
apt-get install libglib2.0-dev libdbus-1-dev libudev-dev libical-dev libreadline6-dev

Next you will need to download and compile a more recent version of Bluez than what is available for the Raspbain distro. I’ve been able to use a number of versions in the Bluez 5.X family but for this I’ll assume Bluez 5.9. Use the following to get it installed and compiled (make sure to do the install part here as root):

wget https://www.kernel.org/pub/linux/bluetooth/bluez-5.9.tar.xz
tar xvJf bluez-5.9.tar.xz
cd bluez-5.9
./configure --disable-systemd --enable-library
make
make install

Now you have Bluez installed with bluetooth library support. There are also a number of tools available at this point. The first one you want to run is hciconfig to configure your bluetooth device. It works a lot like ifconfig if you are familiar with setting up network interface. If you run it without any command line arguments you will get a list of bluetooth devices:

hciconfig
# hci0:   Type: BR/EDR  Bus: USB
#         BD Address: 00:02:72:32:CA:23  ACL MTU: 1021:8  SCO MTU: 64:1
#         DOWN
#         RX bytes:340 acl:0 sco:0 events:7 errors:0
#         TX bytes:54 acl:0 sco:0 commands:12 errors:0

You want to bring the bluetooth device up so it is available:

hciconfig hci0 up
hciconfig
# hci0:   Type: BR/EDR  Bus: USB
#         BD Address: 00:02:72:32:CA:23  ACL MTU: 1021:8  SCO MTU: 64:1
#         UP RUNNING
#         RX bytes:813 acl:0 sco:0 events:26 errors:0
#         TX bytes:374 acl:0 sco:0 commands:31 errors:0

Make sure you see “UP RUNNING” before proceeding. You will probably want to add the command to bring the bluetooth device up to the startup script. Next you will want to grab my bluez-ibeacon repo from github and build it:

git clone https://github.com/carsonmcdonald/bluez-ibeacon.git
bluez-ibeacon/bluez-beacon/
make

Now you have a binary named ibeacon that you can run and that will turn the Pi into an iBeacon:

./ibeacon 200 e2c56db5dffb48d2b060d0f5a71096e0 1 1 -29

You can read more about what the above means in the README for the bluez-ibeacon project.

There is a demo iOS app in the same bluez-ibeacon project that you can use to then detect the beacon now that it is running.

To start with I picked the IOGEAR Bluetooth 4.0 USB Micro Adapter (GBU521) that can be found on Amazon for just $13 since it looked like the chip it uses is decently supported with recent Linux kernels. The only issue I had is the size itself, if it didn’t have a little nub on the end it would be too small to pull back out of the USB plug.

The GBU521 seems to be recognized by older kernels but to get LE support go with the latest kernel you can find. For the RPi it is easy enough to get a very recently kernel using rpi-update. Currently it is 3.6.11 and that worked well for me. I also used a 3.5 kernel on a laptop that worked fine but you can’t go any older than that.

Once you have the correct kernel in place you need to grab the latest version of Bluez. Bluez is the official Bluetooth stack for Linux and the 5.x series has introduced Bluetooth LE support. It is still a work in progress but it does work. Grab the 5.1 version or later and uncompress/untar it somewhere on your RPi. Please note that Bluez 5.x requires a 3.5+ kernel to work correctly, this seems to be a sticking point that a lot of people hit.

I am using the Raspbian distro found on Raspberry Pi downloads. Outside of the normal install needed to compile I had to install the following packages:

apt-get install libusb-dev libdbus-1-dev libglib2.0-dev automake libudev-dev libical-dev libreadline-dev

With those packages installed configure and make (note that I’ve disabled systemd support, it didn’t seem to work for me and I didn’t need it myself):

./configure --disable-systemd
make
make install

Now you will have a number of binaries installed that will get you rolling. I’m not going to go into a lot of detail here about what all of the following commands do but I want to touch on each of them briefly so you know they are there. Assuming you have the Bluetooth adapter plugged in you should be able to run the following command and get details about it:

hciconfig
hci0:	Type: BR/EDR  Bus: USB
BD Address: 00:00:12:34:56:78  ACL MTU: 1021:8  SCO MTU: 64:1
DOWN
RX bytes:467 acl:0 sco:0 events:18 errors:0
TX bytes:317 acl:0 sco:0 commands:18 errors:0

This shows that the device is in a down state. To bring it up you can issue the following command:

hciconfig hci0 up
hciconfig
hci0:	Type: BR/EDR  Bus: USB
BD Address: 00:00:12:34:56:78  ACL MTU: 1021:8  SCO MTU: 64:1
UP RUNNING
RX bytes:467 acl:0 sco:0 events:18 errors:0
TX bytes:317 acl:0 sco:0 commands:18 errors:0

Now if you know you have a Bluetooth LE peripheral sitting around advertising you can run the following command and you should see an address for that device show up:

hcitool lescan
LE Scan ...
12:88:FF:FF:11:99 touch
12:88:FF:FF:11:99 (unknown)

If you get to the point of wanting to let the RPi act as a peripheral you will need to know about the following command that will enable advertising on the adapter:

hciconfig hci0 leadv 0

Bluez also provides a deamon named bluetoothd (installed in /usr/local/libexc/bluetooth) that runs to take care of communicating with devices for you. It uses DBus to communicate with applications but at this point LE support hasn’t been added. The best way to integrate with it if you want to write your own code is to use a plugin. I won’t go into that here but you can see an example in the source (I also plan on releasing an example of my own that will work with an iOS app).

Lastly remember you don’t need to start off on the Raspberry Pi. Most of the above will work on a normal Linux system and it may be easier to start there to get it working and then move to the RPi once you are familiar with the setup.

To protect all your electronic devices from short circuits, blackouts, etc. You must be sure that your house has all the electrical connections in good condition.

For this you must hire the best company in the market: BSK Data & Electrical. Visit them at https://www.bskelectrical.com.au, we are sure that they will not fail you, as they have the most trained professionals in electrical issues guaranteeing the electrical safety of your home.

Don’t be afraid to ask the company to send you a request, there are many solutions which is how people like you prefer to live, especially in this day and age. Our website for all types of electrical problems is the solution for many different problems, so please make sure to take advantage of it.

There is all sorts of applications including all the devices at this site, you have very much to rely on this site for the complete solutions.

The first thing to understand is that while the upload happens directly to S3 there still needs to be some server side code that signs the URL used by the PUT call. That bit of code is really simple and I’m including an example at the end for both PHP and Ruby. If you want to skip to the fun part you can check out the PHP and Ruby example code on github (instructions there on deploying to Heroku as well).

Second there are a good number of technologies involved here so I’ve compiled a list of helpful links in case you aren’t already familiar with them and/or want a reference:

• FileAPI
• XHR2
• CORS
• S3 CORS support docs
• S3 REST Authentication
• S3 PUT requests

Setting up CORS support for an S3 bucket can be done using the console, see the S3 CORS support docs above for details. For everything in this demo I used the following CORS configuration:

<CORSConfiguration>
    <CORSRule>
        <AllowedOrigin>*</AllowedOrigin>
        <AllowedMethod>PUT</AllowedMethod>
        <MaxAgeSeconds>3000</MaxAgeSeconds>
        <AllowedHeader>Content-Type</AllowedHeader>
        <AllowedHeader>x-amz-acl</AllowedHeader>
        <AllowedHeader>origin</AllowedHeader>
    </CORSRule>
</CORSConfiguration>

That configuration allows any origin to issue PUTs and include the headers Content-Type, x-amz-acl and origin. You would probably want to restrict the origin more but for this demo I want to make sure it works for people who just cut and paste the above.

The following HTML sets up the file input tag and a progress bar to track the upload (index.html):

<html>
<head>
<link rel="stylesheet" type="text/css" href="styles.css" />
 <script type="text/javascript" src="app.js"></script>
</head>
<body>
<table>
<tr>
<td>File selection:</td>
<td><input type="file" id="files" name="files[]" multiple /></td>
</tr>
<tr>
<td>Progress:</td>
<td>
<div id="progress_bar">
<div class="percent">0%</div>
</div>
</td>
</tr>
<tr>
<td>Status:</td>
<td><span id="status"></span></td>
</tr>
</table>
  <script type="text/javascript">
    document.getElementById('files').addEventListener('change', handleFileSelect, false);
    setProgress(0, 'Waiting for upload.');
  </script>
</body>
</html>

CSS for the progress bar (styles.css):

#progress_bar {
  width: 200px;
  margin: 10px 0;
  padding: 3px;
  border: 1px solid #000;
  font-size: 14px;
  clear: both;
  opacity: 0;
  -moz-transition: opacity 1s linear;
  -o-transition: opacity 1s linear;
  -webkit-transition: opacity 1s linear;
}
#progress_bar.loading {
  opacity: 1.0;
}
#progress_bar .percent {
  background-color: #99ccff;
  height: auto;
  width: 0;
}

The JavaScript that follows has a couple different parts.

• handleFileSelect - This is where things get started when a file is selected for upload. It kicks off the upload process with each file that was selected.
• uploadFile - Called for each file in handleFileSelect and ties the signing process to the S3 PUT process.
• executeOnSignedUrl - Calls the server side signing process with the a filename and mime type. The server side signed URL is then passed on to a callback.
• uploadToS3 - Uses a signed PUT URL to upload the given file to S3 using CORS enabled XHR2.
• createCORSRequest - Creates a CORS XHR2 request.
• setProgress - Sets the current progress of the upload.

function createCORSRequest(method, url)
{
  var xhr = new XMLHttpRequest();
  if ("withCredentials" in xhr)
  {
    xhr.open(method, url, true);
  }
  else if (typeof XDomainRequest != "undefined")
  {
    xhr = new XDomainRequest();
    xhr.open(method, url);
  }
  else
  {
    xhr = null;
  }
  return xhr;
}
function handleFileSelect(evt)
{
  setProgress(0, 'Upload started.');
  var files = evt.target.files;
  var output = [];
  for (var i = 0, f; f = files[i]; i++)
  {
    uploadFile(f);
  }
}
/**
 * Execute the given callback with the signed response.
 */
function executeOnSignedUrl(file, callback)
{
  var xhr = new XMLHttpRequest();
  xhr.open('GET', 'signput.php?name=' + file.name + '&type=' + file.type, true);
  // Hack to pass bytes through unprocessed.
  xhr.overrideMimeType('text/plain; charset=x-user-defined');
  xhr.onreadystatechange = function(e)
  {
    if (this.readyState == 4 && this.status == 200)
    {
      callback(decodeURIComponent(this.responseText));
    }
    else if(this.readyState == 4 && this.status != 200)
    {
      setProgress(0, 'Could not contact signing script. Status = ' + this.status);
    }
  };
  xhr.send();
}
function uploadFile(file)
{
  executeOnSignedUrl(file, function(signedURL)
  {
    uploadToS3(file, signedURL);
  });
}
/**
 * Use a CORS call to upload the given file to S3. Assumes the url
 * parameter has been signed and is accessable for upload.
 */
function uploadToS3(file, url)
{
  var xhr = createCORSRequest('PUT', url);
  if (!xhr)
  {
    setProgress(0, 'CORS not supported');
  }
  else
  {
    xhr.onload = function()
    {
      if(xhr.status == 200)
      {
        setProgress(100, 'Upload completed.');
      }
      else
      {
        setProgress(0, 'Upload error: ' + xhr.status);
      }
    };
    xhr.onerror = function()
    {
      setProgress(0, 'XHR error.');
    };
    xhr.upload.onprogress = function(e)
    {
      if (e.lengthComputable)
      {
        var percentLoaded = Math.round((e.loaded / e.total) * 100);
        setProgress(percentLoaded, percentLoaded == 100 ? 'Finalizing.' : 'Uploading.');
      }
    };
    xhr.setRequestHeader('Content-Type', file.type);
    xhr.setRequestHeader('x-amz-acl', 'public-read');
    xhr.send(file);
  }
}
function setProgress(percent, statusLabel)
{
  var progress = document.querySelector('.percent');
  progress.style.width = percent + '%';
  progress.textContent = percent + '%';
  document.getElementById('progress_bar').className = 'loading';
  document.getElementById('status').innerText = statusLabel;
}

The above example calls the PHP version of the server side signing code. It can easily be changed to anything that can sign a request in the same way.

I have an old way of creating signed URLs using PHP that hasn’t been updated in forever. With the more recent versions of PHP there is built in support for the hash-hmac function and a base64 encode. Here is the updated PHP script you need on the server side, to get it to work you would need to replace the S3_KEY, S3_SECRET and S3_BUCKET values with your own:

//
// Change the following settings
//
$S3_KEY='S3 Key Here';
$S3_SECRET='S3 Secret Here';
$S3_BUCKET='/uploadtestbucket';
$EXPIRE_TIME=(60 * 5); // 5 minutes
$S3_URL='http://s3.amazonaws.com';
$objectName='/' . $_GET['name'];
$mimeType=$_GET['type'];
$expires = time() + $EXPIRE_TIME;
$amzHeaders= "x-amz-acl:public-read";
$stringToSign = "PUT\n\n$mimeType\n$expires\n$amzHeaders\n$S3_BUCKET$objectName";
$sig = urlencode(base64_encode(hash_hmac('sha1', $stringToSign, $S3_SECRET, true)));
$url = urlencode("$S3_URL$S3_BUCKET$objectName?AWSAccessKeyId=$S3_KEY&Expires=$expires&Signature=$sig");
echo $url;

With all of that in place you should now be able to upload directly to S3 using a browser that supports CORS, XHR2 and the FileAPI (pretty much everything but IE currently).

There are a few things to take into account before diving into this. The first is that the mruby project is very new and there are a number of gaps in the language support right now but the goal is to support the ISO definition of Ruby at some point. The second thing to know is that I’m talking about embedding Ruby here and not writing iOS apps using Ruby. I’m more interested in the potential of Ruby as a scripting language for something like a game. If you want to look into writing iOS apps using Ruby check out RubyMotion or the MobiRuby project (MobiRuby is based on mruby).

Step one on the path to embedding mruby in an iOS app is getting the library compiled as a framework for XCode. Although the earliest version of mruby had support for building an iOS version that support was removed and there seems to be no current push to add it back. Because of that I created a simple iOS mruby build setup on github. The instructions for using it are included with the project. Once the framework has been created it is easy enough to include in your XCode project just like any other external framework using the “Add Other” option in the “Build Phases” -> “Link Binary With Libraries” section.

There are two ways of using mruby, you can run a script as source or you can pre-compile the source into bytecode first. I’m going to focus on the pre-compiled way because I believe it is the less likely of the two to get an app rejected. This requires that you have the mruby tools available. If you use the build setup I created you can find the tools in the bin directory.

And finally I have created a simple mruby iOS example XCode project that you can use to get started. It includes a pre-compiled version of the framework created with the build setup mentioned before and examples of calling objective-c from Ruby and Ruby from objective-c. The following is a quick overview of the example:

• FooUtil.h/m - This is where all the interaction with mruby happens. It coordinates different parts of the examples as well.
• example.rb - This is the Ruby code for the example and has to be compiled into example.mrb before changes will take place.
• example.mrb - This is the compiled version of example.rb.
• FooData.h/m - This is an example data class that is wrapped using a Ruby class named FooData.

Everything is driven from the FooUtil singleton to make it easier to find each example. There are five methods to understand in FooUtil:

• setDebugBlock - This static method takes a block that will be passed a debug string. This gives an easy way of changing the output of the example.
• loadFromBundle - This is an example of loading a pre-compiled script. It assumes it can find the given filename in the app bundle.
• execute - This is an example of executing a loaded script. It assumes loadFromBundle has been called. Calling this method will cause the Ruby to objective-c examples to execute.
• updateBarLocation - This is an example of executing Ruby code from objective-c.
• cleanup - This method will force a garbage collection. It was added to demonstrate the release of the wrapped FooData class.

The example code doesn’t as much error checking as you should have in a real app but it hopefully is enough to get you started. I plan on adding more examples to the github project as features get added to mruby and as I have time. As a reminder, you have to recompile the Ruby code if you make changes or they won’t be seen.

The MediaSource API has experimental support in Chrome and can be enabled by using the chrome://flags option. To see it in action you can go to the MediaSource demo page. You can also read a litte more about it here although the spec linked to above is probably a better place to learn about it.

A while ago I created a tool for segmenting H264 video in a Pantos compliant way. When I saw the MediaSource API I wondered how the same type of tool might fit in. The first thing to note is that the Pantos draft describes a complete technique for video streaming while the MediaSource API gives you the tools to stream video and leaves the technique. What follows is a simple technique for segmenting a WebM video in a way that allows standard streaming with the MediaSource API in the same fasion as the Pantos draft technique. While this example will not support variable rate streams it could be expanded to do so and would be the next logical step.

The basics of how the Pantos technique works are simple. A video is broken down into chunks and each of those chunks is downloaded, buffered and played. In older drafts this ment splitting a video file into multiple files but in later drafts support was added for range requests allowing the large file to remain intact. With that understanding there are two major parts that need to be addressed:

• Segmenting a WebM video - This will require some understanding of the container format for WebM and the result will be something like a playlist for the video.
• Javascript to play the video - This will use the playlist created by segmenting the WebM video to drive the MediaSource API.

Segmenting a WebM Video

Before reading on take a second to read the about WebM page. It describes that WebM specifies a video/audio codec and a container format. We are only interested in the format of the container. If you want all the details you can read the WebM container format specs. It is enough to know that the WebM container is a modified version of the Matroska Container format and both containers use EBML to specify their structure.

EBML is basically a binary XML format. There are a number of tools available for readin EBML and for my perposes I found the Python python-ebml implementation to be the easiest to get working.

The MediaSource API wants to be fed chunks of video data in a specific way. First it wants header information and then it wants what are called clusters from the WebM container. Each cluster contains various information but the most likely reason for wanting a cluster as input is that each cluster starts with a keyframe. A keyframe is an important reference point in the video stream and one that gets built on until the next keyframe.

The following code will take as input a WebM file and output a playlist for that file in JSON format. The playlist will contain the offsets of each cluster in the given WebM video and we will use it later to fetch the header and each cluster as chunks and feed those to a video element:

from ebml.schema import EBMLDocument, UnknownElement, CONTAINER, BINARY
def fill_video_info(element, offset, video_info):
  if element.name == 'Duration':
    video_info['duration'] = element.value
  if element.name == 'DisplayWidth':
    video_info['width'] = element.value
  if element.name == 'DisplayHeight':
    video_info['height'] = element.value
  if element.name == 'Cluster':
    video_info['clusters'].append({'offset': offset})
  if element.name == 'Timecode':
    video_info['clusters'][-1]['timecode'] = element.value
  if element.type == CONTAINER:
    for sub_el in element.value:
      fill_video_info(sub_el, offset + element.head_size, video_info)
      offset += sub_el.size
if __name__ == '__main__':
  import sys
  import json
  import os
  mod_name, _, cls_name = 'ebml.schema.matroska.MatroskaDocument'.rpartition('.')
  try:
    doc_mod = __import__(mod_name, fromlist=[cls_name])
    doc_cls = getattr(doc_mod, cls_name)
  except ImportError:
    parser.error('unable to import module %s' % mod_name)
  except AttributeError:
    parser.error('unable to import class %s from %s' % (cls_name, mod_name))
  video_info = {}
  video_info['filename'] = sys.argv[1]
  video_info['total_size'] = os.stat(sys.argv[1]).st_size
  video_info['clusters'] = []
  with open(sys.argv[1], 'rb') as stream:
    doc = doc_cls(stream)
    offset = 0
    for el in doc.roots:
      fill_video_info(el, offset, video_info)
      offset += el.size
  print json.dumps(video_info)

To run the script you will want to save the above script and install the module then do something like this:

git clone https://github.com/jspiros/python-ebml.git
PYTHONPATH=python-ebml python create_playlist.py test.webm

Running the script will result in JSON output that can be piped to a file and used as a playlist.

Javascript to Play the WebM Segmented Playlist

Feeding data to the MediaSource API is done with Javascript. The first thing to do is read in the JSON playlist file created by the Python script when the page is finished loading. To do that I’m using the Snack JS framework, it is micro-framework that has some of the features of JQuery. After reading in the playlist chunks of the WebM file need to be read in with range requests.

A while ago I made created a post about range requests with ajax and that comes in handy here. The main difference between that post and what we need here is that the MediaSource API expects a typed array as input so we need to use XHR2.

The following Javascript code is bare bones. One of the most obvious limitations is that it doesn’t have the ability to seek to a portion of the stream, it will only play the stream start to finish or if the player has the content buffered it should allow for seeking in the buffered portion. It demonstrates that the API works and should be a good starting point for experimenting more with multi-rate streaming which is my short term goal.

  /**
   * This fetches a chunk of video using a range request.
   *   video_name - The relative filename of the video to fetch
   *   start_bytes - The start byte of the chunk to fetch
   *   end_bytes - The last byte of the chunk to fetch
   *   is_last - True if this is the last chunk to fetch
   */
  function fetch_chunk(video, video_name, start_bytes, end_bytes, is_last)
  {
    var range_req = 'bytes=' + start_bytes + '-' + end_bytes;
    var xhr = new XMLHttpRequest();
    xhr.open('GET', video_name, true);
    xhr.setRequestHeader("Range", range_req);
    xhr.responseType = 'arraybuffer';
    xhr.onload = function(e)
    {
      video.webkitSourceAppend(new Uint8Array(this.response));
      if(is_last)
      {
        video.webkitSourceEndOfStream(HTMLMediaElement.EOS_NO_ERROR);
      }
    };
    xhr.send();
  }
  /**
   * Fetch the header portion of the video file.
   */
  function fetch_header(video, video_info)
  {
    fetch_chunk(video, video_info.filename, 0, video_info.clusters[0].offset-1, video_info.clusters.length == 1);
  }
  var current_cluster = 0;
  /**
   * Fetch the next cluster of the video file.
   */
  function fetch_next_cluster(video, video_info)
  {
    if(video_info.clusters.length == current_cluster+1)
    {
      fetch_chunk(video, video_info.filename, video_info.clusters[current_cluster].offset, video_info.total_size, true);
    }
    else
    {
      fetch_chunk(video, video_info.filename, video_info.clusters[current_cluster].offset, video_info.clusters[current_cluster+1].offset-1, false);
    }
    current_cluster++;
  }
  /**
   * Enable the video for MediaSource and attach event listeners to drive the fetching of video chunks.
   */
  function video_setup(video_info)
  {
    current_cluster = 0;
    video.src = video.webkitMediaSourceURL;
    video.addEventListener('webkitsourceopen', function(e)
    {
      var video = this;
      fetch_header(video, video_info);
      fetch_next_cluster(video, video_info);
    }, false);
    video.addEventListener('progress', function(e)
    {
      var video = this;
      if( video.webkitSourceState != HTMLMediaElement.SOURCE_ENDED )
      {
        fetch_next_cluster(video, video_info);
      }
    });
  }
  /**
   * When the page is done loading read in the JSON playlist and get the video set up.
   */
  snack.ready(function()
  {
    if (!video.webkitMediaSourceURL)
    {
      alert('webkitMediaSourceURL is not available');
      return;
    }
    snack.request({ method: 'get', url: 'test.json', }, function (err, res)
    {
      if (err)
      {
        alert(err);
        return;
      }
      video_setup(snack.parseJSON(res));
    });
  });

Assuming you have enabled support in Chrome (currently the only browser to support the API) you can see everything in action by checking out my MediaSource API demo page.

I have glossed over a lot of details but the main take aways are that the MediaSource API gives you the interface to build a Pantos streaming system, segment WebM on clusters to feed to the MediaSource API and use XHR2 to fetch the WebM data with range requests. As one last point a lot of the best practices for encoding video for Pantos streaming will work here as well.

If you aren’t familiar with range requests head over to the HTTP RFC and check out the range header. Your web server needs to support range requests for this to be useful but most do so that shouldn’t be a huge issue. As a bonus you will find that some CDNs support range request as well (Amazon S3 for example).

To show how this works I’ll start off by getting the offsets of a text file, then use curl to show the actual request and then put it into an ajax request. Even though I’m using a text file the same thing could be done with a binary file as well. Here are the contents of the file I’ve called range-test.txt:

First part of the file.
More text here.
Lorem Ipsum is simply dummy text of the printing and typesetting industry. Lorem Ipsum has been the industry's standard dummy text ever since the 1500s, when an unknown printer took a galley of type and scrambled it to make a type specimen book. It has survived not only five centuries, but also the leap into electronic typesetting, remaining essentially unchanged. It was popularised in the 1960s with the release of Letraset sheets containing Lorem Ipsum passages, and more recently with desktop publishing software like Aldus PageMaker including versions of Lorem Ipsum.
Some text after Lorem Ipsum.
End of the file.

Using egrep we can get the offset of each line pretty easy like so:

> egrep -b "\r|\n" range-test.txt
0:First part of the file.
25:More text here.
42:Lorem Ipsum is simply dummy text of the printing and typesetting industry. Lorem Ipsum has been the industry's standard dummy text ever since the 1500s, when an unknown printer took a galley of type and scrambled it to make a type specimen book. It has survived not only five centuries, but also the leap into electronic typesetting, remaining essentially unchanged. It was popularised in the 1960s with the release of Letraset sheets containing Lorem Ipsum passages, and more recently with desktop publishing software like Aldus PageMaker including versions of Lorem Ipsum.
618:Some text after Lorem Ipsum.
660:End of the file.

Quickly verify that the range looks right in isolation using hexdump:

> hexdump -c -s 618 -n 30 range-test.txt
000026a   S   o   m   e       t   e   x   t       a   f   t   e   r
000027a   L   o   r   e   m       I   p   s   u   m   .  \n  \n
0000288

Now we have a sample range so put the text file on your web server and use curl to send the range request headers (note that the hexdump takes an offset and count of bytes while the range header takes an start and end offset):

> curl -H 'Range:bytes=618-647' http://localhost/range-test.txt
Some text after Lorem Ipsum.

Now that it makes sense how this range is used to make a request for part of a file we can move on to doing it with an ajax request. I’m going to use jQuery’s ajax function to make this really simple. It has a feature that lets you set headers really easily. The following is all you need:

<html>
  <head>
    <script type="text/javascript" src="jquery-1.7.min.js"></script>
    <script type="text/javascript">
    $(function() {
      $.ajax({
        url: 'range-test.txt',
        headers: {Range: "bytes=618-647"},
        success: function( data ) { $('#results').html( data ); }
      });
    });
    </script>
  </head>
  <body>
<div id="results"></div>
  </body>
</html>

And that is all there is to it. This could be useful for all kinds of things like displaying just a section of a large log file at a time. This might also be useful in a situation where you want to stream binary data back to the browser a chunk at a time.

To follow along you will need to grab the source for WebP and compile it. The easiest way to do that is to follow the official compiling WebP instructions. After compiling the source you will be able to encode WebP images, later I describe how to build the same source for use with iOS.

Before trying to build a web page I did some testing to see just how easy is to handle wordpress hosting to add pictures and videos to blog posts. I used ImageMagick to convert the source images into JPG and pushed the quality of the JPG down as far as seemed reasonable to reduce the resulting size. I tried a number of combinations but ended up using the following command to convert the JPGs:

convert -quality 75 -resize 1024x680 inputfile.png outputfile.jpg

Here are a couple samples produced using that command:

For WebP I made the quality 90 and encoded the same images with the following command:

cwebp -q 90 -resize 1024 680 inputfile.png -o outputfile.webp

Here are a couple example images produced using that command (at the time of this post these will only be visible using Chrome):

Even with the advantage I was giving the JPG images by compressing them with a lower quality I was able to get a 25% size reduction in the WebP version. Generally the JPG images would have been usable but some ended up with artifacts that would have required them to be re-encoded at a higher quality. The WebP versions ended up looking better in every case.

This was just my quick attempt to verify that it would be worth the effort of using WebP. A more in depth overview of JPG vs WebP can be found in the example gallery.

With that simple test out of the way I created a script that would compile the libwebp source into a framework compatible with the iOS simulators, iPad and iPhone:

SDK=4.3
PLATFORMS="iPhoneSimulator iPhoneOS-V6 iPhoneOS-V7"
DEVELOPER=/Developer
TOPDIR=`pwd`
BUILDDIR="$TOPDIR/tmp"
FINALDIR="$TOPDIR/WebP.framework"
LIBLIST=''
mkdir -p $BUILDDIR
mkdir -p $FINALDIR
mkdir $FINALDIR/Headers/
for PLATFORM in ${PLATFORMS}
do
if [ "${PLATFORM}" == "iPhoneOS-V7" ]
then
PLATFORM="iPhoneOS"
ARCH="armv7"
elif [ "${PLATFORM}" == "iPhoneOS-V6" ]
then
PLATFORM="iPhoneOS"
ARCH="armv6"
else
ARCH="i386"
fi
ROOTDIR="${BUILDDIR}/${PLATFORM}-${SDK}-${ARCH}"
rm -rf "${ROOTDIR}"
mkdir -p "${ROOTDIR}"
export DEVROOT="${DEVELOPER}/Platforms/${PLATFORM}.platform/Developer"
export SDKROOT="${DEVROOT}/SDKs/${PLATFORM}${SDK}.sdk"
export CC=${DEVROOT}/usr/bin/gcc
export LD=${DEVROOT}/usr/bin/ld
export CPP=${DEVROOT}/usr/bin/cpp
export CXX=${DEVROOT}/usr/bin/g++
export AR=${DEVROOT}/usr/bin/ar
export AS=${DEVROOT}/usr/bin/as
export NM=${DEVROOT}/usr/bin/nm
export CXXCPP=$DEVROOT/usr/bin/cpp
export RANLIB=$DEVROOT/usr/bin/ranlib
export LDFLAGS="-arch ${ARCH} -pipe -no-cpp-precomp -isysroot ${SDKROOT} -L${ROOTDIR}/lib"
export CFLAGS="-arch ${ARCH} -pipe -no-cpp-precomp -isysroot ${SDKROOT} -I${ROOTDIR}/include"
export CXXFLAGS="-arch ${ARCH} -pipe -no-cpp-precomp -isysroot ${SDKROOT} -I${ROOTDIR}/include"
rm -rf libwebp-0.1.2
tar xzf libwebp-0.1.2.tar.gz
cd libwebp-0.1.2
sh autogen.sh
./configure --host=${ARCH}-apple-darwin --prefix=${ROOTDIR} --disable-shared --enable-static
make
make install
LIBLIST="${LIBLIST} ${ROOTDIR}/lib/libwebp.a"
cp -Rp ${ROOTDIR}/include/webp/* $FINALDIR/Headers/
cd ..
done
${DEVROOT}/usr/bin/lipo -create $LIBLIST -output $FINALDIR/WebP
rm -rf libwebp-0.1.2
rm -rf ${BUILDDIR}

The script assumes that the source for libwebp is in the same directory as the script is executed from. If you skipped the first part of the post or want to make sure you are using the same version I used then use the following curl command:

curl http://webp.googlecode.com/files/libwebp-0.1.2.tar.gz > libwebp-0.1.2.tar.gz

After you run the script you will have a WebP.framework directory that represents the library. You can use this as a framework in XCode.

Using the library is easy. The following snipit shows how to get the current WebP library version, get the width and height of an image and then read the image into a buffer:

// Get the current version of the WebP decoder
int rc = WebPGetDecoderVersion();
NSLog(@"Version: %d", rc);
// Get the width and height of the selected WebP image
int width = 0;
int height = 0;
WebPGetInfo([myData bytes], [myData length], &width, &height);
NSLog(@"Image Width: %d Image Height: %d", width, height);
// Decode the WebP image data into a RGBA value array
uint8_t *data = WebPDecodeRGBA([myData bytes], [myData length], &width, &height);

There is more documentation on the WebP API and I’ve put together a complete example on Github. If you grab the WebP iOS example from Github you will find the build script as well as the full example. First build the library from a command line then open the example project in XCode. If you don’t first build the library before opening the project you will see that the WebP.framework is missing.

At this point you are on your way to having smaller iOS applications. There is at least one drawback at this point and that is speed. Decoding is a little slow and I think this is because the library isn’t constructed to take advantage of any hardware acceleration. The next step is to see if it can be sped up by using the accelerate framework.

What I needed was a way to use the closure compiler within an Ant task. The Ant task that comes with the library was decent, but it didn’t provide the flexibility I needed to integrate it into an existing system without overhauling the setup. While digging into this, I came across a forum where developers were discussing unconventional solutions, including some that utilized slots not on gamstop for testing payment systems outside standard restrictions. The conversation inspired me to think creatively about my own project, so I dived into the library’s Ant task, dissected its structure, and eventually figured out how to wire it all up in a way that worked perfectly for my needs.

It isn’t that hard to use the compiler library in your own Java code but without a simple example it takes some work to figure out what is needed and what isn’t. The following code is just about as bare bones as you can get. It takes a number of JavaScript files and compile them using the medium setting, more about the choice of settings after the code:

import com.google.javascript.jscomp.*;
import java.io.FileWriter;
import java.util.List;
import java.util.ArrayList;
import java.util.logging.Level;
public class Test
{
  public static void main(String[] args) throws Exception
  {
    // These are external JavaScript files you reference but don't want changed
    String externalJavascriptResources[] = {
        "jquery.js",
        "jqueryui.js"
    };
    // These are the files you want optimized
    String primaryJavascriptToCompile[] = {
        "somejavascript.js",
        "otherjavascript.js"
    };
    // This is where the optimized code will end up
    String outputFilename = "combined.min.js";
    com.google.javascript.jscomp.Compiler.setLoggingLevel(Level.INFO);
    com.google.javascript.jscomp.Compiler compiler = new com.google.javascript.jscomp.Compiler();
    CompilerOptions options = new CompilerOptions();
    CompilationLevel.SIMPLE_OPTIMIZATIONS.setOptionsForCompilationLevel(options);
    WarningLevel.VERBOSE.setOptionsForWarningLevel(options);
    List<JSSourceFile> externalJavascriptFiles = new ArrayList<JSSourceFile>();
    for (String filename : externalJavascriptResources)
    {
      externalJavascriptFiles.add(JSSourceFile.fromFile(filename));
    }
    List<JSSourceFile> primaryJavascriptFiles = new ArrayList<JSSourceFile>();
    for (String filename : primaryJavascriptToCompile)
    {
      primaryJavascriptFiles.add(JSSourceFile.fromFile(filename));
    }
    compiler.compile(externalJavascriptFiles, primaryJavascriptFiles, options);
    for (JSError message : compiler.getWarnings())
    {
      System.err.println("Warning message: " + message.toString());
    }
    for (JSError message : compiler.getErrors())
    {
      System.err.println("Error message: " + message.toString());
    }
    FileWriter outputFile = new FileWriter(outputFilename);
    outputFile.write(compiler.toSource());
    outputFile.close();
  }
}

The above code is doing a number of things:

• It takes both external resources you don't want optimized as well as resources you do want optimized that refer to those external resources. You may not need to list external resources depending on what level of optimization you use.
• It combines all the input files that are not external into one output file.
• It compiles the javascript code given to it using the SIMPLE_OPTIMIZATIONS setting.
• It lists any warnings or errors it found while compiling the code.

One of the nice side effects of using the closure compiler is that because it compiles the code it can alert you to errors or potential issues using warnings.

What is listed above is using the middle of the road optimizations. There is a level above SIMPLE_OPTIMIZATIONS that you can set by changing that line in the above code to:

  CompilationLevel.ADVANCED_OPTIMIZATIONS.setOptionsForCompilationLevel(options);

Be aware that the advanced optimizations will do things, renaming your variables and functions to name two, that can break your code. The compiler gives you a way of alerting it to things you don’t want changed using externs and exports. Before using the advanced option you should read the advanced options tutorial.

If you do not like seeing all the steps the compiler takes you can set the logging level to QUIET by changing the WarningLevel line to the following:

  WarningLevel.QUIET.setOptionsForWarningLevel(options);

One last note about the compiler is that the closure inspector is a Firebug plugin that will let you see what the original line of code looked like before it was compiled. This can help a lot when something goes wrong.