Getting Started with JRuby Webapps using Rack and Sinatra

By | Published: October 7, 2010

Going to JRubyConf inspired me enough to expand my knowledge of JRuby. This is a quick guide for anyone looking to see what can be done with JRuby and Sinatra.

Of course the first step is to install the Java JDK. Download, install and then set it up in your path. I'm using JDK 6 update 21 on Linux for this post in case it matters.

There are a couple ways to install JRuby after you have Java installed. The hard way is to download JRuby and install it by hand. The easy way is to use Ruby Version Manager to install it. Using RVM will let you switch between Ruby versions as well as giving you a simple command to install JRuby:

rvm install jruby
rvm use jruby

Read More »

Posted in programming | Tagged , , , | Leave a comment

Minimal EC2 Linux Install Using TTYLinux

By | Published: September 14, 2010

If you have ever wondered how to get a Linux EC2 node down to the bare minimum this post is for you. I have been wanting to do this for a long time but it wasn't possible until pv-grub support that was added recently. To make this even more exciting Amazon now offers EC2 micro instances that will go well with this type of bare bones system.

You may wonder why you would want to do this. I'll give you at least two reasons. First there is speed. The configuration I ended up with will boot in about 14 seconds. Almost all of that time is spent waiting on the AWS DHCP server for an IP address. The second reason is security. There is little on this system that you don't need. The majority of what you need is provided by a single application that you can easily monitor for patches. That leaves you to worry about only the tools you must have to get your job done. I'll also include a couple downsides. First this isn't going to be easy for everyone and it can be fairly complicated. The second issue you will run into is that there is nothing in this distribution. If you need something like PHP you might spend a long time building all the support you need. There isn't much to be done about the complexity if you want to do this all yourself. Selection of tools can help you lessen the impact of the bare nature of the system, more on that in a future post.

Read More »

Posted in system administration | Tagged , , | 4 Comments

How to Build and Compile a Custom Linux Kernel for EC2

By | Published: September 9, 2010

I have a long running goal that I'm trying to reach with all these pv-grub for EC2 posts. That goal is to find the smallest/tightest usable node that can be created for EC2. The next step in that path requires a custom Linux kernel. What follows is how to build the latest Linux kernel so that it works on EC2 using pv-grub.

It is important to have a recent kernel since most of what is needed to get a kernel to work on EC2 is now incorporated into the source. These instructions assume the latest kernel is 2.6.35.4 and I've used them with 2.6.35 as well but keep that in mind since the one patch that is required could eventually be merged in. Before getting started it may help to read over how to compile the Linux kernel normally and then my post on running CentOS 5.5 on EC2 using pv-grub.

Read More »

Posted in system administration | Tagged , , | 7 Comments

Using a HTTP Proxy to Debug JAX-WS and SOAP Over HTTPS

By | Published: September 7, 2010

Every once in a while I run into something I need to debug from the network up. Most of the time I can do this using Wireshark but there are a few instances where what I'm really looking for is a man in the middle proxy. I usually find an alternative way to debug what I'm working on but recently I decided to find a combination that worked so the next time I can use it.

Most of the time what I'm working on when I need a proxy has something to do with SOAP and there is actually a MITM proxy in SoapUI but it doesn't have support for HTTPS. If you are working with SOAP and can use HTTP then SoapUI is a great tool. If that doesn't fit the bill then Paros proxy does a good job.

Read More »

Posted in programming | Tagged , | Leave a comment

Converting from VirtualBox or VMWare to EC2 now Easier than Ever

By | Published: August 25, 2010

The new PV-Grub ability introduced by Amazon for EC2 has opened the door for more than just custom kernels, it also makes it a lot easier to turn VirtualBox and VMWare instances into EC2 instances. In the past I have written about transfering VirtualBox images to Xen but to do that with EC2 required a matching kernel exist for your VirtualBox installed OS that was blessed by Amazon. With PV-Grub as long as you can get a kernel for your existing system that is compatible with the EC2 infrastructure you can transfer it. A lot of the most popular distributions already have compatible kernels so that shouldn't be an issue and if you had to you could always compile the kernel by hand.

Some of what follows is exactly the same as my post about installing CentOS 5.5 on EC2 with the stock kernel. Once you get the hang of it you can install just about anything Linux based to EC2. I've broken this post into two parts to try and separate the generic transfer information from the specifics of an example. The first part that goes over the basics of what needs to be done to transfer any VirtualBox or VMWare box to EC2. The second part is an example of transferring an Ubuntu Server install to from VirtualBox to EC2.

Read More »

Posted in system administration | Tagged , , | 5 Comments

Installing Cent OS 5.5 on EC2 with the Cent OS 5.5 Kernel

By | Published: August 10, 2010

Amazon recently introduced the ability to boot a custom kernel using pv-grub on EC2. This opens the door for all kinds of interesting ideas that I've been thinking about for a while, like seeing if I can boot right into a web server and skip all that extra junk that comes with Linux distributions, but that is just me. The main door it is going to open for most people interested in EC2 will be the ability to upgrade the kernel that comes with their distribution. That brings us to how to install Cent OS 5.5 on EC2 and use the kernel that is part of the distribution.

For those who might just be interested in booting a custom kernel using EC2 pv-grub I will try to produce a few more posts with more details on that but for now here are the main things to know:

  • The pv-grup kernels named with hd00 will look on the first partition of the registered device in the /boot/boot/grub directory for a menu.lst file. Use this type of kernel if you create want to use a partitioned disk.
  • The pv-grup kernels named with hd0 will look on the registered device in the /boot/grub directory for a menu.lst file. Use this type of kernel if you don't have a partition on your disk.
  • You won't get anything meaningful back from the boot attempt if your grub menu.lst file is in the wrong place or is not valid. See the end of the post for what a pv-grub error message looks like and some tips on what to do if you see it.
  • The kernel you use does matter but the current mainline Linux kernel (2.6.35) contains everything you need except for a small change to turn off XSAVE. The main thing to know is that not every distribution may have made the change needed to work on EC2.
  • I have tried non-Linux kernels to no avail. See the end of the post for a little more information.

Read More »

Posted in system administration | Tagged , , , | 24 Comments

Compiling WebM into FFMpeg for Windows

By | Published: July 13, 2010

I have updated this post with a newer version of the VP8 patches to FFMpeg and support for libvorbis instead of the built in vorbis support.

Google has released source for the VP8 codec as the WebM project. The WebM project will be an open alternative in the HTML5 video tag codec space and being backed by Google, who will use it for YouTube, will give it a fighting chance. WebM is starting to be integrated into FFMpeg but there are still some patches that need to be applied. While there are some guides to how to build FFMpeg for Linux with VP8 WebM support I wanted to get it compiled for Windows.

First off this won't be done using a Windows install but instead using Virtual Box with a fresh install of Ubuntu 10.04 desktop. It is just faster and easier that way. So go ahead and grab Virtual Box and get Ubuntu installed on it before you continue.

Read More »

Posted in meta | Tagged , , , | 12 Comments

Upgrade to Fedora 13 from Fedora 12

By | Published: May 27, 2010

After a week delay Fedora 13 has been released and it is time to upgrade of course. As always there are a decent number of features in this release but here are a few that stand out to me:

  • boot.fedoraproject.org – A small bootable image to start the install from.
  • IntelliJ IDEA – IntelliJ IDEA community edition. I'm glad to see my favorite IDE make it in along with Netbeans and Eclipse even if it is just the stripped down version.
  • Python 3 – Running along side Python 2.
  • Nouveau DisplayPort – Support for NVIDIA cards with a DisplayPort.
  • Radeon DisplayPort – Support for Radeon cards with a DisplayPort.
  • NetworkManager Command Line – A command line interface for NetworkManager. As much as I find myself hating NetworkManager maybe this will help.
  • Gnome 2.30 – The latest stable version of Gnome.
  • KDE 4.4 – The latest stable version of KDE.

Check out the Fedora 13 feature list for all the major features in this release.

Read More »

Posted in system administration | Tagged , | Leave a comment

Faceted Search With Sphinx

By | Published: May 18, 2010

I decided to use the Sphinx search engine for the GeeQe iPhone app I build last year because it was fast and had a very small memory footprint. Recently I wanted to experiment with a search interface that had facets and wondered if I would need to move away from Sphinx to something like Solr. As it turns out Sphinx can do faceted search almost as well as Solr can. The first half of what follows contains instructions on how to get Sphinx ready for faceted searches. If you are familiar with setting up Sphinx and already have data indexed by Sphinx you may find it better to skip to the second half after reading the intro to faceted search.

You have almost certainly seen faceted searching or faceted browsing already. A lot of online retailers now use facets in their online stores. Here are a few examples:

Read More »

Posted in programming | Tagged , | 1 Comment

How to Create iPad Formatted Videos Using HandBrake or FFMpeg

By | Published: April 13, 2010

This is for those who may want to load a video onto their iPad with iTunes that isn't in the correct format. I needed to do this because I was trying to put an iTunes University video on my iPad a couple days ago and iTunes complained that it wasn't in the correct format for the iPad. I'm not sure exactly how that could be but I decided to take the opportunity to see if I could use HandBrake to easily convert it to an iPad friendly format. There is currently no pre-loaded iPad configuration for HandBrake like there is for the iPhone and iPod Touch so I created a few profiles that can easily be imported into HandBrake to output different sizes for the iPad:

Note that when you import videos using iTunes the iPad puts them in their own Videos app unlike the iPhone where they show up under the iPod app. You will want to find the Videos icon if you don't already know where it is:

iPad Videos Screenshot

After verifying that the above 4×3 version worked for the iTunes University video I went about testing it on a couple other video formats. I tested the Big Buck Bunny video that I also used for my post on iPad video streaming in both 640×360 and 1024×576 output formats. Both resolutions looked great. I also tried converting a DVD. If you decide to convert a DVD you will probably want to turn on de-interlacing in HandBrake. You do that by first selecting the "Picture Settings" option:

HandBrake Picture Settings Screenshot

Then the filters tab and then select the type of de-interlace you want (fast, slow, slowest):

HandBrake DeInterlace Settings Screenshot

If you want to use FFMpeg to do all this you can. The following is a slightly modified version of the streaming command I'm using that will output a high bitrate version of the input video:

ffmpeg -y -i input.avi -acodec aac -ar 48000 -ab 128k -ac 2 -s 1024×768 -vcodec libx264 -b 1200k -flags +loop+mv4 -cmp 256 -partitions +parti4x4+partp8x8+partb8x8 -subq 7 -trellis 1 -refs 5 -coder 0 -me_range 16 -keyint_min 25 -sc_threshold 40 -i_qfactor 0.71 -bt 1200k -maxrate 1200k -bufsize 1200k -rc_eq 'blurCplx^(1-qComp)' -qcomp 0.6 -qmin 10 -qmax 51 -qdiff 4 -level 30 -aspect 16:9 -r 30 -g 90 -async 2 output.mp4
Posted in meta | Tagged , , , | 23 Comments