Category Archives: system administration

Example Git pre-receive and post-receive Hooks to Avoid a Signal 13 Error

Git has a very nice set of example hooks that you can find just by creating a new repo. The following is an easy way to see them all:

mkdir myrepo
cd myrepo
git init
ls .git/hooks/

If you do that you will end up with a list something like this:

applypatch-msg.sample  post-commit.sample   post-update.sample     pre-commit.sample          pre-rebase.sample
commit-msg.sample      post-receive.sample  pre-applypatch.sample  prepare-commit-msg.sample  update.sample

I recently did this thinking I would create a pre-receive hook. Since there isn't a sample for pre-receive I went to the git hooks reference to see if there was anything special I needed to know. I didn't pay any attention to the fact that git will send something in on STDIN when it runs the hook and as it turns out that is important. In some instances if you don't read STDIN you will get an error message like the following when you try to push and the hook gets used:

fatal: The remote end hung up unexpectedly
error: error in sideband demultiplexer
error:  died of signal 13

The fix for this is to make sure you read STDIN when creating pre-receive and post-receive hooks. The following example shell script will show the commit information for each new revision id when you push and the hook is executed:

#!/bin/sh
while read oldrev newrev refname
do
  # Do something with $oldrev $newrev $refname 
  git show --no-color --root -s --pretty=medium $newrev
done

The key part in the above is the "while read" that reads STDIN. So whatever language you write your hook in just make sure you read everything from STDIN before completing the script.

Minimal EC2 Linux Install Using TTYLinux

If you have ever wondered how to get a Linux EC2 node down to the bare minimum this post is for you. I have been wanting to do this for a long time but it wasn't possible until pv-grub support that was added recently. To make this even more exciting Amazon now offers EC2 micro instances that will go well with this type of bare bones system.

You may wonder why you would want to do this. I'll give you at least two reasons. First there is speed. The configuration I ended up with will boot in about 14 seconds. Almost all of that time is spent waiting on the AWS DHCP server for an IP address. The second reason is security. There is little on this system that you don't need. The majority of what you need is provided by a single application that you can easily monitor for patches. That leaves you to worry about only the tools you must have to get your job done. I'll also include a couple downsides. First this isn't going to be easy for everyone and it can be fairly complicated. The second issue you will run into is that there is nothing in this distribution. If you need something like PHP you might spend a long time building all the support you need. There isn't much to be done about the complexity if you want to do this all yourself. Selection of tools can help you lessen the impact of the bare nature of the system, more on that in a future post.

Continue reading

How to Build and Compile a Custom Linux Kernel for EC2

I have a long running goal that I'm trying to reach with all these pv-grub for EC2 posts. That goal is to find the smallest/tightest usable node that can be created for EC2. The next step in that path requires a custom Linux kernel. What follows is how to build the latest Linux kernel so that it works on EC2 using pv-grub.

It is important to have a recent kernel since most of what is needed to get a kernel to work on EC2 is now incorporated into the source. These instructions assume the latest kernel is 2.6.35.4 and I've used them with 2.6.35 as well but keep that in mind since the one patch that is required could eventually be merged in. Before getting started it may help to read over how to compile the Linux kernel normally and then my post on running CentOS 5.5 on EC2 using pv-grub.

Continue reading

Converting from VirtualBox or VMWare to EC2 now Easier than Ever

The new PV-Grub ability introduced by Amazon for EC2 has opened the door for more than just custom kernels, it also makes it a lot easier to turn VirtualBox and VMWare instances into EC2 instances. In the past I have written about transfering VirtualBox images to Xen but to do that with EC2 required a matching kernel exist for your VirtualBox installed OS that was blessed by Amazon. With PV-Grub as long as you can get a kernel for your existing system that is compatible with the EC2 infrastructure you can transfer it. A lot of the most popular distributions already have compatible kernels so that shouldn't be an issue and if you had to you could always compile the kernel by hand.

Some of what follows is exactly the same as my post about installing CentOS 5.5 on EC2 with the stock kernel. Once you get the hang of it you can install just about anything Linux based to EC2. I've broken this post into two parts to try and separate the generic transfer information from the specifics of an example. The first part that goes over the basics of what needs to be done to transfer any VirtualBox or VMWare box to EC2. The second part is an example of transferring an Ubuntu Server install to from VirtualBox to EC2.

Continue reading

Installing Cent OS 5.5 on EC2 with the Cent OS 5.5 Kernel

Amazon recently introduced the ability to boot a custom kernel using pv-grub on EC2. This opens the door for all kinds of interesting ideas that I've been thinking about for a while, like seeing if I can boot right into a web server and skip all that extra junk that comes with Linux distributions, but that is just me. The main door it is going to open for most people interested in EC2 will be the ability to upgrade the kernel that comes with their distribution. That brings us to how to install Cent OS 5.5 on EC2 and use the kernel that is part of the distribution.

For those who might just be interested in booting a custom kernel using EC2 pv-grub I will try to produce a few more posts with more details on that but for now here are the main things to know:

  • The pv-grup kernels named with hd00 will look on the first partition of the registered device in the /boot/boot/grub directory for a menu.lst file. Use this type of kernel if you create want to use a partitioned disk.
  • The pv-grup kernels named with hd0 will look on the registered device in the /boot/grub directory for a menu.lst file. Use this type of kernel if you don't have a partition on your disk.
  • You won't get anything meaningful back from the boot attempt if your grub menu.lst file is in the wrong place or is not valid. See the end of the post for what a pv-grub error message looks like and some tips on what to do if you see it.
  • The kernel you use does matter but the current mainline Linux kernel (2.6.35) contains everything you need except for a small change to turn off XSAVE. The main thing to know is that not every distribution may have made the change needed to work on EC2.
  • I have tried non-Linux kernels to no avail. See the end of the post for a little more information.

Continue reading

Upgrade to Fedora 13 from Fedora 12

After a week delay Fedora 13 has been released and it is time to upgrade of course. As always there are a decent number of features in this release but here are a few that stand out to me:

  • boot.fedoraproject.org – A small bootable image to start the install from.
  • IntelliJ IDEA – IntelliJ IDEA community edition. I'm glad to see my favorite IDE make it in along with Netbeans and Eclipse even if it is just the stripped down version.
  • Python 3 – Running along side Python 2.
  • Nouveau DisplayPort – Support for NVIDIA cards with a DisplayPort.
  • Radeon DisplayPort – Support for Radeon cards with a DisplayPort.
  • NetworkManager Command Line – A command line interface for NetworkManager. As much as I find myself hating NetworkManager maybe this will help.
  • Gnome 2.30 – The latest stable version of Gnome.
  • KDE 4.4 – The latest stable version of KDE.

Check out the Fedora 13 feature list for all the major features in this release.

Continue reading

Fedora 12 Bootable Root EBS on EC2

I recently needed to create a clean EC2 AMI using a fairly new linux distro. It has been a while since I've needed to create a new AMI so I also wanted to move away from the older pre-packaged AMI and boot using EBS. After taking a look at what was currently available publicly I decided I would just create my own EBS bootable AMI using Fedora 12. It wasn't all that complicated but there are a decent number of steps so I figured I would document them for anyone else who might want to give it a try.

I'm going to assume you already understand how to do things like create instances, create EBS volumes and ssh into your running instance using key based authentication. I use the AWS management console for a lot of what follows with the exception of needing to register the AMI and for that you will need the Amazon EC2 API Tools and Amazon EC2 AMI Tools

Continue reading

Building Chromium and Chrome OS with EC2

When the initial cut of the Chromium OS source was released last week I decided to use the opportunity to see if it would run on my EEE PC 900 netbook (check out EEE PC 900 running Chrome OS on Youtube to see the final result). The first roadblock I hit with the build instructions was the Ubuntu requirement (I did give a little effort to getting it working on Fedora first). I don't have an Ubuntu box so I started out trying to use VirtualBox but that was going to take forever so I decided to move things to EC2 and what follows is the result. This isn't meant to be a replacement for the build docs since they are surely going to change, it is more of a cookbook to build Chromium (the browser) and Chromium OS using EC2 (EBS is used as well if you want to cache the source over time).

When I first started down the path of using EC2 I thought I would grab the source each time I wanted to build. I quickly ran into a snag however because it took forever to sync the source and download the Ubuntu repo. Once I had the initial sync of the source I decided I would copy it all to an EBS volume and keep that volume up to date. Using EBS to store the source feels better too since I assume Google expects people to be syncing changes only as opposed to pulling the entire source tree down every time they want to build.

I started out by finding this Ubuntu AMI for a base to work from. For the most efficient compile times I ended up using the High CPU (c1.medium) instance. I started with the default small instance but it was just too slow. With the high cpu instance you are looking at about 45 minutes to build the OS after you have the source synced for the first time and if you add building Chromium in there you are looking at around 55 additional minutes. All told you can have a complete build in less than 2 hours even if there are some source updates needed. For EBS you need a 3G volume for the Chrome OS source plus Ubuntu package repo and a 4G volume for the Chromium source.

Continue reading

Upgrade to Fedora 12 from Fedora 11

Fedora 12 was just released and it is time to upgrade again of course. I almost thought this was going to be a version to yawn at but then I saw that there was going to be a new version of Fedora based on Moblin and it seemed exciting again. Of course that isn't the only thing being upgraded in the latest version of Fedora. Some of the more notable changes in this version:

You can find the complete list of Fedora 12 enhancements as well if you want more details.

Continue reading

Upgrading to Fedora 11 from Fedora 10

It is time again to upgrade if you are using Fedora. Fedora 11 was released a few days ago and contains some nice enhancements.

For people who can a complete re-install is probably best. One reason for that is the inclusion of ext4 in Fedora11. You won't get the benefit of ext4 unless you do a fresh install or upgrade from ext3 to ext4. If you read the upgrade guide that Fedora produces it recommends not doing an upgrade.

Continue reading