I've been using S3 to store semi-transient information like log files from EC2 nodes in the past but recently decided to give Amazon's Elastic Block Store (EBS) a try instead. I quickly realized a downside to using EBS in that there is no mechanism for auto-attach and mounting volumes when an AMI is launched. This is probably something Amazon will fix at some point and allow you to launch a given AMI with an attached EBS volume but until then you need some way of doing it yourself. The following is a simple way of using ruby to make it happen.
I'm going to assume you have already created your EBS volume, if you haven't you can learn more about that from the docs. You will need to make sure ruby is installed on the AMI you are going to use and that the RightScale AWS gem is installed as well.
The following script grabs the instance id from the EC2 metadata URL. It then uses the RightScale EC2 calls to attach the volume to the current EC2 instance. After the attach call it may take a few seconds for the volume to become ready so the script sleeps for a few seconds before calling out to the system to mount the device. One enhancement that is obvious here would be to have the script make a RightScale EC2 call to determine when the volume is really ready and then continue after that.
#!/usr/bin/ruby
require 'rubygems'
require 'right_aws'
require 'net/http'
url = 'http://169.254.169.254/2008-02-01/meta-data/instance-id'
instance_id = Net::HTTP.get_response(URI.parse(url)).body
AMAZON_PUBLIC_KEY='your public key'
AMAZON_PRIVATE_KEY='your private key'
EC2_LOG_VOL='the volume id'
ec2 = RightAws::Ec2.new(AMAZON_PUBLIC_KEY, AMAZON_PRIVATE_KEY)
vol = ec2.attach_volume(EC2_LOG_VOL, instance_id, '/dev/sdh')
puts vol
# It can take a few seconds for the volume to become ready.
# This is just to make sure it is ready before mounting it.
sleep 20
system('mount /dev/sdh /mymountpoint')
I called the script mountlogs.rb and call it out of the local startup script /etc/rc.local so it mounts the disk on startup. This seems to work pretty well as a stopgap until Amazon puts in place a way to auto-attach EBS volumes to instance creation.
Thanks for the script Carson. Do you know if Amazon created an auto-attach method yet? A quick Google and look over the API docs suggests not.
Also, there doesn't seem to be a way of programmatically grabbing the Volume ID. So if your EBS volume dies and you have to re-create from a snapshot, the ID changes and you have to edit your script and possibly rebuild your whole AMI; a far from ideal solution :(
Anyway, thanks again!
James
I don't think they have an auto-attach yet.
I think you could get around rebuilding the AMI by passing the data you need in as a parameter when you launch the instance. The entire solution isn't ideal but until they provide something better it is probably the only way to take care of it.
Some other issues I have seen and read about are that sometimes the EBS volumes get stuck in a mounted state and can't be remounted until they are forced to release. That kind of tosses a wrench into the mix too but again there isn't a lot that can be done about it. Over time I'm sure Amazon will work out those kinks.
Pingback: Symphonious » RightScale AWS Amazon EC2 Library for Ruby
Good stuff!
I have checked that an attached volume state transitions from "available" to "in-use", but when in that state… isn't there any other way of confirming that the volume can safely used (for formatting it, for using it with mdadm or whatever), instead of waiting 20 seconds?
Regards,
David
It can probably be done through the API but I'm not sure if it is worth it. You could also probably look to see if the device is available in a loop.