Brian Parsons

While I haven't *knock on wood* had any EBS failures in Amazon's cloud myself, I have heard the horror stories and that makes me uneasy. Another issue with disks in cloud that I do run into a lot is latency. The disk io in many cases is slower to begin with, and random bouts of latency tend to crop up.

I have addressed both of these problems by deploying RAID 10 on my Amazon EC2 instances. It sounds techie but you don't have to be a rocket scientist to do this. If you are managing an EC2 instance you can do it and I have published a script that will get you there in a few steps.

First you need to have the ec2-api-tools installed and working on a machine. This can be a server but you can also do this on your workstation. For Arch Linux users, there is a package in the AUR.

The key to getting those tools working is setting up your environment variables. I use a little script called awsenv.sh like this:

#!/bin/bash

export AWS_USER_ID="0349-01234-09134"
export AWS_ACCESS_KEY_ID="BLAHDEBLAHBLAHBLAH"
export AWS_SECRET_ACCESS_KEY="somecharsthatmeansnothing"
export EC2_PRIVATE_KEY="/path/to/EC2-key.pem"
export EC2_CERT="/path/to/EC2-cert.pem"

Call it with: $ source awsenv.sh

Now you're ready to grab my script from: https://github.com/bparsons/buildec2raid

Once you have the api tools working, using the script is really easy:

Example:

$ ./buildec2raid.sh -s 1024 -z us-east-1a -i i-9i8u7y7y

This example would create a 1TB (terrabyte) array in the us-east-1a availability zone and attach it to instance i-918u7y7y.

The script does the basic RAID math for you. It uses 8 disks but you can change the DISKS variable near the top of the script if you prefer another topology. I really suggest that you use RAID 10. That way you can pull a slow EBS volume out of your array and then replace it without much hassle.

Once the volumes are created and attached to the instance, you log into the instance and initialize the array:

$ mdadm --create -l10 -n8 /dev/md0 /dev/xvdh*

That starts the array up. Then all you have to do is format it. Here is an XFS example:

$ sudo mkfs.xfs -l internal,lazy-count=1,size=128m -d agcount=2 /dev/md0

If you are new to software RAID you will find it helpful to check out the Linux RAID Wiki

Dont forget to add the mountpoint to your /etc/fstab file and create the /etc/mdadm.conf file:

# mdadm --examine --scan > /etc/mdadm.conf