Ubuntu & EC2 – Device or resource busy

I’m currently in the process of writing automation to deploy replica sets of mongodb to the cloud with a single command. When attaching my EBS volumes I noticed that I was regularly getting an error that stated:

mdadm: cannot open /dev/xvdf: Device or resource busy

I was banging my head against the wall trying to figure out what the issue was. At first I didn’t think it was an issue with my command because after sometime the mdadm command I was using seemed to work; however that was exactly the case, there was an issue with my mdadm command, so here is the command I am now using to build a RAID 10 volume using 4 EBS volumes:

mdadm --create --force --assume-clean -R /dev/md0 -l10 --chunk=256 --raid-devices=4 /dev/xvdf /dev/xvdg /dev/xvdh /dev/xvdi

I figured others might be having the same issue and wanted to share the right way to create the mirrored device.

On a side note, I also setup this fabric task to do the following tasks (this eventually will be called in a larger fab task which does a full blown mongodb deployment and replica set creation):

• Run puppet to make sure directories are properly created
• Run mdadm to build a mirrored lun /dev/md0
• Set readahead to 128 on the mirror and on each of the EBS volumes which make up the mirror
• Setup a new physical volume under LVM control
• Create a new volume group
• Create two logical volumes, one for the mongodb datafiles and the other for the journal
• Leave free space in the volume group for snapshot purposes
• Format the volumes with ext4
• Add the volumes and mount points to /etc/fstab
• Mount the volumes
• Make a symlink from /journal to /data/db.1/journal
• Change ownership of the directories to be owned by the user mongodb and group mongodb

Here is a copy of the fab task if you would like to use it or copy the command into a shell script:

https://gist.github.com/3005807

Hopefully this helps you in your journey with EC2 and with mongodb.