Creating a Custom Image from an Existing Virtual Machine

In this post we describe how to create a custom ExoGENI image from an existing virtual machine.  This post introduces a new script that will capture an image and allow a user to push it into the ExoGENI testbed.  This process replaces the playpen image creation process that was previously supported.

ExoGENI Images: Background:

ExoGENI does not host VM images, nor does it maintain a complete list of all images that can be used in its infrastructure.  Instead, a request for compute resources contains the url of a metadata file that resides on any publicly accessible http server.   In turn, the metadata file refers to an image, kernel, and ramdisk that resides on any publicly accessible http server (not necessarily the same http sever). Some commonly used images are listed in the Image Registry, and new images can be added at the user’s request, so long as the user is volunteering to maintain and host the image. Experimenter tools like Flukes and Jacks populate their lists of available images using this service.

After a slice request containing the new image is submitted, ExoGENI will transfer the metadata file and corresponding image files to the necessary destination aggregates.  A copy of each image will be stored at each rack as long as the rack does not run out of space for storing images.   Although ExoGENI maintains a cached copy of each image the primary copy should remain accessible at its original location.

This model allows a user to use a custom image by creating an image, kernel, ramdisk, and metadata file and making them available on any http server.  However,  building a valid image from scratch is difficult.  For this reason most users will want to capture snapshots of based on existing images as described below.

Capturing the Image:

We have created script that will capture the image along with its kernel and ramdisk, then will create the ExoGENI metadata file. The remainder of this post is a step-by-step example of how to use the script.

1.  Create an ExoGENI virtual machine from an existing image.

The virtual machine must have enough disk space to store a copy of itself.   An easy way to ensure there is enough space is to include an iSCSI storage device.

2. Login to the VM as root.

3. Modify the VM as you wish.

3. Get the snapshot script. Note: future versions of NEuca will include the script.

# wget  http://geni-images.renci.org/images/tools/imgcapture.sh
# chmod +x imgcapture.sh

4. Run the script

The script takes four parameters:

./imgcapture.sh -v -n <name> -d <destination> -s <size> -u <url>

-n <name>:  The name of the new image.
-d <destination>:  The directory to put the image files. This directory must exist before running the script.  If you are using an iSCSI target, set the destination to your storage mount point.
-s <size>:  The size of the image (should be as small as possible).  Run ‘du -h’ and use a size that is approximately 10% larger than the value in the “Used” column for the main disk (likely ‘/dev/vda’).   Note that this value determines the minimum size instance that must me used with this image.  Each instance type can only be used by images smaller than the following:  XO.small < 10G, XO.medium < 25G, XO.large < 50G, XO.xlarge < 75G.
-u <url>:  The http url of the web server where you will host your image.   Used to populate the metadata file.

Example:

# ./imgcapture.sh -n new_image -s 2G -u http://geni-images.renci.org/images/new_image -d /tmp

The script will ask you to confirm the kernel version you would like to include in the package.  Choose the kernel. Likely you will want the newest kernel

Select an available kernel by number from below:
 1) vmlinuz-2.6.32-431.5.1.el6.x86_64
 2) vmlinuz-2.6.32-431.17.1.el6.x86_64
 3) vmlinuz-2.6.32-431.1.2.0.1.el6.x86_64
 4) vmlinuz-2.6.32-279.14.1.el6.x86_64
2
Your chosen kernel is vmlinuz-2.6.32-431.17.1.el6.x86_64. Is this correct? (Y|n)Y

The script will continue to print a lot of messages as it copies many files into the new image.

After the script completes you will have several files in the destination directory.

# ls /tmp/
filesystem initramfs-2.6.32-431.5.1.el6.x86_64.img new_image.tgz new_image.xml vmlinuz-2.6.32-431.5.1.el6.x86_64

The image metadata file is the .xml file.   The file will contain an xml description of your image including the image, kernel, and ramdisk along with the hash of each file.

# cat /tmp/new_image.xml 
<images>
 <image>
 <type>ZFILESYSTEM</type>
 <signature>e2eace6367ed1d618c26cb6891ae3e02202ce012</signature>
 <url>http://geni-images.renci.org/images/new_image/new_image.tgz</url>
 </image>
 <image>
 <type>KERNEL</type>
 <signature>5e76d034e27cb861275fc3d1bd9d97c3905ef5a2</signature>
 <url>http://geni-images.renci.org/images/new_image/vmlinuz-2.6.32-431.5.1.el6.x86_64</url>
 </image>
 <image>
 <type>RAMDISK</type>
 <signature>c999527cc0d8739a3d3f38d48f03fa6c73151cca</signature>
 <url>http://geni-images.renci.org/images/new_image/initramfs-2.6.32-431.5.1.el6.x86_64.img</url>
 </image>
</images>

Transfer image tarball, kernel, ramdisk, and metadata file from the virtual machine to the http server you specified on the script’s command line.   The http server can be any publicly available http server that you have write access to.  Note: you only need to transfer these four files. You do NOT need to transfer the uncompressed filesystem image.

# scp /tmp/initramfs-2.6.32-431.5.1.el6.x86_64.img /tmp/vmlinuz-2.6.32-431.5.1.el6.x86_64 /tmp/new_image.tgz /tmp/new_image.xml pruth@geni-images.renci.org:/images/new_image/.
pruth@geni-images.renci.org's password: 
initramfs-2.6.32-431.5.1.el6.x86_64.img    100% 12MB 2.0MB/s 00:06 
vmlinuz-2.6.32-431.5.1.el6.x86_64          100% 4033KB 3.9MB/s 00:01 
new_image.tgz                              100% 345MB 1.6MB/s 03:38 
new_image.xml                              100% 707 0.7KB/s 00:00

Get the hash of the metadata file:

# sha1sum /tmp/new_image.xml 
159cebda015a203c1c5d9c392b65a86752f62f3c /tmp/new_image.xml

You are now ready to use you new image.  The image url is the url that can be used to access the metadata file on your server.   Add your image’s equivalent to the following to your .flukes.properties files or add the url and hash using the Flukes GUI.  Note:  ‘image1’ should be changed to the next sequential number in your .flukes.properties file.

image1.name=new_image
image1.url=http://geni-images.renci.org/images/new_image/new_image.xml
image1.hash=159cebda015a203c1c5d9c392b65a86752f62f3c

Adding a new compute image to Flukes using the GUI:

Go to Request View Tab of Flukes and click on Compute Images button.

MainWindow

This will open a window titled Images. Click on the New button to add a new image.

ListOfImages

It opens a form where you provide a name of the image, the URL of the xml file and the hash of the xml file. The “sha1sum filename.xml” command will give you the hash of the file.

EnterNewImage

Adding an image Flukes by editing .flukes.properties file:

Edit the .flukes.properties file from your home directory and include the following information in the file. Create .fluke.properties file if it does not exist.

image1.name=new_image
image1.url=http://geni-images.renci.org/images/new_image/new_image.xml
image1.hash=159cebda015a203c1c5d9c392b65a86752f62f3c

Things might go wrong:

1. If your image does not have a kernel or ramdisk in the /boot directory the script will not create the metadata file. To solve this problem install the kernel image from the package manager (e.g. yum or apt-get). Or you can create the metadata file in an editor using the kernel and ramdisk from the base image’s metadata file.

2. If you make changes to your VM that affect the neuca installation you may need to re-install neuca before taking a snapshot of the image.  This might occur if you python version changes significantly.

Neuca has to be reinstalled in that case:

  1. Uninstall the already installed Neuca if it is already there
apt-get uninstall neuca
  1. Download and install neuca-guest-tools
wget http://software.exogeni.net/repo/exogeni/neuca-guest-tools/
       neuca-guest-tools_1.4-1_all.deb
 dpkg -i neuca-guest-tools_1.4-1_all.deb

Have something to add?

Loading Facebook Comments ...