ExoGENI Images: Background
In this post we describe how to create a custom ExoGENI Qcow2 image from an existing virtual machine. This post introduces the Horizon GUI that allow users to an image and script that allows a user to download the image and push it into the ExoGENI testbed. This process replaces the image creation process via a script that was previously supported.
ExoGENI does not host VM images, nor does it maintain a complete list of all images that can be used in its infrastructure. Instead, a request for compute resources contains the url of a metadata file that resides on any publicly accessible http server. In turn, the metadata file refers to a qcow2 image that resides on any publicly accessible http server (not necessarily the same http sever). Some commonly used images are listed in the Image Registry, and new images can be added at the user’s request, so long as the user is volunteering to maintain and host the image. Experimenter tools like Flukes and Jacks populate their lists of available images using this service.
After a slice request containing the new image is submitted, ExoGENI will transfer the metadata file and corresponding qcow2 image file to the necessary destination aggregates. A copy of each image will be stored at each rack as long as the rack does not run out of space for storing images. Although ExoGENI maintains a cached copy of each image the primary copy should remain accessible at its original location.
This model allows a user to use a custom image by creating a qcow2 image and metadata file and making them available on any http server. However, building a valid image from scratch is difficult. For this reason most users will want to capture snapshots of based on existing images as described below.
- Install Python 3.x
- Make sure you have disk space and an available HTTP server etc.
Brief Summary of the steps
With Exogeni Upgrade, users now have access to Horizon dashboard which can be used to create images. The following steps indicate this in detail.
- Create an ExoGENI virtual machine from an existing image.
- Determine Horizon details
- Login to Horizon and create image
- Retrieve the image and put it on your HTTP server
- Create a meta-data description file
- Add image to flukes via GUI
- Add image to flukes via properties file
Determine Horizon Details
- Select the Node and right click to View Properties of the Node in flukes to determine Horizon details for the virtual machine as shown below.
Login to Horizon and create image
- Access dashboard using credentials determined above and go to Compute -> Instances screen. Note down the Project Name shown on the top left corner.
- Select VM from which to create the image and click ‘Create Snapshot’
- Go to Compute -> Images screen to check status of image. It would change from queued to Active.
- Once the image is Active, click on Image Name to find out ID.
Retrieve the image and put it on your HTTP server
Download python image client which is required to download the image. Use the python client to download the image as indicated below.
wget http://geni-images.renci.org/images/tools/image_client.py python3.6 image_client.py -e http://rocky-hn.exogeni.net:8222 -p tenant-Slice1-B7P6GiWwOm -u owner-Slice1-B7P6GiWwOm -w GjxUjA2Pwr -i f95b02a4-064f-4c34-8c80-2dbc887c5af9 -f ./testImage.qcow2
NOTE: Python 3.6 and requests package should be installed before running image_client
Create a meta-data description file
- Generate shasum for the image
sha1sum testImage.qcow2 df9849bcb5bf4c7196b9252238be2cc3b2f0ad9b
- Create a testImage.xml file for the image which should include sha1sum for the image and http link to download the image as shown below
<images> <image> <type>QCOW2</type> <signature>df9849bcb5bf4c7196b9252238be2cc3b2f0ad9b</signature> <url>http://geni-images.renci.org/images/kthare10/testImage/testImage.qcow2</url> </image> </images>
- Get the hash of the metadata file:
sha1sum testImage.xml 159cebda015a203c1c5d9c392b65a86752f62f3c testImage.xml
Add image to flukes via GUI
Go to Request View Tab of Flukes and click on ExoGENI Info -> Compute Images button.
This will open a window titled Images. Click on the New button to add a new image.
It opens a form where you provide a name of the image, the URL of the xml file and the hash of the xml file. The “sha1sum filename.xml” command will give you the hash of the file.
Add image to flukes via properties file
Edit the .flukes.properties file from your home directory and include the following information in the file. Create .fluke.properties file if it does not exist.
image1.name=testImage image1.url=http://geni-images.renci.org/images/kthare10/testImage/testImage.xml image1.hash=159cebda015a203c1c5d9c392b65a86752f62f3c