Grid Appliance On Science Clouds
From Grid-Appliance Wiki
Contents |
Introduction
The Science Clouds are an informal group of small clouds made available by various institutions on a voluntary basis. Science Clouds provide compute cycles in the cloud for scientific communities using Nimbus. This tutorial introduces you to using and interacting with grid appliance on the science clouds.
Getting Started
Please check the list of available science clouds. To gain access to any of these clouds, you would have to check its respective website and contact the administrator for that cloud. Once you have recieved the required credentials to access a cloud, follow the cloud client quick start tutorial. As mentioned in the quick start tutorial, you would be able to see a list of all the appliances currently hosted in the marketplace by issuing the command:
$ ./bin/cloud-client.sh --list
If you do not see a grid-appliance image already listed in the science cloud repository, you can download the image form scienceclouds marketplace, and upload it to your cloud repository. For using the 32-bit image, replace amd64 with i386 in the below commands:
cd /tmp wget http://www.nimbusproject.org/downloads/grid-appliance-amd64.gz #change to cloud client directory ./bin/cloud-client.sh --transfer --sourcefile /tmp/grid-appliance-amd64.gz
Note:
- The appliances are built on Ubuntu Karmic (9.10). Some of the Science Clouds (including FutureGrid clouds) which provide Linux kernels prior to 2.6.31, are not able to run Ubuntu 9.10 or later releases. For such clouds, follow this tutorial to use the grid-appliance images built on Ubuntu Jaunty (9.04).
- Only the base package (grid-appliance-base) is installed. You may install the additional grid-appliance packages as-per your requirements. For more details, please check TestingGridAppliance.
- To use the grid-appliance-nfs package, ensure that the kernel provided in the cloud supports NFS.
- The appliances have been successfully tested on the Science Cloud provided at University of Chicago which provides 64-bit 2.6.31 linux kernel (by default).
- When using the 32-bit appliance on this cloud, the kernel would be 64-bit and the userspace would be 32-bit.
- The appliances have a user named 'griduser' which can be used to submit condor jobs.
Interacting with Grid Appliance
The cloud-client discussed above, in-turn calls the workspace reference client for interacting with the cloud workspace as explained here. To interact with your cloud appliance using the nimbus, you can either use cloud-client or the reference-client directly. The drawback with using the cloud-client is that it does not allow a user to send data to the cloud appliance at the time of creating it.
Using the Cloud Client
At this point, it is expected that you have read the cloud client quick start tutorial. For help and additional help messages, use the below commands:
$ ./bin/cloud-client.sh --help $ ./bin/cloud-client.sh --extrahelp
To start a workspace of the 64-bit grid-appliance, use the below command:
$ ./bin/cloud-client.sh --run --name grid-appliance-amd64.gz --hours 1
Once the workspace reaches "running" state, it's the hostname/IP address would be dispalyed. To log into the Grid Appliance, ssh using the root user.
$ ssh root@<grid_appliance.cloudurl.edu>
The appliance would not connect to any pool at start-up when it is instantiated using the cloud-client. To connect to the default public pool, simply install the grid-appliance-public-pool package in your grid-appliance.
# apt-get install grid-appliance-public-pool
To connect to any other GroupVPN pool, you need to place a floppy disk containing the config files inside the appliance in the /opt/grid_appliance/etc directory and then restart the grid_appliance service. This floppy can be generated using the grid-appliance.org web front-end. The steps are as mentioned below: 1. Stop the grid_appliance service in the grid-appliance
# /etc/init.d/grid_appliance.sh stop # exit
2. Download the floppy to your local system from the grid-appliance website. Transfer it to the Grid Appliance workspace using scp or sftp.
$ scp ./floopy.img root@<grid_appliance.cloudurl.edu>:/opt/grid_appliance/etc/
3. SSH back into grid appliance and start the grid appliance service
$ ssh root@<grid_appliance.cloudurl.edu> # /etc/init.d/grid_appliance.sh start
Using the Workspace Reference Client
Before creating a workspace with the reference-client, you need to prepare a metadata file. Check the sample metadata file shown in sciencecloluds marketplace.
Using the grid-appliance.org web front-end, download a floppy (floppy.zip) for connecting to a GroupAppliance pool of your choice. Encode this floppy to base using openssl or any other tool of your choice:
$ openssl enc -base64 -in floppy.zip -out floppy.zip.b64
You can send this floppy to the cloud using the "--mdUserdata" option while creating the workspace and only the cloud appliance would be able to access it.
Use the below commands to see the various other options provided by the reference-client.
$ sh ./lib/workspace.sh --help $ sh ./lib/workspace.sh --h --deploy
To create an appliance in the UC cloud, you may use the below command:
sh ./lib/workspace.sh --deploy \ --mdUserdata ../floppy.zip.b64 \ --service https://tp-vm1.ci.uchicago.edu:8445/wsrf/services/WorkspaceFactoryService \ --file ../vw-epr.xml \ --metadata ../metadata.xml \ --deploy-mem 1000 \ --deploy-duration 120 \ --trash-at-shutdown Trash \ --exit-state Running \ --authorization /O=Grid/OU=GlobusTest/OU=simple-workspace-ca/CN=host/tp-vm1.ci.uchicago.edu \ --displayname grid-appliance \ --sshfile /home/griduser/.ssh/id_rsa.pub \ --poll-delay 2000
You can use the epr file to terminate a workspace created in this manner:
$ ./bin/cloud-client.sh --terminate --epr-file ../vw-epr.xml
NOTE: An enhancement request (http://bugzilla.mcs.anl.gov/globus/show_bug.cgi?id=7059) is logged with nimbus team to allow an easier way to transfer user files to the VM. Once this feature is implemented, one can use the cloud client directly to send a floppy image during VM instantiation.
