MPI Virtual Cluster Appliance
From Grid-Appliance Wiki
Contents |
Introduction
The purpose of this tutorial is to explain the process of installing mpich2 (a popular open source implementation of MPI - Message Passing Interface) and quickly bring up a virtual cluster to run MPI jobs.
This version of the Grid appliance is tailored to educational applications, where it enables instructors and students to effortlessly create virtual clusters for individuals or small groups ready to run MPI jobs. Using the virtual cluster as a "sandbox" you can then follow MPI tutorials such as the CI-Tutor TeraGrid tutorial
Pre-requisites
1. Introduction to Grid-Appliance
2. Creating Grid-Appliance clusters
3. Grid Appliance On Science Clouds (for Science Cloud users)
Installing the MPI package and configuring the Virtual Cluster
1. Download a Grid Appliance (v 2.04 or higher). Alternatively, you can create your own Grid Appliance starting from a fresh Ubuntu image by following the steps described in TestingGridAppliance.
2. Follow instructions to create your own GroupVPN and appliance pool (Deploying independent appliance pools - PlanetLab).
3. Install the mpi package for grid-appliance by using the commands:
% sudo bash # apt-get update # apt-get install grid-appliance-mpi
4. A secret word is used to distinguish the nodes of your virtual cluster from other mpi nodes in the same virtual network. This secret word is saved in a file located at /etc/mpd.conf . It is advised to change the secret word from the default value ("DefaultSecretWord").
5. Repeat the above steps for all the nodes you want in the virtual cluster.
Starting mpd ring
At this instance, you should have your Grid-Appliance nodes running and they should have the mpi package installed. All the nodes should have acquired their GroupVPN IP and be configured with the same secret word. To run MPI jobs, you need to bring up a ring of deamon process called mpd on all nodes. Run the start_mpd_ring.sh script using the root user on one of the nodes. You do not have to run it on all nodes. This script scans for other mpi appliances in the same GroupVPN and boots the mpd ring. It will also replace the ssh keys used by the nodes in the virtual cluster with new ones, making it secure.
# /opt/mpi/scripts/start_mpd_ring.sh
- Note that this script configures the cluster for non-root users. Before running MPI jobs, you would have to create or configure users as mentioned below.
To stop the mpd ring, run the command:
# /opt/mpi/scripts/stop_mpd_ring.sh
User Management
The MPD ring (started by start_mpd_ring.sh) allows multiple users to run MPI jobs simultaneously. New users created after installing the mpi package will be able to use this MPD ring. Ensure that MPI users belong to the group "users". You can create a new user on all the nodes in the cluster, by using the runall.sh. For example, to create a user "mpiuser" with a password "password", run the below command:
# /opt/mpi/scripts/runall.sh "useradd -b /home -m -s /bin/bash -U --groups users,admin -p `echo password | openssl passwd -crypt -salt xx password` mpiuser"
You can allow an existing user to run MPI jobs too. For example:
#/opt/mpi/scripts/runall.sh "cp /opt/mpi/conf/.mpd.conf /home/griduser/; chown griduser:griduser /home/griduser/.mpd.conf"
Compiling and Running MPI programs
Some examples mpi codes from the tera-grid tutorial are included in /opt/mpi/teragrid_examples
In order to prepare your appliance to run these examples, you need to install the gcc compiler and the Grid appliance NFS packages:
# apt-get install gcc # apt-get install grid-appliance-nfs # apt-get install grid-appliance-autofs
To compile a mpi program use the command mpicc.
To run the mpi program, either copy the compiled executable to the working directory of all the nodes or run it from a shared location. If you have the auto-configured Network File System (NFS and autofs) packages of grid-appliance installed, then you can make directories/files at the submit node available (for reading and/or execution - it is a read-only file system) to other nodes in the virtual cluster. This is shown below:
# su - mpiuser # switch to a user capable of running MPI jobs $ mpdtrace #Verify the nodes in the mpd ring
$ cd /opt/mpi/teragrid_examples/ $ mpicc -o /mnt/local/HelloWorld HelloWorld.c $ mpiexec -np <No_of_processes> /mnt/ganfs/CXXXYYYZZZ/HelloWorld
where CXXXYYYZZZ is the hostname of the node having the compiled executable.
Helper scripts
runall.sh
This script can be used to run a command on all the MPI appliances listed in /opt/mpi/conf/mpd.hosts file.
pushall.sh
Use this script to send a file to all the nodes in the cluster listed in /opt/mpi/conf/mpd.hosts. Usage: "pushall.sh /path/to/filename"
Adding nodes to an existing virtual cluster
Configure the new nodes with the same secret word as in the existing nodes. Stop the mpd ring and run the start_mpd_ring.sh in one of the pre-existing nodes.
