Archer:MICRO42

From Grid-Appliance Wiki

Contents 1 Archer Tutorial at MICRO-42 1.1 At a glance 1.2 Abstract 2 Detailed information and program 2.1 Tutorial audience, goals, and duration 2.2 Setting up your laptop for the tutorial 2.3 Program outline 2.4 Tutorial requirements 2.5 Organizers 3 Acknowledgments

Archer Tutorial at MICRO-42

At a glance

• Where? - With MICRO-42, in New York City (Westin hotel at Times Square)
• When? - Dec. 12th 2009, 1-5pm
• Who? - The intended audience of the tutorial consists of computer architecture researchers, students (graduate and undergraduate), and educators.
• Why? - This hands-on tutorial will teach attendees how to use Archer, an open-source virtual appliance based system designed to help computer architects manage large numbers of simulations and sharing data in an effective way - running batch jobs on a shared infrastructure, on their own resources, or on cloud computing infrastructures.
• Contact info: Check out the main Archer page for more information, and our YouTube videos for a quick overview of Archer. Contact Renato Figueiredo (renato at acis dot ufl dot edu) if you have any questions about the tutorial; you can also join our Google and Facebook user groups.

Abstract

Many research groups are hindered in their ability to perform research because of lack of access to computational resources, tools and datasets. This hands-on tutorial presents Archer, a community-based computing resource for computer architecture research and education. Archer integrates technologies for resource virtualization (e.g. KVM, VMware, Xen, VirtualBox), a robust batch job scheduler (Condor), collaboration and data sharing tools (Wiki, and zero-configuration NFS distributed file systems), to create collaborative infrastructures within a local area or across wide-area resources. With support from the National Science Foundation, the Archer project has deployed a set of hundreds of processor cores across several universities - users are able to tap this shared infrastructure, or use our open-source appliances to deploy Archer pools of their own. The system runs Linux-based unmodified software simulation environments, including Simics, GEMS, FeS2, SESC, SimpleScalar, and PTLsim basic.

Detailed information and program

Tutorial audience, goals, and duration

This tutorial targets computer architecture researchers and students interested in accessing or deploying high-throughput computing resources to run architecture simulations. The tutorial will emphasize hands-on activities – attendees will install and run virtual machines on their own laptops to see how they can gain access and use a large-scale shared distributed computing infrastructure that is currently operational, as well as learn how they can deploy Archer software to manage their own local resource pools. This is a half-day tutorial, consisting of a brief technical introduction to the Archer software and infrastructure as well as hands-on, interactive activities where users can install and run the Archer virtual appliances on their own laptops.

Setting up your laptop for the tutorial

The instructor will bring USB sticks and CDs with these files to the tutorial, but it will speed things up if you do the bulk of download/install/registration prior to the tutorial. Here are the steps.

1. Download and install a virtual machine monitor, if you don't already use one. The simplest one to get started is VMware Player (Windows/Linux) or Fusion (MacOS). Player is free, Fusion has a 30-day evaluation program. You may also use VirtualBox, which is open source and free.
2. Download the Grid appliance image and save to your computer.
3. Create an account on the Grid appliance site; then:
   1. Log in to the Grid appliance site, click on "GroupVPN" (under "User menu", left side of the page) and request to join two groupVPNs (Archer and micro42tutorial)
      1. I'll need to approve this request - you will be notified by email when the requests are approved
   2. Once approved, click on "GroupAppliances" and request to join two appliance groups ("Archer Default" and micro42tutorial)

Program outline

1. Presentation - overview of core technologies in Archer (20 minutes)
   1. Contemporary virtual machine monitors, virtual appliances; virtual private networks
   2. Condor job scheduler
   3. Putting it all together: introduction to the Archer distributed system
2. Hands-on session – Archer appliance installation and user interface (30 minutes):
   1. Archer appliance installation
   2. Basic user interaction with the appliance: user interface, data transfers, “Hello world” job submission
3. Presentation – Archer virtual appliance internals (20 minutes):
   1. Virtual disk organization; software installation; creation of custom appliance modules
   2. Data sharing using NFS (Network File System)
4. Hands-on session – Archer usage in simulation case study: Simics (30 minutes):
   1. Running Simics in batch mode. This tutorial references a package that can extract pre-configured Condor files for a Simics run - you can also download these tutorial files here
   2. Running Simics in interactive mode (optional - may be too slow with the wireless network); Download simics-tutorial-files.zip
   3. Making datasets available via NFS
5. Presentation – Job management with Condor (20 minutes):
   1. Vanilla and standard jobs
   2. Job configuration files, specifying job requirements
   3. Introduction to workflows
   4. Job monitoring, priorities
6. Hands-on session (30 minutes)
   1. Compiling and running SESC with Condor
   2. Download sesc-tutorial-files.zip
   3. Workflow/DAG example
7. Presentation – Deploying Archer beyond a desktop: clusters, desktop pools, cloud resources (30 minutes):
   1. Joining Archer Global with multiple machines - deployment on clusters
   2. Creating and managing user/priority groups
   3. Deploying private Archer Local pools; Web front-end appliance for group management
   4. Path for deployment on cloud resources (e.g. Amazon EC2, FutureGrid)
8. Demonstration session (20 minutes)
   1. This session will guide attendees through a demonstration of creating a private Archer pool.

Tutorial requirements

We recommend tutorial participants to bring a laptop if they wish to follow the hands-on component of the tutorial. A laptop with x86-based processor, 1GB RAM, Windows, MacOS or Linux is sufficient, and we only expect basic skills with Unix command-line interaction to follow the hands-on tutorial.

Instructions will be given in this page prior to the conference for those who want to pre-install the software. The presenters will also bring CDs and USB drives for on-site installation.

Organizers

The main organizer of this tutorial is Renato Figueiredo (University of Florida), in collaboration with the Archer project PIs: Jih-Kwon Peir, Jose A. B. Fortes, Tao Li, P. Oscar Boykin (U. of Florida), Gary Tyson (Florida State U.), Lizy K. John (U. of Texas at Austin), David Kaeli (Northeastern U.), David Lilja (U. Minnesota), Gokhan Memik (Northwestern U.), Sally McKee (Chalmers), Alain Roy (U. of Wisconsin).

Presenter bio: Renato Figueiredo is an Associate Professor with the Advanced Computing and Information Systems (ACIS) Laboratory and the Center for Autonomic Computing at the University of Florida. Figueiredo is the PI of the Archer project, a virtualized shared wide-area infrastructure for computer architecture research and education. In addition to Archer, he has participated in various research projects which apply machine, network and file system virtualization in support of high-performance and high-throughput computing in wide-area distributed systems. His broad research interests are in the areas of resource virtualization, computer architecture, distributed systems, network overlays, and autonomic computing.

Acknowledgments

• Government sponsors

This work is sponsored by the National Science Foundation under CRI collaborative awards 0751112, 0750847, 0750851, 0750852, 0750860, 0750868, 0750884, and 0751091.

• Industry sponsors

The Archer project gratefully acknowledges an academic site license donation for Simics from Virtutech.