FutureGrid 0.1
  • Site
      • 1. INTRODUCTION
        • 1.1. Preface
          • 1.1.1. Citation for Publications
          • 1.1.2. Acknowledgement
          • 1.1.3. Sponsors
          • 1.1.4. About this Manual
          • 1.1.5. Conventions
        • 1. Overview
          • 1.1. Executive Summary
          • 1.2. Project and Account Application
          • 1.3. Services
          • 1.4. Hardware
          • 1.5. Support
      • 2. ACCOUNTS AND PROJECTS
        • 2.1. Project and Account Management
          • 2.1.1. Terminology
          • 2.1.2. Quickstart
          • 2.1.3. Projects and Accounts for XSEDE users
          • 2.1.4. Project Management
            • 2.1.4.1. Create a Portal account
            • 2.1.4.2. Create a Project
              • 2.1.4.2.1. Example Project
            • 2.1.4.3. Join a Project
            • 2.1.4.4. Delete or Deactivate a user from a Project
            • 2.1.4.5. Reporting Results
            • 2.1.4.6. Close a Project
          • 2.1.5. Upload a SSH Public Key
          • 2.1.6. Upload an OpenId
          • 2.1.7. Accessing FutureGrid Resources
          • 2.1.8. Manage a Class on FutureGrid
          • 2.1.9. Mini FAQ
        • 2.2. Using SSH Keys
          • 2.2.1. Using SSH from Windows
          • 2.2.2. Generate a SSH key
          • 2.2.3. Add or Replace Passphrase for an Already Generated Key
          • 2.2.4. Upload the key to the FutureGrid Portal
          • 2.2.5. Testing your ssh key
          • 2.2.6. Testing your ssh key for Hotel
      • 3. STATUS
        • 3.1. Status
      • 4. HPC SERVICES
        • 4.1. Hardware
          • 4.1.1. Compute Resources
          • 4.1.2. Networks
          • 4.1.3. Network Impairments Device (NID)
        • 4.2. HPC Services
          • 4.2.1. Login Nodes
            • 4.2.1.1. SSH Add
            • 4.2.1.2. SSH Config
            • 4.2.1.3. Modules
            • 4.2.1.4. List of Available Modules on Various Machines
            • 4.2.1.5. Filesystem Layout
          • 4.2.2. Message Passing Interface (MPI)
            • 4.2.2.1. MPI Libraries
            • 4.2.2.2. Compiling MPI Applications
            • 4.2.2.3. Batch Jobs
          • 4.2.3. Job Management
            • 4.2.3.1. Job Submission
            • 4.2.3.2. Job Deletion
            • 4.2.3.3. Job Monitoring
            • 4.2.3.4. Job Output
          • 4.2.4. Xray HPC Services
            • 4.2.4.1. Submitting a Job on xray
        • 4.3. ScaleMP vSMP
          • 4.3.1. Accessing ScaleMP
          • 4.3.2. Submitting a job
          • 4.3.3. Developing a job script
            • 4.3.3.1. MPI
            • 4.3.3.2. OpenMP
            • 4.3.3.3. Threaded
            • 4.3.3.4. Throughput
            • 4.3.3.5. Serial
        • 4.4. Genesis II
          • 4.4.1. Introduction
          • 4.4.2. What is GenesisII
          • 4.4.3. Connecting to the GenesisII BES Endpoints
            • 4.4.3.1. Supported Data Staging Protocols
              • 4.4.3.1.1. Stage in
              • 4.4.3.1.2. Stage out
              • 4.4.3.1.3. Endpoint Connection Information
            • 4.4.3.2. India
            • 4.4.3.3. Sierra
              • 4.4.3.3.1. Alamo
              • 4.4.3.3.2. Hotel
          • 4.4.4. Using the Futuregrid GenesisII Endpoints as a Client
            • 4.4.4.1. Non-GenesisII Using a Standards-Compliant Client
            • 4.4.4.2. Using The GenesisII Client
              • 4.4.4.2.1. Acquiring GenesisII Client Package
                • 4.4.4.2.1.1. Linux
                • 4.4.4.2.1.2. Windows
                • 4.4.4.2.1.3. MacOS
              • 4.4.4.2.2. Installing the GenesisII/XCG Client Package
            • 4.4.4.3. Getting Started
              • 4.4.4.3.1. Start Grid Shell
                • 4.4.4.3.1.1. Windows
                • 4.4.4.3.1.2. Linux or MacOS
              • 4.4.4.3.2. Running Jobs
            • 4.4.4.4. Getting Help
        • 4.5. Unicore
          • 4.5.1. UNICORE 6 on FutureGrid User Manual
          • 4.5.2. Introduction
          • 4.5.3. What is UNICORE?
          • 4.5.4. Connecting to the UNICORE BES Endpoints From Other Grid Middleware Clients
            • 4.5.4.1. India Endpoint Info <currently unavailable>
            • 4.5.4.2. Sierra Endpoint Info
          • 4.5.5. Connecting to the UNICORE BES Endpoints Using a UNICORE Commandline Client
            • 4.5.5.1. Installing the UNICORE 6 Commandline Client (UCC)
              • 4.5.5.1.1. Acquire Client Bundle
              • 4.5.5.1.2. Unpack UCC
              • 4.5.5.1.3. Examine UCC Files
              • 4.5.5.1.4. Run Commandline Client
              • 4.5.5.1.5. Installation Conclusion
            • 4.5.5.2. Configuring Client to Connect to FutureGrid U6 Endpoints
              • 4.5.5.2.1. Configuration Overview
            • 4.5.5.3. Setting Up Security in UNICORE
            • 4.5.5.4. Setting Up a Keystore
              • 4.5.5.4.1. Creating a Truststore
              • 4.5.5.4.2. Acquiring the Registry Address
              • 4.5.5.4.3. Connecting to Endpoints w/o Registries
              • 4.5.5.4.4. Preferences File Modifications
              • 4.5.5.4.5. Example Preferences File
              • 4.5.5.4.6. Locating the Preferences File
              • 4.5.5.4.7. Validate Client Setup
              • 4.5.5.4.8. Configuration Conclusion
            • 4.5.5.5. Submitting Jobs to FutureGrid U6 Endpoints
              • 4.5.5.5.1. Getting Started
              • 4.5.5.5.2. Submit Jobs to BES
              • 4.5.5.5.3. Other BES Related Commands
              • 4.5.5.5.4. Job Submission Conclusion
            • 4.5.5.6. References
            • 4.5.5.7. Questions/Comments
          • 4.5.6. Running Jobs on UNICORE Sites
          • 4.5.7. Deploying a New UNICORE 6 Grid
            • 4.5.7.1. Prerequisites
            • 4.5.7.2. Installing the Core Server Bundle
            • 4.5.7.3. Starting/Stopping the UNICORE Servers
      • 5. IAAS/CLOUD SERVICES
        • 5.1. IaaS
          • 5.1.1. Nimbus Clouds
          • 5.1.2. OpenStack Clouds
          • 5.1.3. Eucalyptus Clouds
          • 5.1.4. Virtual Appliances for Training and Education
        • 5.2. OpenStack Havana
          • 5.2.1. Login
          • 5.2.2. Setup OpenStack Environment
          • 5.2.3. Creating the novarc file
          • 5.2.4. List flavors
          • 5.2.5. List images
          • 5.2.6. Key management
          • 5.2.7. Managing security groups
          • 5.2.8. Booting an image
          • 5.2.9. List running images
          • 5.2.10. Use block storage
          • 5.2.11. Set up external access to your instance
          • 5.2.12. Make a snapshot of an instance
          • 5.2.13. Automate some initial configuration
          • 5.2.14. Get the latest version of Ubuntu Cloud Image and upload it to the OpenStack
          • 5.2.15. Delete your instance
          • 5.2.16. How to change your password
          • 5.2.17. Things to do when you need Euca2ools or EC2 interfaces
          • 5.2.18. Horizon GUI
          • 5.2.19. Getting rc files via Horizon
        • 5.3. OpenStack Grizzly
          • 5.3.1. Login
          • 5.3.2. Creating the novarc file
          • 5.3.3. Activate OpenStack tools
          • 5.3.4. List flavors
          • 5.3.5. List images
          • 5.3.6. Standard Images
          • 5.3.7. Key management
          • 5.3.8. Managing security groups
          • 5.3.9. Booting an image
          • 5.3.10. List running images
          • 5.3.11. Use block storage
          • 5.3.12. Set up external access to your instance
          • 5.3.13. Make a snapshot of an instance
          • 5.3.14. Automate some initial configuration
          • 5.3.15. Get the latest version of Ubuntu Cloud Image and upload it to the OpenStack
          • 5.3.16. Delete your instance
          • 5.3.17. How to change your password
            • 5.3.17.1. Things to do when you need Euca2ools or EC2 interfaces
          • 5.3.18. Horizon GUI
            • 5.3.18.1. Getting rc files via Horizon
        • 5.4. Openstack Horizon
          • 5.4.1. Managing Virtual Machine Instances
          • 5.4.2. Snapshots
          • 5.4.3. Floating IP Addresses
          • 5.4.4. Accessing Your Virtual Machine
          • 5.4.5. Uploading an Image
          • 5.4.6. Managing Volumes
          • 5.4.7. EC2 Interface
          • 5.4.8. OpenStack REST Interfaces
        • 5.5. Converting Virtualbox images to OpenStack
          • 5.5.1. Prerequisites
            • 5.5.1.1. Disable SELinux (Only for RedHat-based Linux like CentOS)
            • 5.5.1.2. Configuring the image network interface (eth0) for DHCP
            • 5.5.1.3. Configure the image to allow OpenStack to inject the ssh key
            • 5.5.1.4. Configure udev persistent rules (only CentOS)
          • 5.5.2. Convert your virtual box image to raw format
          • 5.5.3. Convert the image to qcow2 format (optional)
          • 5.5.4. Test your image
          • 5.5.5. Transfer your Image to India
          • 5.5.6. Log into India
          • 5.5.7. Upload your image to OpenStack
          • 5.5.8. Checking Status Image
          • 5.5.9. Test Image in OpenStack
          • 5.5.10. Troubleshooting
          • 5.5.11. Notes:
        • 5.6. Nimbus
          • 5.6.1. Nimbus IaaS
            • 5.6.1.1. What is Nimbus?
            • 5.6.1.2. Nimbus on FutureGrid
            • 5.6.1.3. Getting Started
              • 5.6.1.3.1. Log into hotel
              • 5.6.1.3.2. Download and install cloud-client
              • 5.6.1.3.3. Obtain Your Nimbus Credentials and Configuration Files
              • 5.6.1.3.4. Check Your ssh Key
            • 5.6.1.4. Using the Cloud Client
              • 5.6.1.4.1. Check out the various FutureGrid clouds
              • 5.6.1.4.2. Run a Virtual Machine
              • 5.6.1.4.3. Create a New VM Image
              • 5.6.1.4.4. Save the Changes to a New VM
              • 5.6.1.4.5. Launch Your New VM
              • 5.6.1.4.6. Terminate the VM
              • 5.6.1.4.7. Uploading and downloading images
            • 5.6.1.5. Converting a Xen image to KVM
            • 5.6.1.6. Virtual Clusters
              • 5.6.1.6.1. Cluster Definition File
              • 5.6.1.6.2. Start the Cluster
              • 5.6.1.6.3. Check Out the Virtual Cluster
          • 5.6.2. Launch A VM via Nimbus
            • 5.6.2.1. Options
            • 5.6.2.2. What Happens
          • 5.6.3. cloudinit.d
          • 5.6.4. Phantom
        • 5.7. Eucalyptus
          • 5.7.1. Account Setup
          • 5.7.2. Resources Overview
          • 5.7.3. Testing Your Setup
          • 5.7.4. Available Images
          • 5.7.5. Image Deployment
          • 5.7.6. Logging Into the VM
          • 5.7.7. Shutting down the VM
          • 5.7.8. VM Network Info
          • 5.7.9. Image Management
          • 5.7.10. Volume Management
          • 5.7.11. Status of Deployments
      • 6. PAAS/CLOUD SERVICES
        • 6.1. Using Hadoop in FutureGrid
          • 6.1.1. Running Hadoop as a Batch Job using MyHadoop
            • 6.1.1.1. myHadoop on FutureGrid
              • 6.1.1.1.1. Login into a machine tha has myHadoop installed
              • 6.1.1.1.2. Load the needed modules
              • 6.1.1.1.3. Run the Example
              • 6.1.1.1.4. Details of the Script
              • 6.1.1.1.5. Submission of the Hadoop job
              • 6.1.1.1.6. Persistent Mode
              • 6.1.1.1.7. Customizing Hadoop Settings
              • 6.1.1.1.8. Using a Different Installation of Hadoop
              • 6.1.1.1.9. References
        • 6.2. Using Twister in FutureGrid
          • 6.2.1. What is Twister?
          • 6.2.2. Running Twister on FutureGrid
          • 6.2.3. Run Twister Applications
          • 6.2.4. Papers and Presentations
      • 7. EXPERIMENT MANAGEMENT SERVICES
        • 7.1. Interactive Experiment Management
          • 7.1.1. Host List Manager
            • 7.1.1.1. Host List Manager Setup
            • 7.1.1.2. Provisioning Hosts
            • 7.1.1.3. Managing Host Lists
          • 7.1.2. TakTuk
            • 7.1.2.1. Running Experiments with TakTuk
            • 7.1.2.2. Host List Manager Commands
              • 7.1.2.2.1. hupdate
              • 7.1.2.2.2. hmanage
              • 7.1.2.2.3. hshow
              • 7.1.2.2.4. htag
              • 7.1.2.2.5. hlist
              • 7.1.2.2.6. htaktuk
            • 7.1.2.3. TakTuk Commands
        • 7.2. Precip
          • 7.2.1. Installation
          • 7.2.2. API
          • 7.2.3. Examples
            • 7.2.3.1. Hello World
            • 7.2.3.2. Resources from mulitple infrastructures
            • 7.2.3.3. Setting up a Condor pool and running a Pegasus workflow
        • 7.3. Pegasus
          • 7.3.1. The Pegasus Run-Time Cloud Architecture
          • 7.3.2. Using Pegasus on FutureGrid
        • 7.4. RAIN
          • 7.4.1. Generate and Register an OS Image on FutureGrid using the FG Shell
            • 7.4.1.1. Request access
            • 7.4.1.2. Log into India
            • 7.4.1.3. Start the Shell
            • 7.4.1.4. Generate the Image
            • 7.4.1.5. Image Repository
            • 7.4.1.6. Register Image
            • 7.4.1.7. Start Image
          • 7.4.2. FutureGrid Standalone Image Repository
            • 7.4.2.1. Requirement
            • 7.4.2.2. Software
            • 7.4.2.3. Documentation
              • 7.4.2.3.1. User Manual
              • 7.4.2.3.2. Installation
              • 7.4.2.3.3. Configuration
          • 7.4.3. Manual Image Customization
            • 7.4.3.1. Logging into India
            • 7.4.3.2. Requesting access
            • 7.4.3.3. Obtaining the image
              • 7.4.3.3.1. Getting an image from the repository
              • 7.4.3.3.2. Generating a new image
            • 7.4.3.4. Customizing the image
            • 7.4.3.5. Transfer the image back to India
            • 7.4.3.6. Log into India
            • 7.4.3.7. Upload the image to the repository
            • 7.4.3.8. Register your image in different infrastructures
              • 7.4.3.8.1. Register the image in Openstack
              • 7.4.3.8.2. Register the image in Eucalyptus
              • 7.4.3.8.3. Register the image in HPC
            • 7.4.3.9. Using your Registered Image
          • 7.4.4. RAIN Manual Pages
            • 7.4.4.1. fg-portal-manage
      • 8. PERFORMANCE TOOLS
        • 8.1. PAPI
          • 8.1.1. Availability
          • 8.1.2. References
      • 9. TUTORIALS
        • 2. Tutorials
          • 2.1. Tutorial Topic 0: Accessing FutureGrid Resources
          • 2.2. Tutorial Topic 1: Cloud Provisioning Platforms
          • 2.3. Tutorial Topic 2: Cloud Run-time Map/Reduce Platforms
          • 2.4. Tutorial Topic 3: Grid Appliances for Training, Education, and Outreach
          • 2.5. Tutorial Topic 4: High Performance Computing
          • 2.6. Tutorial Topic 5: Experiment Management
          • 2.7. Tutorial Topic 6: Image Management and Rain
          • 2.8. Tutorial Topic 7: Storage
          • 2.9. Other Tutorials and Educational Materials
      • 10. APPENDIX. SCREENCASTS
        • 10.1. Screencast: Using FutureGrid
          • 10.1.1. Overview
          • 10.1.2. Prerequisites
          • 10.1.3. Instructor Bio
          • 10.1.4. Course Content
            • 10.1.4.1. Creating FutureGrid Accounts and Projects
            • 10.1.4.2. OpenStack on FutureGrid
            • 10.1.4.3. Eucalyptus on FutureGrid
            • 10.1.4.4. Batch Jobs including Hadoop on FutureGrid
          • 10.1.5. Exercises
            • 10.1.5.1. Creating FutureGrid Accounts and Projects
      • 11. APPENDIX. MAINTAINING THIS MANUAL
        • 11.1. Git
        • 11.2. Contributing to the Manual
          • 11.2.1. Organisation of the Manual in Git
          • 11.2.2. Online Editing for Novice Editors
          • 11.2.3. FAQ
          • 11.2.4. Publication of the Manual
        • 11.3. Setting Up the Expert Developers Manual Editing Environment
          • 11.3.1. Python
          • 11.3.2. Virtualenv
          • 11.3.3. Create a github local directory with the manual
          • 11.3.4. Install the Requirements
          • 11.3.5. All-in-one setup script
          • 11.3.6. Watchdog
          • 11.3.7. Transfering a page from the portal to RST
          • 11.3.8. Creating the pages locally
          • 11.3.9. Publishing the pages
          • 11.3.10. Jira
          • 11.3.11. Portal link
          • 11.3.12. Screencast recording tips
            • 11.3.12.1. OSX
            • 11.3.12.2. Changing the prompt
            • 11.3.12.3. Recording of typed in commands
            • 11.3.12.4. Setup quicktime sound recording
      • 12. APPENDIX. MPI RING PROGRAM
        • 12.1. MPI Ring Program
      • 13. APPENDIX. OTHER SECTIONS THAT HAVE NOT YET BEEN REVIEWED OR ARE INTEGRATED IN THE MANUAL
        • 3. Delta GPU User Manual
          • 3.1. Running MPI/GPU program on the Delta cluster
          • 3.2. Running programs on a single GPU
          • 3.3. C-means clustering using CUDA on GPU
        • 13.2. IPOP (IP-over-P2P)
          • 13.2.1. Prerequisites:
          • 13.2.2. Step 1: Deploying VMs to be connected by IPOP
            • 13.2.2.1. Option 1: Deploying a VM on FutureGrid Using Nimbus
            • 13.2.2.2. Option 2: Deploying a VM on FutureGrid Using OpenStack
            • 13.2.2.3. Option 3: Deploying a VM on Amazon EC2
            • 13.2.2.4. Option 4: Deploying a VM on your computer (VMware, VirtualBox)
          • 13.2.3. Step 2: Establishing IPOP connectivity between two VMs
          • 13.2.4. Step 3: Securing communications between Alice and Bob
          • 13.2.5. Troubleshooting and for more information
        • 13.3. IPOP2 (IP-over-P2P)
          • 13.3.1. Prerequisites:
          • 13.3.2. Step 1: Bringing up Alice and Bob VMs
          • 13.3.3. Step 2: Disconnecting Alice and Bob VMs from the PlanetLab P2P overlay
          • 13.3.4. Step 3: Deploy Pam, a P2P Bootstrapping VM
          • 13.3.5. Step 4: Configure Pam to run P2P Bootstrapping Overlay
          • 13.3.6. Step 5: Configure Alice and Bob to use Pam as P2P overlay
          • 13.3.7. Step 6: Send messages between Alice and Bob
          • 13.3.8. 7.1) Inspecting the P2P network
          • 13.3.9. 7.2) Adding fault tolerance by deploying multiple bootstrap nodes
        • 13.4. GA1
          • 13.4.1. Prerequisites:
          • 13.4.2. Hands-on Tutorial:
          • 13.4.3. Reference Material:
          • 13.4.4. Authors:
        • 13.5. GA9
          • 13.5.1. Summary:
          • 13.5.2. Prerequisites:
            • 13.5.2.1. Deploying a Grid Appliance on FutureGrid
        • 13.6. HPSS
        • 13.7. SSH the wrong way ( DO NOT USE THIS TUTORIAL)
          • 13.7.1. Requirement for Windows Users
          • 13.7.2. Instructions for both Windows and Unix users
            • 13.7.2.1. Generate SSH key
            • 13.7.2.2. Check your ssh key
            • 13.7.2.3. Copy the content of your public key
            • 13.7.2.4. Upload the key to the FutureGrid Portal
            • 13.7.2.5. Testing your ssh key
            • 13.7.2.6. Testing your ssh key on Hotel
      • 1. INTRODUCTION
        • 1.1. Preface
          • 1.1.1. Citation for Publications
          • 1.1.2. Acknowledgement
          • 1.1.3. Sponsors
          • 1.1.4. About this Manual
          • 1.1.5. Conventions
        • 1. Overview
          • 1.1. Executive Summary
          • 1.2. Project and Account Application
          • 1.3. Services
          • 1.4. Hardware
          • 1.5. Support
      • 2. ACCOUNTS AND PROJECTS
        • 2.1. Project and Account Management
          • 2.1.1. Terminology
          • 2.1.2. Quickstart
          • 2.1.3. Projects and Accounts for XSEDE users
          • 2.1.4. Project Management
            • 2.1.4.1. Create a Portal account
            • 2.1.4.2. Create a Project
              • 2.1.4.2.1. Example Project
            • 2.1.4.3. Join a Project
            • 2.1.4.4. Delete or Deactivate a user from a Project
            • 2.1.4.5. Reporting Results
            • 2.1.4.6. Close a Project
          • 2.1.5. Upload a SSH Public Key
          • 2.1.6. Upload an OpenId
          • 2.1.7. Accessing FutureGrid Resources
          • 2.1.8. Manage a Class on FutureGrid
          • 2.1.9. Mini FAQ
        • 2.2. Using SSH Keys
          • 2.2.1. Using SSH from Windows
          • 2.2.2. Generate a SSH key
          • 2.2.3. Add or Replace Passphrase for an Already Generated Key
          • 2.2.4. Upload the key to the FutureGrid Portal
          • 2.2.5. Testing your ssh key
          • 2.2.6. Testing your ssh key for Hotel
      • 3. STATUS
        • 3.1. Status
      • 4. HPC SERVICES
        • 4.1. Hardware
          • 4.1.1. Compute Resources
          • 4.1.2. Networks
          • 4.1.3. Network Impairments Device (NID)
        • 4.2. HPC Services
          • 4.2.1. Login Nodes
            • 4.2.1.1. SSH Add
            • 4.2.1.2. SSH Config
            • 4.2.1.3. Modules
            • 4.2.1.4. List of Available Modules on Various Machines
            • 4.2.1.5. Filesystem Layout
          • 4.2.2. Message Passing Interface (MPI)
            • 4.2.2.1. MPI Libraries
            • 4.2.2.2. Compiling MPI Applications
            • 4.2.2.3. Batch Jobs
          • 4.2.3. Job Management
            • 4.2.3.1. Job Submission
            • 4.2.3.2. Job Deletion
            • 4.2.3.3. Job Monitoring
            • 4.2.3.4. Job Output
          • 4.2.4. Xray HPC Services
            • 4.2.4.1. Submitting a Job on xray
        • 4.3. ScaleMP vSMP
          • 4.3.1. Accessing ScaleMP
          • 4.3.2. Submitting a job
          • 4.3.3. Developing a job script
            • 4.3.3.1. MPI
            • 4.3.3.2. OpenMP
            • 4.3.3.3. Threaded
            • 4.3.3.4. Throughput
            • 4.3.3.5. Serial
        • 4.4. Genesis II
          • 4.4.1. Introduction
          • 4.4.2. What is GenesisII
          • 4.4.3. Connecting to the GenesisII BES Endpoints
            • 4.4.3.1. Supported Data Staging Protocols
              • 4.4.3.1.1. Stage in
              • 4.4.3.1.2. Stage out
              • 4.4.3.1.3. Endpoint Connection Information
            • 4.4.3.2. India
            • 4.4.3.3. Sierra
              • 4.4.3.3.1. Alamo
              • 4.4.3.3.2. Hotel
          • 4.4.4. Using the Futuregrid GenesisII Endpoints as a Client
            • 4.4.4.1. Non-GenesisII Using a Standards-Compliant Client
            • 4.4.4.2. Using The GenesisII Client
              • 4.4.4.2.1. Acquiring GenesisII Client Package
                • 4.4.4.2.1.1. Linux
                • 4.4.4.2.1.2. Windows
                • 4.4.4.2.1.3. MacOS
              • 4.4.4.2.2. Installing the GenesisII/XCG Client Package
            • 4.4.4.3. Getting Started
              • 4.4.4.3.1. Start Grid Shell
                • 4.4.4.3.1.1. Windows
                • 4.4.4.3.1.2. Linux or MacOS
              • 4.4.4.3.2. Running Jobs
            • 4.4.4.4. Getting Help
        • 4.5. Unicore
          • 4.5.1. UNICORE 6 on FutureGrid User Manual
          • 4.5.2. Introduction
          • 4.5.3. What is UNICORE?
          • 4.5.4. Connecting to the UNICORE BES Endpoints From Other Grid Middleware Clients
            • 4.5.4.1. India Endpoint Info <currently unavailable>
            • 4.5.4.2. Sierra Endpoint Info
          • 4.5.5. Connecting to the UNICORE BES Endpoints Using a UNICORE Commandline Client
            • 4.5.5.1. Installing the UNICORE 6 Commandline Client (UCC)
              • 4.5.5.1.1. Acquire Client Bundle
              • 4.5.5.1.2. Unpack UCC
              • 4.5.5.1.3. Examine UCC Files
              • 4.5.5.1.4. Run Commandline Client
              • 4.5.5.1.5. Installation Conclusion
            • 4.5.5.2. Configuring Client to Connect to FutureGrid U6 Endpoints
              • 4.5.5.2.1. Configuration Overview
            • 4.5.5.3. Setting Up Security in UNICORE
            • 4.5.5.4. Setting Up a Keystore
              • 4.5.5.4.1. Creating a Truststore
              • 4.5.5.4.2. Acquiring the Registry Address
              • 4.5.5.4.3. Connecting to Endpoints w/o Registries
              • 4.5.5.4.4. Preferences File Modifications
              • 4.5.5.4.5. Example Preferences File
              • 4.5.5.4.6. Locating the Preferences File
              • 4.5.5.4.7. Validate Client Setup
              • 4.5.5.4.8. Configuration Conclusion
            • 4.5.5.5. Submitting Jobs to FutureGrid U6 Endpoints
              • 4.5.5.5.1. Getting Started
              • 4.5.5.5.2. Submit Jobs to BES
              • 4.5.5.5.3. Other BES Related Commands
              • 4.5.5.5.4. Job Submission Conclusion
            • 4.5.5.6. References
            • 4.5.5.7. Questions/Comments
          • 4.5.6. Running Jobs on UNICORE Sites
          • 4.5.7. Deploying a New UNICORE 6 Grid
            • 4.5.7.1. Prerequisites
            • 4.5.7.2. Installing the Core Server Bundle
            • 4.5.7.3. Starting/Stopping the UNICORE Servers
      • 5. IAAS/CLOUD SERVICES
        • 5.1. IaaS
          • 5.1.1. Nimbus Clouds
          • 5.1.2. OpenStack Clouds
          • 5.1.3. Eucalyptus Clouds
          • 5.1.4. Virtual Appliances for Training and Education
        • 5.2. OpenStack Havana
          • 5.2.1. Login
          • 5.2.2. Setup OpenStack Environment
          • 5.2.3. Creating the novarc file
          • 5.2.4. List flavors
          • 5.2.5. List images
          • 5.2.6. Key management
          • 5.2.7. Managing security groups
          • 5.2.8. Booting an image
          • 5.2.9. List running images
          • 5.2.10. Use block storage
          • 5.2.11. Set up external access to your instance
          • 5.2.12. Make a snapshot of an instance
          • 5.2.13. Automate some initial configuration
          • 5.2.14. Get the latest version of Ubuntu Cloud Image and upload it to the OpenStack
          • 5.2.15. Delete your instance
          • 5.2.16. How to change your password
          • 5.2.17. Things to do when you need Euca2ools or EC2 interfaces
          • 5.2.18. Horizon GUI
          • 5.2.19. Getting rc files via Horizon
        • 5.3. OpenStack Grizzly
          • 5.3.1. Login
          • 5.3.2. Creating the novarc file
          • 5.3.3. Activate OpenStack tools
          • 5.3.4. List flavors
          • 5.3.5. List images
          • 5.3.6. Standard Images
          • 5.3.7. Key management
          • 5.3.8. Managing security groups
          • 5.3.9. Booting an image
          • 5.3.10. List running images
          • 5.3.11. Use block storage
          • 5.3.12. Set up external access to your instance
          • 5.3.13. Make a snapshot of an instance
          • 5.3.14. Automate some initial configuration
          • 5.3.15. Get the latest version of Ubuntu Cloud Image and upload it to the OpenStack
          • 5.3.16. Delete your instance
          • 5.3.17. How to change your password
            • 5.3.17.1. Things to do when you need Euca2ools or EC2 interfaces
          • 5.3.18. Horizon GUI
            • 5.3.18.1. Getting rc files via Horizon
        • 5.4. Openstack Horizon
          • 5.4.1. Managing Virtual Machine Instances
          • 5.4.2. Snapshots
          • 5.4.3. Floating IP Addresses
          • 5.4.4. Accessing Your Virtual Machine
          • 5.4.5. Uploading an Image
          • 5.4.6. Managing Volumes
          • 5.4.7. EC2 Interface
          • 5.4.8. OpenStack REST Interfaces
        • 5.5. Converting Virtualbox images to OpenStack
          • 5.5.1. Prerequisites
            • 5.5.1.1. Disable SELinux (Only for RedHat-based Linux like CentOS)
            • 5.5.1.2. Configuring the image network interface (eth0) for DHCP
            • 5.5.1.3. Configure the image to allow OpenStack to inject the ssh key
            • 5.5.1.4. Configure udev persistent rules (only CentOS)
          • 5.5.2. Convert your virtual box image to raw format
          • 5.5.3. Convert the image to qcow2 format (optional)
          • 5.5.4. Test your image
          • 5.5.5. Transfer your Image to India
          • 5.5.6. Log into India
          • 5.5.7. Upload your image to OpenStack
          • 5.5.8. Checking Status Image
          • 5.5.9. Test Image in OpenStack
          • 5.5.10. Troubleshooting
          • 5.5.11. Notes:
        • 5.6. Nimbus
          • 5.6.1. Nimbus IaaS
            • 5.6.1.1. What is Nimbus?
            • 5.6.1.2. Nimbus on FutureGrid
            • 5.6.1.3. Getting Started
              • 5.6.1.3.1. Log into hotel
              • 5.6.1.3.2. Download and install cloud-client
              • 5.6.1.3.3. Obtain Your Nimbus Credentials and Configuration Files
              • 5.6.1.3.4. Check Your ssh Key
            • 5.6.1.4. Using the Cloud Client
              • 5.6.1.4.1. Check out the various FutureGrid clouds
              • 5.6.1.4.2. Run a Virtual Machine
              • 5.6.1.4.3. Create a New VM Image
              • 5.6.1.4.4. Save the Changes to a New VM
              • 5.6.1.4.5. Launch Your New VM
              • 5.6.1.4.6. Terminate the VM
              • 5.6.1.4.7. Uploading and downloading images
            • 5.6.1.5. Converting a Xen image to KVM
            • 5.6.1.6. Virtual Clusters
              • 5.6.1.6.1. Cluster Definition File
              • 5.6.1.6.2. Start the Cluster
              • 5.6.1.6.3. Check Out the Virtual Cluster
          • 5.6.2. Launch A VM via Nimbus
            • 5.6.2.1. Options
            • 5.6.2.2. What Happens
          • 5.6.3. cloudinit.d
          • 5.6.4. Phantom
        • 5.7. Eucalyptus
          • 5.7.1. Account Setup
          • 5.7.2. Resources Overview
          • 5.7.3. Testing Your Setup
          • 5.7.4. Available Images
          • 5.7.5. Image Deployment
          • 5.7.6. Logging Into the VM
          • 5.7.7. Shutting down the VM
          • 5.7.8. VM Network Info
          • 5.7.9. Image Management
          • 5.7.10. Volume Management
          • 5.7.11. Status of Deployments
      • 6. PAAS/CLOUD SERVICES
        • 6.1. Using Hadoop in FutureGrid
          • 6.1.1. Running Hadoop as a Batch Job using MyHadoop
            • 6.1.1.1. myHadoop on FutureGrid
              • 6.1.1.1.1. Login into a machine tha has myHadoop installed
              • 6.1.1.1.2. Load the needed modules
              • 6.1.1.1.3. Run the Example
              • 6.1.1.1.4. Details of the Script
              • 6.1.1.1.5. Submission of the Hadoop job
              • 6.1.1.1.6. Persistent Mode
              • 6.1.1.1.7. Customizing Hadoop Settings
              • 6.1.1.1.8. Using a Different Installation of Hadoop
              • 6.1.1.1.9. References
        • 6.2. Using Twister in FutureGrid
          • 6.2.1. What is Twister?
          • 6.2.2. Running Twister on FutureGrid
          • 6.2.3. Run Twister Applications
          • 6.2.4. Papers and Presentations
      • 7. EXPERIMENT MANAGEMENT SERVICES
        • 7.1. Interactive Experiment Management
          • 7.1.1. Host List Manager
            • 7.1.1.1. Host List Manager Setup
            • 7.1.1.2. Provisioning Hosts
            • 7.1.1.3. Managing Host Lists
          • 7.1.2. TakTuk
            • 7.1.2.1. Running Experiments with TakTuk
            • 7.1.2.2. Host List Manager Commands
              • 7.1.2.2.1. hupdate
              • 7.1.2.2.2. hmanage
              • 7.1.2.2.3. hshow
              • 7.1.2.2.4. htag
              • 7.1.2.2.5. hlist
              • 7.1.2.2.6. htaktuk
            • 7.1.2.3. TakTuk Commands
        • 7.2. Precip
          • 7.2.1. Installation
          • 7.2.2. API
          • 7.2.3. Examples
            • 7.2.3.1. Hello World
            • 7.2.3.2. Resources from mulitple infrastructures
            • 7.2.3.3. Setting up a Condor pool and running a Pegasus workflow
        • 7.3. Pegasus
          • 7.3.1. The Pegasus Run-Time Cloud Architecture
          • 7.3.2. Using Pegasus on FutureGrid
        • 7.4. RAIN
          • 7.4.1. Generate and Register an OS Image on FutureGrid using the FG Shell
            • 7.4.1.1. Request access
            • 7.4.1.2. Log into India
            • 7.4.1.3. Start the Shell
            • 7.4.1.4. Generate the Image
            • 7.4.1.5. Image Repository
            • 7.4.1.6. Register Image
            • 7.4.1.7. Start Image
          • 7.4.2. FutureGrid Standalone Image Repository
            • 7.4.2.1. Requirement
            • 7.4.2.2. Software
            • 7.4.2.3. Documentation
              • 7.4.2.3.1. User Manual
              • 7.4.2.3.2. Installation
              • 7.4.2.3.3. Configuration
          • 7.4.3. Manual Image Customization
            • 7.4.3.1. Logging into India
            • 7.4.3.2. Requesting access
            • 7.4.3.3. Obtaining the image
              • 7.4.3.3.1. Getting an image from the repository
              • 7.4.3.3.2. Generating a new image
            • 7.4.3.4. Customizing the image
            • 7.4.3.5. Transfer the image back to India
            • 7.4.3.6. Log into India
            • 7.4.3.7. Upload the image to the repository
            • 7.4.3.8. Register your image in different infrastructures
              • 7.4.3.8.1. Register the image in Openstack
              • 7.4.3.8.2. Register the image in Eucalyptus
              • 7.4.3.8.3. Register the image in HPC
            • 7.4.3.9. Using your Registered Image
          • 7.4.4. RAIN Manual Pages
            • 7.4.4.1. fg-portal-manage
      • 8. PERFORMANCE TOOLS
        • 8.1. PAPI
          • 8.1.1. Availability
          • 8.1.2. References
      • 9. TUTORIALS
        • 2. Tutorials
          • 2.1. Tutorial Topic 0: Accessing FutureGrid Resources
          • 2.2. Tutorial Topic 1: Cloud Provisioning Platforms
          • 2.3. Tutorial Topic 2: Cloud Run-time Map/Reduce Platforms
          • 2.4. Tutorial Topic 3: Grid Appliances for Training, Education, and Outreach
          • 2.5. Tutorial Topic 4: High Performance Computing
          • 2.6. Tutorial Topic 5: Experiment Management
          • 2.7. Tutorial Topic 6: Image Management and Rain
          • 2.8. Tutorial Topic 7: Storage
          • 2.9. Other Tutorials and Educational Materials
      • 10. APPENDIX. SCREENCASTS
        • 10.1. Screencast: Using FutureGrid
          • 10.1.1. Overview
          • 10.1.2. Prerequisites
          • 10.1.3. Instructor Bio
          • 10.1.4. Course Content
            • 10.1.4.1. Creating FutureGrid Accounts and Projects
            • 10.1.4.2. OpenStack on FutureGrid
            • 10.1.4.3. Eucalyptus on FutureGrid
            • 10.1.4.4. Batch Jobs including Hadoop on FutureGrid
          • 10.1.5. Exercises
            • 10.1.5.1. Creating FutureGrid Accounts and Projects
      • 11. APPENDIX. MAINTAINING THIS MANUAL
        • 11.1. Git
        • 11.2. Contributing to the Manual
          • 11.2.1. Organisation of the Manual in Git
          • 11.2.2. Online Editing for Novice Editors
          • 11.2.3. FAQ
          • 11.2.4. Publication of the Manual
        • 11.3. Setting Up the Expert Developers Manual Editing Environment
          • 11.3.1. Python
          • 11.3.2. Virtualenv
          • 11.3.3. Create a github local directory with the manual
          • 11.3.4. Install the Requirements
          • 11.3.5. All-in-one setup script
          • 11.3.6. Watchdog
          • 11.3.7. Transfering a page from the portal to RST
          • 11.3.8. Creating the pages locally
          • 11.3.9. Publishing the pages
          • 11.3.10. Jira
          • 11.3.11. Portal link
          • 11.3.12. Screencast recording tips
            • 11.3.12.1. OSX
            • 11.3.12.2. Changing the prompt
            • 11.3.12.3. Recording of typed in commands
            • 11.3.12.4. Setup quicktime sound recording
      • 12. APPENDIX. MPI RING PROGRAM
        • 12.1. MPI Ring Program
      • 13. APPENDIX. OTHER SECTIONS THAT HAVE NOT YET BEEN REVIEWED OR ARE INTEGRATED IN THE MANUAL
        • 3. Delta GPU User Manual
          • 3.1. Running MPI/GPU program on the Delta cluster
          • 3.2. Running programs on a single GPU
          • 3.3. C-means clustering using CUDA on GPU
        • 13.2. IPOP (IP-over-P2P)
          • 13.2.1. Prerequisites:
          • 13.2.2. Step 1: Deploying VMs to be connected by IPOP
            • 13.2.2.1. Option 1: Deploying a VM on FutureGrid Using Nimbus
            • 13.2.2.2. Option 2: Deploying a VM on FutureGrid Using OpenStack
            • 13.2.2.3. Option 3: Deploying a VM on Amazon EC2
            • 13.2.2.4. Option 4: Deploying a VM on your computer (VMware, VirtualBox)
          • 13.2.3. Step 2: Establishing IPOP connectivity between two VMs
          • 13.2.4. Step 3: Securing communications between Alice and Bob
          • 13.2.5. Troubleshooting and for more information
        • 13.3. IPOP2 (IP-over-P2P)
          • 13.3.1. Prerequisites:
          • 13.3.2. Step 1: Bringing up Alice and Bob VMs
          • 13.3.3. Step 2: Disconnecting Alice and Bob VMs from the PlanetLab P2P overlay
          • 13.3.4. Step 3: Deploy Pam, a P2P Bootstrapping VM
          • 13.3.5. Step 4: Configure Pam to run P2P Bootstrapping Overlay
          • 13.3.6. Step 5: Configure Alice and Bob to use Pam as P2P overlay
          • 13.3.7. Step 6: Send messages between Alice and Bob
          • 13.3.8. 7.1) Inspecting the P2P network
          • 13.3.9. 7.2) Adding fault tolerance by deploying multiple bootstrap nodes
        • 13.4. GA1
          • 13.4.1. Prerequisites:
          • 13.4.2. Hands-on Tutorial:
          • 13.4.3. Reference Material:
          • 13.4.4. Authors:
        • 13.5. GA9
          • 13.5.1. Summary:
          • 13.5.2. Prerequisites:
            • 13.5.2.1. Deploying a Grid Appliance on FutureGrid
        • 13.6. HPSS
        • 13.7. SSH the wrong way ( DO NOT USE THIS TUTORIAL)
          • 13.7.1. Requirement for Windows Users
          • 13.7.2. Instructions for both Windows and Unix users
            • 13.7.2.1. Generate SSH key
            • 13.7.2.2. Check your ssh key
            • 13.7.2.3. Copy the content of your public key
            • 13.7.2.4. Upload the key to the FutureGrid Portal
            • 13.7.2.5. Testing your ssh key
            • 13.7.2.6. Testing your ssh key on Hotel
  • Page
      • Additional tutorials on FutureGrid-related technologies
        • Tutorial Topic 1: Cloud Provisioning Platforms
        • Tutorial Topic 2: High Performance Computing
        • Tutorial Topic 3: Information Systems
        • Others
          • CloudCom?2010 Tutorials/Screencasts
  • Source

Additional tutorials on FutureGrid-related technologies¶

Page Contents

  • Tutorial Topic 1: Cloud Provisioning Platforms
  • Tutorial Topic 2: High Performance Computing
  • Tutorial Topic 3: Information Systems
  • Others
    • CloudCom?2010 Tutorials/Screencasts

On this page you will find a number of links to tutorials and educational materials on technologies related to or planned for deployment on FutureGrid. For tutorials on using various FutureGrid technologies, please refer to our main tutorials page.

Please contact us at :portal:`help <help>` if you have any corrections or suggestions.

Tutorial Topic 1: Cloud Provisioning Platforms¶

  • FutureGrid Tutorial ON1: Using OpenNebula [advanced]

Tutorial Topic 2: High Performance Computing¶

  • FutureGrid Tutorial VA1: Performance Analysis with Vampir [advanced]
  • FutureGrid Tutorial VT1: Instrumentation and tracing with VampirTrace [advanced]

Tutorial Topic 3: Information Systems¶

  • FutureGrid Tutorial NL1: Netlogger Tutorial [advanced]
  • FutureGrid Tutorial IN1: INCA Tutorial [advanced]

Others¶

CloudCom?2010 Tutorials/Screencasts¶

  • List of CloudCom 2010 tutorials

Back to top

© Copyright 2013, FutureGrid, please contact Gregor von Laszewski about this manual..