• Site
  • 1. INTRODUCTION
    • 1.1. Preface
      • 1.1.1. Citation for Publications
      • 1.1.2. Acknowledgement
      • 1.1.3. Sponsors
      • 1.1.4. About this Manual
      • 1.1.5. Conventions
    • 1. Overview
      • 1.1. Executive Summary
      • 1.2. Project and Account Application
      • 1.3. Services
      • 1.4. Hardware
      • 1.5. Support
  • 2. ACCOUNTS AND PROJECTS
    • 2.1. Project and Account Management
      • 2.1.1. Terminology
      • 2.1.2. Quickstart
      • 2.1.3. Projects and Accounts for XSEDE users
      • 2.1.4. Project Management
        • 2.1.4.1. Create a Portal account
        • 2.1.4.2. Create a Project
          • 2.1.4.2.1. Example Project
        • 2.1.4.3. Join a Project
        • 2.1.4.4. Delete or Deactivate a user from a Project
        • 2.1.4.5. Reporting Results
        • 2.1.4.6. Close a Project
      • 2.1.5. Upload a SSH Public Key
      • 2.1.6. Upload an OpenId
      • 2.1.7. Accessing FutureGrid Resources
      • 2.1.8. Manage a Class on FutureGrid
      • 2.1.9. Mini FAQ
    • 2.2. Using SSH Keys
      • 2.2.1. Using SSH from Windows
      • 2.2.2. Generate a SSH key
      • 2.2.3. Add or Replace Passphrase for an Already Generated Key
      • 2.2.4. Upload the key to the FutureGrid Portal
      • 2.2.5. Testing your ssh key
      • 2.2.6. Testing your ssh key for Hotel
  • 3. STATUS
    • 3.1. Status
  • 4. HPC SERVICES
    • 4.1. Hardware
      • 4.1.1. Compute Resources
      • 4.1.2. Networks
      • 4.1.3. Network Impairments Device (NID)
    • 4.2. HPC Services
      • 4.2.1. Login Nodes
        • 4.2.1.1. SSH Add
        • 4.2.1.2. SSH Config
        • 4.2.1.3. Modules
        • 4.2.1.4. List of Available Modules on Various Machines
        • 4.2.1.5. Filesystem Layout
      • 4.2.2. Message Passing Interface (MPI)
        • 4.2.2.1. MPI Libraries
        • 4.2.2.2. Compiling MPI Applications
        • 4.2.2.3. Batch Jobs
      • 4.2.3. Job Management
        • 4.2.3.1. Job Submission
        • 4.2.3.2. Job Deletion
        • 4.2.3.3. Job Monitoring
        • 4.2.3.4. Job Output
      • 4.2.4. Xray HPC Services
        • 4.2.4.1. Submitting a Job on xray
    • 4.3. ScaleMP vSMP
      • 4.3.1. Accessing ScaleMP
      • 4.3.2. Submitting a job
      • 4.3.3. Developing a job script
        • 4.3.3.1. MPI
        • 4.3.3.2. OpenMP
        • 4.3.3.3. Threaded
        • 4.3.3.4. Throughput
        • 4.3.3.5. Serial
    • 4.4. Genesis II
      • 4.4.1. Introduction
      • 4.4.2. What is GenesisII
      • 4.4.3. Connecting to the GenesisII BES Endpoints
        • 4.4.3.1. Supported Data Staging Protocols
          • 4.4.3.1.1. Stage in
          • 4.4.3.1.2. Stage out
          • 4.4.3.1.3. Endpoint Connection Information
        • 4.4.3.2. India
        • 4.4.3.3. Sierra
          • 4.4.3.3.1. Alamo
          • 4.4.3.3.2. Hotel
      • 4.4.4. Using the Futuregrid GenesisII Endpoints as a Client
        • 4.4.4.1. Non-GenesisII Using a Standards-Compliant Client
        • 4.4.4.2. Using The GenesisII Client
          • 4.4.4.2.1. Acquiring GenesisII Client Package
            • 4.4.4.2.1.1. Linux
            • 4.4.4.2.1.2. Windows
            • 4.4.4.2.1.3. MacOS
          • 4.4.4.2.2. Installing the GenesisII/XCG Client Package
        • 4.4.4.3. Getting Started
          • 4.4.4.3.1. Start Grid Shell
            • 4.4.4.3.1.1. Windows
            • 4.4.4.3.1.2. Linux or MacOS
          • 4.4.4.3.2. Running Jobs
        • 4.4.4.4. Getting Help
    • 4.5. Unicore
      • 4.5.1. UNICORE 6 on FutureGrid User Manual
      • 4.5.2. Introduction
      • 4.5.3. What is UNICORE?
      • 4.5.4. Connecting to the UNICORE BES Endpoints From Other Grid Middleware Clients
        • 4.5.4.1. India Endpoint Info <currently unavailable>
        • 4.5.4.2. Sierra Endpoint Info
      • 4.5.5. Connecting to the UNICORE BES Endpoints Using a UNICORE Commandline Client
        • 4.5.5.1. Installing the UNICORE 6 Commandline Client (UCC)
          • 4.5.5.1.1. Acquire Client Bundle
          • 4.5.5.1.2. Unpack UCC
          • 4.5.5.1.3. Examine UCC Files
          • 4.5.5.1.4. Run Commandline Client
          • 4.5.5.1.5. Installation Conclusion
        • 4.5.5.2. Configuring Client to Connect to FutureGrid U6 Endpoints
          • 4.5.5.2.1. Configuration Overview
        • 4.5.5.3. Setting Up Security in UNICORE
        • 4.5.5.4. Setting Up a Keystore
          • 4.5.5.4.1. Creating a Truststore
          • 4.5.5.4.2. Acquiring the Registry Address
          • 4.5.5.4.3. Connecting to Endpoints w/o Registries
          • 4.5.5.4.4. Preferences File Modifications
          • 4.5.5.4.5. Example Preferences File
          • 4.5.5.4.6. Locating the Preferences File
          • 4.5.5.4.7. Validate Client Setup
          • 4.5.5.4.8. Configuration Conclusion
        • 4.5.5.5. Submitting Jobs to FutureGrid U6 Endpoints
          • 4.5.5.5.1. Getting Started
          • 4.5.5.5.2. Submit Jobs to BES
          • 4.5.5.5.3. Other BES Related Commands
          • 4.5.5.5.4. Job Submission Conclusion
        • 4.5.5.6. References
        • 4.5.5.7. Questions/Comments
      • 4.5.6. Running Jobs on UNICORE Sites
      • 4.5.7. Deploying a New UNICORE 6 Grid
        • 4.5.7.1. Prerequisites
        • 4.5.7.2. Installing the Core Server Bundle
        • 4.5.7.3. Starting/Stopping the UNICORE Servers
  • 5. IAAS/CLOUD SERVICES
    • 5.1. IaaS
      • 5.1.1. Nimbus Clouds
      • 5.1.2. OpenStack Clouds
      • 5.1.3. Eucalyptus Clouds
      • 5.1.4. Virtual Appliances for Training and Education
    • 5.2. OpenStack Havana
      • 5.2.1. Login
      • 5.2.2. Setup OpenStack Environment
      • 5.2.3. Creating the novarc file
      • 5.2.4. List flavors
      • 5.2.5. List images
      • 5.2.6. Key management
      • 5.2.7. Managing security groups
      • 5.2.8. Booting an image
      • 5.2.9. List running images
      • 5.2.10. Use block storage
      • 5.2.11. Set up external access to your instance
      • 5.2.12. Make a snapshot of an instance
      • 5.2.13. Automate some initial configuration
      • 5.2.14. Get the latest version of Ubuntu Cloud Image and upload it to the OpenStack
      • 5.2.15. Delete your instance
      • 5.2.16. How to change your password
      • 5.2.17. Things to do when you need Euca2ools or EC2 interfaces
      • 5.2.18. Horizon GUI
      • 5.2.19. Getting rc files via Horizon
    • 5.3. OpenStack Grizzly
      • 5.3.1. Login
      • 5.3.2. Creating the novarc file
      • 5.3.3. Activate OpenStack tools
      • 5.3.4. List flavors
      • 5.3.5. List images
      • 5.3.6. Standard Images
      • 5.3.7. Key management
      • 5.3.8. Managing security groups
      • 5.3.9. Booting an image
      • 5.3.10. List running images
      • 5.3.11. Use block storage
      • 5.3.12. Set up external access to your instance
      • 5.3.13. Make a snapshot of an instance
      • 5.3.14. Automate some initial configuration
      • 5.3.15. Get the latest version of Ubuntu Cloud Image and upload it to the OpenStack
      • 5.3.16. Delete your instance
      • 5.3.17. How to change your password
        • 5.3.17.1. Things to do when you need Euca2ools or EC2 interfaces
      • 5.3.18. Horizon GUI
        • 5.3.18.1. Getting rc files via Horizon
    • 5.4. Openstack Horizon
      • 5.4.1. Managing Virtual Machine Instances
      • 5.4.2. Snapshots
      • 5.4.3. Floating IP Addresses
      • 5.4.4. Accessing Your Virtual Machine
      • 5.4.5. Uploading an Image
      • 5.4.6. Managing Volumes
      • 5.4.7. EC2 Interface
      • 5.4.8. OpenStack REST Interfaces
    • 5.5. Converting Virtualbox images to OpenStack
      • 5.5.1. Prerequisites
        • 5.5.1.1. Disable SELinux (Only for RedHat-based Linux like CentOS)
        • 5.5.1.2. Configuring the image network interface (eth0) for DHCP
        • 5.5.1.3. Configure the image to allow OpenStack to inject the ssh key
        • 5.5.1.4. Configure udev persistent rules (only CentOS)
      • 5.5.2. Convert your virtual box image to raw format
      • 5.5.3. Convert the image to qcow2 format (optional)
      • 5.5.4. Test your image
      • 5.5.5. Transfer your Image to India
      • 5.5.6. Log into India
      • 5.5.7. Upload your image to OpenStack
      • 5.5.8. Checking Status Image
      • 5.5.9. Test Image in OpenStack
      • 5.5.10. Troubleshooting
      • 5.5.11. Notes:
    • 5.6. Nimbus
      • 5.6.1. Nimbus IaaS
        • 5.6.1.1. What is Nimbus?
        • 5.6.1.2. Nimbus on FutureGrid
        • 5.6.1.3. Getting Started
          • 5.6.1.3.1. Log into hotel
          • 5.6.1.3.2. Download and install cloud-client
          • 5.6.1.3.3. Obtain Your Nimbus Credentials and Configuration Files
          • 5.6.1.3.4. Check Your ssh Key
        • 5.6.1.4. Using the Cloud Client
          • 5.6.1.4.1. Check out the various FutureGrid clouds
          • 5.6.1.4.2. Run a Virtual Machine
          • 5.6.1.4.3. Create a New VM Image
          • 5.6.1.4.4. Save the Changes to a New VM
          • 5.6.1.4.5. Launch Your New VM
          • 5.6.1.4.6. Terminate the VM
          • 5.6.1.4.7. Uploading and downloading images
        • 5.6.1.5. Converting a Xen image to KVM
        • 5.6.1.6. Virtual Clusters
          • 5.6.1.6.1. Cluster Definition File
          • 5.6.1.6.2. Start the Cluster
          • 5.6.1.6.3. Check Out the Virtual Cluster
      • 5.6.2. Launch A VM via Nimbus
        • 5.6.2.1. Options
        • 5.6.2.2. What Happens
      • 5.6.3. cloudinit.d
      • 5.6.4. Phantom
    • 5.7. Eucalyptus
      • 5.7.1. Account Setup
      • 5.7.2. Resources Overview
      • 5.7.3. Testing Your Setup
      • 5.7.4. Available Images
      • 5.7.5. Image Deployment
      • 5.7.6. Logging Into the VM
      • 5.7.7. Shutting down the VM
      • 5.7.8. VM Network Info
      • 5.7.9. Image Management
      • 5.7.10. Volume Management
      • 5.7.11. Status of Deployments
  • 6. PAAS/CLOUD SERVICES
    • 6.1. Using Hadoop in FutureGrid
      • 6.1.1. Running Hadoop as a Batch Job using MyHadoop
        • 6.1.1.1. myHadoop on FutureGrid
          • 6.1.1.1.1. Login into a machine tha has myHadoop installed
          • 6.1.1.1.2. Load the needed modules
          • 6.1.1.1.3. Run the Example
          • 6.1.1.1.4. Details of the Script
          • 6.1.1.1.5. Submission of the Hadoop job
          • 6.1.1.1.6. Persistent Mode
          • 6.1.1.1.7. Customizing Hadoop Settings
          • 6.1.1.1.8. Using a Different Installation of Hadoop
          • 6.1.1.1.9. References
    • 6.2. Using Twister in FutureGrid
      • 6.2.1. What is Twister?
      • 6.2.2. Running Twister on FutureGrid
      • 6.2.3. Run Twister Applications
      • 6.2.4. Papers and Presentations
  • 7. EXPERIMENT MANAGEMENT SERVICES
    • 7.1. Interactive Experiment Management
      • 7.1.1. Host List Manager
        • 7.1.1.1. Host List Manager Setup
        • 7.1.1.2. Provisioning Hosts
        • 7.1.1.3. Managing Host Lists
      • 7.1.2. TakTuk
        • 7.1.2.1. Running Experiments with TakTuk
        • 7.1.2.2. Host List Manager Commands
          • 7.1.2.2.1. hupdate
          • 7.1.2.2.2. hmanage
          • 7.1.2.2.3. hshow
          • 7.1.2.2.4. htag
          • 7.1.2.2.5. hlist
          • 7.1.2.2.6. htaktuk
        • 7.1.2.3. TakTuk Commands
    • 7.2. Precip
      • 7.2.1. Installation
      • 7.2.2. API
      • 7.2.3. Examples
        • 7.2.3.1. Hello World
        • 7.2.3.2. Resources from mulitple infrastructures
        • 7.2.3.3. Setting up a Condor pool and running a Pegasus workflow
    • 7.3. Pegasus
      • 7.3.1. The Pegasus Run-Time Cloud Architecture
      • 7.3.2. Using Pegasus on FutureGrid
    • 7.4. RAIN
      • 7.4.1. Generate and Register an OS Image on FutureGrid using the FG Shell
        • 7.4.1.1. Request access
        • 7.4.1.2. Log into India
        • 7.4.1.3. Start the Shell
        • 7.4.1.4. Generate the Image
        • 7.4.1.5. Image Repository
        • 7.4.1.6. Register Image
        • 7.4.1.7. Start Image
      • 7.4.2. FutureGrid Standalone Image Repository
        • 7.4.2.1. Requirement
        • 7.4.2.2. Software
        • 7.4.2.3. Documentation
          • 7.4.2.3.1. User Manual
          • 7.4.2.3.2. Installation
          • 7.4.2.3.3. Configuration
      • 7.4.3. Manual Image Customization
        • 7.4.3.1. Logging into India
        • 7.4.3.2. Requesting access
        • 7.4.3.3. Obtaining the image
          • 7.4.3.3.1. Getting an image from the repository
          • 7.4.3.3.2. Generating a new image
        • 7.4.3.4. Customizing the image
        • 7.4.3.5. Transfer the image back to India
        • 7.4.3.6. Log into India
        • 7.4.3.7. Upload the image to the repository
        • 7.4.3.8. Register your image in different infrastructures
          • 7.4.3.8.1. Register the image in Openstack
          • 7.4.3.8.2. Register the image in Eucalyptus
          • 7.4.3.8.3. Register the image in HPC
        • 7.4.3.9. Using your Registered Image
      • 7.4.4. RAIN Manual Pages
        • 7.4.4.1. fg-portal-manage
  • 8. PERFORMANCE TOOLS
    • 8.1. PAPI
      • 8.1.1. Availability
      • 8.1.2. References
  • 9. TUTORIALS
    • 2. Tutorials
      • 2.1. Tutorial Topic 0: Accessing FutureGrid Resources
      • 2.2. Tutorial Topic 1: Cloud Provisioning Platforms
      • 2.3. Tutorial Topic 2: Cloud Run-time Map/Reduce Platforms
      • 2.4. Tutorial Topic 3: Grid Appliances for Training, Education, and Outreach
      • 2.5. Tutorial Topic 4: High Performance Computing
      • 2.6. Tutorial Topic 5: Experiment Management
      • 2.7. Tutorial Topic 6: Image Management and Rain
      • 2.8. Tutorial Topic 7: Storage
      • 2.9. Other Tutorials and Educational Materials
        • 2.9.1. <<<<<<< HEAD:doc/source/tutorials.rst
        • 2.9.2. FutureGrid Grid Appliance for Nimbus and Eucalyptus
        • 2.9.3. One-click Hadoop WordCount on Eucalyptus FutureGrid
          • 2.9.3.1. Introduction
          • 2.9.3.2. Prerequisite
          • 2.9.3.3. Login to india.futuregrid.org
          • 2.9.3.4. Download and unzip the “hadoopOneClick.zip” package
          • 2.9.3.5. Run hadoop-one-click.sh
          • 2.9.3.6. Verify output
          • 2.9.3.7. Extensions
        • 2.9.4. One-click Twister K-means on Eucalyptus FutureGrid
          • 2.9.4.1. Introduction
          • 2.9.4.2. Prerequisite
          • 2.9.4.3. Login to india.futuregrid.org
          • 2.9.4.4. Download and unzip the “twisterOneClick.zip” package
          • 2.9.4.5. Run twister-one-click.sh
          • 2.9.4.6. Verify results in the standard output of the scripts
          • 2.9.4.7. Extensions
        • 2.9.5. Virtual Appliances
          • 2.9.5.1. Creating Your Own Appliance:
            • 2.9.5.1.1. Nimbus:
            • 2.9.5.1.2. Eucalyptus:
          • 2.9.5.2. Reusing an Existing Appliance:
            • 2.9.5.2.1. Editing tips for the table:
  • 10. APPENDIX. SCREENCASTS
    • 10.1. Screencast: Using FutureGrid
      • 10.1.1. Overview
      • 10.1.2. Prerequisites
      • 10.1.3. Instructor Bio
      • 10.1.4. Course Content
        • 10.1.4.1. Creating FutureGrid Accounts and Projects
        • 10.1.4.2. OpenStack on FutureGrid
        • 10.1.4.3. Eucalyptus on FutureGrid
        • 10.1.4.4. Batch Jobs including Hadoop on FutureGrid
      • 10.1.5. Exercises
        • 10.1.5.1. Creating FutureGrid Accounts and Projects
  • 11. APPENDIX. MAINTAINING THIS MANUAL
    • 11.1. Git
    • 11.2. Contributing to the Manual
      • 11.2.1. Organisation of the Manual in Git
      • 11.2.2. Online Editing for Novice Editors
      • 11.2.3. FAQ
      • 11.2.4. Publication of the Manual
    • 11.3. Setting Up the Expert Developers Manual Editing Environment
      • 11.3.1. Python
      • 11.3.2. Virtualenv
      • 11.3.3. Create a github local directory with the manual
      • 11.3.4. Install the Requirements
      • 11.3.5. All-in-one setup script
      • 11.3.6. Watchdog
      • 11.3.7. Transfering a page from the portal to RST
      • 11.3.8. Creating the pages locally
      • 11.3.9. Publishing the pages
      • 11.3.10. Jira
      • 11.3.11. Portal link
      • 11.3.12. Screencast recording tips
        • 11.3.12.1. OSX
        • 11.3.12.2. Changing the prompt
        • 11.3.12.3. Recording of typed in commands
        • 11.3.12.4. Setup quicktime sound recording
  • 12. APPENDIX. MPI RING PROGRAM
    • 12.1. MPI Ring Program
  • 13. APPENDIX. OTHER SECTIONS THAT HAVE NOT YET BEEN REVIEWED OR ARE INTEGRATED IN THE MANUAL
    • 3. Delta GPU User Manual
      • 3.1. Running MPI/GPU program on the Delta cluster
      • 3.2. Running programs on a single GPU
      • 3.3. C-means clustering using CUDA on GPU
    • 13.2. IPOP (IP-over-P2P)
      • 13.2.1. Prerequisites:
      • 13.2.2. Step 1: Deploying VMs to be connected by IPOP
        • 13.2.2.1. Option 1: Deploying a VM on FutureGrid Using Nimbus
        • 13.2.2.2. Option 2: Deploying a VM on FutureGrid Using OpenStack
        • 13.2.2.3. Option 3: Deploying a VM on Amazon EC2
        • 13.2.2.4. Option 4: Deploying a VM on your computer (VMware, VirtualBox)
      • 13.2.3. Step 2: Establishing IPOP connectivity between two VMs
      • 13.2.4. Step 3: Securing communications between Alice and Bob
      • 13.2.5. Troubleshooting and for more information
    • 13.3. IPOP2 (IP-over-P2P)
      • 13.3.1. Prerequisites:
      • 13.3.2. Step 1: Bringing up Alice and Bob VMs
      • 13.3.3. Step 2: Disconnecting Alice and Bob VMs from the PlanetLab P2P overlay
      • 13.3.4. Step 3: Deploy Pam, a P2P Bootstrapping VM
      • 13.3.5. Step 4: Configure Pam to run P2P Bootstrapping Overlay
      • 13.3.6. Step 5: Configure Alice and Bob to use Pam as P2P overlay
      • 13.3.7. Step 6: Send messages between Alice and Bob
      • 13.3.8. 7.1) Inspecting the P2P network
      • 13.3.9. 7.2) Adding fault tolerance by deploying multiple bootstrap nodes
    • 13.4. GA1
    • 13.5. GA9
      • 13.5.1. Summary:
      • 13.5.2. Prerequisites:
        • 13.5.2.1. Deploying a Grid Appliance on FutureGrid
    • 13.6. HPSS
    • 13.7. SSH the wrong way ( DO NOT USE THIS TUTORIAL)
      • 13.7.1. Requirement for Windows Users
      • 13.7.2. Instructions for both Windows and Unix users
        • 13.7.2.1. Generate SSH key
        • 13.7.2.2. Check your ssh key
        • 13.7.2.3. Copy the content of your public key
        • 13.7.2.4. Upload the key to the FutureGrid Portal
        • 13.7.2.5. Testing your ssh key
        • 13.7.2.6. Testing your ssh key on Hotel
  • 1. INTRODUCTION
    • 1.1. Preface
      • 1.1.1. Citation for Publications
      • 1.1.2. Acknowledgement
      • 1.1.3. Sponsors
      • 1.1.4. About this Manual
      • 1.1.5. Conventions
    • 1. Overview
      • 1.1. Executive Summary
      • 1.2. Project and Account Application
      • 1.3. Services
      • 1.4. Hardware
      • 1.5. Support
  • 2. ACCOUNTS AND PROJECTS
    • 2.1. Project and Account Management
      • 2.1.1. Terminology
      • 2.1.2. Quickstart
      • 2.1.3. Projects and Accounts for XSEDE users
      • 2.1.4. Project Management
        • 2.1.4.1. Create a Portal account
        • 2.1.4.2. Create a Project
          • 2.1.4.2.1. Example Project
        • 2.1.4.3. Join a Project
        • 2.1.4.4. Delete or Deactivate a user from a Project
        • 2.1.4.5. Reporting Results
        • 2.1.4.6. Close a Project
      • 2.1.5. Upload a SSH Public Key
      • 2.1.6. Upload an OpenId
      • 2.1.7. Accessing FutureGrid Resources
      • 2.1.8. Manage a Class on FutureGrid
      • 2.1.9. Mini FAQ
    • 2.2. Using SSH Keys
      • 2.2.1. Using SSH from Windows
      • 2.2.2. Generate a SSH key
      • 2.2.3. Add or Replace Passphrase for an Already Generated Key
      • 2.2.4. Upload the key to the FutureGrid Portal
      • 2.2.5. Testing your ssh key
      • 2.2.6. Testing your ssh key for Hotel
  • 3. STATUS
    • 3.1. Status
  • 4. HPC SERVICES
    • 4.1. Hardware
      • 4.1.1. Compute Resources
      • 4.1.2. Networks
      • 4.1.3. Network Impairments Device (NID)
    • 4.2. HPC Services
      • 4.2.1. Login Nodes
        • 4.2.1.1. SSH Add
        • 4.2.1.2. SSH Config
        • 4.2.1.3. Modules
        • 4.2.1.4. List of Available Modules on Various Machines
        • 4.2.1.5. Filesystem Layout
      • 4.2.2. Message Passing Interface (MPI)
        • 4.2.2.1. MPI Libraries
        • 4.2.2.2. Compiling MPI Applications
        • 4.2.2.3. Batch Jobs
      • 4.2.3. Job Management
        • 4.2.3.1. Job Submission
        • 4.2.3.2. Job Deletion
        • 4.2.3.3. Job Monitoring
        • 4.2.3.4. Job Output
      • 4.2.4. Xray HPC Services
        • 4.2.4.1. Submitting a Job on xray
    • 4.3. ScaleMP vSMP
      • 4.3.1. Accessing ScaleMP
      • 4.3.2. Submitting a job
      • 4.3.3. Developing a job script
        • 4.3.3.1. MPI
        • 4.3.3.2. OpenMP
        • 4.3.3.3. Threaded
        • 4.3.3.4. Throughput
        • 4.3.3.5. Serial
    • 4.4. Genesis II
      • 4.4.1. Introduction
      • 4.4.2. What is GenesisII
      • 4.4.3. Connecting to the GenesisII BES Endpoints
        • 4.4.3.1. Supported Data Staging Protocols
          • 4.4.3.1.1. Stage in
          • 4.4.3.1.2. Stage out
          • 4.4.3.1.3. Endpoint Connection Information
        • 4.4.3.2. India
        • 4.4.3.3. Sierra
          • 4.4.3.3.1. Alamo
          • 4.4.3.3.2. Hotel
      • 4.4.4. Using the Futuregrid GenesisII Endpoints as a Client
        • 4.4.4.1. Non-GenesisII Using a Standards-Compliant Client
        • 4.4.4.2. Using The GenesisII Client
          • 4.4.4.2.1. Acquiring GenesisII Client Package
            • 4.4.4.2.1.1. Linux
            • 4.4.4.2.1.2. Windows
            • 4.4.4.2.1.3. MacOS
          • 4.4.4.2.2. Installing the GenesisII/XCG Client Package
        • 4.4.4.3. Getting Started
          • 4.4.4.3.1. Start Grid Shell
            • 4.4.4.3.1.1. Windows
            • 4.4.4.3.1.2. Linux or MacOS
          • 4.4.4.3.2. Running Jobs
        • 4.4.4.4. Getting Help
    • 4.5. Unicore
      • 4.5.1. UNICORE 6 on FutureGrid User Manual
      • 4.5.2. Introduction
      • 4.5.3. What is UNICORE?
      • 4.5.4. Connecting to the UNICORE BES Endpoints From Other Grid Middleware Clients
        • 4.5.4.1. India Endpoint Info <currently unavailable>
        • 4.5.4.2. Sierra Endpoint Info
      • 4.5.5. Connecting to the UNICORE BES Endpoints Using a UNICORE Commandline Client
        • 4.5.5.1. Installing the UNICORE 6 Commandline Client (UCC)
          • 4.5.5.1.1. Acquire Client Bundle
          • 4.5.5.1.2. Unpack UCC
          • 4.5.5.1.3. Examine UCC Files
          • 4.5.5.1.4. Run Commandline Client
          • 4.5.5.1.5. Installation Conclusion
        • 4.5.5.2. Configuring Client to Connect to FutureGrid U6 Endpoints
          • 4.5.5.2.1. Configuration Overview
        • 4.5.5.3. Setting Up Security in UNICORE
        • 4.5.5.4. Setting Up a Keystore
          • 4.5.5.4.1. Creating a Truststore
          • 4.5.5.4.2. Acquiring the Registry Address
          • 4.5.5.4.3. Connecting to Endpoints w/o Registries
          • 4.5.5.4.4. Preferences File Modifications
          • 4.5.5.4.5. Example Preferences File
          • 4.5.5.4.6. Locating the Preferences File
          • 4.5.5.4.7. Validate Client Setup
          • 4.5.5.4.8. Configuration Conclusion
        • 4.5.5.5. Submitting Jobs to FutureGrid U6 Endpoints
          • 4.5.5.5.1. Getting Started
          • 4.5.5.5.2. Submit Jobs to BES
          • 4.5.5.5.3. Other BES Related Commands
          • 4.5.5.5.4. Job Submission Conclusion
        • 4.5.5.6. References
        • 4.5.5.7. Questions/Comments
      • 4.5.6. Running Jobs on UNICORE Sites
      • 4.5.7. Deploying a New UNICORE 6 Grid
        • 4.5.7.1. Prerequisites
        • 4.5.7.2. Installing the Core Server Bundle
        • 4.5.7.3. Starting/Stopping the UNICORE Servers
  • 5. IAAS/CLOUD SERVICES
    • 5.1. IaaS
      • 5.1.1. Nimbus Clouds
      • 5.1.2. OpenStack Clouds
      • 5.1.3. Eucalyptus Clouds
      • 5.1.4. Virtual Appliances for Training and Education
    • 5.2. OpenStack Havana
      • 5.2.1. Login
      • 5.2.2. Setup OpenStack Environment
      • 5.2.3. Creating the novarc file
      • 5.2.4. List flavors
      • 5.2.5. List images
      • 5.2.6. Key management
      • 5.2.7. Managing security groups
      • 5.2.8. Booting an image
      • 5.2.9. List running images
      • 5.2.10. Use block storage
      • 5.2.11. Set up external access to your instance
      • 5.2.12. Make a snapshot of an instance
      • 5.2.13. Automate some initial configuration
      • 5.2.14. Get the latest version of Ubuntu Cloud Image and upload it to the OpenStack
      • 5.2.15. Delete your instance
      • 5.2.16. How to change your password
      • 5.2.17. Things to do when you need Euca2ools or EC2 interfaces
      • 5.2.18. Horizon GUI
      • 5.2.19. Getting rc files via Horizon
    • 5.3. OpenStack Grizzly
      • 5.3.1. Login
      • 5.3.2. Creating the novarc file
      • 5.3.3. Activate OpenStack tools
      • 5.3.4. List flavors
      • 5.3.5. List images
      • 5.3.6. Standard Images
      • 5.3.7. Key management
      • 5.3.8. Managing security groups
      • 5.3.9. Booting an image
      • 5.3.10. List running images
      • 5.3.11. Use block storage
      • 5.3.12. Set up external access to your instance
      • 5.3.13. Make a snapshot of an instance
      • 5.3.14. Automate some initial configuration
      • 5.3.15. Get the latest version of Ubuntu Cloud Image and upload it to the OpenStack
      • 5.3.16. Delete your instance
      • 5.3.17. How to change your password
        • 5.3.17.1. Things to do when you need Euca2ools or EC2 interfaces
      • 5.3.18. Horizon GUI
        • 5.3.18.1. Getting rc files via Horizon
    • 5.4. Openstack Horizon
      • 5.4.1. Managing Virtual Machine Instances
      • 5.4.2. Snapshots
      • 5.4.3. Floating IP Addresses
      • 5.4.4. Accessing Your Virtual Machine
      • 5.4.5. Uploading an Image
      • 5.4.6. Managing Volumes
      • 5.4.7. EC2 Interface
      • 5.4.8. OpenStack REST Interfaces
    • 5.5. Converting Virtualbox images to OpenStack
      • 5.5.1. Prerequisites
        • 5.5.1.1. Disable SELinux (Only for RedHat-based Linux like CentOS)
        • 5.5.1.2. Configuring the image network interface (eth0) for DHCP
        • 5.5.1.3. Configure the image to allow OpenStack to inject the ssh key
        • 5.5.1.4. Configure udev persistent rules (only CentOS)
      • 5.5.2. Convert your virtual box image to raw format
      • 5.5.3. Convert the image to qcow2 format (optional)
      • 5.5.4. Test your image
      • 5.5.5. Transfer your Image to India
      • 5.5.6. Log into India
      • 5.5.7. Upload your image to OpenStack
      • 5.5.8. Checking Status Image
      • 5.5.9. Test Image in OpenStack
      • 5.5.10. Troubleshooting
      • 5.5.11. Notes:
    • 5.6. Nimbus
      • 5.6.1. Nimbus IaaS
        • 5.6.1.1. What is Nimbus?
        • 5.6.1.2. Nimbus on FutureGrid
        • 5.6.1.3. Getting Started
          • 5.6.1.3.1. Log into hotel
          • 5.6.1.3.2. Download and install cloud-client
          • 5.6.1.3.3. Obtain Your Nimbus Credentials and Configuration Files
          • 5.6.1.3.4. Check Your ssh Key
        • 5.6.1.4. Using the Cloud Client
          • 5.6.1.4.1. Check out the various FutureGrid clouds
          • 5.6.1.4.2. Run a Virtual Machine
          • 5.6.1.4.3. Create a New VM Image
          • 5.6.1.4.4. Save the Changes to a New VM
          • 5.6.1.4.5. Launch Your New VM
          • 5.6.1.4.6. Terminate the VM
          • 5.6.1.4.7. Uploading and downloading images
        • 5.6.1.5. Converting a Xen image to KVM
        • 5.6.1.6. Virtual Clusters
          • 5.6.1.6.1. Cluster Definition File
          • 5.6.1.6.2. Start the Cluster
          • 5.6.1.6.3. Check Out the Virtual Cluster
      • 5.6.2. Launch A VM via Nimbus
        • 5.6.2.1. Options
        • 5.6.2.2. What Happens
      • 5.6.3. cloudinit.d
      • 5.6.4. Phantom
    • 5.7. Eucalyptus
      • 5.7.1. Account Setup
      • 5.7.2. Resources Overview
      • 5.7.3. Testing Your Setup
      • 5.7.4. Available Images
      • 5.7.5. Image Deployment
      • 5.7.6. Logging Into the VM
      • 5.7.7. Shutting down the VM
      • 5.7.8. VM Network Info
      • 5.7.9. Image Management
      • 5.7.10. Volume Management
      • 5.7.11. Status of Deployments
  • 6. PAAS/CLOUD SERVICES
    • 6.1. Using Hadoop in FutureGrid
      • 6.1.1. Running Hadoop as a Batch Job using MyHadoop
        • 6.1.1.1. myHadoop on FutureGrid
          • 6.1.1.1.1. Login into a machine tha has myHadoop installed
          • 6.1.1.1.2. Load the needed modules
          • 6.1.1.1.3. Run the Example
          • 6.1.1.1.4. Details of the Script
          • 6.1.1.1.5. Submission of the Hadoop job
          • 6.1.1.1.6. Persistent Mode
          • 6.1.1.1.7. Customizing Hadoop Settings
          • 6.1.1.1.8. Using a Different Installation of Hadoop
          • 6.1.1.1.9. References
    • 6.2. Using Twister in FutureGrid
      • 6.2.1. What is Twister?
      • 6.2.2. Running Twister on FutureGrid
      • 6.2.3. Run Twister Applications
      • 6.2.4. Papers and Presentations
  • 7. EXPERIMENT MANAGEMENT SERVICES
    • 7.1. Interactive Experiment Management
      • 7.1.1. Host List Manager
        • 7.1.1.1. Host List Manager Setup
        • 7.1.1.2. Provisioning Hosts
        • 7.1.1.3. Managing Host Lists
      • 7.1.2. TakTuk
        • 7.1.2.1. Running Experiments with TakTuk
        • 7.1.2.2. Host List Manager Commands
          • 7.1.2.2.1. hupdate
          • 7.1.2.2.2. hmanage
          • 7.1.2.2.3. hshow
          • 7.1.2.2.4. htag
          • 7.1.2.2.5. hlist
          • 7.1.2.2.6. htaktuk
        • 7.1.2.3. TakTuk Commands
    • 7.2. Precip
      • 7.2.1. Installation
      • 7.2.2. API
      • 7.2.3. Examples
        • 7.2.3.1. Hello World
        • 7.2.3.2. Resources from mulitple infrastructures
        • 7.2.3.3. Setting up a Condor pool and running a Pegasus workflow
    • 7.3. Pegasus
      • 7.3.1. The Pegasus Run-Time Cloud Architecture
      • 7.3.2. Using Pegasus on FutureGrid
    • 7.4. RAIN
      • 7.4.1. Generate and Register an OS Image on FutureGrid using the FG Shell
        • 7.4.1.1. Request access
        • 7.4.1.2. Log into India
        • 7.4.1.3. Start the Shell
        • 7.4.1.4. Generate the Image
        • 7.4.1.5. Image Repository
        • 7.4.1.6. Register Image
        • 7.4.1.7. Start Image
      • 7.4.2. FutureGrid Standalone Image Repository
        • 7.4.2.1. Requirement
        • 7.4.2.2. Software
        • 7.4.2.3. Documentation
          • 7.4.2.3.1. User Manual
          • 7.4.2.3.2. Installation
          • 7.4.2.3.3. Configuration
      • 7.4.3. Manual Image Customization
        • 7.4.3.1. Logging into India
        • 7.4.3.2. Requesting access
        • 7.4.3.3. Obtaining the image
          • 7.4.3.3.1. Getting an image from the repository
          • 7.4.3.3.2. Generating a new image
        • 7.4.3.4. Customizing the image
        • 7.4.3.5. Transfer the image back to India
        • 7.4.3.6. Log into India
        • 7.4.3.7. Upload the image to the repository
        • 7.4.3.8. Register your image in different infrastructures
          • 7.4.3.8.1. Register the image in Openstack
          • 7.4.3.8.2. Register the image in Eucalyptus
          • 7.4.3.8.3. Register the image in HPC
        • 7.4.3.9. Using your Registered Image
      • 7.4.4. RAIN Manual Pages
        • 7.4.4.1. fg-portal-manage
  • 8. PERFORMANCE TOOLS
    • 8.1. PAPI
      • 8.1.1. Availability
      • 8.1.2. References
  • 9. TUTORIALS
    • 2. Tutorials
      • 2.1. Tutorial Topic 0: Accessing FutureGrid Resources
      • 2.2. Tutorial Topic 1: Cloud Provisioning Platforms
      • 2.3. Tutorial Topic 2: Cloud Run-time Map/Reduce Platforms
      • 2.4. Tutorial Topic 3: Grid Appliances for Training, Education, and Outreach
      • 2.5. Tutorial Topic 4: High Performance Computing
      • 2.6. Tutorial Topic 5: Experiment Management
      • 2.7. Tutorial Topic 6: Image Management and Rain
      • 2.8. Tutorial Topic 7: Storage
      • 2.9. Other Tutorials and Educational Materials
        • 2.9.1. <<<<<<< HEAD:doc/source/tutorials.rst
        • 2.9.2. FutureGrid Grid Appliance for Nimbus and Eucalyptus
        • 2.9.3. One-click Hadoop WordCount on Eucalyptus FutureGrid
          • 2.9.3.1. Introduction
          • 2.9.3.2. Prerequisite
          • 2.9.3.3. Login to india.futuregrid.org
          • 2.9.3.4. Download and unzip the “hadoopOneClick.zip” package
          • 2.9.3.5. Run hadoop-one-click.sh
          • 2.9.3.6. Verify output
          • 2.9.3.7. Extensions
        • 2.9.4. One-click Twister K-means on Eucalyptus FutureGrid
          • 2.9.4.1. Introduction
          • 2.9.4.2. Prerequisite
          • 2.9.4.3. Login to india.futuregrid.org
          • 2.9.4.4. Download and unzip the “twisterOneClick.zip” package
          • 2.9.4.5. Run twister-one-click.sh
          • 2.9.4.6. Verify results in the standard output of the scripts
          • 2.9.4.7. Extensions
        • 2.9.5. Virtual Appliances
          • 2.9.5.1. Creating Your Own Appliance:
            • 2.9.5.1.1. Nimbus:
            • 2.9.5.1.2. Eucalyptus:
          • 2.9.5.2. Reusing an Existing Appliance:
            • 2.9.5.2.1. Editing tips for the table:
  • 10. APPENDIX. SCREENCASTS
    • 10.1. Screencast: Using FutureGrid
      • 10.1.1. Overview
      • 10.1.2. Prerequisites
      • 10.1.3. Instructor Bio
      • 10.1.4. Course Content
        • 10.1.4.1. Creating FutureGrid Accounts and Projects
        • 10.1.4.2. OpenStack on FutureGrid
        • 10.1.4.3. Eucalyptus on FutureGrid
        • 10.1.4.4. Batch Jobs including Hadoop on FutureGrid
      • 10.1.5. Exercises
        • 10.1.5.1. Creating FutureGrid Accounts and Projects
  • 11. APPENDIX. MAINTAINING THIS MANUAL
    • 11.1. Git
    • 11.2. Contributing to the Manual
      • 11.2.1. Organisation of the Manual in Git
      • 11.2.2. Online Editing for Novice Editors
      • 11.2.3. FAQ
      • 11.2.4. Publication of the Manual
    • 11.3. Setting Up the Expert Developers Manual Editing Environment
      • 11.3.1. Python
      • 11.3.2. Virtualenv
      • 11.3.3. Create a github local directory with the manual
      • 11.3.4. Install the Requirements
      • 11.3.5. All-in-one setup script
      • 11.3.6. Watchdog
      • 11.3.7. Transfering a page from the portal to RST
      • 11.3.8. Creating the pages locally
      • 11.3.9. Publishing the pages
      • 11.3.10. Jira
      • 11.3.11. Portal link
      • 11.3.12. Screencast recording tips
        • 11.3.12.1. OSX
        • 11.3.12.2. Changing the prompt
        • 11.3.12.3. Recording of typed in commands
        • 11.3.12.4. Setup quicktime sound recording
  • 12. APPENDIX. MPI RING PROGRAM
    • 12.1. MPI Ring Program
  • 13. APPENDIX. OTHER SECTIONS THAT HAVE NOT YET BEEN REVIEWED OR ARE INTEGRATED IN THE MANUAL
    • 3. Delta GPU User Manual
      • 3.1. Running MPI/GPU program on the Delta cluster
      • 3.2. Running programs on a single GPU
      • 3.3. C-means clustering using CUDA on GPU
    • 13.2. IPOP (IP-over-P2P)
      • 13.2.1. Prerequisites:
      • 13.2.2. Step 1: Deploying VMs to be connected by IPOP
        • 13.2.2.1. Option 1: Deploying a VM on FutureGrid Using Nimbus
        • 13.2.2.2. Option 2: Deploying a VM on FutureGrid Using OpenStack
        • 13.2.2.3. Option 3: Deploying a VM on Amazon EC2
        • 13.2.2.4. Option 4: Deploying a VM on your computer (VMware, VirtualBox)
      • 13.2.3. Step 2: Establishing IPOP connectivity between two VMs
      • 13.2.4. Step 3: Securing communications between Alice and Bob
      • 13.2.5. Troubleshooting and for more information
    • 13.3. IPOP2 (IP-over-P2P)
      • 13.3.1. Prerequisites:
      • 13.3.2. Step 1: Bringing up Alice and Bob VMs
      • 13.3.3. Step 2: Disconnecting Alice and Bob VMs from the PlanetLab P2P overlay
      • 13.3.4. Step 3: Deploy Pam, a P2P Bootstrapping VM
      • 13.3.5. Step 4: Configure Pam to run P2P Bootstrapping Overlay
      • 13.3.6. Step 5: Configure Alice and Bob to use Pam as P2P overlay
      • 13.3.7. Step 6: Send messages between Alice and Bob
      • 13.3.8. 7.1) Inspecting the P2P network
      • 13.3.9. 7.2) Adding fault tolerance by deploying multiple bootstrap nodes
    • 13.4. GA1
    • 13.5. GA9
      • 13.5.1. Summary:
      • 13.5.2. Prerequisites:
        • 13.5.2.1. Deploying a Grid Appliance on FutureGrid
    • 13.6. HPSS
    • 13.7. SSH the wrong way ( DO NOT USE THIS TUTORIAL)
      • 13.7.1. Requirement for Windows Users
      • 13.7.2. Instructions for both Windows and Unix users
        • 13.7.2.1. Generate SSH key
        • 13.7.2.2. Check your ssh key
        • 13.7.2.3. Copy the content of your public key
        • 13.7.2.4. Upload the key to the FutureGrid Portal
        • 13.7.2.5. Testing your ssh key
        • 13.7.2.6. Testing your ssh key on Hotel
• Page
  • 3. STATUS
• « 2.2. Using SSH K...
• 3.1. Status »
• Source