Running Twister on FutureGrid HPC

Twister is widely used by domain scientists for running their scientific applications in parallel fashion. Here, we provide a detail instruction to use Hadoop on FutureGrid test-bed with FutureGrid India Cluster.


  1. FutureGrid HPC account, please apply via FutureGrid portal and request a HPC account.
  2. Linux command experience.

Get HPC compute nodes

Firstly, you will need to login to IU-india with using the command:

Login command: ssh –i [ssh private key] [username]
ssh –i key.pem
Enter passphrase for key 'ssh_id_rsa':
i136 >

Here, “i136” is the headnode (first login node) of IU-india cluster. You will obtain several HPC compute nodes from this node. Noted that, DO NOT run any of your program on the “i136” headnode. Then, on the command prompt, type the following command to get a set of compute nodes:

Obtain compute nodes command: qsub -I -l nodes=[numOfComputeNodes]:ppn=[CpuPerNode],walltime=[hh]:[mm]:[ss]

i136 > module load torque
i136 > qsub -I -l nodes=2:ppn=8,walltime=03:00:00
qsub: waiting for job 44418.i136 to start
qsub: job 44420.i136 ready
i55 > cat $PBS_NODEFILE

As shown above, “i55” and “i56” are the assigned compute nodes where these nodes information are stored to environment parameter $PBS_NODEFILE.

Twister Configuration

We provide a detail instruction to startup twister 0.9. In this example, we use i55 as Driver/Master node, i56 as Worker/Slave node.

Download Twister 0.9

i55 > wget
i55 > tar -zxvf Twister-0.9.tar.gz

Set $TWISTER_HOME, $JAVA_HOME and Worker nodes

Before running Twister configuration script, we need to setup several environment parameters:

i55 > echo "export TWISTER_HOME=/N/u/taklwu/twister-0.9" >> ~/.bashrc
i55 > echo "export JAVA_HOME=/N/soft/jdk1.6.0_20-x86_64" >> ~/.bashrc
i55 > source ~/.bashrc
i55 > vi $TWISTER_HOME/bin/nodes


Within twister 0.9 package, there is a script to automatically configure Twister. Generally, it randomly pick one of the working node as ActiveMQ messaging broker, set working daemon per node, and worker (mapper/reducer) per daemon. Also, it creates Twister required directories such as app_dir and data_dir.

i55 > cd $TWISTER_HOME/bin
i55 > ./
use normal MultiNode Setup
no special processing to nodes
ActiveMQ uri=failover:(tcp://i56:61616)
i55:/N/u/taklwu/twister-0.9/data created.
i56:/N/u/taklwu/twister-0.9/data created.
Change max memory to 16054 MB
copied to i55:/N/u/taklwu/twister-0.9
copied to i56:/N/u/taklwu/twister-0.9
Auto configuration is done.

As shown in the message above “ActiveMQ uri=failover:(tcp://i56:61616)”, i56 is the selected node where ActiveMQ messaging broker will be started.

Download and start ActiveMQ on specific nodes

Now ssh to the selected node, i56, then download and unzip the ActiveMQ package, finally start it up and return the previous master node, i56.

i55 > ssh i56
i56 > wget
i56 > cd apache-activemq-5.4.2/bin
i56 > ./activemq console &
[1] 4009
[taklwu@i56 bin]$ INFO: Using default configuration
(you can configure options in one of these file: /etc/default/activemq /N/u/taklwu/.activemqrc)
INFO: Invoke the following command to create a configuration file
./activemq setup [ /etc/default/activemq | /N/u/taklwu/.activemqrc ]
INFO: Using java '/N/soft/jdk1.6.0_20-x86_64/bin/java'
INFO: Starting in foreground, this is just for debugging purposes (stop process by pressing CTRL+C)
Java Runtime: Sun Microsystems Inc. 1.6.0_20 /N/soft/jdk1.6.0_20-x86_64/jre
  Heap sizes: current=251264k  free=247327k  max=251264k
    JVM args: -Xms256M -Xmx256M -Dorg.apache.activemq.UseDedicatedTaskRunner=true -Dactivemq.classpath=/N/u/taklwu/apache-activemq-5.4.2/conf; -Dactivemq.home=/N/u/taklwu/apache-activemq-5.4.2 -Dactivemq.base=/N/u/taklwu/apache-activemq-5.4.2
ACTIVEMQ_HOME: /N/u/taklwu/apache-activemq-5.4.2
ACTIVEMQ_BASE: /N/u/taklwu/apache-activemq-5.4.2
Loading message broker from: xbean:activemq.xml
 INFO | Refreshing org.apache.activemq.xbean.XBeanBrokerFactory$1@245e13ad: startup date [Sun Oct 30 23:33:22 EDT 2011]; root of context hierarchy
 WARN | destroyApplicationContextOnStop parameter is deprecated, please use shutdown hooks instead
 INFO | PListStore:/N/u/taklwu/apache-activemq-5.4.2/data/localhost/tmp_storage started
 INFO | Using Persistence Adapter: KahaDBPersistenceAdapter[/N/u/taklwu/apache-activemq-5.4.2/data/kahadb]
 INFO | KahaDB is version 3
 INFO | Recovering from the journal ...
 INFO | Recovery replayed 1 operations from the journal in 0.0080 seconds.
 INFO | ActiveMQ 5.4.2 JMS Message Broker (localhost) is starting
 INFO | For help or more information please see:
 INFO | Listening for connections at: tcp://i56:61616
 INFO | Connector openwire Started
 INFO | ActiveMQ JMS Message Broker (localhost, ID:i56-56404-1320032003342-0:1) started
 INFO | jetty-7.1.6.v20100715
 INFO | ActiveMQ WebConsole initialized.
 INFO | Initializing Spring FrameworkServlet 'dispatcher'
 INFO | ActiveMQ Console at
 INFO | Initializing Spring root WebApplicationContext
 INFO | camel-osgi.jar/camel-spring-osgi.jar not detected in classpath
 INFO | Apache Camel 2.4.0 (CamelContext: camel) is starting
 INFO | JMX enabled. Using ManagedManagementStrategy.
 INFO | Found 4 packages with 15 @Converter classes to load
 INFO | Loaded 146 type converters in 0.337 seconds
 INFO | Connector vm://localhost Started
 INFO | Route: route1 started and consuming from: Endpoint[activemq://example.A]
 INFO | Started 1 routes
 INFO | Apache Camel 2.4.0 (CamelContext: camel) started in 0.783 seconds
 INFO | Camel Console at
 INFO | ActiveMQ Web Demos at
 INFO | RESTful file access application at
 INFO | Started SelectChannelConnector@
i56 > exit
i55 >

Start Twister

After you go back to the master node (i55), simply type command ./ & under $TWISTER_HOME/bin.

i55 > ./ &
[1] 7844
Oct 30, 2011 11:34:38 PM org.apache.activemq.transport.failover.FailoverTransport doReconnect
INFO: Successfully connected to tcp://i56:61616
1    [main] INFO  cgl.imr.worker.DaemonWorker  - Daemon no: 0 started with 8 workers.
Oct 30, 2011 11:34:39 PM org.apache.activemq.transport.failover.FailoverTransport doReconnect
INFO: Successfully connected to tcp://i56:61616
0    [main] INFO  cgl.imr.worker.DaemonWorker  - Daemon no: 1 started with 8 workers.

[1]+  Done                    ./

If you can see similar message above, twister has started successfully.

Verify Twister Status

Also you can use command “jps” on each node to make sure Twister (TwisterDaemon) is running.

On master node

# on master node
i55 > jps
7878 TwisterDaemon
7909 Jps

On slave and ActiveMQ node

# on slave and ActiveMQ node
i55 > jps
4265 Jps
4025 run.jar
4185 TwisterDaemon

Next step, try out the Twister Applications section to run Twister Kmeans or Twister Blast.