Using Inca¶

Emulate a User¶

Many monitoring systems run from the perspective of the system administrators and do not test beyond whether a port is open or not. While important, we have found that it is really important to test from the perspective of the user to ensure the software is properly installed and is accessible to users. Therefore, Inca runs from a regular user account and in practice is setup like any other user account. Oftentimes, commands that are printed in the user guides or manual are good tests that can be wrapped into an Inca reporter. Even just one or two simple unit tests can be effective in detecting installation or permission problems.

Consider Load¶

Ideally, we would like to monitor frequently in order to quickly detect problems before the users notice them. However, active measurements introduce load into the system and it is important to not interfere with users. Therefore consider the measurement you are collecting and select a measurement frequency that balances freshness of results with impact on the system; i.e., active measurements that introduce minimal load on the system can be run at a higher frequency than active measurements that introduce medium to high load to the system.?If applicable, you may consider decomposing a measurement into multiple parts so that the lightweight parts that can be run more frequently. For example, checking that a VM is successfully deployed via RAIN introduces load to the system and should probably not be executed more than once every few hours. ?However, we could run a test that queries the list of available VMs every 10 minutes with minimal load to the system.

Test and/or Measure¶

The easiest way to get started with monitoring your component is to write a simple unit test.?This test can be a simple command from the user documentation that every user is likely to run and you can likely leverage an existing Inca reporter to test it as discussed below. Once you gain some familiarity with Inca, you can develop more intricate tests to verify more functionality of your component or to detect frequently occurring problems users are reporting.?As you develop more intricate tests, consider the performance of your component and the data that would be useful to collect in order to detect performance problems and identify bottlenecks.?For example, to measure the performance of RAIN’s invocation mechanism, we would like to know: 1) the time to transfer a VM to the resource, 2) the time to submit the request to Eucalyptus/Nimbus/XCAT, 3) the time before the VM is pingable, and 4) the time before the VM is login-able. Inca provides a flexible data model so any information you can express in a XML document is supported and is described further below. Email notifications can be set if performance drops beyond a threshold and the data can be viewed through the Inca web pages.

Download APIs and Reporters¶

To access the APIs, first download the tar.gz file such as

% wget http://inca.sdsc.edu/releases/latest/Inca-Reporter.tar.gz

Then unzip/untar the tar.gz file

% tar zxvf Inca-Reporter.tar.gz
% ls Inca-Reporter-*
COPYRIGHT  ChangeLog  MANIFEST  Makefile  Packages.gz  bin  cgi-bin  docs  etc  lib  sbin  share  version

The reporters are located in the ‘bin’ directory and the Perl and Python APIs are in ‘lib/perl’ and ‘lib/python’ respectively. Next checkout the latest reporters developed from svn and put them in a ‘bin.svn’ directory

% svn checkout https://futuregrid.svn.sourceforge.net/svnroot/futuregrid/core/trunk/src/performance/benchmarking/framework/inca/bin bin.svn
A    bin.svn/performance.infiniband
A    bin.svn/futuregrid.cluster.partition
A    bin.svn/performance.hpcc
Checked out revision 487.

How to Execute a Reporter¶

All reporters can be run on the command line and will print XML as shown below. ?Help information for a reporter can be printed by adding argument ‘-help’ which will by default print the help info in XML.?To print it in text format, also add ‘-verbose=0’ argument.?For information about the report XML schema, please see the Inca User Guide.

% env PERL5LIB=lib/perl PYTHONPATH=lib/python ./bin/grid.wget.unit -page="http://inca.sdsc.edu"
<?xml version='1.0'?>
<rep:report xmlns:rep='http://inca.sdsc.edu/dataModel/report_2.1'>
  <gmt>2010-10-29T22:53:59Z</gmt>
  <hostname>client64-139.sdsc.edu</hostname>
  <name>grid.wget.unit</name>
  <version>7</version>
  <workingDir>/tmp/Inca-Reporter-5.12450</workingDir>
  <reporterPath>./bin/grid.wget.unit</reporterPath>
  <args>
    <arg>
      <name>help</name>
      <value>no</value>
    </arg>
    <arg>
      <name>log</name>
      <value>0</value>
    </arg>
    <arg>
      <name>page</name>
      <value>http://inca.sdsc.edu</value>
    </arg>
    <arg>
      <name>verbose</name>
      <value>1</value>
    </arg>
    <arg>
      <name>version</name>
      <value>no</value>
    </arg>
    <arg>
      <name>wgetArgs</name>
      <value></value>
    </arg>
  </args>
  <body>
    <unitTest>
      <ID>wget</ID>
    </unitTest>
  </body>
  <exitStatus>
    <completed>true</completed>
  </exitStatus>
</rep:report>

% env PERL5LIB=lib/perl PYTHONPATH=lib/python ./bin/cluster.uptime.performance
<?xml version='1.0'?>
<rep:report xmlns:rep='http://inca.sdsc.edu/dataModel/report_2.1'>
 <gmt>2010-10-29T22:55:33Z</gmt>
 <hostname>client64-139.sdsc.edu</hostname>
 <name>cluster.uptime.performance</name>
 <version>1</version>
 <workingDir>/private/tmp/Inca-Reporter-5.12450</workingDir>
 <reporterPath>./bin/cluster.uptime.performance</reporterPath>
 <args>
  <arg>
    <name>help</name>
    <value>no</value>
  </arg>
  <arg>
    <name>log</name>
    <value>0</value>
  </arg>
  <arg>
    <name>verbose</name>
    <value>1</value>
  </arg>
  <arg>
    <name>version</name>
    <value>no</value>
  </arg>
 </args>
 <body>
  <performance>
    <ID>load average</ID>
    <benchmark>
      <ID>load average</ID>
      <statistics>
        <statistic>
          <ID>fifteen minute</ID>
          <value>0.30</value>
          <units>fraction</units>
        </statistic>
        <statistic>
          <ID>five minute</ID>
          <value>0.30</value>
          <units>fraction</units>
        </statistic>
        <statistic>
          <ID>one minute</ID>
          <value>0.30</value>
          <units>fraction</units>
        </statistic>
      </statistics>
    </benchmark>
  </performance>
 </body>
 <exitStatus>
  <completed>true</completed>
 </exitStatus>
</rep:report>

Using an API¶

Inca accepts any type of data that can be expressed as an XML document with a size limit of 12,000 chars.?The Inca reporter APIs provide will help you print out XML and wrap it within the Inca schema. There are also a number of convenience APIs for writing specific types of reporters. The advantage of using one of these APIs is that the Web status pages already know how to display the data (e.g., in graphs).

Links to the API documentation are below:

• Perl API
• Python API

Please see the Reporter section of the Inca user guide as well.

• Inca User Guide

Often the easiest way to get started is to find a reporter similar to the one you want to write, copy it, and then modify it.  Some common reporter types are shown below.

#!/usr/bin/env perl
use strict;
use warnings;
use Inca::Reporter::Performance;
# create reporter object and parse args
my $reporter = new Inca::Reporter::Performance(
name => 'grid.wget.performance',
version => 7,
description => 'Reports the bandwidth reported by wget',
url => 'http://inca.ucsd.edu',
measurement_name => 'wget'
);
$reporter->addArg('page', 'web page url to download', 'http://cnn.com/index.html');
$reporter->addArg('wgetArgs', 'wget arguments (e.g. --no-check-certificate)', );
$reporter->processArgv(@ARGV); my $page = $reporter->argValue('page');
my $args = $reporter->argValue('wgetArgs');
# run command and parse data
my $wgetCmd = "wget $args -O /dev/null $page";
my $output = $reporter->loggedCommand($wgetCmd);
if($? || ($output !~ /([\d.]+) \wB\/s/)) {
my $err = "wget command failed: $output $!";
$err =~ s/--\d{2}:\d{2}:\d{2}--/--xx:xx:xx--/g;
$reporter->setResult(0, $err);
} else {
# format data in XML
my $benchmark = $reporter->addNewBenchmark('download');
$benchmark->setParameter('page', $page);
my ($pre, $unit) = $output =~ /([\d.]+) (.B\/s)/;
$benchmark->setStatistic('bandwidth', $1, $unit);
$reporter->setResult(1);
}
$reporter->print();

#!/usr/bin/env perl

use strict;
use warnings;
use Inca::Reporter::SimpleUnit;

my $BACKOFF_SECS = 30;
my $reporter = new Inca::Reporter::SimpleUnit(
  name => 'data.access.postgres.unit.connect',
  description => 'Tries to connect to postgres server and list databases',
  version => 8,
  unit_name => 'postgres connect'
);
$reporter->addArg('exec', 'postgres user account', 'psql');
$reporter->addArg('user', 'postgres user account', '');
$reporter->addArg('numTries', 'number of connect attempts before giving up', 5);
$reporter->processArgv(@ARGV);
my $user = $reporter->argValue( 'user' );
$user = "-U $user" if $user ne '';
my $psql = $reporter->argValue( 'exec' );
my $numTries = $reporter->argValue( 'numTries' );

my $success = 0;
my @output;
my $i;
for( $i = 0; $i < $numTries && ! $success; $i++ ) {
  sleep( $BACKOFF_SECS * $i ) if $? != 0;
  @output = $reporter->loggedCommand("$psql $user -l");
  $success = 1 if $?

Commit New Reporter¶

Once you have written a new reporter as described in the previous section, commit it the FutureGrid SVN directory at: https://futuregrid.svn.sourceforge.net/svnroot/futuregrid/core/trunk/src/performance/benchmarking/framework/inca/bin

Then email Shava Smallen <ssmallen@sdsc.edu> and specify the name of the reporter, the resources where you want it deployed, the argument values, and execution frequency.

If you are making a change to an existing reporter, email ssmallen@sdsc.edu and specify the name of the reporter. ?More automated methods for doing this will come later.