5.7. Eucalyptus
Eucalyptus is a software platform that implements
IaaS-style
cloud computing. Eucalyptus provides an Amazon Web Services
(AWS) compliant EC2 based web service
interface for interacting with the Cloud service. Additionally,
Eucalyptus provides services such as the AWS Compliant Walrus and a user
interface for managing users and images. The aim of this tutorial is to
give users an introduction of how to interact with Eucalyptus using the
Eucalyptus EC2 Interface
(Euca2ools). Complete
documentation can be found at the Eucalyptus
Website. A detailed
user guide is available
here.
As of May 1, 2014, FutureGrid is using Eucalyptus version 3.4
on sierra.futuregrid.org. This version of Eucalyptus requires
euca2ools 3.0.2.
After logging in to sierra, make sure to load the euca2ools module:
before using Eucalyptus. euca2ools 3.0.2
is part of Eucalyptus installation packages for CentOS 6 and RHEL 6
described here
Source code installation for other platforms is available here
Eucalyptus 3.4 accounts are provisioned for all FutureGrid users with a
valid portal account and project affiliation. There is no need to apply
for Eucalyptus accounts separately.
Obtaining Credentials
Credentials files are created by default and placed in your home
directory under .futuregrid folder on sierra. There are separate
credentials for each FutureGrid project. For instance, if you are
part of FutureGrid project numbers 100 and 200, there will be two
folders (fg100 and fg200) in .futuregrid/eucalyptus. Each project
folder contains a corresponding credentials zip file. So
fg100:username and fg200:username in Eucalyptus are two different
identities, but they are tied to the same FutureGrid user.
Find your credential zip file in
$HOME/.futuregrid/eucalyptus/fgNNN:
$ cd $HOME/.futuregrid/eucalyptus/fgNNN
$ unzip euca34-{cluster}-fgNNN-{username}.zip
Apply the eucarc file:
If you want to add Eucalyptus environment variables to your .bashrc
then, do this: (if you are planning to switch between different cloud
platforms, it is probably better to use source:
$ cat eucarc >> $HOME/.bashrc
$ source $HOME/.bashrc
Eucalyptus is available to FutureGrid Users on the Sierra
cluster. As we will see later, when we instantiate a Virtual Machine
(VM) we must select the type of VM Image that we are going to
use. The VM Image types available are:
Sierra:
INSTANCETYPE Name CPUs Memory (MB) Disk (GB) Used / Total Used %
INSTANCETYPE m1.small 1 256 5 1 / 24 4%
INSTANCETYPE t1.micro 1 256 5 1 / 24 4%
INSTANCETYPE m1.medium 1 512 10 1 / 12 8%
INSTANCETYPE c1.medium 2 512 10 1 / 12 8%
INSTANCETYPE m1.large 2 512 10 1 / 12 8%
INSTANCETYPE m1.xlarge 2 1024 10 1 / 12 8%
INSTANCETYPE c1.xlarge 2 2048 10 1 / 12 8%
INSTANCETYPE m2.xlarge 2 2048 10 1 / 12 8%
Use euca-describe-availability-zones to test the setup:
$ ssh portalname@sierra.futuregrid.org
Last login: Fri May 11 06:39:02 2012 from 129-79-49-230.dhcp-bl.indiana.edu
Welcome to sierra.futuregrid.org
torque/2.4.8 version 2.4.8 loaded
moab version 5.4.0 loaded
$ module load euca2ools
euca2ools version 3.0.2 loaded
(euca2ools) $ euca-version
euca2ools 3.0.2 (Sparta)
(euca2ools) $ source .futuregrid/eucalyptus/fgNNN/eucarc
(euca2ools) $ euca-describe-availability-zones
AVAILABILITYZONE euca34sierra 198.202.120.44 arn:euca:eucalyptus:euca34sierra:cluster:cc-s38/
List the existing images using euca-describe-images. Remember you
must use the --all option to see public images that you don’t
own:
(euca2ools) $ euca-describe-images --all
IMAGE eki-19543879 fg-kernel/vmlinuz-3.13.0-29-generic.manifest.xml ...
IMAGE eki-7EA73854 fg-kernel/vmlinuz-2.6.32-431.17.1.el6.x86_64.manifest.xml ...
IMAGE emi-763C3C2D fg-image/20140618-euca-centos-6_5.manifest.xml ...
IMAGE emi-F7473A3B fg-image/20140616-euca-ubuntu-14_04.manifest.xml ...
IMAGE eri-13F63599 fg-ramdisk/initrd.img-3.13.0-29-generic.manifest.xml ...
IMAGE eri-1E5C3571 fg-ramdisk/initramfs-2.6.32-431.17.1.el6.x86_64.img.manifest.xml ...
...
Note, if you omit the –all option you will only see your images (not
other images, even if they are public). This differs from the behavior
of previous versions.
Currently the following machine images have been known to work
- emi-763C3C2D
- OS: CentOS release 6.5
- Flavors: m1.small
- username: cloud-user
- emi-F7473A3B
- OS: Ubuntu 14.04 LTS
- Flavors: m1.small
- username: ubuntu
Out of consideration for other users, please use the smallest “flavor” that meets your needs. Resources are limited.
Before running a VM instance, you need to create at least one key
pair. This key pair will be injected into the VM, allowing you to SSH
into the instance. This is done using the euca-add-keypair command:
$ euca-add-keypair $USER-key > $USER-key.pem
Fix the permissions on the generated private key:
$ chmod 0600 $USER-key.pem
When executing the euca-add-keypair command you may run into problems
if you reuse a key name. To see if you have done this, please cat the
key with:
If you see a message that starts with:
-----BEGIN RSA PRIVATE KEY-----
things should be fine. However, if you see:
CreateKeyPairType: Keypair already exists: gvonlasz-key: Could not execute JDBC batch update
You have already created a keypair with that name in your project.
You can check if your key is already in the keypair list:
This will list all the keypairs in your project. If you need to create
a new key, you can delete the old one by:
$ euca-delete-keypair $USER-key
After that you can create a new one as described above.
Now you can start a VM using one of the pre-existing images. We have
provided images based on CentOS 6.5 and Ubuntu 14.04. You can find
these images with:
$ euca-describe-images --all | fgrep fg-image
It will show you an id starting with the prefix “emi-”.:
IMAGE emi-763C3C2D fg-image/20140618-euca-centos-6_5.manifest.xml 663263781684 available public x86_64 machine eki-7EA73854 eri-1E5C3571 instance-store paravirtualized
IMAGE emi-F7473A3B fg-image/20140616-euca-ubuntu-14_04.manifest.xml 663263781684 available public x86_64 machine eki-19543879 eri-13F63599 instance-store paravirtualized
Note that the image name is formatted with the date and linux
distribution name (centos or ubuntu) and version. As we are
continually updating images as security updates are released, the
details in the examples here may differ from what you see.
We use the emi id in the next step. Use the euca-run-instances command
to start the VM:
$ euca-run-instances -k $USER-key -n 1 -t m1.small emi-763C3C2D
RESERVATION r-F8BD3F92 597941386389 default
INSTANCE i-B2EF473E emi-763C3C2D 0.0.0.0 0.0.0.0 pending $USER-key 0 m1.small 2014-06-23T18:53:09.216Z euca34sierra eki-7EA73854 eri-1E5C3571 monitoring-disabled 0.0.0.0 0.0.0.0 instance-store paravirtualized
The initial state of the machine instance will be “pending” while it
is being provisioned and starting up. The euca-describe-instances
command can be used to check the status of the request. The following
image was assigned an ip address and is starting up, as demonstrated
by the “pending” status:
$ euca-describe-instances
RESERVATION r-F8BD3F92 597941386389 default
INSTANCE i-B2EF473E emi-763C3C2D 198.202.120.101 10.128.74.140 pending $USER-test-key 0 m1.small 2014-06-23T18:53:09.216Z euca34sierra eki-7EA73854 eri-1E5C3571 monitoring-disabled 198.202.120.101 10.128.74.140 instance-store paravirtualized
Once started, the status will change to “running”:
$ euca-describe-instances
RESERVATION r-F8BD3F92 597941386389 default
INSTANCE i-B2EF473E emi-763C3C2D 198.202.120.101 10.128.74.140 running $USER-test-key 0 m1.small 2014-06-23T18:53:09.216Z euca34sierra eki-7EA73854 eri-1E5C3571 monitoring-disabled 198.202.120.101 10.128.74.140 instance-store paravirtualized
If you need to delete a deployed VM, you can use the
euca-terminate-instances command with the instance id:
$ euca-terminate-instances i-B2EF473E
The ssh private key that was generated earlier can now be used to
login to the VM. We are using the CentOS image so we log in as the
user cloud-user:
$ ssh -i $USER-key.pem cloud-user@10.128.74.140
[cloud-user@ip-10-128-74-140 ~]$
Note, if you are using the Ubuntu 14.04 image, you will need to use
the username ubuntu instead of cloud-user.
If you see a message something like:
Warning: Identity file $USER-key.pem not accessible: No such file or directory.
this likely means that you are in a directory other than where your
key file is saved. Change to the directory containing your key file,
or use a fully-qualified path in the -i argument
[cloud-user@ip-10-128-74-140 ~]$ uname -a
Linux ip-10-128-74-140 2.6.32-431.17.1.el6.x86_64 #1 SMP Wed May 7 23:32:49 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
When you are finished with your instance, terminate it to release resources for other users:
$ euca-terminate-instances i-B2EF473E
You may continue to see your instance in a “terminated” state in the
output of euca-describe-instances for a period of time after you
terminate it. This is normal.
Note that each VM instance has a public IP address (starting with
198.202) and a private IP address (starting with 10.128). If
you need to access your VM from outside the sierra login node, you
will need to use the public IP address. These IP addresses are listed
in the output of the euca-describe-instances command.
On the VM, the network will show only the VM private IP address:
[cloud-user@ip-10-128-74-140 ~]$ ip addr show eth0
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether d0:0d:b2:ef:47:3e brd ff:ff:ff:ff:ff:ff
inet 10.128.74.140/27 brd 10.128.74.159 scope global eth0
inet6 fe80::d20d:b2ff:feef:473e/64 scope link
valid_lft forever preferred_lft forever
Please take care to only install images that you create yourself or
obtain from trustworthy sources. You are responsible for the images
you use on FutureGrid resources. Any instances that are detetermined
to contain malware, or to be spamming, or attacking other systems on
the network, etc. will be terminated without notice and deleted.
Eucalyptus provides several “starter” images in their github
repository. We
will use one of these to demonstrate basic image management. For more
image management information see the Image Tasks section of the
Eucalyptus documentation
We will use the Fedora 20 image. Note that we have found that some of
the other images at this site do not actually work.
The following steps are executed on the sierra login node. Be sure you
have run module load euca2ools and source eucarc before
proceeding.
First download the image archive:
$ wget http://emis.eucalyptus.com/starter-emis/euca-fedora20-fedora-2013.12.18-x86_64.tgz
Uncompress it:
$ tar xzf euca-fedora20-fedora-2013.12.18-x86_64.tgz
We can see that this archive includes kernel, ramdisk, and root partiton images:
$ tree euca-fedora20-fedora
euca-fedora20-fedora
|-- euca-fedora20-fedora-2013.12.18-x86_64.img
`-- kvm-kernel
|-- initramfs-3.11.10-301.fc20.x86_64.img
`-- vmlinuz-3.11.10-301.fc20.x86_64
Eucalyptus restricts kernel image registration to administrators, so
we have provided the kernel and ramdisk as public images.
We will need to bundle the image, kernel and ramdisk, upload the
bundle, and register it. When uploading the bundle, you must specify a
Walrus bucket with the -b argument to euca-upload-bundle. The
buckets fg-image, fg-ramdisk, and fg-kernel are reserved for
FutureGrid administrative use. We suggest using your Portal username
or project as a bucket name.
First we will find our kernel and ramdisk images:
$ euca-describe-images --all | grep fg-kernel
...
IMAGE eki-1FC235EF fg-kernel/vmlinuz-3.11.10-301.fc20.x86_64.manifest.xml 663263781684 available public x86_64 kernel instance-store
$ euca-describe-images --all | grep fg-ramdisk
...
IMAGE eri-7F3A3ACC fg-ramdisk/initramfs-3.11.10-301.fc20.x86_64.img.manifest.xml 663263781684 available public x86_64 ramdisk instance-store
We’ll use the 3.11.10 kernel and ramdisk which matches what was in the
image archive we downloaded.:
$ euca-bundle-image -i euca-fedora20-fedora/euca-fedora20-fedora-2013.12.18-x86_64.img --kernel eki-1FC235EF --ramdisk eri-7F3A3ACC --arch x86_64
Wrote manifest /var/tmp/bundle-Hf3IL2/euca-fedora20-fedora-2013.12.18-x86_64.img.manifest.xml
Use the generated xml manifest to upload the bundle. Remember to use
your own username or project as bucket name (-b argument):
$ euca-upload-bundle -b fg82 -m /var/tmp/bundle-Hf3IL2/euca-fedora20-fedora-2013.12.18-x86_64.img.manifest.xml
Uploaded fg82/euca-fedora20-fedora-2013.12.18-x86_64.img.manifest.xml
Finally, register the image:
$ euca-register fg82/euca-fedora20-fedora-2013.12.18-x86_64.img.manifest.xml -a x86_64 -n fc-20-image
IMAGE emi-59A4353C
The returned image ID can now be used to start instances with
euca-run-instances as described earlier (for this image you will
need to log in as user fedora). euca-describe-images shows this
as a private image now:
$ euca-describe-images
...
IMAGE emi-59A4353C fg82/euca-fedora20-fedora-2013.12.18-x86_64.img.manifest.xml 597941386389 available private x86_64 machine eki-1FC235EF eri-7F3A3ACC instance-store paravirtualized
To remove an image, first deregister it, then delete the bundle. The
argument value to use for -p is the prefix of the image manifest
name (everything before the .manifest.xml suffix):
$ euca-deregister emi-59A4353C
$ euca-delete-bundle -b fg82 -p euca-fedora20-fedora-2013.12.18-x86_64.img
FutureGrid
0.1