CS270 -- File Systems Class Project

Rich Wolski --- Winter, 2022

• Eucalyptus CLI Documentation

  this page last updated: Mon Oct 1 09:06:45 PDT 2018

  ---

  For the file system project, you will need to use the Eucalyptus cloud that the department has set up. This page will provide some guidance on how to use the cloud to implement your project.

  Getting access to Eucalyptus

  To get an account on the Eucalyptus cloud for CS270, you must email me with the following information
  • The name of your team
  • a list of the team members andf their email addressess
  I will send you and account ID and password once I have recorded your team information.

  Eucalyptus (like Amazon AWS) has a notion of accounts and users. Multiple users can be part of a single account. When an account is created, it is created with a single user with the id admin. For the purposes of this project, it is fine for all team members to share the admin user id to the team's account.

  Using the Eucalyptus Console

  There are two ways to access Eucalyptus: via a web-based console or via a set of command line tools. To use the console, point your browser to https://console.aristotle.ucsb.edu. The account name is the account ID that the TA sent to you. The user name is admin, and the password is the password that the TA sent.

  Please keep that password handy. The "forgot your password" button will not work because we have not attached the account to an email address. If there is a password mishap, email me or the TA and we can reset it for you manually.

  The Eucalyptus console is pretty self-explanatory and there are documents available on-line.

  Using the Command Line Tools

  You are free to use the console for this project. However if you would like to use the command-line tools, the remainder of this tutorial will explain how you can.

  The command-line tools require information from your account that you can access via the Eucalyptus console. The command-line tools are called euca2ools and there are several versions available. For this version of Eucalyptus you will need to make sure you are using euca2ools 3.4.1 or higher. This tutorial will use examples that have been tested on CSIL which is currently running 3.3.0.

  Getting Your Credentials from the Console

  These credentials give you access to the CS270 Eucalyptus cloud. Think of them as being similar to the PGP keys you get when you initialize your user account for ssh. You should put your Eucalyptus credentials in a directory that you protect like the .ssh directory (i.e. where you keep private PGP keys) -- without group or world readable permission.

  To set them up, create a directory in your home directory that will contain your credentials:

  mkdir ~/.euca
  

  I will be using ".euca" in my home directory as an example. It can be any directory but you should set the permissions to be accessible only by you (not group or world).

  Next, in that directory, create a eucarc.ini file. Using a text editor, create a text file that contains the following

  [region cloud.aristotle.ucsb.edu]
  autoscaling-url = http://autoscaling.cloud.aristotle.ucsb.edu:8773/
  bootstrap-url = http://bootstrap.cloud.aristotle.ucsb.edu:8773/
  cloudformation-url = http://cloudformation.cloud.aristotle.ucsb.edu:8773/
  ec2-url = http://ec2.cloud.aristotle.ucsb.edu:8773/
  elasticloadbalancing-url =
  http://elasticloadbalancing.cloud.aristotle.ucsb.edu:8773/
  iam-url = http://iam.cloud.aristotle.ucsb.edu:8773/
  monitoring-url = http://monitoring.cloud.aristotle.ucsb.edu:8773/
  properties-url = http://properties.cloud.aristotle.ucsb.edu:8773/
  reporting-url = http://reporting.cloud.aristotle.ucsb.edu:8773/
  s3-url = http://s3.cloud.aristotle.ucsb.edu:8773/
  sts-url = http://sts.cloud.aristotle.ucsb.edu:8773/
  user = XXX:admin
  
  [user XXX:admin]
  key-id = YYY
  secret-key = ZZZ
  account-id = XXX
  
  [global]
  default-region = cloud.aristotle.ucsb.edu
  

  Go ahead and cut-and-paste this text into your eucarc text file.

  Now go to the console and log in. You will need to get the account number and replace the string XXX with that number. You will also need to generate an ACCESS_KEY (YYY) and a SECRET_KEY (ZZZ). To do so, use the pull down tab in the upper right hand corner which will say "admin@youraccount". Choose the "Manage Credentials" menu option and scroll down to the "Generate Access Keys" section. When you click the "Create Access Keys" button two long strings will appear, one for the ACCESS_KEY and one for the SECRET_KEY. Eucalyptus won't show these to you again so cut-and-paste them into the eucarc file as replacements for YYY and ZZZ respectively.

  Now write the file and make sure it is only readable by you. The ACCESS_KEY and SECRET_KEY are the shared secrets that you need (like your ssh private key). You must protect them or your account can be compromised.

  Next, edit you ~/.bashrc file and add the lines

  eval `euare-releaserole`
  export AWS_DEFAULT_REGION=cloud.aristotle.ucsb.edu
  

  and write it out and type

  source ~/.bashrc
  

  This tells the tools to use the contact strings for aristotle.

  The shell is now enabled to access your account.

  Testing out access

  The CS department has graciously installed euca2ools on all of the CSIL machines so one simple way to work in this class is
  • log into csil.cs.ucsb.edu using ssh
  • create a directory for eucalyptus credentials
  • chmod 700 that directory
  • scp eucalyptus credentials to that directory
  If you go this route, you'll always need to log into a CSIL machine to access Eucalyptus but you won't need to do much with Eucalyptus itself so that might be okay.

  Turns out that euca2ools is available in packaged form for both CentOS/RHEL and OSX. If you have your own machine with either of the operating systems on it, or if you feel up to building them from source, you can try to get euca2ools running on your own machine. You will need version euca2ools 3.4.1 (Newton) will be using Eucalyptus 4.1.2 in this class. Unfortunately, it will not be possible to provide much help or guidance if you try and do a euca2ools install and it doesn't work. If that happens, resort to using the CSIL since the department is able to support it.

  Running a test command

  To test out your Eucalyptus access
  • log into a machine that has both your credentials and euca2ools installed
  • euca-describe-availability-zones
  If the command works, you should see something like

  rich@csil:~/.euca$ euca-describe-availability-zones 
  AVAILABILITYZONE	aristotle	available	
  AVAILABILITYZONE	ucsb	available	
  AVAILABILITYZONE	race	available
  

  To get this output, euca2ools sent a message to the Eucalyptus Cloud Controller that then interrogated an internal component to determine the configured availability zones. In this class there will only be one: nhdc. However, if your credentials were not valid you wouldn't have received this response. This is the "hello world" test for Eucalyptus.

  Eucalyptus Commands

  There are a bunch of Eucalyptus commands which are all documented and most, but not all, of them will work on this installation. When in doubt, just give it a try.

  However, for this class, though, you'll only need to be able to

  • launch and terminate an instance from a pre-installed image
  • create and delete storage volumes
  • attach storage volumes to an instance
  We'll go through these actions here. WARNING: Like AWS, Eucalyptus doesn't have many guard rails. In particular, it doesn't prompt the user for confirmation when a command could cause a potentially catastrophic (from your perspective) data deletion if that is what the command semantics say will happen. There are no backups for the Eucalyptus installation so if it is lost, it is lost. Be SURE that you are making your own backup copies of everything you cannot lose.

  Launching an Instance

  First, it is important to understand some of the cloud resource terminology that Eucalyptus uses. An instance is a VM running in Eucalyptus.

  Using Instances

  There are several steps that must be done when you first start using Eucalyptus before you run your first instance. They involve
  • getting the image ID
  • creating a keypair
  • enabling network port access
  Once you have done these things once, all subsequent instance launches can use the same configuration.

  First, though, it is important to understand some of the cloud resource terminology that Eucalyptus uses. An instance is a VM running in Eucalyptus. To launch an instance, you need to specify an image that contains the operating system software you'd like to have in the instance.

  That is, when you launch an instance, the instance needs an OS (in this class, the OS will be CentOS 6.4) which comes from an image.

  PLEASE READ THIS: Before moving on, it is important to understand that cloud instances are not persistent. That is, when you run an instance, anything you save in the instance will be lost when you shut the instance down.

  Think of an instance as being like a Linux process, except that it has local disk storage. When you kill a process or when a process exits, all of the memory in the process is lost. If you haven't written that memory to disk, then it is gone and you can't get it back.

  An instance is the same way except that it has a disk that is lost as well. When you launch and instance and you install your software, that software is written to a disk that is "ephemeral" which means it will be wiped clean by Eucalyptus when the instance shuts down and there is no way to recover the lost storage.

  Thus, in this class, you will need to make sure that you do not keep anything you cannot lose in your instance. You can either put the source code in GitHub and push it when ever you have code that you can't lose (e.g. very often) or create a volume (see below) that will persist across instances.

  However, instance storage is not persistent.. Please be sure you understand what this previous sentence means completely before moving on. If you are confused, contact me directly. It is vitally important that you not lose your work because you do not understand the cloud storage model.

  With this understanding firmly in mind, the first step, then, is to query Eucalyptus for the images that are currently installed so that you can determine the identifier of the image that you will specify in the launch request for an instance. To see the installed images, type

  euca-describe-images -a
  

  The output from this command is lengthy. In particular, a few colleagues have asked to use the cloud while for their own purposes and they have installed images of their liking. To see the image you'll need for this class, try

  euca-describe-images -a | grep CentOS6.8 | grep "emi"
  

  and you should see something like

  IMAGE	emi-13cbc6f7	000157332541/CentOS6.8	000157332541	available public	x86_64	machine				ebs	hvm
  

  That's the image you'll need. Specifically, to launch an instance from this image, you'll need the image ID which begins with the prefix "emi-". A simple way to get just this ID is

  euca-describe-images -a | grep CentOS6.8 | grep "emi" | awk '{print $2}'
  

  The image ID will not change unless we are forced, at some point, to reinstall Eucalyptus. Assuming that doesn't happen, you can use emi-13cbc6f7 for your instances in this class.

  Next, you will need to create an ssh keypair to use with your instance. When Eucalyptus launches your instance, it will automatically install an ssh key so that you can log into it once the instance is running. Access via ssh is the standard for Linux instances but you first need to create the ssh keys you'd like installed.

  To create a keypair run

  euca-create-keypair -f mykeyname.key mykeyname
  

  This command does two things. First, it generates an ssh key pair and sticks the private key in the file mykeyname.key. Second, it generates an internal name for the keypair and stores the public key inside Eucalyptus. When you launch an instance, you will specify the keyname and Eucalyptus will install the public key in the instance. You can then log in using the private key that has been stored in the file.

  Note that ssh on Linux doesn't like private keys with open permissions. Thus, it is a good idea to immediately

  chmod 600 mykeyname.key
  

  as soon as you create a key

  Also, Eucalyptus isn't big on user and account names. If there is a problem, I won't be able to tell which account or user "owns" a particular instance. The convention, then, is to make "keyname" something descriptive since the keyname shows up in a listing of running instances. In this class, you should make your "keyname" be the name of your team. That way, I can contact you before doing something to an instance that requires some "service."

  The final set up step is to tell Eucalyptus what network ports should be opened for the instance once it is launched. Typically, you want to enable port 22 for TCP since that is the port used by ssh and ICMP ping so you can tell if your instance is "up." Do set up these firewall rules, type

  euca-authorize default -P tcp -p 22 -s 0.0.0.0/0
  euca-authorize default -P icmp -s 0.0.0.0/0 -t -1:1
  

  These commands tell Eucalyptus to allow TCP or ICMP traffic from anywhere (TCP on port 22 only) for ICMP to reach all instances launched in the security group default.

  Launch an Instance

  With this setup, the command to launch an instance is

  euca-run-instances emi-13cbc6f7 -t m1.medium -k mykeyname
  

  This basic command will use the CentOS image and will inject the ssh key. It will also use the default security group so that the firewall rules you set up in the previous step will pertain to this instance.

  It takes a few moments (sometimes minutes) for Eucalyptus to get the instance running. After an instance launch, type

  euca-describe-instances
  

  from time to time. While the instance is starting up, the word pending will show up in the output of this command right next to keyname. Eventually, when the instance is ready, this field in the output will change to the word running. At that point, the instance is ready To log into it you will need the public IP address for the instance. All instances get an IP address which is accessible to networks outside of Eucalyptus, and a private IP address for internal use only. The public IP address is the first IP address listed by

  euca-describe-instances

  for an instance that is running. In this class, all public IP addresses will begin "169.231" and all instance-private IP addresses will begin "10.2."

  To DNS or not to DNS -- that is the question. This installation of Eucalyptus has DNS enabled which means that each instance gets a public DNS entry (which UCSB correctly advertises) and an internal private DNS entry. For reasons that will become clear later in this class when we discuss cloud computing, public DNS will resolve to the private IP address if the resolution takes place within an instance. This mode is called "split horizon" DNS and it is what AWS implements. However, it can make things quite confusing if you are trying to script things and your scripts (or worse your scripting language) don't (doesn't) handle DNS perfectly. Thus, to avoid these headaches, I use IP addresses exclusively when I work with clouds (Eucalyptus or otherwise). You are free to develop your own style -- just understand that DNS isn't everything you think it might be in this context.

  To log into the instance type:

  ssh -i keyname.key centos@public_ip
  

  CentOS images do not allow root logins as a default. Once you have logged in, type

  sudo -s root
  

  and you will become root. If you wish to enable the root login (thereby skipping the sudo step) then you can by typing

  sudo cp ~centos/.ssh/authorized_keys /root/.ssh/authorized_keys
  

  You should then be able to log in as

  ssh -i keyname.key root@public_ip
  

  Logging out of the instance doesn't stop the instance. That is, if you log out, and then come back later and run the ssh command, the instance should still be running. You can always check on the running status of your instance or get its public IP address by running

  euca-describe-instances

  .

  Terminating an Instance

  You are welcome to leave an instance running and to simply log out of it when you are not using it. However, power failures, hardware failures, and system administrator (i.e. your instructor) errors do occur with greater frequency in an academic environment than one might want. Make sure you work is saved in a safe place early and often. There are no backups for this system.

  If you want to shut an instance down, you need to get the instance ID. Run

  euca-describe-instances
  

  and get the instance ID from the second field of the line that begins with the word INSTANCE (it will begin with "i-"). For example,

  euca-describe-instances | grep mykeyname
  

  looks for an instance that was started with the key "mykeyname". It should generate something that looks like

  INSTANCE	i-67aa0bca	emi-13cbc6f7 euca-169-231-235-49.eucalyptus.cloud.aristotle.ucsb.edu euca-10-1-4-188.eucalyptus.internal	running	cs270-aristotle.key	0 m1.medium	2017-10-18T20:34:52.049Z	race monitoring-disabled	169.231.235.49	10.1.4.188 instance-store					hvm sg-e86a4b45			x86_64 
  

  shows the output when I run it. If I wanted to terminate this instance I'd type

  euca-terminate-instances i-67aa0bca
  

  As mentioned before, all data in the instance itself will be lost (including the disk storage) if I were to run this command.

  Creating Persistent Volumes

  Eucalyptus does have a way to create virtual disks that persist across the instance lifecycle. These disks are called volumes and the facility for managing them is called EBS (Eucalyptus Block Store). EBS is generally compatible with the AWS Elastic Block Store if you want to get more documentation.

  In this class, you will need to create an EBS volume to use for the disk storage in your file system. You can also create a volume to hold your code and this volume will persist if your instance is terminated for what ever reason. However, the machine inside Eucalyptus that is implementing EBS volumes might have a disk crash and there are no backups so it is best if you also keep your class code backed up elsewhere.

  To create a volume, you need to specify the availability zone and the size in GB. For this class, you should create volumes no bigger than 30GB in order to prevent us from having disk shortage problems. The command for creating a 30GB volume in AZ 'race' is

  euca-create-volume -z race -s 30
  

  Like instances, it takes a while for Eucalyptus to create your virtual resource. To see the status of the create request type

  euca-describe-volumes
  

  While Eucalyptus is working on your request, the status will be reported as creating. After the volume has been created successfully, the volume will be reported as available. Volumes also get assigned unique identifiers that begin with the string "vol-". You get the ID back from the create call and it is the second field in

  euca-describe-volumes

  .

  Attaching a Volume

  Once a volume is created and it is in the available state it can be attached to an instance so that the instance may access it. Volumes are not shared. That is, a volume may be attached to at most 1 instance at a time. When a volume is attached, its status shows up as in-use and Eucalyptus will deny additional attach requests for it until it is detached.

  To attach a volume to an instance you need

  • the volume ID from

    euca-describe-volumes

  • the instance ID from

    euca-describe-instances

  • a device name to use in the instance for the volume once it is attached
  Because of a bug in libvirt for KVM, the device name is ignored -- sort of (careful with this problem -- see below). However, you must specify it in the call and it is good to try and make it a sane device name. Eucalyptus is using KVM as a hypervisor in the CS270 cloud so device names begin "/dev/vd". The device "/dev/vda" is used by Eucalyptus when it creates the instance. Eucalyptus also initializes /dev/vdb (in this image), formats it as an ext3 file system, and mounts it as /mnt. You should choose a device name that uses a letter other than the letter "a" or the letter "b." For example,

  euca-attach-volume -i i-67aa0bca vol-c7e6ca6c -d /dev/vdb
  

  attached volume vol-c7e6ca6c to instance i-67aa0bca as what Eucalyptus thinks is device /dev/vdb. The only real way to tell when device KVM used, however, is to log into the instance before the attach, and type

  ls /dev/vd*
  

  Then do the attach, then repeat the

  ls

  and see what device name showed up. In this version of KVM (but not previous versions) it appears that it used the letters in order. Thus, the first volume attached shows up as /dev/vdb and then next as /dev/vdd and so on but you definitely want to verify this conjecture before depending on it. Sorry. We have spoken to the folks at Red Hat about this problem numerous times. No luck.

  Detaching a Volume

  You can either detach a volume explicitly with the

  euca-detach-volume

  command or by terminating the instance (the volume will be detached automatically as part of the terminate). However, Eucalyptus does not send a signal to the instance indicating that a terminate or detach is taking place. Thus, if your file system is mounted, and the state has not been flushed to disk, the detach or a terminate can leave the disk in a corrupt condition. Before detaching or terminating, then, it is best to log in, become root, and run the Linux

  sync

  command to allow the instance to flush disk state. Then go ahead with the detach or terminate.

  Image Conditioning

  Once the image is running, you may want to add software packages to it that aren't included in the base CentOS 6.4 image. CentOS and RHEL both use the Red Hat package manager system called rpm which has a convenient user front end called yum. Using the yum utility, you can install additional software as long as it is either part of the standard CentOS repositories or it is located in some repository and you've taught yum where to find it. I'll cover the standard stuff here and let you read the documentation and commentary on how to use yum if you need something beyond what is in the standard repos. You have root within your instance so it is entirely at your disposal. You may install what ever you like as long as it is
  • legal to do so
  • legally available to you, and
  • conforms to UCSB student policies
  For example, I'm a C programmer so I require gcc, make, and gdb. I also use git I like having the man pages installed. Finally, I am using fuse as you will be for this class project. The base CentOS 6.4 image doesn't have these packages installed by default. To install them type

  yum -y install gcc make git fuse fuse-devel man-pages gdb
  

  when logged in as root. If you plan to use Python, or Java, or some other technology, you may need to install them as well. The yum utility should take care of any package dependencies as long as what you want is available as part of the CentOS 6.4 distro. If you go off the reservation, however, you will need to make your own way.

  If you just use the image as it is provided, you will be using a version of Centos 6.8 circa 2013. To get the latest version of Centos 6, however, you need to do an update. The command is

  yum -y update
  

  which takes a while. I believe that the versions of the packages that come with Centos 6 without an update are sufficient to complete the project in C. I don't know, however, if there are important bug fixes to the dependencies that are needed to use FUSE with other languages so it is probably a good idea to update the image when you first start it just to be safe.