Product SiteDocumentation Site

CloudVeneto

Users Guide

Edition 2.3

Legal Notice

Copyright (c) 2018 INFN - "Istituto Nazionale di Fisica Nucleare" - Italy
All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Abstract

This document explains how to use the services of the CloudVeneto.
This document was heavily inspired from the CERN Cloud User Guide
Preface
1. Document Conventions
1.1. Typographic Conventions
1.2. Pull-quote Conventions
1.3. Notes and Warnings
2. We Need Feedback!
1. Overview of the CloudVeneto
1.1. Projects
1.2. Network access
1.3. Getting help
1.4. Acknowledge CloudVeneto / Scientific citations
2. Registration
2.1. Registration in the cloud service
2.1.1. Apply for an account
2.1.2. Apply for other projects
2.2. Manage project membership requests (only for project managers)
2.3. Administer project members (only for project managers)
2.4. Manage account renewals (only for project managers)
3. Getting Started
3.1. Access the Cloud through the Dashboard
3.2. Creating a keypair
3.3. Importing your keypair
3.4. Setting security group(s)
3.5. Password management
3.5.1. Foreword
3.5.2. Setting/changing password
3.6. Switching between projects
3.7. Accessing the Cloud with command line tools
3.8. Accessing the Cloud through the euca2ools EC2 command line tools
4. Managing Virtual Machines
4.1. Creating Virtual Machines
4.1.1. Improve reliability: creating Virtual Machines from Volumes
4.2. Accessing Virtual Machines
4.2.1. Logging to a VM
4.2.2. Copying files to a VM
4.2.3. Giving a VM public access (getting a floating IP)
4.2.4. Multi-user instances
4.3. Accessing other hosts/services from Virtual Machines
4.4. Flavors
4.5. Stopping and Starting VMs
4.6. Contextualisation
4.7. Resizing Virtual Machines
4.8. Snapshotting Virtual Machines
4.9. Deleting Virtual Machines
5. Managing Storage
5.1. Ephemeral storage
5.2. Volumes
5.2.1. Create a Volume
5.2.2. Using (attaching) a Volume
5.2.3. Detaching a Volume
5.2.4. Deleting a Volume
5.2.5. Sharing a volume between multiple (virtual) machines
5.3. Accessing storage external to the Cloud
6. Orchestration
6.1. Creating a template
6.2. Creating a stack
7. Managing Images
7.1. Public Images
7.1.1. Public Images for INFN Padova users
7.2. User Provided Images
7.3. Sharing Images
7.4. Building Images
7.5. Deleting Images
8. Creating Batch Clusters on the Cloud
8.1. Intro: the idea
8.2. Creating Batch Clusters on the Cloud with HTCondor
8.2.1. Prerequisites
8.2.2. The cluster configuration
8.2.3. Start the elastic cluster
8.2.4. How slave nodes are installed and configured
8.2.5. Use the elastic cluster
8.2.6. Use the elastic cluster to run docker containers
8.2.7. How to find the EC2 (AMI) id of an image
8.2.8. Restart the elastic cluster
8.2.9. Troubleshooting the elastic cluster
9. Some basics on Linux administration
9.1. Introduction
9.2. Setting up 'sudo'
9.3. Managing software through the package manager
9.3.1. Managing software on RH distributions
9.3.2. Managing software on DEB distributions
9.4. Adding a user to your VM
9.5. Formatting/resizing a volume you just attached
9.6. Automatically remount volumes on reboot
9.7. Specific instructions relevant for INFN-Padova users
9.7.1. Enabling INFN Padova LDAP based authentication on the Virtual Machine
9.7.2. Install Mathematica (only for INFN Padova users)
10. Miscellaneous topics
10.1. Migrating an image to another cloud
10.2. Migrating an instance to another project/cloud

Preface

1. Document Conventions

This manual uses several conventions to highlight certain words and phrases and draw attention to specific pieces of information.
In PDF and paper editions, this manual uses typefaces drawn from the Liberation Fonts set. The Liberation Fonts set is also used in HTML editions if the set is installed on your system. If not, alternative but equivalent typefaces are displayed. Note: Red Hat Enterprise Linux 5 and later include the Liberation Fonts set by default.

1.1. Typographic Conventions

Four typographic conventions are used to call attention to specific words and phrases. These conventions, and the circumstances they apply to, are as follows.
Mono-spaced Bold
Used to highlight system input, including shell commands, file names and paths. Also used to highlight keys and key combinations. For example:
To see the contents of the file my_next_bestselling_novel in your current working directory, enter the cat my_next_bestselling_novel command at the shell prompt and press Enter to execute the command.
The above includes a file name, a shell command and a key, all presented in mono-spaced bold and all distinguishable thanks to context.
Key combinations can be distinguished from an individual key by the plus sign that connects each part of a key combination. For example:
Press Enter to execute the command.
Press Ctrl+Alt+F2 to switch to a virtual terminal.
The first example highlights a particular key to press. The second example highlights a key combination: a set of three keys pressed simultaneously.
If source code is discussed, class names, methods, functions, variable names and returned values mentioned within a paragraph will be presented as above, in mono-spaced bold. For example:
File-related classes include filesystem for file systems, file for files, and dir for directories. Each class has its own associated set of permissions.
Proportional Bold
This denotes words or phrases encountered on a system, including application names; dialog box text; labeled buttons; check-box and radio button labels; menu titles and sub-menu titles. For example:
Choose SystemPreferencesMouse from the main menu bar to launch Mouse Preferences. In the Buttons tab, select the Left-handed mouse check box and click Close to switch the primary mouse button from the left to the right (making the mouse suitable for use in the left hand).
To insert a special character into a gedit file, choose ApplicationsAccessoriesCharacter Map from the main menu bar. Next, choose SearchFind… from the Character Map menu bar, type the name of the character in the Search field and click Next. The character you sought will be highlighted in the Character Table. Double-click this highlighted character to place it in the Text to copy field and then click the Copy button. Now switch back to your document and choose EditPaste from the gedit menu bar.
The above text includes application names; system-wide menu names and items; application-specific menu names; and buttons and text found within a GUI interface, all presented in proportional bold and all distinguishable by context.
Mono-spaced Bold Italic or Proportional Bold Italic
Whether mono-spaced bold or proportional bold, the addition of italics indicates replaceable or variable text. Italics denotes text you do not input literally or displayed text that changes depending on circumstance. For example:
To connect to a remote machine using ssh, type ssh username@domain.name at a shell prompt. If the remote machine is example.com and your username on that machine is john, type ssh john@example.com.
The mount -o remount file-system command remounts the named file system. For example, to remount the /home file system, the command is mount -o remount /home.
To see the version of a currently installed package, use the rpm -q package command. It will return a result as follows: package-version-release.
Note the words in bold italics above — username, domain.name, file-system, package, version and release. Each word is a placeholder, either for text you enter when issuing a command or for text displayed by the system.
Aside from standard usage for presenting the title of a work, italics denotes the first use of a new and important term. For example:
Publican is a DocBook publishing system.

1.2. Pull-quote Conventions

Terminal output and source code listings are set off visually from the surrounding text.
Output sent to a terminal is set in mono-spaced roman and presented thus:
books        Desktop   documentation  drafts  mss    photos   stuff  svn
books_tests  Desktop1  downloads      images  notes  scripts  svgs
Source-code listings are also set in mono-spaced roman but add syntax highlighting as follows:
package org.jboss.book.jca.ex1;

import javax.naming.InitialContext;

public class ExClient
{
   public static void main(String args[]) 
       throws Exception
   {
      InitialContext iniCtx = new InitialContext();
      Object         ref    = iniCtx.lookup("EchoBean");
      EchoHome       home   = (EchoHome) ref;
      Echo           echo   = home.create();

      System.out.println("Created Echo");

      System.out.println("Echo.echo('Hello') = " + echo.echo("Hello"));
   }
}

1.3. Notes and Warnings

Finally, we use three visual styles to draw attention to information that might otherwise be overlooked.

Note

Notes are tips, shortcuts or alternative approaches to the task at hand. Ignoring a note should have no negative consequences, but you might miss out on a trick that makes your life easier.

Important

Important boxes detail things that are easily missed: configuration changes that only apply to the current session, or services that need restarting before an update will apply. Ignoring a box labeled 'Important' will not cause data loss but may cause irritation and frustration.

Warning

Warnings should not be ignored. Ignoring warnings will most likely cause data loss.

2. We Need Feedback!

For any issues, questions, problems concerning this document please contact the CloudVeneto admins at

Chapter 1. Overview of the CloudVeneto

The CloudVeneto is an OpenStack based cloud.
It implements a IaaS (Infrastructure as a service): it allows the instantiation of Virtual Machines (VMs) of the desired environments (in terms of operating system, installed software, etc.) and of the desired flavors (in terms of processors, memory size, etc.).
Even if it is a single, logical Cloud service, its resources are spread in two different locations: the INFN Padova - University of Padova's "Dipartimento di Fisica e Astronomia" computing centre, and INFN Laboratori Nazionali di Legnaro (LNL).
The CloudVeneto is currently based on the Ocata version of the OpenStack middleware.

1.1. Projects

Projects (also known as tenants) are organizational units in the cloud. A project is used to map or isolate users and their resources.
Projects are given quotas on resource usage, in terms of virtual machines, cores, memory, storage, etc.
A project can have a single user as member (personal project) but the typical use case are shared projects, with multiple users as members, which can map to experiments, organizations, research groups, etc. A user can be on multiple projects at the same time and switch between them.
In the CloudVeneto projects usually map to experiments or other research groups. Among the project's users there is a project manager (usually the team leader) who is responsible to manage (accepting or refusing) membership requests for the project.
A special shared project, called guest is used for all other users: it has a small quota to allow testing the Cloud infrastructure.

Warning

Personal private projects are discouraged and are created only for convincing reasons.

1.2. Network access

Access to Virtual Machines created in the CloudVeneto depends on your affiliation.
  • INFN users:
    VM are by default “visible” from the Local Area Networks (LANs) of both Padova and LNL. This means that e.g. users can access via ssh VMs of the Cloud directly from their desktops located in Padova or Legnaro. It is then possible to control which services/ports can be accessed using the security groups (discussed later) and firewalls on the relevant VMs.
    If it is necessary to log on a VM of the Cloud from the Internet, it is necessary to go through a gate machine. For this purpose users can rely on the existing gate hosts in Padova and Legnaro. Users who don't have an account on such gate hosts can rely on a Cloud specific gate host (gate.cloudveneto.it): please contact if you need an account on such machine.
    If needed, e.g. if a VM should host a service accessible from the Internet, such VM on the Cloud can be given a public IP. If this is the case please contact .
  • UNIPD users:
    VM may be accessed from a gate (gate.cloudveneto.it): please contact if you need an account on such machine.
    If needed, e.g. if a VM should host a service accessible from the Internet, such VM on the Cloud can be given a public IP. If this is the case please contact .
  • "External" users:
    Please contact
From a VM of the Cloud, it is possible to access the Internet, while by default it is not possible to access a host/service hosted in the INFN Padova or Legnaro LANs. If, for some reasons, you need to access some services hosted in Padova or Legnaro from the Cloud, please contact .

1.3. Getting help

If you have problems using the CloudVeneto cloud, please contact the admins at .

Warning

The CloudVeneto provides and infrastructure where you can instantiate your servers but, once activated, your server are managed by you.
This implies that the support crew don't provide support on topics like:
  • How to install/compile/configure your software;
  • ssh / scp basic usage (ask your Department / Institution technicians about that);
  • Basic linux usage (except what's described in this guide);
  • Accessing your VM 'the graphical way'.
INFN-Padova computing and Network service can provide support only to INFN-Padova users and only for instances created using "blessed" images, (as described in Section 7.1, “Public Images”).
Experiences, problems, best practices, etc. can be shared with the other users of the CloudVeneto using the mailing list. By default all CloudVeneto users are member of this mailing list. If you want to be removed from this mailing list please send an e-mail to , writing in the body of the mail:
unsubscribe discuss <your-email-address>
Changes and planned interventions to the service will be posted on the . All registered users to the Cloud are member of this mailing list.

1.4. Acknowledge CloudVeneto / Scientific citations

We kindly ask you to acknowledge the usage of the CloudVeneto infrastructure in any scientific publication or elsewhere. The following quote can be used:
"CloudVeneto is acknowledged for the use of computing and storage facilities."

Chapter 2. Registration

To be able to use the CloudVeneto service, first of all you need to apply for an account. The procedure to be followed is described in this chapter.

2.1. Registration in the cloud service

2.1.1.  Apply for an account

The registration procedure in the cloud is managed through the Horizon Openstack web service interface.
Go to https://cloud-areapd.pd.infn.it/dashboard or https://cloudveneto.ict.unipd.it/dashboard in a browser. The following page should appear:
Click on the Register button. The following page should appear:

2.1.1.1. Enrollment procedure through INFN AAI or UniPD SSO

Once authenticated on your Identity Provider system, you will be redirected to a form like this:
Fill the form with your phone number, the name of a contact person (i.e. the reference person from your Department/Institution), and if needed some notes.
UniPD users: your request will be accepted only if you specify a valid reference person from your Department/Institution. Please refer to the following table to contact the appropriate person:

Table 2.1. Table of reference persons for project creation

Department/Institution Referent
Physics and Astronomy Dept. prof. Alberto Garfagnini
Biology Dept. prof. Giorgio Valle
Geoscience Dept. prof. Manuele Faccenda
Information Engineering Dept. prof. Barbara Di Camillo
Civil and Environmental Engineering Dept. prof. Lorenzo Sanavia
Mathematics Dept. prof. Alessandro Sperduti
Molecular Medicine Dept. prof. Stefano Toppo
Biomedical Sciences Dept. prof. Silvio Tosatto
Chemical Sciences Dept. prof. Antonino Polimeno
Pharmaceutical Sciences Dept. prof. Stefano Moro

For what concerns the Project Action (projects have been discussed in Section 1.1, “Projects”) you have three options:
  • Select Existing Projects
  • Create new project
  • Use guest project
Choose Select Existing Projects if you want to apply membership for one or more existing projects (choose them in the relevant box).
Select Create new project if instead you want to ask the creation of a new project (and you are the leader of the experiment/research group associated to this project). In this case you will have to specify also a Project name and a Project Description. You will also have to specify if this project must be private (a personal project where you will be the only member) or not.
Newly created projects will get a default quota of:
  • 20 VCPU
  • 40 GB RAM
  • 200 GB for Volume (e.g. non ephemeral) storage
Allocation of more resources for UniPD users must be approved by the Cloud UniPD "governance". Mail your request to if you want more resources to be allocated to your project.

Note

Public (i.e. not private) projects are projects where other users can apply for membership. They are supposed to be used for experiments or other research groups.
Personal private projects are discouraged and are created only for convincing reasons.

Note

The person who asks for the creation of a new project is automatically defined as the manager of this project, i.e. he/she will have to manage the membership requests for this project. So the request to create a new project should be done by the relevant experiment/group leader.
Select Use guest project if you want to apply membership for the guest project, i.e. the project with a small quota used to test the Cloud infrastructure.
When you have filled the form, please read the AUP that you need to accept (by clicking the Accept AUP button) to be allowed to have an account.
Finally click on the Register button and you are done.
Your request will be managed by the Cloud adminstrator and by the manager(s) of the project(s) for which you applied membership. You will get an e-mail when your request is approved (and therefore you can start using the CloudVeneto) or if for some reason your request is refused.

2.1.1.2.  Apply for an account using User and Password

If and only if you don't have an account on the UniPD SSO or INFN AAI, click on the icon on the left:
Click then on the Register button below. A form such as the one of the following image will appear.
Fill the form with your personal data (First Name, Last Name, Email Address, Organization (e.g. 'INFN Padova'), Phone number. Choose a User name (please note that it could be changed by the Cloud admins during the registration process) and a Password. Specify the name of a contact person in Padova/Legnaro (e.g. your team leader), and optionally provide some other info in the 'Notes' field.
UniPD users: your request will be accepted only if you specify a valid reference person from your Department/Institution. Please refer to the following table to contact the appropriate person:

Table 2.2. Table of reference persons for project creation

Department/Institution Referent
Physics and Astronomy Dept. prof. Alberto Garfagnini
Biology Dept. prof. Giorgio Valle
Geoscience Dept. prof. Manuele Faccenda
Information Engineering Dept. prof. Barbara Di Camillo
Civil and Environmental Engineering Dept. prof. Lorenzo Sanavia
Mathematics Dept. prof. Alessandro Sperduti
Molecular Medicine Dept. prof. Stefano Toppo
Biomedical Sciences Dept. prof. Silvio Tosatto
Chemical Sciences Dept. prof. Antonino Polimeno
Pharmaceutical Sciences Dept. prof. Stefano Moro

For what concerns the Project Action you have three options:
  • Select Existing Projects
  • Create new project
  • Use guest project
Choose Select Existing Projects if you want to apply membership for one or more existing projects (choose them in the relevant box).
Select Create new project if instead you want to ask the creation of a new project. In this case you will have to specify also a Project name and a Project Description. You will also have to specify if this project must be private (a personal project where you will be the only member) or not.

Warning

Personal private projects are discouraged and are created only for convincing reasons.

Note

The person who asks for the creation of a new project is automatically defined as the manager of this project, i.e. he/she will have to manage the membership requests for this project. So the request to create a new project should be done by the relevant experiment/group leader.
Select Use guest project if you want to apply membership for the guest project, i.e. the project with a small quota used to test the Cloud infrastructure.
When you have filled the form, please read the AUP that you need to accept (by clicking the Accept AUP button) to be allowed to have an account.
Finally click on the Register button and you are done.
Your request will be managed by the Cloud adminstrator and by the manager(s) of the project(s) for which you applied membership. You will get an e-mail when your request is approved (and therefore you can start using the CloudVeneto) or if for some reason your request is refused.

2.1.2. Apply for other projects

After you have been given an account on the CloudVeneto, at any time you can ask the creation of a new project or the membership to an already existing project.
Both operations are performed by accessing the IdentityProjects tab of the OpenStack dashboard and clicking on Subscribe to project as depicted on the following image:
Fulfill your request selecting the relevant choice under the Project action dropdown list.

2.2. Manage project membership requests (only for project managers)

If you are the manager of a project, you will receive membership requests for this project that you will have to manage (approving or refusing them).
When a user applies to be member of a project that you manage, you will receive an e-mail such as this one:
To manage such requests, open the OpenStack web dashboard, log in, and then access IdentitySubscriptions An image such as the following one, with the list of the pending requests, will appear.
To approve a membership request, click on the Approve button (in Actions). A window such as the following one will appear:
Set the expiration date of the account, and click on the Ok button to approve the request.
If, instead, you want to reject the request, select Reject in Actions.

Note

It is therefore up to the project manager to set the expiration date of the members of his/her group.
A user belonging to multiple projects can have different expiration dates for the different projects he/she belongs to.

2.3. Administer project members (only for project managers)

If you are the manager of a project, you can list the members of your project and, if needed, change their role.
Open the OpenStack web dashboard, i.e. go to https://cloudveneto.ict.unipd.it/dashboard in a browser. Log in using the relevant method and access the IdentityProject Members panel. The list of users affiliated to your project will appear:
From here you can also change the role of a specific user (by clicking on 'Toggle Role') from 'Project User' to 'Project manager' or viceversa.

Note

If a user is promoted to Project manager, she will then be allowed to manage affilitation requests to the project, as described in Section 2.2, “Manage project membership requests (only for project managers)”.
From this window you can also remove a specific user from the project you manage.

2.4. Manage account renewals (only for project managers)

When the affiliation of a user for a project is expiring, as manager of that project you will receive an e-mail such as this one:
To renew the affiliation for that user, open the OpenStack web dashboard, log in, and then access IdentitySubscriptions. An image such as the following one will appear:
Click on the Renew button (in Actions). A window, such as the one represented in the following image will appear:
Set the new expiration date and then click the OK button.

Chapter 3. Getting Started

3.1. Access the Cloud through the Dashboard

Once you have been given an account, you can access the functionality provided by the Cloud. There are several ways of interacting with the Cloud. The simplest one is the dashboard, a web based GUI.
To access the production service of the CloudVeneto via the dashboard, you must simply go to https://cloudveneto.ict.unipd.it/dashboard/ or https://cloud-areapd.pd.infn.it/dashboard/ in a browser.
You can now log either using the INFN-AAI credentials, the University of Padova Single Sign On (SSO) system, or using the username and password.

3.2. Creating a keypair

You can now proceed to creating a key-pair. This is a secret key which will allow you to interact with your virtual machine once it is created. This key should be handled with similar security to a password or an ssh key so it should only be stored in a secure directory such as a private area in your home folder.
The steps are as follows:
  • Open the Compute tab on the left side
  • Select Key Pairs
  • In the Key Pairs section, select Create Key Pair.
You will need to give the keypair a name, such as my_key.
On completion of the operation, a file my_key.pem will be downloaded to your computer.

Warning

Be careful not to lose the file you just downloaded since there is no easy way to download it again.
This file should be stored in a safe location. To keep it private, run: chmod 600 my_key.pem

3.3. Importing your keypair

You might already have an ssh key you use to remotely access machines. This mean you already have under the .ssh directory in your home folder a couple of files named id_rsa (or id_dsa) and id_rsa.pub (or id_dsa.pub). If, on the machine you want to log on, your id_rsa.pub has been authorized you can access the machine without providing a password.
Importing your public key allow you to 'inject' it on any newly created VM in your project.
The steps are as follows:
  • Open the Compute tab on the left side
  • Select Key Pairs
  • In the Key Pairs section, select Import Key Pair.
You will need to give the keypair a name (your full username is a good choice), e.g. paolomazzon.
On the "Public Key" field paste the content of your id_rsa.pub file
Finally click on the Import Key Pair button

Warning

Be careful not to paste the content of your private key, the one without the '.pub' extension.
You can now use your key pair when instantiating a VM as an access method.

3.4. Setting security group(s)

Security groups are sets of IP rules (firewall) that define networking access and are applied to all instances within a project using that group. As described in Section 4.1, “Creating Virtual Machines”, when you create an instance you have to specify the security group to be used.
To set such IP rules, users can either add them to the default security group or can create a new security group with the desired rules.
For example the following procedure enables SSH and ICMP (ping) access to the default security group. The rules apply to all instances within a given project using this security group, and should be set (just once) for every project, unless there is a reason to inhibit SSH or ICMP access to the instances.
This procedure can be adjusted as necessary to add additional security group rules to a project, if needed.
  • Log in to the dashboard, choose a project, and click NetworkSecurity Groups. The security groups that are available for this project are shown.
  • Select the default security group and click Manage Rules.
  • To allow SSH access, click Add Rule.
  • In the Add Rule dialog box, enter the following values:
    Rule SSH
    Remote CIDR
    CIDR 0.0.0.0/0

    Note

    To accept requests from a particular range of IP addresses, specify the IP address block in the CIDR box.
  • Click Add.
  • To add an ICMP rule, click Add Rule.
  • In the Add Rule dialog box, enter the following values:
    Rule All ICMP
    Direction Ingress
    Remote CIDR
    CIDR 0.0.0.0/0
  • Click Add.

Warning

If you need to enable some services on a Virtual Machine, besides setting the specific IP rules through security groups, be sure that the relevant ports are also enabled (e.g. via firewalld) on the Virtual Machine.

3.5. Password management

3.5.1. Foreword

Warning

If you access the cloud either through UniPD SSO or INFN AAI you already have a password that you cannot change with this procedure.
You need to use this procedure only if:

3.5.2. Setting/changing password

From the OpenStack dashboard click on your user's name (on the top), select Settings from the dropdown menu and then Manage Password.

Important

Once again: this is the password to authenticate with the Cloud "internal" authentication mechanism. It is uncorrelated from the UniPD SSO or INFN-AAI one.

3.6. Switching between projects

As introduced in Section 1.1, “Projects”, a user can be on multiple projects at the same time. The current project is indicated by the top left dropdown menu near the logo.
To switch between projects just open the dropdown menu (as shown in the following figure) and select one of your available projects.

3.7. Accessing the Cloud with command line tools

It is possible to manage the Cloud using command line tools, even if most of the functionality provided by the Cloud can be accessed through the dashboard web interface. The documentation on the OpenStack site contains extended information on the syntax and installation procedure of the command line tools.

Note

INFN Padova users can find the OpenStack client installed on lx.pd.infn.it.

Important

Command line tools can only be used with the Cloud "internal" authentication mechanism. Even if you normally access the cloud dashboard using the UniPD SSO or INFN-AAI it's now time to set a password.
The OpenStack tools require a set of shell environment variables in order to run. These variables can be obtained from the dashboard and then stored in an 'rc' file that you can source (much like your .profile when logging into a linux server).
The environment variables are different for project you work on.
If you log into the dashboard, you will find API Access under the Compute menu on the left hand side.
Select among Download OpenStack RC file v2.0 and Download OpenStack RC file v3, to download the rc file for your current project (v2.0 or v3). The v3 openrc file requires a quite recent version of the Openstack client .

Warning

Because of a bug, if you downloaded the v2.0 rc file, you have to edit it and replace "v3" with "v2.0" in the OS_AUTH_URL variable setting
This file is different for each of the projects you are working on.
The downloaded rc file should be saved onto the machine you want to run the commands from. If you use csh rather than bash/zsh for your shell, you would need to create a new version using setenv rather than export.
Since the CloudVeneto services are secured using SSL, you will need the Digicert.pem "certification authority" file. This file can be downloaded from here.
Once you get the file you need to edit the RC file to set the OS_CACERT variable like this:
export OS_CACERT=/etc/grid-security/certificates/Digicert.pem

Note

The certificate can be put anywhere on the client as long as the path you specify is consistent.
To test it works, source the rc script file and enter your password to authenticate. The OpenStack command line tools can then be used, e.g.:
$ . SgaraPrj1-openrc.sh 
Please enter your OpenStack Password for project SgaraPrj1 as user sgaravat@infn.it: 
$ openstack server list
+--------------------------------------+--------------+--------+-------------------------+------------+
| ID                                   | Name         | Status | Networks                | Image Name |
+--------------------------------------+--------------+--------+-------------------------+------------+
| 89088351-90d8-4346-8ecf-ad08750b9d9a | tinies-uno-5 | ACTIVE | SgaraPrj1-lan=10.1.1.4  | cirros     |
| 44d12ad6-cc7e-47c3-a6d5-5e2b7c32d542 | tinies-uno-4 | ACTIVE | SgaraPrj1-lan=10.1.1.17 | cirros     |
| 02a40340-d238-4405-b5f3-3d38d9f9b485 | tinies-uno-3 | ACTIVE | SgaraPrj1-lan=10.1.1.12 | cirros     |
| bc6098c7-6ec6-4ac1-8aee-9e1edb33836a | tinies-uno-2 | ACTIVE | SgaraPrj1-lan=10.1.1.6  | cirros     |
| 1a1ab1b8-a3dd-401d-a8c2-cbd30b02e066 | tinies-uno-1 | ACTIVE | SgaraPrj1-lan=10.1.1.10 | cirros     |
| 4e2c6cc7-bc1c-49a1-951e-10b567247588 | dasgara1-2   | ACTIVE | SgaraPrj1-lan=10.1.1.11 | cirros     |
+--------------------------------------+--------------+--------+-------------------------+------------+
$

Note

When you source the rc script you are asked for a password. If the password is wrong, you will be told (with a generic authentication error) only when you issue some OpenStack commands.

3.8. Accessing the Cloud through the euca2ools EC2 command line tools

The CloudVeneto also exposes a EC2 compatible interface, which is a de-facto standard for computational clouds.
The euca2ools are command line tools that can be used to interact with an EC2 based cloud.
You can install the euca2ools package on your dekstop as follows:
CentOS / Fedora
  # yum install euca2ools
Ubuntu / Debian
  # apt-get install euca2ools

Note

INFN-Padova users can find the euca2ools installed on lx.pd.infn.it.
The euca2ools require a set of shell environment variables in order to run. These environment variables are different per project that you work on.
If you log into the dashboard, you will find API Access under the Compute menu on the left hand side.
Select the Download EC2 Credentials option to download the zip file for your current project. This zip file will be downloaded from the browser.
This file should be saved onto the machine where you want to run the commands from, and unzipped into a private directory, e.g:
$ unzip SgaraPrj1-x509.zip 
Archive:  SgaraPrj1-x509.zip
 extracting: pk.pem                  
 extracting: cert.pem                
 extracting: cacert.pem              
 extracting: ec2rc.sh
ec2rc.sh gives the variables for accessing the Cloud with EC2 APIs. If you use a C shell based shell, you would need to adapt this using setenv.
To test it, you can e.g. try the following:
$ . ec2rc.sh 
$ euca-describe-instances -I ${EC2_ACCESS_KEY} -S ${EC2_SECRET_KEY} -U ${EC2_URL}
RESERVATIONr-xvwmks74ee1865a76440481cbcff08544c7d580adefault
INSTANCEi-3b49020eami-2cfcb026tinies-uno-1runningsgaravat-ctest0m1.tiny2018-03-02T12:56:32Znova10.1.1.10instance-storesg-3896bec1
INSTANCEi-ebc7c470ami-2cfcb026tinies-uno-2runningsgaravat-ctest1m1.tiny2018-03-02T12:56:32Znova10.1.1.6instance-storesg-3896bec1
INSTANCEi-bdd57278ami-2cfcb026tinies-uno-3runningsgaravat-ctest2m1.tiny2018-03-02T12:56:32Znova10.1.1.12instance-storesg-3896bec1
INSTANCEi-e5bc209cami-2cfcb026tinies-uno-4runningsgaravat-ctest3m1.tiny2018-03-02T12:56:32Znova10.1.1.17instance-storesg-3896bec1
INSTANCEi-afc80fcdami-2cfcb026tinies-uno-5runningsgaravat-ctest4m1.tiny2018-03-02T12:56:32Znova10.1.1.4instance-storesg-3896bec1
RESERVATIONr-zpz5dkpnee1865a76440481cbcff08544c7d580adefault
INSTANCEi-e93ef61cami-2cfcb026dasgara1-2running1m1.tiny2018-01-16T08:36:44Znova10.1.1.11instance-storesg-3896bec1
$

Warning

For some euca2ools distributions sourcing the ec2rc.sh script is not enough. You need to explictly specify access and secret keys and the endpoint with the relevant command line options, e.g.:
$ euca-describe-instances -I ${EC2_ACCESS_KEY} -S ${EC2_SECRET_KEY} -U ${EC2_URL}

Chapter 4. Managing Virtual Machines

4.1. Creating Virtual Machines

To create a Virtual Machine (VM) using the dashboard, you need to have already logged into the dashboard, created your private key (as explained in Section 3.2, “Creating a keypair”) and set the security group (as discussed in Section 3.4, “Setting security group(s)”) to be used for this VM.
To create a VM proceed as follows:
  • Be sure you have selected the right Project from the dropdown menu on the top.
  • Go to ComputeInstances on the left hand menu. This will display a list of VMs currently running in your project.
  • Select the "Launch Instance" button. A new window appears.
    Here you can enter:
    • Instance name is the name of the machine you want to create.
    • Flavor is the size of the machine you want to create. These are specified using VCPUs (number of virtual CPUs), disk space for the system disk, size for the RAM memory. You are advised to start small (the flavor of a virtual machine can be changed later if required). Flavors are discussed in Section 4.4, “Flavors”.
    • Instance Count is the number of virtual machines to be started.
    • As Instance Boot Source select Boot from Image or Boot from Snapshot and then specify its name.
  • Switch to the Access & Security tab.
    • As Keypair select the keypair you created. This will allow you to log to the VM (usually as root or as an account where you can get admin privileges via sudo) using this SSH key.
    • You can also specify the admin (usually root) password of the instance.

      Warning

      Please note that setting the admin password is not guaranteed to always work (the image can't support the “injection” of this password). It is therefore strongly suggested to rely on the ssh-key mechanisms.
    • Specify the Security group to be used for this VM (security groups are discussed in Section 3.4, “Setting security group(s)”).
  • Now switch to the Networking tab.
    You should see one network called <ProjectName>-lan

    Note

    INFN users could see, besides the <ProjectName>-lan network, also a network called <ProjectName>-wan, if the possibility to use public IP numbers was requested. The former one must be selected if the VM doesn't need to be visible on the Internet. The <ProjectName>-wan network must be selected if instead the VM must have a public IP. It will then be necessary to allocate a public (floating) IP address to this instance, as explained in Section 4.2.3, “Giving a VM public access (getting a floating IP)”.
  • Select Launch to start the virtual machine being created. You will be returned to the Overview screen, where there will be a line with the instance name, ip adress and status. The status should be 'Active' once the install is complete.
Once the status of the machine is 'Active', you can watch the console to see it installing and booting. You can click on the VM name and go to a dedicated window or from this same table you can access a pull down menu on the right hand side under Actions. There you will see various options and among them View Log and Console.
For a Linux machine, select Console to access to the console of the VM.

Note

Virtual Machines instantiated on the Cloud by default aren't registered in the DNS. This means that you'll have to refer to them using their IP numbers.
For Virtual Machines supposed to have a long life, INFN Padova users may ask (contacting ) to have them registered in the DNS. If possible (i.e. if the chosen names are sensible enough and if there are no ambiguities) the registered names in the DNS will be the same as the ones chosen as Instance names.

4.1.1. Improve reliability: creating Virtual Machines from Volumes

By default Virtual Machines are instantiated using the local disk of the Cloud compute node. This means that, in case of failure of the Compute node, it may happen that the virtual machine content is lost.
For production servers which are not fully redundant and load balanced, to improve the availability it is advisable to use an external storage volume for the system disk of the virtual machine. The advantage is also that, if the compute node hosting the virtual machine has to be switched off e.g. for maintenance, the Cloud administrator before doing this operation can live-migrate the instance to another Cloud compute node basically without any service interruption.
The procedure is the following:
  • Create a volume using the desired image as source.
  • Launch a Virtual Machine with that volume as the system disk.
To create a volume from an image, go to the Volumes tab of the Dahsboard, and click on Create Volume.
After having set the Volume Name, set Image for Volume Source and specify the image name in the Use image as a source field.
Set the Size of the root disk (Size (GiB) field).
Complete the operation clicking on Create Volume.

Note

Please select 'ceph' (the default) as type for the volume
Once the volume has been created, you can launch the Virtual Machine.
In the Launch Instance form, that appears after having clicked on the Launch Instance button, please select Boot from volume in the Instance Boot Source field, and specify the name of the previously created volume in the Volume field.
Then proceed as explained for the creation of a "normal" instance.

Note

You can create only one virtual machine from a volume created using an image as source.

4.2. Accessing Virtual Machines

Warning

Please note that cloud VMs are not registered in the DNS, and therefore you must use their IP address.

4.2.1. Logging to a VM

Virtual machines created on the cloud have their IP assigned on a private network associated with the project they belong to, therefore they cannot be accessed directly from the internet. Conversely, there is no limitation on the 'outer' services you can reach from your VM (modulo the services hosted in the INFN Padova/Legnaro LANs, as described in Section 4.3, “Accessing other hosts/services from Virtual Machines”.
If you need to log on your VMs from the Internet you must go through the gate machine
gate.cloudveneto.it
When your account on the cloud is created you also got access to the gate. Contact in case of problems with this account.

Note

Projects that get their private network on the subnet 10.64.0.0/16 are peculiar: the VMs can be accessed from the INFN-Padova or INFN-Legnaro Local Area Networks (LANs). Those projects are created mostly for INFN users.
Assuming that
  • You created a Linux virtual machine using the <ProjectName>-lan network;
  • Your VM got the 10.67.15.3 IP address;
  • You know the default user on the VM. If, as an example, the VM is Ubuntu based there is a default user called 'ubuntu';
  • You stored your my_key keypair in ~/private on the gate machine;
you can access your VM from the gate machine issuing
       ssh -i ~/private/my_key ubuntu@10.67.15.3

4.2.2. Copying files to a VM

Copying files to your VM might be a little more complex since VMs don't have an access facing the internet. You can make it in a two step fashion:
  • Copy your file from your machine to the gate;
  • Copy file from the gate machine to your VM;
This is clearly not very practical.
A more clever approach is to use an SSH tunnel (port forwarding mechanism): you set up a special TCP port on your PC that act as an entrance point and then you inject your files in that port to get them copyed on your VM.
Proceed as follows: we suppose your VM IP address is 10.X.Y.Z.
  • Choose a free TCP port on your machine in the range 1025-65535 (e.g. 2002);
  • Set up the tunnel using your access to one of the gate machines of the cloud (e.g. gate.cloudveneto.it) and providing your TCP port of choice and the IP address of your remote VM:
          ssh -L2002:10.X.Y.Z:22 user@gate.cloudveneto.it
    
    The tunnel needs to stay open as long as you need to copy files toward the VM.
The syntax is explained in the following picture:
  • The green part is your side of the connection (e.g. the port opened on your PC);
  • user@gate..... is the transport part of the connection;
  • 10.X.Y.Z:22 in red is the remote end of the connection.
The blu arrows depict the data flow.
Once you have opened the tunnel you have, on your machine, a direct entry point to your VM. You can use the -P (capital 'p') parameter of the scp command and the remote username on your VM to copy files to the VM:
scp -P 2002 -i ~/private/my_key  my_local_file.txt  remoteuser@localhost:/remote/path/
Please note the remoteuser@localhost part of the command.
If sshfs is installed, you can use it with the tunnel with the following command
sshfs -p 2002 remoteuser@localhost:/remote/path /local/path -o IdentityFile=~/private/my_key
It is a file system client that mounts the remote file system locally.

Note

Tip 1: if the last part of your VM IP is Z, choosing 2000+Z as the local TCP port is a good way to memorize the (local port, remote VM) association of your tunnel.
Tip 2: you can open many ssh tunnels at once by simply repeating the -L 200Z:10.X.Y.Z:22 part of the command, e.g.
ssh -L2002:10.63.1.2:22 -L2003:10.63.1.3:22 -L2010:10.63.1.10:22 user@gate...
You can now specify 2002,2003 or 2010 after '-P' to get to the chosen VM.

4.2.3. Giving a VM public access (getting a floating IP)

If needed, e.g. if a VM should host a service accessible from the Internet, such VM on the Cloud can be given a public IP. For this purpose you will need:
Floating IP addresses can have their associations modified at any time, regardless of the state of the instances involved.
The following procedure details the reservation of a floating IP address from an existing pool of public addresses and the association of that address to a specific instance.
  • From the dashboard click Floating IPs on the Network menu.
  • Click Allocate IP To Project.
    Choose the pool from which to pick the IP address. You must choose:
    • INFN-WAN, if you are a INFN user (i.e. if the IP address of your instances is 10.64.x.y);
    • Unipd-WAN, if you are referring to a University of Padova project (i.e. if the IP address of your instances is 10.67.x.y);
    • CloudVeneto-WAN, in all other cases (i.e. if the IP address of your instances is 10.68.x.y).
    Then click Allocate IP.
  • Click on Associate for the just allocated floating IP.
  • In the Manage Floating IP Associations dialog box, choose the following options:
    • The IP Address field is filled automatically, but you can add a new IP address by clicking the + button
    • In the Ports to be associated field, select a port from the list (the list shows all the instances with their fixed IP addresses).
  • Finally click Associate.
To disassociate an IP address from an instance, click the Disassociate button.
To release the floating IP address back into the pool of addresses, click the More button and select the Release Floating IP option.

Note

By default the possibility to use public IP numbers is disabled and therefore by default it is not possible to allocate a floating IP to an instance. If public IPs are needed for your project, please contact specifying what is/are the relevant service(s) and the port(s) that need to be open.

Important

INFN users will have to create the VM on the <ProjectName>-wan network, if the VM must be given a public IP.
To control which services/ports of your virtual machine can be accessed, be sure you are using the right security group (as discussed in Section 3.4, “Setting security group(s)”) and you have correctly configured firewall (iptables, firewalld etc.) on the relevant VM.

4.2.4. Multi-user instances

By default a VM created by a user is accessible only by the user (also with admin privileges) who created it.
However please note that, as specified in Section 7.1.1, “Public Images for INFN Padova users”, VMs instantiated using the SL6x-INFNPadova-x86-64-<date> and CentOS7x-INFNPadova-x86-64-<date> images allow INFN-Padova system administrators to log (with admin privileges) on such virtual machines.
Please also note that if you have been given access to a virtual machine which was created by someone else, at least the person who created such instance has the "power" to read your data stored in the virtual machine.

Important

CloudVeneto AUP (Access Use Policy) allow to enable other users on the instance you created, but only if such users:
  • are CloudVeneto registered users or
  • have an account at INFN Padova or
  • have an account at INFN Laboratori Nazionali di Legnaro

4.3. Accessing other hosts/services from Virtual Machines

There is no limitation on the 'outer' services you can reach from your VM. However by default it is not possible to access a host/service hosted in Padova or Legnaro.
If, for some reasons, you need to access some services hosted in Padova or Legnaro from the Cloud, please contact .

4.4. Flavors

As shown in Section 4.1, “Creating Virtual Machines”, when an instance has to be created it is necessary to specify the flavor to be used for this VM.
Flavors define the virtual machine size such as:
  • Number of virtual CPU cores (VCPUs)
  • Amount of memory
  • Disk space
Information about the flavors can be seen in the Flavor Details box that appears in the Dashboard when you launch a new instance.

Warning

For what concerns VCPUs, please note that the CloudVeneto is configured to allow some “overbooking” so that a physical core is mapped to 4 VCPUs.
A disk size of 0 means that the size of the disk is the same as that in the image. For other cases, it may be necessary to resize the partitions.

Note

If you find that a specific flavor you require is not available, please contact .

4.5. Stopping and Starting VMs

VMs can be stopped and started in different ways available from the Actions menu of every instance found on the (ComputeInstances) table.

Warning

The cleanest way to shutdown (or reboot) an instance is however to log on the VM and issue from the shell the shutdown or reboot command. In fact if the Soft Reboot or Hard Reboot or Shutdown actions are chosen, there could be problems with networking when the VM is later restarted.
Pause/Unpause allows for temporary suspension of the VM. The VM is kept in memory but it is not allocated any CPU time.
Suspend/Resume stores the VM onto disk and recovers it later. This is faster than stop/start and the VM returns to the status is was when the suspend was performed as opposed to a new boot cycle.

4.6. Contextualisation

Contextualisation is the process to configure a virtual machine after it has been installed. Typical examples would be to create additional users, install software packages or call a configuration management system. These steps can be used to take a reference image and customize it further. Contextualisation is only run once when the VM is created.
Most of the available public images include a contextualisation feature using the open source cloud-init package.
With cloud-init, data to be used for contextualisation are called user data.
Using the Openstack command line tool, the --user_data option of the nova boot command must be used, e.g.:
                                                                                                  
nova boot my_vm --image "SL65-Padova-x86_64-20141023-QCOW2" \                                             
  --flavor m1.xsmall --user_data my_data.txt --key_name my_key
For example to run a command during contextualisation, the #cloud-config directive can be used:
                                                                                                  
cat > cern-config-users.txt << EOF                                                                  
#cloud-config                                                                                             
runcmd:                                                                                                   
 - [ /usr/bin/yum, "install", -y, "cern-config-users" ]                                                   
 - [ /usr/sbin/cern-config-users, --setup-all ]                                                           
EOF
User data can be provided as a gzip file if needed where the user data is larger than 16384 bytes, e.g.:
                                                                                                  
cat > userdata4zip.txt <<EOF                                                                        
#!/bin/sh                                                                                                 
wget -O /tmp/geolist.txt http://frontier.cern.ch/geolist.txt                                              
EOF                                                                                                       
gzip -c userdata4zip.txt > userdata4zip.gz                                                                
                                                                                                          
nova boot my_server --image "SL65-Padova-x86_64-20141023-QCOW2" \                                         
  --flavor m1.xsmall --user_data userdata4zip.gz --key_name my_key
With the #include or Content-Type: text/x-include-url directives, it is possible to specify a list of URLs, one url per line. The userdata passed by the urls can be plain txt, gzip file or mime-multi-part script. E.g.:
                                            
cat userdata.txt <<EOF                                                                              
#! /bin/bash                                                                                              
wget -O /tmp/robots.txt http://www.ubuntu.com/robots.txt                                                  
EOF                                                                                                       
                                                                                                          
cat > userdata4include.txt <<EOF                                                                    
#include                                                                                                  
# entries are one url per line. comment lines beginning with '#' are allowed                              
# urls are passed to urllib.urlopen, so the format must be supported there                                
http://frontier.cern.ch/userdata.txt                                                                      
EOF
cloud-init supply also a method called "multiple part" to supply user data in multiple ways, which means you can use userdata script and cloud-config (or other methods recognized by cloud-init) at the same time. cloud-init provides a script (write-mime-multipart) to generate a final userdata file. Here is an example:
                                                                                                  
cat > userdata4script <<EOF                                                                         
#! /bin/bash                                                                                              
mkdir -p /tmp/rdu                                                                                         
echo "Hello World!" > helloworld.txt                                                                      
EOF                                                                                                       
                                                                                                          
cat userdata4config                                                                                       
#cloud-config                                                                                             
runcmd:                                                                                                   
 - [ wget, "http://slashdot.org", -O, /tmp/index.html ]                                                   
                                                                                                          
cat userdata4include                                                                                      
#include                                                                                                  
# entries are one url per line. comment lines beginning with '#' are allowed                              
# urls are passed to urllib.urlopen, so the format must be supported there                                
http://frontier.cern.ch/userdata.txt
Then use write-mime-multipart to generate userdata4multi.txt and use it to launch an instance:
                                                                                                  
write-mime-multipart -o userdata4multi.txt userdata4script userdata4config userdata4inc                   
                                                                                                          
nova boot my_server --image "SL65-Padova-x86_64-20141023-QCOW2" \                                         
  --flavor m1.xsmall --user_data userdata4multi.txt --key_name my_key
On Internet a lot of documentation (along with examples) is available on cloud-init, such as in the Ubuntu Documentation.

4.7. Resizing Virtual Machines

If the size of a virtual machine needs to be changed, such as adding more memory or cores, this can be done using the resize operation. Using resize, you can select a new flavor for your virtual machine. The operation will reboot the virtual machine and might take several minutes of downtime, so this operation should be planned as it will lead to application downtime.
To resize a VM using the graphical Interface:
  • Detach any attached volume as decribed in Section 5.2.3, “Detaching a Volume”

    Warning

    Failure in doing so might lead to VM and/or Volume corruption!
  • Select the ComputeInstances menu and then Resize Instance option on the Actions.
  • In the Resize Instance box select the desired flavor.
  • After the new flavor has been selected, the status will become 'resize' or 'migrating'.
  • The status will change after several minutes to 'Confirm' or 'Revert Resize/Migrate'. You may need to refresh the web browser page to reflect the new status.
  • Select Confirm Resize/Migrate if you wish to change the instance to the new configuration.
The status will then change to 'Active' once completed.

4.8. Snapshotting Virtual Machines

It is possible to create a new instance from a previously saved snapshot of a VM.
The new instance can also have a different flavor from the source VM (but not smaller disk size!). It is therefore suggested to use the smallest (in terms of disk size) possible flavor for the source VM.
Before creating a snapshot you need to "clean up" the network configurations of the “source” VM. For a RedHat based OS (CentOS, Fedora, ...) this can be done using the following instructions, once logged as root on the “source” VM:
/bin/rm /etc/udev/rules.d/70-persistent-net.rules
/bin/rm /lib/udev/write_net_rules
You may safely ignore warnings raised by these commands (e.g. complaining that the files are missing)
To save a snapshot of your VM:
  • Shutdown your VM: Log in as root to your VM. Please DO NOT shutdown the VM from the Openstack dashboard to make sure all data are correctly flushed on disk;
  • Issue the shutdown -h now or poweroff command;
  • From the ComputeInstances table select the desired VM and click Create Snapshot

4.9. Deleting Virtual Machines

VMs can be deleted using the Terminate Instance option in the OpenStack dashboard.

Warning

This command will immediately terminate the instance, delete all content of the virtual machine and erase the ephemeral disk. This operation is not recoverable.

Chapter 5. Managing Storage

There are several ways of handling disk storage in the CloudVeneto:
  • Ephemeral storage exists only for the life of a virtual machine instance. It will persist across reboots of the guest operating system but when the instance is deleted so is the associated storage. The size of the ephemeral storage is defined in the virtual machine flavor.
  • Volumes are persistent virtualized block devices independent of any particular instance. Volumes may be attached to a single instance at a time, but may be detached or re-attached to a different instance while retaining all data, much like a USB drive. The size of the volume can be selected when it is created within the quota limits for the particular project.

5.1. Ephemeral storage

Ephemeral storage exists only for the life of a virtual machine instance. It will persist across reboots of the guest operating system but when the instance is deleted so is the associated storage. The size of the ephemeral storage is defined in the virtual machine flavor.
Among the flavor details (that are listed in the Dashboard when a VM has to be launched or can be seen using the openstack flavor list command), there is an attribute called 'Ephemeral'. When you use a flavor with an ephemeral disk size different from zero, the instance is booted with an extra virtual disk whose size is indicated by the ephemeral value. This ephemeral disk can be useful where you want to partition the second disk or have a specific disk configuration which is not possible within the system disk configuration.

Warning

Please note that backups are not performed on ephemeral storage systems.

5.2. Volumes

Volumes are persistent virtualized block devices independent of any particular instance. Volumes may be attached to a single instance at a time (i.e. not like a distributed filesystem such as Lustre or Gluster), but they may be detached or re-attached to a different instance while retaining all data, much like a USB drive.

Warning

Please note that backups are not performed on volumes.

5.2.1. Create a Volume

The steps to add a Volume are:
Using the Dashboard, click on ComputeVolumes and then Create Volume. In the "Create Volume” window specify the name of the volume (testvol in the example below) and the desired size (12 GB in the example). As Volume Source specify “No source, empty volume”.
Multiple volume types exist, and you need to specify the type to be used for the volume to be created.

Note

If you are a University of Padova user, please select the ceph volume type.
If you are a INFN user, please select the ceph volume type (the default) unless you have been told by the cloud administrator to use another volume type.
In general different quotas for the different volume types are set. Unfortunately the OpenStack dashboard shows only the overall quota. To see the quota per each volume type you need to use the OpenStack CLI (as explained in Section 3.7, “Accessing the Cloud with command line tools”) and run the cinder quota-usage ${OS_PROJECT_ID} command.
E.g.:
 
$ cinder quota-usage ${OS_PROJECT_ID}
+----------------------------+--------+----------+-------+
| Type                       | In_use | Reserved | Limit |
+----------------------------+--------+----------+-------+
| backup_gigabytes           | 0      | 0        | 1000  |
| backups                    | 0      | 0        | 10    |
| gigabytes                  | 72     | 0        | 400   |
| gigabytes_ceph             | 48     | 0        | 200   |
| gigabytes_equallogic-unipd | 24     | 0        | 200   |
| gigabytes_iscsi-infnpd     | 0      | 0        | 0     |
| per_volume_gigabytes       | 0      | 0        | 5000  |
| snapshots                  | 0      | 0        | 10    |
| snapshots_ceph             | 0      | 0        | -1    |
| snapshots_equallogic-unipd | 0      | 0        | -1    |
| snapshots_iscsi-infnpd     | 0      | 0        | -1    |
| volumes                    | 8      | 0        | 10    |
| volumes_ceph               | 4      | 0        | -1    |
| volumes_equallogic-unipd   | 4      | 0        | -1    |
| volumes_iscsi-infnpd       | 0      | 0        | -1    |
+----------------------------+--------+----------+-------+
$
In this example the project was given 400 GB. For the ceph and equallogic-unipd volume types the quota is 200 GB, while it is not possible to create a volume using the iscsi-infnpd volume type.

Warning

If you try to create a volume using a type for which the quota is over, you will see a generic 'Unable to create volume' error message.

5.2.2. Using (attaching) a Volume

The new defined volume will appear in the Volumes tab.
To attach this volume to an existing instance, click on ActionsManage Attachments ...
... select the relevant Virtual Machine...
...and click on "Attach Volume".
Log in to the instance and check if the disk has been added:
grep vdb /proc/partitions
 253       16   12582912 vdb
If needed, create a file system on it (this will scratch the disk!):
mkfs -t ext4 /dev/vdb
Mount the volume:
mount /dev/vdb /mnt

5.2.3. Detaching a Volume

To detach a volume from an instance, first of all log into the virtual machine that has the volume mounted, and unmount it:
umount /mnt
Then, using the Dashboard, click on Volumes, click on MoreManage Attachments for the relevant volume and select Detach Volume. The detached volume can then be associated to another VM, as described above (you won't have to re-create the file system, otherwise you will lose the content of the volume!)

5.2.4. Deleting a Volume

If a volume is not needed any longer, to completely remove it (note that this step cannot be reverted!):
  • if needed, detach the volume from the associated instance
  • using the Dashboard, click on ComputeVolumes, select the relevant volume and then select Delete Volumes

5.2.5. Sharing a volume between multiple (virtual) machines

As discussed in Section 5.2, “Volumes”, a volume may be attached to a single instance. However it can be shared with other virtual machines of the Cloud (and/or with other hosts) using NFS.
  1. Configure NFS server
    1. Once a volume has been created, formatted and attached to this instance acting as NFS server, create the mount point and mount the volume on this virtual machine:
      mkdir /dataNfs
      mount /dev/sdb /dataNfs
      
    2. Ensure that on this virtual machine the packages providing the rpc.nfsd daemon are installed:
      # yum whatprovides "*/rpc.nfsd"
      
      Loaded plugins: dellsysid, fastestmirror
      Loading mirror speeds from cached hostfile
       * base: centos.fastbull.org
       * epel: mirror.23media.de
       * extras: centos.fastbull.org
       * updates: fr2.rpmfind.net
      1:nfs-utils-1.3.0-0.8.el7.x86_64 : NFS utilities and supporting clients and daemons for the kernel NFS server
      Repo        : base
      Matched from:
      Filename    : /usr/sbin/rpc.nfsd
      
      
      
      1:nfs-utils-1.3.0-0.8.el7.x86_64 : NFS utilities and supporting clients and daemons for the kernel NFS server
      Repo        : @base
      Matched from:
      Filename    : /usr/sbin/rpc.nfsd
      
      # yum install nfs-utils
      
    3. Insert the correct export directive in the /etc/exports file. For example if the volume must be visible in read-only mode to all the virtual machines of the same subnet 10.67.1.* (check the subnet with the ifconfig command) the content of the /etc/exports file will be:
      /dataNfs 10.67.1.*(ro,no_root_squash)
      
      Note that there are no spaces between the '*' and the '('.
      If the volume must be visible in read-write to all the virtual machines of the same subnet 10.67.1.*, you might export it using the async or sync option. In short: async is much faster wrt sync, but can lead to data corruption if the server crashes during write operations (async means that the NFS server will acknowledge data before it's committed to disk, while sync does the opposite: the server will only acknowledge data after it's written out). Sync can be slow in particular when you have to write many files, since the open()/creat() and close() system calls have to wait for the new data to hit disk.
      To export a volume using the async option the /etc/exports will be something like:
                                                                                           
      /dataNfs 10.67.1.*(async,rw,no_root_squash)
      
      To export a volume using the sync option the /etc/exports will be something like:
                                                                                           
      /dataNfs 10.67.1.*(sync,rw,no_root_squash)
      
    4. Check the firewall on the virtual machine. Ensure that the other instances have both UDP and TCP access to ports 111, 2049 and 875:
      • For newer distributions using firewalld issue the following commands:
        firewall-cmd --add-service=nfs
        firewall-cmd --permanent --add-service=nfs
        
      • For older distributions using iptables, the /etc/sysconfig/iptables file should be something like this:
        *filter
        :INPUT ACCEPT [0:0]
        :FORWARD ACCEPT [0:0]
        :OUTPUT ACCEPT [0:0]
        -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
        -A INPUT -p icmp -j ACCEPT
        -A INPUT -i lo -j ACCEPT
        -A INPUT -m state --state NEW -m tcp -p tcp --dport 22 -j ACCEPT
        -A INPUT -m state --state NEW -m tcp -p tcp --dport 111 -j ACCEPT
        -A INPUT -m state --state NEW -m udp -p udp --dport 111 -j ACCEPT
        -A INPUT -m state --state NEW -m tcp -p tcp --dport 875 -j ACCEPT
        -A INPUT -m state --state NEW -m udp -p udp --dport 875 -j ACCEPT
        -A INPUT -m state --state NEW -m tcp -p tcp --dport 2049 -j ACCEPT
        -A INPUT -m state --state NEW -m udp -p udp --dport 2049 -j ACCEPT
        -A INPUT -j REJECT --reject-with icmp-host-prohibited
        -A FORWARD -j REJECT --reject-with icmp-host-prohibited
        COMMIT
        
        Please remember to restart iptables after any change on that file:
        service iptables restart
        
    5. Check the security group (see Section 3.4, “Setting security group(s)”): access to ports 111, 875 and 2049 (IPv4 Ingress both TCP and UDP) should be guaranteed:
    6. Restart nfs server using:
      systemctl restart nfs
      
      or
      service nfs restart
      
  2. Configure client machine(s)
    1. To mount the volume on the other VMs, check that the package providing the mount.nfs command is installed:
      # yum whatprovides "*/mount.nfs"
      Loaded plugins: dellsysid, fastestmirror
      Loading mirror speeds from cached hostfile
       * base: centos.fastbull.org
       * epel: mirror.23media.de
       * extras: centos.fastbull.org
       * updates: fr2.rpmfind.net
      1:nfs-utils-1.3.0-0.8.el7.x86_64 : NFS utilities and supporting clients and daemons for the kernel NFS server
      Repo        : base
      Matched from:
      Filename    : /sbin/mount.nfs
      
      
      
      1:nfs-utils-1.3.0-0.8.el7.x86_64 : NFS utilities and supporting clients and daemons for the kernel NFS server
      Repo        : @base
      Matched from:
      Filename    : /sbin/mount.nfs
      
      # yum install nfs-utils
      
    2. issue a mount command such as this one (assuming 10.67.1.4 is the NFS server):
      mount -t nfs 10.67.1.4:/dataNfs /mnt
      

Warning

It is highly suggested that the VM acting as NFS server is instantiated with enough resources (at least 2 VCPUs and 4 GB of RAM). Moreover it should be used only for hosting the NFS server (i.e. it shouldn't be used for other activities).

Note

Please note that this procedure can be used to mount a volume also on hosts outside the Cloud: it is sufficient to specify the IP address of these hosts in the /etc/exports file on the instance acting as NFS server.

5.3. Accessing storage external to the Cloud

As explained in Section 1.2, “Network access” from an instance of the Cloud by default it is not possible to access a host/service hosted in INFN Padova or Legnaro. This also means that by default on a virtual machine of the CloudVeneto it is not possible to mount a file system exported from a storage server hosted in INFN Padova or Legnaro.

Chapter 6. Orchestration

The CloudVeneto cloud provides an orchestration service, implemented through the OpenStack Heat component, that allows you to spin up multiple instances, and other cloud services in an automated fashion.
In Heat parlance, a stack is the collection of objects, or resources, that will be created by Heat. This might include instances (VMs), volumes, networks, subnets, routers, ports, security groups, security group rules, etc.
Heat uses the idea of a template to define a stack. If you want to have a stack that creates two instances connected by a private network, then your template would contain the definitions for two instances, a network, a subnet, and two network ports.
Either native HOT templates, and AWS CloudFormation (CFN) templates are supported. Templates in HOT (Heat Orchestration Template) format are typically, but not necessarily required to be, expressed as YAML while CFN (AWS CloudFormation) formatted templates are instead typically expressed in JSON.

6.1. Creating a template

This is a working example of a HOT template which:
  • Creates a virtual machine connected to a project network;
  • Creates a storage volume;
  • Attaches this volume to the previously created VM;
heat_template_version: 2015-04-30

description: Template which creates a VM and a cinder volume; volume is then attached to this VM

parameters:
  instance_name:
    type: string
    description: VM Name
    constraints:
    - allowed_pattern: "[a-zA-Z0-9-]+"

resources:

  my_volume:
    type: OS::Cinder::Volume
    properties:
      name: "testVolume"
      size: 3

  my_instance:
    type: OS::Nova::Server
    properties:
      name: { get_param: instance_name }
      image: Centos7x86_64
      flavor: cloudveneto.small
      security_groups: [default]
      key_name: paolomazzon
      admin_pass: heattest
      networks: [{"network": testing-lan}]

  my_volume_attachment:
    type: OS::Cinder::VolumeAttachment
    properties:
      volume_id: { get_resource: my_volume }
      instance_uuid: { get_resource: my_instance }

outputs:
  instance_fixed_ip:
    description: fixed ip assigned to the server
    value: { get_attr: [my_instance, first_address] }

Templates have three sections:
# This is required.
heat_template_version: 2015-04-30

parameters:
  # parameters go here

resources:
  # resources go here (this section is required)

outputs:
  # outputs go here
The resources section specifies what resources Heat should create:
resources:
  my_resource_id:
    type: a_resource_type
    properties:
      property1: ...
      property2: ...
Hardcoded values can be replaced with parameters. The actual value to be used is then specified when the stack is created. In our example a parameter is used for the name of the VM to be created:
parameters:
  instance_name:
    type: string
    description: VM Name
    constraints:
    - allowed_pattern: "[a-zA-Z0-9-]+"
A full description of all the resources that can be used in the stack creation can be found accessing the OrchestrationResource Types menu.
Sometimes we want to extract information about a stack. In our example the output is the fixed IP of the VM created by the stack:
outputs:
  server_ip:
    description: fixed ip assigned to the server
    value: { get_attr: [my_instance, first_address]:
Heat templates allow also to insert user data via cloud-init, e.g:
server01:
    type: OS::Nova::Server
    properties:
      image: sl66
      flavor: cldareapd.xsmall
      user_data_format: RAW
      user_data:
        str_replace:
          template: |
            #!/bin/sh
            yum install -y httpd
            service httpd start
            iptables -I INPUT 4 -m state --state NEW -p tcp --dport 80 -j ACCEPT
            service iptables save
            service iptables restart
Resource startup order can be managed in Heat, as explained in this page. For example it is possible to create a VM only when another one has been successfully started.
The Heat Orchestration Template (HOT) specification is available here.

6.2. Creating a stack

To create a stack using the browser please select the OrchestrationStacks left menu. From here select + Launch stack.
You will be prompted to select a template.
You will then be asked to fill in the parameters of the template and launch the stack.
Then you can follow the status of your stack on the dashboard.

Chapter 7. Managing Images

In a cloud environment, Virtual Machines are instantiated from images. These images are registered in an Image Management Service, in our case provided by the Glance OpenStack component.

7.1. Public Images

Some images in the CloudVeneto are provided by the Cloud administrators. These images are public, and visible to all users. They appear with Visibility equal to Public in the ComputeImages menu.
In the CloudVeneto cloud images are usually provided in QCOW2 format. They are fully resizable also with respect to the disk size.

7.1.1. Public Images for INFN Padova users

The SL6x-INFNPadova-x86-64-<date> and CentOS7x-INFNPadova-x86-64-<date> images are basic SL6.x / CentOS 7.x images which also include cloud-init to perform contextualization based on the user data specified when the VM are instantiated. They also configure CVMFS and the relevant squid servers.
Such images also configure the Padova LDAP server for user authentication. This means that it is just necessary to “enable” the relevant accounts on the VM adding in the /etc/passwd file:
                                                                                     
+name1::::::                                                                                 
+name2::::::                                                                                 
...
and creating their home directories.
Changes done in /etc/passwd could not be applied immediately by the system. In this case a:
                                                                                     
nscd -i passwd
should help.

Note

The SL6x-INFNPadova-x86-64-<date> and CentOS7x-INFNPadova-x86-64-<date> images also allow INFN-Padova system administrators to log (with admin privileges) on the instance.
INFN-Padova computing and Network service () can provide support only for instances created using such images (only to INFN-Padova users).

7.2. User Provided Images

Users can provide their own images and upload them to the Cloud Image service: these images are private, meaning that they are only available to the users of the project they are uploaded for.

Note

Users are not allowed to publish public (i.e. available to all projects) images.
Many open source projects such as Ubuntu and Fedora now produce pre-built images which can be used for certain clouds. If these are available, it is much easier to use them compared to building your own.
Using an Ubuntu image as an example, after you downloaded the image from the relevant web site, to upload such image using the command line tools you need to:
  • Authenticate to OpenStack using the openrc source:
    $ . demo-openrc.sh
    
  • Issue th following command:
    $ openstack image create --private --disk-format=qcow2 \
          --container-format=bare \
          --file trusty-server-cloudimg-amd64-disk1.img ubuntu-trusty
    
Once loaded, the image can then be used to create virtual machines.
Some system software is delivered in ISO image format. For example, these steps show how to create an image from the Freedos ISO available from here
$ openstack image create --private --disk-format=iso \
--container-format=bare --file=fd11src.iso freedos11

Note

In the CloudVeneto cloud image size is limited to 25 GB.

Warning

No backup is currently done on user provided images or snapshots. Therefore users with private images should keep a copy of the images they have uploaded in their private archives.

7.3. Sharing Images

As mentioned before, users are not allowed to publish public images. However images can be shared between different projects. This is currently only possible via the command line tools.
If an image has been uploaded to your currently active project, using the procedure described in Section 7.2, “User Provided Images”, you can then use the openstack image add project operation to share that image with another project.
To share an image, first source the project profile for the project containing the image you want to share (e.g. Fedora-Cloud-Base-27) and find its id with the openstack image list command (d4b02b71-755e-47ad-bb27-1ea5c23bf7cb in the example):
$ openstack image list
+--------------------------------------+----------------------+--------+
| ID                                   | Name                 | Status |
+--------------------------------------+----------------------+--------+
| 7ebe160d-5498-477b-aa2e-94a6d962a075 | CentOS7              | active |
| d4b02b71-755e-47ad-bb27-1ea5c23bf7cb | Fedora-Cloud-Base-27 | active |
| 071668fc-dfeb-4956-996a-d2a487755709 | cirros               | active |
+--------------------------------------+----------------------+--------+

You then need to change (to 'Shared' the visibility of the image:
openstack image set --property visibility=shared d4b02b71-755e-47ad-bb27-1ea5c23bf7cb
You now need to find the id of the project you wish to share the image with. This will generally be done by looking at the openrc file of that project and finding the OS_PROJECT_ID variable (in this example, it is e81df4c0b493439abb8b85bfd4cbe071).
To share the image with id d4b02b71-755e-47ad-bb27-1ea5c23bf7cb to the project whose is is e81df4c0b493439abb8b85bfd4cbe071, use the command:
$ openstack image add project d4b02b71-755e-47ad-bb27-1ea5c23bf7cb e81df4c0b493439abb8b85bfd4cbe071 
+------------+--------------------------------------+
| Field      | Value                                |
+------------+--------------------------------------+
| created_at | 2018-03-19T16:09:21Z                 |
| image_id   | d4b02b71-755e-47ad-bb27-1ea5c23bf7cb |
| member_id  | e81df4c0b493439abb8b85bfd4cbe071     |
| schema     | /v2/schemas/member                   |
| status     | pending                              |
| updated_at | 2018-03-19T16:09:21Z                 |
+------------+--------------------------------------+
A member of the target project (with id e81df4c0b493439abb8b85bfd4cbe071 in our example) needs to accept the image in order to be available in the images list:
$ openstack image set --accept d4b02b71-755e-47ad-bb27-1ea5c23bf7cb
In the target project, the image will appear in the image list:

7.4. Building Images

Users can also build custom images, that can then been uploaded in the Cloud Image service as described in Section 7.2, “User Provided Images”.
There are several tools providing support for image creation. Some of them are described in the Openstack documentation.

7.5. Deleting Images

Images that are not used anymore can be deleted. Deletion of images is permanent and cannot be reversed.
To delete an image, log in to the dashboard and select the appropriate project from the drop down menu at the top left. On the ComputeImages page select the images you want to delete and click Delete Images. Confirm your action pressing the Delete Images again on the confirmation dialog box.

Warning

Don't delete an image if are there are virtual machines created using this image, otherwise these VMs won't be able to start if hard rebooted.

Chapter 8. Creating Batch Clusters on the Cloud

The virtual machines provided by CloudVeneto can also be used to implement batch clusters where users can run their applications (normal jobs or docker containers).
In this chapter we explain how to implement a dynamic batch cluster based on HTCondor and elastiq using ECM.

8.1. Intro: the idea

You create on the cloud a virtual machine that acts as a master for a dynamic batch system (implemented using HTCondor). When you create the master you will need to describe the cluster configuration, as described in Section 8.2.2, “The cluster configuration”.
The master node will be able to spawn new slave nodes (where jobs are executed) when jobs are submitted to the batch system. The elastic cluster will provide a number of virtual resources that scales up or down depending on your needs. The total number of active virtual nodes is dynamic.
The master node will act also as submitting machine: you can log in on this node and submit jobs to the batch system. These jobs will run on the slave nodes, get done, and eventually the slaves will be released.

Note

The master can use a different flavor with respect to the slave nodes.

8.2. Creating Batch Clusters on the Cloud with HTCondor

When you create the master, using the instructions reported in Section 4.1, “Creating Virtual Machines”, you will need to specify some user-data to describe the cluster configuration, as described below.

Note

The master and the slaves must use the same image (which should have all the needed software).

8.2.1. Prerequisites

  • You should be registered in the Cloud as member of a project.
  • You need to have created a SSH key-pair, ad explained in Section 3.2, “Creating a keypair”. This will allow you to log in the master and slave nodes.
  • You need to download the EC2 credentials of the project you want to use (see Section 3.8, “Accessing the Cloud through the euca2ools EC2 command line tools”). You can download them from the dashboard as following:
    Open the dashboard, select the project (drop down menu on top left of the dashboard), go to ComputeAPI Access and here click on Download EC2 credentials.
    You'll get a zip file with a name like: testing-x509.zip (in this case testing is the name of the project). The zip contains the following files:
    $ unzip testing-x509.zip
    Archive:  testing-x509.zip
    extracting: pk.pem
    extracting: cert.pem
    extracting: cacert.pem
    extracting: ec2rc.sh
    
    Extract all these files somewhere safe. The content of your ec2rc.sh file is something like:
    $ cat ec2rc.sh
    
    #!/bin/bash
    
    NOVARC=$(readlink -f "${BASH_SOURCE:-${0}}" 2>/dev/null) || NOVARC=$(python -c 'import os,sys; print os.path.abspath(os.path.realpath(sys.argv[1]))' "${BASH_SOURCE:-${0}}")
    NOVA_KEY_DIR=${NOVARC%/*}
    export EC2_ACCESS_KEY=<access_key>
    export EC2_SECRET_KEY=<secret_key>
    export EC2_URL=https://cloud.cedc.csia.unipd.it:8773/services/Cloud
    export EC2_USER_ID=42 # nova does not use user id, but bundling requires it
    export EC2_PRIVATE_KEY=${NOVA_KEY_DIR}/pk.pem
    export EC2_CERT=${NOVA_KEY_DIR}/cert.pem
    export NOVA_CERT=${NOVA_KEY_DIR}/cacert.pem
    export EUCALYPTUS_CERT=${NOVA_CERT} # euca-bundle-image seems to require this set
    
    alias ec2-bundle-image="ec2-bundle-image --cert ${EC2_CERT} --privatekey ${EC2_PRIVATE_KEY} --user 42 --ec2cert ${NOVA_CERT}"
    alias ec2-upload-bundle="ec2-upload-bundle -a ${EC2_ACCESS_KEY} -s ${EC2_SECRET_KEY} --url ${S3_URL} --ec2cert ${NOVA_CERT}"
    
    where EC2_ACCESS_KEY and EC2_SECRET_KEY are different for each project and user. The elastiq service uses these values to instantiate and to kill VMs in the specific project.
  • You need to identify the image to be used for the master and for the slaves. Currently supported operating systems are RHEL6.x and derivates (SL6.x, CentOS7.x, etc.), RHEL7.x and derivates and Ubuntu. uCernVM based images are also supported. For such image you also need to know the relevant EC2 (AMI) id (see Section 8.2.7, “How to find the EC2 (AMI) id of an image”).
  • You need to set a specific security group to be used for the master node. This security group must include the following rules:
    Direction    Ether Type    IP Protocol    Port Range         Remote IP Prefix
    Egress       IPv4          Any            Any                0.0.0.0/0	
    Egress       IPv6          Any            Any                ::/0	
    Ingress      IPv4          ICMP           Any                0.0.0.0/0 	
    Ingress      IPv4          TCP            22 (SSH)           0.0.0.0/0	
    Ingress      IPv4          TCP            9618               0.0.0.0/0	
    Ingress      IPv4          TCP            41000 - 42000      0.0.0.0/0
    

    Note

    Instead of modifying the rules of an existing security group, we suggest to create a new security group named e.g. "master_security_group". Security groups are discussed in Section 3.4, “Setting security group(s)”.
    The slave nodes will instead use the default security group of your project. This group must include the following rule:
    Direction  Ether Type  IP Protocol  Port Range   Remote IP Prefix   Remote Security Group
    Ingress    IPv4        Any          Any          -                 <master_security_group> 
    
    where <master_security_group> is the name of the security group that was chosen for the master node.
  • You need to download the ECM software. As explained in Section 8.2.2, “The cluster configuration”, this will be used to create the batch cluster configuration:
    $ git clone https://github.com/CloudPadovana/ECM.git
    
  • You need to install the euca2ools package to discover the EC2 (AMI) id of images. Euca commands are also used by ECM to show the list of available images for the cluster.
    yum install euca2ools (on RHEL/CentOS)
    apt-get install euca2ools (on Ubuntu)
    

8.2.2. The cluster configuration

You must properly configure the ecm.conf file stored in the ECM directory (created when you downloaded via git the ECM software)
$ cat ecm.conf                                                                  
FLAVOR_VMS=<flavor_name>                               
MAX_VMS=<max_vms>                                      
MIN_VMS=<min_vms>                                      
JOBS_PER_VM=<jobs_per_vm>                              
IDLE_TIME=<idle_time>                                  
KEY_NAME==<ssh_key_name>
Where:
  • <FLAVOR_VMS> is the name of the flavor to be used for the slave nodes. Flavors have been discussed in Section 4.4, “Flavors”. Available flavors are listed in the dashboard when you try to launch a VM.
  • <MAX_VMS> is the maximum number of slave nodes that can be instantiated.
  • <MIN_VMS> is the minimum number of slave nodes (never terminated, always available).
  • <JOBS_PER_VM> is the maximum number of jobs that will be run in a single slave node.

    Important

    You have to verify that the number of jobs per VM is compatible with the number of VCPUs of the selected flavor.
  • <IDLE_TIME> is the time (in seconds) after which inactive VMs will be killed.
  • <KEY_NAME> is the name (without the .pem extension) of ssh key previously created (see Section 3.2, “Creating a keypair”) to be injected in the batch cluster nodes.

Note

The batch system will use each CPU as a separate job slot. So if you have a flavor with 4 VCPUs, and you submit 1 job, the master will create 1 slave and use 1 of the 4 available VCPUs. If you submit 4 jobs, again the master will create 1 slave, and will use all the 4 VCPUs. Large flavors means less machines to be created but possibly a sub-optimal usage of resources.

8.2.3. Start the elastic cluster

To start the elastic cluster, you only need to instantiate the master node.
When you create such master, you will need to specify some user-data to describe the cluster configuration. The ecm.py script will create such user-data file for you, using as input the ecm.conf file you previously edited. (see Section 8.2.2, “The cluster configuration”).
First of all you have to set the relevant EC2 credentials:
$ source ec2rc.sh
Then you must launch ecm.py file and follow the instructions. We will create a CentOS 6 cluster as an example.
$ python ecm.py

Choose the Operating System (OS) you want 
to use for your master and worker nodes:

1: Fedora
2: Ubuntu
3: uCernVM
4: CentOS7
5: CentOS6

OS type => 5

Important

The same OS will be used to instantiate both master node and worker node(s).
Select the image for your CentOS6 based master and your CentOS6 based WNs:
1: CentOS 6
2: Other image. [WARNING] You have to know the EC2-id of image

Image => 1

Warning

If you choose "Other image" you have to manually insert the image id in EC2 format (see Section 8.2.7, “How to find the EC2 (AMI) id of an image”).
The script will now print something like:
	  Now you can use the master-centos6-2017-03-16-17.45.31 file to instantiate the master node of your elastiq cluster.
Now you have to start the master node. As explained in Section 4.1, “Creating Virtual Machines”. go to 'Instances' and create a new instance with [Launch Instances].
In the details select:
[details]
  • Instance Name => whatever you like
  • Flavor => whatever you like; can be different from the flavor chosen for the slave nodes
  • Image name => The same image chosen for the slaves.
[Access and Security]
  • Key pair => The key-pair that will be used to log on the nodes of the batch cluster
  • Security Group => the security group for the master (choose only this one)
[post creation]
  • Customization Script Source => Select "File" from the dropdown menu and use "Choose File" to upload the user_data_file created by the ecm.py script
Then press launch.
Once you requested the creation of the master node, after some minutes, you will see that the master virtual machine and some (depending on the "MIN_VMS" attribute you defined in the ecm.conf) slave nodes are created.
Get the IP address of master, and log in on this machine using the key you have imported/created before i.e:
$ ssh -i ~/.ssh/id_rsa root@10.64.YY.XXX
For security reasons, as root you can not submit jobs to the HTCondor cluster. So make sure that a 'normal' account exists in the master node. In case you can create it using the command:
# adduser <username>
Create a password for this account:
# passwd <username>
You have to import any external disk, create homes, etc, as you would do in a normal machine.

8.2.4. How slave nodes are installed and configured

The instantiation of slaves nodes is managed by the elastiq service running in the master node.
The min and max number of nodes is set by user in the ecm.conf: the total number of active nodes will change dynamically with jobs need.
The installation of condor and its configuration on slaves is automatically provided by ECM. Inside the user_data_file created by the ecm.py script there is the parameter SLAVE_USERDATA whose value is the script for the installation and configuration of the slave, coded in b64. The original uncoded script used for condor installation and configuration on slaves is stored in the ECM/slave_files directory. There is a file for each Operating system, depending on the system selected for the master. These files provide the basic configuration for condor, to support both the Vanilla and Docker universes.
Generally the user doesn't need to modify these files. If, for some reasons, you need to modify the condor configuration or you need to install additional packages on the slaves, this script can be modified: ECM will take care to code it in b64 and add the new value to the SLAVE_USERDATA parameter in the user_data_file.

8.2.5. Use the elastic cluster

Log to the master node using your unpriviledged account. Check if condor is running with the command:
$ condor_q
-- Schedd: 10-64-20-181.virtual-analysis-facility : <10.64.20.181:41742>
ID      OWNER            SUBMITTED     RUN_TIME ST PRI SIZE CMD

0 jobs; 0 completed, 0 removed, 0 idle, 0 running, 0 held, 0 suspended
Check the status of the cluster:
$ condor_status
Name               OpSys      Arch   State     Activity LoadAv Mem   ActvtyTime

slot1@10-64-22-84. LINUX      X86_64 Unclaimed Idle      0.000 1977  2+12:44:58
slot2@10-64-22-84. LINUX      X86_64 Unclaimed Idle      0.000 1977  2+12:45:25
Total Owner Claimed Unclaimed Matched Preempting Backfill

X86_64/LINUX     2     0       0         2       0          0        0

Total            2     0       0         2       0          0        0
Create you HTCondor 'job file'. A simple example is the following:
$ cat test.classad
	
Universe = vanilla
Executable = /home/<username>/test.sh
Log = test.log.$(Cluster)$(Process)
Output = test.out.$(Cluster)$(Process)
Error = test.err.$(Cluster)$(Process)
Queue <number_of_jobs_to_submit>
where test.sh is the executable you want to run.
Submit your jobs issuing the command:
$ condor_submit test.classad
and check their status with:
$ condor_q
You can find documentation about HTCondor here.

8.2.6. Use the elastic cluster to run docker containers

The HTCondor elastic cluster can also be used to run docker containers. You don't need to install docker on your images: this is done by ECM.
Once the cluster is created, check if Docker is enabled on the slaves:
                                                                                                 
# condor_status -l | grep -i Docker                                                                      
StarterAbilityList = "HasTDP,HasEncryptExecuteDirectory,HasFileTransferPluginMethods,HasJobDeferral,HasJICLocalConfig,HasJICLocalStdin,HasPerFileEncryption,HasDocker,HasFileTransfer,HasReconnect,HasVM,HasMPI,HasRemoteSyscalls,HasCheckpointing"                                                                      
DockerVersion = "Docker version 1.7.1, build 786b29d/1.7.1"                                              
HasDocker = true                                                                                         
StarterAbilityList = "HasTDP,HasEncryptExecuteDirectory,HasFileTransferPluginMethods,HasJobDeferral,HasJICLocalConfig,HasJICLocalStdin,HasPerFileEncryption,HasDocker,HasFileTransfer,HasReconnect,HasVM,HasMPI,HasRemoteSyscalls,HasCheckpointing"                                                                      
DockerVersion = "Docker version 1.7.1, build 786b29d/1.7.1"                                              
HasDocker = true                                                                                         
The following is a simple example which runs a docker container, which is downloaded from docker-hub:
    
$ cat test-docker.classad                                                                                
          
universe = docker                                                                                        
docker_image = debian                                                                                    
executable = /bin/cat                                                                                    
arguments = /etc/hosts                                                                                   
should_transfer_files = YES                                                                              
when_to_transfer_output = ON_EXIT                                                                        
Log = test-docker.log.$(Cluster).$(Process)                                                              
Output = test-docker.out.$(Cluster).$(Process)                                                           
Error = test-docker.err.$(Cluster).$(Process)                                                            
request_memory = 100M                                                                                    
Queue <number_of_jobs_to_submit>

8.2.7. How to find the EC2 (AMI) id of an image

As explained above, to use the Elastic batch cluster you need to know the EC2 (AMI) id of the image you want to use.
First of all you need to install the euca2ools and to download the EC2 credentials for your project, as explained in Section 3.8, “Accessing the Cloud through the euca2ools EC2 command line tools”.
Uncompress (unzip) the EC2 zip file, and source the ec2rc.sh script to set the correct environment:
$ source ec2rc.sh
To discover the EC2 id of your image or snapshot, use the 'euca-describe-images' command:
$ euca-describe-images -I ${EC2_ACCESS_KEY} -S ${EC2_SECRET_KEY} -U ${EC2_URL}

IMAGE   ami-0000031b    None (uCernVM 2.3-0)    beaeede3841b47efb6b665a1a667e5b1        available       public                  machine                         instance-store
IMAGE   ami-00000447    snapshot        36b1ddb5dab8404dbe7fc359ec95ecf5        available       public                  machine                         instance-store
In the example above:
  • the EC2 image id of the uCernVM 2.3-0 image is ami-0000031b
  • there is a snapshot whose EC2 is is ami-00000447.

Note

In case you have snapshot on the output of the euca-describe-images you notice that you have no name associated with the ami-id. To obtain a nicely formatted list of (ami-id, name) couples you can use the following command:
$ euca-describe-images --debug 2>&1 | grep 'imageId\|name' | sed 'N;s/\n/ /'

<imageId>ami-00000002</imageId>       <name>cirros</name>
<imageId>ami-0000000d</imageId>       <name>Fedora 20 x86_64</name>
<imageId>ami-00000010</imageId>       <name>Centos 6 x86_64</name>
<imageId>ami-00000013</imageId>       <name>Centos 7 x86_64</name>
<imageId>ami-0000001b</imageId>       <name>ubuntu-14.04.3-LTSx86_64</name>
<imageId>ami-0000005d</imageId>       <name>matlab-2015a-glnxa64</name>
<imageId>ami-00000057</imageId>       <name>Win7-Pro-X86_64-ENU</name>
<imageId>ami-00000027</imageId>       <name>Fedora 23 x86_64</name>
<imageId>ami-00000069</imageId>       <name>Win7-Photoscan</name>
<imageId>ami-0000002d</imageId>       <name>photo-slave</name>
<imageId>ami-0000004e</imageId>       <name>s-medium-snap</name>
<imageId>ami-0000005a</imageId>       <name>uCernVM 3.6.5</name>
<imageId>ami-000000ae</imageId>       <name>archlinux</name>
<imageId>ami-000000c6</imageId>       <name>ubuntu-16.04.1-LTS x86_64</name>
<imageId>ami-000000c9</imageId>       <name>Fedora 25 x86_64</name>
<imageId>ami-000000d2</imageId>       <name>x2go-thinclient-server</name>
<imageId>ami-000000d5</imageId>       <name>Win7-test</name>
<imageId>ami-000000db</imageId>       <name>matlab-2016b</name>
<imageId>ami-000000d8</imageId>       <name>ubuntu-16.04.1+Matlab_2016b</name>
...
Note that items may appear multiple times...
...
You can also see all the information of an image, e.g.:
$ euca-describe-images --debug ami-00000447 
or:
$ euca-describe-images -I ${EC2_ACCESS_KEY} -S ${EC2_SECRET_KEY} -U ${EC2_URL} --debug ami-00000447 
The returned output will be something like:
<requestId>req-c56c3694-c555-464a-b21d-2c86ccc020be</requestId>
<imagesSet>
<item>
<description/>
<imageOwnerId>36b1ddb5dab8404dbe7fc359ec95ecf5</imageOwnerId>
<isPublic>true</isPublic>
<imageId>ami-00000447</imageId>
<imageState>available</imageState>
<architecture/>
<imageLocation>snapshot/imageLocation>
<rootDeviceType>instance-store</rootDeviceType>
<rootDeviceName>/dev/sda1</rootDeviceName>
<imageType>machine</imageType>
<name>cernvm_230_ldap_pd</name>
</item>
</imagesSet>
</DescribeImagesResponse>

8.2.8. Restart the elastic cluster

This section describes the steps to be followed to restart the elastic cluster (this might be needed e.g. after a reboot).
  • If possible, delete all the slave nodes via dashboard;
  • Log on the master node as root;
  • Check if both condor and elastiq services are already running
    # service condor status
    # service elastiq status
    
    In this case new slaves will be created and will join the cluster in some minutes.
  • if only condor service is running but elastiq isn't, please restart elastiq using the following command:
    # service elastiq start
    
    or
    # elastiqctl restart
    
    and wait for the creation of new slaves that will connect to the cluster in some minutes;
  • If condor isn't running and some elastiq processes are up and running, kill them:
    # ps -ef | grep elastiq
    # kill -9 <n_proc>
    
    and start the condor service using the following command:
    # service condor start
    
    Now the condor_q command should return
    # condor_q 
    -- Schedd:  : <10.64.xx.yyy:zzzzz>
    ID      OWNER            SUBMITTED     RUN_TIME ST PRI SIZE CMD               
    0 jobs; 0 completed, 0 removed, 0 idle, 0 running, 0 held, 0 suspended
    
    and the condor_status command output should be empty (no nodes running)
    # condor_status
    
    
    Then start the elastiq service
    # service elastiq start
    
    In some minutes the minimum number of nodes should connect to the condor cluster and the condor_status command output should show them e.g.:
    # condor_status
    Name               OpSys      Arch   State     Activity LoadAv Mem   ActvtyTime
    slot1@10-64-22-215 LINUX      X86_64 Unclaimed Idle      0.000 1896  0+00:24:46
    slot2@10-64-22-215 LINUX      X86_64 Unclaimed Idle      0.000 1896  0+00:25:05
    slot1@10-64-22-217 LINUX      X86_64 Unclaimed Idle      0.000 1896  0+00:24:44
    slot2@10-64-22-217 LINUX      X86_64 Unclaimed Idle      0.000 1896  0+00:25:05
    slot1@10-64-22-89. LINUX      X86_64 Unclaimed Benchmar  1.000 1896  0+00:00:04
    slot2@10-64-22-89. LINUX      X86_64 Unclaimed Idle      0.040 1896  0+00:00:05
    Machines Owner Claimed Unclaimed Matched Preempting
    X86_64/LINUX        6     0       0         6       0          0
    Total               6     0       0         6       0          0
    

8.2.9. Troubleshooting the elastic cluster

Check the log files
The elastiq log file is /var/log/elastiq/elastiq.log. It is not straightforward to interpret it without knowing its structure. When you start the elastiq service the first part of log file reports the check of cloud user's credentials and of other parameters configured in the elastiq.conf file (i.e. the userdata file for slave nodes)
INFO [__init__.conf] Configuration: ec2.image_id = ami-9f3da3fc (from file)
INFO [__init__.conf] Configuration: ec2.flavour = cldareapd.medium (from file)
INFO [__init__.conf] Configuration: ec2.api_url = https://cloud-areapd.pd.infn.it:8788/services/Cloud (from file)
2INFO [__init__.conf] Configuration: ec2.aws_secret_access_key = [...] (from file)
INFO [__init__.conf] Configuration: ec2.key_name = my_key (from file)
INFO [__init__.conf] Configuration: ec2.user_data_b64 = [...] (from file)
INFO [__init__.conf] Configuration: ec2.aws_access_key_id = [...] (from file)

INFO [__init__.conf] Configuration: quota.max_vms = 3.0 (from file)
INFO [__init__.conf] Configuration: quota.min_vms = 1.0 (from file)
NFO [__init__.main] Loaded batch plugin "htcondor"
DEBUG [htcondor.init] HTCondor plugin initialized
DEBUG [__init__.main] EC2 image "ami-9f3da3fc" found
if your credentials are wrong, you will see in the log file an error as:
ERROR [__init__.ec2_running_instances] Can't get list of EC2 instances (maybe wrong credentials?)
If you specified a wrong ami_id for the image of slave nodes, the error message in the log will be:
ERROR [__init__.main] Cannot find EC2 image "ami-00000000"
Elastiq periodically checks all the VMs. If a VM is correctly added to the condor cluster, it logs:
DEBUG [__init__.ec2_running_instances] Found IP 10.64.22.188 corresponding to instance
otherwise something like:
WARNING [__init__.ec2_running_instances] Cannot find instance 10.64.22.216 in the list of known IPs
WARNING [__init__.ec2_running_instances] Cannot find instance 10.64.22.182 in the list of known IPs
WARNING [__init__.ec2_running_instances] Cannot find instance 10.64.22.236 in the list of known IPs
When elastiq instantiates a new VM it logs something like:
WARNING [__init__.ec2_scale_up] Quota enabled: requesting 1 (out of desired 1) VMs
INFO [__init__.ec2_scale_up] VM launched OK. Requested: 1/1 | Success: 1 | Failed: 0 | ID: i-f026f340
DEBUG [__init__.save_owned_instances] Saved list of owned instances: i-f026f340
and when elastiq deletes an idle VM it logs:
INFO [__init__.check_vms] Host 10-64-22-190.INFN-PD is idle for more than 2400s: requesting shutdown
INFO [__init__.ec2_scale_down] Requesting shutdown of 1 VMs...
In the master node of condor, logs are located in /var/log/condor directory. You may refer to the HTCondor documentation for more information on these log files.
# ls -l /var/log/condor/
total 76
-rw-r--r--. 1 condor condor 24371 Jan 18 08:42 CollectorLog
-rw-r--r--. 1 root   root     652 Jan 18 08:35 KernelTuning.log
-rw-r--r--. 1 condor condor  2262 Jan 18 08:35 MasterLog
-rw-r--r--. 1 condor condor     0 Jan 18 08:35 MatchLog
-rw-r--r--. 1 condor condor 19126 Jan 18 08:42 NegotiatorLog
-rw-r--r--. 1 root   root   13869 Jan 18 08:42 ProcLog
-rw-r--r--. 1 condor condor   474 Jan 18 08:35 ScheddRestartReport
-rw-r--r--. 1 condor condor  2975 Jan 18 08:40 SchedLog
Check the running processes
Processes expected to run are:
  • for the condor service:
    # ps -ef | grep condor
    condor       764       1  0 14:09 ?        00:00:00 /usr/sbin/condor_master -f
    root         960     764  0 14:09 ?        00:00:00 condor_procd -A /var/run/condor/procd_pipe -L /var/log/condor/ProcLog -R 1000000 -S 60 -C 996
    condor       961     764  0 14:09 ?        00:00:00 condor_collector -f
    condor       974     764  0 14:09 ?        00:00:00 condor_negotiator -f
    condor       975     764  0 14:09 ?        00:00:00 condor_schedd -f	  
    
  • for the elastiq service:
    # ps -ef | grep elastiq
    elastiq      899       1  0 14:09 ?        00:00:00 SCREEN -dmS __|elastiq|__ /bin/sh -c python /usr/bin/elastiq-real.py --logdir=/var/log/elastiq --config=/etc/elastiq.conf --statefile=/var/lib/elastiq/state 2> /var/log/elastiq/elastiq.err
    elastiq      952     899  0 14:09 pts/0    00:00:00 /bin/sh -c python /usr/bin/elastiq-real.py --logdir=/var/log/elastiq --config=/etc/elastiq.conf --statefile=/var/lib/elastiq/state 2> /var/log/elastiq/elastiq.err
    elastiq      953     952  0 14:09 pts/0    00:00:01 python /usr/bin/elastiq-real.py --logdir=/var/log/elastiq --config=/etc/elastiq.conf --statefile=/var/lib/elastiq/state
    

Warning

The condor_status information isn't updated very frequently. So it can happen that condor_status shows nodes that have been already removed from the cloud by elastiq.

Chapter 9. Some basics on Linux administration

9.1. Introduction

 
The possession of great power necessarily implies great responsibility.
 
 --William Lamb, 2nd Viscount Melbourne
CloudVeneto provides an infrastructure where your virtual machines can live. After you have activated your virtual machine(s) you are on your own for the most part of the day to day administration tasks.

Warning

We will only focus on Linux VMs, showing differences between the RedHat (CentOS, Fedora, ... ) and Debian (Ubuntu, Mint, ... ) distributions.
Throughout this chapter we will address the former with RH and the latter with DEB.
Some of these everyday tasks might be:
  • Use your VM as root only when needed;
  • Installing/deinstalling software;
  • Adding another user to your VM;
  • Formatting the volume you just attached to your VM;
  • Automatically remount the volume on next reboot.
In the following sections we provide some very small introductory instructions on performing such tasks.

9.2. Setting up 'sudo'

Nobody (not even administrators) use a Unix/Linux system always as root.
If you do you should stop immediately (no jokes!).
Normally you have your user with limited privileges and, when needed, you use su (which stands for 'switch user') to become root, perform the privileged task and then you go back to the normal user.
A more flexible approach is using sudo (Super User DO) to perform the 'one shot' task or to allow certain users to perform only some tasks as the superuser. The configuration of sudo is performed by modifying the /etc/sudoers file or (better) by adding a file in the /etc/sudoers.d directory.
Follow these instructions to allow the user paolo (change this to your username) to perform any command as the superuser providing his own password, or to modify your user privileges (in this case there is already a file with your username in the /etc/sudoers.d directory):
  • Become the root user:
    • RH: using su and providing the root password
    • DEB: using sudo su - and providing your password
  • Create the file /etc/sudoers.d/paolo using your preferred editor;
  • Add this line inside the file:
          paolo            ALL = (ALL) ALL 
    
    If you want the user to do everything without even providing the password put this line instead:
          paolo            ALL = NOPASSWD: ALL
    
  • change the file permissions: chmod 0440 /etc/sudoers.d/paolo
Now when paolo logs in to the VM he is allowed to perform superuser tasks by prefixing the command with sudo.
If you want to limit the user to perform only certain commands (or classes of commands, e.g. installing/deinstalling software) you can look at the sudo documentation on your VM using
man sudoers

9.3. Managing software through the package manager

RedHat and Debian based linux distributions both have their software management system. On RH each software is packaged in rpm form (RPM stands for RedHat Package Manager) while DEB uses deb packages.
Package contents are not only copied on the VM when installed, but are also listed on a database that can be queried to search for new software, to find out which package installed which files and so on. (Note: there is no such functionality on Windows servers...).
Rather than manipulating the packages directly whith the rpm (RH) or dpkg (DEB) commands, you will mostly use a command line package manager such as yum or apt.
We will now try to install the wget package on a RH and a DEB system.

9.3.1. Managing software on RH distributions

Let's try to install the wget software on CentOS or Fedora linux.
We will use yum (dnf on Fedora 21) and rpm for the task.
Since we will be performing operations as the superuser, if you haven't already, please set up sudo first.
  1. Query the system to search for wget (no need to be root for that):
    [paolo@maz03 ~]$ yum search wget
    Loaded plugins: fastestmirror
    Trying other mirror.
    base                                                     | 3.6 kB     00:00     
    extras                                                   | 3.4 kB     00:00     
    updates                                                  | 3.4 kB     00:00     
    (1/4): base/7/x86_64/group_gz                              | 154 kB   00:00     
    (2/4): extras/7/x86_64/primary_db                          |  87 kB   00:00     
    (3/4): updates/7/x86_64/primary_db                         | 3.9 MB   00:00     
    (4/4): base/7/x86_64/primary_db                            | 5.1 MB   00:01     
    Determining fastest mirrors
     * base: mi.mirror.garr.it
     * extras: mi.mirror.garr.it
     * updates: mi.mirror.garr.it
    ============================== N/S matched: wget ===============================
    wget.x86_64 : A utility for retrieving files using the HTTP or FTP protocols
    
      Name and summary matches only, use "search all" for everything.
    
  2. Install the wget package (note that we are using sudo here):
    [paolo@maz03 ~]$ sudo yum install wget
    Loaded plugins: fastestmirror
    Loading mirror speeds from cached hostfile
     * base: mi.mirror.garr.it
     * extras: mi.mirror.garr.it
     * updates: mi.mirror.garr.it
    Resolving Dependencies
    --> Running transaction check
    ---> Package wget.x86_64 0:1.14-10.el7_0.1 will be installed
    --> Finished Dependency Resolution
    
    Dependencies Resolved
    
    ================================================================================
     Package        Arch             Version                   Repository      Size
    ================================================================================
    Installing:
     wget           x86_64           1.14-10.el7_0.1           base           545 k
    
    Transaction Summary
    ================================================================================
    Install  1 Package
    
    Total download size: 545 k
    Installed size: 2.0 M
    Is this ok [y/d/N]: y
    Downloading packages:
    wget-1.14-10.el7_0.1.x86_64.rpm                            | 545 kB   00:00     
    Running transaction check
    Running transaction test
    Transaction test succeeded
    Running transaction
      Installing : wget-1.14-10.el7_0.1.x86_64                                  1/1 
      Verifying  : wget-1.14-10.el7_0.1.x86_64                                  1/1 
    
    Installed:
      wget.x86_64 0:1.14-10.el7_0.1                                                 
    
    Complete!
    
  3. Query the rpm database to see what has been installed:
    [paolo@maz03 ~]$ rpm -ql wget
    /etc/wgetrc
    /usr/bin/wget
    /usr/share/doc/wget-1.14
    /usr/share/doc/wget-1.14/AUTHORS
    /usr/share/doc/wget-1.14/COPYING
    /usr/share/doc/wget-1.14/MAILING-LIST
    /usr/share/doc/wget-1.14/NEWS
    /usr/share/doc/wget-1.14/README
    /usr/share/doc/wget-1.14/sample.wgetrc
    /usr/share/info/wget.info.gz
    /usr/share/locale/be/LC_MESSAGES/wget.mo
    .....
    .....
    /usr/share/locale/sv/LC_MESSAGES/wget.mo
    /usr/share/locale/tr/LC_MESSAGES/wget.mo
    /usr/share/locale/uk/LC_MESSAGES/wget.mo
    /usr/share/locale/vi/LC_MESSAGES/wget.mo
    /usr/share/locale/zh_CN/LC_MESSAGES/wget.mo
    /usr/share/locale/zh_TW/LC_MESSAGES/wget.mo
    /usr/share/man/man1/wget.1.gz
    
  4. You now decide you don't need wget anymore. Remove the package (root needed!):
    [paolo@maz03 ~]$ sudo yum remove wget
    Loaded plugins: fastestmirror
    Resolving Dependencies
    --> Running transaction check
    ---> Package wget.x86_64 0:1.14-10.el7_0.1 will be erased
    --> Finished Dependency Resolution
    
    Dependencies Resolved
    
    ================================================================================
     Package       Arch            Version                     Repository      Size
    ================================================================================
    Removing:
     wget          x86_64          1.14-10.el7_0.1             @base          2.0 M
    
    Transaction Summary
    ================================================================================
    Remove  1 Package
    
    Installed size: 2.0 M
    Is this ok [y/N]: y
    Downloading packages:
    Running transaction check
    Running transaction test
    Transaction test succeeded
    Running transaction
      Erasing    : wget-1.14-10.el7_0.1.x86_64                                  1/1 
      Verifying  : wget-1.14-10.el7_0.1.x86_64                                  1/1 
    
    Removed:
      wget.x86_64 0:1.14-10.el7_0.1                                                 
    
    Complete!
    

9.3.2. Managing software on DEB distributions

Let's try to install the wget software on Debian or Ubuntu linux.
We will use apt and dpkg for the task.
Since we will be performing operations as the superuser, if you haven't already, please set up sudo first.
  1. Update your local cache of available softwares (superuser privileges needed):
    ubuntu@maz03:~$ sudo apt-get update
    sudo: unable to resolve host maz03
    Ign http://nova.clouds.archive.ubuntu.com trusty InRelease
    Ign http://nova.clouds.archive.ubuntu.com trusty-updates InRelease
    Hit http://nova.clouds.archive.ubuntu.com trusty Release.gpg
    Get:1 http://nova.clouds.archive.ubuntu.com trusty-updates Release.gpg [933 B]
    Hit http://nova.clouds.archive.ubuntu.com trusty Release
    Ign http://security.ubuntu.com trusty-security InRelease
    .....
    .....
    Fetched 10.2 MB in 3s (3257 kB/s)                                              
    Reading package lists... Done
    
  2. Query the cache for wget (no privileges needed).
    Note that DEB systems also query descriptions and 'related' softwares.
    ubuntu@maz03:~$ apt-cache search wget
    devscripts - scripts to make the life of a Debian Package maintainer easier
    texlive-latex-extra - TeX Live: LaTeX additional packages
    wget - retrieves files from the web
    abcde - A Better CD Encoder
    apt-mirror - APT sources mirroring tool
    apt-zip - Update a non-networked computer using apt and removable media
    axel - light download accelerator - console version
    axel-dbg - light download accelerator - debugging symbols
    axel-kapt - light download accelerator - graphical front-end
    filetea - Web-based file sharing system
    getdata - management of external databases
    libcupt3-0-downloadmethod-wget - alternative front-end for dpkg -- wget download method
    puf - Parallel URL fetcher
    pwget - downloader utility which resembles wget (implemented in Perl)
    snarf - A command-line URL grabber
    wput - tiny wget-like ftp-client for uploading files
    
  3. Install wget as the superuser:
    ubuntu@maz03:~$ sudo apt-get install wget
    sudo: unable to resolve host maz03
    Reading package lists... Done
    Building dependency tree       
    Reading state information... Done
    The following NEW packages will be installed:
      wget
    0 upgraded, 1 newly installed, 0 to remove and 25 not upgraded.
    Need to get 269 kB of archives.
    After this operation, 651 kB of additional disk space will be used.
    Get:1 http://nova.clouds.archive.ubuntu.com/ubuntu/ trusty-updates/main wget amd64 1.15-1ubuntu1.14.04.1 [269 kB]
    Fetched 269 kB in 0s (1218 kB/s)
    Selecting previously unselected package wget.
    (Reading database ... 51118 files and directories currently installed.)
    Preparing to unpack .../wget_1.15-1ubuntu1.14.04.1_amd64.deb ...
    Unpacking wget (1.15-1ubuntu1.14.04.1) ...
    Processing triggers for man-db (2.6.7.1-1ubuntu1) ...
    Processing triggers for install-info (5.2.0.dfsg.1-2) ...
    Setting up wget (1.15-1ubuntu1.14.04.1) ...
    
  4. Query the deb database and see what files have been installed by wget:
    ubuntu@maz03:~$ dpkg -L wget
    /.
    /usr
    /usr/bin
    /usr/bin/wget
    /usr/share
    /usr/share/man
    /usr/share/man/man1
    /usr/share/man/man1/wget.1.gz
    /usr/share/info
    /usr/share/info/wget.info.gz
    /usr/share/doc
    /usr/share/doc/wget
    /usr/share/doc/wget/copyright
    /usr/share/doc/wget/AUTHORS
    /usr/share/doc/wget/NEWS.gz
    /usr/share/doc/wget/MAILING-LIST
    /usr/share/doc/wget/README
    /usr/share/doc/wget/changelog.Debian.gz
    /etc
    /etc/wgetrc
    
  5. You now decide you don't need wget anymore. Remove the wget software from the system (keep config files).
    Note: you can alternatively 'purge' the software completely as described in the 'purge' section.
    ubuntu@maz03:~$ sudo apt-get remove wget
    Reading package lists... Done
    Building dependency tree       
    Reading state information... Done
    The following packages will be REMOVED:
      wget
    0 upgraded, 0 newly installed, 1 to remove and 25 not upgraded.
    After this operation, 651 kB disk space will be freed.
    Do you want to continue? [Y/n] Y
    (Reading database ... 51129 files and directories currently installed.)
    Removing wget (1.15-1ubuntu1.14.04.1) ...
    Processing triggers for install-info (5.2.0.dfsg.1-2) ...
    Processing triggers for man-db (2.6.7.1-1ubuntu1) ...
    
  6. Discover which files have been left behind by the wget software:
    ubuntu@maz03:~$ dpkg -L wget
    /etc
    /etc/wgetrc
    
  7. Completely remove (purge) all the files installed by wget:
    ubuntu@maz03:~$ sudo apt-get purge wget
    Reading package lists... Done
    Building dependency tree       
    Reading state information... Done
    The following packages will be REMOVED:
      wget*
    0 upgraded, 0 newly installed, 1 to remove and 25 not upgraded.
    After this operation, 0 B of additional disk space will be used.
    Do you want to continue? [Y/n] Y
    (Reading database ... 51119 files and directories currently installed.)
    Removing wget (1.15-1ubuntu1.14.04.1) ...
    Purging configuration files for wget (1.15-1ubuntu1.14.04.1) ...
    

9.4. Adding a user to your VM

You may need to give access to your VM to another user. Given that there are no graphical tools or fancy icons to do the task you are going to user some command line tools.
We are going to add the user 'pemazzon' (Paolo E. Mazzon) to your system.
  1. $ sudo useradd -m -c 'Paolo E. Mazzon' pemazzon
    
    The meaning of parameters is:
    • -m = create a 'home directory' for the user under /home
    • -c = set this as a description of the user
  2. Warning

    It may be necessary to enable password authentications through ssh. Check the file /etc/ssh/sshd_config and be sure that you have
    ChallengeResponseAuthentication yes
    inside. If you modified that file restart the ssh service using
    DEB systems: sudo restart ssh
    or
    RH systems: sudo systemctl restart sshd
  3. Set a password for the user: you will decide a password that will be valid just for the first login. You will force the user to change it immediately.
    $ sudo passwd pemazzon
    
    ... enter twice times the
        password you want to
        set for the user ...
    
  4. Force the user to change his password on first logon:
    $ sudo chage -d 0 pemazzon
    
  5. Mail the user the password you have set.

Important

As specified in the CloudVeneto AUP (Acceptable Use Policy) accounts on a VM can be given only to users who are already CloudVeneto registered users, or to users who have an account at INFN Padova or INFN Labotatori Nazionali di Legnaro.

9.5. Formatting/resizing a volume you just attached

We already show on Section 5.2.2, “Using (attaching) a Volume” how to start using a volume you have attached to your VM. We will give you here some more details.
If you just created an empty volume you first need to create a filesystem on it before you can put some data inside. The volume you just attached is merely 'raw space' and has no concept about files and directories.
You may also think about partitioning your volume, e.g. to split volume space in 'slices', as you may have done installing linux.
Given that in the CloudVeneto you can add as many volumes you want (up to your volume quota, of course) partitioning a volume is simply not recommended.
Suppose now that you have filled the volume space. You have the option to resize it from the cloud dashboard but the result may not be the one you expect until you do some operations from inside your VM.
We are going to resize the volume 'test' from 2 to 4 GB and use the newly available space on a VM.
We will create the volume from scratch. Obviously you can jump to step Section 9.5, “Formatting/resizing a volume you just attached” if you are resizing an existing volume.
  1. Create a 2 GB volume named 'test' and attach it to one of your VM as described in Section 5.2, “Volumes”
  2. Create a filesystem and mount it as described in Section 5.2.2, “Using (attaching) a Volume”
  3. Check the available space is 2 GB and the filesystem is filling up the partition
    ubuntu@maz03:~$ sudo fdisk -l /dev/vdb
    
    Disk /dev/vdb: 2154 MB, 2154823680 bytes
    15 heads, 30 sectors/track, 9352 cylinders, total 4208640 sectors
    Units = sectors of 1 * 512 = 512 bytes
    Sector size (logical/physical): 512 bytes / 512 bytes
    I/O size (minimum/optimal): 512 bytes / 512 bytes
    Disk identifier: 0x00000000
    
    Disk /dev/vdb doesn't contain a valid partition table
    
    ubuntu@maz03:~$ df -k /mnt
    Filesystem     1K-blocks  Used Available Use% Mounted on
    /dev/vdb         2005688  3096   1880992   1% /mnt
    
Let's resize the volume
  1. Umount it first from the VM (if mounted):
    ubuntu@maz03:~$ sudo umount /dev/vdb
    
  2. Detach it from the VM using the dashboard: use "Edit Attachments" and confirm your request.
  3. When the volume is detached the "Extend Volume" option will be available. Select it...
  4. ... and grow the volume to, say, 4GB:
  5. Now attach again the volume to the VM and let's check, from inside the VM, what's happening:
    ubuntu@maz03:~$ sudo mount /dev/vdb /mnt
    ubuntu@maz03:~$ sudo fdisk -l /dev/vdb
    
    Disk /dev/vdb: 4309 MB, 4309647360 bytes
    16 heads, 63 sectors/track, 8350 cylinders, total 8417280 sectors
    Units = sectors of 1 * 512 = 512 bytes
    Sector size (logical/physical): 512 bytes / 512 bytes
    I/O size (minimum/optimal): 512 bytes / 512 bytes
    Disk identifier: 0x00000000
    
    Disk /dev/vdb doesn't contain a valid partition table
    
    The disk size is now 4309 MB, so the system recognize the fact that the volume have grown.
    Let's check the available space:
    ubuntu@maz03:~$ df -k /mnt
    Filesystem     1K-blocks  Used Available Use% Mounted on
    /dev/vdb         2005688  3096   1880992   1% /mnt
    
    we see here that it is still 2 GB!. This is due to the fact that the filesystem has not been touched by the resize operation: the volume service of the cloud has no knowledge of what's inside your volume.
    To use the new space we need to resize the filesystem, obviously from inside the VM, to let it span all the volume:
    ubuntu@maz03:~$ sudo umount /mnt
    ubuntu@maz03:~$ sudo resize2fs /dev/vdb
    resize2fs 1.42.9 (4-Feb-2014)
    old_desc_blocks = 1, new_desc_blocks = 1
    The filesystem on /dev/vdb is now 1052160 blocks long.
    
    ubuntu@maz03:~$ sudo mount /dev/vdb /mnt
    ubuntu@maz03:~$ df -k /mnt
    Filesystem     1K-blocks  Used Available Use% Mounted on
    /dev/vdb         4078888  4120   3873956   1% /mnt
    
    You can now see you have all the 4 GB available.

9.6. Automatically remount volumes on reboot

Connecting a volume to your VM using the 'mount' command is a one-shot solution. If you need to reboot your VM for some reason you will have to re-issue the command again.
Forget to do so might cause the following:
  1. You write data under the /mnt directory (or wherever you mount your volume) thinking you are writing on your volume with, say, 1 TB of space;
  2. The volume is not mounted there so you are writing instead on the same space where your operating system lives;
  3. You eventually fill up your filesystem and your VM crash/starts malfunctioning;
  4. Your VM might not boot anymore and you have to call for help.
We will now create an entry on the /etc/fstab file to remount the volume (the volumes?) upon reboot.

Warning

A big warning! DO NOT edit the /etc/fstab file by transferring it on a windows machine and then back to your VM. Bad things will happen...
The /mnt directory is normally used as the 'mount point' for various devices. Normally you would create a directory under /mnt for each device and attach the device on that directory. Obviously this is not mandatory: you can mount filesystems almost everywhere (e.g. /data, /opt/myprograms and so on.)
All the operations will be performed as the supersuser.
  1. Acquire root privileges
    ubuntu@maz03:~$ sudo su -
    root@maz03:~#
    
  2. Create the 'mount point'
    root@maz03:~# mkdir -p /mnt/volume1
    
  3. Edit the /etc/fstab file: we will use the 'nano' editor for that:
    root@maz03:~# nano /etc/fstab
    
    Your screen should look like this one:
  4. Add a line telling you want to mount the device /dev/vdb under /mnt/volume1 (you have already created an ext4 filesystem on it).
    This should be the content of your file:
  5. Write your file to disk by pressing CTRL+o ...
    ... and confirming with enter.
  6. Exit the editor by pressing CTRL+x. Go back to your normal user by issuing the 'exit' command or by pressing CTRL+d
Now your volume will appear under the '/mnt/volume1' directory everytime your VM boots up. You can also mount the volume just issuing
sudo mount /mnt/volume1
The system will lookup in /etc/fstab and mount the correct volume corresponding to the /mnt/volume1 mount point.

9.7. Specific instructions relevant for INFN-Padova users

In this section we discuss about some specific topics relevant only for INFN-Padova users.

9.7.1. Enabling INFN Padova LDAP based authentication on the Virtual Machine

When creating a custom image, it might be needed to enable a LDAP server to manage authentication for users. This section explains how to enable the INFN Padova's LDAP server for user authentication on the VMs of the Cloud. To do that, the following LDAP client configurations, targeted to SL6.x systems, need to be available on the image used to start the VMs.
First of all, the following packages must be installed:
  • openssl
  • openldap
  • openldap-clients
  • pam-ldap
  • nss-pam-ldapd
  • nss-tools
  • nscd
Then the following files (included in this ldap.t\ ar tar file) must be installed on the Virtual Machine:
  • /etc/openldap/cacerts/cacert.pem
  • /etc/openldap/ldap.conf
  • /etc/pam_ldap.conf
  • /etc/nsswitch.conf
  • /etc/nslcd.conf
  • /etc/pam.d/system-auth-ac
  • /etc/pam.d/password-auth-ac
To do that, it is enough to log on the VM and:
                                                                                     
cd /                                                                                         
tar xvf / path/ldap.tar
Make sure that the following links exist:
                                                                                     
/etc/pam.d/password-auth -> password-auth-ac                                                 
/etc/pam.d/system-auth -> system-auth-ac
Then it is necessary to start the nslcd and nscd services:
                                                                                     
service nslcd start                                                                          
service nscd start                                                                           
chkconfig nslcd on                                                                           
chkconfig nscd on
Then it is just necessary to “enable” the relevant accounts on the VM adding in the /etc/passwd file:
                                                                                     
+name1::::::                                                                                 
+name2::::::                                                                                 
...
and creating their home directories.
Changes done in /etc/passwd could not be applied immediately by the system. In this case a:
                                                                                     
nscd -i passwd                                                                               
should help.

Note

Please note that the SL6x-INFNPadova-x86-64-<date> and CentOS7x-INFNPadova-x86-64-<date> images have already the LDAP client properly configured to use the Padova LDAP server. Using these images it is just necessary to enable the relevant users in /etc/passwd and create their home directories.

9.7.2. Install Mathematica (only for INFN Padova users)

INFN-Padova users can follow these instructions to install Mathematica on their Linux box.
First of all log as root on your VM and mount the directory where the installation kit is available:
                                                                                                 
# mount hellgate.pd.infn.it:/sw/linux mnt/
Go to the relevant directory and launch the installer:
                                                                                                 
cd mnt/Mathematica/<version>/Installer/                                                            
./MathInstaller
The installer will ask the target installation directory (the default is /usr/local/Wolfram/Mathematica/<version>) and the directory where the executables will be linked (default /usr/local/bin).
Finally create the file /usr/local/Wolfram/Mathematica/<version>/Configuration/Licensing/mathpass with the following content:
                                                                                                 
!mathlm.pd.infn.it
That's all! You can now umount the /mnt directory:
                                                                                                 
cd ; umount /mnt

Chapter 10. Miscellaneous topics

10.1. Migrating an image to another cloud

Sometimes it becomes necessary to migrate an image (or snapshot) from one OpenStack cloud to another one.
In short the procedure is the following:
In the source cloud:
  • Download the image/snapshot as a file
In the destination cloud:
  • Upload the downloaded image file to the new environment
To download the image from the source cloud, you need to use the OpenStack CLI. After having sourced the environment script of the source cloud (as explained in Section 3.7, “Accessing the Cloud with command line tools”), run the openstack image list to find the id of the image (photo-slave in the following example):
$ openstack image list | grep photo-slave
| 753e8fa1-3f1b-407d-9294-d22b89ec3184 | photo-slave                 | active |
Then use the openstack image save command, using the relevant id, to download the image file:
$ openstack image save --file photo-slave.qcow2 753e8fa1-3f1b-407d-9294-d22b89ec3184
To upload the image to the destination environment, first of all source the environment script of the target cloud.
Then run the openstack image create command:
$ openstack image create --private --disk-format=qcow2 \         
      --container-format=bare \                                                                                                               
      --file photo-slave.qcow2 photo-slave

10.2. Migrating an instance to another project/cloud

The migration of an instance from one project to another one, or to a different OpenStack cloud, can be done using snapshots.
In short the procedure to migrate an instance is the following: