deploy: Add roles module

This is used to get the target component for each node.
It has a role-component mapping and should be used like:

NS = roles.get(constants.NS, ctx)

where the ctx is a Context() with proper node, role, cluster, setup.

Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
deploy: Add vcluster module

..with the relevant helper methods for creating a local
virtual cluster with KVM. snf-deploy base script will import
this module and use it if vcluster command is requested.

Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
deploy: Cleanup lib module

Remove unused Env, Status, Conf classes. Host and Alias classes have
been substituted by FQDN. The latter will be used for Node and
Cluster info.

Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
deploy: Make snf-deploy script use new modules

Use config, context, status and vcluster modules.

The main method of snf-deploy script will initialize them with
the args passed. Additionally make some minor changes related
to the command line:

 - snf-deploy all -> snf-deploy synnefo.
 - add --setup option
 - add --cluster option
 - use --<name>-dir for template, state, package dirs

Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
deploy: Remove unused utils module

Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
deploy: Refactor components module

Up until now components were simple objects decoupled from fabric
execution. Their methods were just returning a list of bash commands
which utils module was executing in the correct context.

This patch changes this rational. Specifically:

A Component() gets initialized with an execution context that is a
configuration snapshot of the target setup, cluster and node. A
component implements the following helper methods: check, install,
prepare, configure, restart, initialize, and test. All those methods
will be executed on the target node with this order during setup.

Additionally each Component class implements admin_pre, and
admin_post methods which invoke actions on different components on
the same execution context before and after installation. For
example before a backend gets installed, its FQDN must resolve to
the master floating IP, so we have to run some actions on the ns
node and after installation we must add it to cyclades (snf-manage
backend-add on the cyclades node).

Component() inherits ComponentRunner() which practically exports the
setup() method. This will first check if the required components are
installed, will install them if not and update the status of target
node.

ComponentRunner() inherits FabricRunner() which practically wraps
basic fabric commands (put, get, run) with the correct execution
environment.

The fabfile practically gets the target nodes for each role from the
configuration and uses the roles module which knows the
role-component mapping to get the target component with the proper
execution context. Then it just invokes the setup() method.

Each component gets initialized with an execution context and uses
the config module for accessing global wide options. The context
provides node, cluster, and setup related info.

We introduce some helper decorators that wrap methods of Component
class and update its execution context (self, self.ctx):

- parse_* executes the wrapped method and parses the output.
  The desired vars (user_id, backend_id, user_uuid, etc.) are
  stored in the execution context and made available to other
  methods.

- update_admin initializes the admin roles (NS, Astakos,
  Cyclades, etc.) of the current context and make them available
  under self.NS, self.ASTAKOS, etc. These have the same execution
  context of the current components besides the target node
  which gets derived from the corresponding config.

- update_cluster_admin initializes the cluster's admin role
  and make it available under self.MASTER.

- run_cmds will run the list of returned commands in the target node

The update_ns() method of NS component gets the
self.ctx.admin_fqdn and invokes nsupdate accordingly.

Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
deploy: Refactor fabfile

The fabfile is now responsible to find the target nodes for each
execution context. It practically gets the corresponding component
for each role and invokes its setup() method.

To get the execution context for each action we use fabric's execute()
and update the context based on env.host.

Some things can be done concurrently (setup of each VMC node).
To do so, We use the fabric's @parallel decorator.

Additionally we support setup of a single component/role on a target
node or even the execution of a single method of a component.

Signedoff-by: Dimitris Aragiorgis <dimara@grnet.gr>
deploy: Use archipelago related components

Make Cyclades and Pithos require PithosBackend and Archip.
Make VMC require also ExtStorage and ArchipGaneti.

Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
deploy: Make ganeti1 a synnefo backend

This is done by making True the synnefo option in the ganeti1
section. This way the VMC component will require some other
components (Image, GanetiCollectd, ArchipGaneti, etc.) that are
needless in a regular Ganeti cluster.

Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
deploy/ci: Update conf files for ci schema

..and invoke `snf-deploy synnefo` instead of `snf-deploy all`.

Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
docs: Update snf-deploy docs

..to reflect the latest changes.

Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
deploy: Use cname instead of fqdn

..for astakos, cyclades, pithos, cms, db, mq, nfs.

Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
deploy: Add simplejson as requirement

Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
deploy: Install blktap in Achip component

Currently latest version of archipelago uses blktap instead of
xseg. Install official blktap-dkms and our custom blktap-utils
for this purpose.

Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
backend: Change segment type and name

..to posix and archipelago respectively so that it is working
with the latest archipelago version.

Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
deploy: Redirect / to /astakos/ui

Since pithos is the last registered service found in show_urls
synnefo.live redirects to synnefo.live/pithos/ui which eventually
cannot load.

Make apache redirect to /astakos/ui by explicitly defining a
RewriteRule in vhost file.

Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
deploy: Workaround for cert_override.txt

For cyclades, pithos, astakos, and csm use firefox_cert_override.py
script (created by kpap) to manually create proper entries so that
the end user running firefox does not have to accept all
certificates before having a working environment.

This will add /etc/iceweasel/profile/cert_override.txt on client
node with entries for:

astakos.synnefo.live:443
cyclades.synnefo.live:443
pithos.synnefo.live:443
cms.synnefo.live:443
synnefo.live:443

and the corresponding IPs (currently needed for cyclades UI).

For further info about cert_override.txt file see:

https://developer.mozilla.org/en-US/docs/Cert_override.txt

Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>