xenapi: enable user interaction with stack.sh

In XenServer, devstack runs inside a virtual machine. This makes it hard
for the user to interact with stack.sh should a parameter be missing.
This change will create an upstart job (devstack) that runs
stack.sh with console enabled so user can interact with it by running
vncviewer or using XenCenter.

Logging the output is also disabled, stamp files are used instead to
detect the script run status in case install_os_domU.sh is used.

As run.sh.log is removed, standard devstack logging should be used.

The change also removes the environment settings from run.sh, as they
are not needed, they should be specified in localrc. This way user
cannot get different experiences by using unstack.sh/stack.sh or run.sh

Also a proper unstack.sh is called instead of killing screen in run.sh

Change-Id: I7eb12bd74746cc7a1db3aa9fd68ece645a50001d
add assertions for blind grep

blind grep is error-prone.
Add assertions for errors we can not handle

Change-Id: Ibe19085545ecc848498506e8b8ee14e71825b273
Add new stack phase: post-extra

The current existing phases "post-config" and "extra" are not
sufficient to allow local.conf overrides to extra type services
because they run after the services are installed and configured.
This commit introduces a new phase called "post-extra" that runs
after these existing phases. With this change, users are able to
leverage local.conf to provide overridden options to services
like Tempest.

Change-Id: I5d758eebfda804dd1d8cbc3d5cc35ef4dcc8c96f
Closes-Bug: #1249085
split collector service

the ceilometer collector service has been split into two:
ceilometer-collector and ceilometer-agent-notification

Change-Id: I6114fd7f3e063abfa74d48d402dc863bccd249b6
Blueprint: split-collector
Add marconi support to devstack

Marconi has an optional dependency on keystone for authentication. This
code was tested with everything enabled and also with the following
localrc:

    STACK_USER=fedora
    SERVICE_TOKEN=secrete
    ADMIN_PASSWORD=secrete
    MYSQL_PASSWORD=secrete
    RABBIT_PASSWORD=secrete
    SERVICE_PASSWORD=secrete

    disable_all_services
    enable_service qpid
    enable_service key
    enable_service mysql
    enable_service marconi-server

Implements blueprint marconi-devstack-integration
Implements blueprint devstack-support

Change-Id: I13495bcc5c5eb66cee641894e9f84a0089460c8b
Do not create an unnecessary file "ml2"

Previously when running devstack with Neutron ML2 plugin,
an unnecessary file "ml2" is created in devstack directory.
It is because when the first argument is not defined the second
argument becomes the first one.

This change moves the first "options" argument of populate_ml2_config
to the last and checks the given options has a value before adding
them to a file.

Change-Id: I9ff40456798c42216d414d5f8d443e671ab7d497
Close-Bug: #1255853
Adds entries for missing parameters in PLUMgrid plugin

Three configuration parameters were missing for the PLUMgrid plugin.
In this patch all those three have been properly added.

Change-Id: If070aa5eb35678d0984470ebcd43fd99e08bcc8a
Closes-Bug: #1255808
Fix savanna-dashboard git repo clone dest

It breaks savanna d-g jobs due to the ERROR_ON_CLONE=True in d-g
(enforce that nothing will be cloned during the installation).

Change-Id: I0531e1baf7252c31eb63ee5b46c28d1dfa7d0a1b
Attempt to retrieve the vmdk descriptor data-pair

VMDK formats such as monolithicFlat and vmfs require two files to be
fully consumable by the Nova drivers (a descriptor-data pair: *.vmdk and
*-flat.vmdk).
On the upload of the descriptor (*.vmdk), upload_image.sh should attempt to
retrieve the *-flat.vmdk. The same way, the descriptor should be
retrieved when a flat disk is uploaded.
On success, the upload script will be able to use the flat disk as the image
content and the relevant descriptor settings as the image metadata.

Change-Id: I9214754029c46dd60b9e7d606d84d8819a498a8d
Closes-Bug: #1252443
Fix noVNC git repo

Recent change switched base git URL to git.openstack.org.
However, noVNC is only hosted on GitHub.
Change git repo URL for noVNC back to the GitHub one.

Change-Id: Iaa9f570639301be1c29cc400c1c73afcbf637b70
Generate glance image with SCSI bus type for ppc arch

This patch fixes wrong assumptions for bus types of disk and cdrom
on ppc64. Qemu driver assumes IDE bus type for cdrom device, which
is not supported on ppc arch.
Adds capability to add --property key-value to the glance
image-create command.
Using double brackets for portability reasons.

Change-Id: I9f55fa0b6a894a93926e4f8c3d0ea410b5283f9c
Merge "Attempt to retrieve the vmdk descriptor data-pair"
Merge "add assertions for blind grep"
Merge "Fix noVNC git repo"
Merge "xenapi: enable user interaction with stack.sh"
Merge "Add new stack phase: post-extra"
Merge "Do not create an unnecessary file "ml2""
Merge "Adds entries for missing parameters in PLUMgrid plugin"
Merge "split collector service"
Merge "Add marconi support to devstack"