xenapi: enable user interaction with stack.sh

In XenServer, devstack runs inside a virtual machine. This makes it hard
for the user to interact with stack.sh should a parameter be missing.
This change will create an upstart job (devstack) that runs
stack.sh with console enabled so user can interact with it by running
vncviewer or using XenCenter.

Logging the output is also disabled, stamp files are used instead to
detect the script run status in case install_os_domU.sh is used.

As run.sh.log is removed, standard devstack logging should be used.

The change also removes the environment settings from run.sh, as they
are not needed, they should be specified in localrc. This way user
cannot get different experiences by using unstack.sh/stack.sh or run.sh

Also a proper unstack.sh is called instead of killing screen in run.sh

Change-Id: I7eb12bd74746cc7a1db3aa9fd68ece645a50001d
xenapi: display IP and DevStack result on console

The devstack setup service can update /etc/issue, displaying the status
of the installation and the VM's management IP. With this change, after
the devstack service finsihed, the login prompt will look like this:

OpenStack VM - Installed by DevStack
  Management IP:   10.219.3.108
  Devstack run:    SUCCEEDED

DevStackOSDomU login:

This helps people to log in to their system.

Change-Id: Idd6bbd5faf9ced5618cd3e95191bfc3b89473fa2
add assertions for blind grep

blind grep is error-prone.
Add assertions for errors we can not handle

Change-Id: Ibe19085545ecc848498506e8b8ee14e71825b273
enhance logging for depend tracking mode

Add some extra log, to make the console output more understandable

Change-Id: If80cb9dba1031ad268bf9b6266ca83f93071a2ad
Add new stack phase: post-extra

The current existing phases "post-config" and "extra" are not
sufficient to allow local.conf overrides to extra type services
because they run after the services are installed and configured.
This commit introduces a new phase called "post-extra" that runs
after these existing phases. With this change, users are able to
leverage local.conf to provide overridden options to services
like Tempest.

Change-Id: I5d758eebfda804dd1d8cbc3d5cc35ef4dcc8c96f
Closes-Bug: #1249085
Add pip folder to ignore list

Change-Id: I26da80b9e09774b9940d382a32cf69e75c61a527
Switch the base URL to git.openstack.org

The git repositories on github for openstack are mirrors of the primary
repositories, which are hosted on git.openstack.org, so switch as much
as I can to using the primary, rather than the github mirror.

Change-Id: Idcfda49a691582055256b830c61e098f4a271339
Mute false alarm when installing docker

In install_docker.sh, we will restart docker service, then connect
to /var/run/docker.sock with retry mechanism.

At the first contacting with /var/run/docker.sock, when docker
service is not ready, it may complain some error.

Mute this false alarm.

Change-Id: If00a18d2e3ddee951662e272d47ae84215f16ad2
Closes-Bug: #1252087
Adding optional colorized log output for trove

Change-Id: Ibdaed9c2d1527b4c38551efbc147597e2a668b1a
Allow deploying keystone with SSL certificates

Allow providing certificates through environment variables to be used
for keystone, and provide the basis for doing this for other services.
It cannot be used in conjunction with tls-proxy as the service provides
it's own encrypted endpoint.

Impletmenting: blueprint devstack-https
Change-Id: I8cf4c9c8c8a6911ae56ebcd14600a9d24cca99a0
Shorten PKI token logging

Log only the first 12 characters of auth-token for the Swift
API, since PKI based auth-tokens from keystone can huge (>> 2K).

Also tidy up a comment.

Change-Id: Ib784e8ecdcb7e371fe03458c7fd82b4460fa82b9
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Use the swift logging adapter for txn IDs

Change-Id: I2b2308eb9606279cffc1965fc3b86e9597d63e87
Signed-off-by: Peter Portante <peter.portante@redhat.com>
freshen the LDAP support

* Build the base DN from a given domain name
* Remove all hard-coded names to allow configuration of base DN
* Fix manager DN (cn=Manager,dc=...)
* Add ldap init_ldap()
* Add support for clean.sh

Change-Id: Ieb69be9740653645b8e000574ad3fe59a0f97540
split collector service

the ceilometer collector service has been split into two:
ceilometer-collector and ceilometer-agent-notification

Change-Id: I6114fd7f3e063abfa74d48d402dc863bccd249b6
Blueprint: split-collector
Merge "Shorten PKI token logging"
Merge "Use the swift logging adapter for txn IDs"
Make tempest L3 capable plugin aware.

With this patch, the public network config in tempest.conf will be
done for the plugins that support L3.

Change-Id: I820fe300fac45ff92d1281ff0c43ebc137783210
Assign unique name to each fake nova-compute

Without a unique name, the scheduler (and anything else, really) will
consider each of these nova-compute processes as being one and the same,
so only one entry in the services table, only one hypervisor seen by the
scheduler, etc. Assigning unique names lets us simulate an arbitrary
amount of nova-computes which is very handy for benchmarking the
scheduler.

Change-Id: Ie03aad81bd2a8e73b876a9eae934bc00bf2f71e9
Add marconi support to devstack

Marconi has an optional dependency on keystone for authentication. This
code was tested with everything enabled and also with the following
localrc:

    STACK_USER=fedora
    SERVICE_TOKEN=secrete
    ADMIN_PASSWORD=secrete
    MYSQL_PASSWORD=secrete
    RABBIT_PASSWORD=secrete
    SERVICE_PASSWORD=secrete

    disable_all_services
    enable_service qpid
    enable_service key
    enable_service mysql
    enable_service marconi-server

Implements blueprint marconi-devstack-integration
Implements blueprint devstack-support

Change-Id: I13495bcc5c5eb66cee641894e9f84a0089460c8b
Fix neutron log format with colorization

Closes-Bug: #1254817

Change-Id: I6de17ef15c18e2f8ab246934461a2b7c6ae4f95f