xenapi: Tag volumes in boot from volume case

Fixes bug 1064134

Since https://review.openstack.org/#/c/10574, we require that all
openstack volumes are tagged, so as to distinguish them from other
disks that could be deleted on instance termination.

Change-Id: I965f88bf41d39be88f2331ad081f5cec71a65a0f
Declare vpn client option in pipelib

Due to some import refactoring, nova.network.manager isn't always
imported when running pipelib commands, so using the cnt_vpn_clients
option fails. A failing test could not be added because our testing
code imports nova.network.manager so the code config option always
exists in testing.

Fixes bug 1069573

Change-Id: I92b702556bfef2610b86a952dd6f9207fbab102a
nova-compute sends its capabilities to schedulers ASAP

Fixes Bug 1068114.

When a nova-compute runs for the first time, after an entry in
compute_nodes table is created, it takes some time for the
capabilities to be sent to schedulers. It may make the schedulers
send unacceptable requests to the nova-compute. This patch eliminates
it by sending the capabilities before creating the entry in compute_node.

Change-Id: Id9851d9a6f0305256dfa6dfb63ab508e6f519fa2
limit the usage of connection_info

Related to blueprint xenapi-volume-drivers

The connection_info parameter, that is coming from volume contains only
two keys, 'driver_volume_type', and 'data'.
See:
https://github.com/openstack/nova/blob/master/nova/volume/manager.py#L375

parse_volume_info was checking the connection_info for 'auth_method',
and authentication parameters. These informations live inside
connection_data. This patch is fixing this, and covering with a test.

Also rename the generic data variable to connection_data, to make it
easier to read the code.

parse_volume_info thus no longer depends on the whole connection_info,
only on the connection_data.

Also clean-up some copy-paste test setup code.

Change-Id: I5289120d46176c8528c3317a650269c4f384f930
Remove mountpoint from parse_volume_info

Related to blueprint xenapi-volume-drivers

parse_volume_info method did too many things. It was checking the
connection_data for iscsi information, and also checked the mountpoint.
This change extracts the mountpoint conversion, and performs that check
earlier in the volume attach process.

Change-Id: I7c26e813319f6d9de287782ad188b22804d032a9
SanISCSIDriver SSH execution fixes.

In ssh_execute methon in nova/utils there was a problem with cmd passed to it.
From SanISCSIDriver and all other places it is called from cmd is a string.
But in ssh_execute method cmd was processed as a list. This can make problems
with SanISCSIdriver using.

Also it may be useful to pass execute parameter to this SanISCSIDriver class
to have the opportunity of overriding _execute method (as it is in
VolumeDriver class, that is the root of this class hierarchy). That's why
special checking was added.

Fixes bug 1070489

Change-Id: I73f74f9d095a2c4316cab88148afe6a0bde44c5b
Remove is_admin_context from sqlalchemy.api

is_admin_context was used to detect deprecated empty contexts,
and was last updated in 2010

Change-Id: I06c61d6a508bef7ea30054a6f1df3fe6b2d6dfd8
More specific exception handling in migration 091.

The original code would catch and discard any exception generated when
trying to create or drop fkey constraints using the SQLite engine. This
has the side-effect of allowing the DB to become inconsistent (by
leaving a `migration_tmp` table around), causing future migrations to
fail and making bugs even hard to track down.

Long term, we should fix the underlying cause of these exceptions,
either by switching to Alembic or by fixing SQLALchemy-Migrate to fully
support dropping of constraints via a temp table.

For now though, tightening up the exception handling will at least make
any future bugs a bit easier to diagnose.

Fixes bug 1070559

Change-Id: If549d65afa758b6f8a0db6fc8478a5b0c9277e23
Remove deprecated root_helper config

root_helper was deprecated in Folsom and was replaced by rootwrap_config

Change-Id: Ia2f5e67f834c01c07272c61323d01599006b7018
Handle the case where we encounter a snap shot correctly.

Fix the list in the qemu output and break out of the loop
when we encounter that instead of failing to parse.

Fixes bug 1070088.

Change-Id: I5f501c599b94e0ac13892c97d60ec740ac7e641d
Don't require quantumclient when running nova-api.

Handle ImportErrors's when importing the quantumv2 API
in the Nova API servers controller. This makes it possible
to run nova-api when quantumclient is not present.

This was recently broken in commit 83a8418 which changed how we
import the quantum v2 API class.

Fixes LP Bug #1070509.

Change-Id: I3c452c3bd88cb5b431cb23e525ff78e3d677e1a8
Allow loading only selected extensions

To help with speeding up API tests, allow loading only a selected list
of extensions or resources. This is done using init_only=... parameter
for the fakes.wsgi_app() factory.
The change is backwards compatible, so existing tests do not require
modification.

Implements bp:api-tests-speed

Change-Id: Ibdcedd522728cb10294fa69e45732b7f125b3cbb
Speed up api.openstack.compute.contrib tests

Use init_only parameter to select the minimal number of extensions and
resources needed for test execution.
Improvement on a test machine was from 59.770s to 15.218s (508 tests).

Implements bp:api-tests-speed

Change-Id: Id27bd5c29eaee1ef7225b7aeb43210063aaf0d4e
Speed up volume and routing tests

Load only required extensions when testing volume api. Initialise only
required routes for extensions test.
Both result in minor time savings when testing.

Implements bp:api-tests-speed

Change-Id: I8078feda34635c91d4e57bae3bb88475fe848cb8
Fix Incorrect Exception when metadata is over 255 characters

Need to be in sync with Essex. Throw a new exception in api.py when key or value
is over 255 chars. Catch this exception end return back HTTP error 413.

add extra lines in exception.py to make pep8 happy

Fixes bug 1004007

Change-Id: I67694e3a9600b7dd0b80c519327327568751c2a4
Refactor: move find_guest_agent to xenapi.agent

Related to blueprint xenapi-config-drive

find_guest_agent is related to agent, thus moved to that volume. With
all the agent calls in one module, making the agent optional will be as
easy as implementing a null agent.

Change-Id: I5c8cf2fd3d08518e1794a2018429162e4bd767d7
Consistent Rollback for instance creation failures

Fixes Bug 1065092

Adds a single QUOTA.rollback for all exceptions rather than performing
the rollback individually. Ensures that no stale "reservations" are left
in place after a failed instance creation.

Change-Id: I354726b651ae8feb6153d5865e0b14b54c73314b
Return empty list when listing servers with bad status value

API indicates the status parameter is a string, hence status=BALONEY is a valid
filter that should return zero records since no records match that status. The
return should be an empty list and 200 OK.

Fixes bug 1061712

Change-Id: If796110a18dd75ab5c5d4ba41e9c3b58180341f8
Remove database usage from libvirt imagecache module

This removes the database instance list operation from the libvirt
imagecache module and pushes it up into compute/manager.py.

This is a step in the proposed strategy to move db usage out of the virt
drivers up into compute/manager.py in support of bp:no-db-compute.

Change-Id: I2ea39f14304ed4b44c7ac9650703a5f780411c18
Add better log line for undefined compute_driver

If compute_driver is undefined, the message in the log files will be an
exception ending with:

  AttributeError: 'NoneType' object has no attribute 'rpartition'

Provide a better error message instead.

Change-Id: I5f32a376d4f53b0d727e378907ad41c3499121fe