Move volume_clear option to where it's used

The `volume_clear` and `volume_clear_size` option are used in `lvm.py` so we
should move the definition there.

Change-Id: Ic619c2db0bb1b5ccc7dcbec1a500c3dcb1110b26
Correct a maybe-typo in pci_manager

the variable 'dev' in set_hvdevs are only used for list comprehension,
it does not make sense to use 'dev' again for updating self.stale.

Change-Id: I03f31bfcfc56a43e43eb88845e1d562f19a3a09e
baremetal: Remove depenency on libvirt's fetch_image method

It's messy for virt-drivers to depend on each other, so this patch alters
baremetal to use the cross-driver fetch function, instead of the libvirt
version.

Change-Id: I910c621452fa7eb9add417a8738e5757aeb1a48c
libvirt: Fix kwargs for _create_image

Rescue uses the `suffix` and `disk_images` kwargs of `_create_image`, but
passes them in as positional args. We should use explicit kwargs here to be
clear that rescue depends on both of those parameters being present.

In addition, disk mapping was being passed in as kwarg, but is actually a
positional arg, so we should pass it in as a positional arg to be clear that
it's required.

Change-Id: Iaa990729b4bb3569f4c71ceaf1adabed58f73026
Fix swap_volumes

The attach and detach calls to cinder are better handled by cinder this
patch removes them. There is another patch against this bug to modify
cinder.

The attach call is being made against the new volume after the
migrate_volume_completion request. This means it may be done after the
new volume ID in the db has been deleted.

The detach call is being run at the end on the old ID and so may end up
detaching the migrated volume from cinder's perspective.

Change-Id: I3f2d98e9e473905a3de2f02e00c92ae3065e6ae7
Partial-Bug: 1316079
Add warning to periodic_task with interval 0

In Ia227f4c4e69ecf361ab02d1d17a3010303650104, warnings were
added for several options but instance_delete_interval was
left out by mistake.

Closes-Bug: #1188226
Change-Id: I20b396d664e40252743cac4f2a49f4f34882b32f
move dict copy in prep for object-ify flavors

This change is to help split up the patches for object-ifying flavors.
It moves a dict copy of instance_type from the tests to
get_flavor_by_flavor_id().

Change-Id: I0082a56aa637169fd6e1db3aad56877c9f745bc5
refactor flavor db fakes in prep for object-ify flavors

This change is to help split up the patches for object-ifying flavors.
It refactors db fakes to provide default values for fields that are
not nullable in flavor objects.

Change-Id: I18575703cd5c52d4f446c9bca1edd7baf33b416f
refactor flavor manage tests in prep for object-ify flavors

This change is to help split up the patches for object-ifying flavors.
It refactors flavor manage tests to provide default values for fields
that are not nullable in flavor objects.

Change-Id: I59f0254269379ace2186ed90810c5d1a9f09fed7
Update InstanceInvalidState output

Update InstanceInvalidState output because the original format
is not valid and will lead to none readable output.

Change-Id: I8481facd8100ce9a6f87ea208c06d99016a89534
consistently set status as REBUILD when rebuilding

Consider an instance is in SHUTOFF status(stopped). When a rebuild
is issued on this instance, the status does not display as REBUILD.

Similarly, when a rebuild is issued on an instance and the operation
fails, the instance turns to ERROR state. Now if we issue a rebuild
again, the status does not display as REBUILD.

This patch fixes the above mentioned issues by displaying the correct
status i.e REBUILD.

Change-Id: Id11c9d31683a049c61fd751dc0ddabaf22c96cb9
Closes-Bug: #1362733
xenapi: deal with reboots while talking to agent

The latest version of the agent has started changing the hostname
through the official windows APIs, when resetnetwork is called. This
means a reboot might happen after resetnetwork.

A reboot happens when polling for the version. Until now we wait for the
call timeout, before fetching the new dom id. This change ensures that
if we spot a reboot, the plugin exits early rather than keeping polling
the wrong dom id.

Turns out its best to wait for the dom_id to change, before trying
to poll the agent again. Once the dom_id in xenapi has been updated,
the xenstore keys are always in place.

Trying too early leads to lots of reboot detections because we are
retrying with the old dom_id. XenServer continues to return the old
dom_id for a little while after the reboot.

Closes-Bug: #1370999

Change-Id: Id0bf5b64f2b271d162db5bbce50167ab1f665c87
VMware: remove unused variable from tests

Commit 6c4b89a2ad51f92707acc01908b63bc507ad8fe7 added a variable
which was never used.

Change-Id: I6d9391359f25feedbdf822a606c07b0ae9a735be
console: make unsupported ws scheme in python < 2.7.4

Due to a bug in python < 2.7.4, urlparse won't parse a query string
in a URL where the scheme is not recognized:

http://bugs.python.org/issue9374

Commit 1dda5ef75a5c141c316639716820b0c37773a9e3 added some tests for
websocketproxy that uses urlparse to get a token but these don't work
on python < 2.7.4 since the scheme in the tests is "ws://".

In the actual code, raise an exception if python < 2.7.4 and an
unsupported URL scheme is used.

DocImpact: The "getting-started-with-vnc-proxy" page in the
"admin-guide-cloud" docs should be updated with a note that the web
proxy console URLs do not support the websocket protocol scheme (ws://)
on python versions less than 2.7.4.

Change-Id: Ib6e0fc4e50c1ea144a9c538099e23597d66b429b
Closes-Bug: #1376078
Port fping extension to work in v2.1/v3 framework

Ports v2 fping extension and adapts it to the v2.1/v3 API
framework. API behaviour is identical.

- unittest code modified to share testing with both v2/v2.1
- Adds expected error decorators for API methods

Partially implements blueprint v2-on-v3-api

Change-Id: I8dc4ede46826bf6062097a3b4ba6be9a09a1bd12
Make v2.1 API tests use v2 URLs(test_[a-e].*)

Most v2.1 API tests are using v3 URLs, because there didn't exist
a fake router for v2.1 at the time. However, now the fake router
exists.
This makes v2.1 API tests of use v2 URLs for testing v2.1 API as
v2 compatible API for test_[a-e].*.py.

Partially implements blueprint v2-on-v3-api

Change-Id: I9b5901ca294d10a9ded52603f9cc46155b07009d
libvirt: add _get_launch_flags helper method in unit test

In libvirt driver , we have some logic to generate launch_flags in
method _create_domain_and_network, but in unit test we hard code
launch_flags with '0'. This may lead unit test failure in actual test.
We can't extract the logic to a method in libvirt/driver.py, so add
a helper method with same logic for unit test.

Change-Id: I39ce0162b856daf18488e006b0e424c11811e0c0
Use image metadata from source volume of a snapshot

When booting an instance with the block device mapping V2 API and
source=snapshot, the image metadata is not retrieved and stored in
instance system_metadata.  Metadata is not exposed on a volume snapshot
so it is grabbed from the volume that the snapshot is from.

Change-Id: I96d6c2198b2f94470ab9dfee5e4bddb13992d8d5
Closes-bug: #1375478
Support message queue clusters in inter-cell communication

Since cells use oslo.messaging to specify and store the message queue URL,
multiple hosts can be specified by manually modifying that URL in the database.
However, there is no way to specify multiple hosts during cell creation phase.
This patch adds a --broker_hosts option to `nova-manage cell create` command,
which is analogous to the rabbit_hosts option in nova.conf and can be used to
specify multiple message queue servers as a comma separated list. Each server
is specified using hostname:port with both being mandatory. The existing
--hostname and --port options continue to remain but are only considered if no
--broker_hosts is specified.
Internally, each host is converted to a oslo.messaging.TransportHost
and added to the generated TransportURL.
This patch also adds unit tests for creation of the TransportHosts from
user given input.

Change-Id: I14de860b1d12f3e2c0169b58651d580792d6ce0e
Closes-Bug: 1178541
Update NoMoreFixedIps message description

The exception NoMoreFixedIps within Nova is a generic exception.
It is used to indicate that the subnets within the network no longer
have any available fixed IPs for use.

The original message was non specific.  This change set provides an
updated message that indicates which network has exhausted its fixed
IP set.

This is important for operators that are deploying instances with
multiple networks.

Change-Id: Ib8b854381b616f83e82bc573f85e109f019897dc
Closes-Bug: #1366982