Implement methods to modify volume metadata.

There are four methods in nova/volume/cinder.py which are
NotImplemented. They are as follows:
  1. get_volume_metadata
  2. delete_volume_metadata
  3. update_volume_metadata
  4. get_volume_metadata_value
These methods are required in cases where nova needs to modify
a cinder volume's metadata, e.g. attach and detach time. It also
completes the cinder.API class.

Change-Id: I360076ae28db43e661466f556425947813c9040e
Closes-Bug: #1296164
Keep Migration status in automatic confirm-resize

function in compute/manager.py _poll_unconfirmed_resizes
will translate the migrate status from finished to error
whenever finds a problem ,Consider the following case

1) _poll_unconfirmed_resizes runs and founds several migrations to be confirmed
2) user deletes the instance, so its task state will be changed to DELETING
3) _poll_unconfirmed_resizes will find the task state is not None,
it will make the migration error status
4) following code in _delete
if instance.vm_state == vm_states.RESIZED:
self._confirm_resize_on_deleting(context, instance)
will fail because migration status already updated

User is able to do the migration operations when they found error
so we don't need to handle this kind of error for them

Change-Id: Ia79dd56c9c1fa8add7dc138d838cbfbac4523ac3
Closes-Bug: #1301696
VMWare Driver - Ignore datastore in maintenance mode

A datastore can be part of a datastore cluster. Any
datastore which is part of a cluster can be put in maintenance
mode. Instances cannot reside on a datastore if it is in
maintenance mode.
This fix ignores a datastore in the following scenarios
1. Stats update - Ignore a datastore during stats update.
2. Ignore a datastore during instance spawn.

Change-Id: Id0cb3f5fdb4d0bc4884cf5405fadc433d4a2b6ba
Closes-Bug: #1330065
Issue multiple SQL statements in separate engine.execute() calls

Some sqlalchemy drivers (eg: mysqlconnector) don't support
engine.execute("sql stmt 1; sql stmt 2;") and require a separate
execute() call for each SQL statement.  After discussions with
sqlalchemy author, he confirmed it would be better to fix callers rather
than attempt to patch in support for multiple statements.

With this patch, nova-manage db sync succeeds using mysqlconnector.

Change-Id: I57e6ecdafe90110eaffe757f0ef0bf7b41b0f3e3
Merge BadRequest tests of "attach interfaces" API

In test_attach_interfaces, there are BadRequest tests of "attach interfaces".
Most parts of them are duplicated.
This patch merges them for the readability and clarifying their purposes.

Change-Id: Ib9dd2840be46dec6b70fe9e3fda35de88f84b4a9
Adds tests for Hyper-V VM Utils

Adds unit tests for Hyper-V VMUtils and VMUtilsV2 classes.

Co-Authored-By: Bogdan Teleaga <bteleaga@cloudbasesolutions.com>

Partial-Bug: #1220256

Change-Id: I87a13697b0a83dc8905573081f735b542bc9bf17
Standardize logging for v3 api extensions

- add log translation hints
- remove unneeded logging imports

Change-Id: I4845ee70c23336b61e5aac70b9b5021e114f8ac5
libvirt: fix _disk_resize to make sure converted image will be restored

During the process of resizing disk if an image is in qcow2 with
partition less the process converts the image to raw.
After the extend we should to restore the original format in all cases
not only if 'use_cow_images' is configured to True.

Change-Id: I792e0fb986a6c5cf9ac477cef0949c8a40099faa
Closes-Bug: #1298976
libvirt: skip disk resize when resize_instance is False

The method finish_migration is called with an extra
parameter 'resize_instance'. It should be used to know
if it is necessary or not to resize the disks and so
call the method 'disk_resize'.

Change-Id: Ia5a9ad7994fa9abab7cf47d36468c46d97065991
Closes-Bug: #1298981
Make API name validation failure deterministic

Makes the testing of a valid name in the API deterministic
with respect to failing when an empty name is supplied
by the user. Formerly it could fail either of two constraints
- must be at least one character long
- does not match the valid regexp which requires one character

The lack of deterministic behaviour was causing issues with
some unittests when PYTHONHASHSEED is not set to 0 because
of the two failure modes. The second constraint for validation
of names is now modified so it allows empty strings but overall the
behaviour remains the same because of the first constraint.

Change-Id: Iea89bb329a8939af5168926717b5c13bdf2a7f62
Partial-Bug: 1350287
libvirt: add serial ports config

This is the first patch for exposing serial ports on openstack
instances. It contains setting up the instance XML config that creates a
TCP chardevice in libvirt.

Author: Ian Wells
Co-autherd by: Vladan Popovic, Sushma Korati

partialy implements blueprint serial-ports

Change-Id: Ia77909dffe6b48be7e6642cbe8bbab27fb6a627d
Fix 202 responses to contain valid content.

Using a webob.exc to generate 202s makes the response return plain
text in the body, which breaks things that expect to be able to
parse data according to the content type. Using a webob.Response
object makes sure the body of the 202 response is empty.

Change-Id: Ifca3081f573407808a2196e71ac1db87180db6ec
Resolves-bug: #1356058
Fix Trusted Filter to work with Mt. Wilson `vtime`

The Trusted Filter expects that `vtime` is returned from the
attestation server in an ISO8601 format, but the Mt. Wilson
attestation server can return it in a locale appropriate string format
instead. There's no way to configure Mt.  Wilson to do the right thing
so we should just try to parse the returned `vtime` as a string
formatted date.

Closes-Bug: #1353029

Change-Id: Ic7351e0463c014321bdb4fcfeba90ac51460b325
Add unit tests for libvirt domain creation

Tests for:
nova.virt.libvirt.driver.LibvirtDriver.get_guest_config
nova.virt.libvirt.driver.LibvirtDriver._create_domain

Partially implements: bp libvirt-lxc-user-namespaces

Change-Id: I4aada9ac06cd423b04a4c6136165c65e64e378aa
Use flavor in confirm-resize to drop claim

Current the confirm_resize() clean the stashed instance_type information
before invoking the drop_resize_claim, and the drop_resize_claim() try to
fetch the old instance_type in system metadata, which will be wrong.

We change the _cleanup_stored_instance_types() to pass back the
to_be_dropped instance_type, and use that information for drop_resize_claim.

Change-Id: Iba36acc75d3840104746596977b33bcd4e010f2e
Add graphviz to list of distro packages to install

Commit a507d42cf5d9912c2b3622e84afb8b7d3278595b makes the
doc builds fail on warnings, so people are probably going
to be running 'tox -e docs' more often. To run that you
need the graphviz package from the distro, so add it to
the list of packages needed when setting up a development
environment.

Change-Id: I47ffe63fddfbbd73c3fc698490ad07235caaf459
Closes-Bug: #1356983
nova-network: treat instance like object in allocate_for_instance

The allocate_for_instance method gets an instance object from the
compute manager so treat it like an object by using dot notation.

This change updates the docstrings for the base API and nova-network API
methods and updates the nova-network API unit tests to use instance
objects when testing allocate_for_instance.

Part of blueprint compute-manager-objects-juno

Change-Id: Ia6d1d8dd46820d48cd9a3f3d4e98fb48e85d1188
neutronv2: treat instance like object in allocate_for_instance

The allocate_for_instance method gets an instance object from the
compute manager so treat it like an object by using dot notation.

Updates the existing unit tests to use an instance object.

Part of blueprint compute-manager-objects-juno

Change-Id: Ia9305a67d1b9bd8601443da3288f710ab02b3f37
Raise Not Implemented error from V2 diagnostics API

In case any driver does not implement functionality of providing VM
diagnostics, HTTPNotImplemented error will be raised from driver.
Nova V2 API should catch the same and pass it with proper error message
for better debugging.

This patch catch & raise  HTTPNotImplemented exception for
V2 "os-server-diagnostics" API.

Adding UT also for the same.

Change-Id: I485d3487d01eba07e3c34f6ef9b5c707af54ebb7
Image caching: update image caching to use objects

Commit 1023e703bd41c2a42b1159af0d9e907e94440b34 added support
for objects. This patch ensures that the parsing of the instances
is correct.

The patch does the following:
1. In the method _list_running_instances it treats all instances
   as an object
2. Fixes the bug mentioned below. The code was making a string from
   None. This would lead to invalid image details.
3. Ensures that the unit tests actually pass instance objects instead
   of an instance dict
4. In relevant tests updates assertEquals to have the expected parameter
   first

Closes-bug: #1354664

Change-Id: Ib028baab4d4c823f22c371a83f7a813c24d77570