Set vm state error when raising unexpected exception in live migrate

The instance stuck at migrating task_state when unexpected exception.
This is confuse for user, this patch set the vm_state to error.

Change-Id: Ib1b97452bc5e777c66c4d368f71156dbe1e116b7
Partial-Bug: 1397153
Libvirt: SMB volume driver

Currently, there are Libvirt volume drivers that support
network-attached file systems such as Gluster or NFS. This patch
adds a new volume driver in order to support attaching volumes
hosted on SMB shares.

Co-Authored-By: Gabriel Samfira <gsamfira@cloudbasesolutions.com>

DocImpact

Change-Id: I1db3d2a6d8ee94932348c63cc03698fdefff0b5c
Implements: blueprint libvirt-smbfs-volume-support
Use a copy of NEW_NETWORK for test_networks

In test_networks, NEW_NETWORK is used as a request body between unit
tests. "Create a network" API changes a request body internally, so
each test should use a copy of NEW_NETWORK to avoid affecting to the
other unit tests. However one test uses NEW_NETWORK directly.
This patch makes each test use a copy of NEW_NETWORK for avoiding
similar problem in the future.

Change-Id: I9e12fe89956d18dde54bec83c15af258d66e0682
Add a validation format "cidr"

Nova accepts CIDR attributes on network REST APIs and checks its format
on each API. On v2.1 API, the attributes will be checked with JSONSchema.
In addition, Neutron contains its own "cidr" check for its REST APIs.
This patch add a validation format "cidr" by porting Neutron's check from
https://github.com/openstack/neutron/blob/master/neutron/api/v2/attributes.py#L293

Partially implements blueprint v2-on-v3-api

Change-Id: I03601f8fa34fe7617e4f963cdfd4b178eae36565
Adds hacking check for api_version decorator

Because of the implementation of this decorator and the controller's metaclass,
the api_version decorator must be the outermost (ie first) decorator wherever
it is used. This patch adds a hacking check to ensure that this is the case.

This decorator is intended to be used on multiple methods with the same name
which offends F811, so '#noqa' is needed too. This will be fixed in a separate
patch.

Bad:
    @some_decorator # noqa  <-- '# noqa' to avoid F811
    @api_version("2.5")     <-- Error, needs to be the first decorator
    def my_api_method...

Good:
    @api_version("2.5") # noqa   <-- this line still needs '# noqa' for F811
    @some_decorator
    def my_api_method...

Change-Id: I579c0061f03d788c477c5424d4d00ec7a6e721e1
Fix invalid read_deleted value in _validate_unique_server_name()

read_deleted passed to model_query() may have the value 'yes', 'no',
or 'only', but was being passed the value False. The implementation of
model_query caused this to be ignored, and the value of
context.read_deleted used instead.

This change cleans up kwarg parsing in model_query to ensure that all
values explicitly passed in retain their given value, unless they were
explicitly set to None, in which case they are defaulted anyway.  This
last is the expectation of current callers of model_query().

Changes Exception thrown for invalid values to ValueError.

Change-Id: I2d1bbdbd4a391ad3487da905df2e812166a837de
Add cn_get_all_by_host and cn_get_by_host_and_node to ComputeNode

Following the addition of the host field, now we can query by this field.
Here, there are additions for querying N nodes having same host and for
query 1 node having the tuple (host, node)

Here, we also need to provide a compatiblity layer in the ComputeNode
object to make sure that if old compute nodes are still running without
host field, we can still search against them.

Partially-Implements blueprint detach-service-from-computenode

Change-Id: Id23944060fd30da7293f02c31aa71c3150214794
replace httplib.HTTPSConnection in EC2KeystoneAuth

httplib.HTTPSConnection is known to not verify SSL certificates
in Python 2.x. This change replaces use of httplib.HTTPSConnection
with the requests module. It imports config settings related to SSL
verification: ssl.key_file, ssl.cert_file, and ssl.ca_file. It also
adds one config setting: keystone_ec2_insecure. By default, SSL
verification is on, but can be disabled by setting:

keystone_ec2_insecure=true

This patch is based on the keystone middleware ec2 token patch:

https://review.openstack.org/#/c/76476

SecurityImpact
DocImpact
Closes-Bug: #1373992

Change-Id: I8e46d41164e9478b820cad569ba82f25de244620
move eventlet GREENDNS override to top level

Instead of working around this in 3 places, only do the import
override at the top level. There were comments left by mikal to do
this after the pbr switch which is now done.

Change-Id: I3ec281dd016ad904e9f323630324e70a002a037d
Clean bdms and networks after deleting shelved VM

After deleting the shelved offloaded instance with user volumes,
the bdms and networks info should be cleanup. This patch use local
deleting to cleanup the instance info.

Change-Id: Id32385df5b20903d7a019c3590432f3fb1133f90
Closes-bug: #1307791
VMware: clean up unit tests

Remove unnecessary test class

The patch also removes duplicate test methods. These were duplicated
due to the fact that there were two drivers (ESX and VC). Now we
only have the VC driver.

TrivialFix

Change-Id: Idd0f6848ca05e0793506471bb226e2c3530df8be
virt: use instance object for attach in block_device

It will be better if we'll use `instance` like object.

Change-Id: I56b3931da7339d2d350a7d1368dd39c7e398d3e6
Doc: minor fixes to unit testing devref

The unit testing devref had a bit of inconsistent formatting, and a reference
to HACKING.rst which could have been linkifed.  This patch fixes both of those,
and additionally the examples have been updated to reflect the recent split
into functional and unit tests.

Change-Id: I15129808d0d1271c1223e7ab8d8532f8cb2f7293
objects: introduce numa pages topology as an object

Introduces object NUMAPagesTopology also remove the one
from hardware.py which will be not used.

Work-Item: Enhance libvirt driver to report available
           large pages per NUMA node in the host state data

Change-Id: I004c87c5c5b2b42fda04b5754b77a83fadd44281
Partial-Implement: blueprint virt-driver-large-pages
Merge "objects: introduce numa pages topology as an object"
Fix cells RPC version 1.30 compatibility with dict-based Flavors

Version 1.30 of the cells RPC API changed the client to send Flavor
objects instead of dicts. This patch fixes the manager-side compatibility
for <=1.29 clients.

Related to blueprint kilo-objects

Change-Id: Ibde9763071c9bfcc1d7631cbf3452ed0948b4a62
Merge "Libvirt: SMB volume driver"
Merge "Add cn_get_all_by_host and cn_get_by_host_and_node to ComputeNode"
extract fixtures from nova.test to nova.test.fixtures

Start extracting all the setup code in note.test into fixtures which
will let us decompose test setup (and have test base classes that
don't setup everything).

This creates new OutputStreamCapture fixture and StandardLogging
fixture and unit tests for both of them plus tests for the exiting
Conf fixture.

H305/306 have to be turned off because hacking import rules are broken
and continue to believe that fixtures is a nova module not the
absolute one.

Change-Id: I97fdacdf5a6ad2957b0efa1e21ae084d4eb04ab9
move ServiceFixture and TranslationFixture

Move these into the new namespace. No additional testing added at this
point.

Change-Id: I0a45dab39745591a55ffc91b31aed8df5179767a