Libvirt: Always teardown lxc container on destroy

This fixes a bug where shutting down a libvirt-lxc based instance
would leak its underlying nbd device. This was happening because
_teardown_container would only get called if the domain was
present. After this fix, _teardown_container will always get called
on a destroy, which ensures the nbd device gets disconnected.

Change-Id: I8d0044fd038fef80568c6c86e34719ef679aa890
Closes-Bug: #1367363
libvirt: abort init_host method on libvirt that is too old

Nova has a declared minimum supported libvirt and the init_host
method will log an error message if it sees an older version is
installed. Unfortunately people are frequently not seeing this
error message and then filing bugs about things that are broken
on older libvirt versions.

Turn the log message into an exception so that nova-compute
process startup is completely aborted so people will see the
problem immediately rather than some unpredictable time later.

Closes-bug: #1376651
Change-Id: I87d5a41145b05cc2be3559ab528497fe8355ba85
Use reasonable timeout for rpc service_update()

nova.servicegroup.drivers.db.DbDriver._report_state() is called every
service.report_interval seconds from a timer in order to periodically
report the service state. It calls self.conductor_api.service_update().

If this ends up calling nova.conductor.rpcapi.ConductorAPI.service_update(),
it will do an RPC call() to nova-conductor.

If anything happens which causes the RPC reply to be lost or
never sent in the first place, by default the RPC code will
wait 60 seconds for a response (blocking the timer-based
calling of _report_state() in the meantime).
This is long enough to cause the status in the database to get old
enough that other services consider this service to be "down".

if rpc_reponse_timeout is smaller than report_interval then we could
use the existing RPC timeout, but wait longer won't hurt. So the patch
didn't check it and only use report_interval.

Change-Id: I88743183bce1a534812cfe6110c3fc2892058c53
Closes-Bug: #1368989
Add more input validation of bdm param in server creation

Adds a check to ensure that block_device_mapping is a list and
return 400 Bad Request rather than 500 as currently happens.

Change-Id: I30551bcbb7d658734da158dbb8dd7d825c526896
Closes-Bug: 1379973
VMWare: Remove unused exceptions

Nova exceptions UseLinkedCloneConfigurationFault and
VMwareDriverConfigurationException now is present in
oslo.vmware library, so remove it and replace all of
its usages by usages of exceptions from oslo.vmware

Change-Id: I692030cff49965fac2d8da1bb76580fe9a607b61
Use response.text for returning unicode EC2 metadata

When serving meta-data that contains unicode, use the webob Response.text
instead of Response.body.  This also adds a test that triggers the bug in
addition to testing that all meta-data offered by the server can be served.

Closes-bug: #1380792

Change-Id: If2ddaa97bbc89cf574205c0327e3caa02dd503fc
Porting SecurityGroup related controller into v2.1

This patch port SecurityGroup, ServerSecurityGroup and SecurityGroupAction
controller into v2.1 security_groups extension.

This patch just move the v2 code into v2.1 and share the unittest. The other
v2.1 related fix and improvement will be addressed by subsequent patchset.

Partially implements blueprint v2-on-v3-api

Change-Id: I93951ce677b74ebed7db1b2fff1f788344806dba
libvirt: remove pointless HostState class

Now that the get_host_stats method is removed, the only user
of the HostState class passes 'refresh=True' every time. Given
this, there is no point in caching the data in the HostState
class. Instead just populate the data directly in the
get_available_resources method.

Blueprint: virt-driver-get-available-resources-object
Change-Id: I19c391a1bae27c3e8d891ecf77ceeaf170299be3
libvirt: fully parse PCI addresses to integer data type

When parsing node device XML, the PCI addresses were left in their
string form, which is unhelpful for later code wishing to use their
raw values. Fully parse them to the integer form instead.

Blueprint: virt-driver-get-available-resources-object
Change-Id: I46b9c33e08122eb7c28601752ecfc652e09175c3
Clean up the naming of PCI python modules

Something that has been bothering me when looking at the PCI code is
that the modules in /nova/pci/ unnecessarily were prefixed with pci_.
For instance, nova/pci/pci_device.py, whereas all other module directory
organizations for all other code in Nova uses the directory name to
demarcate a submodule.

This patch simply gets the nova/pci/ modules in-line with the standard
set everywhere else in Nova and cuts off the pci_ prefix from the module
names.

Change-Id: Ia5ca13f674f15b41e22b8488efe44eb926e5e4eb
Compute: use instance object for vm_state

Use object format instead of dictionary.

TrivialFix

Change-Id: I6573d2d0d95cbce12733ecb9c9f6d829269d2d10
Pull extra_specs_ops tests from test_host_filters

First in a series of patches to uncruft the 2000+ line
test_host_filters.py unit test file. This patch moves the
extra_specs_ops tests into its own test file.

Change-Id: I200265be6fbc2df1877f01531f443cff8337cebf
Port update method of cloudpipe_update to v2.1(v3)

This patch adds couldpipe_update's `update` method to cloudpipe controller
add validation schema to cloudpipe, have v2 unit test cases shared between
v2.1 and v2.

Partially implements blueprint v2-on-v3-api

Change-Id: Ibf39d7fa7301bcba6e259ae88a73c20f64b7fc36
Remove unuseful parameter in cloudpipe api(v2/v2.1)

project_id in _get_cloudpipe_for_project is useless, remove it.

Change-Id: Ie8cb7a6d8ce0678f04200c9fa9de58b4c92bd40d
VMware: Remove redundant fake.reset() in test_vm_util

We fake.reset() in setUp(), so no need to do it in tearDown() too.

TrivialFix

Change-Id: Ie6e2afd351eefe2ff15d7ec74b9bceb65a00422d
Revert "Replace outdated oslo-incubator middleware"

This is breaking the Juno -> master grenade path. As we are using the old
api-paste.ini file which requires the oslo-incubator copy of request_id

This breakage didn't happen from Icehouse -> master since the icehouse
api-paste.ini didn't have request_id as a middleware. request_id is only
used as middleware in the V21/V3 API and not the V2 API.

Closes-Bug: #1383757
This reverts commit 641de5666f38532252e91735a1cd2611d96c489a.

Change-Id: I1ca6bddf9d9370529e999efcc7eae60622e19e6f
Merge "Libvirt: Always teardown lxc container on destroy"
Merge "Compute: use instance object for vm_state"
Merge "Revert "Replace outdated oslo-incubator middleware""
Merge "libvirt: remove pointless HostState class"