Fixes get_vm_storage_paths issue for Hyper-V V2 API

Add get_vm_storage_paths function to vmutilsv2.py

get_vm_storage_paths implementation on Hyper-V V2 API changed
compared to previous V1 API. In previous V1 APIs, using
Msvm_ResourceAllocationSettingData class can get storage
paths, but in V2 API, only using Msvm_StorageAllocationSettingData
class can get the storage paths.

Closes-Bug: #1248463

Change-Id: Ica73221524f162a6ffcd9dc64ee33c85fb5ad31d
Fix a lazy-load exception in security_group_update()

When security_group_api is nova, the update_security_group api raises http
500 error: group rules fails with lazy load exception. Because the rule
infos of the security group will be return in response body, but the rule
attribute hasn't been preloaded.

Closes-Bug: #1244018
Change-Id: Ie07879ac22e4add90f75968778bf334915e8349c
Check Neutron port quota during validate_networks in API

Unless ports are passed into Nova it will create ports on the
requested networks as part of the network allocation in
the compute manager. However if the user exceeds their
port quota the instance will end up in an Error state, having
first been re-scheduled a number of times.

It would be much better if the quota failure was detected as
part of the network validation in the API server, so that an
error can be reported to the user and the creation failed.

A full fix would include reserving or creating the ports at
this stage, but there is no reservation mechanism in the
Neutron API, and port creation depends in some cases
on mac addresses only available on the compute manager.

Instead this change just validates the quota and adjusts the
max_count to be consistent with that quota, which doesn't
guarantee that the create will work, but does catch the
majority of cases.

Refs bug: 1172808

Change-Id: Iaaee059a6746fad68049712f94b2f8cfea6ab8dc
LXC: Image device should be reset in mount() and teardown()

When fixing bug#1208387, image device name is returned in function
setup_container. But it's missed to reset in function mount() and
teardown().

This patch is to fix it. We now just return the device name when
mounting disk image during setting up container and remove the
member variable 'device' from Class _DiskImage added in last patch.

Closes-bug: #1243108
Related-bug: #1208387

Change-Id: I2aaed6f3367edce4fb23b78f0e979440954609a8
libvirt: consider minimal I/O size when selecting cache type

The "none" cache type is not usable when a file-system
has stricter alignment and minimal I/O size requirements,
for example the xfs file-system created with 4kiB sector size,
requires all DIRECT I/O operation has to be done with
 integer multiple of 4 kiB count/size.

qemu assumes direct I/O is doable with 512 byte minimal size,
so the  _supports_direct_io method needs to check is
the direct I/O doable with 512 byte size.

Change-Id: I5e1ba4e8afa87519cb0ed1120c213f7c318ccef2
XenAPI: Speedup host_ref cannot change - get it once

The host we run on is fixed - getting this each time from XAPI
is expensive

Change-Id: Ifee9a778f3ed855ef52b001de3fc2ebbd0b972cf
Refactor PXE DHCP Option support.

The previous support imported pxe into driver, but driver is meant to
be entirely decoupled from specific baremetal drivers.

Co-Authored-By: Robert Collins <rbtcollins@hp.com>

Change-Id: Icc7c40fe06b520aa220906122f5f6be8d79a3ac4
VMWare: bug fix for Vim exception handling

The exception handling in the Vim module was too broad and did not treat
the suds.MethodNotFound exception. This would result in a exception
that "__deepcopy__" is not supported in the VIM module.

In the case of an exception the VMwareAPISession class may be deleted.
The patch also ensures that the deletion is able to logout if necessary.

Closes-bug: #1228847

Change-Id: Ic85be9b3407c444db4b3a074108c01d9141c61be
Enable remote debugging for nova

This patch added 2 command line parameters which are used to
connect to an external debugger such as pycharm or eclipse.
This feature is used when you want to connect to a nova
service via a debugger running on a different host.
To use it you start the nova service with  the following
command line parameters

--remote_debug-host <where the debugger is running>
--remote_debug-port <port> it's listening on>.

DocImpact
Closes-bug: #1251021
Change-Id: I6ede9bf0813eafbeb91d858c297d4c160aafceba
Remove smoketests

We don't use them anymore. Also remove boto_v6 since only smoketests
have was using it.

Change-Id: Iee206fd4ba6cd502ded794f2de302062604d4186
Refactor time conversion helper function for objects in db api

Refactor the code to use common time conversion code for objects. Strip
timezone info out from datetimes, which is no need for database.

This is a part of: blueprint make-instance-action-use-objects

Change-Id: Id7f8b48da1499ded3418ceebe4680e066420e2a3
barematal: Cleanup the calls to assertEqual

Switch the ordering of assertEqual calls to match the correct
(expected, observed) ordering. This makes test failures easier to read.

Change-Id: Iecad55e33be4ab8f7ddff3b4cdbe08aa6c1b8660
Partial-Bug: #1260134
Remove docker's unsupported capabilities

The 'supports_recreate' flag needs the driver to support the evacuate
operation. The 'has_imagecache' flag needs the driver to implement the
'manage_image_cache' method. Docker does not currently support these
two functions, so the capabilities flags should not be set.

Partial-Bug: #1261636

Change-Id: I191d0d8cd35aece52462c0477f2d89897bb762c1
Image cache: move all of the variables to a common place

In addition to this it creates a base ImageCacheManager class.

This moves common image cache config from the libvirt driver into
a place that can be shared between drivers, nova.virt.imagecache
The libvirt specific image caching variables are untouched.

The other image cache fields that were in the nova.compute.manager
have also been moved to the common place.

These common configuration can be used by the
blueprint multiple-image-cache-handlers and the
blueprint vmware-image-cache-management.

DocImpact
    The following configuration variables are now in the DEFAULT
    section:
        image_cache_manager_interval
        image_cache_subdirectory_name
        remove_unused_base_images
        remove_unused_original_minimum_age_seconds

Change-Id: I92080f20c3d21ec348aff7d0a9795e773bcc2649
Merge "XenAPI: Speedup host_ref cannot change - get it once"
Clarify rebuild_instance's recreate parameter.

As far as I can tell this parameter actually implies that none of the
cleanup happens - it's used when evacuating an instance to a separate
hypervisor, so the new hypervisor has rebuild(recreate=True) passed
in. The docstring didn't make that clear, so this trivial patch
hopefully does that.

blueprint: baremetal-preserve-ephemeral
Change-Id: I42463cf981fe5dbd6b3b15f7a247d5a8ccfa7279
Add configurable 120s timeout ovs-vsctl calls

By default, or with a secs of 0, ovs−vsctl waits forever for a response
from the database. This option limits runtime to approximately 120 seconds.
If the timeout expires, ovs−vsctl will exit with a SIGALRM signal. (A
timeout would normally happen only if the database cannot be contacted,
or if the system is overloaded.)

The user can update the timeout by setting the ovs_vsctl_timeout
configuration variable.

DocImpact

Closes-bug: 1229018

Change-Id: Ica64ae98fb6162d2119021972127a82417d5ee7a
Merge "Remove smoketests"
Merge "Add configurable 120s timeout ovs-vsctl calls"
Merge "Image cache: move all of the variables to a common place"