Calculate default security group into quota usage

Create the related quota usage when creating the default
security group for user.

Change-Id: Ide1de160814a5937f71e07f3ab10ce2bba8886f1
Closes-bug: 1271381
Ensure that exception raised in neutron are handled correctly

We are now logging exceptions so we need to use save_and_reraise.

The patch set has also added log messages for exceptions that were
raised. This will be very helpful for debugging and troubleshooting.
This is done in the following cases:
 - unable to access a port when validating networks
 - unable to find a requested floating ip

Change-Id: I946126c2fb5aa229f917f8cb36ecb23aab528e14
Make libvirt use Flavor object instead of using VirtAPI

This makes the libvirt driver use the Flavor object instead of
getting that information through the VirtAPI/conductor call.

Related to blueprint compute-manager-objects

Change-Id: I484c98068a46c9780d82c886779eb58e783f9b6f
Group kvm image_meta tests for get_disk_bus

Refactoring test_get_disk_bus to eliminate duplicate code by
adding a loop to the image_meta tests.

Change-Id: Id8f342a0739db01635c2ca925788d7a258f8a478
Use six.moves.urllib.parse instead of urlparse

To keep Python 3.x compatibility, use six.moves.urllib.parse
to replace urlparse.

Change-Id: I23df3455d6293711572a48165964644cb18d175b
Add V3 api for pci support

All pci passthrough code is merged into nova, but the APIs are not supported.
This patch provides a resource: endpoint to show PCI information. This is a
part of PCI APIs, for more information see the blueprint.

Partially implement blueprint pci-api-support

Change-Id: I05c502cb7564b8b71f12d8f06249124b3d2b370f
libvirt: Uses available method get_host_state

An available method should be used to get the host_state.

Change-Id: I24004c0ed4c167d8f54a60a440ee250bb3236c1b
libvirt: improve unit test time

A number of tests would invoke code that would sleep. This
patch adds a mocked function for the sleep.

Change-Id: Iecdc8488393cb118aa77bc5c33bcfde33ceb4733
Closes-bug: #1279820
Convert remove_fixed_ip_to_instance to objects

Related to blueprint icehouse-objects.
Related to blueprint compute-manager-objects.

Change-Id: I443784d95cf0556dc4e6f9ac7f91466c766819b6
Convert post_live_migration_at_destination to objects

Related to blueprint icehouse-objects.
Related to blueprint compute-manager-objects.

Change-Id: Icaf0346c852d2f6a804675c975e54a7df642c1ab
XenAPI: Add the support for updating the status of the host.

  We added a function into the xenhost plugin to get information
about PCI devices. Roughly we run the lspci command on dom0. This
information will be used to get the list of pci devices  that are
passed on the pciback.hide dom0 command line.

  The hide option is used to hide the devices from the normal guest
drivers and assign them to the pciback kernel driver at boot on dom0
instead of their normal driver. We will parse the output of the lspci
command to find which device is using the pciback kernel driver and
thus to know if it has been passed to the pciback.hide option.

  This information will be used to perform the match with the
list of pci devices provided in pci_whitelist into /etc/nova/nova.conf.

Implements: blueprint pci-passthrough-xenapi

Change-Id: I465fc5d29f3c47ab0079adcfcc2d7d6501bd4b20
XenAPI: Add the mechanism to attach a pci device to a VM.

 Before booting a VM we check if there are PCI devices reserved
by this VM and if there are any we attach them.

 A PCI device can be asked by a VM by choosing a flavor that
request PCI devices. See the wiki about pci passthrough for more
details:

https://wiki.openstack.org/wiki/Pci_passthrough#Create_a_flavor

 We are attaching a PCI device by adding a key "pci" to the
"other-config" parameter of a VM.

Change-Id: I4aca96b11f931f70cf6442adac64e867ce1affb6
Implements: blueprint pci-passthrough-xenapi
Refactor get_iscsi_initiator to a common location

Refactor get_iscsi_initiator to a common location so that other
drivers can use it, without reaching into virt/libvirt. This patch
updates the current baremetal volume driver to the new shared location.

Change-Id: I30539f4bb917e486739f797bb6a9579e1cdffd22
MetricsWeigher: Added support of unavailable metrics

This patch added the support of unavailable metrics in the metric
weigher. Based on the configuration, if an unavailable metric is met,
the metric weigher might either throw an exception, or return an
configurable weight value.

This is part of the blueprint utilization-aware-scheduling.

DocImpact: Added support of unavailable metrics. 2 new config options
are added, 1) 'required' is a boolean option indicating whether all
the metrics set by 'weight_setting' are needed to be available when
calculating the weight. 2) 'weight_of_unavailable' is a float option
specifying the value to be returned if the metrics is unavailable. See
etc/nova/nova.conf.sample for details.

Change-Id: Icb951605aae28487b00a12d172a3ac4dafda2cdb
libvirt: remove retval from libvirt _set_host_enabled()

This is an internal method since commit e1fdd8f and the return value
is currently unused, so remove it.

Change-Id: I63f4ea7fdb5218c985cad73e7c795b4329f7b4c9
VMware: raise more specific exceptions

In certain cases the exception handling for backend errors
would be handled too broadly. This patch elevates the backend exception
to the application so that it can treat specific errors.

In the application we can now handle the following specific exceptions:
 - FileAlreadyExists - for example when moving a directory to a directory
                       that already exists
 - InvalidProperty - for example when using neutron and opaque networks
                     are not supported
 - AlreadyExists - for example a port group already exists
 - NotAuthenticated - for example the operation is denied as because a
                      session is not established

Related-Bug: #1266579
Related-Bug: #1266580
Closes-bug: #1276539

Co-authored-by: Matthew Booth <mbooth@redhat.com>
Change-Id: I0873f570d97345208c8d9b5bcbb5a980a7a58f9d
VMware: prevent race for vmdk deletion

Prevent a race condition for the deletion of a file when there
is more than one process trying to erase the file.

In the application we can now handle the following specific exceptions:
 - CannotDeleteFile - the file cannot be deleted
 - FileFault - a file access exception
 - FileLocked - an attempt is made to lock a file that is
                already in use
 - FileNotFound - the specific file does not exist

Change-Id: I58e07b9e630c063f894cbbb138612a39e912acd5
Closes-bug: #1266580
Treat port attachment failures correctly

Raise a more appropriate exception following the failure of a
port attachment.

Closes-bug: #1245683

Change-Id: I77e9568c8675b0dd6a0c8c741e71e936ed3998a1
Remove XML support from schemas v3

The XML support was removed from the V3 API plugins,
this part of the code was not used again.

Partially implements blueprint remove-v3-xml-api

Change-Id: I7bc3ed2386d8d678eb13ac4592e4cce2ba8a1602
Imported Translations from Transifex

Change-Id: I7473dd1635f326fb9a50a95e42c57341c36a5609