Calculate default security group into quota usage

Create the related quota usage when creating the default
security group for user.

Change-Id: Ide1de160814a5937f71e07f3ab10ce2bba8886f1
Closes-bug: 1271381
Ensure that exception raised in neutron are handled correctly

We are now logging exceptions so we need to use save_and_reraise.

The patch set has also added log messages for exceptions that were
raised. This will be very helpful for debugging and troubleshooting.
This is done in the following cases:
 - unable to access a port when validating networks
 - unable to find a requested floating ip

Change-Id: I946126c2fb5aa229f917f8cb36ecb23aab528e14
Use six.moves.urllib.parse instead of urlparse

To keep Python 3.x compatibility, use six.moves.urllib.parse
to replace urlparse.

Change-Id: I23df3455d6293711572a48165964644cb18d175b
Add V3 api for pci support

All pci passthrough code is merged into nova, but the APIs are not supported.
This patch provides a resource: endpoint to show PCI information. This is a
part of PCI APIs, for more information see the blueprint.

Partially implement blueprint pci-api-support

Change-Id: I05c502cb7564b8b71f12d8f06249124b3d2b370f
libvirt: Uses available method get_host_state

An available method should be used to get the host_state.

Change-Id: I24004c0ed4c167d8f54a60a440ee250bb3236c1b
libvirt: improve unit test time

A number of tests would invoke code that would sleep. This
patch adds a mocked function for the sleep.

Change-Id: Iecdc8488393cb118aa77bc5c33bcfde33ceb4733
Closes-bug: #1279820
Refactor get_iscsi_initiator to a common location

Refactor get_iscsi_initiator to a common location so that other
drivers can use it, without reaching into virt/libvirt. This patch
updates the current baremetal volume driver to the new shared location.

Change-Id: I30539f4bb917e486739f797bb6a9579e1cdffd22
VMware: raise more specific exceptions

In certain cases the exception handling for backend errors
would be handled too broadly. This patch elevates the backend exception
to the application so that it can treat specific errors.

In the application we can now handle the following specific exceptions:
 - FileAlreadyExists - for example when moving a directory to a directory
                       that already exists
 - InvalidProperty - for example when using neutron and opaque networks
                     are not supported
 - AlreadyExists - for example a port group already exists
 - NotAuthenticated - for example the operation is denied as because a
                      session is not established

Related-Bug: #1266579
Related-Bug: #1266580
Closes-bug: #1276539

Co-authored-by: Matthew Booth <mbooth@redhat.com>
Change-Id: I0873f570d97345208c8d9b5bcbb5a980a7a58f9d
VMware: prevent race for vmdk deletion

Prevent a race condition for the deletion of a file when there
is more than one process trying to erase the file.

In the application we can now handle the following specific exceptions:
 - CannotDeleteFile - the file cannot be deleted
 - FileFault - a file access exception
 - FileLocked - an attempt is made to lock a file that is
                already in use
 - FileNotFound - the specific file does not exist

Change-Id: I58e07b9e630c063f894cbbb138612a39e912acd5
Closes-bug: #1266580
Remove XML support from schemas v3

The XML support was removed from the V3 API plugins,
this part of the code was not used again.

Partially implements blueprint remove-v3-xml-api

Change-Id: I7bc3ed2386d8d678eb13ac4592e4cce2ba8a1602
VMware: factor out management of controller keys and unit numbers

This patch encapsulates the allocation of controller key and unit
number for a new virtual disk/CDROM into utility function.
This way high-level methods like spawn, rescue, etc. don't have
to deal with implementation details like the above and the
algorithm for allocating controller_key and unit_number is
clearly defined and can be easily reused.

Change-Id: Ia6d5db5d5c5fa25f680eae82892dcb9b2be864fa
Closes-Bug: #1259981
libvirt: configuration element for virtual controller

While Libvirt can automatically configure a controller for most
of the devices, some need to have an explicit controller element
to be provided.

Partially implements blueprint libvirt-virtio-scsi-driver

Change-Id: Id0b6c597362cfb66af0d8c5d43f777e4fa3267fe
libvirt: add virtio-scsi disk interface support

A new virtio-scsi cotroller will be presented to the guests,
when hw_disk_bus_model will be set to virtio-scsi in Glance
image metadata:

glance image-update \
       --property hw_scsi_model=virtio-scsi \
       [image]

DocImpact
Partially implements blueprint libvirt-virtio-scsi-driver
Change-Id: I2723494130055687a9dfc58b1a27e586116fdff3
Use disk_bus when guessing the device name for vol

Even though the disk bus could be set through hw_disk_bus property,
the libvirt driver didn't guess the proper bus and device name for the
root volume and expected it to be defined in the BDM.

With this patch, we consider the predefined disk_bus when going to guess
the volume device name and bus instead of just relying on the one
defined in the BDM.

Change-Id: Ic6434ee2b29dc20126ec3b470d7340a0f8bf62e9
Closes-Bug: 1281989
Correct the assert() order in test_libvirt_blockinfo

I'm not a big believer in changing the order of assert arguments
for the fun of it. On the other hand, this file was inconsistent
(and mostly backwards). I just wasted a while trying to figure out
what was wrong with my code to discover it was the arguments being
passed in the wrong order. So I am fixing that.

Change-Id: If5cb6c12a26fedbcd1f15c2a3f310e7c8347b586
Merge "Calculate default security group into quota usage"
Merge "Correct the assert() order in test_libvirt_blockinfo"
Refactor driver BDM attach() to cover all uses

Add separate flags to allow the execution to skip the
volume_api.check_attach step, and to be able to call the
driver.attach_volume if needed. This will make the method modular enough
to be used in all cases where we actually "attach" volumes to instances.

Check attach should always be done in the API, followed immediately by a
reservation, to minimize the chance of a race. However currently we
do the check_attach step on the compute side during boot, so it needs to
stay. Other operations like attach_volume will have it reserved causing
this check to fail on the compute side, so we lay down the groundwork to
be able to skip it when calling attach().

Likewise, when booting, we don't need to call the virt driver
attach_volume method, as all the devices will be taken care of by the
boot code path.

Part of blueprint: use-new-bdm-format-in-attach

Change-Id: Iff3890e39a8c75b1693df0918084ecf4492c4e91
Merge "Use six.moves.urllib.parse instead of urlparse"
Merge "libvirt: Uses available method get_host_state"