Libvirt:Instance resize confirm issue against NFS

when I test resize-confirm on NFS (KVM) using at least two
compute nodes, OSError will occur when try to cleanup the
backup instance directory and will make instance status
"Error".

One way to cleanup resize is using delay_on_retry and
maximum attempts setting. Replace rmtree() because
execute() have more parameters choices.

Closes-Bug: #1248019

Change-Id: Ifb9a500bbba805af0317307c2e7d6903dcd02ad1
Sync middleware audit, base, and notifier from oslo

Pull in middleware modules required to support CADF auditing
reports from Nova.  audit.py requires notify.py, and
notify.py requires base.py.  This is why those 3 files are
being synced, and not others in the
openstack/common/middleware directory from oslo.

The following 3 change IDs included in this sync provide
the main parts of the CADF functions being synced:

Ie67cf4a7 Add audit middleware
Iead2431a Middleware: add notifier
I05ff58   Move wsgi module to deprecated package

The following change IDs additionally included in this sync
provide minor updates to the files that were made after the
main parts of the CADF functions were merged in oslo:

I032616   Changed header from LLC to Foundation based on
          trademark policies
I840c43bf add support to ignore certain request
I2099e1ee Enable hacking H404 test
I04b0cf   middleware: code cleanup

blueprint support-standard-audit-formats

Change-Id: Id15f95a0c354270ba2d5aecb12c4f910905e8c20
baremetal: volume driver refactoring and tests

Refactor the LibVirtVolumeDriver to test each step of
attaching and detaching a volume. And add tests.

Closes-Bug: 1240470
Change-Id: I51906bec6ba5d6af71997df277bb368cff81b3b3
Raise better exception if duplicate security groups

Previously a 500 error was raised if one was trying to launch an instance
with a security group that shared the same name as another security group.
With this patch a more clear error message is raised.

Closes-bug: #1161472

Change-Id: I285bedbb2e0b3f4cd24cfa3a9b17131ad0200afa
Merge "Raise better exception if duplicate security groups"
Merge "Libvirt:Instance resize confirm issue against NFS"
Merge "Sync middleware audit, base, and notifier from oslo"
Merge "baremetal: volume driver refactoring and tests"
XenAPI: Speedup host_ref cannot change - get it once

The host we run on is fixed - getting this each time from XAPI
is expensive

Change-Id: Ifee9a778f3ed855ef52b001de3fc2ebbd0b972cf
XenAPI: get_info was very expensive

Getting the whole VM record is expensive and slowed down Tempest tests
significantly.  Replaced with getting each required item.

When getting power state, only need that specific value, so reduce the
calls to get_info too.

Change-Id: I93bcef5b692a531aac41518dc29d5d4f46cded65
XenAPI: Tidy calls to get_all_ref_and_rec

get_all_records is more efficient than getting them individually in the
old get_all_ref_and_rec implementation.

Replaced some cases of get_all_records with a faster and more
specific API call.

Change-Id: I4725c3f1fdebdf1737941cb7db69b19afb22df9f
XenAPI: Speedup get_vhd_parent_uuid

In many cases the VDI record has already been retrieved.
Use it rather than expensively getting a new VDI record from XAPI.

Change-Id: I469e2a415c1f1924010f47d3927ca18c45a7b8f8
XenAPI: Use get_VALUE in preference to get_record()['VALUE']

Replaces calls to get_record where only a single item is retrieved
from the record with the specific get_VALUE call.

Change-Id: I52e20c4f4c9d6a6ad019694916ee057cc96b18a6
XenAPI: Synchronize on all VBD plug/unplug per VM

Dynamically adding or removing a device needs to be synchronized
to work around a bug in XenServer 6.1 and 6.2 where simultaneous
events can be partially ignored, leading to a failure to plug or
unplug.

Change I041365ed863a1a1c55e88025dff42d01c35494a2 attempted to
work around the issue by retrying - however that does not catch
all failures.  This change should eliminate some of the known
INTERNAL_ERROR failures, but not all, so I041 should not be
reverted.

Change Ie7a64c481a9e3874df9d5924fbb050ac7ab117db does the same
as this, but only for VBD.plug.

Change-Id: If78d4e5f1dd84f1d4a588f25b8f887a7b125bbea
Closes-bug: 1217972
XenAPI: Monitor the GC when coalescing

Add a new host plugin to monitor whether the garbage collector is running.
This is then used by the wait_for_vhd_coalesce function to know whether
waiting any more is likely to produce the expected results.

Increased the number of times we loop waiting for coalesce as the loop
now has knowledge that something is still happening.

Change-Id: Idcc0738945bd1aee8cdb52c9cfabd798f85c31db
Closes-bug: 1258169

Conflicts:

	nova/virt/xenapi/vm_utils.py
XenAPI: Use direct IO for writing config drive

If you don't use direct IO when writing to the device, the unplug
immediately following this may not succeed as the device may still
be in use.

Also add a sync to ensure the device is not being used at the point
of unplug

Change-Id: I68f09835648b36772a7839af9718123a4c60734f