Fix LibvirtGenericVIFDriver.get_config() for quota

When Nova is used with Neutron and Open vSwitch, network quotas on the
bandwidth are ignored by libvirt: the "<bandwidth>" tag is not generated in
libvirt.xml of the virtual machine. This change fixes this issue but also for
the network types: "bridge", "network" and "direct".

Commands to create the flavor with download limited to 1 MB/sec and create a
virtual machine with this flavor:
---
nova flavor-create --ephemeral=1 victor_test_vif 50 256 1 1
nova flavor-key victor_test_vif set quota:vif_inbound_average=1000
nova boot --flavor=victor_test_vif --image=cirros-0.3.1-x86_64-uec victor_test
---

Commands to upload a file to this virtual machine with SCP to check that the
bandwidth is limited to 1 MB/sec (replace 10.0.0.3 with the IP of the virtual
machine):
---
ip netns
sudo ip netns exec qrouter-128db593-a0db-40c3-84c7-e6383d40c75f bash

dd if=/dev/urandom of=random10MB bs=1024 count=10240
scp random10M cirros@10.0.3:
---

Change-Id: I0f0111fc79fe90900e38df022034b208f1129088
Closes-Bug: #1254664
DB migration 209: Clean up child rows as well

Fixes bug 1252891

In order to add missing foreign key constraints, rows that reference a
nonexistant key need to be purged. Unfortunately, DB migration 209 does
not clean up any rows in child tables that themselves have a foreign key
constraint on the tables with the missing foreign keys. This can result
in an integrity violation causing the migration to fail.

When purging the rows, make sure to purge rows in any child tables first.

Change-Id: I25bd899383c72cb0670819b030759630ac89acaa
Log exception in _heal_instance_info_cache

Previously, if errors occurred in _heal_instance_info_cache() they would
go unnoticed as their log level was debug and the stace trace was not
displayed. This patch changes that so the exception is logged so periodic
failures here can be more easily detected and fixed.

Change-Id: I15bb3e7e29462e930f6159be9f4bbf4980b1270c
Closes-bug: #1257940
Merge "Log exception in _heal_instance_info_cache"
Merge "DB migration 209: Clean up child rows as well"
Merge "Fix LibvirtGenericVIFDriver.get_config() for quota"
XenAPI: Speedup host_ref cannot change - get it once

The host we run on is fixed - getting this each time from XAPI
is expensive

Change-Id: Ifee9a778f3ed855ef52b001de3fc2ebbd0b972cf
XenAPI: get_info was very expensive

Getting the whole VM record is expensive and slowed down Tempest tests
significantly.  Replaced with getting each required item.

When getting power state, only need that specific value, so reduce the
calls to get_info too.

Change-Id: I93bcef5b692a531aac41518dc29d5d4f46cded65
XenAPI: Tidy calls to get_all_ref_and_rec

get_all_records is more efficient than getting them individually in the
old get_all_ref_and_rec implementation.

Replaced some cases of get_all_records with a faster and more
specific API call.

Change-Id: I4725c3f1fdebdf1737941cb7db69b19afb22df9f
XenAPI: Speedup get_vhd_parent_uuid

In many cases the VDI record has already been retrieved.
Use it rather than expensively getting a new VDI record from XAPI.

Change-Id: I469e2a415c1f1924010f47d3927ca18c45a7b8f8
XenAPI: Use get_VALUE in preference to get_record()['VALUE']

Replaces calls to get_record where only a single item is retrieved
from the record with the specific get_VALUE call.

Change-Id: I52e20c4f4c9d6a6ad019694916ee057cc96b18a6
XenAPI: Monitor the GC when coalescing

Add a new host plugin to monitor whether the garbage collector is running.
This is then used by the wait_for_vhd_coalesce function to know whether
waiting any more is likely to produce the expected results.

Increased the number of times we loop waiting for coalesce as the loop
now has knowledge that something is still happening.

Change-Id: Idcc0738945bd1aee8cdb52c9cfabd798f85c31db
Closes-bug: 1258169

Conflicts:

	nova/virt/xenapi/vm_utils.py
XenAPI: Use direct IO for writing config drive

If you don't use direct IO when writing to the device, the unplug
immediately following this may not succeed as the device may still
be in use.

Also add a sync to ensure the device is not being used at the point
of unplug.

Closes-bug: 1259163

Change-Id: I68f09835648b36772a7839af9718123a4c60734f