Check Neutron port quota during validate_networks in API

Unless ports are passed into Nova it will create ports on the
requested networks as part of the network allocation in
the compute manager. However if the user exceeds their
port quota the instance will end up in an Error state, having
first been re-scheduled a number of times.

It would be much better if the quota failure was detected as
part of the network validation in the API server, so that an
error can be reported to the user and the creation failed.

A full fix would include reserving or creating the ports at
this stage, but there is no reservation mechanism in the
Neutron API, and port creation depends in some cases
on mac addresses only available on the compute manager.

Instead this change just validates the quota and adjusts the
max_count to be consistent with that quota, which doesn't
guarantee that the create will work, but does catch the
majority of cases.

Refs bug: 1172808

Change-Id: Iaaee059a6746fad68049712f94b2f8cfea6ab8dc
remove vmwareapi.vmops.get_console_output()

This code has been unused since commit
2ce3461595a490c9a02272ee4a1ab1eb55c27f60.

Change-Id: Ice72b0aba8af49b07528b30f46c012d69b712735
List NotImplementedError as a client exception for vnc

Not all drivers implement the get_vnc_console method.  In that case,
they will raise NotImplementedError.  List this exception as a client
exception so that the rpc layer does not log a traceback when this
happens.

Change-Id: Id30fd35b36cbd222b2d8787f00fa586d1ed4f11a
sync oslo service to fix SIGHUP handling

this is a sync of the oslo service code to fix a bug found in
the sighup handling. Previously it was not possible to use
devstack's unstack.sh because the service module would always
restart under a SIGHUP, not only when the process was running as
a service. SIGHUP on a non-detached process should act like
SIGTERM. This change fixes that.

Change-Id: Ib260d82519fde5c5cb24a2c0c8a7cccd0b96b84f
Related-Bug: #1263122
Merge "sync oslo service to fix SIGHUP handling"
Merge "remove vmwareapi.vmops.get_console_output()"
Merge "List NotImplementedError as a client exception for vnc"
Merge "Check Neutron port quota during validate_networks in API"
XenAPI: get_info was very expensive

Getting the whole VM record is expensive and slowed down Tempest tests
significantly.  Replaced with getting each required item.

When getting power state, only need that specific value, so reduce the
calls to get_info too.

Change-Id: I93bcef5b692a531aac41518dc29d5d4f46cded65
XenAPI: Tidy calls to get_all_ref_and_rec

get_all_records is more efficient than getting them individually in the
old get_all_ref_and_rec implementation.

Replaced some cases of get_all_records with a faster and more
specific API call.

Change-Id: I4725c3f1fdebdf1737941cb7db69b19afb22df9f
XenAPI: Speedup get_vhd_parent_uuid

In many cases the VDI record has already been retrieved.
Use it rather than expensively getting a new VDI record from XAPI.

Change-Id: I469e2a415c1f1924010f47d3927ca18c45a7b8f8
XenAPI: Use get_VALUE in preference to get_record()['VALUE']

Replaces calls to get_record where only a single item is retrieved
from the record with the specific get_VALUE call.

Change-Id: I52e20c4f4c9d6a6ad019694916ee057cc96b18a6
XenAPI: Monitor the GC when coalescing

Add a new host plugin to monitor whether the garbage collector is running.
This is then used by the wait_for_vhd_coalesce function to know whether
waiting any more is likely to produce the expected results.

Increased the number of times we loop waiting for coalesce as the loop
now has knowledge that something is still happening.

Change-Id: Idcc0738945bd1aee8cdb52c9cfabd798f85c31db
Closes-bug: 1258169

Conflicts:

	nova/virt/xenapi/vm_utils.py
XenAPI: Use direct IO for writing config drive

If you don't use direct IO when writing to the device, the unplug
immediately following this may not succeed as the device may still
be in use.

Also add a sync to ensure the device is not being used at the point
of unplug.

Closes-bug: 1259163

Change-Id: I68f09835648b36772a7839af9718123a4c60734f