Add UNSHELVING and RESCUING into IoOPSFilter consideration state

IoOpsFilter will take instance in following state
task_states.RESIZE_MIGRATING, task_states.REBUILDING, task_states.RESIZE_PREP,
task_states.IMAGE_SNAPSHOT, task_states.IMAGE_BACKUP
into consideration ,if the instance on this host exceed the maximum
number of instance allowed to do I/O the scheduler will fail on the host

UNSHELVING and RESCUING are also I/O related so IoOps need to include them

Change-Id: I141470254c824e7e57e6734ae0e016c432b85c00
Closes-Bug: #1295558
No longer any need to pass admin context to aggregate DB API methods

No longer need to pass an elevated context because the check of
admin has been removed in following patch:
https://review.openstack.org/67026

This patch removes the admin context elevated
method in nova/scheduler sub-folder since it's not required any more.

Change-Id: Ib768c97c9e84a9db44d866fa5fbfc4665ad7c73e
Partial-Bug: #1279216
Nova utils: add in missing translation

The method validating the hypervisor version would throw
an exception that did not have a translation.

Change-Id: I6284d313a553757b90bd6b2c4946c6aa84ea17cd
Scheduler: remove test scheduling methods that are not used

The test code no longer uses the following methods:
 - fake_get_group_filtered_hosts
 - fake_get_group_filtered_affinity_hosts

This patch removes those methods.

Change-Id: I0a005fb1301ce78cc2ec365429015c08d58f446e
Fix AvailabilityZone check for hosts in multiple aggregates

Check for multiple AZ was distinct when adding an host to an aggregate
and when updating an aggregate (or its metadata), which was leading to
possible corner-cases where logics were different.

This patch proposes a single way for checking for both API methods and
also adds an extra control on the possibility to have an aggregate with
its metadata AZ set to the configuration default value (eg. 'nova') and
where it was possible to subsequently have one host in two disctinct AZs
Fixed that scenerio as it is a gap, Nova as of now preventing to have
one host in two distinct AZs.

Change-Id: Ida6aa507f34f36a436adaafe1f7609963c59a9bc
Partial-Bug: #1277230
Add specific doc build option to tox

Add doc venv to align with other OpenStack projects (such as keystone,
heat and nova-specs) who are using 'docs' for there doc builds.

Change-Id: I37771db339d864b2b938d1c02d15d38b07093493
Ignore errors when deleting non-existing vifs

If an instance fails during its network creation (for example
if the network-vif-plugged event doesn't arrive in time) a
subsequent delete will also fail when it tries to delete the
vif, leaving the instance in a Error(deleting) state.

This can be avoided by including the "--if-exists" option to
the ovs-vsctl command so that the command doesn't fail if the
vif doesn't exist.

Change-Id: Ic613c9d6e0619f51fcd931e62f6d6bce404ebe81
Closes-bug: #1308544
Set the volume access mode during volume attach

cinderclient supports an optional keyword argument named 'mode'
for volumes.attach(), which allows you to set the access mode of
a volume at attach time. The keyword argument is not passed in
by nova when it attaches a read-only volume, making the volume
access mode default to 'rw' instead of the desired 'ro'. This
patch finds the access mode of the volume and passes it to
cinderclient, so the correct access mode is used at attach
time.

Change-Id: Ib62f454292e60114f96debbad96c4a356dc32a21
Closes-Bug: #1307088
Fix instance cross AZ check when attaching volumes

For CONF.cinder_cross_az_attach=False, we currently only check the desired AZ
during scheduling but don't actually check the final AZ of the node.

When attaching volumes to running instances we need to check the actual
AZ on the instance and not the desired AZ at boot time. The desired AZ
attribute is optional at boot and may not be set or the desired AZ
could be different from the final AZ.
When we are booting with a volume however we don't know the final AZ so we
check on the desired AZ.

Closes-Bug: #1255347
Change-Id: Ied912c171100a450754d315027bf5c407ac067bb
libvirt: Refresh volume connection_info after volume snapshot

The following patch is related to the guest assisted snapshot
functionality.  When you take a snapshot of a volume
(e.g. GlusterFS) attached to a running instance, a new snapshot
file is created, i.e. volume-<uuid>.<snapshot-uuid>.  The
instance uses this file as the active volume.  If you shutdown
and restart the instance, nova will reattach the base volume
(volume-<uuid>) to the instance instead of the snapshot volume
(volume-<uuid>.<snapshot-uuid>).  The expected behavior is to
have the snapshot volume reattach to the instance.  This is
caused by stale data being returned from the database when
_get_instance_volume_block_device_info is called during
_power_on.  To fix this bug, this patch calls
refresh_connection_info to update the database in both
volume_snapshot_create and _volume_snapshot_delete methods of the
libvirt driver.

Change-Id: I0f340a3f879580e7981d97863bc299e33d71aa84
Closes-Bug: #1304695
Fix a minor comments error

Seems a copy-paste error. This test case is test bigger than oversize
limitation.

Change-Id: Ie0ce557032b62a0d3a955a5b7ce8bbbe41fef7cd
Network: replace neutron check with decorator

Commit a581823ecee286c5c6329c9c373f0fea0b6b729f added in a
validator that would check if neutron was enabled.

It is much cleaner to do this as a decorator. The decorator
here needs to preserve the function signature.

Change-Id: I355e4bcee5e93a7f1459672e10ad7793d2bf8a49
Rename quota_injected_file_path_bytes

Rename quota_injected_file_path_bytes to
quota_injected_file_path_length, and change help message to
'Length of injected file path'.

DocImpact

Change-Id: Id6b9c03b37664601937e4bc9d4a8d8026c4d3b63
Closes-bug: 1258175
Remove comments since it's pointless

Remove some comments for rescue/unrescue since it's pointless.

Change-Id: I1fa76355599fcd7a6999a16854361f7edd752a93
Raise more information on V2 API volumes when resource not found

NotFound exception is caught in volumes.py on v2 api
and raise HTTPNotFound exception but exception information are
not well specified. This patch adds more information
to the raised exception.

Change-Id: I2bd48503460662fcc648b31ade7156059b370f2b
Closes-Bug: #1275755
libvirt: Handle unsupported host capabilities

Neither libvirt-xen nor libvirt-lxc support
VIR_CONNECT_BASELINE_CPU_EXPAND_FEATURES so check for the proper error
code that is returned by libvirt.

The existing code was performing this check improperly by trying to
catch an enum value; instead, we need to catch the `libvirtError`
exception and then check its return code.

Co-Authored-By: Rick Harris <rconradharris@gmail.com>
Co-Authored-By: Andrew Melton <andrew.melton@rackspace.com>
Closes-Bug: 1297962
Change-Id: Ie8bf5c9d1f2a27c387f8b2f54a9bb729fa2f0985
Convert address to str in fixed_ip_obj.associate

A caller of associate may pass in a netaddr.IPAddress argument which
will currently fail when it gets to sqlalchemy.  Following the
convention of other methods the netaddr.IPAddress will be cast to a
string.

Change-Id: I67c1876b447217b37104660ba87ed967f477b863
Closes-bug: 1317668
vmware:Don't shadow builtin function type

We should avoid to assign builtin function with other value.

Closes-Bug: #1319615
Change-Id: I9c52524a16b6a74ad76466473338bf86414f0c50
VMware: spawn refactor - phase 1 - fetch_image

One wonders why this inner method existed at all. It did nothing but
add log messages. The inner method has been stripped and the log
messages abandoned because it turns out that they added no meaningful
information at all. It is not clear what intention this method had
if any.

partial blueprint vmware-spawn-refactor

Co-authored-by: Tracy Jones <tjones@vmware.com>
Change-Id: Ib346b5739d11a7f1fc0a145673bb2701b4b59455
Fixes Hyper-V iSCSI target login method

Hyper-V is trying to log in to portals/targets every time a volume
is attached, even if the portal/target was already connected. This
is not an issue with most volume backends, but it causes the attach
to fail when using Storwize as a Cinder backend.

This patch fixes the issue by checking if the target or portal is
already connected, and simply perform an update in this case,
instead of attempting to create a new connection.

Also, a retry loop is introduced for the case when the iSCSI target
login fails. As in this loop it is checked whether or not the
operation completed successfully, this method does not rely anymore
only on a sleep interval to ensure that the connection to the target
has been made. Even so, on V2 volume utilities, the sleep interval
is still needed to avoid having a new connection attempt before the
last one finished.

The method which gets the according mounted disk must raise an
exception when the device number is -1, which signifies that the
disk has not been mounted properly.

Co-Authored-By: Jay Bryant <jsbryant@us.ibm.com>

Closes-bug: #1221525

Change-Id: I92d2a586704fa3e857f46432bb571c81e86d183b