Move volume_clear option to where it's used

The `volume_clear` and `volume_clear_size` option are used in `lvm.py` so we
should move the definition there.

Change-Id: Ic619c2db0bb1b5ccc7dcbec1a500c3dcb1110b26
Don't truncate osapi_glance_link or osapi_compute_link prefixes

When osapi_glance_link_prefix or osapi_compute_link_prefix is
defined and used to update the links return in API responses, do
not drop the path component of the overriding link prefix.

Change-Id: I290e3f85b94e28d25a54621c1699081cbbf48456
Closes-Bug: 1304099
Rollback quota when confirm resize concurrently completed

quota is reserved when confirm_resize is executed, if a concurrent
operation updated the status of migration, currently compute layer
will directly return back without rollback the quota and it can
lead to quota inconsistent issue layer.

Change-Id: I2217e9a4523d2783eccd8ef31c1fd65a604e2cc3
Closes-Bug: #1301690
Correct a maybe-typo in pci_manager

the variable 'dev' in set_hvdevs are only used for list comprehension,
it does not make sense to use 'dev' again for updating self.stale.

Change-Id: I03f31bfcfc56a43e43eb88845e1d562f19a3a09e
Adds verbosity to child cell update log messages

Child Cells frequently update their parents with their capabilities and
capacities. The debug log message contains the capabilities or capacities
that are being updated. This patch adds the names of the parent cells to
which the information is being sent to the debug log message.

Change-Id: Ic34191da191369e64a016c8300a8da294aaa9e07
Closes-Bug: 1351301
Add decorator expected_errors for ips v3 extension

Add decorator expected_errors for ips v3 extesion, that's required
for all the v3 extension.

Change-Id: Iad071fd77bd3571087badc1d836275bd64d8dfc8
Adjust audit logs to avoid negative disk info

When requesting more disk than is available, the audit
logs will report negative values. This may confuse the
user or make him/her think the tracking is innaccurate.

This patch seeks to remove any confusion. It is in the
same vein as https://review.openstack.org/#/c/93261/
which addresses the same issue in regards to cpu and
memory audit logs.

Change-Id: I184985573b20144717ac40f1f3c26709030d5a03
Closes-bug: #1350542
Lock attach_volume

There are some issues with instance and volume cleanup when the volume
is not in a fully attached state so it will be safer to not attempt a
terminate_instance while there are attachments in progress.

Change-Id: I4347794e51004a881bf4ef5ee30f65ac28773e51
Closes-Bug: #1355348
baremetal: Remove depenency on libvirt's fetch_image method

It's messy for virt-drivers to depend on each other, so this patch alters
baremetal to use the cross-driver fetch function, instead of the libvirt
version.

Change-Id: I910c621452fa7eb9add417a8738e5757aeb1a48c
libvirt: Fix kwargs for _create_image

Rescue uses the `suffix` and `disk_images` kwargs of `_create_image`, but
passes them in as positional args. We should use explicit kwargs here to be
clear that rescue depends on both of those parameters being present.

In addition, disk mapping was being passed in as kwarg, but is actually a
positional arg, so we should pass it in as a positional arg to be clear that
it's required.

Change-Id: Iaa990729b4bb3569f4c71ceaf1adabed58f73026
Add test for get_instance_disk_info to test_virt_drivers

Commit 2bed16c89356554a193a111d268a9587709ed2f7 added
get_instance_disk_info to the base ComputeDriver class
but didn't add a test for it in test_virt_drivers, so
it's added here.

This also fixes the wrong method signature in the fake
virt driver.

Note that this wouldn't have found the problem for the
hyperv or vmware drivers due to bug 1209421.

Change-Id: I5765720d8c3e0a7962b493fa49359514f3136544
Related-Bug: #1358719
Fix sample files miss for os-aggregates

There are some sample files for os-aggregates V3 API assigning
'deleted' element with '0', which should be assigned to 'false'.
This patch fixes it.

Change-Id: I6d7814ce98bc0c701d4eabefcbe64100345cb2d3
Closes-Bug: #1294556
Add _security_group_ensure_default() DBAPI method

According to the convention we use for DB API methods, every public
method should have a private counterpart. The difference between
those two is that the latter accepts an optional 'session' argument,
which makes it possible for the method to participate in the existing
database transaction.

security_group_ensure_default() was called inside instance_create()
and thus implicitly created a new concurrent db transaction, which
is now fixed by making a call to a private counterpart and passing
the session instance.

Closes-Bug: #1362233

Change-Id: Ifa1b6430328fde1ab0984f726bd8392bbd3f7ee5
Remove libvirt legacy LVM code

The Change-Id Ib36b962971fd1f66ea9a0818e91fec59e118e686 fixed an
issue which could have caused volumes no longer being able to be
cleaned up due to it's reliance on the instance_name_template
configuration option.

There was code in place to handle both disk formats which are
the legacy format (volume name prefixed by instance name) and
the new one (volume name prefixed by instance uuid).

This commit removes the legacy format which was due to be
removed from the code.

UpgradeImpact - this removes support for really old lvm names that
could have been created in Havana clouds. That means there is a
possible failure to cleanup of lvm volumes that existed uniterupted
since Havana.

Change-Id: I28497a650fbd012643080951f7d86ab8780fd59c
Fix NoopQuotasDriver.get_settable_quotas()

get_settable_quotas() always failed with KeyError. While this driver
is not really useful, we still want it to be working.

Closes-Bug: #1363014

Change-Id: Ia6986453338cee05f28ff081c6bb1279072c3d80
Allow hard reboots when still attempting a soft reboot

Commit cc0be157d005c5588fe5db779fc30fefbf22b44d introduced a
regression in the reboot code. Previously, a hard reboot could be
performed when an instance was in soft reboot.

This fixes the regression by allowing a hard reboot when the instance
is in one of the two new task states.

Closes-bug: 1362799
Change-Id: Id8aaf852b0cccb75a79794a8bf9eb94e21550b81
the headroom infomation is incomplete

If multiple resources beyond the quotas in 'limit_check' from
nova/quota.py. The headroom information about the resources
beyond the quotas in OverQuota Exception is incomplete.

Change-Id: I790c22e3618a3558dfad64bf46e37e4e49f6385e
Closes-Bug: #1334965
Return hydrated net info from novanet add/remove_fixed_ip calls

Compute manager expects hydrated network info from network_api

Change-Id: I36e48742144555004cbd1f71429a943c7e2952b0
Closes-bug: #1350846
Fix swap_volumes

The attach and detach calls to cinder are better handled by cinder this
patch removes them. There is another patch against this bug to modify
cinder.

The attach call is being made against the new volume after the
migrate_volume_completion request. This means it may be done after the
new volume ID in the db has been deleted.

The detach call is being run at the end on the old ID and so may end up
detaching the migrated volume from cinder's perspective.

Change-Id: I3f2d98e9e473905a3de2f02e00c92ae3065e6ae7
Partial-Bug: 1316079
Add a policy for handling retrospective vetos

For some time, we've been informally implementing a "revert early"
policy around handling -2 worthy complaints after a patch has been
merged. These reverts can be contentious, so it's appropriate we make
it very clear what the policy and procedure here is in order to lessen
the contentiousness of these reverts.

Co-authored-by: Michael Still <mikal@stillhq.com>

Change-Id: I90e4153a80c859a928630809aef31e7313f08906