Add test for get_instance_disk_info to test_virt_drivers

Commit 2bed16c89356554a193a111d268a9587709ed2f7 added
get_instance_disk_info to the base ComputeDriver class
but didn't add a test for it in test_virt_drivers, so
it's added here.

This also fixes the wrong method signature in the fake
virt driver.

Note that this wouldn't have found the problem for the
hyperv or vmware drivers due to bug 1209421.

Change-Id: I5765720d8c3e0a7962b493fa49359514f3136544
Related-Bug: #1358719
Fix sample files miss for os-aggregates

There are some sample files for os-aggregates V3 API assigning
'deleted' element with '0', which should be assigned to 'false'.
This patch fixes it.

Change-Id: I6d7814ce98bc0c701d4eabefcbe64100345cb2d3
Closes-Bug: #1294556
Add _security_group_ensure_default() DBAPI method

According to the convention we use for DB API methods, every public
method should have a private counterpart. The difference between
those two is that the latter accepts an optional 'session' argument,
which makes it possible for the method to participate in the existing
database transaction.

security_group_ensure_default() was called inside instance_create()
and thus implicitly created a new concurrent db transaction, which
is now fixed by making a call to a private counterpart and passing
the session instance.

Closes-Bug: #1362233

Change-Id: Ifa1b6430328fde1ab0984f726bd8392bbd3f7ee5
Remove libvirt legacy LVM code

The Change-Id Ib36b962971fd1f66ea9a0818e91fec59e118e686 fixed an
issue which could have caused volumes no longer being able to be
cleaned up due to it's reliance on the instance_name_template
configuration option.

There was code in place to handle both disk formats which are
the legacy format (volume name prefixed by instance name) and
the new one (volume name prefixed by instance uuid).

This commit removes the legacy format which was due to be
removed from the code.

UpgradeImpact - this removes support for really old lvm names that
could have been created in Havana clouds. That means there is a
possible failure to cleanup of lvm volumes that existed uniterupted
since Havana.

Change-Id: I28497a650fbd012643080951f7d86ab8780fd59c
Fix NoopQuotasDriver.get_settable_quotas()

get_settable_quotas() always failed with KeyError. While this driver
is not really useful, we still want it to be working.

Closes-Bug: #1363014

Change-Id: Ia6986453338cee05f28ff081c6bb1279072c3d80
Allow hard reboots when still attempting a soft reboot

Commit cc0be157d005c5588fe5db779fc30fefbf22b44d introduced a
regression in the reboot code. Previously, a hard reboot could be
performed when an instance was in soft reboot.

This fixes the regression by allowing a hard reboot when the instance
is in one of the two new task states.

Closes-bug: 1362799
Change-Id: Id8aaf852b0cccb75a79794a8bf9eb94e21550b81
Allow forceDelete to delete running instances

Currently attempting to force delete a running instance
fails while a normal delete can succeed. This changes the
behaviour of force_delete so any instance can be deleted
and not just ones which have already been soft deleted.

Also catch InstanceIsLocked exceptions at the API layer
and now also don't need to catch InstanceInvalidState
exceptions since they are no longer raised

DocImpact: forceDelete will now delete any instance and not
  just ones which have already been soft deleted

Change-Id: I06a476701ccb47d8f7fd6e43c394ccb3fac2b3aa
Closes-Bug: 1351001
the headroom infomation is incomplete

If multiple resources beyond the quotas in 'limit_check' from
nova/quota.py. The headroom information about the resources
beyond the quotas in OverQuota Exception is incomplete.

Change-Id: I790c22e3618a3558dfad64bf46e37e4e49f6385e
Closes-Bug: #1334965
Return hydrated net info from novanet add/remove_fixed_ip calls

Compute manager expects hydrated network info from network_api

Change-Id: I36e48742144555004cbd1f71429a943c7e2952b0
Closes-bug: #1350846
Fix swap_volumes

The attach and detach calls to cinder are better handled by cinder this
patch removes them. There is another patch against this bug to modify
cinder.

The attach call is being made against the new volume after the
migrate_volume_completion request. This means it may be done after the
new volume ID in the db has been deleted.

The detach call is being run at the end on the old ID and so may end up
detaching the migrated volume from cinder's perspective.

Change-Id: I3f2d98e9e473905a3de2f02e00c92ae3065e6ae7
Partial-Bug: 1316079
Add a policy for handling retrospective vetos

For some time, we've been informally implementing a "revert early"
policy around handling -2 worthy complaints after a patch has been
merged. These reverts can be contentious, so it's appropriate we make
it very clear what the policy and procedure here is in order to lessen
the contentiousness of these reverts.

Co-authored-by: Michael Still <mikal@stillhq.com>

Change-Id: I90e4153a80c859a928630809aef31e7313f08906
Pass certificate, key and cacert to glanceclient

When glance is configured with ssl nova does not pass it's certificate,
key, or cacert.  Updated the code to pass the certificate, key, and cacert used in the
nova.conf file, if they are set.  Removed nested if statement.

Change-Id: I23fcd070994e435cb42aa09c824d13553503a7c5
Closes-Bug: #1279381
always set --no-hosts for dnsmasq

Many network modes in nova-network disable reading host entries from
/etc/hosts. This includes running vish-ha (multihost), or if you ever
set a dns server. However if you don't do these things we continue to
use /etc/hosts.

It's not clear that this does anything other than create
inconsistency. There is a long standing bug that exists around this
because the impacts include exposing localhost as a dns entry on the
network.

Change-Id: Ibccf0999bfd4b37f2d948af34640260f95c54572
Closes-Bug: #1201873
Remove unused oslo.config import

Commit 46e88320e6e6231550f3e2b40312c51f55e059f5 added
the oslo.config import in one of it's patch series but
in the revision that was ultimately merged, the import
was no longer used - but we forgot to remove the import.

This cleans that up.

Change-Id: I3fcfdab625be68a19162b5053ad4f6f180a72a4d
Update invalid state error message on reboot API

This change adds more detail to the error message
when a request to soft reboot an instance in an
invalid state fails, indicating that specifically
soft reboot is the request that failed, when
a hard reboot might still succeed.

Closes-Bug: 1365683

Change-Id: I4b12aa261d1c4fa05bacd5311793ed6e0478e59b
Clean up LOG import in floating_ips_bulk v2 api

Remove useless code of LOG in floating_ips_bulk.py

Change-Id: I66f20a23d75b2c9c2e7cc2425f015a95fefb62d8
Add sqlite dev packages to devref env setup doc

While setting up a new Ubuntu Trusty development
system without devstack I ran into an undocumented
native package dependency for sqlite, so add those
here.

Change-Id: Ib82ea58e85f6ccab6008f2d19a31d0e890136ca7
Don't elevate context when rescheduling

The rescheduled launch should be executed with the same context
as the original request, so make sure that we aren't passing an
elevated context to reschedule.

Change-Id: Ic528f10067fa2977cc375c41b9c65d3517524ff8
Resolves-bug: 1366181
Network: enable instance deletion when dhcp release fails

In the event that the 'dhcp_release' fails we should continue
with the deletion of the instance.

This will no longer the leave the instance in ERROR state.

Change-Id: Ie8259180a9df12865940907c728a8d9da3c9fda0
Closes-bug: #1289397
VMware: prevent race condition with VNC port allocation

When spawning some instances, nova VMware driver could have a race condition
in VNC port allocation. This fix ensures that the lock is done on the
actual setting in the VM configuration spec.

Co-authored-by: Marcio Roberto Starke <marcio.starke@serpro.gov.br>

Change-Id: I70fab021bbf2df418df53e5f47e19cf16dbe45ac
Closes-bug: #1357372