don't clear the database dicts in the tearDown method

This is superfluous as the two dicts (self.engines and
self.test_databases) are re-initialised during setUp anyway and aren't
used in between

Change-Id: Icafa9bdf3b0193f31752e6104ecac0e49476c812
Map libvirt error to InstanceNotFound in get_instance_disk_info

When getting instance disk info, the instance may be destroyed/deleted, and a
libvirtError will be raised in XMLDesc method, so catching and mapping it to
InstanceNotFound and the caller can handle it correctly.

Fixes bug #1064581

Change-Id: I07fed3e82e10dad4cb84ae5c8650ada351c24e78
Tests for PXE bare-metal provisioning helper server.

a part of blueprint general-bare-metal-provisioning-framework.

The deploy helper was landed without tests. This patch reinstates the
tests.

Change-Id: I690ec077e175dc37c2cc5f18170513b0e5bca103
Co-authored-by: Mikyung Kang <mkkang@isi.edu>
Co-authored-by: David Kang <dkang@isi.edu>
Co-authored-by: Ken Igarashi <igarashik@nttdocomo.co.jp>
Co-authored-by: Arata Notsu <notsu@virtualtech.jp>
Co-authored-by: Devananda van der Veen <devananda.vdv@gmail.com>
Fix state sync logic related to the PAUSED VM state

A VM may get into the paused state not only because the user request
via API calls, but also due to (temporary) external instrumentations.
Before the virt layer can reliably report the reason, we simply ignore
the state discrepancy. In many cases, the VM state will go back to
running after the external instrumentation is done.

Fix bug 1097806.

Change-Id: I8edef45d60fa79d6ddebf7d0438042a7b3986b55
Add Compute API validations for block device map

This patch adds validations to return 400 BadRequest to
Create server(s) with block device mappings where appropriate
and related unit tests

Fixes bug 1094244

Change-Id: I778d3ed4b0871561774594e950ce98b6a1ecfcb9
Ignore auto-generated files by lintstack

Part of blueprint lintstack

Change-Id: I6ca0ec71c7fa083a86920a7ae055ef8e9d19c512
Go back to the original branch after pylint check

Previously we save the current commit hash before running lintstack.
This is no problem for Jenkins but not very friendly if we run
lintstack locally. Every time after it's done, git is in detached
HEAD mode. This patch remembers the current branch name and
automatically go back to the branch if available. Otherwise we fall
back to the commit hash.

Part of blueprint lintstack

Change-Id: I145df3027022fc9a8302bc6da9ab300f677f3386
Refactor: extract method: driver_dict_from_config

Related to blueprint xenapi-volume-drivers

Extract method to create driver registries, so that it could be re-used
in xenapi for a similar purpose.

Change-Id: Ifd25d12335ac96a15dfea4c5ab7f315ff7ddfd8b
XenAPI: Fix volume detach

fixes bug 1101229

The code-cleanup and code-move changes from this patch:
https://review.openstack.org/#/c/19219/
introduced a problem: the volume_utils.find_sr_from_vbd method was
called after the vbd has already been destroyed. This change fixes the
issue by moving the sr discovery before the vbd destruction, and tests
the call order by using side effects.

Change-Id: Ide4f8ac810f98bb192909f5f0408affc940e7446
Run_as_root to ensure resize2fs succeed for all image backends

Unlike other file based image backend, operation over LVM
logical volumes requires root to functioning correctly. Resize2fs
on logical volumes also need run_as_root.

Otherwise, when running nova-compute as non-root user, instance
creation with flavor disk size larger than image size would fail.
Caused by resize2fs failure.

Fixes LP# 1101137

Change-Id: I16a2aa4382be58843270ff23e11bfb1e20c568d2
More consistent libvirt XML handling and cleanup

Fixes bug 1081373

The overall goal of this effort is to make handling of libvirt XMLs
more consistent. First, the on-disk XML file in the instances
directory is never to be used by the libvirt driver. It will be
generated via to_xml(). Second, anytime it is safe to re-define the
XML using the version we programatically generate, we do so.

Extended to_xml() to support a new write_to_disk argument. When true,
this the resulting XML will be written to the instances directory.
Renamed _get_domain_xml to _get_existing_domain_xml, as it should only
be used when we need the XML for a defined domain. It will continue to
fall back to using to_xml() if the defined version is not available.

Hard reboots will now assume nothing about the defined state within
libvirt. Rather, we generate and define the domain every time. This
eliminates the possibility of the hypervisor and Nova database going
out of sync (i.e. volume attached in defined XML with Nova having no
knowledge). We no longer need to pass an xml argument into
_hard_reboot() since it is always generated.

Resume state on boot has been updated with logic similar to that of
_hard_reboot. Again, after a hypervisor reboot (or detection of downed
VMs on compute start), we should not assume the defined XML is valid.
The state or configuration of the VM, from Nova's perspective, may
have been changed while the system was down.

The post_live_migration_on_destination() method used to call to_xml()
without block_device_info. This would result in an on-disk XML without
any volume mappings. The compute manager has been updated to pass bdi
so that this XML can be generated and written correctly. The Mox stub
for this function has been replaced by a stub in the fake virt driver.

Change-Id: I4aa1068fa3aa54fdb7e690d46b6cf2e41bb20bc9
Prevent metadata updates until instance is active.

This avoids a race condition where setting metadata too early in the
build process can cause a build failure due to the VM not existing
in Xen.

bug 1100866

Change-Id: I702128799997d91d5f0c0ac189f619bfdf011988
fix floating ip test that wasn't running

the associate_floating_ip negative test wasn't running because
the indent was wrong. This was uncovered when looking for unknown
names with pyflakes.

This makes the test run, and fixes the mispelling of floating to
make it actually pass.

Change-Id: I74397b8592d51a7fed249c96f17d3e71cdf82a7e
Sync latest cfg from oslo-incubator

The main change is:

  c5984ba Move logging config options into the log module

Re-generate nova.conf.sample to reflect the options move.

Change-Id: I3913ea54465658d93dc56e014dfe5d911b0541d6
Allow snapshots of paused and suspended instances

Fixes bug 1100556

Remove the restriction in the API of snapshotting instances which
are in paused or suspended states and update the libvirt driver
to deal with this accordingly.

Other drivers may need to be updated accordingly.

Change-Id: Iabeb44f843c3c04f767c4103038fcf6c52966ff3
Generate coverage even if tests failed.

A test failing doesn't make coverage useless, so we should always
generate it.

Change-Id: I3ed9e54213c7fb6756906553920d19c5075e1adb
Fix get Floating ip pools action name to match with its policy

Get Floating IP Pools is always responding 403 (Forbidden) cause its method
name, which is also being used as action name, doesn't match with the name in
policy.json file.
I opted for this approach cause it also fixes the different names between the
extension (which is named floating **ip** pools instead of floating pools) and
the backend.
Another good practice would be change the default_floating_pool option to
default_float_ip_pool, but that would require change the conf file and possibly
break some compatibility.

Fixes bug 1091668

Change-Id: I2627e8b0f45c5c39bbbf1a4a12b448e1992fa4b4
fix mysql race in tests

because of testr, we can't assume the migrations tests run in
sequence, so they make run all at the same time. This is fine for
our in memory sqlite db, but when talking to a real external
database, not so good.

Since a second mysql test was added, we started racing on these
migrations them often hitting at the same time causing random fails
in gate.

Fix this by moving all the mysql tests into 1 test function, so we
won't race.

Fixes bug #1101874

Change-Id: I7627f3bd510266c99af018d355f960b352573759
fix misspellings in logs, comments and tests

Flagged with: https://github.com/lyda/misspell-check
Run with: git ls-files | misspellings -f -

Fixes bug: 1100083
Change-Id: Icf1f844fea8ad0a1101d1dc64b9a126608e9536e
Unpin testtools.

Change-Id: If760a7b6f8771d9f9ac2722c300032e715e34151