Fix double reboot issue during soft reboot

Using the ID of domain in Libvirt to recognize the hard reboot has been
implemented or not, if the ID changed, we believe the domain has been rebooted,
return True and break from soft reboot.

Fixes: bug #1046356

Change-Id: Iec2f9e8225cfe2779f84d2095667f3c0e621e935
PXE driver should rmtree directories it created

Baremetal PXE driver was failing to delete the per-instance tftpboot and
image directories which it created when the instance was deleted. This
happened partly because of dangling files within the directory, and
partly because 'unlink' does not remove directories and the error was
squelched.

Now, when destroy()ing an instance, PXE driver will call shutil.rmtree()
on the per-instance directories which it created.

Fixes bug 1101048.

Change-Id: I682d0264288add488ea23e5d5200357b7be52dd9
Baremetal/utils should not log certain exceptions

unlink_without_raise was logging exceptions when it failed to unlink a
file, which is confusing since it is often called on non-existing files.

create_link_without_raise was also logging exceptions when it failed
to create a symlink.

This patch corrects this behaviour; both functions now explicitly check
for the type of error they expect, and then suppress that. If another
type of OSError is encountered, a warning is logged. This patch also
adds unit tests for both functions.

fixes bug 1097931.

Change-Id: Ie36f59dce34a6c67765770c5f968c003003acc88
ensure failure to inject user files results in startup error

This was the case in Essex but was inadvertantly changed in:
folsom-2-95-g0d166ca.

* nova/virt/disk/api.py: Refactor to allow specifying
mandatory injection items, that result in an exception
on failure to inject.
* nova/virt/libvirt/driver.py: Specify that user 'files'
are mandatory items and thus result in VM startup failure
unless injected successfully.
* nova/tests/test_virt_disk.py: A new test for the
separate warning and error cases.

Fixes bug: 1095744
Change-Id: Idab5c4294c1cb52098ce44a7aae957a44fb2674f
Avoid stuck task_state on snapshot image failure

Fixes bug LP 1101136

Previously if the glance interaction failed prior to an
instance being snapshot'd or backed up, the task state
remained stuck at image_snapshot/backup.

The normal task state reversion logic did not kick in,
as this is limited to the compute layer, whereas the
intial glance interaction occurs within the API layer.

Now, we avoid this problem by delaying setting the task
state until the initial initial image creation/retrieval
has completed.

Change-Id: Id498ae6b3674306743013e4fe99837da8e2031b5
Fix double reboot during resume_state_on_host_boot

Re-basing my commit introduced a mistake I did not catch before the
merge occurred.  resume_state_on_host_boot is calling both
_create_domain_and_network followed by _hard_reboot. It should only
call _hard_reboot.

Change-Id: I52be2c0d0c85abe6386842c980f26b3c5713e3e3
Add version constraint for coverage

Python coverage lower than 3.6 causes issues when running the whole
test suite. Some arguments are randomly mixed up with script names,
resulting in unittest failures like this:

UnexpectedMethodCallError: Unexpected method call.  unexpected:-  expected:+
- exists.__call__('/tmp/buildd/.../nova/virt/images.py') -> None
+ exists.__call__('/some/path') -> True

Change-Id: Iadb6d046e3acc75c6ed5abffcfc1abda82aeee4f
VMware Compute Driver Volume Management

blueprint vmware-compute-driver

Attach and Detach iSCSI volume
Get volume connector

Change-Id: I25e0c79ffb0b762726fb931233f7beeb53092a34
Added sample tests to floating-ip-pools API.

Partially implements blueprint nova-api-samples
Fixes bug 1070165

Change-Id: Ia7d99354ae815cdd8a41b012f568877751a3d8bd
Fixes task_log_get and task_log_get_all signatures

Fixes signatures for the above methods in nova/db/api.py
based on the recent TaskLog refactor on sqlalchemy/api.py.
Also adds unit tests for TaskLog.

Fixes Bug 1102652.

Change-Id: Id7ce74b8941d9eb6a50dfbfaa62e0fe05dd467c2
Avoid db lookup in info_from_instance().

The method info_from_instance() in nova.notifications was doing a db
lookup for the system_metadata for an instance.  This patch updates that
code to get that data from the instance that's passed in instead.  The
rest of the patch are related changes to make that happen.

metadata_to_dict() was needed here.  It lived in nova.compute.utils.
nova.compute.utils already imported nova.notifications, so using it from
there would have created a circular import.  Move the method to
nova.utils instead and update the tree to use it from its new location.

I also noticed that the xen driver had a copy of metdata_to_dict().
This patch removes it and uses the common implementation in nova.utils.

'system_metadata' was added to _extra_keys of the Instance db model so
that it would show up in a serialized instance.  Tests failed without it
as the result of getting instances via the conductor API did not include
system_metadata.  Now it's there.

Part of bp no-db-compute.

Change-Id: I451355fb26ae29f13b71438f7896c448b59f97b0
Fix hacking N302 import only modules

* Includes some general tools/hacking cleanup
* Fix several N302 cases
* Disable N302 until all cases are fixed

Change-Id: Iddba07ff13e10dc41a6930749044bb8c0572d279
Refactored run_command for better naming

This distinguishes this difference between
running commands on the local compute node and
the remote system, the VIOS.

bp powervm-compute-resize-migration

Change-Id: I57967e291f52d2745cd7d6d53553e37d9de07a6b
Merge "Fix double reboot during resume_state_on_host_boot"
Merge "Fix hacking N302 import only modules"
Increase maximum URI size for EC2 API to 16k

The EC2 API supports both HTTP GET and POST agnostically.  It also supports
user-data of 16k -- meaning that client tools could generate 16k URIs.
The WSGI default limit is 8k; this raises it.

Fixes bug 1098646.

Change-Id: Idec460d88b2affab970c9d9f39fa61295db035c5
Support optional 4 arg for nova-dhcpbridge

dnsmasq will call the add, old, and del actions with an optional
4 argument of hostname if the hostname is known. Since we populate
the hostname in the DHCP host file its always passed to us.

Add an optional 4 argument for hostname for the add, old, and del
actions.

Change-Id: I1a6c095324be2424d6c1d469877b254f88fba4aa
Fixes: bug #1103783
Merge "Increase maximum URI size for EC2 API to 16k"
Merge "Support optional 4 arg for nova-dhcpbridge"
Merge "Refactored run_command for better naming"