Let resource_tracker report right migration status

A shutoff instance can be resized, but resource_tracker only think
active instance is in migration state. This patch let it report
right migration status.

Closes-Bug: #1247610

Change-Id: Ic6ce9c7d8fdecaf812831eb302186133f0ee71b1
add hints to api_samples documentation

Based on my own experience and questions I recently answered in
the openstack-nova channel, these hints could help developers.

Change-Id: Id7350f1a6a834b0de8456fdc28c2db890f2971de
Fix AttributeError(s) from get_v4/6_ips_by_interface

Add some defensive code to prevent AttributeError especially
when virtual_interface_get_by_uuid returns None. Add some basic
tests for good and bad data returned from
virtual_interface_get_by_uuid

Change-Id: I0ee1bb2087a6ed2619ac343534e4ade84e08d9bd
Closes-Bug: 1246087
Add flavor-access to core for V3 API

Adds the flavor-access plugin to the core list for the V3 API.
This is part of the process of adding all of the flavor related
extensions to core, with the exception of rxtx, as per the discussion
here: https://etherpad.openstack.org/p/NovaV3APICore

Note that since the alias changes from os-flavor-access to flavor-access
the api samples move directory and quite a few of other api samples
as well as flavor access is included by default.

Partially implements blueprint nova-v3-api

DocImpact

Change-Id: I9abc7df8f4620d462b0d381a2c0ed180c48f220e
Add flavor-extra-specs to core for V3 API

Adds the flavor-extra-specs plugin to the core list for the V3 API.
This is part of the process of adding all of the flavor related
extensions to core, with the exception of rxtx, as per the discussion
here: https://etherpad.openstack.org/p/NovaV3APICore

Partially implements blueprint nova-v3-api

Change-Id: Id4b4fad5e584bdf2bbbaaae579a07e28c312ef34
Remove used_limits extension from the V3 API

Remove the used_limits extension from the V3 API because
it shows usage for limits which are no longer going to be displayed
by the limits extension. See https://review.openstack.org/#/c/39872/
The limits are not being displayed there because they are already
available through the quotas extension.

Although the quotas extension currently only displays the quota limit
and not usage, the following proposed changeset adds display
of usage: https://review.openstack.org/#/c/32303/. And this is done
in a way which is much more consistent in terms of naming and interface
than used_limits does.

So we no longer need the used_limits extension

Partially implements blueprint nova-v3-api

Change-Id: I8ec1f300335ca4fc3b86323cc3af6bc1591da7bb
Fix limits v3 follow API v3 rules

* Add expected_errors decorator.
* Remove 'absolute_limit' from response, that already return by
extension quota_sets. And update api samples.

Partially implements bp nova-v3-api

Change-Id: Iea3ff494b8abed77fdc4b9e5e552784ed3733c34
Use 400 instead of 422 for invalid input in v3 servers core

We return 400 when input is invalid for v3 api. This patch fix
this case for servers core.

DocImpact

Change-Id: If802d4dcba6ed25f7e9cb9d568a3ffd78f0f6c68
Closes-Bug: 1251538
Removed unused instance object helper function

Remove code that is no longer used

Change-Id: Iaf99824ad28aa03206a997e69d9f6097ba034941
Remove V3 API version of coverage extension

As discussed at the summit in the QA session, this removes
the coverage extension from the V3 API. The coverage extension
doesn't work properly and has security issues. An alternative technique,
though more limited in nature will soon be proposed.

Partially implements blueprint nova-v3-api

Change-Id: Ib4296288fefeb7e1e11bb6e44bd1cf355f29b9b9
Remove V2 API version of coverage extensions

As discussed at the summit in the QA stream, this removes
the coverage extension from the V2 API.

According to the API Change Guidelines
https://wiki.openstack.org/wiki/APIChangeGuidelines
we should not be doing this. However the coverage
extension is a rather special case. It has security issues
such that it shouldn't be used in a production cloud
environment. It also doesn't work properly so its usefulness
to measure API coverage is limited. And in the past the presence of
this plugin has been used as a reason not to accept
an alternative technique to measure API coverage.

This is the proposed replacement

https://review.openstack.org/#/c/25882/

Change-Id: I07d798129ee277a6f7691c25f88c07a5204c0943
Add a driver method to toggle instance booting

Add a driver method to toggle the ability to boot an instance.

The only implementation provided is for the xenapi virt driver.  Other
drivers will log a warning and treat the method as a no-op.

Implements: blueprint cleanup-running-deleted-shutdown

Change-Id: Ie448c681d653b126cb12d3a48d79ab4febde8828
VMware: Always upload a snapshot as a preallocated disk

Currently, a snapshot operation always produces a preallocated
disk type. When the snapshot is uploaded to glance, the source
image metadata is uploaded with the new snapshot and then some
of the metadata properties are updated. The update doesn't
include the disk_type property, this can be problematic. If
the original image has a sparse disk type and a snapshot is
taken, then the snapshot which is a preallocated disk type is
uploaded to glance with the disk_type property set to sparse,
which creates a disk type mismatch and causes errors when a vm
is spawned from that snapshot. This patch addresses the issue
by correcting the image metadata used when uploading a snapshot
to glance.

Also, the image_upload function in vmwareapi/fake.py has been
modified to address the change. Any function call to
image_upload (from the unit tests) will assert that the
metadata for the image to be uploaded must have the disk_type
value to be preallocated.

Closes-Bug: 1247296
Change-Id: I71cfa8d4d1eb7911abb489153292fb0d1e614765
Clean up how test env variables are parsed

Oslo and cookiecutter and other projects were using
variable to parse test environment vaiables. Let us
do this in Nova

This is copied from 739627bf79619f8 Use cleaner version from
cookiecutter OpenStack template

Change-Id: I4c2b4072981985e7632fc775ffc98216fbfc0508
VMware: add support for VM diagnostics

The drivers now return diagnostics for a requested VM.

Implements blueprint vmware-vm-diagnostics

VMware specific keys will be namespaced.

DocImpact

Change-Id: I08f199f7e3b4edafe60ce73d2fc833f8c5a5574e
objects: declare some methods as static

A lot of code is composed of static methods, but they are not declared
as is.

Change-Id: I64be630b560e331aed9d1ab9a68152da52eb0d82
Use model_query() instead of session.query in db.instance_destroy

Unify db.instance_destroy() method to use model_query() as others
methods use.

Side effect we won't try to soft delete already soft deleted instance

Change-Id: I36d5bb42b16901c260ff4276e31a6aec46ffa8a4
handle migration errors

handle errors:
   MigrationPreCheckError, NoValidHost,
   InvalidSharedStorage and InvalidLocalStorage
in _migrate_live so that the 'reason' is passed up

Closes-Bug: 1250523

Change-Id: I3d7e0626223f2606c5e7205e5022e26585506cfb
Fix missing format specifier in ImagePropertiesFilter log message

Commit a52259e introduced some changes to debug log messages in the
ImagePropertiesFilter but with that came a bug due to a missing format
specifier.

Closes-Bug: #1253875

Change-Id: Ic60b7ba179bd43b80c500be084a1809b6b0b32ca
Use of logging in native thread causes deadlock connecting to libvirtd

When connecting to libvirtd daemon we were calling the _connect
method in a native thread. The use of logging in this method hit this
bug in eventlet https://bitbucket.org/eventlet/eventlet/issue/137/
causing nova-compute to hang.

Closes-Bug: #1252409

Change-Id: I0d22c3ed1b3288aeb1afae130f11a4f38dc6440a