Correct the state for PAUSED instances on reboot

If an instance is in the PAUSED state before the compute node is
restarted, it will be left in an inconsistent state (impossible to
delete and/or restart it) after the node boots again.

This patch adds a check in the periodic task sync_power_state so when
this situation is detected the instance is stopped and left in a state
from where it can be started again or deleted.

Closes-Bug: #1265494
Change-Id: I7266ace596598522c28a3839091ef50faf4463c2
Don't detach root device volume

Detach a root device volume may cause various guest error. Although
we can argue that it's user response to not detach the root device
volume, a check in nova will be more helpful.

We can either give a warning message, or not allow it. This patch
forbids this operation, considering that even in real system, remove
root device does not make much sense.

Closes-Bug: #1279300

DocImpact

Change-Id: I01f086be668b35442a2f139f4d7dc96884a3bda6
Signed-off-by: Yunhong Jiang <yunhong.jiang@intel.com>
Reverts change to default state_path

Reverting the change to the default value of state_path introduced by
I94502bcfac8b372271acd0dbc1710c0e3009b8e1 for the reasons set out
in my -1 review of the same that seems to have been skipped when the
change was accepted.

As implemented the change will break any existing systems that are using
the default value of state_path with no warning period, which goes beyond
the scope of change for UpgradeImpact

A better approach would be to introduce a new config value that is used in
the code, but is over-ridden by "state_path" if it is still in the config file,
and log a warning that state_path is deprecated and will be removed in the future.

That way the change to default behavior can be introduced at the start of the
next release cycle by removing the deprecated state_path config value, and
users have until then to prepare.

It does mean that we keep the insane default for a bit longer, but I think that's
much better that breaking a running system.

Closes-bug: #1291515

Change-Id: I7bb70c551ec9616276552d2e32076b66a1d3ce01
Fix docstring for shelve_offload_instance in compute manager

The shelve_offload_instance method in the compute manager is using an
instance object now but the docstring for the instance param said it was
a dict.

Change-Id: I9376f09402ac00e570a58f9b58b2f4aa56a7e16d
Move log statement to expose actually info_cache value

When debugging an info_cache issue I noticed the logs  display:

"Updating cache with info: None update_instance_cache_with_nw_info"

Though we are not actually updating the info_cache with None as if
nw_info is not isinstance(NetworkInfo we fetch the latest nw_info
from neturon. This patch just moves the log statement so the actual
value is exposed.

Change-Id: I3a74191c31a053682bb2558c59228025ceb0cf28
Closes-bug: 1292298
Remove aggregate_get() from conductor API

The aggregate_get() method of the conductor API is no longer used.
Remove it from the client code and leave a note on the manager side
for it to be removed in the next major version bump.

This is cleanup, but part of a blueprint being completed during
feature freeze.  It allows us to drop more code in the rpc version
bump.

Part of blueprint rpc-major-version-updates-icehouse

Change-Id: I893f58bbcae21112f186348cefcbfdd3bcbb8c06
add support for host driver cleanup during shutdown

Provides a place for any drivers to potentially
setup graceful-shutdown code.

VMware driver's proper driver-lifecycle code included.
This is critical in environments where the the vmware
driver is setup and torn down at high frequency.

Prevents run-away vSphere by closing stateless
HTTP management sessions gracefully.

Change-Id: I67a91613643540243ab1210b333ed8e121f05802
related to bug: 1262288
Closes-bug: 1292583
Remove aggregate_get_by_host() from conductor API

The aggregate_get_by_host() method of the conductor API is no longer
used.  Remove it from the client code and leave a note on the manager
side for it to be removed in the next major version bump.

This is cleanup, but part of a blueprint being completed during
feature freeze.  It allows us to drop more code in the rpc version
bump.

Part of blueprint rpc-major-version-updates-icehouse

Change-Id: Ib28fab80adbe796ecaded040d4c33ea05659e509
Remove instance_get() from conductor API

The instance_get() method of the conductor API is no longer
used.  Remove it from the client code and leave a note on the manager
side for it to be removed in the next major version bump.

This is cleanup, but part of a blueprint being completed during
feature freeze.  It allows us to drop more code in the rpc version
bump.

Part of blueprint rpc-major-version-updates-icehouse

Change-Id: Id2ed674c2d4c491bc7b0b0c79a64bef84fb05db4
Remove migration_update() from conductor API

The migration_update() method of the conductor API is no longer
used.  Remove it from the client code and leave a note on the manager
side for it to be removed in the next major version bump.

This is cleanup, but part of a blueprint being completed during
feature freeze.  It allows us to drop more code in the rpc version
bump.

Part of blueprint rpc-major-version-updates-icehouse

Change-Id: Id385255cbfff8c7a42a9a1551044746b7f6344bd
Fix equal_any() DB API helper

equal_any() query helper is meant to produce a WHERE clause combining
a few conditions with OR operator in order to select the rows for
which the given column is equal to ANY of passed values. In fact,
it uses AND operator instead of OR, which means that an incorrect
WHERE clause will be returned, if more than one value is passed to
equal_any().

This is caused by the fact that the or_() function from SQLAlchemy
expression language is used incorrectly: it accepts unpacked
instances of BinaryExpression, but a list of ones is passed.
Unfortunately, in 0.7.x and 0.8.x branches of SQLAlchemy passing
a list of values to or_() won't raise an exception immidiately,
and the result will be an iterable, which explains why we didn't
see equal_any() was actually broken before. Besides, equal_any() is
rarely used in our code base and the test didn't cover the case when
more than 1 value is passed to the function.

This effectively fixes the error when running Nova unit tests
with SQLAlchemy 0.9.x releases.

Closes-Bug: #1292285

Change-Id: If4c0f1d3e016e2affb0d49293c62ca9df0d033f7
Fix sqlalchemy utils test cases for SA 0.9.x

To be able to run migration scripts on SQLite we've been providing
a few workarounds for sqlalchemy-migrate and SQLAlchemy 0.7.x and
0.8.x branches in the form of sqlalchemy utils module. In 0.9.x
releases a few workarounds become obsolete (due to improvements made
to schema reflection for SQLite in SA), but our test cases still
expect SA to raise exceptions in several cases.

As long as our utils can work well with 0.9.x SA releases with our
workarounds and without them, we can just skip the test cases
expecting exceptions to be raised when SA version > 0.9.0 is used
and continue to run them for older releases of SA.

Closes-Bug: #1292284

Change-Id: Ie6cdb017b64b393cff43957b2047b5aae142d54e
Make sure instance saves network_info when we go ACTIVE

This makes sure that when we do a save() on our instance during
creation, that we don't save the empty network_info we created
earlier over a potentially-full one built during the process.
This may happen since we wrap the networkinfo in the async helper.
This also moves the network_info.wait() out from under a conditional
which could cover up other issues leading to a case where it is
None (which shouldn't be possible). If we didn't call wait(),
then saving it wouldn't be productive or correct.

Change-Id: I9bd8dd0dce9e17cd128c7495acd52aa82119c57c
Related-bug: 1249065
Remove block_device_mapping_destroy() from conductor API

The block_device_mapping_destroy() method of the conductor API is no
longer used.  Remove it from the client code and leave a note on the
manager side for it to be removed in the next major version bump.

This is cleanup, but part of a blueprint being completed during
feature freeze.  It allows us to drop more code in the rpc version
bump.

Part of blueprint rpc-major-version-updates-icehouse

Change-Id: Iffdb6f77cd611918ca27f64162c1b4f13a46815b
Merge "Fix equal_any() DB API helper"
Merge "Move log statement to expose actually info_cache value"
Merge "Remove aggregate_get() from conductor API"
Merge "Fix sqlalchemy utils test cases for SA 0.9.x"
Merge "add support for host driver cleanup during shutdown"
Merge "Remove aggregate_get_by_host() from conductor API"