Only load necessary instance info for use in sync power state

During sync power state, instance is refreshed from the db. If the
instance gets deleted, this can result in InstanceInfoCacheNotFound
being thrown due to info_cache being loaded and therefore refreshed.

Sync power state does not make use of any extra instance attrs, so
this problem can be avoided by not loading any extra attrs in the
first place.

Two callers of _sync_instance_power_state() are updated to not load
extra attrs. 1) The periodic sync and 2) the lifecycle event handler.

Change-Id: I150bd95da65263981608dd9a033232bd142e737f
Closes-Bug: #1367845
Don't modify columns_to_join formal parameter in _manual_join_columns

The _manual_join_columns method was modifying the columns_to_join list
passed to it which, if not copied by the original caller, can introduce
bugs in the calling code if the original columns_to_join list is re-used
later, e.g. making nova objects with an expected_attrs list that has
been mutated by the DB API call.

This change makes a copy of the original list and passes back the new
list for the DB API methods to use, and adds some documentation to
hopefully clarify intent and usage.

A test is added to demonstrate the expected behavior of the utility
method.

Closes-Bug: #1393491

Change-Id: I49511f608291b66642ced00a3c13833e18fab3d7
Add shelve and unshelve info into devref doc

The diagram on http://docs.openstack.org/developer/nova/devref/vmstates.html
does not currently show the "SHELVED_OFFLOADED" state and the
associated "shelve", "shelveOffload" and "unshelve" state transitions.

Change-Id: I50eb07a84336160b8120ca294598576eeb982d28
Closes-Bug: #1394215
Compute: Catch binding failed exception while init host

While compute starts it will init all instances,
if an exception is raised from one instance
(e.g NovaException during plug_vifs), then the
compute process exits unexpectedly because of
this unhandled exception.
This commit changes the NovaException to more
appropriate VirtualInterfacePlugException and
catches it during init host, as well as the
instance is set to error state, with this change
the compute process can be started normally even
if this VirtualInterfacePlugException is raised.

Closes-bug: #1324041

Change-Id: Ia584dba66affb86787e3069df19bd17b89cb5c49
Share server access ips tests between V2 & V2.1

As server access ips should behave same in V2 and V2.1,
access ips tests can be shared among V2 & V2.1 to cleanup
duplicated test code.

This patch share those tests between V2 & V2.1

Partially implements blueprint v2-on-v3-api

Change-Id: Ife0b3f35ec6bd0d3c0052d2a8c0f314638a07ca3
Check for floating IP pool in nova-network

There are no checks in nova-network to see if a specified
floating IP pool exists or not before attempting to allocate
an IP address. If a pool does not exist, nova throws an
unrelated error - 'No more floating ips available'. The
following patch adds a check for the existence of the IP pool
and throws a FloatingIpPoolNotFound if it is not found.

Change-Id: I72cb8dd044f7ac094287bad7df9167b8e77a4b5b
Related-Bug: #1355623
Parse "networks" attribute if loading os-networks

"networks" attribute of "create a server" API should be used when
Neutron/os-network extension is loaded as TODO which is removed with
this patch mentions. V2.1 os-network extension has been implemented
with I94b7b476cbb02725a396725a379417b64afd58a7 already, so this patch
enables "networks" attribute for os-network extension.

Partially implements blueprint v2-on-v3-api

Change-Id: Ibe59ddc53d074f5b8147e24b35066d38d73c7c33
Remove unused db.api.fixed_ip_get_by_address_detailed

Change-Id: I23ccad0460f38db5314180fda865153bd8784c01
libvirt: Fixes live migration for volume backed instances

Live migration fails for volume backed instances in LibvirtDriver
because _is_shared_block_storage() incorrectly identifies volume
backed disks as local. This is fixed by passing the block_device_info
parameter to get_instance_disk_info, which allows for correct
filtering of volume backed disks.

Change-Id: I1437b2a7d5a62615b0099114ed1b5b1110f56de2
Closes-bug: 1392773
Modify v21 alias name for compatible with v2

This patch changes some of the alias names to be compatible with v2.
so the extension info output will be as same with v2.
 consoles -> os-consoles
 flavor-extra-specs -> os-flavor-extra-specs
 flavor-manage -> os-flavor-manage

Partially implements blueprint v2-on-v3-api

Change-Id: Ia8c6a0144e43134667636db9ba862b26f265a433
VMware: remove flag in tests indicating VC is supported

The ESX driver is no longer supported so this code is no longer
needed.

Change-Id: I03d2ea7935040b656c4a32de219d018f0ab89161
extract the timeout setup as a fixture

Includes tests for the nova specific parts of this.

Change-Id: I591f3c270804dfb01eab7c8c5d43b87a7abcdeda
simplify database fixture to the features we use

The Database fixture has existed for a long time, and has a large
amount of vestigial code that is no longer usable as it's masked by
other test setup. For instance, the conf fixture always sets the db
connection string to in memory db.

Simplify what the Database fixture is doing to just the inmemory
schema caching and reconstruction. Also provide a test case
demonstrating that this both sets up the db correctly, as well as
resets it after we've made changes.

Change-Id: Idc9b587c705c1c1ab4e1173ad1e8e244037284d7
Merge "libvirt: Fixes live migration for volume backed instances"
Merge "Remove unused db.api.fixed_ip_get_by_address_detailed"
Inline _instance_extra_get_by_instance_uuid_query

This function didn't do anything special, and only had 1 caller.

Change-Id: I8ff717a05893d8e2adbdf0c17bd4b81677d15b8c
Merge "extract the timeout setup as a fixture"
Extend clean_shutdown to the compute rpc layer

Adds the clean_shutdown parameter added to stop, resize, rescue,
shelve and shelve_offload in Juno to the compute rpc layer.  This
will in turn allow us to extend the API to provide a
forced-shutdown option to those operations.

Change-Id: Ie24c66e5006e64ef8b45d9d5056cc7876481a2f8
Partially-Implements: blueprint user-defined-shutdown
Add support for clean_shutdown to stop in compute api layer

Change Iec3dfd17725440958aac395ebc471e51afd6522e added
clean_shutdown support to the compute manager in Juno.
This change is part of a set to extend that to the compute
API layer, which will in turn allow us to extend the REST API
to provide a forced-shutdown option to those operations.

Change-Id: I38707cf869705f983ab35e549d95dc4be4811ece
Partially-Implements: blueprint user-defined-shutdown
Merge "Inline _instance_extra_get_by_instance_uuid_query"