Catch InstanceInLocked exception for rescue and instance metadata APIs

When VM is locked by nova, VM API should not execute.
There are so many api that cannot be executed when vm is locked.
rescue/unrescue
update_instance_metadata/delete_instance_metadata
the above APIs should be updated to catch InstanceIsLocked exception

Change-Id: Ic88f3d190894d4b65f54b34660efd9ae83fb0ee4
Closes-Bug: #1266960
Refactor Network API

Base network API class to have all the methods that can be implemented
by specific network api's.

 * Move common logic and decorators to network/base_api.py
 * Base class base_api.NetworkAPI with current methods from
   nova & neutron, that will raise NotImplementedError()
 * Refactor some tests with mock

When this get merged, every review adding new methods to neutron or
nova network, must have the same methods added to the base NetworkAPI.

Implements: bp refactor-network-api

Change-Id: Ia507e80dfd1e0b7552c5d50e84ac9e642ca8c9fe
Updated from global requirements

Change-Id: I580d7706618c05ef24e1cae0bb0f887968aca26c
Recover from POWERING-* state on compute manager start-up

If a compute manager is stopped / fails during POWERING-ON or
POWERING-OFF operations then the instance will be left stuck in
a transitional task_state.

This change handles both these states by simply either retriggering
the power on or power off request on start-up of the compute manager.
If the operation had previously completed then retrying will
essentially be a no-op so it is safe to handle instances in this
state in this way.

Closes bug: #1304333
Change-Id: I262be6b13a764bc1355be34cdc0180e8e87f260b
XenAPI: Add host information to glance download logs

This change adds a context field to the xenapi session
call_plugin_serialized_with_retry method to provide better
logging information.  This is then used by the glance image
downloading to provide the host IP we are contacting.  This
saves having to dig through other logs to get this information.

I added a mock unit test to test_glance to verify the download
retry mechanism works since this wasn't verified before.
It is the first mock test in test_glance.py

Change-Id: I4de2b9e61ce25fb8ab693eac2f67559d0eb01a77
Xen: Do not resize root volumes

Boot from volume on Xen wants to resize the volume
when the image size doesn't jive with the volume size.
This patch doesn't attempt the resize.

Closes-Bug: #1296839
Change-Id: Ic03b7997f61f1a22362dafd86664db681e54087e
support local debug logging

Add OS_DEBUG as an environmental variable that will let you run
locally with debug logs when trying to debug a particular unit
test. Inspired by similar function in the neutron base tests.

History: I70790aa39c0774726de71e5fa5751c45e7e34ffd was enabled on
March 10th to address 2 issues, make it possible to see DEBUG logs
during test runs, and test that the LOG.debug functions that we happen
to execute as a side effect of unit tests did not have syntax errors
(it's important to note it was not guarunteed coverage for all debug
messages).

The net effect was largely a DDOS on the testing system. This
generated a 190MB subunit_log per test run. It meant on failures in
Nova unit tests now generated such an enormous dump that they became
non indexable in elastic recheck (it overloaded the indexers), so the
bug classification system was blinded to any nova unit test bugs.

It was thus reverted in: Ia1cfc3d874ddf4f8ee2037b487d22f968585854e

This patch solves the first concern, which is developer targetted
testing. It mirrors what is done in Neutron for cross project
consistency. There was a competing approach in requiring OS_LOG_LEVEL
to be set, however it had the same problem as the initial patch,
defaulting to DEBUG means that it's a DOS (at 15k unit tests, that
just can't be the default). Given that Neutron already established
some conventions here, using existing conventions also seems better.

This *does not* address concern #2. That is expected to be done in a
separate patch using an ephemeral string buffer logger. However
getting that correct has proved tricky enough that the approach to the
problem was:

 1. revert - Ia1cfc3d874ddf4f8ee2037b487d22f968585854e
 2. add targeted debug logging (this patch)
 3. add back side effect debug testing - being worked by joe

Change-Id: I04ff2f0650f1e6035d8de5c5802b96ce4294891b
Add RBAC policy for ec2 API security groups calls

The revoke_security_group_ingress, revoke_security_group_ingress, and
delete_security_group calls in the ec2 API were not restricted by policy
checks.  This prevented a deployer from restricting their usage via
roles or other checks.  Checks have been added for these calls.

Closes-Bug: #1290537
Change-Id: I4bf681bedd68ed2216b429d34db735823e0a6189
Merge "Xen: Do not resize root volumes"
Merge "Refactor Network API"
Merge "support local debug logging"
Merge "Recover from POWERING-* state on compute manager start-up"
Merge "Updated from global requirements"
Merge "Catch InstanceInLocked exception for rescue and instance metadata APIs"
Merge "XenAPI: Add host information to glance download logs"
Merge "Add RBAC policy for ec2 API security groups calls"