Include proper Content-Type in the HTTP Headers

Nova Metadata Service was not setting the Content-Type
in the HTTP Response headers. Because of this, clients
were receiving the default Content-Type which is "text/html".
The changes done in this patch will include proper Content-Type
in the HTTP response headers.

Closes-Bug: #1243878
Change-Id: Ia65cdf8099c85f63d7f44f93835c606d679a7ab3
Use osapi_glance_link_prefix for image location header

When an image is created the location header returned is generated based
on the request url that nova-api receives.  If SSL termination happens
upstream the request url is an http request, and the location header is
wrongly generated with that.  The osapi_glance_link_prefix config option
is available for almost precisely this purpose, so let's use it.

Change-Id: I24e46ed8fc0763f9ddeec65e0d79590dd7b86bef
Closes-bug: 1298005
Catch InstanceInLocked exception for rescue and instance metadata APIs

When VM is locked by nova, VM API should not execute.
There are so many api that cannot be executed when vm is locked.
rescue/unrescue
update_instance_metadata/delete_instance_metadata
the above APIs should be updated to catch InstanceIsLocked exception

Change-Id: Ic88f3d190894d4b65f54b34660efd9ae83fb0ee4
Closes-Bug: #1266960
Remove unnecessary check for CONF.notify_on_state_change

Already checked CONF.notify_on_state_change at beginning, no
need to check it again

Change-Id: I757ff7b71bafdee9e2e6009e7489db520704c8cc
Remove redundant tests from Qcow2TestCase

Qcow2TestCase had exact copies of two tests in the parent
_ImageTestCase so remove them. I'm assuming the tests originated
in Qcow2TestCase but later more image types were added and the
tests were pulled up into the common parent mixin since they
applied to all types of images, but the redundant tests from
Qcow2TestCase weren't cleaned up.

Related-Bug: #1302211

Change-Id: I031516b555ead5a3769fb62d90e373aac1b6baf1
Updated from global requirements

Change-Id: I580d7706618c05ef24e1cae0bb0f887968aca26c
Recover from POWERING-* state on compute manager start-up

If a compute manager is stopped / fails during POWERING-ON or
POWERING-OFF operations then the instance will be left stuck in
a transitional task_state.

This change handles both these states by simply either retriggering
the power on or power off request on start-up of the compute manager.
If the operation had previously completed then retrying will
essentially be a no-op so it is safe to handle instances in this
state in this way.

Closes bug: #1304333
Change-Id: I262be6b13a764bc1355be34cdc0180e8e87f260b
XenAPI: Add host information to glance download logs

This change adds a context field to the xenapi session
call_plugin_serialized_with_retry method to provide better
logging information.  This is then used by the glance image
downloading to provide the host IP we are contacting.  This
saves having to dig through other logs to get this information.

I added a mock unit test to test_glance to verify the download
retry mechanism works since this wasn't verified before.
It is the first mock test in test_glance.py

Change-Id: I4de2b9e61ce25fb8ab693eac2f67559d0eb01a77
Fix bug detach volume fails with "KeyError" in EC2

Modified volume['instance_uuid'] to volume.get('instance_uuid'),
to avoid "KeyError", when detach a volume with "attaching" status.

Change-Id: Ic1aacab212c2a2026d18ef57ad9d77fdc6d86b47
Closes-Bug: #1280532
support local debug logging

Add OS_DEBUG as an environmental variable that will let you run
locally with debug logs when trying to debug a particular unit
test. Inspired by similar function in the neutron base tests.

History: I70790aa39c0774726de71e5fa5751c45e7e34ffd was enabled on
March 10th to address 2 issues, make it possible to see DEBUG logs
during test runs, and test that the LOG.debug functions that we happen
to execute as a side effect of unit tests did not have syntax errors
(it's important to note it was not guarunteed coverage for all debug
messages).

The net effect was largely a DDOS on the testing system. This
generated a 190MB subunit_log per test run. It meant on failures in
Nova unit tests now generated such an enormous dump that they became
non indexable in elastic recheck (it overloaded the indexers), so the
bug classification system was blinded to any nova unit test bugs.

It was thus reverted in: Ia1cfc3d874ddf4f8ee2037b487d22f968585854e

This patch solves the first concern, which is developer targetted
testing. It mirrors what is done in Neutron for cross project
consistency. There was a competing approach in requiring OS_LOG_LEVEL
to be set, however it had the same problem as the initial patch,
defaulting to DEBUG means that it's a DOS (at 15k unit tests, that
just can't be the default). Given that Neutron already established
some conventions here, using existing conventions also seems better.

This *does not* address concern #2. That is expected to be done in a
separate patch using an ephemeral string buffer logger. However
getting that correct has proved tricky enough that the approach to the
problem was:

 1. revert - Ia1cfc3d874ddf4f8ee2037b487d22f968585854e
 2. add targeted debug logging (this patch)
 3. add back side effect debug testing - being worked by joe

Change-Id: I04ff2f0650f1e6035d8de5c5802b96ce4294891b
Add RBAC policy for ec2 API security groups calls

The revoke_security_group_ingress, revoke_security_group_ingress, and
delete_security_group calls in the ec2 API were not restricted by policy
checks.  This prevented a deployer from restricting their usage via
roles or other checks.  Checks have been added for these calls.

Closes-Bug: #1290537
Change-Id: I4bf681bedd68ed2216b429d34db735823e0a6189
Read deleted instances during lifecycle events

This prevents us from failing to find the associated instance if
the lifecycle event happens after the instance has been deleted,
as in the case of a ... delete.

Change-Id: I819ddf8ff68edc407d7932bf43771c440f514f26
Closes-bug: #1304968
Merge "support local debug logging"
Merge "Recover from POWERING-* state on compute manager start-up"
Merge "Updated from global requirements"
Merge "Catch InstanceInLocked exception for rescue and instance metadata APIs"
Merge "XenAPI: Add host information to glance download logs"
Merge "Add RBAC policy for ec2 API security groups calls"
Merge "Read deleted instances during lifecycle events"
Merge "Use osapi_glance_link_prefix for image location header"