xenapi: refactor - extract get_stream_funct_for

This change extracts the get_stream_funct_for function. As we would like
to support tar.gz compressed raw images, the stream function for them
should be different from the plain ones.

related to blueprint xenapi-supported-image-import-export

Change-Id: I27681da069ba2cd23670ef41f54ed9ac5af31ae6
xenapi: refactor - extract get_virtual_size

This change extracts a function get_virtual_size. This is needed,
because we would like to support tar.gz files, in which case the image's
virtual size will be determined by looking at the tar.gz stream. The
first tarinfo will tell us the size.

related to blueprint xenapi-supported-image-import-export

Change-Id: If34bb8849ad2dacf471f1bec5e72a37ddda79464
Remove trivial cases of unused variables

Removes trivial cases of unused variables from V3 API version of
servers.py. This was applied to the V2 API but not the V3 API
in commit 9f7eaca80d259201844f097fec24d1e9fe9fb5e3.

Change-Id: I42c1eb73a502afb61bf9c2dd81b79266e0966b5f
Fix message for server name with whitespace

Change-Id: I9007ada4db6502da084fe44ab585e62e689e2f4f
Fixes: bug #1207625
Refactor integrated tests to support V2 and V3 API testing Part 2

This changeset moves the APISampleTestBase class out into its own
file so the V3 versions of the tests can cleanly import it separate
from the V2 code.

Partially implements v3-api-unittests
Partially implements v3-api-specification

Change-Id: If0994934edb2ab19004353f9a4089ce525ed31dd
Port integrated tests for V3 API

Ports most of the integrated tests to the V3 API. The
test api samples test will be ported to the V3 API in a later
changeset

Partially implements blueprint v3-api-unittests

Change-Id: I00a93471d95760e6c97534808c5a93522710b20d
Adds infra for v3 API sample creation

Adds the required infrastructure for the v3 API sample creation.
- v3 API samples and the test cases are in their own v3 directory
- API samples from "core" are now put in their own (per plugin)
  directory rather than in the root directory
- Testcases to create api samples are now placed in individual
  files rather than one giant one. I think it makes it clearer what
  testcases we do have and avoids merge conflict pain

Adds API samples for basic servers interaction

Fix for bug in servers.py where if no extension was loaded which
extends create xml deserialization an exception would occur (fixed
here because it causes the api sample tests ported to fail).

Partially implements blueprint v3-api-unittests

Change-Id: I653af49dcc499c18f14b6e1764711a9922354069
Update mailmap

Change-Id: I7cc1c069aefca3f61328fa92d7366b5b755febaf
Handle InstanceInvalidState of soft_delete

Since commit 99c51e34230394cadf0b82e364ea10c38e193979, instance
is not able to be soft_deleted if instance has never been active.
But there are circumstances when instance fails on initial build.
Thus if we set reclaim_instance_interval in nova.conf, we cannot
delete such instances in any way.

This patch fixes this by handling InstanceInvalidState exception
when try to soft_delete instance. The delete() function is called
for instance which has never been active.

Bug 1202158

Change-Id: I7d0d9414d7436e56b6a3e5861ee3c2f02473df2b
Fix IVS vif to correctly delete interfaces on unplug

The IVS unplug method was incorrectly calling the OVS
specific interface removal methods. This corrects it
to call the IVS removal methods.

Fixes: bug #1214532
Change-Id: Ie718e167d5bdcfae0d915ef85976aeeb3449ad18
Set VM back to its original state if cold migration failed

If migration of a VM in the STOPPED state failed due to
NoValidHost, the VM was erroneously set to ACTIVE.

The fix sets the VM back to its original state instead.

Fix bug 1204905

Change-Id: I67ccac9ff4104fbd63c545b9f9399206a7febae7
Remove duplicate variable _host_state

* The variable _host_state is initialized twice in __init__() method of libvirt driver.
This patch removes the second occurrence.

Fixes bug #1214431

Change-Id: I13f175abcbce7752afb0d8c69ee209928afa1eb3
Implement hard reboot for powervm driver

Implements the OS reboot API in the powervm driver. Soft reboot is not
supported as it requires additional configuration on the LPAR when it's
spawned so this patch simply performs an immediate hard reboot.

Also adds tests for start_lpar and stop_lpar which didn't exist before.

Closes-Bug: #1205089

Change-Id: Iad64dda79f46d18c7aa5bee12ed41e7a01853d34
Fix instance metadata access in xenapi

In preparation for objects, use the helper method in utils to get instance
metadata.

Related to blueprint unified-object-model

Change-Id: I552369937854ec195d1f9abb1835b4e6afef7a70
Log exception details setting vm_state to error

This commit switches to LOG.exception() for exceptions that trigger
setting an instance into an error vm_state. Prior to this commit just
an error message was printed which provided no context for debugging
an actual failure. By switching to LOG.exception() instead of
LOG.error() a traceback will be printed to show where the exception
came from.

Fixes bug 1215068

Change-Id: If3aaf3d8ec61c3e902c299f3e5889dcafa144a9a
Use ExceptionHelper to bypass @client_exceptions

Rather than stubbing out catch_client_exception() to bypass the wrapping
of exceptions in ClientException, use the existing ExceptionHelper to do
it.

This is import for oslo.messaging, because there's no public API method
for us to stub out.

blueprint: oslo-messaging
Change-Id: I32c83ef15fece35526739c4fcdfccf7a275c7033
Disable periodic tasks during integration tests

While debugging various test failures, I was confused several times by
weirdness caused by periodic tasks running during the tests.

It doesn't seem to make much sense to allow this to happen, even if it
doesn't appear to affect the tests.

Change-Id: I75d34b5ee2bf76b64ae8ba108d9c78b741ce4574
Enable consoleauth service during ec2 tests

Otherwise delete_tokens_for_instance() fails if we time out calls()
in the fake RPC driver when there are no consumers for a topic.

blueprint: oslo-messaging
Change-Id: Id27f20144bc3dc273d72b2dad59c94bddd61f03f
Add CastAsCall fixture

In Ia7f40718533e450f00cd3e7d753ac65755c70588 we made the EC2 integration
tests treat all RPC cast()s as call()s. The idea being that rather than
insert sleeps in the tests to ensure that a cast()ed operation is
completed before we continue, we instead just do the operation with
call() and thereby block until it is completed.

If we take a step back, the approach is questionable - if there's no way
for us to block on the completion of an async task in our unit tests,
what hope do users have?

But, it is a sweet little hack, and is better than sleeping in tests.

With oslo.messaging, it will be both harder to implement this hack and
seemingly needed more often. Encode the hack as a fixture so it can be
easily re-used.

blueprint: oslo-messaging
Change-Id: I5f9e74f46e347594103484460e81da46835eeb7e
Fix api samples image service stub

If you enable CastAsCall with test_api_samples, test_post_backup_server
fails because the ImageService.detail() method returns None and you
get:

  TypeError: object of type 'NoneType' has no len()

Fix by returning an empty list.

blueprint: oslo-messaging
Change-Id: I0cc413d6d428caebb1f67cbc701a36227dead567