Remove duplicate test from V3 version of test_hosts

Removes duplicate test_host_startup_service_unavailable
test.

Partially implements blueprint v3-api-unittests

Change-Id: I87327fa577666bcd2924432253d02cd67865e948
Should finish allocating network before VM reaches ACTIVE

This patch can fix issues like tempest caused by async
network allocation. Nova uses NetworkInfoAsyncWrapper
to allocate network in an async manner, and this allows
one to start querying for network info before you know
you will need it. Nova will invoke refresh_cache method
to update instance_cache_info after excuting the
allocate_for_instance method as well.

However, nova rest api will query instance_cache_info
directly, thus maybe get the empty network info before
instance_cache_info table has been updated asynchronously.
Thus will cause VM can exist without a known address
accidentally, the caller is difficult to distinguish
this situation, unless waiting for some internal process
or it will not have an address. So should finish
allocating network before the VM reaches ACTIVE status.
For more detail can refer the bug description.

BTW, almost all create server related unit tests have
coverred this case.

Closes-Bug: #1210483
Change-Id: Iab542df1be7d5c9661d717ec596d2a37b975169a
Add test cases for resume_state_on_host_boot

part of blueprint libvirt-driver-tests

Co-authored-by: Hirofumi Ichihara <ichihara.hirofumi@lab.ntt.co.jp>
Change-Id: I234c3613eb30a593fd1b0e29655551694b7e24f7
Fix changes_since for V3 API

The V3 API changed the changes-since parameter to changes_since
for consistency with other parameters. However, the db level
expects changes-since as the filter. So this change transforms
changes_since to changes-since before passing it to the compute
layer.

Closes-Bug: 1220045

Change-Id: Id34ffe8aa75b20d11dcb1b4a9ba36e2586dae1e3
Increase volume created checking retries to 60

In some cases 30 retries with 1 sec interval is not enough for
volume to be created.
Increased max tries to 60

Fixes bug 1213953

Change-Id: I6bcd395a4bac940387384cc85691d7219cf90e63
Revert "Importing correlation_id middleware from oslo-incubator"

This reverts commit 26ecf7b02377b860b4412aa554ed2392f42f8d40.

This patch was merged in preparation for
https://review.openstack.org/#/c/29480/ which was never merged.

Change-Id: Iefa1478af03f748ccb5f27f8bd4ad0f5b24af070
xenapi: Fix download_handler fallback

In order to make image-download a bit more robust, we attempt to download from
a preferred handler and if that fails, we fall back to a default handler.

If the preferred handler *is* the default-handler, then we should just raise
the exception instead of retrying.

The existing code was checking to see with whether the handler *instance*
compared as equal to the default-handler *instance*. Since the two objects are
instantiated separately, they'll always be different objects, so never compare
as equal.

The correct check is to see if the handler is an instance of the default
handler's class.

Fixes bug 1210659

Change-Id: Id038d221fd2f90c36673f30d4ff8acdc8dbf098e
Add V3 API samples for instance-actions,extenions

Partially implements blueprint v3-api-unittests

Change-Id: Ie90973734943b3d22cb288b18916a6ef2d9f51de
update .mailmap

Change-Id: Ibdad9607e4a046cb0c78b68ae4255679e17ff218
Add missing indexes back in from 152

Migration 152 lost indexes on upgrade. This was fixed in migration 194
and 212 but one index is still missing on the migrations table.

This patch ensures the indexes are present on downgrade of 152 to be
consistent and also adds the missing index back in with a new migration.

Fixes bug 1208652
bp: db-sync-models-with-migrations
Change-Id: I9f42b3fbe891187382135f8e789f13bb2d35d450
Fixes race cond between delete and confirm resize

If we delete an instance after a confirming resize operation,
and if the instance is deleted firstly at the dest compute node,
then the *_resize dir and network info of this instance will be
left alone on the source compute node, never be cleaned up.
Please refer the bug page to get the reproduce steps

Fixes bug 1202484

Change-Id: Ib1e3d976373bd7f4d086d4f9716b0bca4b383bab
Catch more accuracy exception for _lookup_by_name

When an instance is not found by libvirtd, _lookup_by_name in libvirt
driver raises InstanceNotFound. This patch catchs this exception instead
of NotFound who is the father of InstanceNotFound.

Closes-Bug: #1222663
Change-Id: I1b4ce8f6c44eef29617a42151152cda2159a0088
Add V3 API samples for flavors,flavor-rxtx,flavor-access

Partially implements blueprint v3-api-unittests

Change-Id: I2b572ec13190b946daebec4e64f9b9b362d5ac6e
Add V3 API samples for flavor-manage,flavor-extra-specs

Partially implements bp v3-api-extension-versioning

Change-Id: I54881677005cc085e5a2a604fc83eb4225ffd5d0
xenapi: ensure pool based migrate is live

The current code is missing a flag to pool_migrate that should be
included to ensure the migrate is live. This change adds that flag to
both uses of VM.pool_migrate, namely the live-migrate call and the
host_maintenance call.

Fixes bug 1201795
Change-Id: I265baaba5e9bd9132bd5e1c6f9fb18cefab02cfb
xenapi: move kernel/ramdisk methods to vm_utils

A previous change has now removed the duplication between spawn and
finish_migration, into create_kernel_and_ramdisk. This change moves the
code into vm_utils, and covers the code with some unit tests.

This will makes is easier to share code between spawn and finish_resize
and gives a more logical public interface to vm_utils.

I have also simplified the call to destroy_kernel_ramdisk in spawn so
that can be shared more easily between spawn and finish_migration in a
later change.

Change-Id: I73523d93be0a7afa9fe814af280c582e47064d69
xenapi: tidy ups to some spawn related methods

There were server methods using during spawn that are only used
internally, but not named that way. They have been renamed in this
change.

There were also several methods that are currently called in a single
place, but had logic around the call that should have been inside that
method. This has been tided up, so it will be easier to re-use and/or
move those calls in future refactoring.

This is one of the building blocks towards sharing more code between
spawn and finish_migration.

Change-Id: I424daf8334ad9904572b9df511d8e6bc1ff6d21f
xenapi: add tests for finish_migration and spawn

Before making finish_migration and spawn share more code, lock down the
current code with unit tests.

There are also some methods that are split up, to help with some of the
next refactoring steps.

Change-Id: Icfc214ea2c7470705f5605816b8313e0ba2a1429
xenapi: stop injecting the hostname during resize

The windows PV tools will set the hostname of a server to the value
given in the hostname xenstore key. If we inject that during resize, the
users will be unable to change their server hostname, and in addition it
will be reset to the original name.

Fixes bug 1207829
Change-Id: I49ba9d9f4f8b4a86488e427d99f69236bd959c78
xenapi: regroup spawn steps for better progress

Currently the operations are not very logically grouped. Now we can see
all the individual actions more clearly, we can re-arrange the
operations so each step make more sense, and should be better at
reporting progress.

Rather than checking very late that there is no duplicate VM and enough
memory, I have moved these checks before the expensive operation of
downloading the disk.

After this change there are 10 clear steps during spawn:
* initial step so progress is not zero during download disks
* download and create disks step
* download and create kernel and ramdisk step
* create the VM record, now the disks have been downloaded
* setup the disks; resizingi up disk, and attaching, adding rescue disk
* injecting instance data, currently this means writing to xenstore
* setting up the networking, including the initial firewall setup
* starting the VM and waiting for it to boot
* configuring the VM using the agent (takes a long time on windows)
* applying the final firewall filters, now the VM is running

With this in place, it will be easy to decide which steps are required
and which are not required when reusing the spawn code to implement
finish_migration in a later change.

Change-Id: I0e33dd7d298390630f632e116c8866a64e769102