xenapi: move kernel/ramdisk methods to vm_utils

A previous change has now removed the duplication between spawn and
finish_migration, into create_kernel_and_ramdisk. This change moves the
code into vm_utils, and covers the code with some unit tests.

This will makes is easier to share code between spawn and finish_resize
and gives a more logical public interface to vm_utils.

I have also simplified the call to destroy_kernel_ramdisk in spawn so
that can be shared more easily between spawn and finish_migration in a
later change.

Change-Id: I73523d93be0a7afa9fe814af280c582e47064d69
xenapi: tidy ups to some spawn related methods

There were server methods using during spawn that are only used
internally, but not named that way. They have been renamed in this
change.

There were also several methods that are currently called in a single
place, but had logic around the call that should have been inside that
method. This has been tided up, so it will be easier to re-use and/or
move those calls in future refactoring.

This is one of the building blocks towards sharing more code between
spawn and finish_migration.

Change-Id: I424daf8334ad9904572b9df511d8e6bc1ff6d21f
Remove _assert_compute_node_has_enough_memory from filter_scheduler.py

Now the function _assert_compute_node_has_enough_memory has no use
in filter_scheduler.py, just remove it to make code more clean.

Change-Id: Ie6f104368140b69a60e124ea96ef6bf7715fe34e
Wrong arguments when calling safe_utils.getcallargs()

When calling safe_utils.getcallargs(), it will get all argnames from
definition of a function. As every function in Nova has a argument named
`context`, so argnames will always start with `context`.

In nova/exception.py#81, *args is a list of arguments which does not
start with
`context`, however, argnames starts with `context`, this will lead to a
mismatching when zip it into a dict.

Solution for this is that when calling getcallargs(), pass context as
one argument.

Change-Id: I4ce81e1862fb2d741f15747b04aa6bf1a2243364
Fixes: bug #1207153
Enable fake driver can live migration

Now two methods of virt/fake.py does not implemented
1) ensure_filtering_rules_for_instance
2) unfilter_instance

Also the method live_migration does not call
post_live_migration_at_destination, this will cause compute
driver has no chance to update task state for the VM in DB.

Fix bug 1223090

Change-Id: I2fe552faa5f73b47b97e8fefc5c4de04d6785f8c
pci passthrough bug fix:hasattr dones not work for dict

hasattr only work in object model, scheduler does not use the
object model yet. so fall back to dict model.

Closes-Bug: #1226348

Change-Id: Ifab4165be67abfe2bbe9ac7280b8715cb7f04f1c
Signed-off-by: Yongli He <yongli.he@intel.com>
Retry on deadlock in instance_metadata_delete

Add the retry_on_deadlock decorator to the instance_metadata_delete
function in the sqlalchemy db API.  A tempest test failure shows that it
would benefit from it.

Change-Id: I39547fbbbabc310b78493df0dc055bc320995770
Closes-bug: #1226076
Prune node stats at compute node delete time

This commit addresses the situation in which a compute node is deleted, but
its compute node stats still remain in the database as **not** deleted. This
causes ill side effects in compute_node_get_all when it's retrieving host stats
as it doesn't expect there to be compute node stats for which there is no
corresponding compute node (i.e., causing some nodes' stats to be empty).

As such, when a compute node is deleted, its stats should also be implicitly
deleted.

The new test case that's been created fails without the code changes, which
illustrates the problem that compute node stats are empty when they should
not be.

Also included is a simple DB migration script that will update old stats that
were not marked soft-deleted as they should have been.

Change-Id: Ief0f7cf1a506e71898b5a45a0513d34167432d67
Closes-Bug: #1224712
MultiprocessWSGITest wait for workers to die bug

MultiprocessWSGITest.test_terminate_sigterm
fails because the wait for workers to die is unreliable.

Parent process can now wait on child or grand-child
processes to complete before moving on. This allows
for grand-child (aka worker) processes to take as much
time as they need to shutdown. NOTE: If the processes
still take too long, they will be killed by the test
framework throwing the "TIMEOUT" exception.

Reverts to using a ping signal to child process
to determine if the child process is live. Waits for
the signal call to fail, signaling process death.

closes bug #1218133

Change-Id: I08e17005508fe412f462773f9ab212cf0f10382f
Convert TestCases to NoDBTestCase

NoDBTestCase is faster because it skips initializing a database for
tests which don't actually talk to the database. Its also nicer
because its more explicit about the dependancies of a given test.
This is a first pass at moving test cases over to the database
lacking version, but further work is required here.

Change-Id: Ia28de4ebea4f354cba06bcdc66bb01d4b332943a
Partial-Bug: 1227473
Typo: certicates=>certificates in nova.conf

The help message for cinder_ca_certificates_file and
neutron_ca_certificates_file spelled certificates wrong.

As this is user facing, we should fix it.

Fixes bug 1227780

Change-Id: If9372fffb655d21069c005351ebb593d8e16e056
Merge "Prune node stats at compute node delete time"
Send notification for any updates to instance objects

Before objects, a notification would be sent for any change to an
instance and the notifications.send_update method would handle any
filtering that needed to be done prior to sending the event.

Currently, the instance objects code is pre-filtering the notification
by only checking if vm_state or task_state changed which is a
regression.

This patch removes the filter in the instance objects code so that
notifications are again sent for any changes to the instance and lets
the notifications.send_update method handle the event as it did before.

Closes-Bug: #1220186

Change-Id: Ib6ceedfe55b021be0fe7fa910e97d46c0cd8f0cd
Merge "Retry on deadlock in instance_metadata_delete"
Merge "Send notification for any updates to instance objects"
Merge "Wrong arguments when calling safe_utils.getcallargs()"
Merge "Enable fake driver can live migration"
Merge "MultiprocessWSGITest wait for workers to die bug"
Merge "Typo: certicates=>certificates in nova.conf"
Merge "Convert TestCases to NoDBTestCase"