libvirt: virt rescue method now supports objects

Commit a37a96fd13b933123e80ad24e37fc37d73c16059 ensured
that the compute API and manager are now getting objects
for the rescue method.

This patch adds support to the virt driver code.

Change-Id: I2ae89f937b1983a2b0599e2eded343aa8358dbba
libvirt: virt unrescue method now supports objects

Commit 65ad230af5ece020822fa963dd30fd641a54ae91 ensured
that the compute API and manager are now getting objects
for the unrescue method.

This patch adds support to the virt driver code.

Change-Id: I5cca3fa997042b164c21fe2a13c2311f4b18ec06
Fix cold-migrate missing retry info after scheduling

If the resize failed, nova-compute will try to rescheduling this resize
task. But the retry info is missed at resize compute task. This patch
add the retry info back.

Change-Id: I1b60704955e2fb218d047394dd20afe7f971c03d
Do not allow resize to zero disk flavor

For some reason nova allows creating a flavor with a disk_size of 0,
when an instance is booted using such flavor its disk will use the
native base image size as the size of the ephemeral root volume (from
the ops docs:
http://docs.openstack.org/openstack-ops/content/flavors.html).

Even though having the disk_size=0 means that the instance's disk can
have any imaginable size, nova will still allow to resize that instance
to any flavor defining disk_size >= 0 but it won't shrink the disk if it
is bigger than the new disk_size.

This patch add check to prevent resize to zero disk flavor except
when both current and new disk are 0.

DocImpact

Change-Id: Iffc2f821cc51ccf555ea1e3d7589147375aeb56a
Closes-bug: 1305860
Replace nova.utils.cpu_count() with processutils.get_worker_count()

cpu_count() was moved from nova to oslo-incubator via commit
85f178489a128a04a7ee3ed44018403caa109ef0 so it can be used in other
projects.

This change removes the duplicate code from nova and uses the renamed
method from oslo.

Change-Id: I03da77adf64e0495880343b7086f5907a2c5f75d
VMware: create utility function for reconfiguring a VM

A utility method has been created that simplifies the process of
reconfiguring a VM.

Change-Id: I184e2483726e29819164175493cfd1559592358b
Add assert_called check for "brclt addif" test

I92ee96e3aa34f7cd154a8ce4b33852b96cabe8c9 adds a negative test for
"brclt addif" but some mock arguments are not used/checked.
This patch adds assert_called_once_with check.

Change-Id: I8f3a0e4957343c8d6c903e1b893c58fda4916d91
vmware: remove unused delete_disk fake vim method

This is unused since the driver moved away from using DeleteVirtualDisk
in I786365847673e5192a21b654cba951b2e7a6f291

Change-Id: I126b4c069178720250f0dc7fcb924cd3c6c450e5
Fix error in rescue rpcapi that prevents sending objects

The change restores the rescue_instance rpc's ability to send
instance objects from version 3.11 onwards, that was inadvertently
broken by https://review.openstack.org/#/c/62314

Unit test modified to check that rpc version passes instance object
instead of dict.

Change-Id: Icffea82da623d3aab4661554b33bd5959bfe2bd8
Closes-Bug: #1327145
Fix run_instance() rpc method to pass instance object

Change I8a206f63cab1e4f939d2c97c258312a8fa540aff converted run_instance()
to use instance objects, but although it changed the rpc version to 3.27
it didn't update the rpcapi method, so at the moment it still passes a
dict.

Related to blueprint compute-manager-objects-juno

Change-Id: I0e511a5c4f1adce901f8812a935b98886c216dc9
Update compute rpcapi tests to use instance object instead of dict

Bugs such as 1327145 when the rpc code is accidentally reverted
to passing dicts instead of objects are not currently detected by
the unit tests because these are always called with a dict.

This change updates all compute rpcapi unit test cases to pass
an instance object to all cases where the version has been
converted to objects, so that such issues are now detected.

Related to blueprint compute-manager-objects-juno

Change-Id: I9b974e3a8e8e6dcbf87b1bdc854f05e94e7a35ce
Enforce task_state is None in ec2 create_image stop instance wait loop

We're hitting races in the gate where the instance.vm_state is STOPPED
but the task_state is POWERING_OFF so when the compute_api.start method
is called, we're in an invalid task state and fail.

The compute manager's stop_instance method is correctly setting the
vm_state to STOPPED and the task_state to None when the instance is
powered off via the virt driver, so we must be hitting this from races
in the ec2 API as noted in the TODO above the method definition.

This change simply checks the task_state in addition to the vm_state
in the wait loop before continuing. The error message is also updated
for context by including the instance uuid, vm_state and task_state,
and removes the timeout value in the message since it was in
milliseconds, not seconds, to begin with.

There is already a unit test that covers this change (which was racing,
hence the bug). There are no changes to that unit test since it's really
an integration test that's running through the compute API and compute
manager code, so the fix tests itself.

Closes-Bug: #1334345

Change-Id: I13f0c743cadda6439ae15607a9ef6e4e4985626d
libvirt: remove volume_driver_method API

Replace the 'volume_driver_method' API with _connect_volume
and _disconnect_volume methods to remove the pointless
introspection based calling technique. This adds a '_' to
the names to show that they are internal methods only.

Related-bug: #1333219
Change-Id: Ife3fb60dd8aef8adf6bbdd2465971e156430fa0d
libvirt: remove hack from ensure_filtering_rules_for_instance

The ensure_filtering_rules_for_instance method had a hack
where the test suite caller could pass in a module to
replace the greenthread.sleep call. This is utterly crazy
when mock can trivially replace the greenthread.sleep call
directly.

Partial-bug: #1333219
Change-Id: I449290aa366f5c85cc63a15d0b1704339be4d73a
Neutronv2 api does not support neutron without port quota

If quota on ports are disabled in neutron (see below), neutronv2 api
set port quota to None and quota check raises a TypeError.

  # neutron server /etc/neutron/neutron.conf
  [quotas]
  quota_items=network,subnet,router,floatingip

This change set port quota to -1 (unlimited) if quota on ports are
disabled.

Change-Id: I89c530dd57a7beb35695be124ca8511148378458
Closes-Bug: #1333315
VM diagnostics: introduce Diagnostics model object

There is no formal definition for the VM diagnostics. This patch
introduces a model object that will be used by the virt drivers
to return instance diagnostics.

The model will have a version. Each update to the model will require
a version change.

Part of the blueprint v3-diagnostics

Change-Id: I5527bc9904417489ec9bde844c9ff24cb02233fd
Fix wrong lock name for operating instance external events

This patch correct the wrong lock name for operating instance events.
Currently it use function object InstanceEvents._lock_name as the lock
name, the lock name will be like "<function _lock_name at 0x7fe6a7edec08>".
So when process the server external event, it will use one lock for all
the events that may belong to different instances. This isn't efficient.
This patch correct it to invoke InstanceEvents._lock_name to get the lock
name as '<instance-uuid>-events'. Then the lock is per instance.

Change-Id: I57712afe893933fde6ea7ab3f5d6328f04c65a8a
Closes-Bug: #1330431
Pass instance to _error_out_instance_on_exception

This is part of a series of patches to eventually get instance objects
passed to _instance_update so that we don't have to make a separate call
to update the instance via conductor when using an object already.  We
start with _error_out_instance_on_exception, then need to handle
_set_instance_error_state and then will change _instance_update.

Adds unit tests for the three different exception blocks handled
in the context.

Part of blueprint compute-manager-objects-juno

Change-Id: Ib3e64089a45cfab37e9c3570b94fed59b0d8f22c
Pass instance to _set_instance_error_state

This builds on commit 77a8478 so that _error_out_instance_on_exception
can pass the instance to _set_instance_error_state.  The next change in
the series will pass the instance down to _instance_update so if we have
an instance object we can avoid the direct call to conductor for state
changes and just update the instance object directly, which allows us to
remove instance.refresh() in the tests to pick up out-of-band database
updates on instance objects.

Part of blueprint compute-manager-objects-juno

Change-Id: I0a4607d368d3c226ca5a60d017d27c199c0dfe89
Merge "Neutronv2 api does not support neutron without port quota"