Add external_instance_event() method to compute manager

This would allow external services to send notifications in to
the compute manager, which could have one or more threads waiting
to be unblocked. It would also provide a way for us to process
generic events like "update your instance cache for $instance".

Related to blueprint admin-event-callback-api

Change-Id: If87cc7fdb1ebcfa2a1d33f5e864c43ed0290beb6
Add os-server-external-events API

Related to blueprint admin-event-callback-api

Change-Id: Ib09d5370622809e4bf1b5825eb9c223a115f7d33
Add os-server-external-events V3 API

Related to blueprint admin-event-callback-api

Change-Id: I93957840ac251cf6a22474897daf80a164ba35c6
Make compute manager's virtapi support waiting for events

This makes compute manager's virtapi expose a context manager
that lets virt drivers do something that doesn't complete until
an external event is triggered.

Related to blueprint admin-event-callback-api

Change-Id: I00b2821200f89a04f27963153805280c0063fd57
Make compute manager prune instance events on delete and migrate

This makes compute manager ensure that any unprocessed instance events
get removed from the list when an instance is deleted or migrated.

Related to blueprint admin-event-callback-api

Change-Id: I03816632a4710ac5e9b92707783dfeef096bfc24
Remove run-time dependency on fixtures module by the nova baremetal

Tripleo still uses this baremetal driver. The dependency on the python fixtures
was introduced by https://review.openstack.org/#/c/71219/

This dependency is causing devtest to stop working.

Change-Id: Id6deccce13698b083af492635ff88533c059fd35
Add add_members to InstanceGroup object

There is a db API method that safely adds a list of members to an
instance using a db session.  Expose it via the InstanceGroup object.

Part of blueprint instance-group-api-extension

Change-Id: I1b8349a82148bddf5cfbb7be3c8667f292df6602
Add instance to instance group in compute.api

When a user specifies an instance group via a scheduler hint, add the
instance to the instance group during instance creation in the compute
API.

Part of blueprint instance-group-api-extension

Change-Id: Ibad59412efecbd25c01e190c33da4bf28342e95b
Add get_hosts to InstanceGroup object

Add a method to the InstanceGroup object that will return a list of
hosts for all non-deleted instances in the group.

Testing this uses a nova.tests.utils method for creating an instance.
A tweak was needed in this test code to ensure that the user_id and
project_id were set on the instance that was created.  Otherwise, this
testing this new method on the InstanceGroup object failed since it
was filtering on the project_id.

Change-Id: Iaf27649ac30b4aa59a3d86f997aa30cfc3d29506
Initial scheduler support for instance_groups

Scheduler support for blueprint instance-group-api-extension

The scheduling code now uses the instance group object. This
patch set adds support for 'anti-affinity' and 'affinity'
policies.

Co-authored-by: Russell Bryant <rbryant@redhat.com>
Change-Id: I539424192727cf2a768d6b24ba9cf36dc56c9304
Fix anti-affinity race condition on boot

The scheduler enforces anti-affinity policy.  However, since
scheduling can happen in parallel for more than one instance, it's
possible for the scheduler to schedule an instance to a host that will
end up violating anti-affinity.

This patch updates the instance build code in the compute manager to
do an anti-affinity policy sanity check.  If the policy would be
violated by spawning the instance locally, an exception is raised so
that it gets rescheduled.

Part of blueprint instance-group-api-extension

Change-Id: I1358c36dff47b4ecd07841535aa0653d0f6ba823
Fix BDM legacy usage with objects

When we're doing an upgrade against Havana, our backleveled
compute RPC version will cause us to run block_device.legacy_mapping(),
which attempts to convert a BlockDeviceMapping object into an old-
style dict. This trips over a common problem with objects where
things expecting a list or a dict decide the object is a dict and
try to iterate it as such.

This makes the small tweak necessary and adds a test that verifies
that it works.

Change-Id: I98fe496720ad1c4fce44c0c7fea8bfa1c2721812
Make network_cache more robust with neutron

Currently, nova treats neutron as the source of truth for which ports are
attached to an instance which is a false assumption. Because of this
if someone creates a port in neutron with a device_id that matches one
of their existing instance_ids that port will eventually show up in
nova list (through the periodic heal task).

This problem usually manifests it's self when nova-compute
calls to neutron to create a port and the request times out (though
the port is actually created in neutron). When this occurs the instance
can be rescheduled on another compute node which it will call out to
neutron again to create a port. In this case two ports will show
up in the network_cache table (since they have the same instance_id) though
only one port is attached to the instance.

This patch addresses this issue by only adding ports to network_cache
if nova successfully allocated the port (or it was passed in). This
way these ghost ports are avoided. A follow up patch will come later
that garbage collects these ports.

Closes-bug: #1258620
Closes-bug: #1272195

Change-Id: I961c224d95291727c8614174de07805a0d0a9e46
Correct network_model tests and __eq__ operator

The current network_model tests mix IP() with FixedIP()
in several places. This patch corrects this issue. This was discovered
while adding logic to determine if the network_cache needs to be updated.
While updating the __eq__ operator to check all of the objects fields
rather than just id there tests started failing.

Closes-bug: # 1260891
Related to blueprint: smarter-network-cache-update

Change-Id: I6d144aecf95725146aa0fc4f4fe5a3f16c434d14
Store neutron port status in VIF model

This patch adds a field into the VIF model called status
which is True if a port in neutron has a status of 'ACTIVE' or
if a port is marked as admin_state_up=Fase (as the user has marked
the port to be down). Otherwise this value will be false. This is
needed for the admin-event-callback-api work. This allows nova-compute to
check the vif status to determine if it needs to wait on a vif-plugged
event before starting a instance.

Related to blueprint admin-event-callback-api
Change-Id: I25d03739afc1f3d74d2830e08ccb418e8cfd664e
Merge "Remove run-time dependency on fixtures module by the nova baremetal"
Merge "Add external_instance_event() method to compute manager"
Merge "Add os-server-external-events API"
Merge "Fix BDM legacy usage with objects"
Merge "Add os-server-external-events V3 API"