Add instance to instance group in compute.api

When a user specifies an instance group via a scheduler hint, add the
instance to the instance group during instance creation in the compute
API.

Part of blueprint instance-group-api-extension

Change-Id: Ibad59412efecbd25c01e190c33da4bf28342e95b
Add get_hosts to InstanceGroup object

Add a method to the InstanceGroup object that will return a list of
hosts for all non-deleted instances in the group.

Testing this uses a nova.tests.utils method for creating an instance.
A tweak was needed in this test code to ensure that the user_id and
project_id were set on the instance that was created.  Otherwise, this
testing this new method on the InstanceGroup object failed since it
was filtering on the project_id.

Change-Id: Iaf27649ac30b4aa59a3d86f997aa30cfc3d29506
Initial scheduler support for instance_groups

Scheduler support for blueprint instance-group-api-extension

The scheduling code now uses the instance group object. This
patch set adds support for 'anti-affinity' and 'affinity'
policies.

Co-authored-by: Russell Bryant <rbryant@redhat.com>
Change-Id: I539424192727cf2a768d6b24ba9cf36dc56c9304
Fix anti-affinity race condition on boot

The scheduler enforces anti-affinity policy.  However, since
scheduling can happen in parallel for more than one instance, it's
possible for the scheduler to schedule an instance to a host that will
end up violating anti-affinity.

This patch updates the instance build code in the compute manager to
do an anti-affinity policy sanity check.  If the policy would be
violated by spawning the instance locally, an exception is raised so
that it gets rescheduled.

Part of blueprint instance-group-api-extension

Change-Id: I1358c36dff47b4ecd07841535aa0653d0f6ba823
Fix BDM legacy usage with objects

When we're doing an upgrade against Havana, our backleveled
compute RPC version will cause us to run block_device.legacy_mapping(),
which attempts to convert a BlockDeviceMapping object into an old-
style dict. This trips over a common problem with objects where
things expecting a list or a dict decide the object is a dict and
try to iterate it as such.

This makes the small tweak necessary and adds a test that verifies
that it works.

Change-Id: I98fe496720ad1c4fce44c0c7fea8bfa1c2721812
Error out failed migrations

This change causes migrations to have their status set to error when
either the resize_instance or finish_resize methods raise an exception.
This prevents continuous logging of stalled migrations.

Change-Id: Ie752e4833d28fd679c6d1abbc9da5f0ef57f5ec4
Closes-bug: 1258275
Make network_cache more robust with neutron

Currently, nova treats neutron as the source of truth for which ports are
attached to an instance which is a false assumption. Because of this
if someone creates a port in neutron with a device_id that matches one
of their existing instance_ids that port will eventually show up in
nova list (through the periodic heal task).

This problem usually manifests it's self when nova-compute
calls to neutron to create a port and the request times out (though
the port is actually created in neutron). When this occurs the instance
can be rescheduled on another compute node which it will call out to
neutron again to create a port. In this case two ports will show
up in the network_cache table (since they have the same instance_id) though
only one port is attached to the instance.

This patch addresses this issue by only adding ports to network_cache
if nova successfully allocated the port (or it was passed in). This
way these ghost ports are avoided. A follow up patch will come later
that garbage collects these ports.

Closes-bug: #1258620
Closes-bug: #1272195

Change-Id: I961c224d95291727c8614174de07805a0d0a9e46
Correct network_model tests and __eq__ operator

The current network_model tests mix IP() with FixedIP()
in several places. This patch corrects this issue. This was discovered
while adding logic to determine if the network_cache needs to be updated.
While updating the __eq__ operator to check all of the objects fields
rather than just id there tests started failing.

Closes-bug: # 1260891
Related to blueprint: smarter-network-cache-update

Change-Id: I6d144aecf95725146aa0fc4f4fe5a3f16c434d14
Store neutron port status in VIF model

This patch adds a field into the VIF model called status
which is True if a port in neutron has a status of 'ACTIVE' or
if a port is marked as admin_state_up=Fase (as the user has marked
the port to be down). Otherwise this value will be false. This is
needed for the admin-event-callback-api work. This allows nova-compute to
check the vif status to determine if it needs to wait on a vif-plugged
event before starting a instance.

Related to blueprint admin-event-callback-api
Change-Id: I25d03739afc1f3d74d2830e08ccb418e8cfd664e
Merge "Remove run-time dependency on fixtures module by the nova baremetal"
Merge "Add external_instance_event() method to compute manager"
Merge "Add os-server-external-events API"
Merge "Fix BDM legacy usage with objects"
VMware: boot from iso support

The code enables the VMware driver to boot from an ISO
image.

Implements blueprint vmware-iso-boot

Change-Id: I568df99d6530f898d5a1885256604cf050b33dba
Merge "Add os-server-external-events V3 API"
Merge "Make compute manager's virtapi support waiting for events"
Merge "Make compute manager prune instance events on delete and migrate"
Merge "Make network_cache more robust with neutron"
Merge "Correct network_model tests and __eq__ operator"
Merge "Store neutron port status in VIF model"