Hypervisor uptime fails if service is disabled

Getting the uptime of a hypervisor fails because the db query
used to get the service filters out disabled hosts.

Instead we replace that query with one that always gets the
service if it exists, and a check that the sevice is up to
avoid an rpc call to a non running service.

Fixes bug 1195052

Change-Id: I97bc8a5002bd0f333207403d5b1878649612b762
Reassign MAC address for vm  when resize_revert

In the case of a resize or migrate on the same host, we need to assign
a temp mac address to the source LPAR so we don't have a conflict when
another LPAR is booted with the same mac address as the original LPAR.
We need reassign the original mac address, replacing the temp mac on
the old instance during revert resize. The new instance has been
destroyed before this method, so remove the precondition to make sure
original mac address be set.

Fixes bug #1196404

Change-Id: I709292a7725f0a21f04f1c3c79f2fafc4437e402
Check system_metadata type in _populate_instance_for_create

The _populate_instance_for_create method is passed a dictionary
of base instance values during create but it wasn't checking to
see if system_metadata was a dict or not which can lead to a
TypeError when booting an instance via the cells scheduler.

Fixes bug 1195549

Change-Id: I60b853eda48726b3bb752b7e3542b0fca74bc926
Fix compatibility with older confirm_resize() calls

Commit ff4194bec79067bf77ab25caec1e40ad3709ea07 bumped the RPC
version and format for compute_confirm_resize() without providing
compatibility with older installations. This patch converts it
to a instance-dict style of primitive before sending it, if we're
pinned to an older version.

Change-Id: Ic631b70c005bfd0efe6965ad024890cc2caf947c
Remove notify_on_any_change option

This option isn't really needed, since the case it defends is already
covered by notify_on_state_change. Let's simplify the configuration a
bit by removing it.

DocImpact

Change-Id: I35071abdea2d2380d79c07e9134e095bda65a13d
Signed-off-by: Julien Danjou <julien@danjou.info>
Fix power_state lookup in confirm_resize

Previously the compute manager's confirm_resize method was trying to get
the current power_state of the resized/migrated instance from the
hypervisor layer to account for a case where the user migrated from a
stopped instance and then powered it on via the hypervisor (below
openstack) to test it before confirming the migration.  This doesn't
work in the migrate case because the confirm_resize request is serviced
on the source compute node, and the instance no longer lives there so
_get_power_state results in an InstanceNotFound.  The vm_state would
then be set to ACTIVE even though the instance's power_state is still
SHUTDOWN.

This patch fixes this problem by just getting the instance power_state
from the database since the scenario of powering on the migrated
instance outside of openstack is not supported.

Fixes bug 1197514

Change-Id: If2ea83cd317aa8897b7b6bd6b43a2067e1074e33
Exceptions raised by quantum validate_networks result in 500 error

The exceptions raised by validate_networks should be subclassed
from Invalid so they result in a 400 error

These exceptions are also used in allocate_for_instance

Fixes bug 1193421

Change-Id: If708e413d81a9d32bf81e7821d876736c4ac40ef
Fix debug message for GroupAntiAffinityFilter

Fix bug 1198109

The log message in not accurate for GroupAntiAffinityFilter, it
should be fixed as "Group anti affinity"

Change-Id: Ib58c2934626c28349dd49bafd1b3e781fcbe9cb9
Update instance.node on evacuate

The evacuate API works like a migration from a dead node. As part of
migration, the instance.node property is updated to reflect the target
node that the instance is migrating to. This is later used when the
migration is confirmed/reverted. The problem is when an instance is
being evacuated, the node property isn't getting updated and the
instance will have the old (dead) node value set on it after it's
evacuated. This patch updates instance.node when the instance is
evacuated.

Fixes bug 1198284

Change-Id: I4901fb937160e955df7c97c37265783e844fdbef
Remove references to deprecated DnsMasqFilter

In https://review.openstack.org/#/c/16803/ compatibility
for FLAGFILE has been added. It is time to remove this
again for Havana.

Change-Id: I1304132320f7d7d381b9e6548337afed52f1e96a
Allow filters to only run once per request if their data is static

Currently the filter_all() method of each scheduler filter is run
once for each instance in a request, but for many filters the data
doesn't change during a request.

For example the data used by the AZ filter is pretty static, and
doesn't filter any more hosts on successive runs.  However it is
fairly expensive to run on a large system.

Similarly the ServiceGroup information used by the compute filter
is cached as part of the host status, and doesn't need to be
evaluated more than once per request.

Only filters that are based on data that may change within a
request, for example resource consumption, need to be evaluated
for each instance.

This blueprint introduces a new attribute that allows filters to
declare that they only need to be run once per request. The default
behaviour is left so that a filter is evaluated once for each
instance in a request.

The following existing filters are changed to only evaluate once
per request:

  AggregateInstanceExtraSpecsFilter
  AggregateMultiTenancyIsolation
  AggregateTypeAffinityFilter
  AllHostsFilter
  AvailabilityZoneFilter
  ComputeCapabilitiesFilter
  ComputeFilter
  DifferentHostFilter
  GroupAntiAffinityFilter
  ImagePropertiesFilter
  IsolatedHostsFilter
  SameHostFilter
  SimpleCIDRAffinityFilter

In addition the function that does the check is defined in the
filter base class in a way that a filter that wants to run
(for example) once for every 10 instances in a request could
overload the function and implement its own behaviour.

Implements: bp once-per-request-filters
Fixes: bug 1189557

Change-Id: Ia668f16414da86441323c58b1bbef5f88c81b90c
Add a VIF driver for IOVisor engine

Implements blueprint iovisor-vif-driver

Unit Test is included
Follows the same structure that OVS and Qbh drivers
Includes the rootwrap config to allow calls to ifc-ctl for managing IOVisor engine.

Change-Id: I346b4d84a6e48d56f307349ed681355d26710ecd
Sync harmless changes from oslo-incubator

Includes these commits:

  3948aee Highlighting the deprecated nature of 'log-format'.
  b21fc56 Fix bad default for show_deleted
  d28fa69 python3: Add python3 compatibility.
  0bf03b7 Add network_utils.urlsplit
  e456727 Remove useless logging in networks_utils
  7119e29 Enable hacking H404 test.
  b41862d Use param keyword for docstrings
  2f01388 Use Python 3.x compatible except construct
  e3545f8 Enable hacking H402 test
  484a1df Enable hacking H403 test
  35660da Enable hacking H401 test
  874249e Add support for requirements.txt.
  b135234 Remove the notifier and its dependencies from log.py
  926b3e9 Fixes import order nits

Changes which may look not look so harmless:

 - the common get_admin_context() method is not used by Nova
 - network_utils.urlsplit() is added but not used yet
 - suppport for requirements.txt is added to patch_tox_venv but
   actually it doesn't use those files

Change-Id: I03e67f4648dcaf57620f11e63cfd8e7dbe3665cb
Sync notifier change from oslo-incubator

Syncs this commit:

 54fcea2 notifier: do not rely on CONF.host

Change-Id: I74a8f9755c48d2c74026afe75413112c653f3ae4
Replace get_instance_metadata call in api.ec2.cloud._format_instances

The _format_instances method in api.ec2.cloud populates an instance
dictionary from the sqlalchemy instance model objects, and populates the
metadata by calling compute.api.get_instance_metadata.

This is unnecessary, since the instance objects already have metadata
attached. The helper class is needed until the migration to nova objects
is complete.

Partially fixes bug #1192715

Change-Id: Ib5a593bd36c8d6a531b9494b9861cca9a40ed08a
Validate volume_size in block_device_mapping

Reject invalid values for volume_size upfront.
For compatibility with old python-novaclient, an
empty volume_size is ignored.

Create a common _validate_int_value() function for
validating that a parameter is an integer in a given
range. Use it also for min_count/max_count to share code.

Fixes LP Bug#1199539

Change-Id: I18ac285d5b3c8200d7706c7d577809b15d6ab9e8
Sync db.models.Instance* with migrations.

This patch syncs models with migrations for:
    - InstanceIdMapping
    - InstanceMetadata
    - InstanceSystemMetadata
    - InstanceTypeExtraSpecs
    - InstanceTypeProjects
    - InstanceTypes
    - InstanceGroupMember
    - InstanceGroupPolicy
    - InstanceGroupMetadata

Blueprint: db-sync-models-with-migrations

Change-Id: Ice28b9d545f60f7df381e3fcf2c86a2b8e7350d3
Sync db.models.Security* and db.models.Volume*.

This patch syncs models with migrations.

Change-Id: I2e4a4cf28376d933be47bc496933e6964e19a56e
Clarify flavorid vs instance_type_id in db

And add comment about future direction.

Partially implements bp flavor-instance-type-dedup

Change-Id: Ia6c1bad5c3cc35e772c32803c27a50a7728045e5
Fix duplicate fping_path config option

This is causing an error in the config file tool generator. Just use
the config option in the original copy of this code.

Change-Id: I5e7c3069913eea11633f35273183c3c3b46f4969