Introduce ImagePropertiesFilter scheduler filter

Commit d39137fa added functionality to the ComputeFilter to filter on
architecture, hypervisor_type, and vm_mode. The existing ArchFilter
already filtered on architecture. This patch merges the ComputeFilter
functionality introduced in d39137fa with the ArchFilter to create the
ImagePropertiesFilter.  The ImagePropertiesFilter uses image properties
specified in the request_spec to filter hosts.

This patch also adds the ImagePropertiesFilter to the list of default
filters specified by the scheduler_default_filters FLAG.

Fixes LP Bug #1037339

DocImpact

Change-Id: Ifa6fccf2db266b0fe3457d58fc81fdb50ffcbc0f
Keep the ComputeNode model updated with usage

Keep the compute host's ComputeNode model in sync with the
level of resource usage.  This enables the ComputeNode
model to be used as a basis for scheduling decisions
rather than forcing scheduler to calculate free
resources from an instance_get_all on each request.

Resources like memory and disk are claimed as instances are built
or deleted.  There is also support for configurable compute node
stats (a generic key/value store) for extensible advertising of other
usage stats that may be useful for a particular scheduler
implementation.  Additionally, there is a periodic task on the
compute host that audits actual resource consumption at the virt
layer to ensure that the database stays in sync.

This change partially implements blueprint: scheduler-resource-race

This patch complements:
https://review.openstack.org/#/c/9540/  (build re-scheduling support)

Change-Id: Ibbe3839a054f8b80664b413d47f766ca8d68e3f2
xenapi: Tag nova volumes during attach_volume

1. Set other_config:'osvol':'True' on VBD during attach.
2. Use this (instead of nodestroys list) to determine which volumes
should not be destroyed on instance terminate.
3. Ensure that the SRs for the attached volumes are forgotten when
instance is terminated.

In the virt/xenapi layer, there is no way to determine which of the
VDIs are nova volumes. This information is required when terminating
and instance, to ensure the volumes are kept intact, as well as other
operations like migrate.

Fixes bug 1030143.

Change-Id: Id1717e64bc29092ce9ffe13b7c254a3867fe8342
Debugged extra_specs_ops.py

The current extra_specs_ops.py under nova/scheduler/filters has a bug:
it crahses if <or> is not followed by a word. For example, if
"<or> gpu <or>" is given in extra_specs, current code tries to pop a
word after the second <or> that causes a crash. This patch lets it
ignore the "<or>" keyword if no word follows the second "<or>" keyword.

Debugged bug 1038979

Change-Id: I80a6eafd7ee57a2f3ccbf2fb869655ab84e92cf2
Code clean up

Some code clean up in the file doc/source/conf.py
to make the code more pretty.

Change-Id: Icb25428739725c530977a011bf28f17fda1c29fb
Implements sending notification on metadata change

It would be good to have a notification sent every time there is change
in the metadata. This merge prop implements sending a
compute.instance.update event for the above change.

fixes bug 1034547

Change-Id: I451796b6a83c4782df6f72f083755527ebd62a52
Remove stub_network flag

Fixes bug #816025

remove stub_network flag and replace usage with stubbing out of relevant
calls in tests.
In nova/tests/fake_network.py , I add set_stub_network_methods and
unset_stub_network_methods. They can be userd for tests.

Change-Id: Ie8fa95d9ab62cced00f0917cffe75ebc6c94bb3e
Merge "Keep the ComputeNode model updated with usage"
Merge "Remove stub_network flag"
Have compute_node_get() join 'service'

Fixes bug 1039633

compute_node_get() is used by hypervisors extension and it accesses
compute_node['service'].  Lazy joins fail now and then..

Also fixes line continuation indents with other compute_node calls in
sqlalchemy/api.py

Change-Id: I3ac02aad739e96a4866a64cabe859cf484341784
Add deprecated warning to SimpleScheduler

Fixes bug 1037829

Change-Id: I83abd3b58bc7d253616ad355458214859bb4112d
Make glance image service check base exception classes

Fixes bug 1039675

glanceclient can raise HTTPNotFound (as well as NotFound) it appears,
but glance image service is only converting NotFound ->
InstanceNotFound.  Same applies to 'Forbidden' and other exceptions.

This patch converts 'exc_type is NotFound'-like checks to use
'isinstance' instead, which will cover HTTPNotFound, etc.

Change-Id: I0982875e667121cee3da3cfe0124499cad484fa6
Merge "Implements sending notification on metadata change"
Merge "Make glance image service check base exception classes"
Cleanup pip dependencies

Fix bug 1039745

* move feedparser to tools/test-requires
* remove python-daemon

Change-Id: I8da6ee6854e2064b44f3161b84f28d0bfc7f438d
Remove unused exceptions from nova/exception.py

Removed exceptions:

ImagePaginationFailed
MelangeConnectionFailed
InvalidSignature
InvalidInstanceType
InstanceNotSuspended
DestinationHostUnavailable
SourceHostUnavailable
VolumeNotFoundForInstance
NoVolumeTypesFound
ListingImageRefsNotSupported
KernelNotFoundForImage
ProjectMembershipNotFound
UserRoleNotFound
NetworkBusy
NetworkIsDuplicated
FixedIpNotFoundForHost
AuthTokenNotFound
AccessKeyNotFound
NoInstanceTypesFound
LDAPUserNotFound
LDAPGroupNotFound
LDAPGroupMembershipNotFound
GlobalRoleNotAllowed
LDAPUserExists
LDAPGroupExists
LDAPMembershipExists
ProjectExists
ZoneRequestError
SolidFireAPIStatusException
LDAPObjectNotFound
UserExists
UserNotFound

Change-Id: I8a3caaebdeed82f19733893e62f57648bdfc45bd
Merge "Cleanup pip dependencies"
Merge "Have compute_node_get() join 'service'"
Merge "Add deprecated warning to SimpleScheduler"
Merge "Debugged extra_specs_ops.py"