Optimize instance queries in compute manager

Some instance queries against the db through the conductor can be
optimized to return a reduced set of instances as needed for the job
at hand. Most of these are part of periodic tasks, so the win is
kind of big.

Compute methods where queries can be made more efficient:

_get_instances_on_driver
_poll_rebooting_instances
_poll_rescued_instances
_reclaim_queued_deletes
_run_image_cache_manager_pass

Resolves bug 1169970.

Change-Id: I7c2fab48944e34765b3fff8ce10bc64a5cd826c8
Sync service and threadgroup modules from oslo

blueprint use-oslo-services

Change-Id: I8641dbed0a83a6d57c0b6c1a02796eb3e633fafd
Convert Nova to use Oslo service infrastructure.

Convert Nova to use the Oslo versions of:
* Service
* Launchers

Also add Service timers to the Service ThreadGroup.

blueprint use-oslo-services

Change-Id: Id8ab017f4525afd69fed322311f2d5cc3b6d6f98
Improve unit tests for DB archiving.

Test all tables that have a uuid column and no constraints that make
generic testing difficult, rather than just a couple of tables.

Make the teardown code more generic.

Add test_archive_deleted_rows_2_tables, which tests that we can
archive rows from two tables in one call.

Change-Id: If08aac3c6869820c72dfcb1bf6d22d19c2995c8a
'm1.tiny' now has root_gb=1

`root_gb=0` indicates that a disk-resize shouldn't occur which is not
something that a default flavor should have enabled. This is
especially true of one named 'tiny' since the unresized image could
potentially be very large, leading to the unexpected result of a
'tiny' instance consuming a massive amount of disk.

The no-resize behavior is still supported, so if it's being relied on
for testing, additional, non-default flavors can still be created that
enable it.

DocImpact
Fixes bug 1175383

Change-Id: Ifa5827c7f87dae95214bc4a585adce5735a05d83
Use prettyxml output

minidom's toprettyxml doesn't render well so using etree instead

Change-Id: Iad05d55cfb071b11cb6f6b30aa9f23f33fd8039c
The vm_state should not be modified until the task is complete.

Several methods in nova.compute.api alter the vm_state value before
proceeding with the actual work. The initial vm_state may be changed to
something other than the initial vm_state (which is a problem: since we
do not know what the initial state actually was, we cannot reasonably
recover and set the corresponding ERROR if a problem with the operation
occurs).

Actually done:

Removed vm_state update before executing task

Changed methods:
- reboot
- pause
- unpause
- suspend
- resume
- rescue
- unrescue

Fixes: bug #1158509
Change-Id: I5ee7aebf3c4831dd577ed5ab9aa155095152f8a8
baremetal: drop 'prov_mac_address' column

bm_nodes.prov_mac_address is no longer used. So drop it.

Change-Id: I3c7294da8f62581aba8c948171cb02f80cf054ef
Editable default quota support

Implement blueprint edit-default-quota

DocImpact

Using the class quotas named `default` as the default editable quotas.

We can use the following novaclient command to update default quota:

nova quota-class-update default <key> <value>

Change-Id: I5a5001fadcbd61d550ebd5bdc33613b0ffdf29b2
Strip exec_dirs prefix from rootwrap filters

Make the rootwrap filters more portable by removing
the explicit exec_dirs prefix from them. This allows
a binary to be installed in any of the exec_dirs and
still be used by rootwrap, which improves cross-distro
portability.

Change-Id: Ieccb8c480e0e116a976d0920ab900305a5615015
Remove locals() from nova/cells/*

Remove use of locals() from nova/cells/* to satisfy pyflakes, etc.

Change-Id: If430226b84ce1fa974771ea1ed5b8d50480a70c0
Fix require_context() decorators.

require_context() and require_admin_context() decorators
don't copy attributes of a decorated function (func_name,
__doc__, __module__) to a wrapper function. This makes
introspection and debugging harder and should be fixed.

Fixes bug 1181606.

Change-Id: I0926df92a40d3fc09b94a0c4b1ded63039abf18b
Add memorycache import into the oslo config.

Resolves bug 1181633.

Change-Id: I86af10a2bb0f8afa5e2cf0d14e1839b872c21c90
Make nova-api use servicegroup.API.service_is_up().

Fix bug 1168861

Change-Id: I1e7e8f7519ec627b96736f74ee7123d101973544
Merge "Strip exec_dirs prefix from rootwrap filters"
Merge "Optimize instance queries in compute manager"
Merge "Remove locals() from nova/cells/*"
Merge "Improve unit tests for DB archiving."
Added attribute 'ip' to server search options.

Allows all tenants to use attribute 'ip' in server search options.
For this fix, added attribute 'ip' to server search options.
Modified the existing test case to verify 'ip' search for all tenants.

Implements: blueprint add-attribute-ip-in-server-search-options

Change-Id: Ic24ee67d729f77d56c4fe3e1d4110865550c2154
Added validation for networks parameter value

A validation check is added for networks parameter value.
In server create request body, if networks parameter value
is not stored in a list then 'Bad networks format' error
is thrown with bad request exception.A new test case is
added to verify whether it is returning bad request exception
when invalid syntax is used.

Fixes: bug #1175683

Change-Id: I457a1626c56281f89d8c4a606ccc0d0e3aea05fa