Copy the RHEL6 eventlet workaround from Oslo

Tests in oslo-incubator now need the horrendous hack to workaround an
issue with eventlet on RHEL6. We've moved the patch_tox_venv tool
and redhat-eventlet.patch into oslo-incubator, so add them to
openstack-common.conf.

Change-Id: I1acfec282b64d7582144b155119bedb1741db2f7
Import new additions to oslo's processutils.

trycmd() and ssh_execute() have been moved into oslo. Copy the
oslo version here. Conversion will occur in a later review.

Change-Id: I69926af33ff824f712ae2b8e4023233afe1dff2f
Convert to using newly imported processutils.

This change moves nova over to using the newly imported trycmd() and
ssh_execute() implementations from oslo's processutils.

Change-Id: Ied2bb1eeb0cbdc6903a4028a60c89ca9e23656b4
Fix format error in claims.

I have now seen two bug reports today about log message formatting
in the claims code. I'm not 100% sure what's going on here, but
I've removed the use of locals() in the hope it makes it clearer.
I've also moved to treating limits as floats because I think it
makes the messages clearer, but it is not a functional change.

Resolves bug 1175923.

Change-Id: I8b7b31f2637d1ab41d0fb2f274545181f9fddd9d
Catch volume create exception

Catch InvalidInput exception from cinder when the parameters
are invalid such as an invalid size. Currently the exception
is not caught which results in a 400 Bad Request correctly being
returned, but also leaving a traceback in the logs. Correctly
catching the exception and explicitly raising an HTTPBadRequest
removes the production of the traceback

Fixes bug #1098048

Change-Id: Ia66e67f302cf20e96417f4ff3c1430b755a4a861
Include list of attached volumes with instance info

Fixes bug 1112998

Change-Id: I1c3eb578339aabbcfed107043f39e30daf633c4a
Remove duplicate gettext.install() calls

When commit 799a925 moved us to console scripts entry points, we moved
the gettext.install() call into nova/cmd/__init__.py but forgot to
remove it from some of the commands.

Since that move, we've switched to using gettextutils.install() which
supports a $NOVA_LOCALEDIR environment variable ... so calling install()
from these commands will actually break that support.

Change-Id: I1f959704551b35e95c5470a7aef900312db2e223
Optimize instance queries in compute manager

Some instance queries against the db through the conductor can be
optimized to return a reduced set of instances as needed for the job
at hand. Most of these are part of periodic tasks, so the win is
kind of big.

Compute methods where queries can be made more efficient:

_get_instances_on_driver
_poll_rebooting_instances
_poll_rescued_instances
_reclaim_queued_deletes
_run_image_cache_manager_pass

Resolves bug 1169970.

Change-Id: I7c2fab48944e34765b3fff8ce10bc64a5cd826c8
Optimize db.instance_floating_address_get_all method

Optimize db.instance_floating_address_get_all
Was:
Get FixedIps with corresponding instance_uuid
For each FixedIp get associated FloatIp (N request)
Now:
Get only list of FixedIps ids
Get all corresponding FloatingIp with one query (select in)
Get only addresses from FloatingIps (because we are using only it)

Add temporary test for db.instance_floating_address_get_all to
ensure that it works as expected.

Change-Id: Id3c515de0ca35707bbfe46b991c2d7cf591e431b
Make sure that hypervisor nodename is set correctly in FakeDriver

This resolves an issue where the wrong host name is used when
executing some sample_api tests individually.

Resolves bug 1177802.

Change-Id: I29af026e29ff8c70bb6b4c81f1bdf28cdea11140
Raise InstanceInvalidState for double hard reboot

A hard reboot is not allowed if the instance task_state is
REBOOTING_HARD so check for that state and raise InstanceInvalidState.
This returns the correct status of 409 to the caller.

Previously the hard reboot would fail the the expected_task_state check
on db.instance_update_and_get_original which would log a traceback and
return a 500 to the caller.

Bug 1157237

Change-Id: Ifb6111790352da5c404289b218c50d8104384301
Pass None to image if booted from volume in live migration

The destination check in live migration will fetch the image
information from glance and it will throw ImageNotFound if
the instance is booted from volume since there is no image id
on the instance

Fix bug 1170596

Change-Id: Ie683a3ca5d6430c52ead77b41f98fbcb4114ea1e
Architecture property updated in snapshot libvirt

During an instance snapshot if the architecture property is defined
in the image it is updated twice in the snapshot image.
The snapshot is created with the metadata values from the
"instance_system_metadata" table and then, if defined, the architecure
property is updated from the value in the base image in that moment.

There is no reason to treat this property differently from others.
Also, it didn't respect the "non_inheritable_image_properties" option.

This change removes the special behavior for the architecture property
in the libvirt driver during a snapshot.

Fixes: bug #1176173
Change-Id: Ic5b4e3d462c08df167e14522fa25a7cf857c6aad
Improve message and logging for corrupt VHD footers

 * Handles the output of vhd-util check a bit more intelligently.
 * Changes it to use LOG like in other modules
 * Adds a necessary import (errno)

Change-Id: I75d6b9e081159bc29732a721deb7a557e5f2ee06
Adds useful debug logging to filter_scheduler

On a large system (>500 hosts) the amount of logging
information if everything is at DEBUG is vasts, due to
the number of hosts checked for each filter

However at the moment some useful information is only
available within the filter's debug entrys (such as
parts of the request_spec).   Also instance uuids are
not logged making it hard to search for instance entries.

This set of changes allows it to be used in a very
large install with debug turned off for everything
except nova.filters

Specifically:
 - Log.Info the instance_uuids at the start of schedule
 - Log.debug the request_spec at the start of schedule
 - Log.debug hosts in weighted order
 - Log.info which host has been allocated to a specific
   instance_uuid
 - Log.debug how many hosts are returned by each filter

To get a count from each filter nova.filters had to be
changed to generate a list after each filter rather than
building a recursive generator object.   Although this
new approach is less elegant it does provide simple and
useful insight into the behaviour of the filters, and s
the impact on execution time on a system with several
hundred hosts and 10 filters was negligible

Change-Id: Ibbdd14f87b1dfd3cee8bf3cf6388e40b474e530a
Extract getting instance's AZ into a helper module.

An instance's availability zone logic may used by many modules,
so move it from extend API into helper module `nova.availability_zones`.
Also add some tests for the new method.

part of bp: different-availability-zone-filter

Change-Id: I5916a0b09d5dddec338e8e36503953720dfd6bcd
Fix header issue for baremetal_deploy_helper.py

Fix header issue

baremetal_deploy_helper.py is not a python script which can not
be run directly, so there is no need to keep "#!/usr/bin/env python"
but change it to "# vim: tabstop=4 shiftwidth=4 softtabstop=4"

Change-Id: Ic4957673605cd71dc19fc38a2626bdaa35d102c0
Sync service and threadgroup modules from oslo

blueprint use-oslo-services

Change-Id: I8641dbed0a83a6d57c0b6c1a02796eb3e633fafd
Detach volume fails when using multipath iscsi

Enabled multipath for libvirt iscsi volume driver. Attach volume worked,
however, detach volume failed. It failed because it can't find multipath
device "/dev/mapper/xxxxxxxx" from connection_info and block device
mapping table.

The fix is to retrieve multipath device info dynamically at detach time.

Fixes: bug #1178893

Change-Id: I53a3888daef93b57bf7f57438f74cee05e794b52
Show the cause of virt driver error

Make sure that the virt driver loading errors are shown with the
stacktrace attached. Otherwise the source of an import error can be
completely hidden. This is important here, since virt driver can fail
due to various issues, like quantum vif problem.

Change-Id: Icb90d348e4a4ddef5768ca32836bd95d94981499