Copy the RHEL6 eventlet workaround from Oslo

Tests in oslo-incubator now need the horrendous hack to workaround an
issue with eventlet on RHEL6. We've moved the patch_tox_venv tool
and redhat-eventlet.patch into oslo-incubator, so add them to
openstack-common.conf.

Change-Id: I1acfec282b64d7582144b155119bedb1741db2f7
Add cpuset attr to vcpu conf in libvirt xml

Currently the instances can use all of the pcpu of compute node, the
host may become slow when vcpus of instances are busy, so we need to
pin vcpus to the specific pcpus of host instead of all pcpus.
Also added a cache of total vcpus num in libvirt driver here.

DocImpact: please refer to the bp page.
Implements blueprint: instance-vcpu-pin
Change-Id: I6b6e805c15b249cc5e7517e26619d79bb0183de8
Enable live block migration when using iSCSI volumes

Fix for bug 1160323.

DocImpact
Depends on a version of XAPI supporting relax-xsm-sr-check.
Currently no release versions support this, so the change at
https://github.com/xen-org/xen-api/pull/1116 would need to be applied
to the source to enable this.  XenServer 6.2 is expected to
support this mode, and possibly some future hotfixes for XenServer 6.1.

It also depends on a configuration change which must be documented and added to devstack:
* Set "relax-xsm-sr-check = true" in /etc/xapi.conf

This commit makes the following changes:
* Attach the SR on the destination host prior to migrate
* Returns the SR-ref from the pre_migrate call (API change)
* Populates the XS VDI map with the sr-ref on the destination host
* Removes the connection to the SR from the source host post-migrate
* Added plugin to determine if iSCSI block migration is enabled

Associated tempest test at https://review.openstack.org/#/c/26478/

Change-Id: I917d9cf094190d636f4b9e14f6c8e728ff85af0e
Fixes: bug 1160323
Import new additions to oslo's processutils.

trycmd() and ssh_execute() have been moved into oslo. Copy the
oslo version here. Conversion will occur in a later review.

Change-Id: I69926af33ff824f712ae2b8e4023233afe1dff2f
Convert to using newly imported processutils.

This change moves nova over to using the newly imported trycmd() and
ssh_execute() implementations from oslo's processutils.

Change-Id: Ied2bb1eeb0cbdc6903a4028a60c89ca9e23656b4
Adds instance root disk size checks during resize

Fixes bug: #1163844

The Hyper-V driver does not support disk resizes to a smaller size.
This patch verifies the disk size compatibility before the migration
starts in order to avoid data losses.

Change-Id: Ie99bf8779d583e97b911c9a136cee1bca9a1ecdc
Fix format error in claims.

I have now seen two bug reports today about log message formatting
in the claims code. I'm not 100% sure what's going on here, but
I've removed the use of locals() in the hope it makes it clearer.
I've also moved to treating limits as floats because I think it
makes the messages clearer, but it is not a functional change.

Resolves bug 1175923.

Change-Id: I8b7b31f2637d1ab41d0fb2f274545181f9fddd9d
Add unitests for VMware vif, and fix code logical error.

Fix code logical error that cause variable referenced before assignment.
Add unittests for vmwareapi vif.

Fix bug #1177635

Change-Id: I6656e8b77417061eca2a56d50820f5385c0ae987
Fixes KeyError bug with network api associate

Fixes bug in network api associate function where it would always
raise a KeyError when associating or disassociating a project.

Fixes bug 1171284

Change-Id: Iae0e57e0a961b8a3377b38dad72094188755a3e8
Catch volume create exception

Catch InvalidInput exception from cinder when the parameters
are invalid such as an invalid size. Currently the exception
is not caught which results in a 400 Bad Request correctly being
returned, but also leaving a traceback in the logs. Correctly
catching the exception and explicitly raising an HTTPBadRequest
removes the production of the traceback

Fixes bug #1098048

Change-Id: Ia66e67f302cf20e96417f4ff3c1430b755a4a861
Include list of attached volumes with instance info

Fixes bug 1112998

Change-Id: I1c3eb578339aabbcfed107043f39e30daf633c4a
Remove duplicate gettext.install() calls

When commit 799a925 moved us to console scripts entry points, we moved
the gettext.install() call into nova/cmd/__init__.py but forgot to
remove it from some of the commands.

Since that move, we've switched to using gettextutils.install() which
supports a $NOVA_LOCALEDIR environment variable ... so calling install()
from these commands will actually break that support.

Change-Id: I1f959704551b35e95c5470a7aef900312db2e223
Optimize instance queries in compute manager

Some instance queries against the db through the conductor can be
optimized to return a reduced set of instances as needed for the job
at hand. Most of these are part of periodic tasks, so the win is
kind of big.

Compute methods where queries can be made more efficient:

_get_instances_on_driver
_poll_rebooting_instances
_poll_rescued_instances
_reclaim_queued_deletes
_run_image_cache_manager_pass

Resolves bug 1169970.

Change-Id: I7c2fab48944e34765b3fff8ce10bc64a5cd826c8
Optimize db.instance_floating_address_get_all method

Optimize db.instance_floating_address_get_all
Was:
Get FixedIps with corresponding instance_uuid
For each FixedIp get associated FloatIp (N request)
Now:
Get only list of FixedIps ids
Get all corresponding FloatingIp with one query (select in)
Get only addresses from FloatingIps (because we are using only it)

Add temporary test for db.instance_floating_address_get_all to
ensure that it works as expected.

Change-Id: Id3c515de0ca35707bbfe46b991c2d7cf591e431b
Make sure that hypervisor nodename is set correctly in FakeDriver

This resolves an issue where the wrong host name is used when
executing some sample_api tests individually.

Resolves bug 1177802.

Change-Id: I29af026e29ff8c70bb6b4c81f1bdf28cdea11140
Fix zookeeper import and tests

Import zookeeper.membership rather than zookeeper.membersip.
Also fixed some issues with setting up the tests for the zookeeper
servicegroup driver.  Config options were not being set before
initializing the driver leading to failures.

There is no added test for this because the bug is indistinguishable
from not having the zookeeper python modules installed, which leads to
skipping these servicegroup tests.

Bug 1177776

Change-Id: Idd6dca2e03169399b930cc1fc1a85636497cb0b5
Raise InstanceInvalidState for double hard reboot

A hard reboot is not allowed if the instance task_state is
REBOOTING_HARD so check for that state and raise InstanceInvalidState.
This returns the correct status of 409 to the caller.

Previously the hard reboot would fail the the expected_task_state check
on db.instance_update_and_get_original which would log a traceback and
return a 500 to the caller.

Bug 1157237

Change-Id: Ifb6111790352da5c404289b218c50d8104384301
Pass None to image if booted from volume in live migration

The destination check in live migration will fetch the image
information from glance and it will throw ImageNotFound if
the instance is booted from volume since there is no image id
on the instance

Fix bug 1170596

Change-Id: Ie683a3ca5d6430c52ead77b41f98fbcb4114ea1e
Add sqlalchemy migration utils.check_shadow_table method

This method check that our table with name `xxx`:
1) Has shadow table `shadow_xxx`
2) All columns in `xxx` are also in `shadow_xxx` (with same type)
3) There is no extra columns in `shadow_xxx`

Will be used to check that all shadow tables are up-to-date.

blueprint db-improve-archiving

Change-Id: I0b366afe1b7d8703ab9c49a68977202b82d844fc
Add sqlalchemy migration utils.create_shadow_table method

This method allows to create shadow table by existing table.

Two use cases:
a) create shadow table by table name
b) create shadow table from table instance

blueprint db-improve-archiving

Change-Id: Ib3a3797782b89a9a89baa60738edcabde7fe1299