Reset migrating task state for MigrationError exceptions

Create an exception type for pre-check errors and add it to the list
of exceptions for which instance task state is reset to ACTIVE to
avoid cases where an instance get stuck in MIGRATING state.

Convert existing instances of raise MigrationError in scheduler and
driver pre-checks to have them raise the new MigrationPreCheckError
exception instead.

In addition, also make sure that expected exceptions are passed over
RPC silently.

Resolves bug 1171526.

Conflicts:
	nova/scheduler/filter_scheduler.py
	nova/scheduler/manager.py
	nova/virt/xenapi/vmops.py

Change-Id: I00fa0962e555f895dd98861684ec7767fed3b37b
(cherry picked from commit 9eb3dc7ed44d99652a17238724842e3067d46120)
Make resize/migrated shared storage aware

Fixes bug 1177247 (for stable/grizzly)

Added some logic to check for whether or not we are on a shared
filesystem and set shared_storage accordingly. We perform similar
checks in other areas of the code, typically through RPC calls.
However, all the resize/migrate code is slated to be refactored for
Hava, so the idea was to keep this patch as minimally intrusive as
possible.

When shared_storage is true, we pass that on to the cleanup call
so that it no longer executes an rm via SSH, which was ultimately
destroying the original instance directory.

Change-Id: Ie9decedd373c000211c171df64e1e96fe78e5081
Cherry-Pick: 9290bddd9f270d8ea4fbd6d953a8634473979cd5
Don't make nova-compute depend on iSCSI

Because not all volume drivers require iSCSI it makes sense to not
set the initiator in the connector and let the volume drivers
return an error if iSCSI is required.

bug #1007084

Change-Id: I5da472e5013fdc0c733ecb12b919c1068873bdc6
(cherry picked from commit 3ba0c171c0b6f3516662bc0ab660b14cd6d34d8a)
Fix a race where a soft deleted instance might be removed by mistake

When a soft deleted instance is restored there is a tiny risk that it
might be deleted if the reclaim periodic task runs at the same time.
This happens because the restore operation sets deleted_at to None
which makes the reclaim job think it is ok to delete the instance.

This can be resolved by restricting reclaims to only those instances
whose task_state is None.

Resolves bug 1186243.

Conflicts:
	nova/compute/manager.py
	nova/db/sqlalchemy/api.py
	nova/tests/compute/test_compute.py

Change-Id: Ia138e2d504b7482c46900583f019a32c70513245
(cherry picked from commit cdf10381caad217ae9eb99c69a4226e40ba81db4)
Allow confirm_resize if instance is in 'deleting' status

Deleting an instance puts it into 'deleting' task_state.
If the instance is in 'resized' vm_status, this tries a
confirm_resize on the instance. confirm_resize currently expects
the task_state to be None. It should instead expect
None/deleting task_state.

Fixes bug: 1189391

Conflicts:
        nova/tests/compute/test_compute.py

Change-Id: I6a3f82eab9ed2242bfd38bc1ac760dbb4b3d8c4a
(cherry picked from commit 0fed513b695cd15aaca24d24926a1c7d13e63509)
Remove junk file when ftp transfer failure

During spawn with powervm driver, the image transfer to the IVM host
may fail in some conditions, eg. IVM host doesn't have enough disk
space. It leaves the junk file in IVM host, and this will disturb
next spawn after resolving environment issue, need clean it.

Fixes bug #1188884

Change-Id: I6347dec4cfda61bed0960159e8a3402fcffa296b
Fix resizes with attached file-based volumes

Bug 1190364

get_instance_disk_info() now takes an additional, optional argument
of block_device_info. When passed in, we use it to filter out
volumes, such that they are not returned. Previously, any devices
classified as "file" as parsed from the instance XML were considered
local disks. This assumption is incorrect, since some volumes are
simply files, rather than block devices.

Added a new test, based almost entirely off the existing test case
for this function, which represents an XML file with two fake
file-based (NFS-like) volumes and associated block_device_info to
claim those volumes.

Change-Id: I4809a8fd68eb0f709347aa7cad0fa85389ade41c
(cherry picked from commit e71081b856b06513cc1e56a9adb2a619967f0eae)
Avoid redefining host to none in get_instance_nw_info(...)

The host parameter is required and a part of the method signature.
When it is redefined from kwargs.get('host') the supplied value
gets redefined to None. This removes the redefinition and keeps the
parameter value intact.

Fixes bug #1194178

Change-Id: I779cbce909935a4b794c16cefb0128f16f8563fb
(cherry picked from commit b0a2a2d29f24cea15d866f352d4beae5da965605)
Don't deallocate network if destroy time out

Fixes bug #1169057
Sames as bug 1180178, if libvirt returns time out error during destroy
an instance, we also can't deallocate the network of the instance,
otherwise the fixed ip of it may be re-used by another new instance,
and resulted in the new instance can't be accessed by the fixed ip.

Change-Id: I04dfd21bb1b7168abc71f39e836126a11dad6d8d
(cherry picked from commit d925044e09509495e7474d5f625bbd45d8985afa)
Disassociate fixed IPs not known to dnsmasq

Currently, fix_ip disassociation only happens in release_fixed_ip(),
which is called by dhcp-bridge script when ip is released. But there
are circumstances when instancs fail to get lease from dnsmasq(instances
fail to boot or instances fail to spawn). In such case, DHCPRELEASE
packet sent to dnsmasq would never trigger dhcp-bridge to run. Thus the
fixed ip would not be disassociated from instance until
_disassociate_stale_fixed_ips() begins to run.

Actually, this patch borrows the idea from Vishvananda's
3f6739e2a8acdf20d01c0b5039a33f26d879587b commit: explicitly disassociate
fixed ip which is not known to dnsmasq, instead of via the dhcp-release
script.

Steps to reproduce:
1. Create an instance which doesn't send dhcp request to dnsmasq.
2. Delete the instance.
3. The fixed ip associated with the instance is not returned to pool.

Bug 1180766

Change-Id: Icc022ea2fb7af08fb7354e3a49c77ad5ae58f34c
(cherry picked from commit 383ca6cf5ff9172a43339dbf845245fbdf2f8210)
Fix nic order not correct after reboot

Previously if an instance was rebooted or migrated the original nic order
would not be maintained. This patch retrieves the correct nic order from
info_cache so that the order is mantained.

Fixes bug 1187092

Change-Id: I3f5488cc14c692fde7d5bd3376389e2f2f5550ff
(cherry picked from commit bcd12a776e8cde8c4b64e7c1a21a7387ece55b31)
Update instance.node on evacuate

The evacuate API works like a migration from a dead node. As part of
migration, the instance.node property is updated to reflect the target
node that the instance is migrating to. This is later used when the
migration is confirmed/reverted. The problem is when an instance is
being evacuated, the node property isn't getting updated and the
instance will have the old (dead) node value set on it after it's
evacuated. This patch updates instance.node when the instance is
evacuated.

Fixes bug 1198284

Change-Id: I4901fb937160e955df7c97c37265783e844fdbef
(cherry picked from commit 3da3dfb733237ed2c908d8770961d8d6656ea1a3)
Make flavors is_public option actually work

When you create a flavor, you can set an is_public flag to be True or
False.  It is True by default.  When False, the intention is that the
flavor is only accessible by an admin, unless you use the flavor_access
API extension to grant access to specific tenants.

Unfortunately, the only place in the code where this was being enforced
was when listing flavors through the API.  It would filter out the
non-public ones for a non-admin.  Otherwise, the flavor was accessible.
You could get the details, and you could boot an instance with it, if
you figured out a valid flavor ID.

This patch adds enforcement down in the db layer.  It also fixes one
place in the API where the context wasn't passed down to enable the
enforcement to happen.

Fix bug 1194093.

(cherry picked from commit b65d506a5f9d9b2b20777a9aceb44a8ffed6a5de)

Conflicts:
	nova/api/openstack/compute/contrib/flavor_access.py
	nova/api/openstack/compute/contrib/flavormanage.py
	nova/api/openstack/compute/flavors.py
	nova/compute/api.py
	nova/db/sqlalchemy/api.py
	nova/tests/api/openstack/compute/contrib/test_flavor_access.py
	nova/tests/api/openstack/compute/contrib/test_flavor_disabled.py
	nova/tests/api/openstack/compute/contrib/test_flavor_manage.py
	nova/tests/api/openstack/compute/contrib/test_flavor_rxtx.py
	nova/tests/api/openstack/compute/contrib/test_flavor_swap.py
	nova/tests/api/openstack/compute/contrib/test_flavorextradata.py
	nova/tests/api/openstack/compute/test_flavors.py
	nova/tests/db/test_db_api.py

Change-Id: I5b37fa0bb19683fe1642fd81222547d4a317054e
eventlet.monkey_patch() for spice and vnc proxy

The VNC proxy and spice apps did not run eventlet.monkey_patch(). These
services uses rpc to talk to nova-consoleauth, so eventlet is required.

Note that this is a stable/grizzly patch only. The issue does not
affect master (havana) anymore as it was resolved as a side effect of
switching everything in bin/ over to using entry points. The monkey
patching is done in nova/cmd/__init__.py now.

Change-Id: I61cb1212d283e41e6fda88a9521a9e54143af1fb
Force reopening eventlet's hub after fork

With this we reopen eventlet's hub after a fork (triggered from
websockify when a new client connects) to prevent sharing epoll's fd
with the parent, which may cause erratic behaviour.

This caused novncproxy/spicehtml5proxy to stop working when it had more
than two clients connected.

Fixes bug #1193031

Change-Id: I3ff9001543b84b1037597da243422490bb611657
Add inst_type parameter

Add inst_type parameter in order to generate the bandwidth tag into
the libvirt configuration.

Change-Id: I63c1fb51ea637d461755b309c7464f4b4d49a4b2
Fixes: bug #1185030
(cherry picked from commit 08f8773b9a4fe7fea248b4501230ebd6e6ef2fde)
Naming instance directory by uuid in VMware Hyper.

Like Libvirt Hyper, Using EC2 template for naming instance
directory may cause conflict. we follow what Libvirt does to use
instance uuid instead of EC2 template naming. To keep backwards
compability, when fails to find a instance by uuid, fall back to use
EC2 name.

Fix bug #1186944

(cherry picked from commit e517e5668ab20214432e06ef87f7b16776edaf7c)

Conflicts:

	nova/tests/vmwareapi/db_fakes.py
	nova/virt/vmwareapi/vmops.py

Change-Id: Iaba099225a9b3b0390cae4c5fbd82d7e6c11f0e0
Merge "Fix nic order not correct after reboot" into stable/grizzly
Merge "Naming instance directory by uuid in VMware Hyper." into stable/grizzly
Fix aggregate_get_by_host host filtering

aggregate_get_by_host never actually joins the _hosts relationship,
so aggregates with non-matching hosts are still included in the output.

This change adds the join and improves the tests to cover the bug.

Fixes bug 1201277.

Conflicts:

	nova/db/sqlalchemy/api.py
	nova/tests/db/test_db_api.py

Change-Id: I87ebfe3de193b9ae47e3a2c8350a49d9cc863fb0
(cherry picked from commit 13513a182a54dffb62900a8464529459dd836136)