Make resize/migrated shared storage aware

Fixes bug 1177247 (for stable/grizzly)

Added some logic to check for whether or not we are on a shared
filesystem and set shared_storage accordingly. We perform similar
checks in other areas of the code, typically through RPC calls.
However, all the resize/migrate code is slated to be refactored for
Hava, so the idea was to keep this patch as minimally intrusive as
possible.

When shared_storage is true, we pass that on to the cleanup call
so that it no longer executes an rm via SSH, which was ultimately
destroying the original instance directory.

Change-Id: Ie9decedd373c000211c171df64e1e96fe78e5081
Cherry-Pick: 9290bddd9f270d8ea4fbd6d953a8634473979cd5
Don't make nova-compute depend on iSCSI

Because not all volume drivers require iSCSI it makes sense to not
set the initiator in the connector and let the volume drivers
return an error if iSCSI is required.

bug #1007084

Change-Id: I5da472e5013fdc0c733ecb12b919c1068873bdc6
(cherry picked from commit 3ba0c171c0b6f3516662bc0ab660b14cd6d34d8a)
Fix a race where a soft deleted instance might be removed by mistake

When a soft deleted instance is restored there is a tiny risk that it
might be deleted if the reclaim periodic task runs at the same time.
This happens because the restore operation sets deleted_at to None
which makes the reclaim job think it is ok to delete the instance.

This can be resolved by restricting reclaims to only those instances
whose task_state is None.

Resolves bug 1186243.

Conflicts:
	nova/compute/manager.py
	nova/db/sqlalchemy/api.py
	nova/tests/compute/test_compute.py

Change-Id: Ia138e2d504b7482c46900583f019a32c70513245
(cherry picked from commit cdf10381caad217ae9eb99c69a4226e40ba81db4)
Allow confirm_resize if instance is in 'deleting' status

Deleting an instance puts it into 'deleting' task_state.
If the instance is in 'resized' vm_status, this tries a
confirm_resize on the instance. confirm_resize currently expects
the task_state to be None. It should instead expect
None/deleting task_state.

Fixes bug: 1189391

Conflicts:
        nova/tests/compute/test_compute.py

Change-Id: I6a3f82eab9ed2242bfd38bc1ac760dbb4b3d8c4a
(cherry picked from commit 0fed513b695cd15aaca24d24926a1c7d13e63509)
Remove junk file when ftp transfer failure

During spawn with powervm driver, the image transfer to the IVM host
may fail in some conditions, eg. IVM host doesn't have enough disk
space. It leaves the junk file in IVM host, and this will disturb
next spawn after resolving environment issue, need clean it.

Fixes bug #1188884

Change-Id: I6347dec4cfda61bed0960159e8a3402fcffa296b
Avoid redefining host to none in get_instance_nw_info(...)

The host parameter is required and a part of the method signature.
When it is redefined from kwargs.get('host') the supplied value
gets redefined to None. This removes the redefinition and keeps the
parameter value intact.

Fixes bug #1194178

Change-Id: I779cbce909935a4b794c16cefb0128f16f8563fb
(cherry picked from commit b0a2a2d29f24cea15d866f352d4beae5da965605)
Don't deallocate network if destroy time out

Fixes bug #1169057
Sames as bug 1180178, if libvirt returns time out error during destroy
an instance, we also can't deallocate the network of the instance,
otherwise the fixed ip of it may be re-used by another new instance,
and resulted in the new instance can't be accessed by the fixed ip.

Change-Id: I04dfd21bb1b7168abc71f39e836126a11dad6d8d
(cherry picked from commit d925044e09509495e7474d5f625bbd45d8985afa)
Disassociate fixed IPs not known to dnsmasq

Currently, fix_ip disassociation only happens in release_fixed_ip(),
which is called by dhcp-bridge script when ip is released. But there
are circumstances when instancs fail to get lease from dnsmasq(instances
fail to boot or instances fail to spawn). In such case, DHCPRELEASE
packet sent to dnsmasq would never trigger dhcp-bridge to run. Thus the
fixed ip would not be disassociated from instance until
_disassociate_stale_fixed_ips() begins to run.

Actually, this patch borrows the idea from Vishvananda's
3f6739e2a8acdf20d01c0b5039a33f26d879587b commit: explicitly disassociate
fixed ip which is not known to dnsmasq, instead of via the dhcp-release
script.

Steps to reproduce:
1. Create an instance which doesn't send dhcp request to dnsmasq.
2. Delete the instance.
3. The fixed ip associated with the instance is not returned to pool.

Bug 1180766

Change-Id: Icc022ea2fb7af08fb7354e3a49c77ad5ae58f34c
(cherry picked from commit 383ca6cf5ff9172a43339dbf845245fbdf2f8210)
Fix nic order not correct after reboot

Previously if an instance was rebooted or migrated the original nic order
would not be maintained. This patch retrieves the correct nic order from
info_cache so that the order is mantained.

Fixes bug 1187092

Change-Id: I3f5488cc14c692fde7d5bd3376389e2f2f5550ff
(cherry picked from commit bcd12a776e8cde8c4b64e7c1a21a7387ece55b31)
Update instance.node on evacuate

The evacuate API works like a migration from a dead node. As part of
migration, the instance.node property is updated to reflect the target
node that the instance is migrating to. This is later used when the
migration is confirmed/reverted. The problem is when an instance is
being evacuated, the node property isn't getting updated and the
instance will have the old (dead) node value set on it after it's
evacuated. This patch updates instance.node when the instance is
evacuated.

Fixes bug 1198284

Change-Id: I4901fb937160e955df7c97c37265783e844fdbef
(cherry picked from commit 3da3dfb733237ed2c908d8770961d8d6656ea1a3)
eventlet.monkey_patch() for spice and vnc proxy

The VNC proxy and spice apps did not run eventlet.monkey_patch(). These
services uses rpc to talk to nova-consoleauth, so eventlet is required.

Note that this is a stable/grizzly patch only. The issue does not
affect master (havana) anymore as it was resolved as a side effect of
switching everything in bin/ over to using entry points. The monkey
patching is done in nova/cmd/__init__.py now.

Change-Id: I61cb1212d283e41e6fda88a9521a9e54143af1fb
Force reopening eventlet's hub after fork

With this we reopen eventlet's hub after a fork (triggered from
websockify when a new client connects) to prevent sharing epoll's fd
with the parent, which may cause erratic behaviour.

This caused novncproxy/spicehtml5proxy to stop working when it had more
than two clients connected.

Fixes bug #1193031

Change-Id: I3ff9001543b84b1037597da243422490bb611657
Naming instance directory by uuid in VMware Hyper.

Like Libvirt Hyper, Using EC2 template for naming instance
directory may cause conflict. we follow what Libvirt does to use
instance uuid instead of EC2 template naming. To keep backwards
compability, when fails to find a instance by uuid, fall back to use
EC2 name.

Fix bug #1186944

(cherry picked from commit e517e5668ab20214432e06ef87f7b16776edaf7c)

Conflicts:

	nova/tests/vmwareapi/db_fakes.py
	nova/virt/vmwareapi/vmops.py

Change-Id: Iaba099225a9b3b0390cae4c5fbd82d7e6c11f0e0
Merge "Fix nic order not correct after reboot" into stable/grizzly
Merge "Naming instance directory by uuid in VMware Hyper." into stable/grizzly
Fix aggregate_get_by_host host filtering

aggregate_get_by_host never actually joins the _hosts relationship,
so aggregates with non-matching hosts are still included in the output.

This change adds the join and improves the tests to cover the bug.

Fixes bug 1201277.

Conflicts:

	nova/db/sqlalchemy/api.py
	nova/tests/db/test_db_api.py

Change-Id: I87ebfe3de193b9ae47e3a2c8350a49d9cc863fb0
(cherry picked from commit 13513a182a54dffb62900a8464529459dd836136)
Fix filtering aggregate metadata by key

Filtering by metadata key has no effect in the following two functions:

    aggregate_metadata_get_by_host
    aggregate_host_get_by_metadata_key

Rely on the explicit join instead of joinedload to load the _metadata
relationship in these functions (indicated by the contains_eager
option), so that filtering by key still applies when the metadata is
collected.

Fixes bug 1201283.

Conflicts:

	nova/tests/db/test_db_api.py

Change-Id: I3e44b5db8d67787bfa14f0e57d30a02a271a8833
(cherry picked from commit 7e8fa8f3e6acac0e510cd1fc5af591f9a321d03a)
VNC console does not work with VCDriver

Introduces get_vnc_console_vcenter as a way to get
a modified vnc_console object that connects to the
ESXi host underneath vCenter control.

* adds new fake classes for testing
* documents classes, methods & API logic

fixes bug: #1178369

(cherry picked from commit 271fc68c1852e3764b7c64d71cd28ac3f803ecba)

Conflicts:

	nova/virt/vmwareapi/fake.py

Change-Id: I48430cb9bc9615e02ca9af235f97853f3f0bdafd
Don't pass empty image to filter on live migration

If live migration for a volume backed instance without an image supplied
is initiated with no destination supplied, image is set to None.

This value will cause one of the default filters - the
ImagePropertiesFilter to throw an exception in it's host_passes method,
as it expects an image to be a dictionary.

We solve this by leaving out the image entirely before engaging filters.

This patch is intended for stable/grizzly only, since live migration
functionality is being heavily refactored in Havana at the moment and
the code from the patch that introduces this bug
(Ie683a3ca5d6430c52ead77b41f98fbcb4114ea1e) was
accidentally dropped in I6de33ada6dc377e20f8df07da92244f2c150b9fe.

Fixes bug: 1199811

Change-Id: I8a5f1b176addbe09c8d8b5f1d1136d05af53ce3e
Merge "Disassociate fixed IPs not known to dnsmasq" into stable/grizzly