Reset migrating task state for MigrationError exceptions

Create an exception type for pre-check errors and add it to the list
of exceptions for which instance task state is reset to ACTIVE to
avoid cases where an instance get stuck in MIGRATING state.

Convert existing instances of raise MigrationError in scheduler and
driver pre-checks to have them raise the new MigrationPreCheckError
exception instead.

In addition, also make sure that expected exceptions are passed over
RPC silently.

Resolves bug 1171526.

Conflicts:
	nova/scheduler/filter_scheduler.py
	nova/scheduler/manager.py
	nova/virt/xenapi/vmops.py

Change-Id: I00fa0962e555f895dd98861684ec7767fed3b37b
(cherry picked from commit 9eb3dc7ed44d99652a17238724842e3067d46120)
Fix powervm driver resize instance error

During a resize operation in powervm, the logical volume
of the instance will be copied and compressed to a file.
Current logic decompresses it twice before copying this file
to a new volume which will cause the new instance to not
boot properly.

Fixes bug #1169848

Change-Id: I91f536919511aafbf7dbe14abbee48dbceb189b2
(cherry picked from commit 5643bd5d9bf17335e109d4b2d93e161f4d12c6f9)
Support FlatDHCP network for VMware Hyper.

Auto create bridge when using FlatDHCP network, Add unittests for vif.

Fix bug #1171783
Fix bug #1177635

Change-Id: Ibb75c8e3174283f0f5e94bacfd61ee93e8bd6227
(cherry picked from commit bc66b8aa968a0c0b74e2f21ed0fc856f8933c6d3)
Volume IO usage gets reset to 0 after a reboot / crash

Check for reset of block device stats before updating volume usage cache.

If an instance is reboot or crashes, the results of block device stats for
that domain get reset. Add check that if block device stats are lower than
the current stats in the database, then add current values to totals and
update.

Change-Id: I3011872f8d80a1519771f8423001650dcbcc7f5b
Fixes bug: 1169097
(cherry picked from commit a4cb456631f0e70685fbc361eced9baa477a8c56)
Return proper error message when network conflicts

Fixes bug 1158075
When creating a network which is already existing, we got
an internal server error which can't show the error reason.
This patch returns conflict error(HTTP 409) instead of
internal server error(HTTP 500), so that users can know
the error reason from the error message.

Change-Id: I9e39a7152e37756b34b1c8d77de057012743f0f7
(cherry picked from commit 8a80c4c0968a9deceaa6d99629c6d7f7889769e5)
Catch InstanceNotFound in instance_actions GET

Catch the InstanceNotFound exception in the instance_actions
index function and handle the error gracefully so a stacktrace is
not generated in the log files. This patch does not change the
API

Fixes bug 1182867

Change-Id: I0764f8abda7176deefe0d103bd86acd0af78780d
(cherry picked from commit 8eede5e9814f3305e0bb66eeb93623ba25b16e9a)
set ERROR state when scheduler hits max attempts

Presently when scheduler raises NoValidHost due to max attempts
being reached, the instance remains in a build state.
Exception handler for NoValidHost in manager.run_instance() needs
request_spec[instance_uuids] to know which host to put into an
error state in _set_vm_state_and_notify().
schedule_run_instances() was popping instance_uuids from the
request_spec prior to a call to _schedule().
Changed pop of instance_uuids prior to call to _schedule() to be a get.
Added pop of instance_uuids to beneath call to _schedule() as
individual creates do not need them.

Conflicts:
	nova/scheduler/filter_scheduler.py

Change-Id: I9654820e01d5611763e9e673f15f46b947d09e6d
Fixes: bug #1182056
(cherry picked from commit aefc28dd481354edd0f3b5aec18db006680b2ffe)
Clean up failed image transfers in instance spawn

Boot one instance with powervm driver, will transfer the image
file to host. Need clean up failed image transfers that may occur
during instance spawn.

Fixes bug #1173017
(cherry picked from commit 68cc1cd511298208dde59878bfbef474a3625c9f)

Change-Id: Id06fa84bee63adc4af96a339526d75540978263f
Don't update API cell on get_nwinfo

Every call to get_instance_nw_info() updates an instance's info_cache.
The purpose of this is to make sure we're constantly healing the
info_cache table.  However, this causes a lot of unneeded updates
traveling up to the API cell.  Cells has its own periodic task that
heals all instance data.

Fixes bug 1180304

Conflicts:
	nova/network/api.py
	nova/tests/cells/test_cells_messaging.py

Change-Id: I07774712a6358999fd1fdd0f5a90c6f05e3d8d6e
(cherry picked from commit 4251bba3bd456f7ba1c7481bb84d839a5fafc757)
Don't delete sys_meta on instance delete

Unfortunately, we require to access instance_system_metadata to get
data (specifically at least instance_type data) for instances that have
been deleted.

There's 2 cases where this is true:

1) nova-api supports showing deleting instances when you specify a
'changes-since' param.
2) The _usage_audit_log periodic task pulls all instances during the
audit period, which includes instances that have been deleted during
that period.

This reverts commit 17bca438954638d74035d560e826a26a532b3ea3, which was
attempting to fix a bug where we leave instance_system_metadata entries
undeleted from the DB.  There's not an easy way to query for the deleted
sys_meta data in an efficient manner, and leaving the entries undeleted is
the lesser of the 2 evils for now.

Fixes bug 1185190

Change-Id: I898f0546c49126dcc56a4237120082f95dc82304
(cherry picked from commit 4885aa28706a1858f4fc51a0d2c661eec05139c0)
Don't snat all traffic when force_snat_range set

When using /proc/sys/net/bridge/bridge-nf-call-iptables=1, bridged
traffic gets snatted even though it is being bridged out to an
external gateway. This can lead to asymmetric routes where traffic
goes out the fixed network and comes back on the float network
and can lead to packets being blocked by firewalls. Work around
this problem by only fallback snatting when the traffic is
going to one of the force_snat_ranges. If force_snat_range is not
set it defaults to the existing behavior of snatting all traffic
that doesn't hit an earlier rule.

Fixes bug 1182143

Change-Id: If496dad2308a68a6a4a9fd43d695943bc5034ac4
(cherry picked from commit 769782be844e4c74fdc0aad0ff704b06c874c3ad)
Bump stable/grizzly next version to 2013.1.3

Change-Id: Ie9b5fb202effaa2d42270d64642369adf7b11840
Make nova-api use servicegroup.API.service_is_up().

Fix bug 1168861

Change-Id: I38e06c8832ec054755daa70898a303383557041b
(cherry picked from commit 616e5d965bd8a2923f00ba4ff194e6a6ee2468cd)
Fix bug where consoleauth depended on remote conductor service

Fixes bug 1186123

Conductor API was being imported directly so ignoring use_local FLAG

Change-Id: I055a5301d4c18d201d67f114047f503114751603
(cherry picked from commit 6c6e43e466a5383b3ef3c852c9b2749f0fad64dc)
Handle instance directories correctly for migrates.

In Grizzly we changed how we name instance directories. Unfortunately
the decision as to what directory name to use is based on whether
there is a pre-existing directory on the compute node with the old
name.

In the case of live migration, that doesn't work because we have
libvirt copy the directory across for us. So, we need to pass
around the directory name in the migration request. This is better
anyway, as it means that ops types don't need to handle an instance
directory magically renaming itself at some point.

Resolves bug 1175286.

Change-Id: Ib102e644a8742726fa2049303354d466b8109e0b
(cherry picked from commit 5c11a8c537eb4349e02e8f42468b441c259fbfc2)
Fix assumed port has port_security_enabled

Previously if adding a security group to an instance it would fail
if the port did not contain port_security_enabled=True. This patch
also changes the unit tests so that networks by default are not
associated with port_security_enabled=True because this is only true
if a plugin is running the port_security extension.

Fixes bug 1189059

Change-Id: Ieebb7aafba01c18164b86a01a3c289a6957321e4
(cherry picked from commit eddab7ba0e45c47fa8503b29484ce296f46c0d27)
Call virt.driver.destroy before deallocating network.

Call virt driver destroy before releasing the instance's
ip address, so that certain failures can be handled.
For instance, if destroy raises InstancePowerOffFailure,
do not deallocate network resources.

Fixes bug 1180178

Change-Id: I9bde984d2393bd73a73eae6065defb2ed7c01492
(cherry picked from commit dacb187fcd685c15fd084fdfce8c5e165130ff54)
Make resize/migrated shared storage aware

Fixes bug 1177247 (for stable/grizzly)

Added some logic to check for whether or not we are on a shared
filesystem and set shared_storage accordingly. We perform similar
checks in other areas of the code, typically through RPC calls.
However, all the resize/migrate code is slated to be refactored for
Hava, so the idea was to keep this patch as minimally intrusive as
possible.

When shared_storage is true, we pass that on to the cleanup call
so that it no longer executes an rm via SSH, which was ultimately
destroying the original instance directory.

Change-Id: Ie9decedd373c000211c171df64e1e96fe78e5081
Cherry-Pick: 9290bddd9f270d8ea4fbd6d953a8634473979cd5
Fix nova-compute fails to start if quantum is down

When nova-compute starts it needs to check the state of instances that
were previously running and needs to query quantum. Previously, if
it queried quantum and quantum raised an error or was unreachable
nova-compute would terminate. This patch adds a retry loop to prevent
this from happening.

Fixes bug 1186357

Change-Id: I0d865e464f7bf60a0b6dc4ca8a42436d16e29790
(cherry picked from commit 6fda12bb14ab91592910f9ebbde9ef9f8d588b08)
Regenerate missing resized backing files

Fixes bug 1185588 (for stable/grizzly)

The only way to reliably determine whether we are using a legacy
(pre-Grizzly) backing structure is to look at the backing file
reference for the instance/disk itself. Thus, if we determine the
backing file contains an underscore, we use the value following it
as our legacy_backing_size, which is then used to create the
necessary backing file.

Change-Id: I145f71068c972a604e5361ae79d6315286136786
Cherry-Pick: 6e768363450774cfee90e41aa5c40af780d3e04a