Handle compute node not available for live migration

This patch handles exception.ComputeServiceUnavailable by restoring
instance's vm_state and instance's task_state after live migration
failure caused by unavailable source/dest compute node.

Raises detailed HTTPBadRequest explanation for this exception.

Fixes bug 973393 and bug 1051881

Change-Id: If825b61fad9c4e3030f2e6c5002907255eaf3661
Fix update quota with invalid value

If update quota with a value which is not integer, it will response
with http status code 500. Catch the exception and continue for
updating other quotas.

Fix LP# 1111327

Change-Id: I0ab3b10ff106a2e6c7677d1acdb55c2f35c492db
Ensure that LB VIF drivers creates the bridge if necessary

Fixes bug 1117770

In order for libvirt to use the bridge name the bridge must exist.

Change-Id: Ie24cfce8454e0c1bbfd9058787a1fd704790f03c
Changes GA code for tracking cross-domain.

Change-Id: I731d6b160588b59f01abe98699c144fdf4ae12ad
Properly write non-raw LVM images on creation

Fixes bug 1075333

When force_raw_images=False and libvirt_images_type=lvm were set
together, selecting anything other than a RAW image would result in
an unbootable instance. This is because the downloaded image was
being directly dd'd onto the new LVM root, which will obviously not
work for any format other than RAW.

Simple change to use qemu-img to read from the image (letting it
decide the format) and write to the LVM root as raw.

Change-Id: I36494e6d0671c07f7511c10ef886c8cf84c18df7
Libvirt: Implement snapshots for LVM-backed roots

Uses the built-in LVM snapshotting mechanism to create a new
snapshot, use qemu-img to extract the full root into an image file,
then delete the snapshot volume. Although we typically zero deleted
LVM volumes manually, this is not necessary for snapshots, as LVM
handles it for us (per the documentation).

Change-Id: I82f21488671ffa3be3d0feba73ff631fa2a51ec0
Fixes: Bug 1120809
Flags: DocImpact
make vmwareapi driver pass quantum port-id to ESX

bug 1121349

Similar to how the libvirt and xenapi drivers need to communicate
the quantum port-id (aka the nova vif_uuid when quantum is in use)
to the compute layer in order to identify each vNIC, vmwareapi
needs to do the same.

Change-Id: I8b4bf81bfbad63e61a6ade9f0173e478b3aa45bd
Refine PowerVM MAC address generation algorithm

Change MAC address generation algorithm on PowerVM to generate
an additional 4 bits of data and improve comments around it.

Change-Id: Icdfba54161bcb043c17127c26c57503af497d001
Remove locking declarator in ServiceGroup __new__()

The lock-based synchronization is not re-entrant, which
would introduce a deadlock with the new conductor patch.
Because race condition is not be possible in the
eventlet threading model with GIL, we simply remove the
synchronized declarator.

Change-Id: I03e52227385dafd2b2b66bca18cab8445c91f3be
test_(dis)associate_by_non_existing_security_group_name missing stub

test_(dis)associate_by_non_existing_security_group_name() is passing because
it is not finding the instance which also returns HTTPNotFound not because
it cannot find the security group. This patch adds the missing stub so
that it it checks for the non existing security group.

Fixes bug 1120832

Change-Id: I67a2b82ac329193c2438038719f8c052de8081a3
Implements resize / cold migration on Hyper-V

Blueprint: hyper-v-compute-resize

Resize / cold migration is implemented by copying the local disks to a remote
SMB share, identified by the configuration option HYPERV.instances_path_share
or, if empty, by an administrative share with a remote path corresponding to
the configuration option instances_path.

The source instance directory is renamed by adding a suffix "_revert" and
preserved until the migration is confirmed or reverted. In the former case
the directory will be deleted and in the latter renamed to the original name.

The VM corresponding to the instance is deleted on the source host and
recreated on the target. Any mapped volume is disconnected on the source
and reattached to the new VM on the target host.

In case of resize operations, the local VHD file is resized according to
the new flavor limits. Due to VHD limitations, an attempt to resize a disk
to a smaller size will result in an exception.

In case of differencing disks (CoW), should the base disk be missing
in the target host's cache, it will be downloaded and reconnected to the
copied differencing disk.

Same host migrations are supported by using a temporary directory
with suffix "_tmp" during disk file copy.

Unit tests have been added for the new features accordingly.

Change-Id: Ieee2afff8061d2ab73a2252b7d2499178d0515fd
Update docs about testing.

Add an entry to the HACKING file about testr. While in there, noticed a
reference to the now-defunct nova/testing dir. Fixed that, moved the testing
README into nova/tests and remove the nova/testing dir.

Change-Id: Ibf6fb82658ba73eee9123fa53b340d0b72afb292
Adding support for AoE block storage SANs.

Adding support for AoE (ATA over Ethernet) block storage SANs.
AoE initiator driver is supported on all Linux initiators
(released under GPL), ATA over Ethernet (AoE) Linux driver
for all 3.x and 2.6 kernels.

Implements blueprint libvirt-aoe

   - Fix (minor) Redundant exception caching

Change-Id: I00de13eb2d4bd1105b9eb68f1465825414e2d5d7
Fix nova-compute use of missing DBError

Apparently exception.DBError disappeared at some point, but nova-
compute was still using it. Since it was a stretch in the first
place, and nothing else in exception.py looks reasonable, this
adds exception.DBNotAllowed.

Fixes bug 1120390

Change-Id: Ifbfa0b09f3dc011ad87f6f3b06eb0fdaa7247ec0
Make the metadata paths use conductor

This patch makes the metadata service use conductor instead of hitting the
database directly.

It adds a get_ec2_ids() method to conductor, which encapsulates a bunch of
very small id translation database queries. It also does the AZ lookup
via conductor.

Fixes bug 1120402

Change-Id: Iceef753b1451ff53e29dfbf97a696b952b47d5a3
Ensure that only one IP address is allocated.

Fixes bug 1122899

Change-Id: I03f7509977a37bfcb8f1698a83638be3079065a0
Add an extension to show the type of an ip.

There is currently no way to differentiate a fixed ip from a floating
ip in the servers list. This extension adds the OS-EXT-IPS:type param
so that users can determine if it is a fixed address or a floating
address.

Fixes bug 1117784

DocImpact

Change-Id: I8709a4f2f6792faa3d8e0cdcd330d751ef6633ae
Simplify and optimize az server output extension.

Displaying two different az results to users is confusing, so just
return the az that the instance is actually in. Also, cache the
result from looking up the az by host for an hour to avoid making
lots of expensive db requests.

DocImpact

Change-Id: Ib39bf94c608874695aab00e61035e64f9594a985
Sync jsonutils from openstack-common

Pulls in new jsonutils to get the configurable max_depth in
jsonutils.to_primitive.

Change-Id: I4878a033257839ec3b2ac5223440b3a9034877cc
xenapi: Remove unecessary exception handling

The volume code was catching exceptions from the XenAPI only to re-raise
more-generic StorageError exceptions. This practice makes it much more
difficult to discover the root-cause of issues in production by making
the message in the instance-faults table less useful.

The solution is to have the original root-cause exception propogate all
the way up to the instance-faults table where the developer will be able
to make sense of it.

Other cleanups:

* Remove useless logging and stop logging opaque-refs in some places

* Remove dead code (create_iscsi_storage)

* Remove one-off functions (forget_sr_if_present,
  introduce_sr_unless_present); just use forget_sr and introduce_sr

* Removed unused `exception` import

* Added Openstack copyright lines

Future Work:

There are lots of other places where unecessary exception handling is
occuring; this patch just addresses this in the attach/detach code-path.

Fixes bug 1122733

Change-Id: I8c382fb505303e604ff2e86afcf302efe3d6851d