Re-attach volumes after instance resize

Fixes bug 1051600.

Update the compute manager to pass block_device_info to three
additional driver entry points and update all virtualization drivers
to accept it as an optional argument.

Within libvirt, migrate_disk_and_power_off() will now iterate and disconnect
any existing connections.  finish_migration() simply updates its use
of to_xml(), passing block_device_info, which returns a libvirt XML
fully populated with volumes.  Finally, finish_revert_migration() no
longer uses the on-disk XML file, instead generating it via to_xml; as
the former lacks volume configuration.

Change-Id: I7dccd03ef9cc7d2848f07764d4def3787e41e202
Fixes snat rules in complex networking configs

There is currently no way to disable nova's snat rule, which causes
problems for some network configurations with an external gateway.
This patch allows the snat to be completely disabled by setting
a blank value for routing_source_ip.

This patch also makes the snat rule a little more specific, only
snatting traffic that is destined for the public interface (or
the floating interface if it is defined). This allows the snat
to work without changing routing_source_ip and also ensures that
an instance assigned multiple ips from different floating pools
will connect from the appropriate address for each pool

Fixes bug 1048765

Change-Id: I18be88a3dbb7a9f4762db7beefc94e07b8310100
Make DeregisterImage respect AWS EC2 specification

The return value of current deregister_image implementation is the AMI id.
Howerver, according to Amazon EC2 API documentation, the return value of
DeregisterImageResponse should be boolean.

See more detailed documentation in:
http://docs.amazonwebservices.com/AWSEC2/latest/APIReference/ApiReference-query-DeregisterImage.html

The example response:
<DeregisterImageResponse xmlns="http://ec2.amazonaws.com/doc/2012-08-15/">
  <requestId>59dbff89-35bd-4eac-99ed-be587EXAMPLE</requestId>
  <return>true</return>
</DeregisterImageResponse>

This patch changes the response and test case for deregister_image.

Fixes bug 1051752.

Change-Id: Ifd31b88ec4f575f50244079cdeed4c9ff57206f8
Fix volume deletion when device mapper is used

Call dmsetup remove if there is a /dev/mapper/nova--volumes-
element present.

Resolves bug 979020

Change-Id: Iddaaed411a77dda4bd32f9a97687ff17744119eb
Fixes error handling during schedule_run_instance

If there are not enough hosts available during a multi-instance launch,
every failing instance should be updated to error state, instead of
just the first instance. Currently only the first instance is set
to Error and the rest stay in building.

This patch makes a number of fixes to error handling during scheduling.

 * Moves instance faults into compute utils so they can be created
   from the scheduler.
 * Moves error handling into the driver so that each instance can be
   updated separately.
 * Sets an instance fault for failed scheduling
 * Sets task state back to none if there is a scheduling failure
 * Modifies chance scheduler to stop returning a list of instances
   as it is not used.
 * Modifies tests to check for these states.

In addition to the included tests, the code was manually verified on
a devstack install

Fixes bug 1051066
Fixes bug 1019017

Change-Id: I49267ce4a21e2f7cc7a996fb2ed5d625f6794730
Fix volume id conversion in nova-manage volume

  The nova-manage volume commands weren't updated when the switch
  from int volume-id's to uuid's was made.  This updates the
  param2id method to call the appropriate conversion, and doesn't
  attempt to cast the value to an int.

  There is an additional issue with the nova-manage commands with the
  introduction of Cinder.  The calls in nova-manage are direct to the
  nova database which when using Cinder is not valid.  Add an error message
  when trying to use delete or re-attach commands as they don't work.

  This change deprecates the nova-manage delete function as it's now
  implemented as an admin extension (nova commit id: If795599d and
  cinder commit id: I29f4b892).

  The reattach command is only valid for nova-volumes, a similar extension
  will need to be added to cover cinder at which time the nova-manage volume
  reattach can be deprecated as well.

  Partial fix for bug #1051603

Change-Id: I666d4e627dee4a2025f7135560ee36c40f4bf17a
Merge "Fixes error handling during schedule_run_instance"
Raise BadRequest while creating server with invalid personality

Handle UnicodeDecodeError raises from compute api while trying to
create server with invalid personality content and throw it as
HTTPBadRequest

Fixed bug 1050409

Change-Id: I27d47bbc9ed89abfa9827512fbfb3b16a0d87160
Merge "Make DeregisterImage respect AWS EC2 specification"
Add console output extension to API samples test.

Change-Id: I0ac1f893b06882acaa7c61dade61eb991a5d0d04
Signed-off-by: Matthew Treinish <treinish@linux.vnet.ibm.com>
Merge "Fix volume deletion when device mapper is used"
Adds new volume API extensions

Adds following extensions:
1. Create volume from image
2. Copy volume to image

Added unit tests.

Implements: blueprint create-volume-from-image

Conflicts:

	cinder/api/openstack/volume/contrib/volume_actions.py
	cinder/tests/api/openstack/fakes.py
	cinder/tests/api/openstack/volume/contrib/test_volume_actions.py
	cinder/tests/policy.json
	nova/api/openstack/volume/volumes.py
	nova/flags.py
	nova/tests/api/openstack/volume/test_volumes.py
	nova/tests/test_volume.py
	nova/utils.py
	nova/volume/api.py
	nova/volume/manager.py

This is based on a cherry-pick of cinder commit
2f5360753308eb8b10581fc3c026c1b66f42ebdc with bug fixes
8c30edff982042d2533a810709308b586267c0e9 and
ffe5036fa0e63ccde2d19aa0f425ec43de338dd7 squashed in.

Change-Id: I9c73bd3fa2fa2e0648c01ff3f4fc66f757d7bc3f
add get_location method for images

This is useful for determining whether the backend storage for the
image in Glance is the same as the storage used in a volume driver.

The direct_url is only available in the v2 images API, so add a
version parameter to each request. As more parts of the v2 API are
used, this parameter will become more useful.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>

Conflicts:

	nova/image/glance.py

This is based on a cherry-pick of cinder commit
171a57a6879683dac5f219a0e8f1080ea687105d

Change-Id: I5ca06b6d80ffe4118f8dd381ed11283573bce71f
add ability to clone images

Given the backend location from Glance, drivers can determine
whether they can clone or otherwise efficiently create a volume
from the image without downloading all the data from Glance.

For now implement cloning for the RBD driver. There's already a
Glance backend that stores images as RBD snapshots, so they're
ready to be cloned into volumes. Fall back to copying all the
data if cloning is not possible.

Implements: blueprint efficient-volumes-from-images
Signed-off-by: Josh Durgin <josh.durgin@inktank.com>

Conflicts:

	nova/volume/api.py
	nova/volume/driver.py

This is based on a cherry-pick of cinder commit
edc11101cbc06bdce95b10cfd00a4849f6c01b33

Change-Id: I71a8172bd22a5bbf64d4c68631630125fcc7fd34
Merge "Fixes snat rules in complex networking configs"
make ensure_default_security_group() call sgh

This change makes sure that sgh is called when default
security group is created.

Fixes bug 1050982

Change-Id: I483e33d8977b2bfbd7456a7c7c0d1fe7803708f8
Merge "Add console output extension to API samples test."
Merge "Fix volume id conversion in nova-manage volume"
Correct db migration 91

* Snapshot.id gets the new snapshot uuid rather than snapshot.volume_id
* Foreign keys are dropped before and recreated after updating id fields
* Snapshot id <-> uuid queries use snapshot_id_mappings.c.id rather
  than volume_id_mappings.id
* Snapshot id <-> uuid queries are executed before passing the new id
  values into subsequent UPDATE queries
* Thoroughly inspect the expected modifications in a new functional test
* Fixes bug 1052244
* Fixes bug 1052220

Change-Id: I22c820e5747562251c6447ac678c80dd9e0e2e20
Merge "Adds new volume API extensions"