Fix building quantumapi network model with network list

Bug 1171636 occurs when _build_network_info_model() receives a list
of networks that does not fully map all of an instance's ports.
It crashes in _ensure_requested_network_ordering() when the list of
all ports is sorted by the subset of networks provided.

This change simplifies the early part of that function to only
consider ports within the networks provided, and to always sort those
ports in network order. This allows us to remove some error path
code from the loop below that iterates over the ports (which contained
a fatal typo, by the way) which is now dead code.

The affected code is really untested at the moment, due to the
structure of the method, making it hard to add a unit test to cover
this fix. In order to make the change clear from the existing code,
this fix is provided here first, and the following patch restructures
the method and add tests for this change and the rest of the surrounding
code.

Fixes bug 1171636

Change-Id: Idf519a88415844451b18959b38cc48ce92c090d1
Make _build_network_info_model testable

This restructures the quantumapi _build_network_info_model()
method to be testable and adds tests for each of the smaller
components. It also provides verification for bug 1171636 by
ensuring that ports returned from the client that are not in the
set of networks passed don't cause the failure detailed in that
bug.

Change-Id: I82f20f68122fc38e72c31e52283529614252c174
Rename _check_image_size to _get_and_check_image_metadata

This method has functionality which doesn't seem to match what
it actually does (which is getting image metadata and performing
basic checks on that data).

Change-Id: I818fb5079105cca8acbdf464d47a92f7b1ebb398
Fix EC2 instance bdm response

Make the bdm response respect Amazon EC2 spec:
 * use ec2 volume id instead of uuid
 * use volume attach_status instead of status

References:
http://docs.aws.amazon.com/AWSEC2/latest/APIReference/ApiReference-ItemType-EbsInstanceBlockDeviceMappingResponseType.html

Resolves bug 1165973.

Change-Id: Ib68397b79f63e01e92ed4bebbc245ba5e0a3ad11
Volume IO usage gets reset to 0 after a reboot / crash

Check for reset of block device stats before updating volume usage cache.

If an instance is reboot or crashes, the results of block device stats for
that domain get reset. Add check that if block device stats are lower than
the current stats in the database, then add current values to totals and
update.

Change-Id: I3011872f8d80a1519771f8423001650dcbcc7f5b
Fixes bug: 1169097
Fix key error when create lpar instance failed

With existing code, when create lpar instance
failed, it throws a key error because the parameter
is not being passed to PowerVMLPARCreationFailed
function in exception module.

The fix for this is that in module operator, passing
the 'instance_name' to the called function in exception
module.

Fixes bug 1173303
Change-Id: I688eecf7b649b67ca62f72d6d33fd7cd774047c3
Prevent rescuing a VM with a partially mounted volume.

If a VM goes into rescue mode with a partially mounted volume, the
volume won't re-appear after the VM is unrescued.

Fix bug 1158942

Change-Id: I1e104236c41c59e67a0f0e9ef26143c57f6e0094
xenapi: Don't swallow missing SR exception

The existing code would catch a missing SR exception and bail out of the
`update_status` method before finishing, causing required keys to be missing.
The end-result was code breaking later on when it tried to access these keys.

Since an SR is required to generate these required stats keys, we shouldn't
swallow the StorageRepositoryNotFound exception and instead should fail-fast.

Fixes bug 1175357

Change-Id: I3d65ae807fb2d86277bc0061c1967a87792b6662
Call format_message on InstanceTypeNotFound exception

Put the exception message from InstanceTypeNotFound into the
HTTPBadRequest rather than the exception itself.  Prevents a Circular
reference error during serialization later.

Bug 1175414

Change-Id: I4811e727311af670cd964a6d16ac5fce0fe3186e
Checks if volume can be attached

This patches makes a boot fail fast if one of the volumes cannot
be attached at boot time. Currently the request is accepted but
spawning fails in addition to leaving the volume state as
available which is an inconsistent state

Change-Id: I35b2034f8aa9574fcb8a34b58ead9f5e9ef27664
fixes: bug 1166770
Merge "Rename _check_image_size to _get_and_check_image_metadata"
Move db.fixed_ip_* tests from DbApiTestCase to FixedIpTestCase

During bp db-api-tests we will create for each model
ModelNameTestCase that contains tests for all methods that works
with this model..

So we shouldn't have DbApiTestCase at all. This patch will move
from DbApiTestCase methods that tests fixed_ip_* methods to already
existing FixedIpTestCase and a little bit improve it.

blueprint db-api-tests

Change-Id: I914c448d26c2b04a9fc3aa9614b7adf6f35b54d0
Merge "Call format_message on InstanceTypeNotFound exception"
Merge "Volume IO usage gets reset to 0 after a reboot / crash"
Merge "Fix building quantumapi network model with network list"
Merge "Make _build_network_info_model testable"
Merge "Move db.fixed_ip_* tests from DbApiTestCase to FixedIpTestCase"
Merge "Checks if volume can be attached"
xenapi: Always set other_config for VDIs

The existing code only set `other_config` on a VDI when it was created using
`create_vdi`. This patch updates the code so that `other_config` is also set
when a VDI is created from a dom0 plugin or during a migration.

Also included is a one-time script to set the other_config for existing
VDIs that were affected by this bug.

Fixes bug 1162029

Change-Id: I4fa856754487f77ce9c8e39d45eee7ea5187123e
Merge "Prevent rescuing a VM with a partially mounted volume."