Fix building quantumapi network model with network list

Bug 1171636 occurs when _build_network_info_model() receives a list
of networks that does not fully map all of an instance's ports.
It crashes in _ensure_requested_network_ordering() when the list of
all ports is sorted by the subset of networks provided.

This change simplifies the early part of that function to only
consider ports within the networks provided, and to always sort those
ports in network order. This allows us to remove some error path
code from the loop below that iterates over the ports (which contained
a fatal typo, by the way) which is now dead code.

The affected code is really untested at the moment, due to the
structure of the method, making it hard to add a unit test to cover
this fix. In order to make the change clear from the existing code,
this fix is provided here first, and the following patch restructures
the method and add tests for this change and the rest of the surrounding
code.

Fixes bug 1171636

Change-Id: Idf519a88415844451b18959b38cc48ce92c090d1
Make _build_network_info_model testable

This restructures the quantumapi _build_network_info_model()
method to be testable and adds tests for each of the smaller
components. It also provides verification for bug 1171636 by
ensuring that ports returned from the client that are not in the
set of networks passed don't cause the failure detailed in that
bug.

Change-Id: I82f20f68122fc38e72c31e52283529614252c174
Fix baremetal get_available_nodes

Baremetal driver get_available_nodes was incorrectly returning only a
list of unprovisioned nodes, leading update_availabile_resources to
sometimes delete a baremetal node that had an instance provisioned
to it -- sometimes even while the provisioning was still in process.

Fix bug 1174952

Change-Id: I4f08a2f6539a5b6cd12245ac6b4dc308767ae0cd
Update to using oslo periodic tasks implementation.

Convert nova to using the oslo periodic tasks implementation. There
are no functional changes in this review.

Change-Id: I767e0ad17781d5f9d5e987e0a4ad65796243ae5c
Convert the cache key from unicode to a string.

The memcached API expects string format, not unicode.
Convert the key passed to the memory cache to a string
before sending it to memcached. Since both the function
name and the id that form the key are ascii format, a
simple conversion from unicode to str is sufficient.

Change-Id: I9d15858ad4e10aa183444cccda9c33676059e55a
Fixes: bug #1174487
Rename _check_image_size to _get_and_check_image_metadata

This method has functionality which doesn't seem to match what
it actually does (which is getting image metadata and performing
basic checks on that data).

Change-Id: I818fb5079105cca8acbdf464d47a92f7b1ebb398
Sync jsonutils from oslo

This is a sync of commit 9dcf688ea80f52cdb5413514198b2aa81d5a4e09 from
the oslo-incubator project. It should improve the serialisation
performance in general and tests runtime specifically.

Change-Id: I6b0e7baeb24951ce3c0bc129d20c65daa0d8fa61
Sync small and safe changes from oslo

Ran the oslo-incubator sync script and brought across only the files
that have small and safe changes.

Change-Id: I41b400c189e52a2e7f642a03ad143d821aaa807c
Merge "Convert the cache key from unicode to a string."
Volume IO usage gets reset to 0 after a reboot / crash

Check for reset of block device stats before updating volume usage cache.

If an instance is reboot or crashes, the results of block device stats for
that domain get reset. Add check that if block device stats are lower than
the current stats in the database, then add current values to totals and
update.

Change-Id: I3011872f8d80a1519771f8423001650dcbcc7f5b
Fixes bug: 1169097
Merge "Fix baremetal get_available_nodes"
Merge "Update to using oslo periodic tasks implementation."
Reset migrating task state for MigrationError exceptions

Create an exception type for pre-check errors and add it to the list
of exceptions for which instance task state is reset to ACTIVE to
avoid cases where an instance get stuck in MIGRATING state.

Convert existing instances of raise MigrationError in scheduler and
driver pre-checks to have them raise the new MigrationPreCheckError
exception instead.

In addition, also make sure that expected exceptions are passed over
RPC silently.

Resolves bug 1171526.

Change-Id: I00fa0962e555f895dd98861684ec7767fed3b37b
Merge "Sync jsonutils from oslo"
Merge "Sync small and safe changes from oslo"
Merge "Reset migrating task state for MigrationError exceptions"
Prevent rescuing a VM with a partially mounted volume.

If a VM goes into rescue mode with a partially mounted volume, the
volume won't re-appear after the VM is unrescued.

Fix bug 1158942

Change-Id: I1e104236c41c59e67a0f0e9ef26143c57f6e0094
xenapi: Don't swallow missing SR exception

The existing code would catch a missing SR exception and bail out of the
`update_status` method before finishing, causing required keys to be missing.
The end-result was code breaking later on when it tried to access these keys.

Since an SR is required to generate these required stats keys, we shouldn't
swallow the StorageRepositoryNotFound exception and instead should fail-fast.

Fixes bug 1175357

Change-Id: I3d65ae807fb2d86277bc0061c1967a87792b6662
Call format_message on InstanceTypeNotFound exception

Put the exception message from InstanceTypeNotFound into the
HTTPBadRequest rather than the exception itself.  Prevents a Circular
reference error during serialization later.

Bug 1175414

Change-Id: I4811e727311af670cd964a6d16ac5fce0fe3186e
Checks if volume can be attached

This patches makes a boot fail fast if one of the volumes cannot
be attached at boot time. Currently the request is accepted but
spawning fails in addition to leaving the volume state as
available which is an inconsistent state

Change-Id: I35b2034f8aa9574fcb8a34b58ead9f5e9ef27664
fixes: bug 1166770