Downgrade MySQL to the same state it used to be

virtual_interfaces_instance_uuid_fkey exists as a KEY, not a UNIQUE KEY in
MySQL which causes trouble when downgrading.

Don't re-create the virtual_interfaces_instance_uuid_fkey on downgrade as it
wasn't upgraded properly and therefore still exists. This forces the downgrade
to look exactly like it was before the upgrade.

Fixes bug 1207344

Change-Id: Id1700e1cbba669df2dd8e5c5a842b859577aa32c
Remove the indexes on downgrade to work with MySQL

Fixes MySQL downgrade on migration 203.
Add tests to check the indexes.

Fixes bug 1207174

Change-Id: If50fb49e1c76c2a7512141dae5f1a5818dc1b699
Fix migration downgrade 146 with mysql

Add availability zones back in individually to work with MySQL downgrades
on migration 146.

Fixes bug 1207956

Change-Id: I48b3ce27b92f231e280ee5af7a3e1b8ce6c4b018
VMware: fix rescue/unrescue instance

Fixes bug 1199955

The problem is due to the way in which the VM's are named on vCenter.
That is, in Nova a VM has a name and a UUID. In vCenter the VM only
has a name. The VM name on vCenter is Nova's instance UUID. When the
rescue/unrescue operation takes place a new VM is spawned/destroyed.
The Nova VM has the name '<name>-rescue'. The problem is that the
new rescue name was not passed correctly to the vCenter, that is,
it should have been '<uuid>-rescue' but it was just '<uuid>'. This
caused the opertaion to fail as there is already a VM with that name
on vCenter. A new name should have been passed which is '<uuid>-rescue'.

Change-Id: Icfe6e5021b29026788bec8f3526f94a8ec0a65e2
Remove unsafe XML parsing

Move to using xmlutils for XML parsing. Fixes missing patch
for the V3 API in commit 0ef2c55d804e0dab58cb5419a0cafd6a805315c7
and fixes bug 1210529

Change-Id: I6dc6111844277565d075a4ff5a08fdee5bc99344
Fix instance_group_delete() DB API method

Table 'instance_groups' has two columns that can be used to
unambigously identify the specific row:
  - 'id' (Integer, primary_key=True, autoincrement=True)
  - 'uuid' (String)

The former is used internally to bind related instances by FKs
(InstanceGroupMember, InstanceGroupPolicy and InstanceGroupMetadata),
and the latter is accepted by public methods of DB API. This must
be a miss in DB schema design, because 'uuid' could be used for both
use cases.

instance_group_delete() deletes the instance group (and all related
instances) given its UUID value. When related entities are deleted
they are filtered by the FK value - the 'group_id' column, however,
the 'group_uuid' value is passed.

Such DELETE statement is executed successfully (though WHERE clause never
evaluates to True, so the query is logically incorrect) on MySQL and SQLite,
which are often too 'soft' when checking data types of passed values.

This patch fixes the instance_group_delete() method and renames the 'group_uuid'
argument of _instance_group_model_get_query() to 'group_id', because this one
is actually always called with id value rather than with uuid.

Closes-Bug: #1211307

Change-Id: Ic8bcb4b218f570f26420904aeb74eb14567235fb
Fix ReservationTestCase on PostgreSQL

setUp() of ReservationTestCase calls quota_reserve() passing
timeutils.utcnow() as a value of 'until_refresh' argument.
until_refresh column is actually of type int rather than
datetime or varchar, so these tests fail on PostgreSQL.

Blueprint: db-api-tests-on-all-backends

Change-Id: Ib57579db2080f44bd41b91e83b0eee6f77f51840
Allow swap_volume to be called by Cinder

This allows Cinder to call swap_volume as part of a volume migration.
Cinder creates the new volume and calls Nova to perform the swap. When
Nova finishes, it calls Cinder to complete the migration, where it
updates the volume metadata so that the volume ID remains unchanged.
Thus, if Cinder detects that the migration was initiated by Cinder, it
will return the original volume ID to keep using. Otherwise, it will
return the new volume's ID for Nova to use, which is what it had done
before.

Change-Id: If352f2e11701148a5aeccaf7fd6379528fa8be31
Remove "N309 Python 3.x incompatible construct"

This check was implemented as H231 in Hacking 0.6 or newer
(I0bd89fa05d88de34ca62ce59f427468ede226126),
so the duplicate implementation can be removed.

Change-Id: I003d8611621adc53f1afcfd77691f345397670b0
Fix exception handling in V3 API coverage extension

Commit 616098dcd36b20e01d38898b8942003df664e6ac fixed the exception
handling in the V2 version of the coverage extension so backtrace
information is not lost when an exception is reraised. This applies
the same fix to the V3 version

Change-Id: I29ec1492c2def9c60c8ff471683914ae2943966e
Upgrade to Hacking 0.7

There was a bug in hacking 0.6 that broke H202, assertRaises Exception
too broad, so switch to Hacking 0.7 and fix the one H202 bug.

Change-Id: Ia3fb22d1524022479d7068cb84a3bdf7ca19f408
VMware: Added check for datastore state before selection

The driver only checked for the datastore type. The datastore could
be in an inaccessible state. In such cases creating an instance to
such datastore will fail. The fix to filter out inaccessible data-
stores.

The datastores have a property called accessible. The fix is to use
this property to filter out inaccessible datastores.

fix bug #1194078

Change-Id: I372fc3adc0e89c48d86f1f70164263ed35184c5d
Fix ratelimiting

Current master does not respect ratelimiting, since parsing of the
api-paste.ini is faulty. api-paste.ini limited user limiting by
setting a line as follows:
user:<user-id>:(GET, *, ".*", 4, minute)

This was passed to the Limiter as kwargs with "user" as a key.
Thus multiple user limiting is not possible as well as extracting
the id of the user was bound to fail, since we checked on the key
with startswith("user:")

An example config in the api-paste.ini has to look as follows:
limits = (POST, "*", .*, 10, MINUTE)
limits.<user-id1>:(GET, "*", .*, 4, minute)
limits.<user-id2>:(GET, "*", .*, 2, minute)

The code has been modified to be able to handle such configs

Fixes bug: 1210869

Change-Id: Iaec669b282a567d42fbe21a6f64e7bc99161053b
Stub out entry points in LookupTorrentURLTestCase

Torrent URL lookup functions are registered via entry-points.

If the tests run in an environment where the production lookup function is
present, it will use that; not what we want for tests.

Instead, we should stubout entry-points so we have a mock URL lookup function.

Fixes bug 1211396

Change-Id: Ibc2b34bfdcecaad4c53fb00e925a8b21965d8577
xenapi: Added iPXE ISO boot support

This patch adds support for a new kind of Glance image, a specially crafted ISO
which supports iPXE booting, giving customers a means to roll their own image.

Two virt-layer modifications were needed. The first was adding configurations
for the iPXE ISO feature (network to use, boot menu, mkisofs_cmd). The second
was the ability to inject networking info into the ISO after it was
downloaded.

To use this feature, operators should enable the `ipxe_boot` image-property.

DocImpact

Implements blueprint xenapi-ipxe-iso-boot-support

Change-Id: I33acf9dfdff0a5ed9797723a142bc451348e8549
Updates tools/config/README

The sample command line lacks the arguments required for executing
the script. Added the command line used in check_uptodate.sh

Fixes bug #1204204

Change-Id: I475491872577632c472695a3b1089caaf1f27d7c
Add new compute method for building an instance

The run_instance method on the compute manager handles rescheduling of a
failed build by casting back to the scheduler.  In order to maintain
backwards compatibility while modifying the build process to go through
the conductor a new method is being added to co-ordinate building an
instance.

This is one part of a series of patches to build up the
build_and_run_instance method.

The conductor will be switched over to calling this method once it
reaches a level of consistency with the current run_instance method.

Part of bp query-scheduler

Change-Id: I879805efa4a0a045082c02b10926e1278136b72c
nova.conf configurable gzip compression level

Currently only used by XenAPI.

Sets GZIP environment variable before spawning tar -cz to force the
compression level. Defaults to none, which means the GZIP environment
variable is not set and the default (usually -6) is used.

Implements: blueprint image-compression-mode

Change-Id: I02e136b7215a4f2cefc259f40bde9c1b205737ed
Fix Fibre Channel attach for single WWN

The code allowed for only a string of the WWN or a list of them.
Unfortunately unicode is also returned, for which the attach fails.
This patch allows for unicode as well.

Change-Id: I1d81abc3f8569ce8b3fb25c0e9c8aefd10e9aae8
Fixes: bug 1214413
Fixes sync issue for user level resources

fixed/floating ips and networks can't be synced properly because they're
tracked per user in the quota table but they're not tied to a user so
it's not feasible to grab a count of how many are being used by any
particular user. So the quota_usages table can get into a bad state if
it sync with user level.
Sync these resources with project level, and set quota with project
level to solve the problem.

Fixes bug 1214523

Change-Id: Icd94e95fb706ee90e0737e85cfa50f05ff3c0ff8