Enforce sqlite-specific flow in drop_unique_constraint

The nova.db.sqlalchemy.apy.drop_unique_constraint method has a
sqlite-specific flow for dropping unique constraints because it can't
use alter table like mysql or postgresql. The code is explicitly
checking if the db engine is mysql or postgresql and then falls into a
block which assumes the backing db is sqlite. This can cause problems if
the backing db is not sqlite.

This patch simply switches the checks so that the sqlite-specific flow
is only ran if the backing db is sqlite, everything else will default to
the 'alter table' flow.

Change-Id: I23b9bf33d18df3ac36106d5800a8ecdd1a4018df
libvirt fix resize/migrates with swap or ephemeral

Fixes bug 1191608

The code was not properly guarding against backing files not
existing with QCOW2 images. As such, get_disk_backing_file() would
return None, then the subsequent call to basename() on None would
cause an exception.

This bug was introduced with the fix for bug 1185588 and commit
6e768363.

Change-Id: I4b1237ddf01b9398bfdc141180260136b36cae5a
Don't deallocate network if destroy time out

Fixes bug #1169057
Sames as bug 1180178, if libvirt returns time out error during destroy
an instance, we also can't deallocate the network of the instance,
otherwise the fixed ip of it may be re-used by another new instance,
and resulted in the new instance can't be accessed by the fixed ip.

Change-Id: I04dfd21bb1b7168abc71f39e836126a11dad6d8d
Add troubleshoot to baremetal PXE template.

We recently added an optional troubleshoot parameter to
diskimage_builder which defaults to 0 (false).

This commit adds the option to the Nova pxe_config.template
so it will be easier for people to find should they need to
enable troubleshooting mode.
Change-Id: I169a5e3b7a9d80e18e389ae4645d2c5bbdae2a6f
Preserve network order when using ConfigDrive

Pass network_info to be used by InstanceMetadata instead of fetching
it by API and losing originally requested network order. This is
similar to what has been done for "content" where the original data
is lost after the initial call.

This issue only affects the ConfigDrive code path as the original
network_info is used when using file injection.

HyperV and XenApi are still probably affected by the bug but
unaffected by this fix. The fix is the same for these drivers. Simply
pass a network_info and it will be used instead of fetching the
network info by API.

Change-Id: Ie673b725cb47bf491009db99f6cb1258d46b0a69
Fixes: bug #1156844
Revert "fixes nova resize bug when force_config_drive is set."

This reverts commit 8ae97f24b0c806a438456170959ae527ce33ad01

This issue was already tracked and fixed in bug 1163598.
The user's packaged version was just lagging behind the released fix.

Change-Id: I9260ce7b00b0500b168f5a385696361b4f8e73fd
Clarify the compute API is_volume_backed_instance method

Update the is_volume_backed_instance to be more robust and
check for volume_id or snapshot_id to see if a block device
mapping is really backed by a cinder volume. As per the blueprint
https://wiki.openstack.org/wiki/BlockDeviceConfig#snapshot_id_and_volume_id
I used the volume_id to do this as seems consistent with the
rest of the tests.

Add unit test to enforce this.

The motivation behind this is that we rely on this method at HP and
the test ServersActionsAllJsonTest.test_server_create_image
in nova.tests.integrated.test_api_samples highlighted weakness in
this method and where it could fail now that every instance has an
associated block device mapping for its root device.

Change-Id: Ia4b623a94d0285d8aeb97e9fd72a63f5f87efe7a
Avoid redefining host to none in get_instance_nw_info(...)

The host parameter is required and a part of the method signature.
When it is redefined from kwargs.get('host') the supplied value
gets redefined to None. This removes the redefinition and keeps the
parameter value intact.

Fixes bug #1194178

Change-Id: I779cbce909935a4b794c16cefb0128f16f8563fb
Update pyparsing to 1.5.7

The version of cliff we are using depends on pyparsing>=1.5.7 but
python-quantumclient pulls 1.5.6 because it appears first in the
requirements processing.

The requirements change is in
    https://review.openstack.org/#/c/33406/

Fixes bug #1191866

Change-Id: Ib1c11b53813ff96b750f5ffe4997ad01907d32c3
Sync rpc from oslo-incubator.

The most significant change included here is:

6d42ced Remove the amqp_rpc_single_reply_queue option from Havana

Change-Id: I4322a8dbfe9b666cd741c3c520a192296c56cd7b
Exeption message of 'live migration' is not appropriate

Fix bug 1194367

When do live-migration with RESTAPI body as:
{
    "os-migrateLive": {
        "host": "0443e9a1254044d8b99f35eace132080",
        "block_migration": false
    }
}

Customer would get the following exception:
{
    "badRequest": {
        "message": "host and block_migration must be specified.",
        "code": 400
    }
}

Update the exception to the following which is more accurate:
host, block_migration and disk_over_commit must be specified.

Change-Id: I87fbc8e9d287cd73084ae5e36dd7d6c0bcaec41c
Sync models.Console* with migrations.

Blueprint: db-sync-models-with-migrations

Change-Id: I2cceab6484b6d69b3f1aa1492e49d76a765b758d
Make sure periodic cleanup of instances continues on error

If an instance cleanup fails with an exception, no more instances
will be cleaned up. If the failure is permanent, this will block
other instances from being cleaned up for as long as it remains.

Resolves bug 1192712.

Change-Id: Ibdd69817ee0e6d0963637f38a8be3949e6e053ed
Make sure periodic instance reclaims continues on error

If an instance reclaim operation fails with an exception, no more
instances will be reclaimed. If the failure is permanent, this will
block other instances from being reclaimed for as long as it remains.

Resolves bug 1195029.

Change-Id: I0379d16a1f74344ef44c843ce19a505e31e79bfa
Remove locals() from compute directory

Replace locals() with dictionary of the values to be printed

Change-Id: Iec926e859a5b340a04d9ada7c770b58eecd450b6
Sane rest API rate limit defaults

The previous rate limit defaults were unusable in any deployment.
Rate limiting to 10 POSTS per minute and 50 servers per day seems
to low, especially when we can use quotas to actually limit the amount
of resources a user can consume.

Update docstring to explain what the rate limiting is used for.

Fixes bug 1178529

DocImpact changed default values

Change-Id: I8cc93423f76d9b0a5135adf69babc4ff355a0951
Port consoles extension API into v3 part1

This changeset only copies the v2 files into the appropriate v3
directories unchanged. The copy as-is will not be loaded by either
the v2 or v3 extension loaders. The second changeset will then
make the changes required for it to work as a v3 extension.

This is being tried in order to make reviewing of extension
porting easier as gerrit will display only what is actually
changed for v3 rather than entirely new files.

Partially implements: bp v3-api-extension-versioning

Change-Id: I60fd601e28ed69c2597cfc8742b2e17e706e2a1f
Port consoles extension API into v3 part2

This patch contains the changes required to adapt the consoles
extension and the corresponding unittest to the v3 framework.

Partially implements bp v3-api-extension-versioning

Change-Id: I1f5aa7e1a7f9e6d959bb16ea6c4c3d2cd17958b9
Notification support for host aggregate related operation

Add notification for the following operations related to host
aggregation:
1) Send notification when create a new host aggregation
2) Send notification when delete a host aggregation
3) Send notification when add host to a host aggregation
4) Send notification when delete host from host aggregation
5) Send notification when update host aggregation for aggregate
name and available zone
6) Send notification when update host aggregation meta data

Change-Id: I2dc42addb587c4765b41e464697f35b1f844fde9
Implements: blueprint notification-host-aggregate
xenapi: Add configurable BitTorrent URL fetcher

Add configurable torrent URL fetcher for sourcing image torrents.

The code uses entry points to find a "driver" style function.  If no
driver is supplied, the torrent file will continue to be found at:

CONF.xenapi_torrent_base_url + "/" + image_id + ".torrent"

DocImpact

Change-Id: Ib2b90c32a5b2c20dd1863058cf1a523672cf2fb3