Enforce sqlite-specific flow in drop_unique_constraint

The nova.db.sqlalchemy.apy.drop_unique_constraint method has a
sqlite-specific flow for dropping unique constraints because it can't
use alter table like mysql or postgresql. The code is explicitly
checking if the db engine is mysql or postgresql and then falls into a
block which assumes the backing db is sqlite. This can cause problems if
the backing db is not sqlite.

This patch simply switches the checks so that the sqlite-specific flow
is only ran if the backing db is sqlite, everything else will default to
the 'alter table' flow.

Change-Id: I23b9bf33d18df3ac36106d5800a8ecdd1a4018df
libvirt fix resize/migrates with swap or ephemeral

Fixes bug 1191608

The code was not properly guarding against backing files not
existing with QCOW2 images. As such, get_disk_backing_file() would
return None, then the subsequent call to basename() on None would
cause an exception.

This bug was introduced with the fix for bug 1185588 and commit
6e768363.

Change-Id: I4b1237ddf01b9398bfdc141180260136b36cae5a
Port fping extension to v3 API Part 1

This changeset only copies the v2 files (implementation and test)
into the appropriate v3 directories unchanged. The copy as-is
will not be loaded by either the v2 or v3 extension loaders. The
second changeset will then make the changes required for it to
work as a v3 extension.

This is being done in order to make reviewing of extension porting
easier as gerrit will display only what is actually changed for v3
rather than entirely new files

Partially implements blueprint nova-v3-api

Change-Id: I55d15ab5f9f59a83dc67bac2d2f41ceee06085d5
Don't deallocate network if destroy time out

Fixes bug #1169057
Sames as bug 1180178, if libvirt returns time out error during destroy
an instance, we also can't deallocate the network of the instance,
otherwise the fixed ip of it may be re-used by another new instance,
and resulted in the new instance can't be accessed by the fixed ip.

Change-Id: I04dfd21bb1b7168abc71f39e836126a11dad6d8d
port Simple_tenant_usage  API into v3 part1

This changeset only copies the v2 files into the appropriate v3
directories unchanged. The copy as-is will not be loaded by either
the v2 or v3 extension loaders. The second changeset will then
make the changes required for it to work as a v3 extension.

This is being tried in order to make reviewing of extension
porting easier as gerrit will display only what is actually
changed for v3 rather than entirely new files.

Partially implements: bp v3-api-extension-versioning

Change-Id: I1618987cc905a6734b409eb9a7fe37c65b6aac28
Remove junk file when ftp transfer failure

During spawn with powervm driver, the image transfer to the IVM host
may fail in some conditions, eg. IVM host doesn't have enough disk
space. It leaves the junk file in IVM host, and this will disturb
next spawn after resolving environment issue, need clean it.

Fixes bug #1188884

Change-Id: I7b8d719995c2702c89cf55cef5f34bde9d01c1b9
Port hypervisor API into v3 part1

This changeset only copies the v2 files into the appropriate v3
directories unchanged. The copy as-is will not be loaded by either
the v2 or v3 extension loaders. The second changeset will then
make the changes required for it to work as a v3 extension.

This is being tried in order to make reviewing of extension
porting easier as gerrit will display only what is actually
changed for v3 rather than entirely new files.

Partially implements: bp v3-api-extension-versioning

Change-Id: I251dc18dda9756574c308a08423fd805a3e91182
Add troubleshoot to baremetal PXE template.

We recently added an optional troubleshoot parameter to
diskimage_builder which defaults to 0 (false).

This commit adds the option to the Nova pxe_config.template
so it will be easier for people to find should they need to
enable troubleshooting mode.
Change-Id: I169a5e3b7a9d80e18e389ae4645d2c5bbdae2a6f
Preserve network order when using ConfigDrive

Pass network_info to be used by InstanceMetadata instead of fetching
it by API and losing originally requested network order. This is
similar to what has been done for "content" where the original data
is lost after the initial call.

This issue only affects the ConfigDrive code path as the original
network_info is used when using file injection.

HyperV and XenApi are still probably affected by the bug but
unaffected by this fix. The fix is the same for these drivers. Simply
pass a network_info and it will be used instead of fetching the
network info by API.

Change-Id: Ie673b725cb47bf491009db99f6cb1258d46b0a69
Fixes: bug #1156844
Port extended status extension to v3 API Part 1

This changeset only copies the v2 implementation file into the
appropriate v3 directory unchanged. The copy as-is will not be
loaded by either the v2 or v3 extension loaders. The second
changeset will then make the changes required for it to work as a
v3 extension.

This is being done in order to make reviewing of extension porting
easier as gerrit will display only what is actually changed for v3
rather than entirely new files.

Partially implements blueprint nova-v3-api

Change-Id: Ia5b80b70a7a459bf973de1c1aea8dc650e763b8d
Revert "fixes nova resize bug when force_config_drive is set."

This reverts commit 8ae97f24b0c806a438456170959ae527ce33ad01

This issue was already tracked and fixed in bug 1163598.
The user's packaged version was just lagging behind the released fix.

Change-Id: I9260ce7b00b0500b168f5a385696361b4f8e73fd
Port extended-server-attributes into v3 part1

This changeset only copies the v2 files into the appropriate v3
directories unchanged. The copy as-is will not be loaded by either
the v2 or v3 extension loaders. The second changeset will then make
the changes required for it to work as a v3 extension.

This is being tried in order to make reviewing of extension
porting easier as gerrit will display only what is actually
changed for v3 rather than entirely new files.

Partially implements: bp v3-api-extension-versioning

Change-Id: Idc9eed087a82f897d504a5c79e520883e537d060
Clarify the compute API is_volume_backed_instance method

Update the is_volume_backed_instance to be more robust and
check for volume_id or snapshot_id to see if a block device
mapping is really backed by a cinder volume. As per the blueprint
https://wiki.openstack.org/wiki/BlockDeviceConfig#snapshot_id_and_volume_id
I used the volume_id to do this as seems consistent with the
rest of the tests.

Add unit test to enforce this.

The motivation behind this is that we rely on this method at HP and
the test ServersActionsAllJsonTest.test_server_create_image
in nova.tests.integrated.test_api_samples highlighted weakness in
this method and where it could fail now that every instance has an
associated block device mapping for its root device.

Change-Id: Ia4b623a94d0285d8aeb97e9fd72a63f5f87efe7a
Remove usage of locals() from powervm virt package

Fixes bug 1171936

Change-Id: I9ff8c392071102737c70a807f9bfc584729c4142
Fix the filtered characters list from console-log

In fffcaea1 ^M was added to the list of characters to filter from the
console-log output but, because of a typo in the regexp, the
character '-' was added to that list too and has been removed since
then.

Fixes bug #1194032

Change-Id: Ia02b0b4c7027cbd09a51042ca1b5d28890413626
Avoid redefining host to none in get_instance_nw_info(...)

The host parameter is required and a part of the method signature.
When it is redefined from kwargs.get('host') the supplied value
gets redefined to None. This removes the redefinition and keeps the
parameter value intact.

Fixes bug #1194178

Change-Id: I779cbce909935a4b794c16cefb0128f16f8563fb
Update pyparsing to 1.5.7

The version of cliff we are using depends on pyparsing>=1.5.7 but
python-quantumclient pulls 1.5.6 because it appears first in the
requirements processing.

The requirements change is in
    https://review.openstack.org/#/c/33406/

Fixes bug #1191866

Change-Id: Ib1c11b53813ff96b750f5ffe4997ad01907d32c3
Sync db.models and migrations.

This patch syncs models with migrations for:
- DNSDomain
- AggregateMetadata
- BandwidthUsage

Blueprint: db-sync-models-with-migrations

Change-Id: I09c48bac5ed780a7ec1b89cc8ce0684f4903d9dd
Skip security group code when there is no network.

This patch adds a simple check to the nova/quantum API that immediately
returns an empty list if there are no configured networks available.

Fixes bug 1191044

Change-Id: I34ac4dd56c71f99c7ed9ff632d6dc7036d91008d
Fix types in test_ec2_ids_not_found_are_printable

This test fails on PostgreSQL when trying to compare
the VolumeIdMapping.id column (Integer) with a passed
string value.

Blueprint: db-api-tests-on-all-backends

Change-Id: Ie529d25029667a5279f9c70f0f3f081b87fb6acb