Let resource_tracker report right migration status

A shutoff instance can be resized, but resource_tracker only think
active instance is in migration state. This patch let it report
right migration status.

Closes-Bug: #1247610

Change-Id: Ic6ce9c7d8fdecaf812831eb302186133f0ee71b1
baremetal: Make volume driver use a correct source device

attach_volume() have to take the source device path from returning of
LibVirtVolumeDriver.connect_volume() rather than connection_info.

It also adds a test for attach_volume().

Change-Id: I3bee80560342104e2df614066301171624497a04
Closes-Bug: #1208707
Fix limits v3 follow API v3 rules

* Add expected_errors decorator.
* Remove 'absolute_limit' from response, that already return by
extension quota_sets. And update api samples.

Partially implements bp nova-v3-api

Change-Id: Iea3ff494b8abed77fdc4b9e5e552784ed3733c34
Use 400 instead of 422 for invalid input in v3 servers core

We return 400 when input is invalid for v3 api. This patch fix
this case for servers core.

DocImpact

Change-Id: If802d4dcba6ed25f7e9cb9d568a3ffd78f0f6c68
Closes-Bug: 1251538
Fix sample parameter of agent API

The parameter of agent update API has been changed from "para" to
"agent" since Nova v3 API, and this patch fixes the sample doc.

Partially implements blueprint v3-api-specification

Change-Id: Ic9c0f11a637e48daa9457ab9ae923fefc0fb350d
Fix empty selector XML bug

The XML serialization fails when it's trying to create a datum selector with no
values. In that case the obj is an empty string so the selection of datum from
it just fails.
Instead of raising an exception the fix is returning an empty string, that
allows to serialize Selector with an empty object.

Fixes bug 1223358

Change-Id: If5d564c9b037eb0a107f4688ffa1aceb5281dc42
VMware: Always upload a snapshot as a preallocated disk

Currently, a snapshot operation always produces a preallocated
disk type. When the snapshot is uploaded to glance, the source
image metadata is uploaded with the new snapshot and then some
of the metadata properties are updated. The update doesn't
include the disk_type property, this can be problematic. If
the original image has a sparse disk type and a snapshot is
taken, then the snapshot which is a preallocated disk type is
uploaded to glance with the disk_type property set to sparse,
which creates a disk type mismatch and causes errors when a vm
is spawned from that snapshot. This patch addresses the issue
by correcting the image metadata used when uploading a snapshot
to glance.

Also, the image_upload function in vmwareapi/fake.py has been
modified to address the change. Any function call to
image_upload (from the unit tests) will assert that the
metadata for the image to be uploaded must have the disk_type
value to be preallocated.

Closes-Bug: 1247296
Change-Id: I71cfa8d4d1eb7911abb489153292fb0d1e614765
Clean up how test env variables are parsed

Oslo and cookiecutter and other projects were using
variable to parse test environment vaiables. Let us
do this in Nova

This is copied from 739627bf79619f8 Use cleaner version from
cookiecutter OpenStack template

Change-Id: I4c2b4072981985e7632fc775ffc98216fbfc0508
Removes os-personalities extension from the V3 API

As per the discussion on the mailing list here:

http://lists.openstack.org/pipermail/openstack-dev/2013-September/015490.html

and discussions at summit, this patch removes the os-personalities
from the V3 API.

DocImpact

Part of blueprint v3-api-remove-extensions

Change-Id: I1a57ca812a9d71c0b1bab5d16ffad8eb82cba539
Remove traces of V3 personality extension from api samples

Removes occurences of the personality extension from the V3 API
samples as the extension has been removed. Cleanup of consistency
of image and flavor ref examples for the api samples.

DocImpact

Partially implements blueprint v3-api-remove-extensions

Change-Id: I43a34097904d3d7e280039c3d5d76479a49e92d0
fakevirt: return hypervisor_version as an int instead of a string

The compute_nodes.hypervisor_version type in the database is an integer
not a string. This is only caught in postgres and not MySQL. Changing
the fakevirt driver to return the proper type allows for the
test_virt_drivers test to make check this for all drivers it currently
tests.

Change-Id: I25e4dba855b954a2451bced9a53172b51eb47238
Change evacuate test hostnames to preferable ones

The hostname grammar in RFC952 does not allow for underscores in
hostnames. However, some evacuate tests use them in each hostname.
This patch changes them to hyphens instead.

Change-Id: Id961e74a0bbeefbc843678edd74c02466b6d6885
Compact pre-Icehouse database migrations <= 150.

Compacts the pre-Icehouse database migrations <= 150 into a
single migration (150_havana.py).

Pre-Icehouse users will now need to upgrade to Havana before
running any Icehouse migrations.

Tested on MySQL and PostgreSQL schemas on Fedora 19 to ensure the
new compacted migration generates an identical schema to the
previous migrate scripts in Havana.

Specific changes include:

 1) Make migrations.instance_uuid a String 36.

 2) Add migrations_instance_uuid_and_status_idx index.

 3) rename instance info caches seq (postgresql only, dropped from
    baseline)

 4) Add source_node, dest_node columns to migrations. Indexes...

 5) Add new volume usage cache table.

 6) Drop aggregates.availability_zone column.

 7) Drop services.availability_zone column.

 8) Add instance actions tables.

 9) Convert PostgreSQL to INET.

 10) Add host to instance faults.

Implements: blueprint compact-havana-db-migrations

Change-Id: Ic0c318967b69af32536b5ab6999affc7755ec442
Compact pre-Icehouse database migrations <= 160.

Compacts the pre-Icehouse database migrations <= 160 into a
single migration (160_havana.py).

Pre-Icehouse users will now need to upgrade to Havana before
running any Icehouse migrations.

Tested on MySQL and PostgreSQL schemas on Fedora 19 to ensure the
new compacted migration generates an identical schema to the
previous migrate scripts in Havana.

Specific changes include:

 1) Make task_log period_begining, period_ending a DateTime.

 2) Change type of deleted column.

 3) Create shadow tables.

 4) Add task log unique constraint.

 5) Add security_group_default_rules table.

 6) Add networks unique constraint.

 7) Revert ip column length.

NOTE: This patch removes the 184_sqlite_upgrade.sql and
184_sqlite_downgrade.sql migrations since these were causing
new test failures and the standard DB migration seems to take
care of it.

Implements: blueprint compact-havana-db-migrations

Change-Id: If977e5f02fffbfb1bc257b3beddf2d7133e6c7b9
VMware: Fix unhandled session failure issues

VMware driver has a re-try mechanism to handle session expiration
failures. Due to a minor bug in the exception handling module, this
failure was unhandled.

The patch fixes this issue and has added tests.

Closes-Bug: #1252827
Change-Id: Ie91adb4b4b57b7cefeed855cdbe4710da86294f0
Document when config options were deprecated

When going through the code and looking for what code was deprecated and
when, its easier to know if there is an explicit comment. Otherwise one
has to trace through 'git log' and 'git blame' to be 100% sure that its
ok to remove the code.

Change-Id: I57b5861e57168c44729934757e732de962f3d91a
Clean up test cases for compute.manager._check_instance_build_time

This patch cleans up the unit tests related to the
_check_instance_build_time method so that they are actually verifying
the expected vm_state value when the instance is updated.

This patch also removes the test_instance_build_timeout test case
since it was completely redundant with the other one modified
here. I checked the git history and both test cases were added
in the same commit which confirms that they weren't added for
different reasons, i.e. separate bug fixes/use cases.

Change-Id: Ib384c84e2954bd303661386256053b3c1ad38239
Fix HTTP response code for network APIs and improve error message

This patch corrects the http response codes for network resources to be
consistant with other resources by returning a 400 instead of a 404
if the desired resource is not found.

This patch makes the following api changes:

v2-api: NetworkNotFound now returns 400 instead of 404 and improved error
        message which network was not found instead of
        'The resource could not be found'.

v3-api: PortNotFound and NetworkNotFound now returns a 400 instead of a 404.

Closes-bug: #1245696
DocImpact

Change-Id: I66eb0c0ab926e0a8d1e2c9cfe1f7fd579ea3da21
Don't log stacktrace for UnexpectedTaskStateError

An UnexpectedTaskStateError is actually an expected behavior and we
don't need to log a stacktrace every time it happens. This stacktrace
is one of the most common ones found in nova-compute during gating jobs.

Change-Id: I8b33f72dc0f4beaca969de084481d66987faa974
Configuration element for describing video drivers.

The configuration is needed to allow the users to select
a non default video driver.

Partially implements blueprint libvirt-video-driver-selection
Change-Id: Ic6b3e44cfb7e22e0a215f7b159963df35ee1ebac