Fix instance can not be deleted after soft reboot

The reason is that:
1. soft reboot will wait for instance become to 'shutdown', and then start it
2. delete operation also wait for this, and then clean up the instance
3. if soft reboot found the instance become to 'shutdown' firstly, it will
start it immediately
4. then the delete operation will go to the _wait_for_destroy loop, and the
loop may be endless
5. when we delete the instance again, because the lock was hold by the delete
operation before, this one will wait the lock and don't implement actually.
So the domain id is checked during _wait_for_destroy loop, if it changed and
the instance is still running, we need to destroy it again.
If the domain is booted after _wait_for_destroy, it may result in
unfilter_instance failed because the nwfilter is in use, so doing the same
thing as above.

Fixes Bug #1111213

Change-Id: I98dc902dd86fa828f5821465c611953e08f9f637
Prevent the unexpected with nova-manage network modify.

"nova-manage network modify" can misinterpret arguments if the user
provides values where they are unexpected, specifically the
--disassociate-project and --disassociate-host options. If the user
provides a project or hostname value for either of these values, it will
be interpreted as a positional argument and modify handler may behave as
if other options were provided (e.g. project or host).

This change handles the presence of a project or host option along with
--disassociate-project or --disassociate-host as an error, printing a
message and exiting immediately with an error code. While there are
other approaches (e.g. perform the disassociate modifications while
ignoring the others), preventing unexpected or inconsistent results by
doing nothing seems safest.

Change-Id: Iff7817fe3ed19a4ff330b4029f9c558d3b405e9b
Retry bw_usage_update() on innodb Deadlock

Adds a new decorator _retry_on_deadlock() to sqlalchemy api.  This patch
makes bw_usage_update() use it.

Fixes bug 1129622

Change-Id: I0293c62d2dd5ac036445bc639cabbd05ba016e83
Add an extension to show image size.

Adds image size data that is returned from Glance into the API response.
This becomes really useful for users when they take a snapshot and want
to track the storage used by it.  Since this information is already
being pulled by Nova we should display it.

DocImpact
Change-Id: I8a2d4001c21bcc54cac1a2992034dfc9fbe39d7e
Fix instance directory path for lxc

Commit 12b8bd380e22a40e10e9d2758f5a0d594517de51 triggers a regression
when starting a STOPPED LXC instance. Instance referecne is needed to
correctly setup container rootfs path.

Change-Id: Id55b7f3a631e80a0da3f52c55784414105a61142
Fix pep8 issues with test_manager.py

Fixes E231 (missing whitespace) problems with
nova/tests/network/test_manager.py

Change-Id: I99d4c3bf7ed1a137fbfb26e9fe2f31d9104d5ae4
More gracefully handle TimeoutException in test

Sometimes although the test_killed_worker_recover test passes, the
cleanup of a forked child takes too long and a TimeoutException
occurs. This patch better handles the TimeoutException and
ensures the child process is killed.

Fixes bug #1128574

Change-Id: I39ea2fbb6c9cb95ad10b68bfa216686467951a93
ensure we run db tests in CI

CI had a set of environmental variables that refused to allow the
tests from failing gracefully. These seem to have gotten lost
along the way in refactoring.

Change-Id: Ib19b5d472c51ed72b8acf1571dd19f41474532c5
Refactor nwfilter parameters

* libvirt < 1.0.1 does not allow dynamically updating filter
  parameters inside an interface block of theinstance XML. To
  address this we move the nwfilter paramters from the instance
  XML to the per-instance base nwfilter.

Relates to bug #1124722

Change-Id: I50ec5dcffa8ff0232ae0aba0a08fce5151962238
Make add-fixed-ip update nwfilter wth in libvirt

Add inject_network_info to the libvirt driver  so that the
add-fixed-ip call will trigger a refresh of the nwfilters.

Change-Id: I00eccf34b8f92fb1e26a72e4bddb47dd39c4558f
Fixes: bug #1124722
Sync rpc from oslo-incubator.

This includes the following changes:

 * use oslo.config directly
 * implementation of rpc-amqp-fast-reply-queue
 * simplifications to bin/nova-rpc-zmq-receiver
 * add new join_consumer_pool API call
 * a bug fix to ensure kombu messages get acked even when an exception
   occurs in the message callback

Change-Id: I813ce0ee7d5442d7b5b0f9d477949adabc83174e
Add API Sample tests for Hypervisors extension.

Partially implements blueprint nova-api-samples
Fixes bug 1091750

Change-Id: Ie8e6881ea3533cd43259548594d3debe377345a5
Removed duplicate spawn code in PowerVM driver

After introducing resize/migrate in PowerVM there was
some code that was duplicated after a rebase. The commit
removes the duplicate code. The functionality of
resize/migrate remains the same as introducted in
Change I72345626

Bug #1130680

Change-Id: I1fe3d8e1f2a19fd40a03c8c38d90c01711356b38
Allow exit code 21 for 'iscsiadm -m session'.

Updates the libvirt volume driver so that it properly handles
exit code 21 in the active session check. This fixes issues
in running the command on Fedora 17.

This behaviour was broken in 933365a (multipath libvirt iscsi).

Fixes LP Bug #1130718.

Change-Id: I80de4c09ba857d550734f9bd0dfc0c40bfd5036a
Improve I/O performance for periodic tasks

Unfortunately, periodic tasks to update local resource and
driver status come with performance penalties, especially on LVM
image backend type.

This is because low level implementation relies 'vgs' command to
query LVM info, which itself has a I/O performance price and
extravagant use of local_gb_xxx query make things even worse.

This commit centralize such query in one single call
(volume_group_info) and reduce invocation times in periodic tasks
accordingly.

Partially resolves LP #1130697

Change-Id: I165f96dd3e4fdad770b8a9643f5a038d353312ff
Fix handling of source_groups with no-db-compute.

The moving of data around in no-db-compute broke source groups. This
is a combination of a few errors. First, the instance_type data wasn't
being retrieved from system_metadata. Second, the instance_type data
was too deep in the heirarchy and being ignored. Finally, source
groups require a nw info call which causes a db access by
nova-compute.

Fixes bug 1122316

Change-Id: Iccb6e5d336c0b2c8ba41c41ab2e046fc9617835a
Merge "Allow exit code 21 for 'iscsiadm -m session'."
Merge "Sync rpc from oslo-incubator."
Merge "Add API Sample tests for Hypervisors extension."
Add better status to baremetal deployments.

This patch introduces a few new baremetal states, which are used to
track the deploy process. Now, nova-baremetal-deploy-helper updates the
bm_nodes record directly when it begins and finishes deploying an image
to that node.

The next patch will add a LoopingCall inside driver.spawn() to wait for
the deploy to complete.

Also, since there can not be >1 active deployment per node, there
is no need to have a separate table for storing them. This patch drops
the table bm_deployments and adds the important information it contained
to bm_nodes. Since the previous behavior was to mark a deployment as
deleted once it completed, there is no need to copy any data from
bm_deployments prior to dropping the table -- assuming that no active
deployments are in process when the migration is run.

Since this is the first migration for the baremetal database, it also
adds a new test class, TestBaremetalMigrations, and refactors the
test_migrations.py file to allow for multiple test classes.

partially implements fix for bug 1096723

Change-Id: Iad30b462d49c88fc19babed43a2fb8540b1fad30