Fix migration 147 downgrading with no aggregates

Add some extra bulletproofing to downgrading migration 147 where
there may have been no aggregates.

Change-Id: I6e4aabf4fff462e9ad53083621077ce0e0a80a43
Ensure deleted instances' status is always DELETED

Currently when deleting failed instances that were never assigned a
host, their vm_state is left in 'error' instead of 'deleted'. This is a
problem when invoking the Index Instances REST API with a Changes-Since
query parameter since that API returns both deleted and non-deleted
instances, but currently the failed deleted instances are incorrectly
marked as status:ERROR instead of status:DELETED.

This patch modifies the server ViewBuilder to ensure the status is
correctly set.

Change-Id: I614714676bab4986564b236d74f2c0d77400e48c
Closes-bug: #1243301
(cherry picked from commit 38018d32f9db5b9222436d0f860244ce38ba97fb)
fix missing host when unshelving

An instance has the hostname of the host on which it is running.
The hostname is deleted when shelving. However, the hostname is not
recreated when unshelving.

This patch sets the hostname when unshelving.

Change-Id: I2e431718198321c46b9335e6fb7ab7be3943fab6
Closes-Bug: #1237868
(cherry picked from commit 9532e4ed969e47826801b14aff40dd9e356f2677)
Libvirt:Instance resize confirm issue against NFS

when I test resize-confirm on NFS (KVM) using at least two
compute nodes, OSError will occur when try to cleanup the
backup instance directory and will make instance status
"Error".

One way to cleanup resize is using delay_on_retry and
maximum attempts setting. Replace rmtree() because
execute() have more parameters choices.

Closes-Bug: #1248019

Change-Id: Ifb9a500bbba805af0317307c2e7d6903dcd02ad1
(cherry picked from commit 17d131661740e452e9729d9ac8881e6fede23dd7)
Make non-admin users can unshelve a server

If non-admin users try to unshelve a server, nova-compute fails due to
"User does not have admin privileges" while getting hypervisor_hostname.
This patch changes the user context to admin's one temporarily to get the
hypervisor_hostname.

Closes-Bug: 1245312
Change-Id: I84dce1f7694d09c1281da13c6008bb3aa5940f85
(cherry picked from commit 13fa2caaad796ee66c432e0d19fd6f759234a964)
Fix docstring for disk_cachemodes

The docstring for the disk_cachemodes option implied a syntax that is
incorrect for a ListOpt.  Change the example to match exactly what you
would put in the config file for it to work.

Change-Id: Ie999d2799dba6c47facdc771e1cf6081d28d6b85
Closes-Bug: #1106423
(cherry picked from commit 1df9b26b51ff3a153b93d79e25bef454e08a8e38)
Fix difference between mysql & psql of flavor-show

If you create a new flavor using a deleted flavorid,
the result of flavor-show is different between mysql and psql.

This issue is caused by the different processing for index in each db.

We need to specify the sort-type to avoid the issue.

Change-Id: Ib1029e80c1b981e1ec86d954b63f83650c9b1cc1
Closes-Bug: #1288636
(cherry picked from commit 4bbb4d88595cfd5f75528d8789be94d0ec9fbe62)
Update aggregate should not allow duplicated names

It is not allow to create 2 aggregates with the same name
but if update an existing one with a name already used
by other aggregate that check should be made.
This patch doesn't allow to change the name to an existing
one.

Change-Id: Icfe315594beaec94229e5a4a4c1bb80fc366b66e
Closes-Bug: #1288296
(cherry picked from commit 2b2decfd1b5b0d68813bc7e5b5e4349a1ab592aa)
Fix: Unshelving an instance uses original image

The driver uses the property image_ref to boot the
the instance.
This patch updates the image_ref property with the
shelved_image_id stored during the shelving.

Change-Id: I2304618bb5a995ea43f2a36069cfa0f3787c9792
Closes-Bug: #1257594
(cherry picked from commit 60455490c2c664dd41364ede29add64967a08844)
network_info cache should be cleared before being rescheduled

If an instance fails to boot due to a non-networking error the instance
then gets rescheduled and launched on another compute node. In these cases
deallocate_for_instance() is called which deletes the network ports
allocated though the info_cache for the instance is never cleared. This patch
adds a call to update_instance_cache_with_nw_info() which causes the cache
to get cleared out. Note: the cache is only cleared if the instance hasn't
been marked for deletion. This is due to how instance_info_cache_update()
is implemented.

Change-Id: If967884c9a6276f5949a7a04b597cedcce12ba09
Closes-bug: #1285158
(cherry picked from commit 3538a9cd06e1e46ec5698edf4018c8d3075b619b)
Fix bug detach volume fails with "KeyError" in EC2

Modified volume['instance_uuid'] to volume.get('instance_uuid'),
to avoid "KeyError", when detach a volume with "attaching" status.

Change-Id: Ic1aacab212c2a2026d18ef57ad9d77fdc6d86b47
Closes-Bug: #1280532
(cherry picked from commit fa72e49845179e29cf918e0742ef735ae868bb4c)
Update ensure()/reconnect() to catch MessagingError

The error handling code that gets connections reset if necessary
caught ConnectionError. It really needs to catch MessagingError,
which ConnectionError inherits from. There are other types of
MessagingErrors that may occur, such as InternalError, and they need
to cause the connection to reset, as well.

This fix has already been merged into oslo.messaging.

Closes-bug: #1303890
Change-Id: Ic5082b74a362ded8b35cbc75cf178fe6e0db62d0
(cherry picked from commit 234f64d608266f43d8856ff98c89ceba6699d752)
(cherry picked from commit fae88f6805f6a16a60c23da67ce06cc0c78a706d)
Fix for qemu-nbd hang

NBD device once used seem to run into intermittent trouble when
used with mount repeatedly. Adding code to explicitly flush the
device buffers using 'blockdev --flushbufs'.

NOTE(dmllr): Also merged regression fix:

    Consolidate the blockdev related filters

    Seeing a "/usr/local/bin/nova-rootwrap: Unauthorized command" error
    in the logs when  "blockdev --flushbufs" is executed because of the
    existing blockdev in compute.filters. We need to merge both into a
    single RegExpFilter

    Change-Id: Ic323ff00e5c23786a6e376e67a4ad08f92708aef

(cherry-picked from dd3f96e91581f465a52d10a212eb51b147dc38b5)
(cherry-picked from 31b2791e6be1768c410de5fa32283fcb637bda57)

Closes-Bug: #973413
Partial-Bug: #1254890
Change-Id: I2b7053b9a069d6e82f6f6baf9ad480efa4388d91
Update log message for add_host_to_aggregate

When add host to aggregates with availability zone, if the host
is already in one availability zone, then the host cannot be added
to another availability zone.

When user do above operation, nova api will report error message,
but the error message does not include blank space between
<availability zone> and zone name, also the error message include
two periods.

The fix was adding blank space and removes one period for the message.

Change-Id: I7307288be194d3ce6de05b77433bbe2626ae46d9
Closes-bug: #1237303
(cherry picked from commit c2ae5fb5659faeadc6f4d8afe27eb68e3c0ade0c)
Remove unnecessary steps for cold snapshots

Up until now when we created cold snapshots we were stopping the
instance, create an internal snapshot, extract the snapshot to a file
and then delete the internal snapshot before bringing up the instance.

If the instance is shut down, there's no concurrent writer, so the image
can be directly extracted without taking an internal snapshot first,
because the snapshot and the current state are the same.

In this patch the creation and deletion of the internal snapshot are
removed to eliminate the extra steps and optimize the creation of
snapshots a bit.

Closes-Bug: #1242819
Closes-Bug: #1247185

Change-Id: I429fa2e1e26aea321eecbf14edd11263fff525ae
(cherry picked from commit 8c985874c7885f31871204d3f83ce547fefc5fb6)
Merge "network_info cache should be cleared before being rescheduled" into stable/havana
Add context as parameter for two libvirt APIs

When finish_revert_migration is called, the caller of
finish_revert_migration already includes context as a parameter,
but finish_revert_migration did not reuse this parameter and still
re-generate the context inside finish_revert_migration, we should
add context as a new parameter for finish_revert_migration so that
the functions inside of it can reuse context when needed.

For the function _create_domain_and_network in libvirt/driver.py,
it set context as an optional parameter, but context isn't really
an optional parameter for this method. We should always pass a
context down because it might be needed somewhere inside
_create_domain_and_network().

The patch includes two parts:
1) Add context as a new parameter for finish_revert_migration
2) Set context as a required parameter for _create_domain_and_network

Closes-Bug: #1251261
(cherry picked from commit 81fc3967bca0f154d72d5de7875b646e1b6c47c8)

Conflicts:
	nova/tests/compute/test_compute_mgr.py
	nova/virt/vmwareapi/vmops.py

Change-Id: I72d131fc8dce903400e3080d51ba0fc0a1320751
Bump stable/havana next version to 2013.2.4

Change-Id: I079d8a5175e9642522142f64b096b09a40127131
VMWare: add power off vm before detach disk during unrescue

Non Hot Plug type disk like IDE can only be detached when the VM is power off.

Close-bug: #1279199
(cherry picked from commit 1e1915aaaca38b5691794e0e052a42b9d95dd3c2)

Conflicts:

	nova/virt/vmwareapi/vmops.py

Change-Id: Ib1f387a41abe2b52357854e90c2535ebb7b43f18
VMware: ensure rescue instance is deleted when instance is deleted

If the user creates a rescue instance and then proceeded to delete
the original instance then the rescue instance would still be up
and running on the backend.

This patch ensures that the rescue instance is cleaned up if
necessary.

The vmops unrescue method has a new parameter indicating if
the original VM should be powered on.

Closes-bug: 1269418
(cherry picked from commit efb66531bc37ee416778a70d46c657608ca767af)

Conflicts:

	nova/tests/virt/vmwareapi/test_vmwareapi.py
	nova/virt/vmwareapi/vmops.py

Change-Id: I3c1d0b1d003392b306094b80ea1ac99377441fbf