Fix `NoopQuotaDriver.get_(project|user)_quotas` format

The quota API extension expects `get_project_quotas` and `get_user_quotas` to
return a dictionary where the value is another dictionary with a `limit` key.

The `DbQuotaDriver` adhered to this spec, but the `NoopQuotaDriver` didn't.

This fixes the `NoopQuotaDriver` to return the results in the correct format.

Fixes bug 1244842

Change-Id: Iea274dab1c3f10c3cb0a2815f431e15b4d4934b1
(cherry picked from commit 711a12b4029cd1544d26d147d8a67e110e056124)
Fix interface-attach removes existing interfaces from db

The following commit 394c693e359ed4f19cc2f7d975b1f9ee5500b7f6 changed
allocate_port_for_instance() to only return the ports that were created
rather than all of the ports on the instance which broke the attach-interface
code.

This patch fixes this issue by remove the sync decorators from:
allocate_for_instance, allocate_port_for_instance, and
deallocate_port_for_instance to just _build_instance_nw_info which
is called in all these cases instead.

Closes-bug: #1223859

(cherry picked from commit 1957339df302e2da75e0dbe78b5d566194ab2c08)

Conflicts:
	nova/network/neutronv2/api.py

Change-Id: I66eb0c0ab926e0a8d1e2c9cfe1f7fd579ea3aa27
libvirt: Allow delete to complete when a volume disconnect fails

If an instance which was booted from a volume fails to spawn, the
instance is left in an ERROR state.  If the failure is in the networking
stage Nova will not have attached the volume via the Cinder driver,
and so the volume remains in an available state.  In this state the
volume may be deleted or assigned to another instance.
During the subsequent delete an exception may be thrown from
the cinder driver which prevents the instance from completing the
deletion process leaving the instance stuck in Error and unable to
be deleted.

Volume exceptions should be logged and ignored when the instance
is being deleted.

Closes-Bug #1222979

(cherry picked from commit 506a8f58cf4b8cecf90b647c7deba47da2a4dfec)
Conflicts:
	nova/tests/virt/libvirt/test_libvirt.py

Change-Id: Icb3796b0ddba25cf344953a649b2e762fab6d782
VMware: fix bug when more than one datacenter exists

In the case that there was more than one datacenter defined on the VC,
then spawning an instance would result in an exception. The reason for this
was that the nova compute would not set the correct datacenter for the
selected datastore.

The fix also takes care of the correct folder selection. This too was a
result of not selecting the correct folder for the data center.

The 'fake' configuration was updated to contain an additional data
center with its on datastore.

Closes-Bug: #1180044
Closes-Bug: #1214850

Co-authored-by: Shawn Harsock <hartsocks@vmware.com>

(cherry picked from commit a25b2ac5f440f7ace4678b21ada6ebf5ce5dff3c)

Conflicts:

	nova/tests/virt/vmwareapi/test_vmwareapi.py
	nova/virt/vmwareapi/fake.py

Change-Id: Ib61811fffcbc80385efc3166c9e366fdaa6432bd
Ensure MTU is set when the OVS vif driver is used

Set the MTU on the relevant devices when the OVS vif driver
is used:
    - veth devices
    - ovs vif device

Closes-bug: 1260697

Conflicts:

	nova/network/linux_net.py

Change-Id: I217f43227c099f17ab9a225b1ef7377d3de28114
(cherry picked from commit e34a085f7a524747dd93e9645c6e0e69631075d4)
Setup destination disk from virt_disk_size

When running live-migration --block-migrate on a qcow2 backed
VM without cow image, the destination qcow2 file should be created
with the virtual disk size. For raw images, the virt_disk_size
is set to disk_size to ensure that virt_disk_size is always the
size of the disk that should be re-created.

Update unit tests to be more strict and check for sizes to be correct.

Closes-Bug: #1257355

Change-Id: Ie3be46024f06b9f59af92f5e3918a1958386d4f1
Merge "libvirt: Allow delete to complete when a volume disconnect fails" into stable/havana
Merge "Setup destination disk from virt_disk_size" into stable/havana
Merge "Fix interface-attach removes existing interfaces from db" into stable/havana
Merge "VMware: fix bug when more than one datacenter exists" into stable/havana
Avoid deadlock when stringifying NetworkInfo model

In the libvirt driver, we log information about the instance we're
about to generate XML for, which includes a NetworkInfo model. In
reality, this is a NetworkInfoAsyncWrapper which acquires a lock
on the model. Since we're in the middle of a log statement, we also
hold the logging lock. This happens right after we've fired off an
async request to update information about the instance, which first
acquired the network model lock and then acquired the logging lock
to make a seemingly innocuous log record.

The resulting deadlock is fixed by this patch by stringifying the
NetworkInfo object before making the log call in the libvirt
driver.

Change-Id: I81f35197ab7c74daa84fb51fbaa2df28c025da13
Closes-bug: 1276268
(cherry picked from commit 072e4ad20a6a7f4fff3b59eb7ef3f6fef9aa19d1)
Add HEAD api response for test s3 server BucketHandler

Current nova test s3 server lacks HEAD for BucketHandler. And it becomes
an issue for Boto after version 2.25.0 [1], which changes underlying
implementation of get_bucket method from GET to HEAD request.

This change fixes this gap by adding HEAD response to test s3 server.
And also added cases for testing bucket existence per Boto document
suggestion[2], which applies for both Boto prior to or after version
2.25.0

[1] http://docs.pythonboto.org/en/latest/releasenotes/v2.25.0.html
[2] http://docs.pythonboto.org/en/latest/s3_tut.html#accessing-a-bucket

Change-Id: If992efa40f7f36d337d1b9b1f52859aa0ae51874
Closes-bug: #1277790
(cherry picked from commit 033f3776c4bc6d0db14b1b9da7c24e207e9628ab)
Merge "Fix `NoopQuotaDriver.get_(project|user)_quotas` format" into stable/havana
libvirt: handle exception while get vcpu info

If an exception is raised while get a libvirt domain's vcpu info,
the update_available_resource periodic task will be failed, which
will result in the resource of this compute node will never be
reported.

This patch add an exception handling to avoid this situation.
Closes-bug: #1270008

Change-Id: I69109402416989fdaa421f8dbc72953bd067c407
(cherry picked from commit 844df860c38ac38550b8d1739fd53131cd7fd864)
hyperv: Retry after WMI query fails to find dev

During long stress runs the WMI query that is looking for
the iSCSI device number can incorrectly return no data.
If the query is retried the appropriate data can then be
obtained.

This commit adds a retry loop, calling
basevolumeutils.get_device_number_for_target to avoid this situation.
It also handles the case where the devices list returned in
get_device_number_for_target is empty.  The retry loop is
implemented with new mounted_disk_query_retry_count and
mounted_disk_query_retry_interval configuration options.

Unit tests have been added to check the good and bad paths for
get_mounted_disk_from_lun.

DocImpact
Closes-bug: 1247901
Change-Id: I082c4b1694efcd20cce65293cd330b7a0cf7d470
(cherry picked from commit d143540ad1b69ec93c2b7bfadd1f654c4d8c7a34)
Add db2 communication error code when check the db connection

Closes-Bug: #1244092
Change-Id: I17882e407afa00167e550a565081219f9e432a31
Don't raise MySQL 2013 'Lost connection' errors

Although this code error is part of the `ping_listener` tests, it
missing from the list of known - and ignored - connection errors.

The code error 2013 refers to a connection lost during a query, see:
https://dev.mysql.com/doc/refman/5.0/en/error-messages-client.html#error_cr_server_lost

Without this error code, the session won't try to reconnect to mysql,
which makes `max_retries` useless.

Closes-bug: #1276510
Change-Id: If2667cf83ac656666ffe00aa90e3454516b080db
Fix changes-since filter for list-servers API

On list-servers API, "changes-since" parameter filters out servers
which have been created at the same date as the specified timestamp.
For example, the following server is filtered out with "changes-since=
2013-12-05T15:03:25":

  mysql> select display_name, updated_at from instances;
  +--------------+---------------------+
  | display_name | updated_at          |
  +--------------+---------------------+
  | vm01         | 2013-12-05 15:03:25 |
  +--------------+---------------------+

This bug causes some Tempest test failures, if the timestamp of server
creations are the same as the "changes-since" timestamp in second unit.
In addition, the behavior would be wrong from the viewpoint of its name
which includes "since".

This patch chanegs the server list for including the ones created at
the specified timestamp.

Change-Id: Icc10a1363503b3553d810c6be20b86b3da7ac1a0
Closes-Bug: #1236585
(cherry picked from commit d87bd44d0211a633efd545dc7b2027a613897b85)
Bump stable/havana next version to 2013.2.3

Change-Id: Ia5b940faf01df6cae58df1f4b4f5e8fa2f3f078a
VMware: Always upload a snapshot as a preallocated disk

Currently, a snapshot operation always produces a preallocated
disk type. When the snapshot is uploaded to glance, the source
image metadata is uploaded with the new snapshot and then some
of the metadata properties are updated. The update doesn't
include the disk_type property, this can be problematic. If
the original image has a sparse disk type and a snapshot is
taken, then the snapshot which is a preallocated disk type is
uploaded to glance with the disk_type property set to sparse,
which creates a disk type mismatch and causes errors when a vm
is spawned from that snapshot. This patch addresses the issue
by correcting the image metadata used when uploading a snapshot
to glance.

Also, the image_upload function in vmwareapi/fake.py has been
modified to address the change. Any function call to
image_upload (from the unit tests) will assert that the
metadata for the image to be uploaded must have the disk_type
value to be preallocated.

Closes-Bug: 1247296
(cherry picked from commit 11e327f888a6dbde990d75196560083fddecf394)

Conflicts:

	nova/tests/virt/vmwareapi/test_vmwareapi.py
Change-Id: I71cfa8d4d1eb7911abb489153292fb0d1e614765