Add log_handler to implement the publish_errors config option

Synced from Olso.
Commit Id is Ibf28ba17f81b5ddeda653f6a3501ecac18ffcd42

This handler sends all error logs as rpc notifications.
Note: when using publish_errors you also need to set the
notification_driver option too.

Change-Id: Ic97b9c8b84d51c84a53ace5b86cc2e72c744f136
Closes-bug: #1240349
(cherry picked from commit 1c4e279e187d1b864680c9be7849130cb7f12801)
Fix `NoopQuotaDriver.get_(project|user)_quotas` format

The quota API extension expects `get_project_quotas` and `get_user_quotas` to
return a dictionary where the value is another dictionary with a `limit` key.

The `DbQuotaDriver` adhered to this spec, but the `NoopQuotaDriver` didn't.

This fixes the `NoopQuotaDriver` to return the results in the correct format.

Fixes bug 1244842

Change-Id: Iea274dab1c3f10c3cb0a2815f431e15b4d4934b1
(cherry picked from commit 711a12b4029cd1544d26d147d8a67e110e056124)
Include image block device maps in info

In cases for example soft/hard reboot, we were just getting volume and
snapshot block device maps. For block devices with a source type image,
Nova would fall back on a local disk location and fail.

This change allows us to include block devices with a source of an
image, so that we correctly get the right the location of the block
device.

Closes-Bug: #1245719
Change-Id: I1f726293e85183d4b84c05b635a7c606a092992f
(cherry picked from commit 85d0ace169be513c30b09e13a35fa7d912f5b380)
Fix interface-attach removes existing interfaces from db

The following commit 394c693e359ed4f19cc2f7d975b1f9ee5500b7f6 changed
allocate_port_for_instance() to only return the ports that were created
rather than all of the ports on the instance which broke the attach-interface
code.

This patch fixes this issue by remove the sync decorators from:
allocate_for_instance, allocate_port_for_instance, and
deallocate_port_for_instance to just _build_instance_nw_info which
is called in all these cases instead.

Closes-bug: #1223859

(cherry picked from commit 1957339df302e2da75e0dbe78b5d566194ab2c08)

Conflicts:
	nova/network/neutronv2/api.py

Change-Id: I66eb0c0ab926e0a8d1e2c9cfe1f7fd579ea3aa27
libvirt: Allow delete to complete when a volume disconnect fails

If an instance which was booted from a volume fails to spawn, the
instance is left in an ERROR state.  If the failure is in the networking
stage Nova will not have attached the volume via the Cinder driver,
and so the volume remains in an available state.  In this state the
volume may be deleted or assigned to another instance.
During the subsequent delete an exception may be thrown from
the cinder driver which prevents the instance from completing the
deletion process leaving the instance stuck in Error and unable to
be deleted.

Volume exceptions should be logged and ignored when the instance
is being deleted.

Closes-Bug #1222979

(cherry picked from commit 506a8f58cf4b8cecf90b647c7deba47da2a4dfec)
Conflicts:
	nova/tests/virt/libvirt/test_libvirt.py

Change-Id: Icb3796b0ddba25cf344953a649b2e762fab6d782
Fix migration 185 to work with old fkey names

Due to different versions of MySQL it is possible to end up with
different FK's then what is expected. For example the FK constraint
on instance_info_caches changed from instance_info_caches_ibfk_1
to instance_info_caches_instance_uuid_fkey and on virtual_interfaces
from virtual_interfaces_ibfk_1 to
virtual_interfaces_instance_uuid_fkey meaning databases who have
upgraded from before folsom may have the old fkey.

This patch adds a check for those with the old fkey name to make sure
it is dropped before changing unique constraints (as is required by
MySQL). It also fixes the fkey name to be as defined in 133 if it
wasn't before (which will persist on downgrade being consistent with
those who have come in after folsom).

Closes-Bug: #1245502

Change-Id: Ib0dcd04fcb9ed776c76d40561181abad2e14f76f
(cherry picked from commit c620cafb700ca195db0bd0ef9d62a0c9459bdc38)
Add context as parameter for resume

Now for KVM when resume an instance with block storage, nova
compute will throw exception and failed to resume the VM.

The root cause is that when resume a VM with block storage,
libvirt driver needs to call conductor via rpcapi to update
block device, but the function of resume() do not have context,
this will cause RPC api failed.

Change-Id: I712777ed1d893a2b6463d30c407b0a677e37b602
Closes-Bug: #1241337
(cherry picked from commit de4158861000dedc204314c545cc5682aa38f5f1)
Allow compute manager to accept 3.0 messages

We are currently trying to support live upgrades from havana to
icehouse.  This change is required to make that happen.  In a live
upgrade situation, you have icehouse nova infrastructure, with a mix of
havana and icehouse compute nodes as you roll through and upgrade them.
While in this situation, the havana compute nodes must be able to accept
3.0 messages.

Accepting 3.0 messages is trivial to do and extremely low risk.  It does
not change the behavior of a havana-only deployment at all.  The only
additions is some pass-through code to the current compute manager.
This code will only get hit during a live upgrade situation.

Note that there is one other change needed to complete this.  With only
this patch, live upgrades are supported, but any operations that require
compute nodes to talk to each other will not work.  This includes
resizes and migrates.  That would be unfortunate, since live migrations
are something that people *definitely* want to be able to do during an
upgrade procedure.

The additional changes needed to allow resizes and migrates during live
upgrades are included in another patch.

Partial-bug: #1258253
Change-Id: Ia801609b6b3fe88e0d0b16f4c0d37bf25dbfa842
Allow resizes and migrates during live upgrade

This patch completes the changes needed to havana's compute rpc api to
allow live upgrades to icehouse.  Specifically, these changes are needed
to allow resizes and migrates to work.  Before havana compute nodes get
put in a mixed environment with icehouse compute nodes, they need to be
speaking version 3.0 messages.

Change-Id: If09bd38c9d8c3beb5b95107c497699dec47aa3ef
Closes-bug: #1258253
Updated from global requirements

Change-Id: I95c80f36e090cea29e53721fba067c0004011ca4
Prevent spoofing instance_id from neutron to nova

Previously, one could update a port's device_id in neutron to be
that of another tenant's instance_id and then be able to retrieve
that instance's metadata. This patch prevents this from occurring by
checking that X-Tenant-ID received from the metadata request matches
the tenant_id in the nova database.

DocImpact - This patch is dependent on another patch in neutron
            which adds X-Tenant-ID to the request. Therefore to
            minimize downtime one should upgrade Neutron first (then
            restart neutron-metadata-agent) and lastly update nova.

Change-Id: I93bf662797c3986324ca2099b403833c2e990fb4
Closes-Bug: #1235450
Fix performance of Server List with Neutron for Admins

https://review.openstack.org/#/c/30048/ optimized the
SecurityGroupsOutputController for the case where the server
list only contains one server, but in all other cases the current
code calls the Neutron driver in a way that makes it retrieve
all ports and security groups visible to the tenant.

For users with a Neutron admin role this retrieves all ports
and all SecGroups in the system, which on a large system is a
major performance issue and often leads to client timeouts.
Normally these users have further qualified their query to a
specific tenant or host, or are maybe just trying to get their
own list of servers.

If instead we pass the pre-filtered list of servers to Neutron
it can query for ports associated with just the device IDs of
the servers.  We then use that list of ports to build a
specific list of security groups IDs to query for.

This approach means that we can remove the initial
optimization for a single server case.

Closes-Bug: 1228384

(cherry picked from commit 18c3ac4a8935b9997dbb55181d5dbb5232ef2c27)

Conflicts:
	nova/tests/api/openstack/compute/contrib/test_neutron_security_groups.py

Change-Id: I6f72533056e4f336f7578cc883fd1a125c2048a9
Merge "Prevent spoofing instance_id from neutron to nova" into stable/havana
Bump stable/havana next version to 2013.2.2

Change-Id: I3a7170180717bd62fce4cbc2c3a9107187f89fb1
Enforce permissions in snapshots temporary dir

Live snapshots creates a temporary directory where libvirt driver
creates a new image from the instance's disk using blockRebase.
Currently this directory is created with 777 permissions making this
directory accessible by all the users in the system.

This patch changes the tempdir permissions so they have the o+x
flag set, which is what libvirt needs to be able to write in it and

Closes-Bug: #1227027
Change-Id: I767ff5247b4452821727e92b668276004fc0f84d
(cherry picked from commit 8a34fc3d48c467aa196f65eed444ccdc7c02f19f)
Make Serializer able to backlevel objects

This improves handling the situation where nova-api sends newer
objects to an older compute node. In this case, the serializer detects
the IncompatibleObjectVersion error and asks conductor to backlevel it.
This means we call into the conductor with the primitive object that we
can't handle, and basically ask it to hydrate it and call
obj_make_compatible() on our behalf and pass it back to us.

This eliminates the need for API to know whether it's talking to
an older compute and allows a newer conductor to assist older
compute nodes during upgrades to a significant degree.

Adapted for havana/stable from master commit: 0eb7e35

Related-Bug: #1258256
Change-Id: I4b17c8382619e4f73b83c026bf68549ac67a68a2
VMware: fix bug when more than one datacenter exists

In the case that there was more than one datacenter defined on the VC,
then spawning an instance would result in an exception. The reason for this
was that the nova compute would not set the correct datacenter for the
selected datastore.

The fix also takes care of the correct folder selection. This too was a
result of not selecting the correct folder for the data center.

The 'fake' configuration was updated to contain an additional data
center with its on datastore.

Closes-Bug: #1180044
Closes-Bug: #1214850

Co-authored-by: Shawn Harsock <hartsocks@vmware.com>

(cherry picked from commit a25b2ac5f440f7ace4678b21ada6ebf5ce5dff3c)

Conflicts:

	nova/tests/virt/vmwareapi/test_vmwareapi.py
	nova/virt/vmwareapi/fake.py

Change-Id: Ib61811fffcbc80385efc3166c9e366fdaa6432bd
Merge "Include image block device maps in info" into stable/havana
use 'os_type' in ephemeral filename only if mkfs defined

Currently for undefined os-types it will use the default mkfs
command, but use the meta 'os_type' in the name of the
ephemeral file (e.g. ephemeral_20_abcdef). Which can result
in a lot of files (DoS?)
This change will only use 'os_type' in the ephemeral filename
if there is a specific mkfs command defined, otherwise it will
use 'default' (e.g. ephemeral_20_default)

Modifed the tests to test for:
  os_type=''
  os_type=None
  os_type='test' - with no mkfs command specified
  os_type='test' - with a mkfs command specified

Closes-Bug: 1253980

Backport of Original Change-Id: Ie4c10f99ce690c5e4ef181624bd688c38923855c
to stable/havana

Change-Id: Ifa2b94e79dabd586d7e904da247d099360229313
Merge "Make Serializer able to backlevel objects" into stable/havana