Add _wrap_db_error() support to SessionTransaction.commit()

This patch adds _wrap_db_error() to the commit()
and rollback() methods of sqlalchemy.orm.session.SessionTransaction,
which is the object that is dealt with when one invokes
code of the form "with session.begin():".  The context manager
form does not invoke the commit() method on the
SQLAlchemy Session directly, and instead calls the one
local to the SessionTransaction.

In order to intercept this, we must build a subclass
of SessionTransaction with the appropriate wrapping, and
then patch it into the object that is returned by
Session.begin(), ensuring that it is compatible with
_wrap_db_error().

This whole approach is legacy; newer oslo.db versions
intercept errors at the point at which they occur
via engine events.

Tests are omitted here as we are relying upon the tests
that were added to the corresponding oslo-incubator code.

Closes-bug: #1370191

Change-Id: Ie0456e6daa86c99cf6fbe56ca5dfd8a618f14232
Ignore errors when deleting non-existing vifs

If an instance fails during its network creation (for example
if the network-vif-plugged event doesn't arrive in time) a
subsequent delete will also fail when it tries to delete the
vif, leaving the instance in a Error(deleting) state.

This can be avoided by including the "--if-exists" option to
the ovs-vsctl command so that the command doesn't fail if the
vif doesn't exist.

Change-Id: Ic613c9d6e0619f51fcd931e62f6d6bce404ebe81
Closes-bug: #1308544
(cherry picked from commit 476dc5efcdb11c9859b062dc69a8c205e69be861)
Fix instance cross AZ check when attaching volumes

For CONF.cinder_cross_az_attach=False, we currently only check the desired AZ
during scheduling but don't actually check the final AZ of the node.

When attaching volumes to running instances we need to check the actual
AZ on the instance and not the desired AZ at boot time. The desired AZ
attribute is optional at boot and may not be set or the desired AZ
could be different from the final AZ.
When we are booting with a volume however we don't know the final AZ so we
check on the desired AZ.

Closes-Bug: #1255347
Change-Id: Ied912c171100a450754d315027bf5c407ac067bb
(cherry picked from commit 18a1486293b15c5713ac1352b9b9b30fdf2b7413)
Fixes missing ec2 api address disassociate error on failure

Fixes ec2 api address disassociate so that the ec2 error
InvalidAssociationID.NotFound is returned when it is passed an
invalid association rather than returning success. Note this is
slightly different than the original bug report but the type
of failure has changed since the bug was originally filed.

Change-Id: I10034a2fed62d4e93bc384aef3ff594b66e285f2
Closes-Bug: 1179816
(cherry picked from commit 00d8c334bbfaef3ffec4b04b494e609740b96253)
Fix CellStateManagerFile init to failure

Currently, specifying a cells_config file in nova.conf causes
CellStateManager to fail and in turn stops the nova-cells service from
starting. The reason is that CellsManager creates an instance of
CellStateManager with no arguments. CellStateManager __new__ runs and
creates an instance of CellStateManagerFile which runs __new__ and
__init__ with cell_state_cls and cells_config_path set. At this point
__new__ returns CellStateManagerFile and the new instance's __init__
method is invoked (CellStateManagerFile.__init__) with the original
arguments (there weren't any) which then results in:
2014-04-29 11:52:05.240 16759 TRACE nova self.state_manager =
cell_state_manager()
2014-04-29 11:52:05.240 16759 TRACE nova TypeError: __init__() takes
exactly 3 arguments (1 given)

It seems reasonable for CellStateManagerFile to derive the
cells_config_path info for itself so I have updated the code with that
change and added unit tests to catch this bug and to check that the
correct managers are still returned

Closes-Bug: #1314677
(cherry picked from commit 695191fa89387d96e60120ff32965493c844e7f5)

Conflicts:
	nova/tests/cells/test_cells_state_manager.py

Change-Id: I9021640515142a3ca95c2d9e7b03e19b529bc175
VMware: validate that VM exists on backend prior to deletion

Attempting to delete an instance that does not exist on the backend
throws an exception (for example UnregisterVM may return this when
running the CI).

Fixes: bug #1300788

(cherry picked from commit 98fb5e56ef96c0c6a8754276e5f8c9c0b7e50fed)

Change-Id: I1b95caec2dad6580db576aa6f80f47913b7c066e
Merge "VMware: validate that VM exists on backend prior to deletion" into stable/icehouse
Merge "Fixes missing ec2 api address disassociate error on failure" into stable/icehouse
Merge "Ignore errors when deleting non-existing vifs" into stable/icehouse
share neutron admin auth tokens

Admin tokens in nova.network.neutronv2 were being created once per
"thread" (i.e., green thread) and shared. For example, during instance
creation, a single token was created then reused 6 times.  However,
this scheme was far from optimal. Given that tokens, by default, have
a shelf life of 24H, a single token can be shared by _all_ admin
nova.network.neutronv2 calls in a 24-hour period.

DocImpact
Signed-Off-By: Peter Feiner <peter@gridcentric.ca>
Closes-Bug: 1255594

Change-Id: I5c743aa86030e4ab000dcdefb6a7183305512303
(cherry picked from commit 5c01b9e618af44640332871757ff613b2f0b66e2)
postgresql: use postgres db instead of template1

In PostgreSQL template1 database is kind of special, as all new databases are
copied from it, when you do CREATE DATABASE. This implies a few restrictions:
e.g. you can't have more than one active session to template1, if one of them
tries to create a database. It will fail with:

    ERROR: source database "template1" is being accessed by other users

Using of postgres database instead of template1 resolves this issue.

Closes-Bug: #1355929

Change-Id: I8b1a36163984fbef6be66964456072d32da6a9d0
(cherry picked from commit 02fa15b4caaa414930b6ddf6a3a9fe8019751c26)
Merge "Fix instance cross AZ check when attaching volumes" into stable/icehouse
Merge "postgresql: use postgres db instead of template1" into stable/icehouse
Fix XML UnicodeEncode serialization error

The generic Nova XMLSerializer code will currently attempt
to cast to str the value for all leaf nodes. This patch
ensures that no attempt is made to convert unicode which
can cause a UnicodeEncode error. We don't need to convert
unicode for XML text and regardless we encode to UTF-8 at
a later point.

Change-Id: I8135d2b9a67db62b0eafdd301b7fdb67a5dd72cc
Closes-Bug: #1279172
(cherry picked from commit 53fe8696314fb73ca9943fce998d96fa6d0414b4)
(cherry picked from commit 3136cfc11f034cf0b17888f7348e80d38d563c2e)
Nova api service doesn't handle SIGHUP properly

When SIGHUP signal is sent to nova-api service, it doesn't complete
processing of all pending requests before terminating all the
processes.

Closes-Bug: #1334647

Change-Id: I3f7bd0ba5ef1bb716a97635252abe251053a669d
(cherry picked from commit 224a3de466d48301ab638b60ca47d19ad6b1b1dc)
Merge "share neutron admin auth tokens" into stable/icehouse
Merge "Nova api service doesn't handle SIGHUP properly" into stable/icehouse
Merge "Fix CellStateManagerFile init to failure" into stable/icehouse
Retry on closing of luks encrypted volume in case device is busy

We're seeing races in the gate on detach of an encrypted volume where
the device is busy so the detach fails, which causes the delete of the
volume on the cinder side to fail because the volume is still in use and
then Tempest times out waiting for the volume to be deleted.

This simply adds a retry to close the encryption device.  I use
attempts=3 since that's what we use for lvremove.

Closes-Bug: #1374458

Change-Id: Ia0667da7dee247ba8c3338b296244b2a71d1045d
(cherry picked from commit eef97cdf4bb7f426d7feb394ef54510db8b1656b)
Make the block device mapping retries configurable

When booting instances passing in block-device and increasing the
volume size, instances can go in to error state if the volume takes
longer to create than the hard code value (max_tries(180)/wait_between(1))
set in nova/compute/manager.py

def _await_block_device_map_created(self,
                                    context,
                                    vol_id,
                                    max_tries=180,
                                    wait_between=1):

To fix this, max_retries/wait_between should be made configurable.
Looking through the different releases, Grizzly was 30, Havana was
60 , IceHouse is 180.

This change adds two configuration options:
a)  `block_device_allocate_retries` which can be set in nova.conf
by the user to configure the number of block device mapping retries.
It defaults to 180 and replaces the max_tries argument in the above method.
b) `block_device_allocate_retries_interval` which allows the user
to specify the time interval between consecutive retries. It defaults to 1
and replaces wait_between argument in the above method.

The cherry-picked patch has been amended to set the default configuration
options to the same values that were previously hard-coded:
  block_device_allocate_retries=180
  block_device_allocate_retries_interval=1

DocImpact
Closes-Bug: #1332382
Change-Id: I16e4cd1a572bc5c2cd91fc94be85e72f576a8c26
(cherry picked from commit 66721eb2c0f53fc4260b2f0aa9a3811da0f7ddbd)