Add fix for reservation index to icehouse

Cinder doesn't have backportable migrations, so ensure the index
is added every time the migration runs.

DB migration in master that does the same thing was made in the
following commits:
  4d56b91628518629b47997b93acd9cb1e48f87f5
  08b90190085a11bc99112472944c193672061e1f
  479d6050f3f07e12488e6bb9a40b0aec3452a19c

Change-Id: I8bdab1a70f0076312aba4d0b6c4417b335947680
Resolves-bug: #1348720
Add retry_on_deadlock to db update methods

There's some known races that have been addressed in
Nova via a retry_on_deadlock deocrator.  Some known
cases in Cinder exist as well when dealing with updates
to quotas and reservations_expire.

This patch introduces the decorator to Cinder and hits
the methods known to be susceptible to deadlock as well
as some others that seem as though they'd fall into the
same category.

Leverages work that's already been done in Nova

Co-Authored-By: Vish Ishaya <vishvananda@gmail.com>

Change-Id: Ic807f4f8a333e4d5477f86e41eea40191637087c
Partial-bug: #1350466
(cherry picked from commit 201adf31a57ea4d456ef5daf1eca2ed3b64fca85)
volume_image_metadata missing from volume list

The volume_image_metadata field was missing from the volume list
because we tried to filter VolumeGlanceMetadata against a non-existent
project_id field. Filtering should be done on the Volume associated
to the VolumeGlanceMetadata instead.

Move project_id filtering out of model_query as it assumes
the project_id field is in the queried model itself.
Build the filter ourselves against Volume.

Fix glance metadata SQL query performance

The query built to retrieve glance metadata associated to a volume
was sub-optimal: all rows of volume_glance_metadata were returned.

Fix it by properly joining the 2 tables with volume_id field.

Closes-bug: #1337526
Closes-bug: #1349936
Change-Id: I6053708296f2b5e24513dc87ed63da0f67c220ae
(cherry picked from commit 06dd7f28d1d3ec8619df0d25ddfe977d538897b3)
(cherry picked from commit 8f8a8a6ffe14cf7bb461dc26d178bf8acc8ddf12)
Cache snapshots in request for extension

The extended snapshots extension makes a separate db request for
snapshots which is horribly inefficent. In addition it causes an
issue when using all_tenants=1, because the second request doesn't
pass in the filter. Fix this by caching the snapshot in the request
object like we do for volumes and retrieve the cached object in the
extension.

Change-Id: Ia943438c8d48accb7f62c43f07642c3da2fe34ab
Resolves-bug: #1358960
(cherry picked from commit 4f0d1d36136bcb33f578b2999812f36cadd71cec)

Conflicts:
	cinder/api/contrib/extended_snapshot_attributes.py

Change-Id: Ia943438c8d48accb7f62c43f07642c3da2fe34ab
Fix performance issues with brocade zone driver.

Removed firmware checks from common path as the ssh invoke_shell
was taking 5 seconds to complete for each firmware check.
Refactored to only do cfgsave in non-activate case.
Removed extra calls to switch to get the active zoneset.

Closes-Bug: #1292119
(cherry-picked from commit: 371a58fd1d3fb1c2db7e1234f9c51f22b1b8787e)
Change-Id: Ie1c2c0c603f6dd68439f2c2db74b21ea2dd0fc11
Prevent tenant viewing volumes owned by another

Bug introduced by 0505bb268942534ad5d6ecd5e34a4d9b0e7f5c04 allows any
tenant to get the details of a volume belonging to any other tenant
if the UUID is known.

This commit allows only the tenant or an admin to get a volume.

Change-Id: I0268d374f7529d89068dcbf3c1cb9ab3d60d4115
Closes-Bug: #1356368
(cherry picked from commit 5868e8f285d23b56ca6123dab760342c57bf8c80)
Honor volume:get policy

The fix for bug 1356368 hard-coded a policy check (same as
rule:admin_or_owner) for volume:get.  While in most cases this is
what people want, it'd be good we honor policy setting.

Note that before commit 0505bb268942534ad5d6ecd5e34a4d9b0e7f5c04,
DB query volume_get() actually acted as the policy checker for
volume:get, and it raised VolumeNotFound if context.project_id didn't
match volume['project_id'].  The check_policy() in volume:get didn't
get a chance to raise PolicyNotAuthorized exception.  So in this
change we keep the same behavor.

Change-Id: If43cec5cce977b9220296709b4e243b35b06ecd5
Related-bug: #1356368
(cherry picked from commit d6d75f868d5da77c2c8e20d0562555a14a6f91ec)
VMware: Disable suds caching

The default cache implementation in suds store pickled objects in a
predictable path in /tmp which could lead to attacks. This patch
turns off suds caching to address this security issue.

Change-Id: I7daaa25a0677004e03896298e9c3026d5c33c6ac
Closes-Bug: #1341954
(cherry picked from commit 6a41fe9c5c98a14a355fa81b41aae2c4b0ce0a7b)
Mock glance client object in version unit tests

This patch changes the glance client version unit tests
to mock the Client object completely.  Previously the
tests were ensuring the right version of the client was
returned, but that required too much knowledge of glance's
implementation and ended up breaking the tests when glance
changed the implementation details of the Client class.

The new code tests if cinder is calling the integration
point correctly for the version, rather than if glance
is correctly returning the right client; that should be
a glance test, not a cinder test.

Closes-Bug: 1366596
Change-Id: I979175e6c3b5f98076bde6d36ca52fb8b03009b8
(cherry-picked from 04abab869d28e007d9b869122740f70cd839daf7)
Merge "Mock glance client object in version unit tests" into stable/icehouse
Merge "VMware: Disable suds caching" into stable/icehouse
Block sqlalchemy-migrate 0.9.2

Breaks unit tests.

Conflicts:
	requirements.txt

Closes-Bug: #1368391
Change-Id: Ic814a91038800201603baf33ca40e4586944343b
(cherry picked from commit eb01ccba28b0feabe6220c125776fee3c2c52505)
Updated from global requirements

Change-Id: I6d26b2a5ee5e351d14ca2217eaa808ba505d8535
Merge "Block sqlalchemy-migrate 0.9.2" into stable/icehouse
Merge "Fix performance issues with brocade zone driver." into stable/icehouse
Merge "Add fix for reservation index to icehouse" into stable/icehouse
Merge "Add retry_on_deadlock to db update methods" into stable/icehouse
Update cinder.conf.sample

Whether we should be checking this in the gate or not for
stable inparticular is a whole seperate argument.

Change-Id: I84160f7ff6c4c3b9926c6af6144d077111e36a08
Don't clear _mounted_shares list in remoteFS while updating

This fix makes the updating of the _mounted_shares list in
remoteFS more of an atomic operation.  Previously this list
would be cleared, then rebuilt.  That allowed a race
condition where operations that took place during this
update would have a list of 0 shares to work with.

Closes-Bug: #1366083
cherry-picked from commit 9cc9cf111ef8a0265df5323fecaed12b0dabe312

Change-Id: I740d8f1b87db911242326a38ff398b81c5974cea
Fix KeyError exception in NetApp CDOT iscsi driver volume create.

Prior to commit 98aa91b, when the NetApp CDOT iscsi driver experiences
an NaAPIError interacting with a backend vserver, it catches (and logs)
the NaAPIError, but then throws an irrelevant KeyError because the Path
metadata field has not been set properly.  The KeyError obfuscates and
buries the real error message.

In the cinder pools implementation via commit 98aa91b this problem no
longer exists, but there is a need for a fix in codebase (e.g. icehouse)
where cinder pools will not be implemented.

Note that there are multiple ways that an NaAPIError can be thrown.  In
launchpad bug 1363763 an NaAPIError was thrown, but then buried behind a
KeyError exception, because of an attempt to create a cinder volume with
a Netapp qos policy-group when the backend container on the filer
already itself had a qos policy-group attached to it (outside of
Openstack).  The fix supplied here addresses all types of NaAPIError,
not just errors related to QOS.

Change-Id: Idc93ddddefdf27b7edf0813a14d8562fb5dcd175
Closes-Bug: 1363763