GET details REST API next link missing 'details'

When executing a pagination query a "next" link is included in the
API reply when there are more items then the specified limit.

See pagination documentation for more information:
http://docs.openstack.org/api/openstack-compute/2/content/
Paginated_Collections-d1e664.html

The caller should be able to invoke the "next" link (without
having to re-format it) in order to get the next page of data.
The documentation states "Subsequent links will honor the
initial page size. Thus, a client may follow links to traverse
a paginated collection without having to input the marker parameter."

The problem is that the "next" link is always scoped to the non-
detailed query.

For example, if you execute "/v2/<tenant>/volumes/detail?limit=1",
the "next" link does not have the URL for a detailed query and is
formatted as "/v2/<tenant>/volumes?limit=1&marker=<marker>". In this
case the "next" link needs to be scoped to "/v2/<tenant>/volumes/detail".

The user could work around this issue my manually inserting '/details'
into the "next" link URL.

Test code is included to verify that the '/details' URL is correctly added
when the "next" link is included in a detailed pagination query. Also,
existing tests were changed to ensure that the correct controller function
(ie, 'index' vs. 'detail) are invoked for the appropriate query
(ie, non-detailed or detailed) -- 'index' was previously alwayed invoked
for detailed URL requests.

Change-Id: Ib00d6deb25255fac1db0f7bf4ecd3c8d30e1c39d
Closes-bug: 1299247
(cherry picked from commit 2981cdbb03c6f1239a58fedb260796667b8154ab)
Specify lld in tgt config backends

Otherwise when running tgt-admin --delete, it might forget
to invoke tgtadmn with "--lld iscsi" in order to delete the low-level
iscsi exports, which can cause crashes or hangs later on.

Closes-Bug: #1311640
Change-Id: I436c67c30565089236e987d84ac2609eedd2d2c7
(cherry picked from commit 872b7ca0fc5ce4911fdaadee464d46be5a38058d)
vmware: Fix problems with VIM API retry logic

Currently the VIM APIs including session creation API are retried for
cases which are unrelated to connection or session overload problems.
This change fix the retry logic as follows:
* Add an exception to indicate connection problem and raise it
  appropriately from the VIM API client
* Modify the expected exceptions in the retry decorator to include
  only connection and session overload related exceptions

This change also fixes the base class of VimFaultException
(VolumeBackendAPIException -> VimException). This change is required
so that we can handle the exceptions thrown by the VIM API client in
a consistent manner. Currently if we need to handle VIM API related
exceptions, we have to catch both VimException and VimFaultException.
For example, the API for checking active session raises
VimFaultException in some cases and we are handling only VimException.
Due to this, when session expires, the session re-establishment and API
retry fails (intermittently) since the check for active session throws
VimFaultException in some cases which is not handled.

Change-Id: I72cfc7777c3ce693b8598633f822c12c2cee2235
Closes-Bug: #1294598
Closes-Bug: #1302514
(cherry picked from commit bc115eaa477d30826ac17986a6e45844b456e206)
Fix handling multiple WWPNs on preferred FC node

There was a bug in the code that checked to see if the
appropriate WWPN for a preferred node was available on the storage
host in intialize_connection.  It was assumed that there would only
ever be one preferred WWPN.  The code just checked to see if the list of
preferred WWPNs was in the list of available WWPNs.  This would only work
if all of the WWPNs were available on the storage host.  This resulted in
volumes not being able to be mounted when they should have been
available.

This fix changes the check of the preferred WWPNs to check each
item in the list until one is found.  The first one found is used
as the target WWPN.  If no match is found, the old default behavior
of selecting the first available WWPN is used.  I have added a
warning message for this case as it is quite possible that this
won't work in the user's environment as was the case that uncovered
this bug.

Note that this change is only relevant for systems that are not using
multipath as systems that use multipath don't choose one WWPN, they
just send the whole available list back.

Change-Id: I8d3239da24a8f9fbd8309ba5c90b05b29a068754
(cherry picked from commit 9bfc5b0bf58ecb7de5904cbd41db6de7330187e3)
Close-bug: 1308298
Storwize/SVC driver detach volume failed

If config two backends on the same storage, but the protocol is
different, one is FC and the other is iSCSI,  that will create two
hosts on the storage.

When detaching the volume which is attached to the iSCSI host, the
driver will get FC host by default, that causes volume status keeps
'detaching'.

Change-Id: Idbdb3861574197549625b4a6b6141197e42857ff
(cherry picked from commit a221a30b32aa94077f352719b2637b51c1809069)
Closes-Bug: #1310559
Retry lvremove with ignore_suspended_devices

A lvremove -f might leave behind suspended devices
when it is racing with udev or other processes
still accessing any of the device files. The previous
solution of using lvchange -an on the LV had the
side-effect of deactivating origin LVs alongway in
the thick volume case, which was undesired.

It turns out retrying the deactivation twice and
ignoring the suspended devices on the second iteration
avoids the hang of all LVM operations after an initial
failure.

Change-Id: I0d6fb74084d049ea184e68f2dcc4e74f400b7dbd
Closes-Bug: #1317075
Related-Bug: #1270192
(cherry picked from commit da9597aed0186e68dbf1c7304b30e49f8e6a54ff)
Don't leave snapshots on the floor

The current cloning process in the LVM
driver uses a snapshot to do the data
copy in the background.  Trouble here
is that if the creation or activation
of the new volume fails, we don't
cleanup the snapshot we created.

Just move the create_volume and activate
calls down into the try block so the finally
clause can do the cleanup for us.

Change-Id: If0fbab9d9f39102cdc7d287527be22c4a2b35934
Closes-Bug: #1312718
(cherry picked from commit 38ed0523125efeaf8383b36d1db169b42e2f5eb6)
Ensure flushing of IO prior to removing FC device

This patch flushes IO for each SCSI device that
brick removes from the system.  The patch also
flushes a multipath device prior to removing
each underlying LUN associated with the
multipath device.   We were seeing kernel
IO errors in the syslog without calling flush
on the block device, during a detach for a
multipath FibreChannel copy image to volume
action.

Change-Id: I2fae3dccd4fcab22c8f59dcd297581ee9577b779
Closes-Bug: 1318950
(cherry picked from commit 383f460d6d0431d3dbaf32edb7fcb68c25ff1929)
Updated from global requirements

Change-Id: I5c10231456066d6eddafb2c6ca3a4ede0d2d7d66
Merge "Don't leave snapshots on the floor" into stable/icehouse
Get updated model info on volume transfer

On some drivers CHAP credentials are tied to accounts
which may be unique for each OS and/or backend Tenant.

For example:
  Volume is created with project_id xyz
  Backend creates an account for xyz with specific CHAP

  Volume is transferred to project_id abc

Currently we don't update the model_info, so even though
on the backend we updated the owner and CHAP settings we
never propogated that back up to the DB object so the volume
can't be used by the new owner.

This patch just adds an option to return model_update
on the accept_transfer call to the driver and updates
the db accordingly.

Also adds a call in the SF driver to actually get the
new model info to be fed back to the manager.

Closes-Bug: #1349475

Conflicts:
	cinder/volume/drivers/solidfire.py

Change-Id: Ie0447cdad69c4fbee99b4b6b1d3cacdfdd14137d
Set python hash seed to 0 in tox.ini

New tox (>=1.7.0) sets a random python hash seed by default. This is
generally good for testing because it will help keep projects working
regardless of the hash seed, but cinder unittests don't currently pass
with a random hash seed so set it to the python default seed.

This change will allow us to use new tox again and remove the
restriction on tox<=1.6.1 to run unittests.

Redundant setenvs for specific test envs are removed to prevent them
from overriding the PYTHONHASHSEED value.

Note this change will need to be backported to the stable branches to
keep unittests there working with new tox as well.

Change-Id: Ia55031528d082a1b11da6890447c5584e9e27551
(cherry picked from commit 03293715b1e950cf406018058e4c8a514c0f977f)
Merge "Set python hash seed to 0 in tox.ini" into stable/icehouse
Merge "Retry lvremove with ignore_suspended_devices" into stable/icehouse
Merge "GET details REST API next link missing 'details'" into stable/icehouse
Merge "Ensure flushing of IO prior to removing FC device" into stable/icehouse
Merge "vmware: Fix problems with VIM API retry logic" into stable/icehouse
Fixes HP LeftHand driver with Paramiko 1.13.0

With Paramiko 1.13.0, the method exec_command now returns Unicode.
This causes a problem when the driver tries to build the XML returned
from the LeftHand array. The XML header returned from the array defines
the encoding as encoding=UTF-8. Therefore, we must now ensure the
encoding passed to the parser is utf-8.

Change-Id: I7b504626e2d9a0ee2b62820b11f56eb136e31987
closes-bug: #1298608
(cherry picked from commit 57e8cdb9dd8e24919bc2243438a6bded53491b3e)
Merge "Fix handling multiple WWPNs on preferred FC node" into stable/icehouse
Fix solidfire accept_transfer

Previous fix has a mistake which makes the situation worse.  This change
fix volume['project_id'] and volume['user_id'] and adds unit test for
accept_transfer.

Closes-bug: # 1349475

Change-Id: I7b2bcefa4c8d173689cc63ee5aa9e00397c7f8ca
(cherry picked from commit 5dbdb11e8d8659a6b823c9552052b443050886ca)