GET details REST API next link missing 'details'

When executing a pagination query a "next" link is included in the
API reply when there are more items then the specified limit.

See pagination documentation for more information:
http://docs.openstack.org/api/openstack-compute/2/content/
Paginated_Collections-d1e664.html

The caller should be able to invoke the "next" link (without
having to re-format it) in order to get the next page of data.
The documentation states "Subsequent links will honor the
initial page size. Thus, a client may follow links to traverse
a paginated collection without having to input the marker parameter."

The problem is that the "next" link is always scoped to the non-
detailed query.

For example, if you execute "/v2/<tenant>/volumes/detail?limit=1",
the "next" link does not have the URL for a detailed query and is
formatted as "/v2/<tenant>/volumes?limit=1&marker=<marker>". In this
case the "next" link needs to be scoped to "/v2/<tenant>/volumes/detail".

The user could work around this issue my manually inserting '/details'
into the "next" link URL.

Test code is included to verify that the '/details' URL is correctly added
when the "next" link is included in a detailed pagination query. Also,
existing tests were changed to ensure that the correct controller function
(ie, 'index' vs. 'detail) are invoked for the appropriate query
(ie, non-detailed or detailed) -- 'index' was previously alwayed invoked
for detailed URL requests.

Change-Id: Ib00d6deb25255fac1db0f7bf4ecd3c8d30e1c39d
Closes-bug: 1299247
(cherry picked from commit 2981cdbb03c6f1239a58fedb260796667b8154ab)
Specify lld in tgt config backends

Otherwise when running tgt-admin --delete, it might forget
to invoke tgtadmn with "--lld iscsi" in order to delete the low-level
iscsi exports, which can cause crashes or hangs later on.

Closes-Bug: #1311640
Change-Id: I436c67c30565089236e987d84ac2609eedd2d2c7
(cherry picked from commit 872b7ca0fc5ce4911fdaadee464d46be5a38058d)
vmware: Fix problems with VIM API retry logic

Currently the VIM APIs including session creation API are retried for
cases which are unrelated to connection or session overload problems.
This change fix the retry logic as follows:
* Add an exception to indicate connection problem and raise it
  appropriately from the VIM API client
* Modify the expected exceptions in the retry decorator to include
  only connection and session overload related exceptions

This change also fixes the base class of VimFaultException
(VolumeBackendAPIException -> VimException). This change is required
so that we can handle the exceptions thrown by the VIM API client in
a consistent manner. Currently if we need to handle VIM API related
exceptions, we have to catch both VimException and VimFaultException.
For example, the API for checking active session raises
VimFaultException in some cases and we are handling only VimException.
Due to this, when session expires, the session re-establishment and API
retry fails (intermittently) since the check for active session throws
VimFaultException in some cases which is not handled.

Change-Id: I72cfc7777c3ce693b8598633f822c12c2cee2235
Closes-Bug: #1294598
Closes-Bug: #1302514
(cherry picked from commit bc115eaa477d30826ac17986a6e45844b456e206)
Fix handling multiple WWPNs on preferred FC node

There was a bug in the code that checked to see if the
appropriate WWPN for a preferred node was available on the storage
host in intialize_connection.  It was assumed that there would only
ever be one preferred WWPN.  The code just checked to see if the list of
preferred WWPNs was in the list of available WWPNs.  This would only work
if all of the WWPNs were available on the storage host.  This resulted in
volumes not being able to be mounted when they should have been
available.

This fix changes the check of the preferred WWPNs to check each
item in the list until one is found.  The first one found is used
as the target WWPN.  If no match is found, the old default behavior
of selecting the first available WWPN is used.  I have added a
warning message for this case as it is quite possible that this
won't work in the user's environment as was the case that uncovered
this bug.

Note that this change is only relevant for systems that are not using
multipath as systems that use multipath don't choose one WWPN, they
just send the whole available list back.

Change-Id: I8d3239da24a8f9fbd8309ba5c90b05b29a068754
(cherry picked from commit 9bfc5b0bf58ecb7de5904cbd41db6de7330187e3)
Close-bug: 1308298
Storwize/SVC driver detach volume failed

If config two backends on the same storage, but the protocol is
different, one is FC and the other is iSCSI,  that will create two
hosts on the storage.

When detaching the volume which is attached to the iSCSI host, the
driver will get FC host by default, that causes volume status keeps
'detaching'.

Change-Id: Idbdb3861574197549625b4a6b6141197e42857ff
(cherry picked from commit a221a30b32aa94077f352719b2637b51c1809069)
Closes-Bug: #1310559
Ensure flushing of IO prior to removing FC device

This patch flushes IO for each SCSI device that
brick removes from the system.  The patch also
flushes a multipath device prior to removing
each underlying LUN associated with the
multipath device.   We were seeing kernel
IO errors in the syslog without calling flush
on the block device, during a detach for a
multipath FibreChannel copy image to volume
action.

Change-Id: I2fae3dccd4fcab22c8f59dcd297581ee9577b779
Closes-Bug: 1318950
(cherry picked from commit 383f460d6d0431d3dbaf32edb7fcb68c25ff1929)
Merge "GET details REST API next link missing 'details'" into stable/icehouse
Merge "Ensure flushing of IO prior to removing FC device" into stable/icehouse
Merge "vmware: Fix problems with VIM API retry logic" into stable/icehouse
Fixes HP LeftHand driver with Paramiko 1.13.0

With Paramiko 1.13.0, the method exec_command now returns Unicode.
This causes a problem when the driver tries to build the XML returned
from the LeftHand array. The XML header returned from the array defines
the encoding as encoding=UTF-8. Therefore, we must now ensure the
encoding passed to the parser is utf-8.

Change-Id: I7b504626e2d9a0ee2b62820b11f56eb136e31987
closes-bug: #1298608
(cherry picked from commit 57e8cdb9dd8e24919bc2243438a6bded53491b3e)
Merge "Fix handling multiple WWPNs on preferred FC node" into stable/icehouse
Fix solidfire accept_transfer

Previous fix has a mistake which makes the situation worse.  This change
fix volume['project_id'] and volume['user_id'] and adds unit test for
accept_transfer.

Closes-bug: # 1349475

Change-Id: I7b2bcefa4c8d173689cc63ee5aa9e00397c7f8ca
(cherry picked from commit 5dbdb11e8d8659a6b823c9552052b443050886ca)
Merge "Specify lld in tgt config backends" into stable/icehouse
Merge "Storwize/SVC driver detach volume failed" into stable/icehouse
vmware: Force chunked transfer for upload-to-image

The upload-to-image operation downloads (using stream-optimized HTTP NFC
transfer) the virtual disk corresponding to the volume to a pipe from
where the glance image client reads the data for upload. Due to a recent
change in the glance image client, success in seeking the input file
object results in forgoing the chunked transfer in favor of regular
transfer. The glance image client expects an IOError if the input file
object is a pipe. Currently the pipe seek() is a NOP and this results in
upload-to-image failure. This changes fixes such failures by throwing
an IOError indicating an illegal seek.

Conflicts:
	cinder/volume/drivers/vmware/read_write_util.py

Change-Id: I2e69fc4103559d49d2cee16ea04787a252dbf879
Closes-Bug: #1295239
(cherry picked from commit cf18199571b7965a8fe77db3d910b0e798f12946)
Bump stable/icehouse next version to 2014.1.3

Bump stable/icehouse next version to 2014.1.3

Change-Id: I65a087895be6652ae8a2683728722566e3f42f58
Signed-off-by: Chuck Short <chuck.short@canonical.com>
Merge "vmware: Force chunked transfer for upload-to-image" into stable/icehouse