CinderException args to strings when exceptions

CinderException message and kwargs are converted to strings if the
argument is itself an exception.

Exceptions passed as arguments to CinderException happen in many areas
of the code. This causes an error when oslo.messaging tries to serialise
the exception.

Change-Id: I1399c939bbca47ab8362ac3bbe0e9a349c6d5572
Closes-Bug: #1301249
NetApp NFS: Do not reference dst_img_local before assignment

This patch moves the assignment of the dst_img_local variable to before the
try/finally block that references it so that the finally block will have this
variable no matter where in the try block an error is raised.

Change-Id: Iab48b46559f18a3d6b044abe4bc3e615e7fee075
Closes-Bug: 1309047
Use oslo network utils function to set tcp_keepalive

Earlier, tcp_keepidle was set to 600s and other parameters left to default
values. A new function was added to oslo-incubator for setting tcp keepalive
settings, and cinder should use this to set any values we want.
In addition to the existing cfg option for tcp_keepidle, new values were added
for tcp_keepalive(to turn on/off this feature, tcp_keepintvl(to specify gap
between successive probes) and tcp_keepcnt(to specify number of probes). To
mimic existing behavior, tcp_keepalive defaults to True, and tcp_keepidle to
600(seconds). The other two default to None, and the oslo code will ignore
them.
Ran update.py in oslo-incubator to copy changed modules into cinder.
Changes are:
network_utils.py (which contains the function we need, others are deps)
edd73c9 Merge "Improve help strings"
fe3389e Improve help strings
8a0f567 Remove str() from LOG.* and exceptions
f61a4e7 Fix python26 compatibility for RFCSysLogHandler
da6d713 Revert setting oslo-incubator logs to INFO
0d18381 Set default log levels for oslo.messaging and oslo-incubator

gettextutils.py
9912e5d Add API for creating translation functions
6cc96d0 Fix test_gettextutils on Python 3

jsonutils.py
9e5a393 jsonutils.load() accepts file pointer, not string
a6b2aec On Python <= 2.6, use simplejson if available
e3a1d9c Use six.moves.xmlrpc_client instead of xmlrpclib
3a31bba Python3 support for xmlrpclib
e6494c2 Use six.iteritems to make dict work on Python2/3

log.py
edd73c9 Merge "Improve help strings"
fe3389e Improve help strings
8a0f567 Remove str() from LOG.* and exceptions
f61a4e7 Fix python26 compatibility for RFCSysLogHandler
da6d713 Revert setting oslo-incubator logs to INFO
0d18381 Set default log levels for oslo.messaging and oslo-incubator

Current HEAD in OSLO:
----------------------------------------------
Merge: 2640847 e433899
Date:   Mon May 5 19:41:28 2014 +0000
Merge "Enable configuring tcp keepalive"
----------------------------------------------

Ran generate_sample to generate cinder.conf.sample

Implements blueprint configure-tcp-keepalive
Change-Id: I2e4c2327803c25e54ed771f25ef973bb543b9355
Limit formatting routes when adding resources

By default, routes.mapper.Mapper.resource adds a bunch of formatted
routes that accept anything after a '.'. Our spec says only .xml
and .json are accepted so limit the formatting to those valuse.

This allows identifiers with a '.' in urls. A few tests were added
to the extensions test to prove that .xml and .json are stripped
but other values are not.

Closes-bug: #1307956
Change-Id: I8f0b4ae3b10ea93890f6506662905bd5823412f5
Handle volumes no longer existing in resume delete

init_host resumes deletes for volumes with a status of 'deleting'. It
does these sequentially, and sometimes a volume could be deleted under
it, causing a bad stacktrace. This will gracefully handle those
situations and move on. Added coverage for resume volume delete being
successful as well.

Change-Id: I5b30265609b65804f2da77d9daa84b640de7939c
Closes-Bug: #1322340
Remove second get call to list/show volumes

Recently it was observed that with large numbers of volumes
that things like "cinder list" could take extremely long to
return.

For example, running cinder list on a system with 1000 volumes took
greater than 30 seconds to return.  It turns out that the cause of
this is the addition of visible admin_metadata.

There's two problems with this:
1. The original patch probably shouldn't have gone in
   data is either admin data or it's not, selectively picking
   pieces of admin data out to provide to the user just creates
   complications and introduces confusion.
2. The REAL issue here is that since the standar gets are made
   with the standard user context, the add_visible_admin_metadata
   would go through and do an elevated context get on every single
   volume individually.  This is what caused the horrible performance
   issue on cinder list with large numbers of volumes.

Running as admin, or removing the second call drops this down to about 3
seconds for the same 1000 volume list.

This patch removes the secondary admin context get_call.  Instead where
we expect to do display the visible admin_meta, we pass in a flag
requesting that the volume object we're getting has the appropriate
metadata.  This way we can elevate the context if needed and avoid
iterating through the gets again.

This patch also cleans up the get_visible_admin_meta methods, and
consolidates both V1 and V2 to use the utils method.

Change-Id: I3fb7aefb7d8a5664b0a3fb3958f509b5cd621320
Closes-Bug: 1317606
3PAR volume detach with host in a host set

When deleting the last VLUN for a host, try to
delete the host.  If the host is in a host set
this will fail.  When that happens, log a nice
message including the reason, but don't fail
or hang the delete VLUN.  Leave hosts in host
sets.

Closes-Bug: #1317134

Change-Id: Ie3f8f94c05a6a318de866598f9e9c4c5d84926f4
hp_lefthand_rest_proxy no handler for logger during tests

Create the proxy during do_setup() instead of __init__()
so that the logger is ready for an import error during tests.

Closes-Bug: #1325116
Change-Id: I88145d97df5d746e92bc8302bf516ca9eaa05d00
Make begin_detaching fail if volume not "in-use"

Like it's counterpart from Nova's volume-attach functionality standpoint
- reserve_volume, begin_detaching should fail if the volume is not in
  the correct state to be detached. This will prevent nova from
attempting to check and then detach, which is inherently racy.

Change-Id: Ie87eb0c9aea068affc94032ab2e53a91888d272a
Partial-bug: #1302774
Remove Quota Exception Stack Traces from Cinder Logs

Added an if clause to the Cinder API FaultWrapper so that
a warning is printed rather than an error and stack trace
if an exception of type QuotaError is thrown.

Closes-Bug: 1324294
Change-Id: If91db0f443afc212af3893e27dd36f19b15c43bf
Fixed the comment spelling error - voumes to volumes

Closes-bug: #1325481
Change-Id: I38a1d9df98c709d3ca173fa61315ad3f7ca9ffdc
Ensure flushing of IO prior to removing FC device

This patch flushes IO for each SCSI device that
brick removes from the system.  The patch also
flushes a multipath device prior to removing
each underlying LUN associated with the
multipath device.   We were seeing kernel
IO errors in the syslog without calling flush
on the block device, during a detach for a
multipath FibreChannel copy image to volume
action.

Change-Id: I2fae3dccd4fcab22c8f59dcd297581ee9577b779
Closes-Bug: 1318950
Delete image on upload-to-image failure

On upload-to-image failure before initiating the data transfer or during
data transfer, the source volume status is restored properly whereas the
image created remains in queued or saving state. This change deletes the
image during such failures.

Change-Id: I0aa64798d2bc5bf19b79dd3b88dcd107ff369c42
Closes-Bug: #1298042
Merge "Ensure flushing of IO prior to removing FC device"
Merge "Fixed the comment spelling error - voumes to volumes"
Merge "Make begin_detaching fail if volume not "in-use""
Merge "NetApp NFS: Do not reference dst_img_local before assignment"
LIO: Don't add ACL for local initiator name

Currently cinder-rtstool requires /etc/iscsi/initiatorname.iscsi
to be present in order to create a LUN/ACL/portal successfully.
However, this is not required since Cinder will create the
required ACLs dynamically at attach time as well.

Removing this path means that iscsi-initiator-utils no longer
needs to be installed locally when using a remote Nova compute
node.

Closes-Bug: 1328580

Change-Id: Ieb5e28e3f81987ab8518b861fff885f371969a56
Fix Brocade FC driver's use of the fc_fabric_names

The Brocade Fibre Channel Zone Manager driver wasn't
stripping out whitespace in the fc_fabric_names
prior to using them.  This would cause breakage later on
in the driver.

Change-Id: I6d16a35026fdd2f30a92f9dff16435a6c39b699a
Closes-Bug: 1328681
Merge "3PAR volume detach with host in a host set"