Have task/flow logging go to a separate log file

To be able to point the taskflow logging at a task log
we need to be able to separate the logs coming from cinder
from the logs coming from taskflow (as well as the listener
that acts as the intermediary between both projects).

To start the process of having a detailed task log
location, add-in a new log setting that can be used for just
this purpose.

This is not the most ideal way of doing this, since it just
provides a sample logging configuration adjustment, but there
does not appear to be a better way (due to how
cinder/openstack/common/logging.py can't be modified or adjusted
with a better option).

Part of blueprint task-logging

Change-Id: I5e3946c71110ee1b923c2258f3a7eb83c358426b
VMware: Handle exceptions raised by image update

Exceptions raised by glance image update are not handled while writing
to image. This results in volume being stuck in uploading state for
a long time even after image transfer failure. This patch fixes the
exception handling.

Change-Id: I7651c96968caa49c5421d68ee4dbe9e55f7f53b1
Closes-Bug: #1357239
Partial-Bug: #1269345
Create RemoteFSSnapDriver class

Move RemoteFSDriver into its own file.

Refactor pieces needed for qcow2 snapshots into a class.

Also add self.SHARE_FORMAT_REGEX.

Implements bp: remotefs-snaps

Change-Id: I8f9eaed125f4b916b835a63b04239058e34492a9
GlusterFS: Use image_utils for tempfile creation

Besides the aesthetical aspect of replacing ad-hoc code
with an utility function, it's more correct
to perform image conversions in CONF.image_conversion_dir,
which is taken care by the image_utils.temporary_file()
context guard.

Change-Id: Ied5607156f0ab1f1f8cfc889d5b57f3486d6b407
Pass an empty context to the notifier

The notifier API expects a context as a first parameter. During the port
to oslo.messaging, a bug was introduced here and the first argument in
the notifier call was removed. This patch puts it back.

Change-Id: Id2f47757ee4af8f8c39a8a1e6b11dcad163f5b7b
Closes-bug: #1346466
VMware: Disable suds caching

The default cache implementation in suds store pickled objects in a
predictable path in /tmp which could lead to attacks. This patch
turns off suds caching to address this security issue.

Change-Id: I7daaa25a0677004e03896298e9c3026d5c33c6ac
Closes-Bug: #1341954
Sync latest strutils from oslo-incubator

We have had a request from Tristan Cacqueray to pull in the latest
strutils.  The request is driven by a need to get improvements in
place that make strutils.mask_password more secure.

-------------------------------------

Head of oslo-incubator:
commit 80a08a413fb0f23a056eca2d273b167f0a09bab6
Merge: 83c4098 d73f3b1
Author: Jenkins <jenkins@review.openstack.org>
Date:   Mon Aug 25 14:32:36 2014 +0000

    Merge "Remove unused/mutable default args"

--------------------------------------

This sync pulls in the following change:

66142c34 - Make strutils.mask_password more secure

--------------------------------------

Change-Id: Ie51d316a30bed40996db6cd860dbb9cb173e5ac6
Partial-bug: 1345233
Fixing 3PAR excessive FC port usage

Updating the 3PAR FC driver so that it can detect if there is
only a single FC path available.  When a single FC path is
detected only a single VLUN will be created instead of one for
every available NSP on the host.  This will prevent a host
from using extra FC ports that are not needed.  If multiple
FC paths are available all the ports will still be used.

Closes-Bug: 1360001
Change-Id: I4b2bdf12289dd255dd959675991fa9e2e0fcbd1e
Fixes terminate_connection live migration issue

Reverts the changes to cinder/volume/manager.py added in
commit b868ae707f9ecbe254101e21d9d7ffa0b05b17d1 as calling
remove_export in terminate_connection causes Nova live
migration to fail when volumes are attached.

Change-Id: I38f3db336db22bb096a8855bcbbfeba16e517b80
Closes-Bug: #1361738
Add retype method to xiv/ds8k driver interface.

Change-Id: I1389a3391a17c80e47aad1ed3e7209ec47333e41
Consistency Groups

This patch enables Consistency Groups support in Cinder.
It will be implemented for snapshots for CGs in phase 1.

Design
------------------------------------------------
The workflow is as follows:

1) Create a CG, specifying all volume types that can be supported by this
CG. The scheduler chooses a backend that supports all specified volume types.
The CG will be empty when it is first created.   Backend needs to report
consistencygroup_support = True.  Volume type can have the following in
extra specs: {'capabilities:consistencygroup_support': '<is> True'}.
If consistencygroup_support is not in volume type extra specs, it will be
added to filter_properties by the scheduler to make sure that the scheduler
will select the backend which reports consistency group support capability.

Create CG CLI:
cinder consisgroup-create --volume-type type1,type2 mycg1

This will add a CG entry in the new consistencygroups table.

2) After the CG is created, create a new volume and add to the CG.
Repeat until all volumes are created for the CG.

Create volume CLI (with CG):
cinder create --volume-type type1 --consisgroup-id <CG uuid> 10

This will add a consistencygroup_id foreign key in the new volume
entry in the db.

3) Create a snapshot of the CG (cgsnapshot).

Create cgsnapshot CLI:
cinder cgsnapshot-create <CG uuid>

This will add a cgsnapshot entry in the new cgsnapshots table, create
snapshot for each volume in the CG, and add a cgsnapshot_id foreign key
in each newly created snapshot entry in the db.

DocImpact
Implements: blueprint consistency-groups

Change-Id: Ic105698aaad86ee30ef57ecf5107c224fdadf724
XtremIO cinder iSCSI & FC volume drivers for Juno

XtremIO Direct Driver has been contributed to Juno release.
support all minimum required features by Juno for both iSCSI and FibreChannel
Certificate Test Results
    https://bugs.launchpad.net/cinder/+bug/1336844

Implements: blueprint xtremio-cinder-volume-driver

Change-Id: I93ade3064532ec60662e92c56e5fefd4035a0d5b
Merge "GlusterFS: Use image_utils for tempfile creation"
Merge "Create RemoteFSSnapDriver class"
Merge "Fixes terminate_connection live migration issue"
Merge "Consistency Groups"
Introduce Hitachi storage volume driver

This patch introduces Hitachi storage volume driver.

Implements: blueprint hitachi-block-storage-driver

Certification test result for FC:
https://bugs.launchpad.net/cinder/+bug/1336661/+attachment/4189194/+files/FC%20tmp.pTAkWV3eWb

Certification test result for iSCSI:
https://bugs.launchpad.net/cinder/+bug/1336661/+attachment/4189195/+files/iSCSI%20tmp.1Q7C1rkzTY

Change-Id: Ie9b5df6d223b47d176c4e80fcf7e110543ce1d37
Signed-off-by: Seiji Aguchi <seiji.aguchi.tr@hitachi.com>
Merge "Have task/flow logging go to a separate log file"
Merge "XtremIO cinder iSCSI & FC volume drivers for Juno"
Merge "Fixing 3PAR excessive FC port usage"