Update gpfs driver volume creation process

Modify gpfs driver to set file permissions in
a more consistent way.

Modify image_utils.resize_image to allow caller
to request it be run as root.

SecurityImpact
Change-Id: Ic01d91c0d660c74095e8d2b212279b39b9b9dc05
Partial-Bug: #1260679
Implement retype in HP LeftHand driver

This adds the ability to modify the
type of an existing HP LeftHand volume to
another type supported on the same backend.

Main use case for this is changing provisioning,
(full vs. thin), adaptive optimization and data
protections settings for a volume on-demand.

Change-Id: I78e53431708bcb3151c5dc4016e65d1a655e03c6
Implements: blueprint retype-support-for-hplefthand-driver
Add support for backing up volume metadata

This commit adds a new versioned api to the backup driver base
class which all backup driver implementations should use in order
to support backing up volume metadata as well as data. The general
rule here is that we backup all metadata from a set of db tables
and can then be selective about what we restore.

As part of this commit, Ceph backup driver implements the new api
to backup volume metadata (including glance metadata) thus
supporting restoring volumes as bootable.

It is expected that the other backup driver implementations will
follow suit and adopt this approach.

Implements: blueprint cinder-backup-volume-metadata-support
Fixes: bug 1137908
Change-Id: Icf2f1202d827d2435e09f5dad0f6f022f35dceee
Use six.moves cStringIO instead of cStringIO

to keep Python 3.x compatibility, use six.moves.cStringIO to
replace StringIO

Change-Id: Id89c6156d9bba0007e8b16cd2cc28413a7cc5fd3
Closes-Bug: #1280964
GPFS:  Implement volume backup and restore

Add support in gpfs driver for volume backup interface
to backup or restore a volume using backup service
specified in cinder.conf.

Implements: blueprint gpfs-volume-backup
Change-Id: I6f6730bab87c7e7547a66e1b57909f761f87d347
Merge "Add support for backing up volume metadata"
IBM XIV and DS8000 Driver reorganizing (IBM dir)

    * renamed:    cinder/volume/drivers/xiv_ds8k.py
                  -> cinder/volume/drivers/ibm/xiv_ds8k.py
    * updated all references to old xiv_ds8k paths
    * regenerated sample.conf

Change-Id: Iab094c78a0e4eb1753fe8809089ba1337d95e2e0
Partially implements: blueprint ibm-volume-driver-reorg
Sync request_id, request_utils for cinder

This is in preparation for work being done on request ID tracing across
OpenStack services.

Per discussion in blueprint cross-service-request-id , the standard header
name for the request ID across OpenStack projects should be
x-openstack-request-id. The request_id middleware was created for just this
purpose, as oslo change Ic7967cd62e7b743343d70f751b9238339171e013 . Using the
same middleware allows for consistency across projects.

There is an additional oslo module, request_utils, introduced as
change Id085c4444fee2bb68b80738bfe77ccb0ba1908ec to allow uniform logging of
request IDs. Bring that in as well so we have it when necessary.

Implements: blueprint add-standard-req-id-header
Change-Id: If080df2a323347924c93a912538dab5ffeffe982
Ensures NetApp iSCSI driver correctly compares int values for size

The NetApp iSCSI driver needs to verify that the new size requested
during an extend operation does not exceed the max_resize of the
lun.  However, the driver was doing a >= compare against str values
of the sizes rather than the int values.  This caused the incorrect
code path to be used in certain situations.

Change-Id: I8bc66c71db6b469c7adf00ce8b5091513fccb740
Closes-Bug: 1281279
Remove unused function

Romove unused function stub_out_rate_limiting in
cinder/tests/api/fakes.py

Change-Id: Ib6fea599eacb140108ca4293cdd089b010a8b979
Closes-Bug: #1282098
Merge "Implement retype in HP LeftHand driver"
Update 3PAR drivers to pass cert test

This patch updates both the HP 3PAR Fibre Channel
and HP 3PAR iSCSI block storage drivers so that they
can pass the driver certification tests.  We also updated
the unit tests to use mock instead of mox.

These versions of the drivers require the new 3.0.0 hp3parclient,
which can be downloaded from the pypi repository:
https://pypi.python.org/pypi/hp3parclient
The new hp3parclient requires the 3.1.3 firmware on the HP 3PAR.

The driver certification results:
Related-Bug: #1278575
Related-Bug: #1278577

Closes-Bug: #1279137

Change-Id: I72e0a76c865e54c58e41cc4409555d9042e30267
DocImpact: Document new driver requirements.
Merge "Sync request_id, request_utils for cinder"
Add support for qos_specs feature to 3PAR drivers

This patch includes the ability to set the qos-specs for the
existing 3PAR qos settings (maxIOP and maxBWS) in addition to
the new settingis listed below. We would prefer that the
new qos-specs associated to a volume type be used, but this
patch also supports setting these values the old way in extra-specs
for backwards compatibility.

This patch also adds two additional personas to the list of
valid personas that are now support by the 3PAR backends.

DocImpact: Implements new qos settings
minIOPS - The minimum IOPs per second
minBWS - The minimum bandwidth per second
latency - The QOS I/O target latency
priority - The QOS scheduling priority (high, normal, low)
The priority defaults to normal

Two new personas added include HPUX and WindowsServer

Change-Id: I7f94a493919dc2d34daac91d684369d1f51c974c
Implements: blueprint add-qosspec-support-to-3par-drivers
HP LeftHand Backend assisted volume migrate

This patch implements volume migrate using native LeftHand APIs.

Limitations:
1. Same LeftHand backend
2. Volume cannot be attached
3. Volumes with snapshots cannot be migrated
4. Source and Destination clusters must be in the same management group
5. Volume re-type not supported

Change-Id: I503d5a10ee59db14130c676a5c3a07abf9a2b7af
Implements: blueprint native-lefthand-volume-migrate
Merge "GPFS:  Implement volume backup and restore"
Merge "IBM XIV and DS8000 Driver reorganizing (IBM dir)"
Merge "HP LeftHand Backend assisted volume migrate"
Ensure return for db api functions

Functions in db/api.py should return whatever the implementation of
those functions return. This is important for qos_specs_update which
returns something for the sqlalchemy implementation, but for others as
well for future implementations.

Added 'return' for functions that were missing it.

Change-Id: I384b1cf915b2d2238fd75d23b17558cae2cf9f50
Closes-Bug: #1272365
Add HP MSA Fiber Channel driver

This commit implements a driver for the HP MSA arrays. Only Fiber
Channel is supported for now.

Implements: blueprint add-msa-2040-driver
DocImpact
Change-Id: I75232cafadf7f6d15c5959d6c054f2e0f4d14e2c