Change default policy for volume_extension:volume_tenant_attribute

The volume_tenant_attribute extension should be accessible by
the owning tenant and the admin. Change default policy from
'rule:admin_api' to 'rule:admin_or_owner'.

Change-Id: I886cd717741c86b5083aeaf16e3e7ea5b4e525b4
Closes-bug: #1269312
Nexenta iSCSI driver: fixed volume_migration

Fixed volume_migration updating volume provider_location.
Fixed temporal snapshot deleting after volume migrated.

Change-Id: Id23749cb3724e1510865425232b38b9fc32690f8
Closes-Bug: #1263258
Brick support for pNFS

This change implements a retry mechanism
during mounting nfs share. It tries mounting
using pnfs if no specific nfs version is specified
in the nfs options. In case of any failure during mounting
it falls back to regular nfs mount which will internally
try to mount with highest supported version. If nfs version
is specified in options then regular mount will be allowed.

Change-Id: I502b375548707ce7021cbacf1cd4f5532804aeb3
Implements: blueprint pnfs
Remove unused methods

 o cinder/volume/driver.py
   - def _get_iscsi_initiator(self):

     This method was removed in the following commit.

       commit cb6faab4d99801e76195802659013a5ccdc6b5b5
       Author: Walter A. Boring IV <walter.boring@hp.com>
       Date:   Wed Jul 10 15:22:06 2013 -0700

           Add Brick Fibre Channel attach/detach support.

 o cinder/volume/utils.py
   - def get_host_from_queue(queuename):

     This method was added in the following commit.

       commit 6c708d12f58eb20fce6733f1f6fd08d978570775
       Author: Michael Basnight <mbasnight@gmail.com>
       Date:   Mon Feb 11 15:54:45 2013 -0600

           Update cinder-volume to enable multi volume support

     But there is no evidence that was used.

   - def notify_usage_exists(context, volume_ref, current_period=False):

     This method was removed in the following commit.

       commit d09b7c78b3c905badb1112434eb9c09c2c8ffb5a
       Author: Ollie Leahy <oliver.leahy@hp.com>
       Date:   Thu Dec 12 11:01:34 2013 +0000

           Allow user to specify audit period.

Change-Id: I819679e03df9a2a848004d14e3f55821c88834a4
Removes use of timeutils.set_time_override

The set_time_override function in timeutils was written as a
helper function to mock utcnow for unittests before 'mock' was
generally used. Now that we have mock and fixture, we no longer
need to use it.

Change-Id: I057d4bc3a8b1d4e96e6830cd3071dc96a05496dc
Partial-Bug: #1266962
VMware: changing volumeops tests from mox to mock.

Changing unit tests implementation of volumeops to mock. In the process
also moving all these unit tests into a separate class for better
test organization.

Partial-Bug: #1261097
Change-Id: I08b2341b22ed9c8f63930a2c73a9737cbe7ef540
Deactivate LV before removing

With certain versions of LVM2, removing an active LV can end up with

  Unable to deactivate open XXX
  libdevmapper exiting with 1 device(s) still suspended.

which causes any lvm command afterwards to hang endlessly on
trying to access the suspended volume. This seems to be caused
by a race with udev, so lets be conservative and do the deactivation,
then wait for udev and then finish the removal.

Closes-Bug: #1270192

Change-Id: I4703133180567090878ea5047dd29d9f97ad85ab
NetApp eseries iscsi driver implementation

This change introduces the iscsi
driver for NetApp eseries family of storage
systems. E-series is another  storage
solution offered by NetApp.

Related-Bug: 1278567
Implements: blueprint netapp-eseries--cinder-driver
Certification results: https://launchpadlibrarian.net/166301064/eseries_cinder_certification.log

Change-Id: I8b55d1f1c8de12052281c7cae0c2f89a9d2c3dd1
Remove rabbit_notifier (Sync notifier with oslo d6e1ba7)

rpc_notifier is available since grizzly, and rabbit_notifier
was printing deprecation messages since then. Remove it
for icehouse. This includes a sync to oslo-incubator, which
includes the following commits:

  aff0171 Remove "vim: tabstop=4 shiftwidth=4 softtabstop=4" from headers
  8575d87 Removed copyright from empty files
  8b2b0b7 Use hacking import_exceptions for gettextutils._
  6fa29ae Trivial: Make vertical white space after license header
          consistent
  d7ae4ff Remove deprecated Grizzly features

DocImpact: Remove all references to rabbit_notifier and replace
it with rpc_notifier.

Change-Id: I889b43dfcebe6574dfa00ea50818f85732860d98
Add encrypted flag to volumes

Currently, the only way to determine whether a volume is encrypted
is by retrieving the encryption metadata about the volume (through
cinder.api.contrib.volume_encryption_metadata) and checking the
encryption_key_id value.  This patch adds an "encrypted" flag to
the basic volume api to enable other services (like Horizon: see
patch https://review.openstack.org/#/c/71125) to easily tell
whether a volume is encrypted using a basic get call,instead of
requiring an additional call.

Implements blueprint encrypt-cinder-volumes
https://blueprints.launchpad.net/nova/+spec/encrypt-cinder-volumes

Change-Id: Id8e422135f17795de06589930afd0309fde28fd1
NetApp api fix structure conversion methods

This fixes the issue in api element creation
when data structures are supplied to be converted
into NaElements. It could not handle multiple
keys with same name and also not converted different
types properly into NaElement. This change fixes the issue.

Change-Id: I84f1ee97a73bbad88bbb64d4a9ac4e1b29cad741
Closes-bug: #1287643
Use six.moves cStringIO instead of cStringIO

to keep Python 3.x compatibility, use six.moves.cStringIO to
replace StringIO

Change-Id: Id89c6156d9bba0007e8b16cd2cc28413a7cc5fd3
Closes-Bug: #1280964
Fix invalid facilities documented in rootwrap.conf

The values user0 and user1 do not map to valid facility values.
local1, etc.  Using user0 results in a  pri value that does not map
back to a facility of the same name in syslog.

RFC5424 suggest values values of local0 through local7.  Setting
syslog_log_facility to one of those values results in a message with a
priority that can be mapped back to the original string value.

This fix adjusts the comment in rootwrap.conf to suggest the local
prefix instead of the user prefix.

Change-Id: I1c1233ab2386994672d22341fbe3e097409dbcbc
Closes-bug: #1280879
test_volume unittest fails if ran only this module

Using stevedore ExtensionManager to load entry_points(extensions)
before running any test case.

In BaseVolumeTestCase we have stubbed 'os.path.exists' call,
because of this stevedore is not able to load entry points paroperly.
Stevedore internally use pkg_resources and pkg_resources use
'os.path.exists' to check if entry_points.txt file exists or not.
Because of stubbibg it is returning True even if file do not exists.

Change-Id: I4fd6bee2e9f33c45ad3deacfe7359dc56171205e
Closes-Bug: #1263175
Fix missing package dependency for requests

The package named requests is used by NetApp eseries
driver for doing client/server communications. This entry
is absent in requirements.txt. The patch fixes it.

Closes-bug: #1291325

Change-Id: I044ecce433d3d9e0409762a3e4551e1ce17f607d
Sync oslo imageutils, strutils to cinder

Changes in imageutils:
===============================
bec3a5e Implements SI/IEC unit system conversion to bytes
8b2b0b7 Use hacking import_exceptions for gettextutils._
aad179d Fixing misspelled encryption key in QemuImgInfo
12bcdb7 Remove vim header
2bd46eb Refactors byte size extraction logic

Changes in strutils:
====================
bec3a5e Implements SI/IEC unit system conversion to bytes
e53fe85 strutils bool_from_string, allow specified default
8b2b0b7 Use hacking import_exceptions for gettextutils._
84d461e Fix a bug in safe_encode where it returns a bytes object in py3
12bcdb7 Remove vim header
3970d46 Fix typos in oslo
1a2df89 Enable H302 hacking check
67bd769 python3: Fix traceback while running python3.
b0c51ec Refactors to_bytes

Change-Id: I765bcd40f959162874bfda412b69e673c6e5f732
Related-bug: #1189635
Related-bug: #1193765
Related-Bug: #1257829
Use len instead of for-loop to get the end index

Use len() instead of for-loop to get the end index in file
cinder/tests/test_backup_tsm.py

Change-Id: I3a7b0e418f8af4881e21eb5273a03dbcd317c27d
Closes-Bug: #1282084
Update 3PAR drivers to pass cert test

This patch updates both the HP 3PAR Fibre Channel
and HP 3PAR iSCSI block storage drivers so that they
can pass the driver certification tests.  We also updated
the unit tests to use mock instead of mox.

These versions of the drivers require the new 3.0.0 hp3parclient,
which can be downloaded from the pypi repository:
https://pypi.python.org/pypi/hp3parclient
The new hp3parclient requires the 3.1.3 firmware on the HP 3PAR.

The driver certification results:
Related-Bug: #1278575
Related-Bug: #1278577

Closes-Bug: #1279137

Change-Id: I72e0a76c865e54c58e41cc4409555d9042e30267
DocImpact: Document new driver requirements.
Add x-openstack-request-id to cinder responses

x-openstack-request-id is the common header name for request ID going forward.
Currently, only glance and neutron return this header. Using the request_id
middleware is a convenient way to ensure this header is in the response. We will
also be using the middleware to generate the request ID.

The ID will be generated by the middleware and is inserted into the request
environment. The current code generates the request ID in the constructor of
RequestConext; with this change, the ID is generated beforehand and passed in to
the RequestContext constructor. Not much different here: a request ID is
generated and is available for use in the handling of the request.

On the response end, the middleware is again used, this time to attach the
x-openstack-request-id header, using the value of the generated request ID. The
current code attaches the x-compute-request-id header.

This change is a necessary step toward logging the mapping of request IDs across
OpenStack services.

UpgradeImpact: api-paste.ini is modified

Change-Id: I30c4e5efa77f9ebffb485d2ea6a6c3f78eb1573f
Implements: blueprint add-standard-req-id-header
Add support for qos_specs feature to 3PAR drivers

This patch includes the ability to set the qos-specs for the
existing 3PAR qos settings (maxIOP and maxBWS) in addition to
the new settingis listed below. We would prefer that the
new qos-specs associated to a volume type be used, but this
patch also supports setting these values the old way in extra-specs
for backwards compatibility.

This patch also adds two additional personas to the list of
valid personas that are now support by the 3PAR backends.

DocImpact: Implements new qos settings
minIOPS - The minimum IOPs per second
minBWS - The minimum bandwidth per second
latency - The QOS I/O target latency
priority - The QOS scheduling priority (high, normal, low)
The priority defaults to normal

Two new personas added include HPUX and WindowsServer

Change-Id: I7f94a493919dc2d34daac91d684369d1f51c974c
Implements: blueprint add-qosspec-support-to-3par-drivers