Add encrypted flag to volumes

Currently, the only way to determine whether a volume is encrypted
is by retrieving the encryption metadata about the volume (through
cinder.api.contrib.volume_encryption_metadata) and checking the
encryption_key_id value.  This patch adds an "encrypted" flag to
the basic volume api to enable other services (like Horizon: see
patch https://review.openstack.org/#/c/71125) to easily tell
whether a volume is encrypted using a basic get call,instead of
requiring an additional call.

Implements blueprint encrypt-cinder-volumes
https://blueprints.launchpad.net/nova/+spec/encrypt-cinder-volumes

Change-Id: Id8e422135f17795de06589930afd0309fde28fd1
Fix invalid facilities documented in rootwrap.conf

The values user0 and user1 do not map to valid facility values.
local1, etc.  Using user0 results in a  pri value that does not map
back to a facility of the same name in syslog.

RFC5424 suggest values values of local0 through local7.  Setting
syslog_log_facility to one of those values results in a message with a
priority that can be mapped back to the original string value.

This fix adjusts the comment in rootwrap.conf to suggest the local
prefix instead of the user prefix.

Change-Id: I1c1233ab2386994672d22341fbe3e097409dbcbc
Closes-bug: #1280879
Sync oslo imageutils, strutils to cinder

Changes in imageutils:
===============================
bec3a5e Implements SI/IEC unit system conversion to bytes
8b2b0b7 Use hacking import_exceptions for gettextutils._
aad179d Fixing misspelled encryption key in QemuImgInfo
12bcdb7 Remove vim header
2bd46eb Refactors byte size extraction logic

Changes in strutils:
====================
bec3a5e Implements SI/IEC unit system conversion to bytes
e53fe85 strutils bool_from_string, allow specified default
8b2b0b7 Use hacking import_exceptions for gettextutils._
84d461e Fix a bug in safe_encode where it returns a bytes object in py3
12bcdb7 Remove vim header
3970d46 Fix typos in oslo
1a2df89 Enable H302 hacking check
67bd769 python3: Fix traceback while running python3.
b0c51ec Refactors to_bytes

Change-Id: I765bcd40f959162874bfda412b69e673c6e5f732
Related-bug: #1189635
Related-bug: #1193765
Related-Bug: #1257829
Add x-openstack-request-id to cinder responses

x-openstack-request-id is the common header name for request ID going forward.
Currently, only glance and neutron return this header. Using the request_id
middleware is a convenient way to ensure this header is in the response. We will
also be using the middleware to generate the request ID.

The ID will be generated by the middleware and is inserted into the request
environment. The current code generates the request ID in the constructor of
RequestConext; with this change, the ID is generated beforehand and passed in to
the RequestContext constructor. Not much different here: a request ID is
generated and is available for use in the handling of the request.

On the response end, the middleware is again used, this time to attach the
x-openstack-request-id header, using the value of the generated request ID. The
current code attaches the x-compute-request-id header.

This change is a necessary step toward logging the mapping of request IDs across
OpenStack services.

UpgradeImpact: api-paste.ini is modified

Change-Id: I30c4e5efa77f9ebffb485d2ea6a6c3f78eb1573f
Implements: blueprint add-standard-req-id-header
3PAR: Backend assisted volume migrate

This patch implements volume migrate using native 3PAR APIs.

Prerequisites:
1. Same 3PAR backend
2. Volume cannot be attached
3. Source and Dest CPGS are in the same domain

Steps:
1. Create a temporary volume in the destination with a copy of the source
2. Wait for the copy to complete
3. Delete the source
4. Rename the temporary to the same name as the original

Change-Id: Ia42503d41bf8afd908cd3ffb0288f0d353f313f1
Implements: blueprint native-3par-migrate-volume
Add support for FC zone lifecycle management

Allows automated zone lifecycle management in the attach/detach
entry points of the volume manager for FC volumes (when fabric
zoning is enabled). Zones are configured automatically as part
of the active zone set in the FC SAN to provide a more flexible
and secure way of controlling access. FC zone manager enables
SAN vendors to add support for pluggable implementations.

Change-Id: I86e02e11afec71ffa25dfbcbd0dc3ddcc35741ea
Implements: blueprint cinder-fc-zone-manager
Driver for IBM SONAS and Storwize V7000 Unified

The IBM NAS Driver currently implemented to work over NFS protocol,
and implements required set of features for OpenStack Icehouse release,
using the features of these storage products. It allows IBM® SONAS®
and Storwize® V7000 Unified filers to host storage for OpenStack
environment by integrating with Openstack Cinder service.

DocImpact

Change-Id: I0d630bf7e69454b88e9d1e93b4099631f6138204
Implements: blueprint ibm-nas-volume-driver
Related-Bug: #1281152
Merge "Driver for IBM SONAS and Storwize V7000 Unified"
Merge "Add x-openstack-request-id to cinder responses"
Fix race in test_delete_backup

Unit test sporadically failed due to timing issue - the deletion
timestamp on the backup was exactly the same as timeutils.utcnow(),
causing the AssertGreater check to fail.  Change to AssertGreaterEqual.

Change-Id: Ia66d0d4bdae3242780a91f0d384cbd17caa8c197
Closes-bug: #1284227
Merge "Add support for FC zone lifecycle management"
Merge "Sync oslo imageutils, strutils to cinder"
Merge "Add encrypted flag to volumes"
Merge "3PAR: Backend assisted volume migrate"
Added 3par initiator target map for FCZM

This patch adds the required initiator target map
for the Fibre Channel Zone Manager.   The
FCZM looks for the initiator_target_map to try and
automatically zone up the endpoints.

Change-Id: I61aaf31752f8a5eebb672887c9a943584df66995
Closes-Bug: #1284275
3PAR: Delete missing snapshot stuck in error_del

If the snapshot is deleted outside of cinder, attempting
to delete it again results in the snapshot state stuck in
error_deleting.

This is because the current code raises a NotFound exception,
but the base cinder code does not catch this and update its db.

Instead, log a warning, and simply tell the base cinder code that it was
successfully deleted.

Change-Id: I5c5d24bc113e320346ff4f1f4f62fffdf7cddfc4
Closes-Bug: #1283233
Implement retype in IBM GPFS Driver and refactor

1. Add retype method to GPFS driver.  Currently only storge_pool
   attribute is handled.  Future patches will extend this.
2. Refactor migrate method to handle migration locally if source
   and target are in the same gpfs cluster.
3. Move driver into 'ibm' directory.
4. Refactor based on pylint feedback.
5. Replace mox with mock in unit test.
6. Improve exception handling and logging.

DocImpact

Implements: blueprint gpfs-volume-retype
Change-Id: Ibc3bc367e5d42940c69d6747774c82c8cad22a35
Merge "Fix race in test_delete_backup"
Merge "Fix invalid facilities documented in rootwrap.conf"
Merge "Added 3par initiator target map for FCZM"