NetApp api fix structure conversion methods

This fixes the issue in api element creation
when data structures are supplied to be converted
into NaElements. It could not handle multiple
keys with same name and also not converted different
types properly into NaElement. This change fixes the issue.

Change-Id: I84f1ee97a73bbad88bbb64d4a9ac4e1b29cad741
Closes-bug: #1287643
Fix missing package dependency for requests

The package named requests is used by NetApp eseries
driver for doing client/server communications. This entry
is absent in requirements.txt. The patch fixes it.

Closes-bug: #1291325

Change-Id: I044ecce433d3d9e0409762a3e4551e1ce17f607d
Sync log.py from oslo

This change syncs log.py from oslo-incubator with the
following commit:
* Backport 'ident' from python 3.3 for Oslo's SysLogHandler
  79e8a9a08daf563aa8a8d9280c9a6a27dcafc8f2
* default connectionpool to WARN log level:
  1978114030c275c571a1786644a5e0d51ada29c0
* Fix deprecated messages sent multiple times
  52b64466177b58c8e80bc72c5ce1e940782c969a
* Remove None for dict.get()
  86707cd5c01a2de6b9ba386d7ea4f72c9c7286b1
* Rename Openstack to OpenStack
  61ff7a66fef51ba2a783a5ed87e14256a6e9432c
* Use ContextFormatter for imparting context info
  dda47c9cef19c5d8f050cf5081535b4061024457

Change-Id: I14c3d7eccc596783e380ebd57110337d3ce49038
Closes-bug: 904307
Add snapshot related data to limits api

When calling the limits api we can only get Volume and Gigabyte data,
the snapshot related limit value should be returned too.

Change-Id: I6a5e1e2d1aa6abbac5d5dacd042223ce49207c1f
Closes-bug: 1282924
vmware: Storage policy based volume placement.

This change adds a new feature to vmdk cinder driver. Users will be
able to specify a vSphere storage policy in a volume-type. Creating
a volume with this volume-type associates the volume with the
storage policy in vSphere. The driver places this volume only on a
datastore that satisfies this storage policy.

Implements: blueprint vmdk-storage-policy-volume-type
Change-Id: I84585e7d5fc8b28f8cfed98cb621b5b6ce9435e0
vmware: check datastore availability during create

The vmdk driver does a lazy creation of the volume's backing vmdk
only at attach time. This is done to save on the huge copy cost
if the vmdk needs to be moved 'closer' to the VM instance at
attach time.

Still during a create volume call the driver needs to verify
availability of a suitable datastore that matches the given
storage profile as well as capacity to accommodate the given
volume size. This check needs to happen at three places - create
volume, create volume from snapshot and create volume from another
source volume.

Implements: blueprint vmdk-storage-policy-volume-type
Change-Id: I327bbc4930b4ab4e81627ac7bc48c2048e2078ee
vmware: default global pbm policy configuration

Adding support for a global pbm policy configuration for the vmdk
driver. Setting the 'pbm_default_policy' in cinder.conf will be
used as the default storage profile name to be used when creating
a volume without associated vmware:storage_profile extra spec.

Also renaming 'vmware-pbm-wsdl' to 'pbm-wsdl-location' to use
nova driver's naming convention.

Implements: blueprint vmdk-storage-policy-volume-type
Change-Id: I7fad167b7be6a479db88fb4d15d07f29afd023b0
vmware: PBM wsdl file configuration

Currently the PBM wsdl files have to be downloaded to the local file
system and needs to be configured in 'pbm_wsdl_location'. If it is
manually configured then PBM feature is enabled.

This patch does away with the manual configuration of
'pbm_wsdl_location'. PBM wsdl files are now shipped with the driver.
VC version is fetched from the backend server and the corresponding
wsdl file is used. If the backend is a VC version that is less than
5.5 then PBM feature is turned off.

Implements: blueprint vmdk-storage-policy-volume-type
Change-Id: I559006b532fa39aab6d54cb0b21d5f7afba62af3
NetApp implementation for copy offload in clustered nfs driver

The Copy offload binary is a special RPC implemented in Ontap that
allows NFS clients to ask the server to copy data between
volumes in the same cluster efficiently. The special
binary will be developed by NetApp and distributed to its customers.
It will address two copy cases after efficient image clone failed.
First when image cache file is present in a different share than
the one holding volume. Second when glance is backed by nfs share
which is on the same cluster as nfs driver backend. Instead
of regular http download the copy offload workflow will be used to copy
image to share and volume. Resubmitting it as there was a problem
with build in previous submission.

Change-Id: I0847d47cecdec8be2ade06d0ea944cc3fa6f476b
Implements: blueprint copyoffload
Fix HP LeftHand Performance issue with AO

Setting AdaptiveOptimization (AO) to ‘true’, the default value,
at volume create time significantly slows down the operation on
the LeftHand array. If at create time, AO is set to ‘true’, it
will result in an update operation following the create operation
to set this value. Therefore, it is best to not specify the value,
when ‘true’, and let it default to ‘true’.

Change-Id: I2c8860e3f25a7bcaa2d2efefeffc1c11319f33e2
Closes-Bug:#1285925
Sync latest Oslo config code for i18n

This sync pulls the latest Oslo config code over
to Cinder.  This sync is needed to include a fix
to config generator which is needed as part of
the work to enabled i18n messages.  Config generator
was failing when lazy message translation was enabled.

The following changes are included with this patch. Each
patch includes the file(s) the patch touches:
-> c178e56 Add basic Python 3 tests
--> cinder/openstack/common/__init__.py
-> 12bcdb7 Remove vim header
--> cinder/openstack/common/__init__.py
-> 547ab34 Fix Copyright Headers - Rename LLC to Foundation
--> cinder/openstack/common/__init__.py
-> 44b6ea3 Import oslo.config configuration file generator
--> cinder/openstack/common/config/__init__.py
-> dd9aa2b Remove unused variables
--> cinder/openstack/common/config/generator.py
-> 5dce17b Use entry points to discover options in libraries
--> tools/config/generate_sample.sh
--> cinder/openstack/common/config/generator.py
-> e8e636c generator: add an EXTRA_LIBRARIES env variable
--> cinder/openstack/common/config/generator.py
-> e3dddd7 generator: use EXTRA_* env vars in the bash script
--> tools/config/generate_sample.sh
--> cinder/openstack/common/config/generator.py
-> 6da13e8 generator: rename EXTRA_MODULES_FILE to RC_FILE
--> tools/config/generate_sample.sh
-> 763eedf Fix DictOpt support in config sample generator
--> cinder/openstack/common/config/generator.py
-> e839886 Config generator fails with lazy messages
--> cinder/openstack/common/config/generator.py
-> 343686b Add check_uptodate to tools/config
--> tools/config/check_uptodate.sh

Oslo version:
-> 0f24d82 Fix migration.db_version when no tables
-> Date: Sat, 22 Feb 2014 00:32:18 +0000

Change-Id: I26a95fe96b08d6340b0fce1b9e2949c8e661a946
Closes-Bug: 1280826
Related-bp: i18n-messages
Re-enable lazy translation

This patch re-enables lazy translation to enable REST API
responses to be translated to the requested locale (I.E. setting
'Accept-Language: zh-CN'.)

This patch also removes the unnecessary manual import of
'cinder.openstack.common.gettextutils _' in Cinder code.  Since _ is
still being imported in the cinder.openstack.common files from Oslo
we still need to set gettexttutils.enable_lazy().  enable_lazy will
be able to be removed once the manual imports of _ have been removed
from Oslo.

implements-bp: i18n-messages
Change-Id: I0b5a35a2cc28d97d12ceccccd58565b487fef428
Include next link when default limit is reached

The /volumes and /volumes/details APIs support pagination and a
"next" link should be included when more data is available. When
the default "osapi_max" limit is reached then the "next" link is
not included in the API reply. In this case, the caller cannot
determine if there are any more volumes and has no marker value
such that they can retrieve the rest of the volumes.

The fix for this is to include the "next" link when the number of
volumes being returned is the maximum limit, even if the "limit"
parameter is not supplied.

Change-Id: I2f04192e67f80232b4019194f718625dbaf78fa6
Closes-bug: 1288429
Convert cinder utils tests to use mock

Convert the tests using mox to use mock.  Also add several
new tests to increase overall test coverage.

Closes-bug: 1285905
Change-Id: Ic7ed128e860aec50f135a9fd8728b868b27640ba
Merge "Fix HP LeftHand Performance issue with AO"
gpfs driver: fix logging problems

Fixed the string formatting template issues
Fixed the name collision on the _ variable by changing the variable name to err

Closes-Bug: #1289382

Change-Id: Id690c2b6dcd5e9732330e6b98be34819586b78fc
Merge "Include next link when default limit is reached"
Clean Up EMC VNX Direct Driver in Cinder

This patch cleans up issues discovered during the review of
EMC VNX Direct Driver.

https://review.openstack.org/#/c/73672/

Implements blueprint emc-vnx-direct-driver
Closes-Bug: #1287944

Change-Id: I4002ef9ea14e2d843dd8cbccffa025997a54c738
Merge "Clean Up EMC VNX Direct Driver in Cinder"
GPFS unit tests: increased coverage, uses mock

Increased unit tests coverage for the GPFS driver 100%.
All tests make usage of mock.
This helped to uncover bugs in the error path (see 1289382).

Closes-bug: 1290921

Change-Id: Ic78ac9b49c6e22231b966052f65d8ae07824379e