NetApp api fix structure conversion methods

This fixes the issue in api element creation
when data structures are supplied to be converted
into NaElements. It could not handle multiple
keys with same name and also not converted different
types properly into NaElement. This change fixes the issue.

Change-Id: I84f1ee97a73bbad88bbb64d4a9ac4e1b29cad741
Closes-bug: #1287643
Fix missing package dependency for requests

The package named requests is used by NetApp eseries
driver for doing client/server communications. This entry
is absent in requirements.txt. The patch fixes it.

Closes-bug: #1291325

Change-Id: I044ecce433d3d9e0409762a3e4551e1ce17f607d
Sync log.py from oslo

This change syncs log.py from oslo-incubator with the
following commit:
* Backport 'ident' from python 3.3 for Oslo's SysLogHandler
  79e8a9a08daf563aa8a8d9280c9a6a27dcafc8f2
* default connectionpool to WARN log level:
  1978114030c275c571a1786644a5e0d51ada29c0
* Fix deprecated messages sent multiple times
  52b64466177b58c8e80bc72c5ce1e940782c969a
* Remove None for dict.get()
  86707cd5c01a2de6b9ba386d7ea4f72c9c7286b1
* Rename Openstack to OpenStack
  61ff7a66fef51ba2a783a5ed87e14256a6e9432c
* Use ContextFormatter for imparting context info
  dda47c9cef19c5d8f050cf5081535b4061024457

Change-Id: I14c3d7eccc596783e380ebd57110337d3ce49038
Closes-bug: 904307
Add snapshot related data to limits api

When calling the limits api we can only get Volume and Gigabyte data,
the snapshot related limit value should be returned too.

Change-Id: I6a5e1e2d1aa6abbac5d5dacd042223ce49207c1f
Closes-bug: 1282924
Quota delete operation in cinder

This change adds the quota-delete operation to cinder.

It is adding the api and the tests, using the methods already
implemented in the db to destroy all the quotas for a specific
project.

Implements blueprint quota-delete

DocImpact

Change-Id: Ia040a20abb831c311eb13624ddecbbf0f12fdd9f
GlusterFS: Set permissions on qcow2 snapshot files

The GlusterFS driver sets specific permissions on the
volume file when volumes are created.  Setting them on the
snapshot files at snapshot creation time is also required,
otherwise volume attaches will fail for volumes with snapshots
when using QEMU's libgfapi functionality.

This does not weaken the current security model in any
significant way, as it only lines up the permissions used
for snapshot files with those already used for volume files.

This will eventually be addressed by functionality in libvirt:
https://bugzilla.redhat.com/show_bug.cgi?id=1052014

Closes-Bug: #1286376

Change-Id: I93944724c557c907eb3821a061e7434f09ff20bc
Check if snapshot is deleted cleanly.

This patch added a check in delete_snapshot to make sure the
snapshot is deleted cleanly.  Without this check, if someone
tries to delete the source volume immediately after the snapshot
deletion, it will fail with error that the volume still has
snapshot.

Closes-Bug: #1286699

Change-Id: Idf4508025b403fa572584e5204b3891375fed6c6
Add EMC VNX Direct Driver in Cinder

This patch implements a driver based on the Cinder iSCSIDrver.
It performs volume operations on VNX using the NaviSecCLI command
line tool.  It supports all required driver features.

Implements blueprint emc-vnx-direct-driver

Change-Id: Iec1786612cc19452aca806b41c6be664680a923b
vmware: Storage policy based volume placement.

This change adds a new feature to vmdk cinder driver. Users will be
able to specify a vSphere storage policy in a volume-type. Creating
a volume with this volume-type associates the volume with the
storage policy in vSphere. The driver places this volume only on a
datastore that satisfies this storage policy.

Implements: blueprint vmdk-storage-policy-volume-type
Change-Id: I84585e7d5fc8b28f8cfed98cb621b5b6ce9435e0
vmware: check datastore availability during create

The vmdk driver does a lazy creation of the volume's backing vmdk
only at attach time. This is done to save on the huge copy cost
if the vmdk needs to be moved 'closer' to the VM instance at
attach time.

Still during a create volume call the driver needs to verify
availability of a suitable datastore that matches the given
storage profile as well as capacity to accommodate the given
volume size. This check needs to happen at three places - create
volume, create volume from snapshot and create volume from another
source volume.

Implements: blueprint vmdk-storage-policy-volume-type
Change-Id: I327bbc4930b4ab4e81627ac7bc48c2048e2078ee
vmware: default global pbm policy configuration

Adding support for a global pbm policy configuration for the vmdk
driver. Setting the 'pbm_default_policy' in cinder.conf will be
used as the default storage profile name to be used when creating
a volume without associated vmware:storage_profile extra spec.

Also renaming 'vmware-pbm-wsdl' to 'pbm-wsdl-location' to use
nova driver's naming convention.

Implements: blueprint vmdk-storage-policy-volume-type
Change-Id: I7fad167b7be6a479db88fb4d15d07f29afd023b0
vmware: PBM wsdl file configuration

Currently the PBM wsdl files have to be downloaded to the local file
system and needs to be configured in 'pbm_wsdl_location'. If it is
manually configured then PBM feature is enabled.

This patch does away with the manual configuration of
'pbm_wsdl_location'. PBM wsdl files are now shipped with the driver.
VC version is fetched from the backend server and the corresponding
wsdl file is used. If the backend is a VC version that is less than
5.5 then PBM feature is turned off.

Implements: blueprint vmdk-storage-policy-volume-type
Change-Id: I559006b532fa39aab6d54cb0b21d5f7afba62af3
Remove str() from LOG.* and exceptions

This commit removes the use of str() from LOG.* messages
and exceptions.

The reason we need to remove str() is because str() will fail if
it gets a Unicode string that has something that doesn't translate
to ASCII in it.  If such a situation is encountered you will lose
the message string in question.  In most cases, the use of str() is
unnecessary for LOG.* and exception messages.  Using %s is smart
enough to figure out what to do with what it is passed.  It first
tries to str() it, if this fails it falls back to using unicode.  Either
way, the result will then be something that gettextutils can
handle and translate.

Change-Id: I6eb81043edd9fa5e035d81ee81e8439340546d24
Closes-bug: 1274245
Related-bp: i18n-messages
Merge "Check if snapshot is deleted cleanly."
Merge "Quota delete operation in cinder"
Merge "Remove str() from LOG.* and exceptions"
Merge "Add EMC VNX Direct Driver in Cinder"
Merge "GlusterFS: Set permissions on qcow2 snapshot files"
NetApp implementation for copy offload in clustered nfs driver

The Copy offload binary is a special RPC implemented in Ontap that
allows NFS clients to ask the server to copy data between
volumes in the same cluster efficiently. The special
binary will be developed by NetApp and distributed to its customers.
It will address two copy cases after efficient image clone failed.
First when image cache file is present in a different share than
the one holding volume. Second when glance is backed by nfs share
which is on the same cluster as nfs driver backend. Instead
of regular http download the copy offload workflow will be used to copy
image to share and volume. Resubmitting it as there was a problem
with build in previous submission.

Change-Id: I0847d47cecdec8be2ade06d0ea944cc3fa6f476b
Implements: blueprint copyoffload
Fix HP LeftHand Performance issue with AO

Setting AdaptiveOptimization (AO) to ‘true’, the default value,
at volume create time significantly slows down the operation on
the LeftHand array. If at create time, AO is set to ‘true’, it
will result in an update operation following the create operation
to set this value. Therefore, it is best to not specify the value,
when ‘true’, and let it default to ‘true’.

Change-Id: I2c8860e3f25a7bcaa2d2efefeffc1c11319f33e2
Closes-Bug:#1285925