NetApp fix for compression and dedup stats

This change fixes the incorrect stats reported
for compression and deduplication by NetApp
iscsi and nfs backend.

Change-Id: I3e97d953b3ea2231c3eb1f950b0ca6705b3b2286
Closes-Bug:#1236839
(cherry picked from commit 592d860481e04312448d9ae0338066bd475fad7e)
NetApp fix mirrored stats

This fixes the incorrect reporting of mirrorred
statistics for NetApp iscsi and nfs backend.
The uninitialized and broken snapmirror is
not reported and only snapmirrored state
is reported back.

Closes-Bug:#1242808

Change-Id: I4b9f788e1f99b78d12db23bd92b6a277d5f64cb3
(cherry picked from commit a9852d6e727316d9bc07caee3096c67dc98dbb7a)
NetApp fix ssc volume filtering inconsistency

This fixes the situation when the actual netapp
volume data structure becomes inconsistent
because of operating on the actual object
than the copy of it. It also introduces some checks
before calculating volume stats.

Change-Id: I626f9ca9aa8c7cc41ba4aa6058f14cc040f7301f
Closes-Bug:#1243081
(cherry picked from commit 1d503937e841da27f5e3441775b2d208e44f5e75)
NetApp fix for 7mode iscsi volume stats

This fixes the bug for reporting correct
capacity information in case of 7mode
iscsi drivers.

Change-Id: Ie41009eca866830173809211d58470025be847e3
Closes-Bug: #1238967
(cherry picked from commit e4bde7adda4e885fe847e0bacc4d3fc84547e723)
Check if dir exists before calling listdir

Changes along the way to how we clean up and detach after
copying an image to a volume exposed a problem in the cleanup
of the brick/initiator routines.

The clean up in the initiator detach was doing a blind listdir
of /dev/disk/by-path, however due to detach and cleanup being
called upon completion of the image download to the volume if
there are no other devices mapped in this directory the directory
is removed.

The result was that even though the create and copy of the image
was succesful, the HostDriver code called os.lisdir on a directory
that doesn't exist any longer and raises an unhandled exception that
cause the taskflow mechanism to mark the volume as failed.

Change-Id: I488755c1a49a77f42efbb58a7a4eb6f4f084df07
Closes-bug: #1243980
(cherry picked from commit 1766a5acc5c948288b4cd81c62d0c1507c55f727)
Unit test fails in pbuilder environment

When running tests inside a pbuilder environment we don't
have /dev/disk/by-path directory and so the test_host_driver
test fails. Fix test by overriding the isdir method to
return True and pretend that the directory is there.

Closes-bug: #1244418

Change-Id: Ifd39d796b7f250b1ec45227a4262ebb33d5656bf
(cherry picked from commit ea51db596906c663e0c04b546b3ca79873dd74f0)
GlusterFS: set correct filename when cloning volume

When cloning a volume, the volume[name] field was populated
with incorrect data, resulting in an unexpected filename containing
the volume data.  This results in failures of later operations on
that cloned volume.

Change-Id: I067ed44cebdc8e91e9ded326953fd0c99d003f05
Closes-Bug: #1244238
(cherry picked from commit bfb66019edd197141ea1462ba78dd27a2ed0e40d)
Merge "NetApp fix for compression and dedup stats" into stable/havana
Merge "NetApp fix mirrored stats" into stable/havana
Merge "GlusterFS: set correct filename when cloning volume" into stable/havana
Merge "Unit test fails in pbuilder environment" into stable/havana
Use pipe between ceph backup diff export/import

We now use a piped transfer between the rbd export-diff
and import-diff for incremental backups/restores as
opposed to holding the entire diff in memory.

Change-Id: I33476d9b3934781413af5cd2867a11d825a5d78e
Fixes: bug 1244464
(cherry picked from commit d384d28e1c2c7c3ce0bd676fb1c2bdb8a2d98a03)
Allow spaces in quoted SSH command arguments

The check_ssh_injection() method was rejecting arguments with spaces
even when they were quoted, this was causing problems with some volume
driver commands such as commands for a storage pool with spaces in the
name.

Note that this backport also fixes a typo that has been fixed separately
in master with commit eb0f2e4dd538a79184efbb23d7e404147dfe877b .

Closes-Bug: #1244415
Change-Id: Ie4b809e1b39fdb752cf634e6d3c0a3924d8ac52b
(cherry picked from commit 2737c76cb2fb436f117a4f635aebca7a01691d88)
Merge "NetApp fix for 7mode iscsi volume stats" into stable/havana
Merge "NetApp fix ssc volume filtering inconsistency" into stable/havana
Don't zero out thin provisioned LV's on delete

Thin provisioned LV's don't need secure delete to protect
from data leakage.  Also, zeroing these out kinda defeats
the purpose of using thing provisioning.

This patch add a check for the lvm type and if it's thin simply
returns from the lvm.clear_volume() method.

Change-Id: Ie6764209018152565295291efc6fbba553698ae6
Closes-Bug: #1240299
(cherry picked from commit de4392314f9b0def8fab65679ec5668aec98fbee)
Fail when image is bigger than the volume

When creating a new volume from a qcow2 image stored in glance, we will
be given the physical size of the image instead of the virtual size.
Most drivers will convert that image to raw after downloading for
resizing it to the requested volume size afterwards.

If the virtual size of the image is bigger than the requested one, the
resulting volume might end up being unusable after the resize even
though the creation went good.

This patch will make the volume creation fail if the virtual size of
the image exceeds the one of the requested volume.

Closes-Bug: #1235358
Change-Id: I254cd9e3acf5d9eb7eb913b37d27a14e97568dec
(cherry picked from commit 6e287c0f2bb7d4994d50f1763f412277e4dac6f7)
Merge "Allow spaces in quoted SSH command arguments" into stable/havana
Merge "Check if dir exists before calling listdir" into stable/havana
NetApp fix free space as zero during 1st vol stats update

NetApp clustered ontap drivers report space as zero
till first 60 seconds of driver start. This is causing
discomfort for some performance sensitive deployements.
This was due to async nature of the NetApp stats collection job.
Job is changed to sync at driver start to improve customer
experience.

Change-Id: I7d5cbf590897a0d328ece3a60516c92c0ad0ee7f
Closes-bug: #1253660
(cherry picked from commit 585f34ff7798ad271121d432e0cae8820bc54389)