Using HttpNfcLease to transfer vmdk files.

The current VMware driver supported only "sparse" and "preallocated"
vmware_disktype property set in a "vmdk" glance image. Both of these were just
copied over as *-flat.vmdk files into the vmfs or nfs file system of the
underlying datastore. This was used during copy_image_to_volume() api.
Unfortunately for a vsan datastore this work flow breaks since there is no
access to the flat vmdk file in the underlying datastore.

This patch introduces a new vmware_disktype for a glance image called
"streamOptimized". This is a format generated when a VM/vApp is exported using
the HttpNfc APIs. AS the name suggests this is a highly optimized format for
streaming in chunks and thus would result in much faster upload / download
speeds. The driver's copy_volume_to_image() implementation now always uploads
the vmdk contents using HttpNfc api so that the glance image ends up in the
"streamOptimized" disk type. Also the driver's copy_image_to_volume()
implementation now understands a "streamOptmized" disk type and uses HttpNfc to
import that vmdk into a backing VM.

Note that the same "streamOptmized" glance image format will also be supported
by VMware nova driver. This change is in a different patch -
https://review.openstack.org/#/c/53976/

Patch Set 4: Removing changes to requirements.txt that got in by mistake.
Patch Set 5: Fixing a small bug around progress updates.
Patch Set 6: Addressing comments from Avishay.

Fixes bug: 1229998

Change-Id: I6b55945cb61efded826e0bcf7e2a678ebbbbd9d3
NetApp fix free space as zero during 1st vol stats update

NetApp clustered ontap drivers report space as zero
till first 60 seconds of driver start. This is causing
discomfort for some performance sensitive deployements.
This was due to async nature of the NetApp stats collection job.
Job is changed to sync at driver start to improve customer
experience.

Change-Id: I7d5cbf590897a0d328ece3a60516c92c0ad0ee7f
Closes-bug: #1253660
Add default quota class into DB during migration

For some time now use_default_quota_class has been the
default setting for Cinder.  Cinder, however, has not been putting
any defaults for the default quota class into the database.  This
resulted in any command that queried for the default quotas to cause
the message "Deprecated: Default quota for resource: <resource> is set
by the default quota flag: <quota flag>, it is now deprecated.  Please use
the default quota class for default quota."

This commit resolves this issue by setting the default value for volumes,
snapshots and gigabytes in the quota_class table at migration time if there
is not already a class_name of 'default' in the quota_classes table.

Unit tests are included with this commit.

Closes-bug 1233763
Change-Id: I457ed8a9b78492eda22e31dfc198b2ee051d3ece
Remove unused code in test_admin_actions.py

Change-Id: I66fa378005d0e8b181274a47b1746a1fe787e716
Closes-Bug: #1253931
Pass the size when fetching image in xenapi driver

In commit 6e287c0f2bb7d4994d50f1763f412277e4dac6f7 the fetch_to_raw
method was changed to accept the volume size so it can fail when the
image doesn't fit in the destination volume. In this commit we also
updated all the volume drivers to pass the volume size to that method so
it could perform the check. The xenapi driver missed that change.

This commit updates the xenapi driver to pass the volume size when
fetching the image to enable the introduced check.

Change-Id: I89bc4b3b1c1e6b06f5c529166cf9ffa1fe600c96
Related-Bug: #1235358
Fix ./run_tests.sh -V --virtual-env-name

Currently if --virtual-env-name is specified, the tests fail
because it still creates .venv/ but then looks in the specified
location to use the venv.

This makes --virtual-env-name work as expected.

Change-Id: I036cd08f7b7b9dc87e7012f8dd198a26375297a2
Merge "Using HttpNfcLease to transfer vmdk files."
Merge "Add default quota class into DB during migration"
Continue to delete volumes that DNE in rbd backend

If a volume has already been deleted in the RBD backend, catch the
exception and let Cinder continue removing it in the database.

Closes-Bug: #1254318
Change-Id: I99b591ce78d82c6eaabff0459d9c523e85c8e65f
Migrate volume should check para "host" in request

The server doesn't check whether the parameter "host" is in request
body. So the 500 error has been thrown.

We should catch the KeyError and transfer the KeyError to
400 (HTTPBadRequest) instead of 500.

Closes-Bug: #1253904
Change-Id: I3fb07113816a87f284b47e32bacd57f78a32676c
Fix docstring for _migrate_volume_completion

"Migrate a volume to the specified host." as the description of
_migrate_volume_completion func is incorrect.

Closes-Bug: #1253910
Change-Id: Ica944d9a7fa962ff56a9f0107ac1519623d2e551
Initialize and terminate connection raise 500 err

Initialize and terminate connection should check whether "connector" is
in request body. It throws 500 error if "connector" is not present. We
should catch the KeyError and transfer it to 400 (HTTPBadRequest)
instead of.

Closes-Bug: #1253944
Change-Id: If38419592701c8a14df52f94fd46ed0fc7a17e04
Ensure 'status' in update_snapshot_status

The function os-update_snapshot_status doesn't check whether the param
"status" is in request body. It throws 500 error. We should catch the
KeyError and return 400 (HTTPBadRequest) instead.

Change-Id: If3775e3c0299cf0edbdda5081a6633b38462011b
Merge "Remove unused code in test_admin_actions.py"
Merge "Ensure 'status' in update_snapshot_status"
Merge "Fix ./run_tests.sh -V --virtual-env-name"
Merge "Migrate volume should check para "host" in request"
Merge "Initialize and terminate connection raise 500 err"
Merge "Fix docstring for _migrate_volume_completion"
Merge "NetApp fix free space as zero during 1st vol stats update"