Brick connector revised fix for NFS drivers

This change fixes the error that comes
while getting brick connector and attaching volumes
in case of NFS drivers in cinder. The attribute for
mount point base was not passed to attach_volume method
as the connector initialization logic is common for all
types of protocols. It is fixed by populating
the required parameter in the RemoteFsConnector
by extracting it from connection properties. The infrastructure
for fixing this situation for remote fs drivers is implemented
in this fix. The drivers need to override the method
_get_mount_point_base in their implementation. NFS driver
implements it. GlusterFs implementation of the method to follow
in a separate patch. Hence marking it partial fix.

Change-Id: Id61a437fca1870529fa8b85f7fc2f73eae665294
Partial-Bug: #1238085
Using HttpNfcLease to transfer vmdk files.

The current VMware driver supported only "sparse" and "preallocated"
vmware_disktype property set in a "vmdk" glance image. Both of these were just
copied over as *-flat.vmdk files into the vmfs or nfs file system of the
underlying datastore. This was used during copy_image_to_volume() api.
Unfortunately for a vsan datastore this work flow breaks since there is no
access to the flat vmdk file in the underlying datastore.

This patch introduces a new vmware_disktype for a glance image called
"streamOptimized". This is a format generated when a VM/vApp is exported using
the HttpNfc APIs. AS the name suggests this is a highly optimized format for
streaming in chunks and thus would result in much faster upload / download
speeds. The driver's copy_volume_to_image() implementation now always uploads
the vmdk contents using HttpNfc api so that the glance image ends up in the
"streamOptimized" disk type. Also the driver's copy_image_to_volume()
implementation now understands a "streamOptmized" disk type and uses HttpNfc to
import that vmdk into a backing VM.

Note that the same "streamOptmized" glance image format will also be supported
by VMware nova driver. This change is in a different patch -
https://review.openstack.org/#/c/53976/

Patch Set 4: Removing changes to requirements.txt that got in by mistake.
Patch Set 5: Fixing a small bug around progress updates.
Patch Set 6: Addressing comments from Avishay.

Fixes bug: 1229998

Change-Id: I6b55945cb61efded826e0bcf7e2a678ebbbbd9d3
NetApp fix free space as zero during 1st vol stats update

NetApp clustered ontap drivers report space as zero
till first 60 seconds of driver start. This is causing
discomfort for some performance sensitive deployements.
This was due to async nature of the NetApp stats collection job.
Job is changed to sync at driver start to improve customer
experience.

Change-Id: I7d5cbf590897a0d328ece3a60516c92c0ad0ee7f
Closes-bug: #1253660
Fix typo in cinder.volume.API.

'_valid_availabilty_zone' --> '_valid_availability_zone'

Change-Id: Ib79772039d663f0eb4fc8994cc8edc5154d75a02
Closes-Bug: #1252915
To fix test_get_dss_rp in test_vmware_vmdk.py

The test "test_get_dss_rp" in test_vmware_vmdk.py is not testing
the positive case as intended. Rather, it tests the negative case
(no datastores) which is already covered by
"test_get_dss_rp_without_datastores".

Change-Id: I5b4802e22218217e0338fc721fc0ce1efeaee4f1
Closes-Bug: #1251126
Add default quota class into DB during migration

For some time now use_default_quota_class has been the
default setting for Cinder.  Cinder, however, has not been putting
any defaults for the default quota class into the database.  This
resulted in any command that queried for the default quotas to cause
the message "Deprecated: Default quota for resource: <resource> is set
by the default quota flag: <quota flag>, it is now deprecated.  Please use
the default quota class for default quota."

This commit resolves this issue by setting the default value for volumes,
snapshots and gigabytes in the quota_class table at migration time if there
is not already a class_name of 'default' in the quota_classes table.

Unit tests are included with this commit.

Closes-bug 1233763
Change-Id: I457ed8a9b78492eda22e31dfc198b2ee051d3ece
Don't do glance v2 calls when config is set to v1

Fixes bug 1252549

Change-Id: I6d9509bfecd2f73040a0ad27aff8fc5f6fd0a899
Remove suffixes from LVM brick test vgs output

When fake_execute() is called to retrieve volume group information, the
--nosuffix flag is passed but ignored.  This patch honors the nosuffix
flag and removes the unit suffix from the volume group sizes in the
returned output.

Change-Id: I45f41bc58f4ffb4bd527336c0f760350cff7c355
Remove dead code from test_get_volume_stats()

The _emulate_vgs_execute() routine is never called because the function
that calls it (get_all_volume_groups) is overridden to return fake
output without executing vgs.

Change-Id: I3d3f536f7d85128547dc9e758e3930412ebed121
Add support for extend volume in GPFS vol driver

Add support for extend volume support to the GPFS volume
driver.

Ensure that requested volume size is implemented for each
of the volume creation paths.

Replace instances where units.GiB could be used but was not.

Implements: blueprint gpfs-extend-volume-support
Change-Id: I79295205f4147fca109f5cb59497a6e1fbe6d296
Merge "Remove suffixes from LVM brick test vgs output"
Merge "Don't do glance v2 calls when config is set to v1"
Merge "Fix typo in cinder.volume.API."
Remove unused code in test_admin_actions.py

Change-Id: I66fa378005d0e8b181274a47b1746a1fe787e716
Closes-Bug: #1253931
Merge "Add support for extend volume in GPFS vol driver"
Pass the size when fetching image in xenapi driver

In commit 6e287c0f2bb7d4994d50f1763f412277e4dac6f7 the fetch_to_raw
method was changed to accept the volume size so it can fail when the
image doesn't fit in the destination volume. In this commit we also
updated all the volume drivers to pass the volume size to that method so
it could perform the check. The xenapi driver missed that change.

This commit updates the xenapi driver to pass the volume size when
fetching the image to enable the introduced check.

Change-Id: I89bc4b3b1c1e6b06f5c529166cf9ffa1fe600c96
Related-Bug: #1235358
Fix ./run_tests.sh -V --virtual-env-name

Currently if --virtual-env-name is specified, the tests fail
because it still creates .venv/ but then looks in the specified
location to use the venv.

This makes --virtual-env-name work as expected.

Change-Id: I036cd08f7b7b9dc87e7012f8dd198a26375297a2
Add greenthread.sleep() to parent wait()

Add an eventlet.greenthread.sleep() to wait() to keep
cinder-volume parent from looping after catching SIGTERM
and hogging the CPU. This was already added to the running
state but is missing from the SIGTERM caught state when
the parent waits to reap the child.

Change-Id: Ia871e31db5bf9ec4e86f926f8f6c4f0f4ecb7925
Closes-Bug: #1254089
Pull latest service module from Oslo

Get latest service module from Oslo to prepare for multi-process API service implementation.
Below are the commits included in this pull.

Changes being pulled into in service module are:
* e7bc8c9 2013-11-20 | Merge "os._exit in _start_child may cause unexpected exception"
* 96a2d4e 2013-11-07 | os._exit in _start_child may cause unexpected exception
* 1771a77 2013-11-05 | Adjust import order according to PEP8 imports rule
* 3110c0f 2013-10-17 | Use multiprocessing.Event to ensure services have started
* b5fba9e 2013-09-18 | Move comment in service.py to correct location
* 11cc74f 2013-08-26 | Fixes issue with SUGHUP in services on Windows
* 825ace5 2013-06-17 | Add service restart function in oslo-incubator
* c935d1c 2013-07-16 | Merge "Allow launchers to be stopped multiple times"
* dc8aa79 2013-07-08 | Allow launchers to be stopped multiple times
* 1a2df89 2013-06-25 | Enable H302 hacking check
* 52e857a 2013-06-19 | Ignore any exceptions from rpc.cleanup().
* 5518ad3 2013-05-16 | Add graceful service shutdown support to Launcher

And these dependent modules
 - cinder/openstack/common/eventlet_backdoor.py
    * 1dcc747 2013-07-15 | Fix stylistic problems with help text
    * 1a2df89 2013-06-25 | Enable H302 hacking check
    * c7c55b2 2013-06-20 | Improve usability when backdoor_port is nonzero
 - cinder/openstack/common/gettextutils.py
    * 3970d46 2013-11-02 | Fix typos in oslo
    * 88db9c8 2013-10-03 | When translating if no locale is given use default locale
 - cinder/openstack/common/jsonutils.py
    * 3d7504b 2013-09-23 | Ensure that Message objects will be sent via RPC in unicode format
    * 1807d32 2013-08-22 | jsonutils: make types py3 compatible
    * bdef862 2013-08-22 | jsonutils: do not require xmlrpclib
    * ded9bd6 2013-08-04 | Make dependency on netaddr optional
    * 7b7566b 2013-06-25 | Add netaddr.IPAddress support to to_primitive()
 - cinder/openstack/common/local.py
    * cb2a2b6 2013-06-28 | Modify local.py to not be dependent on Eventlet
    * 547ab34 2013-03-11 | Fix Copyright Headers - Rename LLC to Foundation
 - cinder/openstack/common/log.py
    * a82e889 2013-11-14 | Merge "Do not name variables as builtins"
    * 2251cb5 2013-11-13 | Do not name variables as builtins
    * 25c5854 2013-11-13 | Adds admin_password as key to be sanitized when logging
    * cbfded9 2013-11-11 | Default iso8601 logging to WARN
    * 76b0cd1 2013-11-04 | Add mask password impl from other projects
 - cinder/openstack/common/loopingcall.py
    * 1a2df89 2013-06-25 | Enable H302 hacking check
 - cinder/openstack/common/threadgroup.py
    * 9d3c34b 2013-10-25 | Add a link method to Thread
    * 1a2df89 2013-06-25 | Enable H302 hacking check
 - cinder/openstack/common/timeutils.py
    * f3b5f17 2013-11-12 | Add helper method total_seconds in timeutils.py
    * 53ebd30 2013-10-18 | python3: use six.text_types for unicode()
    * 3bc6f79 2013-09-19 | Fix timeutils.set_override_time not defaulting to current wall time
    * af76064 2013-08-29 | Optimize timeutils.utcnow_ts()
    * df3f2ba 2013-07-26 | BaseException.message is deprecated since Python 2.6
    * d28fa69 2013-06-27 | python3: Add python3 compatibility.

Partial bp: multi-process-api-service

Change-Id: Ifd25eae9eb2d6ae53bcf1665c3d5b7db4144433c
Merge "Using HttpNfcLease to transfer vmdk files."