NetApp fix clone image compatibility issue with ssc

It makes clone image functionality compatible
with the ssc feature. Hence it makes clone
image sensitive to volume type and provision
on the right share matching volume type
extra specs.

Change-Id: If980239fedf784a1cca7abd6ee3d00f8625b9b11
Closes-Bug:#1226752
Changed header from LLC to Foundation based on trademark policies

Fixes: Bug 1214176

Change-Id: Ieda1a75992abec647c267faa7727f5da4f7a88a3
Fix finish_volume_migration() on SQLAlchemy 0.8.x

In SQAlchemy 0.8.x "Unconsumed column names" warning became
an exception, so refering to a non-existent column in insert()
or update() call raises an error.

finish_volume_migration() calls Session.update() method passing
values of two non-existent columns as arguments (volume_metadata,
volume_admin_metadata, volume_type). These two are not table columns
at all, but rather SQLAlchemy models relationships.

As SQLAlchemy ORM implements Unity of Work pattern, we should not
really track changes to a model instance manually at all, because
Session class already does it for us. finish_volume_migration()
is refactored to take benefit of this fact.

Fixes bug 1206561

Change-Id: I4513e3155a7dc6dcbd1c95aa9c14d1e1e5d02ab4
Fix mandatory and optional args for create_volume

Calls to create_volume (both volume_rpcapi and scheduler_rpcapi) pass
mandatory args as optional and reverse. This patch fixes the calls to be
according to the function definition.

Also, fix the call to create_volume in the migration code where
allow_reschedule defaults to True, but we don't pass request_spec so it
prints a warning and disables it.

Change-Id: I9509f88866e0ae182f90ebdfca1fcf415ca60d19
Closes-Bug: #1230069
Fix translation of CinderExceptions in REST API

When creating a Fault from a CinderException wrapped in an
HTTPException, we were converting the inner explanation to unicode
before it was able to reach the Fault's call() method which is where
translation occurs, and unicode objects can't be translated.

This patch preserves the CinderException's Message object and puts it in
another Message object as the explanation to the HTTPException so it can
be translated.

Fixes bug: #1229967

Change-Id: Ida71908b639da32b4b85846a117ef21da2fe685b
Set vol driver initialized before deleting volumes

Move the call to set_initialized before calling delete_volume in
init_host.  The delete_volume method has a precondition that the
driver be initialized.

Closes-Bug: #1232177

Change-Id: Ie73bd30e0ab80d489ab61ba8a60fac51ccb505b5
Validate force_host_copy API param for migration

For the force_host_copy parameter of volume migration, make sure that we
get a boolean or string that we can convert to boolean.

Change-Id: I7a77ca1780a4ef80bc351aa89df0efaaea0d7cf4
Closes-Bug: #1232698
Add support for querying the quotas usage

QuotaSetsController's _get_quotas already supports querying for
the usage. This patch allows sending a parameter "usage" to the
REST API when asking for quotas

Change-Id: I2103f41ae7aac1ed9f071949db4fb205552694f5
Fix Huawei HVS driver attaching volume error

If iSCSI initiator is not added to host, we will get https errors
for we find the initiator info by sending url with this initiator name.

This patch fixes the way of getting the initiator info.
First, get the all initiator info.
Then, find the one we need by name.

Closes-bug: #1230296
Change-Id: I92620374923fa136ee71fe6eb3af6e4c78a3d66b
VMware: Usng RetrvProprtisEx & does multi ESX scan

The existing code uses deprecated API RetrieveProperties for querying through
property collector. The public doc asks to use RetrievePropertiesEx instead.

RetrievePropertiesEx works by retrieving results in serveral batches rather than
retrieval in a single API call. This works well in environments containing
large number of VMs or ESX hosts.

There are three steps involved 1. Get first batch 2. Continue retrieving using
a token 3. Cancel retrieval when not needed

Batch size is user configurable, default is 100 results per batch.

The review also fixes a careless programming error leading to scanning only
1 ESX host under the vCenter server for datastores.

Fixes bug: 1227078
Fixes bug: 1229654
DocImpact

Change-Id: I85a6f6a7e2d764b9843efbd0dd347fba3ef19310
Create volume revert fails for non admin user

Done fixes in revert method of EntryCreateTask task
to use elevated context while destroying volume.

Fixes bug #1230159

Change-Id: I41b7f2ae4e0235df40bcdaa511256775228751aa
VMware ESX: Fixes vol clone & clone from snapshot

The driver was earlier performing create from a source volume and create from
a snapshot point by copying the source volume container folder.

This is not valid in case of vSAN datastore where the volume data is not a
'file' in the container folder unlike VMFS/NFS datastores. So copying folder
will only copy descriptor file and hence the source and cloned volume will
end up writing to the same data location, eventually corrupting it.

The fix is to copy only the VMDK file (descriptor + data) and not the entire
container folder. The implementation does the following:
1. Create a volume backing
2. Delete the current VMDK
3. Make a copy of source volume's VMDK file to the destination using virtual
   disk manager. In case of create from snapshot, we copy the VMDK file
   corresponding to the snapshot point.

Fixes bug: 1233034

Change-Id: I51c9870c11a6d66f9f47c1ad9aaa2abc3dd0533a
Fixes backup with multiple volume backend

The backup service now keeps a dictionary of managers
for each type of volume backend. If multi backend is
not in use (and volume_type is None) the current
volume_driver is used.

Change-Id: I66e91f3ce2cffaf2e2d07edfddc08b1480e2a37c
Fixes: bug 1228223
v2 api - return bootable attr value on volume list

Add bootable to the list of values returned by display_list.  This
was returned in v1 api, and is still in v2 header, but was missed
in the new implementation.

Closes-Bug: #1232287

Change-Id: If7460b1c8ab4af417117c4bf6cfdccc5fcf21f46
Add XML response tests for qos specs manage ext

Make sure XML serializing is working for qos spec index and show, as
well as association show.

Change-Id: If678cf9201c3ba218a485999e11fc034a06495d4
Fixes ceph backup import errors

For some reason, using non-rbd volume driver whilst using
Ceph backup causes what looks to be a cyclic import error.
Ensuring that the backup.drivers.ceph module is imported
into volume.drivers.rbd ONLY if it is needed i.e. if both
drivers are in use, seems to remedy this.

Change-Id: Ieae0c3451cfb1750eca8cfcde5e0be9e7eb028d0
Fixes: bug 1231412
usedevelop in tox

We want to run code inside of the venv, not outside.

Change-Id: I92c5e35674933ac4d9d15eda32f3b9339dbedbd3
Merge "Set vol driver initialized before deleting volumes"
Merge "Add XML response tests for qos specs manage ext"
Merge "Fix Huawei HVS driver attaching volume error"