Increase test coverage brick/initiator/connector

Covered next methods:
	disconnect_volume_multipath_iscsi;
	get_multipath_iqn;
	get_iscsi_devices;
	get_multipath_device_name;

Change-Id: I0eaffa6d1408446f1f13e56016642df5337015a2
Allow volume create from source unless in error status

This patch restores the source volume status checking behavior
to allow create_volume function to proceed unless it is in
error status. In FibreChannel attached environment, it is typical
to have a volume attached before the volume clone starts. The
operating system owns the volume will ensure the consistency.

Fixes: bug #1214235
Change-Id: Ie639f2f36e2939399303ca0658c169124801e86b
Fix volume_glance_metadata deletion

If volume_glance_metadata does not exist for a certain volume,
it returns an exception and does not delete volume from db.
Avoid deleting it if it does not exist.

Bug: 1209367
Change-Id: Ia607ec5ecfc115c1a593f14236ee93158938a8da
flake8 H202 error in test_image_utils.py

H202 will detect following case:
self.assertRaises(Exception, foo)

Replace Exception with test.TestingException to pass the H202 checking.

Bug #1214365

Change-Id: Idbd13d11995e6a22e99bb9d63c344a5b1a75f987
Clean up the sqlalchemy migrate manage.py script.

This way we can run it without figuring out passing in
the correct url and repository information.

Change-Id: Ib0f2c9d5c3888a1c094c8b636d735832a6242b7e
emit warning while running flake8 without virtual env

run_tests.sh -N -p
it will call the flake8 installed on your host to detect PEP8, and
the flake8 plugin "OpenStack hacking" may not installed on your
host, so this command may not detect the OpenStack Style Commandment
supplied by hacking(e.g H202).

run_tests.sh -p
it will call the flake8 from virtual env, flake8 plugin "OpenStack
hacking" installed in virtual env will be triggered.

The result from "run_tests.sh -p" should be trusted, and jenkins uses
virtual env to run flake8 too.

When "-N" is enabled, emit warning to remind user.

Bug #1208584

Change-Id: Ie08e5fa2b44088dad135e11583c046650d65df1e
GPFS use clone_image for creating volumes

If both source and target of gpfs create volume from image operation
are backed by gpfs storage use clone_image method for implementing
the move of image data to the new volume.

The copy_image_to_volume method is used only if this is not true,
and uses image_utils.fetch_to_raw to move image data to the new
volume.

Fixes bug #1213248

Change-Id: I3958febe67cc86bc3cb608288f7d064f74d3a731
Fix python 3 pep8 errors for print

The new auto-pulled requirements files exposed
python 3 compat issues in a number of modules.
We added these to tox.ini ignore temporarily,
this change updates the print routines in bin/cinder
and removes the debug cruft that was in the unit tests.

Will update requirements and test-requirements
appropriately in a follow up patch.

Change-Id: I76ea24f4dc1f61c4f8a1d202b0554c90daf2c9cc
Refactor cinder/tests/test_volume.py

Refactor test_volume.py to remove code duplication.
- remove iterant definition of the 'expected' dict from
'test_create_delete_volume' and 'test_create_delete_snapshot'
method;
- move 'test_copy_volume_to_image_status_available',
'test_copy_volume_to_image_status_use' and
'test_copy_volume_to_image_exception' methods to new class
 CopyVolumeToImageTestCase;
- move 'test_volume_get_active_by_window' and
'test_snapshot_get_active_by_window' to new class
GetActiveByWindowTestCase;
- create base class BaseVolumeTestCase for TestCase classes.
bp cinder-tests-improvement

Change-Id: I5d3644375d321e613418ea8e544557d33401ff77
Remove unused methods from cinder.utils

These methods were left over from the split from Nova and are unused still.

Remove methods:
	fetchfile()
	trycmd()
	generate_uid()
	last_octet()
	get_my_linklocal()
	parse_mailmap()
	str_dict_replace()
	flatten_dict()

Change-Id: I5d41dec66246814bb1edb6f4993d5307587a00f9
3PAR driver add missing domain check on QOS cmd

The 3PAR drivers need to support CPGs that are not part of a
virtual domain on the 3PAR backend. This patch now checks
if the domain is set before calling the createvvset command.
This instance was overlooked in the following patch that has
already been merged: https://review.openstack.org/#/c/42697/

Also fixes: bug 1212328

Change-Id: I0375dfc89104775582b9258f18f97c683e6148b1
Dont crash service if sf cluster isnt available

Currently if the SolidFire driver is configured but the cluster
isn't available for some reason the ensure_export call will raise
an unhandled exception and crash the volume-service.

We should be able to handle things like loosing connectivity to
a single back-end without impacting the other volume-service
backends.

We'll wrap the ensure_export call in a try block here and
return None in the case that the connection can't be made.
This will keep the service from crashing and log an error
message that the connection timed out.

Additional work would include adding a periodic retry task
to the manager to try and start the backend service for us
on some regular interval incase the device comes back.

Fixes: bug 1215064

Change-Id: Ice3f517d220c40113074bb77adbb10d5e32abd0b
Backup driver for IBM Tivoli Storage manager (TSM)

An implementation of Cinder backup driver using TSM as a backend for
Cinder backups. The driver is a wrapper for the TSM command line
utility dsmc, and uses TSM image backup and restore.

Re-added make_dev_path from cinder/utils.py which was removed by commit
d65425453d215ad64e2bf31b66a3b613c6e7f879

Change-Id: Id105c91ffd3ca953a9fc67dc13f31c3b885bccd7
Merge "Fix python 3 pep8 errors for print"
Move root task class to base file.

Start cleaning up the large create_volume file by
splitting up the base task and task naming helper
into its own file.

Change-Id: Ibff80dc8b10507d49374737b4760039f51d8e1ba
Merge "Backup driver for IBM Tivoli Storage manager (TSM)"
Merge "Allow volume create from source unless in error status"
Implement QoS support for volumes

This patch is to implement generic Quality-of-Service(QoS) support for volumes.
The goal is to add an interface so that cloud/Cinder admins can use to set
volume QoS, which can be enforced either in hypervisor or on Cinder back-end
or both. QoS specifications are added as a standalone (only visible to admin)
entity.  So admin can create/update/delete and associate/disassociate QoS
specifications to other entities, in this case volume types.

Note that while it's possible for Cinder to set the granularity of QoS control
to every single volume, this patch puts the control granularity to the level
of volumes of the same type to minimize the impact of other Cinder parts.
In other words, the design is to bond QoS with volume types. So Cinder admin
can associate volume types with QoS specifications, and volumes of same volume
type share the same QoS specifications.

QoS can mean a lot different things that it's unlikely we can come up with a
interpretation that all vendors can agree on.  So the approach this
implementation takes is to make Quality-of-Service specs as free-from, i.e.
expressed as key/value pairs.

Changes:
 - Add a quality_of_service_specs table, using adjacency list relation to store
 a specs entry and its detailed specs in key/values. Note that to be able to
 distinguish where should the QoS specs be consumed, each QoS specs entity
 will have a 'consumer' (i.e. fixed key) with the value of where admin would
 like the QoS policy to be enforced/consumed, currently these three values are
 considered valid: 'front-end' (Nova Compute), 'back-end' (Cinder back-end),
 'both'. The default value for 'consumer' is 'back-end';
 - Add a new API extension 'qos_specs_manage' to allow list/create/update/
 delete/associate/disassociate of QoS specs;
 - Add volume/qos_specs internal API for qos specs manipulation;
 - Add 'qos_specs' info to data structure when
 initialize_connection() is called.
 - Add 'qos_specs' to request_specs and filter properties for
 a volume create request.

 TODO
 - Modify 'type_manage' API extension to be able to accept qos info.
 - Modify volume_types.create() to accept qos info and do the checks.

DocImpact

implement blueprint: pass-ratelimit-info-to-nova

Change-Id: Iabc61b941aaff10395b30e2045e3421369a317e2
Merge "Refactor cinder/tests/test_volume.py"
Storwize/SVC: allow setting of I/O group

Allow setting the I/O group via config option or volume type. Also
updated comments and checks regarding iSCSI multipath, which is in Nova
and is not controlled by the driver.

Fixes: bug 1215363

Change-Id: Ia67fcb9991e21990cc81830145b8fd934a23488e