Fix Huawei drivers to support other host OSs

Huawei drivers create Linux hosts by default when attaching volumes.
This patch makes them also support Windows, XenServer, AIX, etc.
The default OS is still Linux if it is not specified.

Users need to configure the host OS types in Huawei XML configuration
file. They need to set the items like this:
<Host OSType="Windows" HostIP="10.10.0.1, 10.10.0.2, ..." />
<Host .../>

When attaching a volume, the driver will get the host IP from nova. We
compare that IP with the IP in "HostIP" to get the corresponding OS type.

Closes-bug: #1229759

(cherry picked from I36fd52b97f790f1c68eaf24b6c12e7ef5d16145d)

Change-Id: I6792aaae050b126088447a0df65e10c425b84102
Remove suffixes from LVM brick test vgs output

When fake_execute() is called to retrieve volume group information, the
--nosuffix flag is passed but ignored.  This patch honors the nosuffix
flag and removes the unit suffix from the volume group sizes in the
returned output.

Change-Id: I45f41bc58f4ffb4bd527336c0f760350cff7c355
(cherry picked from commit a3ebe1fbaf90b0419a90c7b7dd6f76f59c676752)
LVM: Create thin pools of adequate size

Thin pools in LVM are quite different from volume groups or logical
volumes and their differences must be taken into account when providing
thin LVM support in Cinder.

When you create a thin pool, LVM actually creates 4 block devices.  You
can see this after thin pool creation with the following command:

    $ dmsetup ls

    volumes--1-volumes--1--pool       (253:4)
    volumes--1-volumes--1--pool-tpool (253:3)
    volumes--1-volumes--1--pool_tdata (253:2)
    volumes--1-volumes--1--pool_tmeta (253:1)

In the above command, a thin pool named 'volumes-1-pool' was created in
the 'volumes-1' volume group.  Despite this, the 'lvs' command will only
show one logical volume for the thin pool, which can be misleading if
you aren't aware of how thin pools are implemented.

When you create a thin pool, you specify on the command line a size for
the pool.  LVM will interpret this size as the amount of space requested
to store data blocks only.  In order to allow volume sharing and
snapshots, some amount of metadata must be reserved in addition to the
data request.  This amount is calculated by LVM internally and varies
depending on volume size and chunksize.  This is why one cannot simply
allocate 100% of a volume group to a thin pool - there must be some
remaining space for metadata or you will not be able to create volumes
and snapshots that are pool-backed.

This patch allocates 95% of a volume group's free space to the thin
pool.  By doing this, we allow LVM to successfully allocate a region for
metadata.  Additionally, any free space remaining will by dynamically
used by either data or metadata if capacity should become scarce.

The 95/5 split seems like a sane default.  This split can easily (and
probably should) be made user-configurable in the future if the user
expects an abnormal amount of volume sharing.

Change-Id: Id461445780c1574db316ede0c0194736e71640d0
Closes-Bug: #1245909
(cherry picked from commit d72914f739b1467ad849dd47fddd321965fed928)
Returns thin pool free space calculated from actual usage

This change adds a hidden method which calculates the thin
pool free space from the data_percent LVM options and reports it
accordingly in _update_volume_stats()

Change-Id: Id6a69644505ca10811db458ea90ed10f643054b2
Closes-Bug: 1249782
(cherry picked from commit 1679acd53d1f0c330edf583afe8b347a7304499c)
LVM: Activate Thin Pool LV upon initialization

If the LVM thin pool is not active, space calculation fails,
as data_percent is not known.  Activate the pool upon initialization,
since we intend to use it anyway.

Slightly refactors _get_thin_pool_free_space so that it is clear
which missing/unexpected value is breaking things if a similar error
occurs for some other reason.

Closes-Bug: #1260773

Change-Id: I7cb187746c1ac297b82254c6efa37ba1c5fbb3e1
(cherry picked from commit 6ccc654a0509c4e308713f9e6fd95266cd112c9d)
Add default quota class into DB during migration

For some time now use_default_quota_class has been the
default setting for Cinder.  Cinder, however, has not been putting
any defaults for the default quota class into the database.  This
resulted in any command that queried for the default quotas to cause
the message "Deprecated: Default quota for resource: <resource> is set
by the default quota flag: <quota flag>, it is now deprecated.  Please use
the default quota class for default quota."

This commit resolves this issue by setting the default value for volumes,
snapshots and gigabytes in the quota_class table at migration time if there
is not already a class_name of 'default' in the quota_classes table.

Unit tests are included with this commit.

Closes-bug 1233763
Change-Id: I457ed8a9b78492eda22e31dfc198b2ee051d3ece
(cherry picked from commit 7d2641688454d9064b691e4aab4b5d8b14d75305)
Change RBD delete failure log level to warn

This is a recoverable issue in the backend, so we don't have to provide
the message on the error level.

Change-Id: I35711876b2c088ad28f32abd39248dc9a467d00d
Closes-Bug: #1256259
(cherry picked from commit a2f2a0e0d2f9516d86ef5988e083f70804c3977c)
Fixes HP LeftHand driver with Paramiko 1.13.0

With Paramiko 1.13.0, the method exec_command now returns Unicode.
This causes a problem when the driver tries to build the XML returned
from the LeftHand array. The XML header returned from the array defines
the encoding as encoding=UTF-8. Therefore, we must now ensure the
encoding passed to the parser is utf-8.

closes-bug: #1298608
(cherry picked from commit 57e8cdb9dd8e24919bc2243438a6bded53491b3e)

Conflicts:

	cinder/tests/test_hplefthand.py
	cinder/volume/drivers/san/hp_lefthand.py

Change-Id: I7b504626e2d9a0ee2b62820b11f56eb136e31987
vmware: Fixes create volume from image error

While creating a volume with an image specified as the volume source,
the VMDK driver tries creating the volume backing file on each host
in the inventory (one by one; until any attempt succeeds). However,
if the attempted host is in a bad connection state, the volume
creation fails and results in the volume going into error state.
Effectively, user cannot create a volume from image in such scenario.
This patch catches the VimFaultException during backing file
creation, so that driver picks the next available host for volume
creation.

Change-Id: I8ffdb849f41e8c28137ad8c0fa1f86c54b5e6fe9
Closes-Bug: #1326366
Merge "Remove suffixes from LVM brick test vgs output" into stable/havana
Merge "LVM: Create thin pools of adequate size" into stable/havana
Merge "Returns thin pool free space calculated from actual usage" into stable/havana
Ensure routing key is specified in the address for a direct producer

Porting this fix from oslo.messaging.  This fixes the impl_qpid.py
driver to allow it to work with the latest stable upstream QPID broker
(version 0.28).  See the Apache Qpid Jira bug
https://issues.apache.org/jira/browse/QPID-5557

Change-Id: If71f78e50f8a9b3acfd1e9d02c8271f17c4ebee7
Closes-Bug: #1300318
(cherry picked from commit a90f0c6cfc44bc3c3fecec2f3b7281902cd6ec19)
(cherry picked from commit 4c6735fb598fbba0fea056487ac51d519d2b099f)
Merge "Fix Huawei drivers to support other host OSs" into stable/havana
Do not show quota of removed volume types in Default Quotas panel

Fixes the issue where quota for removed volume types still show up
in Admin -> Defaults -> Default Quotas view.

Change-Id: I2a822ac08b8347a8d76e8de70deca0e34fafe6e3
Closes-Bug: #1266906
(cherry picked from commit a7b6d82fd170bc594ceff791eb10b900a61c9175)
Merge "vmware: Fixes create volume from image error" into stable/havana
Merge "Change RBD delete failure log level to warn" into stable/havana
Handle volumes no longer existing in resume delete

init_host resumes deletes for volumes with a status of 'deleting'. It
does these sequentially, and sometimes a volume could be deleted under
it, causing a bad stacktrace. This will gracefully handle those
situations and move on. Added coverage for resume volume delete being
successful as well.

Change-Id: I5b30265609b65804f2da77d9daa84b640de7939c
Closes-Bug: #1322340
(cherry picked from commit b8a064f10a4dad097ef40f401015897cf9b424c3)
Merge "Do not show quota of removed volume types in Default Quotas panel" into stable/havana
Updated from global requirements

Change-Id: Id642f0727039484154628bbb7f9ec5f5eda2f633