delete.start/delete.end notification for hostless

Third party GUIs may rely on notifications to track the
progress of volume creation and deletion.  In the case that
a volume is being deleted after a failed attempt to create
(the volume is listed in the database but is not actually
resident in a backing store) the path that is taken in
volume.api.delete() makes no notifications of deletion
occurring.

This patch adds a volume_utils.notify_about_volume_usage
call to the beginning and end of the delete with a
delete.start and delete.end respectively.  The notifications
serve as triggers for GUIs to refresh the state of the
volume.  This change makes the hostless delete path's
functionality more consistent with the other paths through
the delete code.

Change-Id: I091b9d277834b341105569d41a48ef5c1fc105ce
Closes-Bug: 1257053
Fixes case insensitive for resp body

Create metadata for a volume with key-value set, which key in uppercase
and lowercase(e.g.{"key": "v1", "KEY": "V1"), the server accept the
request and return the key-value set {"key": "v1", "KEY": "V1"}. But the
server just add one metadata because the server is not case sensitive.

The patch will modify the resp body with the one which the server added.

update_all has the same ploblem.

Fixes errors on v2 unittest without difficulty.

DocImpact
Closes-Bug: #1258004

Change-Id: Ic337c0a351ac234493e1d73b86ba87520f32289a
Adds lock for create from vol/snap to avoid race conditions

This patch protects create from volume/snapshot by using a
lockfile to protect the operation from concurrent deletes of
the volume/snapshot used in the create operation.

Currently, if a volume/snapshot is deleted while a volume is
being created from it that delete may complete during the
create operation thus leaving the new volume in error or stuck
state. This lock will ensure that:

(a) if a create of VolA from snap/volB is in progress, any
    delete requests for snap/volB will wait until the create
    is complete.

(b) if a delete of snap/volA is in progress, any create from
    snap/volA will wait until snap/volA delete is complete.

Co-authored-by: Takashi Natsume <natsume.takashi@lab.ntt.co.jp>
Closes-Bug: 1251334
Change-Id: Ie4bc0af789ab232593f55aa2f6b34345eb9b9929
GlusterFS: Remove glusterfs_disk_util option

The glusterfs_disk_util option lets one choose whether to
use df or du to calculate space usage.  This serves no real
useful purpose.  Should remove this option to simplify things.

Since this does not meaningfully impact behavior, removal of
this option shouldn't require any special deprecation efforts.

(This same change was made for the NFS driver in 5bf7b9be.)

DocImpact: Remove config option

Change-Id: I302692454b43de15688d65873a72a3dada0c67cb
LVM migrate: Use keywords for the brick instance

In the `migrate_volume` method a new instance is created when the
dest_vg is not equal to the source vg. This new brick instance is
created using positional arguments instead of keywords. However, some of
those arguments are passed in the wrong positions.

This patch uses keywords for the misplaced arguments.

The patch also changes `test_lvm_migrate_volume_proceed` in order to
fully test the happy path and catch things like this.

Closes-bug: #1258128
Change-Id: I75cf91171709554053fd5b52a4aae4e176e8364e
Convert lvm_mirrors to int

When the location_info is parsed and `lvm_mirrors` extracted, it is not
being converted to int. This causes some boolean checks to return True
even when there are not mirrors enabled because of the u'0'.

Closes-bug: #1258133
Change-Id: I67c463d9f139fe326bbcab4d193c45459129b514
LVM migration: Check if name is equal to dest_vg

The existence check of the destination volume group is wrong. It
currently checks if there's a VG with a name equal to the source's vg
name instead of checking if it's equal to the dest_vg.

Closes-bug: #1258203
Change-Id: Ia5d4acb24b94c6aa832107c7eb4b6996985af97f
Brick LVM: Set C locale when gathering PV info

Some locales use commas as decimal separators, which means
that code breaks when someone expects to be able to use the
PV size as a float.

Change-Id: Idceb405e021ea8a0f44e19be43bad1307fbbb51c
Related-Bug: 1219963
Returns thin pool free space calculated from actual usage

This change adds a hidden method which calculates the thin
pool free space from the data_percent LVM options and reports it
accordingly in _update_volume_stats()

Change-Id: Id6a69644505ca10811db458ea90ed10f643054b2
Closes-Bug: 1249782
Sync rpc fix from oslo-incubator

Sync the following fixes from oslo-incubator:

ef406a2 Create a shared queue for QPID topic consumers
e227c0e Properly reconnect subscribing clients when QPID broker restarts

Change-Id: I934a6ea6e9eb510513640870750ea37e6f076df9
Closes-bug: #1251757
Closes-bug: #1257293
Add RBD test for volume not existing during delete

For cases when the volume does not exist in the backend, but Cinder
still knows about it, we just return from the rbd driver and let Cinder
do its thing.

Change-Id: I6150756bdd48259ec8d550fa7a5dc7a9f882ca22
Merge "LVM migrate: Use keywords for the brick instance"
Merge "Convert lvm_mirrors to int"
Merge "Fixes case insensitive for resp body"
Merge "Returns thin pool free space calculated from actual usage"
Merge "Sync rpc fix from oslo-incubator"
Merge "Adds lock for create from vol/snap to avoid race conditions"
Merge "Brick LVM: Set C locale when gathering PV info"
Sync matchmaker_ring.py from oslo-incubator

Cinder does not currently include the matchmaker driver, which is the
preferred configuration with ZeroMQ.

Change-Id: I8514f1780e6492f9d37134d282a5196709347bd8
Partial-Bug: #1185690
Merge "LVM migration: Check if name is equal to dest_vg"