Improve Cinder API internal cache interface

Improve the internal caching system used by the API layer
by borrowing the implementation used by Nova.

Unlike the previous implementation, this new interface makes it clear
which resource types are added or retrieved from the cache.

This change also adds other resources to the cache:
- backups
- snapshots
- volume types

It's now possible to remove database access those extensions:
- extended_snapshot_attributes
- volume_host_attribute
- volume_mig_status_attribute
- volume_tenant_attribute

Closes-bug: #1358524
Change-Id: I1fc673d0c01c41faa98292d5813d4471b455d712
Fix unnecessary snap of glance image, with non-raw images

When the glance image format is not raw, the glance image stored on
GPFS should not be snapped. Because, we copy the image to cinder volume
repository converting the image format to 'raw'. So, no copy_on_write
is supported.

Change-Id: I9cc3ffb2f36b65876b05ecb9f4654461be4516b7
Closes-Bug: #1378964
SQL scripts should not manage transactions

Cinder SQL scripts assume too much about the environment and the way in
which they are executed by sqlalchemy-migrate. Particularly, they manage
transactions on their own even though sqlalchemy-migrate do it itself,
plus it assumes that scripts are executed in one go (even though
standard Python DB-API 2.0 does not mention multi-statement SQL commands
to be executed on DB cursors.

blueprint enable-mysql-connector

Change-Id: I10c58b3af75d3ab9153a8bbd2a539bf1577de328
Closes-Bug: 1368391
NetApp fix eseries unit test mock clean

This patch fixes the issue of mock not getting
cleaned for requests in unit tests.

Closes-Bug: #1353506

Change-Id: Iab401021d7f180ff1f2bf3ed79166699112cc367
NetApp fix for default host type in eseries

This fixes the issue where the default host type
provided in mapping should be high performing LnxALUA
type for eseries. It also makes it configurable in case
users want to configure a different host type.

Closes-Bug: #1365884

Change-Id: I30992ca69c25c3c02334470aae90c32731a5f3f4
ZFSSA iSCSI vol create fails with vol type option

Vol create with volume-type option is not working since
volume_backend_name contains the class name as
predefined string. No matter what was specified in cinder.conf
as volume_backend_name, volume creation failed.
Multi-backend option and using extra specs to create custom volumes
won't work.
The fix is to look whether volume_backend_name is part of the
configuration or falls into the class name in case there is
no backend name.

Change-Id: I1bc501dd4c5689d96c7beb720b64112df1770232
Closes-Bug: 1373621
DocImpact
Deprecate / obsolete NetApp volume extra specs

The NetApp Data ONTAP (Cluster-mode) NFS & iSCSI drivers for Juno support
the Cinder pools feature, but the drivers are reporting two qualified
extra specs that must be converted to unqualified extra specs in order to
be used by the Cinder scheduler's capability filter. Furthermore, there
are four extra specs that must be deprecated due to having the pools
feature.  Warnings will be logged during volume creation if any of the
obsolete or deprecated extra specs are seen in the volume type.

Change-Id: I4dbd667610e481356304a12b8dae84cff61aa9d9
Closes-bug: 1374630
Handle eqlx SSH connection close on abort.

EqualLogic array CLI operation timeout causes the
SSH thread to be aborted. This would cause SSH
sessions to be orphaned and hit a max connection
limit on the array. This fix catches these aborts
and makes sure the connection is closed.

Change-Id: I9392fd5dd79eb44f252bf50217f17cc473e6f2f0
Closes-Bug: 1374613
Clarify InvalidInput exception when the size is missing

In the case where size is None, the exception message
'Volume size  must be an integer...' is misleading.

Change-Id: Ida34a84f8b3d156e3dca54de594c991b5ef73295
Refuse invalid qcow2 backing files

Don't allow qcow2 files that are pointing to backing files outside of:

volume-<id>
volume-<id>.<snap-id>
volume-<id>.tmp-snap-<snap-id>

(optionally prefixed with /mnt/path)

Closes-Bug: #1350504

Change-Id: Ic89cffc93940b7b119cfcde3362f304c9f2875df
Windows SMBFS: Handle volume_name in _qemu_img_info

The volume_name is now parsed to the _qemu_img_info wrapper. As
this method is not prone to security issues because this driver
does not support raw images (at least not yet), we don't have to
perform any checks on the backing image file path.

Thus, this method simply ignores this argument that will be parsed
by the base class methods.

Related-Bug: #1350504

Change-Id: I801a6338250ec2dc631c4058543f7d0088b3e4d4
Merge "Deprecate / obsolete NetApp volume extra specs"
Merge "Refuse invalid qcow2 backing files"
Merge "Handle eqlx SSH connection close on abort."
Merge "Clarify InvalidInput exception when the size is missing"
Merge "ZFSSA iSCSI vol create fails with vol type option"
Merge "Windows SMBFS: Handle volume_name in _qemu_img_info"
Merge "NetApp fix eseries unit test mock clean"
Sync latest processutils from oslo-incubator

An earlier commit (Ia92aab76fa83d01c5fbf6f9d31df2463fc26ba5c) failed
to address ssh_execute(). This change set addresses ssh_execute.

------------------------------------------------

oslo-incubator head:

commit 4990535fb5f3e2dc9b397e1a18c1b5dda94ef1c4
Merge: 9f5c700 2a130bf
Author: Jenkins <jenkins@review.openstack.org>
Date:   Mon Sep 29 23:12:14 2014 +0000

    Merge "Script to list unreleased changes in all oslo projects"

-----------------------------------------------

The sync pulls in the following changes (newest to oldest):

6a60f842 - Mask passwords in exceptions and error messages (SSH)

-----------------------------------------------

Change-Id: Ie0caf32469126dd9feb44867adf27acb6e383958
Closes-Bug: #1377981
coraid: allow setting default repository

Currently, the Coraid driver requires storing Storage Repository
name in volume type key named 'coraid_repository' or different key name
if a value of the 'coraid_repository_key' option changed in the cinder
configuration file.

In case of creating a volume without volume type associated or creating
a volume with volume type that doesn't have an appropriate key with
the repository name, volume creation fails.

That is an inconvenient behaviour, especially in case of using that with
Tempest that creates new volume types but obviously doesn't know
that it should add keys with the repository name.

In order to fix that, introduce a 'coraid_default_repository' config
option that allows to set a default repository to use if one is not
specified in the volume type key.

In case if neither volume type nor config file provide a repository
name, throw CoraidException with description of the problem.

DocImpact

Change-Id: Ib205553cdee0af0036a774f6926433f4452ec023