Adding image location selection strategies

An image can be saved to more than one backend storage by multi-location
mechanism, Glance should have a capability to select the best backend
storage for improving image consuming performance. This change
implements two strategy modules: "location_order" and "store_type", and
allow Admin configure some strategy-specified options for Glance as
their deployment required.

As an end user, he can get VM or consuming image faster base on this
enhancement. This is not only for 'download' transport handling on the
API server side but also for client since they can obtain locations by
standard 'direct URL' interface.

docImpact
Implements bp: image-location-selection-strategy

Change-Id: I86f192aeae8e5f21a72f946552f6507654c25a6c
Signed-off-by: Zhi Yan Liu <zhiyanl@cn.ibm.com>
fix bug:range() is not same in py3.x and py2.x

in python3.x, there is no method named xrange(),it
has replaced by range(),which is equal to xrange() in
python2.x. so we must fix this issue.we use six module
to fix this issue.

Change-Id: If4fd7478865e8024025af3e57e42f2a24e1e2bbc
Closes-Bug: #1268439
Sync unhandled exception logging change from Oslo

copy changes from 1 Oslo commit: Ibf00173a07510b2ec0c81a29624004f17810a4e4
This is related to the bug fix for adding error type to unhandled
exception log message

Change-Id: I5645894aad5944f6a91032ae9ac1f2a0904661f7
Closes-Bug: #1265561
Prevent E500 when delayed delete is enabled

If delayed delete is enabled the user token was not being propagated
properly to the registry in all cases. This resulted in an authentication
error when the API attempted to connect to the registry, which in turn
caused a 500 error from the API.

Fix for bug 1238604.

Change-Id: Ifafc1613db034508abfce7ba5e4df582e6056c10
Tests added for glance/cmd/cache_pruner.py

Related to blueprint glance-cmd+unittests

Integration test test_cache_pruner_main added to cover
chain of calls between glance/cmd/cache_pruner.py and glance/image_cache
module.
Unit test test_cache_pruner_main_runtime_exception_handling added to
cover exception handling for main() method of cache_pruner.py

Change-Id: I7d7b182260a407aed1288a5d136e03cbe029523f
Restore image status on duplicate image upload

Glance should not allow already uploaded images to be modified by
another upload. Currently, when configured with the local filesystem
backend, Glance prevents duplicate upload of an already uploaded image
BUT it changes the image status to "saving".

This commit adds a status transition state machine to the domain.Image
class and modifies the domain.Image.status setter function to verify
status transitions based on this state machine - only target states that
can be reached from the image's current state are permitted.

Tests have also been agumented to verify that the original image (meta)
data does not change in case of a conflicting upload.

Closes-Bug: #1241379

Change-Id: I62c5acae4c29abf0691d8279b51c59008f9c0047
Retry failed image download from Swift

If a request to Swift for an image fails before the full image is
downloaded, this will reconnect to Swift and download the remainder of
the image without having to start from scratch.

DocImpact

blueprint retry-swift-download

Change-Id: Ifde68415707506b56ffac1939c6750b216b87a87
Check first matching rule for protected properties

When using roles to define protected properties, the first matching rule
in the config file should be used to grant/deny access. This change
enforces that behaviour.

Fixes bug 1271426

Change-Id: I11ece25ae85ff868516bcd1839a4e430e9c51370
Add multifilesystem store to support NFS servers as backend

Currently to configure multiple nfs server as a backend
using filesystem store, you cannot mount all disks to a single directory.
Filesystem store allows administrator to configure only single directory with
filesystem_store_datadir parameter in the glance-api.conf.

With multifilesystem store, administrators can configure multiple directories
to store the glance image with help of filesystem_store_datadirs option.
Each directory can be coupled with its priority.

Format for filesystem_store_datadirs in glance-api.conf:
--------------------------------------------------------
filesystem_store_datadirs = </path/to/store>:<priority>
...
filesystem_store_datadirs = </path/to/store>:<priority>

Example:
filesystem_store_datadirs = /var/glance/store
filesystem_store_datadirs = /var/glance/store1:100
filesystem_store_datadirs = /var/glance/store2:200

Note:
-----
1. Either filesystem_store_datadir or filesystem_store_datadirs option
must be specified in glance-api.conf
2. Store with priority 200 has precedence over store with priority 100
3. If no priority is specified, default priority '0' is associated with it.
4. If two filesystem stores have same priority store with maximum free space
will be chosen to store the image.
5. If same store is specified multiple times then BadStoreConfiguration
exception will be raised.

blueprint: glance-multifilesystem-store

DocImpact: Multifilesystem support can be used by specifying multiple
directory paths to filesystem_store_datadirs param in glance-api.conf as
discussed above.

Change-Id: Ibb04ac14c472cd863c5e285b6dc6a08c69014fe8
Enable gating on H501

H501 - "avoid locals() on string formatting" passes
currently, so no reason to blacklist it.

Change-Id: I9fb1823a598679c0e95c88a8ce4f93629508a80c
Sync gettextutils from Oslo

Sync the latest code from Oslo to support Glance i18n.

afdbc0a Fix E501 in individual openstack projects
9d529dd Make Message keep string interpolation args
ec6d900 Add support for locales missing from babel
221c37d Allow the Message class to have non-English default locales
0e1af5b Implementation of translation log handler
2cfc1a7 Translation Message improvements

Partially implements bp i18n-messages

Change-Id: Ia56a94e42ebe1d9b539b7bc513b0342e9b2450b3
Merge "Sync unhandled exception logging change from Oslo"
Merge "Restore image status on duplicate image upload"
Merge "Adding image location selection strategies"
Merge "fix bug:range() is not same in py3.x and py2.x"
Merge "Check first matching rule for protected properties"
Merge "Enable gating on H501"
Check --store parameter validity before _reserve

Currently the value of the --store parameter is checked after the call
to _reserve is made. Thus the image is actually created on the
database even though the store is invalid, and then returns a E400.

Change-Id: Ic27ed30e605794342161530e8a8eacd6e6deb5d9
Closes-bug: #1260314
Merge "Tests added for glance/cmd/cache_pruner.py"
Merge "Sync gettextutils from Oslo"