Moving eventlet.hubs.use_hub call up

Under the design concept of eventlet, we need to call
eventlet.hubs.use_hub() before the application starts
doing any I/O. And it makes custom log handlers register
correct event listeners.

Closes-bug: 1247889
Change-Id: I3d43021c8cb6ed630123781f3686d62e5159edfc
Signed-off-by: Zhi Yan Liu <zhiyanl@cn.ibm.com>
Adding status field to image location -- scrubber changes

Adding a status field to image's each location property, each location
status can be 'active', 'pending_delete' and 'deleted'.

Under location's status information Scrubber service can make cleanup
based on DB records also but not a dedicated queue-file for each image.

This is third part of this change which covered scrubber DB queue
enablement and double queue supporting.

Partially-Implements BP: image-location-status

Change-Id: I8ee9e9254c8371bbbce8b28d643e3d474b2361bc
Signed-off-by: Zhi Yan Liu <zhiyanl@cn.ibm.com>
Fixes the failure of updating or deleting image empty property

When property protection feature enabled, end user couldn't uses
Glance v2 API to update or delete a property of an image successfully
which value is empty, client will receive a http 500 error and a
relevant error log could be found in glance-api service log.

Closes-bug: #1332103

Change-Id: I1f9f181cea714e6ba26388d125bb7023e7a14305
Signed-off-by: Zhi Yan Liu <zhiyanl@cn.ibm.com>
Enable F821 check: undefined name 'name'

Enable F821 and fix usage of undefined variables.

In glance/common/client.py in BaseClient class image_iterator()
function returns instances of unexisting classes:
- SendFileIterator
- ImageBodyIterator
Since we have not these classes functionality now, we will use
utils.chunkreadable() function instead.

Delete image_iterator() function.

Change-Id: Ie1a8c66e5df2836e6e9aadfb1013d69fc6f710e6
Merge "Moving eventlet.hubs.use_hub call up"
Do not log password in swift URLs in g-registry

There was a debug level log with the locations added to it.
This change fixes the log to not contain that sensitive info.

Fixes bug 1348838

Change-Id: I3c5b29616c8d76bed17dbd31a8f4fc7ccd2dd945
Quota column name 'key' in downgrade script

006_mysql_downgrade didn't correctly quote 'key' column of
'image_properties' table when creating index, the index wasn't created
as needed, and then operator could not run any upgrade script
successfully since 006_mysql_upgrade script could not to handle this
unexpected db status.

Note, this issue doesn't impact one-way upgrade migration operation,
since bug existed in downgrade script. But once operator do a
reciprocating migration operation, this issue will rise up as a blocker.

Closes-bug: 1351413

Change-Id: Ie1ed1b7f06b8a0aeede70c3d4dd061c857e448da
Signed-off-by: Zhi Yan Liu <zhiyanl@cn.ibm.com>
Trivial fix typo: Unavilable to Unavailable

This patch fixes a typo in an exception message.

Change-Id: I204b0bff94970d8743872cd184b8431d30dd1341
Merge "Fixes the failure of updating or deleting image empty property"
Merge "Do not log password in swift URLs in g-registry"
Merge "Trivial fix typo: Unavilable to Unavailable"
Merge "Enable F821 check: undefined name 'name'"
Fix RBD store to use READ_CHUNKSIZE

Change 4e0c563b8f3a5ced8f65fcca83d341a97729a5d4 was incomplete and
missed a variable in the rbd store.

This patch fixes the issue by using store.READ_CHUNKSIZE instead of
store.chunk_size

Change-Id: I2dc5b1514d3e86ec129c1fc88de096d98a64f806
Closes-Bug: #1352509
VMware store: Use the Content-Length if available

Change I579084460e7f61ab4042632d17ec0f045fa6f5af changed the VMware
store to use chunked encoding to upload data to the underlying backend.
We should do that only if the image size is not provided or zero and not
all the time.

This patch addressed the issue by checking the image_size before upload.

Change-Id: If348be7fd24fd05146b0c86bef45b79f600cc0f7
Closes-Bug: #1336970
Fix image killed after deletion

If an image upload fails after the image was deleted during
the upload, the image status currently gets set to 'deleted'
then 'killed' once the upload fails. This patch fixes this so
that if the image was deleted it stays in 'deleted' state.

Permitted from_states for transitioning to 'killed' are
'queued' and 'saving'. We now enforce this transition and
raise an exception if it is violated.

Closes-Bug: 1236868
Co-authored-by: Hirofumi Ichihara <ichihara.hirofumi@lab.ntt.co.jp>
Co-authored-by: Zhi Yan Liu <zhiyanl@cn.ibm.com>
Change-Id: I5552220ce264c0d09780bbd0eb7610e7cff0b820
Integrate OSprofiler and Glance

*) Add osprofiler wsgi middleware
This middleware is used for 2 things:
1) It checks that person who want to trace is trusted and knows
secret HMAC key.
2) It start tracing in case of proper trace headers
and add first wsgi trace point, with info about HTTP request

*) Add initialization of osprofiler at start of serivce
Set's olso.messaging notifer instance (to send notifications to
Ceilometer)

*) Add tracing support for sqlalchemy

*) Add profiler CONF group that has to options:
1) Enable or disable profiler
2) Enable or disable sqlalchemy tracing
(the reason why we put this in Conf options, is that usually
db requests create a lot of trace info)

*) Glance-registry wasn't setted properly to send notifications.

To test it you need 2 things:
1) Enable ceilometer in devstack:
http://paste.openstack.org/show/85724/
2) Initialize osprofiler and send out API request. You can use
one of follow approaches:
a. Run this script (pls provide proper credentials)
http://paste.openstack.org/show/85722/
b. Use the function https://review.openstack.org/#/c/111184/
added from glanceclient CLI directly.

DocImpact

Change-Id: I45a19f5eb5304c2b78a9e12cbc0744941a807304
Co-author: Zhi Yan Liu <zhiyanl@cn.ibm.com>#
Fix rally performance job in glance

Recently we merged non backward compability change in rally
https://review.openstack.org/#/c/112045/

So we need to fix plugin

As well change name to something reasonable

Change-Id: I040959e9d1e36f524e88174c9cbbeb7db2f4b66b
Merge "Quota column name 'key' in downgrade script"
Merge "Fix rally performance job in glance"
Merge "Adding status field to image location -- scrubber changes"