Improved coverage for glance.api.*

Added new tests for glance.api.v2.image_data,
glance.api.v2.image_members, glance.api.v2.images,
glance.api.v2.schemas, glance.api.v2.image_tags,
glance.api.cached_images.

Part of blueprint improve-api-unit-tests

Change-Id: I2bd39e12366dd1b185bf60a8cd1d03a162ba5896
Catch exception.InUseByStore at API layer

If user create an image with Ceph backend driver, and then create
an instance based on the image. As a result, user will run into
500 error when deleting the image. Because rbd driver is raising
exception.InUseByStore()at [1] when the image is in use, but no
code to catch the exception at REST API layer.

[1] https://github.com/openstack/glance/blob/master/glance/store/rbd.py#L290

Closes-Bug: 1345942

Change-Id: I350c235944aab226dc96de169be2087e2b599ab4
Fix image killed after deletion

If an image upload fails after the image was deleted during
the upload, the image status currently gets set to 'deleted'
then 'killed' once the upload fails. This patch fixes this so
that if the image was deleted it stays in 'deleted' state.

Permitted from_states for transitioning to 'killed' are
'queued' and 'saving'. We now enforce this transition and
raise an exception if it is violated.

Closes-Bug: 1236868
Co-authored-by: Hirofumi Ichihara <ichihara.hirofumi@lab.ntt.co.jp>
Co-authored-by: Zhi Yan Liu <zhiyanl@cn.ibm.com>
Change-Id: I5552220ce264c0d09780bbd0eb7610e7cff0b820
Fix migration on older postgres

Since the move to oslo.db, sqlalchemy no longer raises
ProgrammingError.  Instead, it is wrapped in a
oslo.db.exception.DBError. This causes migration to fail on older
postgres, which relies on catching this error to use the correct
constraint name.

Change-Id: Icf7b20eb4e09f3c91aa3df3bcea497f810a6f907
Bump osprofiler requirement to 0.3.0

ofprofiler.notifier.create, which is used by recently merged
commit 94b670c199fcc39784edd90b323b0396914ce66a, does not seem
available in osprofiler 0.1.1.

Closes-Bug: #1354500
Change-Id: I563558efcf4e909e2a97003bf84a0785091fd360
Adding status field to image location -- scrubber queue switching

Adding a status field to image's each location property, each location
status can be 'active', 'pending_delete' and 'deleted'.

Under location's status information Scrubber service can make cleanup
based on DB records also but not a dedicated queue-file for each image.

This is fourth part of this change which covered using scrubber DB queue
in store. This change allow cloud admin/operator deploy a central
Scrubber service(s) on one or two hosts only, before this Glance require
deploy it on each API service host since Scrubber only support load
pending delete jobs from local file.

Partially-Implements BP: image-location-status

Change-Id: I6910b55de8c3b203560d75ff3d1675eda31ae786
Signed-off-by: Zhi Yan Liu <zhiyanl@cn.ibm.com>
Use @mock.patch.object instead of mock.MagicMock

Some unittests replace methods by mock objects, so they are leave mocked
when unittest finished. It can cause a lot of issues, because when we
try to get access to these attributes in the next test, we get mock
object instead of real attribute. Use @mock.patch.object decorator to
get attributes `unmocked` after test.

Change-Id: I23b3926a7671e0669315836da787ec88a812d67b
Merge "Fix image killed after deletion"
Fix bad indentation in glance

Fix for bad indentation warnings from pylint.

Closes-Bug: #1356406

Change-Id: I425de4975a4468495a000e408e8d7cf94b646912
Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
Merge "Adding status field to image location -- scrubber queue switching"
Merge "Use @mock.patch.object instead of mock.MagicMock"
Merge "Fix bad indentation in glance"
Ensure constant order when setting all image tags

The newer version of tox (>=1.7.0) sets a random hash seed, which makes the
order of sets unpredictable. This patchset fixes the test_image_tag_set_all
test, which otherwise broke with a randomized PYTHONHASHSEED. This new tox
behaviour is consistent with python (>=3.3), meaning this change should avoid
bugs on that platform at a later date.

Change-Id: Ie281b04fc9c4c2a96a9eec9bde7b05229ebf57fc
Partial-bug: #1348818
Do not assume order of images

This fixes the test_tag_lifecycle test, which broke with a randomized
PYTHONHASHSEED.

Previously, the test assumed that the values in the list of tag names
would remain constant. The test now sorts the results and compares to a
sorted set.

Change-Id: If5a2ef683fcf55520c7d539f1b90aefaba4739d5
Partial-bug: #1348818
Merge "Fix migration on older postgres"
Enforce image_size_cap on v2 upload

image_size_cap should be checked and enforced on upload

Enforcement is in two places:
- on image metadata save
- during image save to backend store

Closes-Bug: 1315321
Change-Id: I45bfb360703617bc394e9e27fe17adf43b09c0e1
Co-Author: Manuel Desbonnet <manuel.desbonnet@hp.com>
Changes HTTP response code for unsupported methods

Requests for resources with an unsupported HTTP method now return a HTTP
response 405 (Method Not Allowed) or 501 (Not Implemented) rather than a
404 (Not Found) for everything.

For example, attempting to DELETE on /v2/images will now return a 405
instead of a 404 and will provide a response header 'Allow' that lists
the valid methods for the resource.

Attempting to use NON_EXISTENT_METHOD on /v2/images will now return a
501.

Attempting to GET on /v2/non_existent_resource will, as expected, return
a 404.

Fixed for v1 and v2.

Change-Id: I5406f8ee423d3d5e66c56a26a7009b4f438a7e0f
Closes-Bug: #1297362
Merge "Do not assume order of images"
Merge "Ensure constant order when setting all image tags"
Merge "Enforce image_size_cap on v2 upload"