Move tests/unit/storage/base under m/tests/storage

If we want third-party backends to inherit from these tests, we need to
ship them along with Marconi. This patch moves storage base tests under
marconi/tests/sotrage.

Implements blueprint refactor-system-tests

Change-Id: Ic7390ead37694bdac9eb341da6f239b8c1178e1e
Move queues' unittests under tests/unit/queues/

This patch moves queues' unit tests under a queues package to reflect
Marconi's source structure.

Change-Id: Ibe009aba89d615333b8626c73008c55ee3face8f
proxy: adds oslo.cache, config support, & stevedore

This patch brings together oslo.cache, oslo.config, and stevedore to
provide pluggable, hierarchical catalogue caching for marconi proxy.

Here's the list of changes:
- add configuration/driver loading at the app level
- remove select from proxy storage driver - unnecessary intelligence
  at storage layer
- node.weighted_select -> partition.weighted_select (clearer name)
- forwarding logic further refactored, placed in own module
- caching logic placed in lookup module
- selector passed down at app level to handle round-robin state
  globally
    * open to becoming configurable
- adds several TODOs for a better proxy

Change-Id: I3bc568315e685486d63cdce3ec278c89e3f2b2bc
Implements: blueprint placement-service
test(proxy): fix proxy-transport tests

Change-Id: Ibc05d6949221367a21b493f75ff0b4f0350ee265
Merge "Move tests/unit/storage/base under m/tests/storage"
Merge "Move queues' unittests under tests/unit/queues/"
fix: age presented as negative

Adds unit tests to verify that the bug never appears again. The unit
test is effective for storage backends that use timeutils to measure
the age of a message.

Bug manifests itself in both claims and messages.

Change-Id: I7fec4533236df5b232c0500835ac8e1d9d261d09
Closes-Bug: 1229358
fix(mongodb): Marker generation creates a bottleneck under heavy load

This patch changes markers so that they are generated using a per-queue
side counter. A heuristic is used to mitigate a race condition. Due to the
new semantics, partial inserts are no longer possible due to collisions,
which ended up simplifying the retry logic for posting messages.

As a consequence of this patch, the last message posted no longer needs
to remain in the queue indefinitely, rendering marconi-gc unnecessary,
and so it has been removed.

Also, since the mongod GC worker runs once a minute, the queries no longer
filter out expired-but-not-yet-gc'd messages; on average, a message may
live more than 30 seconds passed it's expected lifetime, but I do not
think that this will harm or complicate any application building on top of
Marconi, practically speaking. That being said, it is worth calling out
in documentation.

Closes-Bug: #1218602
Change-Id: I34e24e7dd7c4e017c84eb5929ce37ad4c9e5266a
perf(mongodb): Combine project and queue message fields

This patch merges the message and queue collections' project name and
queue name fields into a single 'p_q' field, named to denote the
compound nature of the scoped queue name string.

This should reduce index complexity, making for faster inserts and
finds. This hypothesis obviously needs to be verified during the
review process for this patch.

Change-Id: I3da09806ad50c2d79f1f0c03bee68f1cc4ee787d
Implements: blueprint api-perf-tuning
WSGI: added the control for non empty X-PROJECT-ID

Change-Id: I6fbf0344fd45f9581a28a55ef852139b32e93ed7