Expand controller abbreviations in WSGI resources

Several properites were expanded in the interest of readability. When in
doubt, be a little more verbose; you'll spend a lot more time reading
code than writing it.

Change-Id: I352bc8fab70c4c3707003bf528a37e61742aceb5
Implements: blueprint bp/grizzly-debt
Properly handle message post with an empty iterable.

Storage drivers expect at least one item in messages iterable,
so I modified the WSGI driver to check when this is not
the case and return an error response to the client rather
than blindly passing the messages generator into the
driver's post method.

Fixes: bug #1169838

Change-Id: Ia2e512138a465af70110d88e8e11ae58ee5e4aab
Don't use BaseException.message

This patch fixes the deprecation warning when testing under py26, and
gets us one step closer to Python 3 compatibility.

Change-Id: Ia6ac113a917eca075ef5abee937634bd4e488fa5
Merge "Don't use BaseException.message"
Add System Tests

This patch adds the System level testing framework &
the initial set of tests for Marconi.This will allow
developers to run the system tests (regression suite for
Marconi system level tests), against local Marconi server.

Refer tests/system/README.rst file for details on
how to add new tests and run the existing tests.

Change-Id: I2f47e03091acc260293b19edad69ff5c83888ec3
Implements: blueprint system-tests
Merge "Add System Tests"
Replace gunicorn with wsgiref

The patch replaces gunicorn with wsgiref since it doesn't make sense to
have gunicorn as dependency. Lets let deployers choose whatever the
prefer to use as container.

The patch also removes lib/* since marconi_paste is not needed anymore,
the wsgi app can now be accessed through:

    `marconi.transport.wsgi.app:app`

Backward incompatible change: bind refers now to the host and a new
config variable was introduced to specify the port it should bind to.

Fixes bug: #1187280
Implements blueprint: transport-wsgi

Change-Id: I9f7767ace5c6553e75e2f4587032d7d64b9537c4
Merge "Replace gunicorn with wsgiref"
Wrap transport's app with keystoneclient's middleware

Instead of using paste and depending on the middleware chain, use
ceilomenter's strategy and wrap transport's app using the auth_token
middleware.

NOTE: Tests with successful auth are missing

Implements blueprint remove-paste
Change-Id: I61e7d1fae6b80114e22c0a43b4e391e2d5443123
Make sure transport's conf are registered

Import transport and let it register CLI parameters before loading the
arguments into oslo.config.

Change-Id: I1be537c216df6855b8e5b12951715bd5bb8ec9ea
Pass content length to req.stream.read() in WSGI transport

This update modifies the WSGI transport driver so that it
always passes in a length when reading from the request
stream. This is necessary because WSGI servers are not
required to provide a stream that supports "read all"
semantics. For example, wsgiref will simply block if
read() is called without arguments.

Fixes: bug #1189648
Change-Id: I267e3bce36f0e11d931d9e9800bf3ad3aca7a804
Add Verification for List Queue

This patch adds more verification for List Queue API call.
It also cleans up some files, to keep the style consistent.

blueprint system-tests

Change-Id: Ieda206a98f22e7642784122b5c7b8fd45f2fd39e
Guarantee FIFO/once-and-only-once delivery when using MongoDB

This patch modifies the way message markers are generated and
used, such that Marconi can guarantee FIFO for a single message
producer posting to a single queue (FIFO for multiple producers
is not guaranteed.) At the same time, these changes guarantee
that observer clients will recieve messages once-and-only once
by removing race conditions inherent in timestamp-based markers.

Along the way, some technical debt was also addressed, particulary
regarding style, as well as some minor optimizations.

A new service, marconi-mongo-gc, was also added, and is required
when using the MongoDB storage driver for Marconi. This service
was necessary due to the special requirement that at least the
most recent message always remain in each queue, which is an
a-priori assumption upon which the FIFO/pagination algorithm
is based, and arises from the particular constraints imposed
by MongoDB's semantics.

Note: While implementing this blueprint, many calculated tradeoffs
were made in an attempt to balance performance, risk, readability,
and maintainability. The goal was to create a resonable baseline
implementation that can be iterated upon pending comprehensive
system and performance testing. Due to the many subtleties
of solving the FIFO/once-and-only-once problem for the MongoDB
driver, future contributors should excercise extreme caution when
modifying the algorithm introduced in this patch.

Changes include:
* Align text in comments
* Add counter to queue, messages
* Markers are now converted over to using monotonic counter
* Handle DuplicateKeyError
* Return resources in body as a response to a message POST
* Added mongo driver claims tests
* Return 503 when no messages were enqueued due to marker conflict
* Added backoff sleep between retries
* Added marconi-mongo-gc service. This is a new required servi

Implements: blueprint message-pagination
Change-Id: Ifa0bb9e1bc393545adc4c804d14c6eb2df01848c
Merge "Add Verification for List Queue"
Merge "Guarantee FIFO/once-and-only-once delivery when using MongoDB"
Move log.setup to Bootstrap and use cfg.CONF opts

This patch uses cfg.CONF as ConfigOpts in common/config instead of
creating a new instance of it. This is needed since most of Oslo's
modules use the global CONF object to register their config parameters.

The patch also moves log.setup call into Bootstrap and calls it after
configs are loaded.

Fixes bug: #1190524

Change-Id: Ib601418a7dbcad84c79b640b93bc5798ec4a62c4
Log retry attempts resulting from parallel message posts

This patch just adds a couple log entries, one when the first
attempt to post messages fails due to a race condition, and the
other when the post finally succeeds. Adding transaction IDs
will improve traceability, but that's another bp. :D

Change-Id: I188dae91f425e1effb6a12551e9d9f35c9549f71
Implements: blueprint bp/production-logging
Migrate to pbr

Change-Id: I0d5888f9d58d819ac65561b222cb2e93f260d627
Rename pip-requires / test-requires into (test-)requirements.txt

Change-Id: Ica6e7da1f7ed9f9fd62302d73309630aa536116d
Removing cliff from dependencies

Change-Id: Icf08010838e7249628bb068343c78d1d5bad8a1c