fix: rename _payload_uplimit into _paging_uplimit

Although "message_paging_uplimit" also limit the maximum number of
IDs can be supplied in a URI, which is not quite a "page" (in bulk
deletion), but we don't need the configuration to be too precise.

Change-Id: I0737146f1212c82db18de35e35206d3932a46628
Don't use claim creation time to calc claim's age

Mongodb's driver uses Claim's creation time to estimate claims age
instead, it should use claim's update time. Since there's no update /
creation time in mongodb's claims, the patch estimates claims age by
doing:

    delta(expiration - ttl, now)

This change fixes the issue bellow and makes mongodb's claims compliant
with Marconi's specs.

Change-Id: I72c968236ecd239e7b885b293238e2920807a57c
Closes-Bug: #1210662
Merge "Don't use claim creation time to calc claim's age"
Ignore *.egg dirs

This patch adds *.egg to .gitignore

Change-Id: Ieb2ff6bc04a101fb700419ccfabbc947725de481
fix: fetching nonexistent message/claims raises 503s

The proposed fix moves away from throwing an exception everytime a
malformed ID is encountered and instead returns None. This way,
partial bulk get and bulk deletes are possible.

Storage drivers affected:
- mongo
- sqlite

Changes/expected behavior:
- GET /v1/queues/exists/messages/malformed => 404
- GET /v1/queues/exists/messages?ids=malformed,malformed => 204
- DELETE /v1/queues/exists/messages?ids=malformed,malformed => 204
- GET /v1/queues/exists/claims/malformed => 404
- PATCH /v1/queues/exists/claims/malformed => 404
- DELETE /v1/queues/exists/claims/malformed => 204

In partcular, regarding bulk_get on messages, malformed IDs or
messages that are not found by ID are now ignored. bulk_delete also
works in this fashion now.

Regression tests are added to the unit test suite to reflect these
changes.

Change-Id: I4f14cd8b4cfd1dd190dccd8724c20f5fac99c629
Closes-Bug: #1203842
test: delete msg w/ invalid claim isn't deleted

Adds a test to verify the above scenario. It should be returning 204
in this case without the message being affected.

Change-Id: I74cf8ab5f7d94128ae2157a51a5e37c01ef9ca64
restructure: rm -rf tools, move test requires

To pave the path for the project structure of the future, tools/ is
being eliminated.

New commands/utils should be added as entry points and placed in cmd/

tools/system-test-requires -> system-test-requirements.txt

Change-Id: I14b3189f2b2a1cee6508952a46274e00514addac
Use req.get_param's store feature in lieu of utils.purge

This patch removes the use of utils.purge since it is no
longer needed in light of Falcon's relatively new "store"
feature, which causes the target dict to only receive the
value of the param if it is found in the request, thus
avoiding clobbering default kwarg values, which was the
original intent of utils.purge.

As a bonus, this approach is also ~40% faster.

Change-Id: I6b3d82b1f700045e50b3131902b2c7b6dc6ea13c
Implements: blueprint v1-obvious-optimizations
Don't wait for ack when deleting expired messages

This patch simply sets write concern to 0 in the MongoDB driver,
when removing expired messages. This shortens the amount of time
that garbage collection takes, and slightly reduces the amount
of traffic going between app and db servers, as well as between
master and secondary nodes.

This is a safe thing to do, since the operator will be running
multiple GC daemons, so that in the case of a failure, the
operation will be retried in the near future.

The downside is that persistent failures to collect expired
messages will cause the DB to grow out of control; however,
it is expected that if this operation is failing, more than
likely message CRUD in general is failing, which mitigates
the risk just mentioned.

Change-Id: I90950721361dacffa508624281d2e2ea8cf53724
Merge "Ignore *.egg dirs"
Merge "Use req.get_param's store feature in lieu of utils.purge"
Merge "Don't wait for ack when deleting expired messages"
Merge "restructure: rm -rf tools, move test requires"