Add support for rate-limiting the sending API.

Summary:
Limit each account to a quota of 300 sent messages (by default) in a rolling 24
hour window. The quota value can be changed via the DEFAULT_SENDING_LIMIT
config value.

Fixes T449.

Test Plan: Unit test added.

Reviewers: spang, kav-ya

Reviewed By: kav-ya

Maniphest Tasks: T449

Differential Revision: https://review.inboxapp.com/D796
Allow database queue pool max overflow to be set in config.

For the sync workload we really want a small pool size and a large max
overflow, but in any case it makes sense for both parameters to be tunable.
Search tests -
Refresh ES index after recreation so tests pass with repeated creation/deletion.
Test bulk indexing.
Handle event create API requests that lack Content-Type header.
Remove deleted UIDs after adding new ones in IMAP sync.

Summary:
Say we sync a draft update back to the IMAP backend, so delete the UID for the
old draft and add a new one. In normal mailsync, if we first sync the delete
and then the add, the odds are high that the actual draft object gets deleted
as being orphaned, and that we then create a new different draft object once we
sync the add. This would cause a baffling 404 error if you went to send the
draft or update it again. (T621)

As aesthetically unappealing as this solution is, it drastically lessens the
likelihood of that scenario in an expedient way.

Test Plan:
Repeatedly update draft and check that sync runs correctly without
deleting it.

Reviewers: spang

Reviewed By: spang

Differential Revision: https://review.inboxapp.com/D794
[SCHEMA] Search index operations through the transaction log.

Summary:
We currently re-index all the namespaces (i.e. all messages, threads) in the database periodically.
This suffers from a few key shortcomings:
1. Changes due to sync, syncback are not reflected in search until the next re-index is complete (run as an hourly cron job right now).
2. Deletes are not supported.
3. Is obviously slow/inefficient.

So, leverage our transaction logic instead.
The SearchIndexService - run as a separate process - periodically polls the Transaction table (set to 30s),
translates the database changes for the messages, threads of all namespaces to the corresponding
Elasticsearch index operations and performs them.

* Support for index insert, update, delete operations.

* Support for process restarts - the id of the last processed (i.e. indexed) transaction is
persisted to/retrieved from the database.

Note:
The current implementation of the SearchIndexService is namespace agnostic.
An alternate implementation would be to perform the polling/indexing per-namespace and store the
last_processed_txn per-namespace too.
I don't have a compelling argument for the latter, which is more complicated too, and hence went with the former.

The `searchindexcursor` table will only ever have one row (Alembic uses the `alembic_version` table in a similar way too).

Migration included.
Test underway.

Test Plan:
Tested locally, manually.
Will be tested on staging.

Reviewers: emfree

Reviewed By: emfree

Differential Revision: https://review.inboxapp.com/D801
Script to check for extra eas folders.
Fix argument.
Add content_id to API so clients can support embedded images

Summary:
The HTML body of a message can contain image tags which reference a `cid` aka Content-ID. This is a string that represents which MIME attachment to display inline. Since we're not modifying the HTML, we need to expose the Content-ID value so clients can (optionally) load and render the image inline via our API.

More info on Content-ID: https://www.ietf.org/rfc/rfc2392.txt

Test Plan: passes tests

Reviewers: mg, spang

Reviewed By: spang

Subscribers: kav-ya, emfree

Differential Revision: https://review.inboxapp.com/D756
[tests] Verify that Google calendars events are actually synced back

Summary:
This is probably not the prettiest test ever: there's a lot of copy paste from test_events (I'm welcoming your suggestions on how to solve this btw -- I tried to use fixtures and stuff but this just wouldn't work with py.test).

The test already paid itself by catching not one but two bugs!

Test Plan: Ran the test

Reviewers: emfree, kav-ya

Reviewed By: kav-ya

Projects: #testing

Differential Revision: https://review.inboxapp.com/D788
Search index bulk deletes should ignore parent relation.
Bump python-dateutil version in requirements.txt
Dockerfile now builds using new dynamic notation, modify config parser to override mysql host and port information from docker links
Correctly update thread tags in IMAP sync.

Summary:
With this commit, we update a thread's tags by aggregating over all ImapUid
folders and labels to figure out what the tags should be. Previously, we
wouldn't remove the inbox tag if the \Inbox *label* was removed from an All
Mail UID -- only if we synced a deleted Inbox UID. This wouldn't work if the
Inbox folder syncs lagged such that we never actually saw an Inbox UID for the
message.

Fixes T632.

Test Plan:
Run syncs and check that the inbox tag is removed from archived
threads. Automated tests to come, but I want to put this on staging sooner
rather than later.

Reviewers: spang, khamidou

Reviewed By: khamidou

Maniphest Tasks: T632

Differential Revision: https://review.inboxapp.com/D810
Fix bin/regen-labels script for large datasets.
s/namespace/namespace_id for API calendar representation.
Do not raise SearchEngineError on deleting an ES index that does not exist.
Properly update draft objects after sending failure; remove smtpconn_retry.

Summary:
* It doesn't make sense to have retry logic in both the syncback service and in
  the smtpconn_retry decorator; doing so can cause sending actions to get stuck
  for an arbitrarily long amount of time.

* Properly set a draft object's 'state' property to 'sending failed' if we
  can't send it.

Test Plan: Unit tests added.

Reviewers: kav-ya

Reviewed By: kav-ya

Differential Revision: https://review.inboxapp.com/D829
Support pagination in search queries -
return total number of matches in the search response.
Improve the heartbeat mechanism.

Summary:
This revision changes the Redis database from 0 to 1 (see
/etc/inboxapp/config.json) - NOTE: we can use any database from 1 to 15;
introduces support for the heartbeat mechanism for EAS; improves both the
location and the parameters of calls to publish() to improve reporting; avoids
unnecessary branches when retrieving a Redis client; improves error handling
with better logging.

In addition, this revision overhauls the bin/monitor-syncstatus script, which
is now bin/inbox-syncstatus. The script now prints JSON on stdout and exits
with either 0, 1, or 2, which makes it play nicely with our sensu-based
monitoring.

Test Plan:
```
vagrant up
vagrant ssh
cd /vagrant
bin/inbox-start

vagrant up
vagrant ssh
cd /vagrant
bin/inbox-auth ADDRESS

vagrant up
vagrant ssh
cd /vagrant
bin/inbox-syncstatus --redis-hostname='localhost' --redis-port=6379 --redis-database=1
```
Happy monitoring :)

Reviewers: emfree, kav-ya

Reviewed By: emfree, kav-ya

Subscribers: khamidou, jennie, spang

Differential Revision: https://review.inboxapp.com/D771