Bump python-dateutil version in requirements.txt
Dockerfile now builds using new dynamic notation, modify config parser to override mysql host and port information from docker links
Correctly update thread tags in IMAP sync.

Summary:
With this commit, we update a thread's tags by aggregating over all ImapUid
folders and labels to figure out what the tags should be. Previously, we
wouldn't remove the inbox tag if the \Inbox *label* was removed from an All
Mail UID -- only if we synced a deleted Inbox UID. This wouldn't work if the
Inbox folder syncs lagged such that we never actually saw an Inbox UID for the
message.

Fixes T632.

Test Plan:
Run syncs and check that the inbox tag is removed from archived
threads. Automated tests to come, but I want to put this on staging sooner
rather than later.

Reviewers: spang, khamidou

Reviewed By: khamidou

Maniphest Tasks: T632

Differential Revision: https://review.inboxapp.com/D810
Fix bin/regen-labels script for large datasets.
s/namespace/namespace_id for API calendar representation.
Do not raise SearchEngineError on deleting an ES index that does not exist.
Properly update draft objects after sending failure; remove smtpconn_retry.

Summary:
* It doesn't make sense to have retry logic in both the syncback service and in
  the smtpconn_retry decorator; doing so can cause sending actions to get stuck
  for an arbitrarily long amount of time.

* Properly set a draft object's 'state' property to 'sending failed' if we
  can't send it.

Test Plan: Unit tests added.

Reviewers: kav-ya

Reviewed By: kav-ya

Differential Revision: https://review.inboxapp.com/D829
Support pagination in search queries -
return total number of matches in the search response.
Improve the heartbeat mechanism.

Summary:
This revision changes the Redis database from 0 to 1 (see
/etc/inboxapp/config.json) - NOTE: we can use any database from 1 to 15;
introduces support for the heartbeat mechanism for EAS; improves both the
location and the parameters of calls to publish() to improve reporting; avoids
unnecessary branches when retrieving a Redis client; improves error handling
with better logging.

In addition, this revision overhauls the bin/monitor-syncstatus script, which
is now bin/inbox-syncstatus. The script now prints JSON on stdout and exits
with either 0, 1, or 2, which makes it play nicely with our sensu-based
monitoring.

Test Plan:
```
vagrant up
vagrant ssh
cd /vagrant
bin/inbox-start

vagrant up
vagrant ssh
cd /vagrant
bin/inbox-auth ADDRESS

vagrant up
vagrant ssh
cd /vagrant
bin/inbox-syncstatus --redis-hostname='localhost' --redis-port=6379 --redis-database=1
```
Happy monitoring :)

Reviewers: emfree, kav-ya

Reviewed By: emfree, kav-ya

Subscribers: khamidou, jennie, spang

Differential Revision: https://review.inboxapp.com/D771
Include 'action' in the sync status monitoring output.
Move SyncStatus initialization outside of try/except.
Simplify bin/inbox-syncstatus usage.

Add defaults to parameters and a --pretty switch to ease reading the
output of the script.
Bump python-dateutil version in requirements.txt

I ran the regression test against a virtual machine with version 2.3.
The regression passes without troubles. In addition, one of our staging
machine is already running with version 2.3 (I don't know why this is
the case so we would better update it).
Support adding new IMAP folders without restarting.

Summary:
We didn't support adding new IMAP folders without restarting the IMAP monitor
and thus the IMAP folder sync engines handling an IMAP account since
prepare_sync, which was the function doing the "dirty work", was called only
once in sync.  We now support it by calling sync() on a regular basis within
the IMAP monitor.

In addition, this revision refactor the Gmail monitor. Conversely to a classic
IMAP account, Gmail accounts don't require to pick up new IMAP folders since
Gmail doesn't really support them. Thus, it overrides sync calling prepare_sync
over and over.
Moreover, this revision also removes code duplications between IMAP and Gmail
monitor. This requires adding the should_block property to the IMAP folder sync
engine, which is then overloaded by the Gmail folder sync engine (which has its
own policy). The former always returns True to make sure we only have one IMAP
folder sync engine in the 'initial' state per IMAP monitor.

Fix https://review.inboxapp.com/T586.

Test Plan:
Sync an IMAP account (e.g., Fastmail), add a new IMAP folder from the webapp
make sure Inbox picks it up.

Reviewers: khamidou, emfree

Reviewed By: emfree

Subscribers: spang

Differential Revision: https://review.inboxapp.com/D675
Eager-load attributes necessary to construct Message representation.

Summary:
This makes the /messages API endpoint a fair bit faster (~ 3x as
measured against example requests from production).

Test Plan: Run example requests; unit tests.

Reviewers: kav-ya

Reviewed By: kav-ya

Differential Revision: https://review.inboxapp.com/D840
Fix regression in Gmail-special-folder-handling.

This fixes a regression introduced in commit e9624a728, in which syncing Gmail
threads marked e.g. as starred or important would fail if the corresponding
folder was disabled from appearing in IMAP, or not present for some other
reason when we synced folder names.
s/subqueryload/joinedload to prevent weird bugs.

tests.test_drafts.test_draft_filtering was failing sporadically for some
strange reason apparently related to subquery loading?
Use pipeline to batch execute Redis fetches

Summary:
Instead of sequential hgetall() calls, use a pipeline to
execute calls in bulk for a big speed improvement.

Test Plan:
Tested locally with staging redis DB. Will need to test
against production redis in case the count parameter needs tweaking.

Reviewers: siro

Reviewed By: siro

Differential Revision: https://review.inboxapp.com/D848
Preserve IMAP folder sync order.

We want to make sure that we sync e.g. the Inbox folder before other ones
during initial sync.
Add merge to InboxSession

Summary: Adds merge to InboxSession.

Test Plan: Tested trivially. The motivation for this change is to unbreak a test elsewhere.

Reviewers: emfree

Reviewed By: emfree

Differential Revision: https://review.inboxapp.com/D857