Create archive folder when needed

Summary: It seems someone tried to archive emails for a non-gmail provider on Beta. This resulted into a large number of errors (https://app.getsentry.com/inbox/beta/group/29293091/). This diff fixes this bug (T328).

Test Plan: Added an archived tag to a thread in yahoo. Checked that the folder was created.

Reviewers: spang, emfree

Reviewed By: emfree

Maniphest Tasks: T328

Differential Revision: https://review.inboxapp.com/D367
Update outdated comment.
Better (but not faster) system tests

Summary:
Charles,

I made a couple changes to the system tests:
- for_all_available_providers now sets up and teardowns test accounts (fixes T311).
- I extended test_sending to archive the email and then trash it.

I found a race condition in mailsync - sending would crash if tried to early in the sync because the folder mapping isn't stored in the db. I added a 30 seconds sleep to handle this. I don't know if it's worth it to go after this bug.

Test Plan: Ran the tests for a couple accounts.

Reviewers: charles

Reviewed By: charles

Maniphest Tasks: T311, T315

Differential Revision: https://review.inboxapp.com/D360
[event/calendar] Add event owners and calendar support.

Summary:
Integrate a bunch of API changes as well as adding calendar support.

(sorry for the long diff)

Test Plan: review the code, sync calendar events across providers, run regresion.

Reviewers: emfree

Reviewed By: emfree

Differential Revision: https://review.inboxapp.com/D365
[events/participants] Provide initial support for participant RSVPs.

Summary:
We will need a way for participants to be able to indicate that they
are attending a particular event. This diff provides the API changes and tests
to provide such support. Issues such as access to this link need to be provided
through redwood white-listing the appropriate event endpoint.

Test Plan: Run tests which exercise success and failure cases of API.

Reviewers: emfree

Reviewed By: emfree

Differential Revision: https://review.inboxapp.com/D366
Validate public_id values in URL paths.

We should consider changing the HasPublicID mixin design so that comparisons to
an invalid public id return no results, instead of raising StatementError. But
in the interim, just validate these URL fragments at the API level.
Vagrantfile security improvements

Summary:
- Add SHA256 'box_download_checksum' to ensure integrity of boxes after
  downloading them over plaintext HTTP.

- Warn if 'vagrant-rekey-ssh' plugin is missing.

- Try to bind forwarded ports to localhost-only by default.  Note: Some
  (all?) versions of the vmware_fusion provider ignore this setting.

Test Plan: Ran 'vagrant up' on my machine, and looked at the results of 'netstat -tnlp'.

Reviewers: mg, emfree

Reviewed By: emfree

Differential Revision: https://review.inboxapp.com/D373
[SCHEMA] Mutable drafts.

Summary:
Updating a draft does not create a new draft; merely updates the existing attrs +
bumps the version.

Key resulting changes -
* Draft update, delete, send API calls now require a 'version' in the request body.
This is used to determine if the operation can proceed.

* The X-INBOX-HEADER set at message creation and used to identify messages on the remote
(for reconciliation during mailsync, draft deletes) is now set to the version rather than
the public_id.

* remote_delete_draft fns operate on the version rather than the public_id.
We introduce an `extra_args` column in the ActionLog table - we don't really care about
the Message record for `delete_draft`; the version == inbox_uid of the Message may have been
updated since the time of the ActionLog record creation.
Therefore, store the inbox_uid too.

Test Plan:
api/test_drafts.py, general/system/test_drafts.py updated + pass.

Reviewers:
emfree

Differential Revision:
https://review.inboxapp.com/D364
Support the 'attachment' tag on threads.

Summary:
If at least one message on a thread has attachments, add the 'attachment'
tag to that thread.

Note that this code doesn't properly handle the case in which you create a
draft with attachments, then update it to a draft that has no attachments.
Since we're in the process of redoing draft versioning, I propose that we make
a bug to fix this after the new drafts stuff lands.

Fixes T323.

Test Plan:
Run migration script, and run a fresh sync. Each time, check
responses to /n/ns_id/threads?tag=attachment. Also update drafts unit
tests.

Reviewers: kav-ya

Reviewed By: kav-ya

Maniphest Tasks: T323

Differential Revision: https://review.inboxapp.com/D363
Correctly set references header for reply drafts.

Summary: Fixes 338.

Test Plan: None

Reviewers: emfree

Reviewed By: emfree

Differential Revision: https://review.inboxapp.com/D375
Apply distinct() to filtered API results.

Summary:
If we don't do this, the problem is that when we join threads to messages, the
limit parameter is applied to the resulting thread-message pairs, not to
threads alone. This leads to wrong behavior when multiple messages on a thread
match, say, the recipient address you're filtering on.

Fixes T336 and T337.

Test Plan: Regression test added.

Reviewers: kav-ya

Reviewed By: kav-ya

Maniphest Tasks: T336

Differential Revision: https://review.inboxapp.com/D376
Keep calendar table when running bin/clear-db

Otherwise you get 'Cannot delete or update a parent row: a foreign key
constraint fails', because the accoutn table maintains a foreign key reference
to a default calendar.
Fix Message.references values in test DB dump.

We save them as lists when syncing, but they were saved as tab-joined strings
in the test database dump. Which is probably why unit tests didn't catch T338.
Properly convert HTML character refs in snippets.

Summary:
Instead of dropping character references like &#225; or &amp;, convert them to
unicode. Fixes Github issue #82.

Test Plan: Regression test added.

Reviewers: khamidou

Reviewed By: khamidou

Differential Revision: https://review.inboxapp.com/D370
More efficiently update contacts from messages.

Summary:
When we update contacts after syncing a message, it's faster* to do one database
query to get existing contacts, instead of doing a bunch of queries serially,
one for each participant in the message.

* faster = ~50-100ms less time for messages with many recipients.

Test Plan: Unit tests added.

Reviewers: khamidou

Reviewed By: khamidou

Differential Revision: https://review.inboxapp.com/D379
Draft reconciliation should use version, not public_id.
tests/general/test_reconcile_message updated too.
Vagrantfile: Fix `config.vm` -> `override.vm` :-(

Summary:
Oops.  This fixes the following error:

    The checksum of the dowloaded box did not match the expected
    value. Please verify that you have the proper URL setup and that
    you're downloading the proper file.

    Expected: b79e900774b6a27500243d28bd9b1770e428faa3d8a3e45997f2a939b2b63570
    Received: 9a8bdea70e1d35c1d7733f587c34af07491872f2832f0bc5f875b536520ec17e

Test Plan:
1. Wipe out my ~/.vagrant.d and my .vagrant directory
2. Run `vagrant up` on Linux (using the default virtualbox provider)

Reviewers: emfree

Reviewed By: emfree

Differential Revision: https://review.inboxapp.com/D381
Also install EAS-specific dependencies in setup.sh, when applicable.
Fix system auth tests.
Wrap greenlet-spawning code inside a try/finally block

Summary:
Eben,

This is a patch extracted from the account deletion diff. It simply wraps the initial sync code inside a try/finally block so that the greenlets we spawn during the initial sync get killed when the main greenlet is killed.

Test Plan: Ran a sync.

Reviewers: emfree

Reviewed By: emfree

Maniphest Tasks: T330

Differential Revision: https://review.inboxapp.com/D380