[Testing] Basic end-to-end tests

Summary:
Christine, Charles, emfree,

I've started working on basic end-to-end tests. So far, the code only tests email sending and thread deletion.
The diff is quite short but here's some things I'd like you to take a look at:

1. I wrote object-oriented tests, which doesn't seem to be the blessed way in pytest. My reasoning was it would be simpler to define a main test class with all the test code and extending it using mixins for every provider. I'm open to suggestions to make the code simpler.

2. What's the scope of this testing? I decided on testing only whether calls on the API propagate to the IMAP backend. I guess we are planning on testing things the other way too? What's the priority?

3. How clean are we about testing? Is it okay to query the db to get an user's password? Right now user credentials are hard-coded into mixins which is kinda bad.

Test Plan: Run the tests.

Reviewers: spang, charles

Subscribers: emfree

Projects: #inbox-sync

Differential Revision: https://review.inboxapp.com/D316
[default_folder_map] Add support for using default folder mappings and looking at flags on folders to determine what system folders they are.

Summary:
Previously we had a folder map for each provider, but many of
them shere common names. Additionally, some providers use flags on
certain folders do describe what they are. This is particularly useful
in the situation where the actual folder names might be in a different
language. Use these flags and a default folder map to clean up the
providers.py file so that it only needs to special-case certain folders
that don't fit into the default mapping.

Test Plan: Review the code.

Reviewers: emfree

Reviewed By: emfree

Differential Revision: https://review.inboxapp.com/D348
Update docs to clarify that message recipients will never be prepopulated.

Test Plan: Look at the docs as served by redwood.

Reviewers: bengotow

Differential Revision: https://review.inboxapp.com/D338
Condstore support for MobileMe

Summary:
Briefly, there was a couple bugs with the mailsync code:

1. Some backends need to receive a condstore-enabling message to activate condstore. This is done using a "STATUS CONDSTORE" command.

2. MobileMe sends HIGHESTMODSEQ values bigger than Integer. I changed the db field to be a BigInteger instead. I made the same change to UIDVALIDITY because it's probably the same.

Test Plan: Ran a sync with MobileMe, added a gmail account, checked that condstore worked for both.

Reviewers: emfree, charles

Reviewed By: charles

Projects: #inbox-sync

Maniphest Tasks: T311

Differential Revision: https://review.inboxapp.com/D352
[event_participants] Initial support for adding particiapnts to events.

Summary:
Adds the ability to sync participant information and provide it via the REST API.
Fixes T299

Test Plan: run regression, sync google account with events.

Reviewers: emfree

Subscribers: kav-ya

Maniphest Tasks: T299

Differential Revision: https://review.inboxapp.com/D342
[events] Initial support for iCloud and Outlook.com/Hotmail

Summary:
[events] Initial support for iCloud and Outlook.com/Hotmail
Also makes the Events / Contacts interface more generic.

Fixes T306

Test Plan: Add events on calendars through web interface, see them show up from the API

Reviewers: emfree

Reviewed By: emfree

Subscribers: mg

Maniphest Tasks: T306

Differential Revision: https://review.inboxapp.com/D345
[events] fix merging issues.
[events] Quick fixes for error handling.
[testing] Add support for authing outlook and gmail accounts.

Summary:
The testing system needs to start from an unverified account and go
through the auth process. This code adds support for consenting to the oauth
providers such as Outlook.com/Hotmail and Gmail.

Test Plan: review the code, test sending and the like on other providers.

Reviewers: emfree, khamidou

Reviewed By: khamidou

Differential Revision: https://review.inboxapp.com/D353
Reset account.state='running' on a subsequent successful retry.

Summary:
I think this entire flow could use some refactoring,
this is intended to be a temp fix.

Test Plan: None

Reviewers: emfree

Reviewed By: emfree

Differential Revision: https://review.inboxapp.com/D354
Fix check for accounts.py file in system tests.
Make system auth tests work with EAS.

Summary:
They weren't working with EAS, because only inbox.auth.handler_from_provider
turns an 'unknown' provider into 'eas'.

Test Plan:
Run py.test tests/system/test_auth, as well as bin/inbox-auth, on
test accounts for Gmail, Outlook, Yahoo and EAS. Also verify that none of the
affected code is called from redwood.

Reviewers: charles

Reviewed By: charles

Differential Revision: https://review.inboxapp.com/D355
[test/base_dump] Update test database structure that was for some reason inconsistent.
[test/base_dump] Update test database structure that was for some reason inconsistent.
Remove outdated comment about ZeroRPC sync interface.
Don't use an SMTPConnectionPool.

Summary:
We don't really need to maintain an SMTP connection pool, because in general we
only rarely need an SMTP connection for an account. Currently, once we open a
connection we try to keep it around forever, which is not ideal.

Instead, we just open and close a connection on an as-needed basis. This
resolves issues we were seeing around messages not being sent because of
improperly handled SMTPServerDisconnected errors.

Test Plan:
Use system tests to send messages for various providers; use added
raise_once utility in inbox/util/debug.py to check that sending succeeds when
retrying after an SMTPServerDisconnected error.

Reviewers: kav-ya

Reviewed By: kav-ya

Differential Revision: https://review.inboxapp.com/D357
Silence the Google apiclient logger.

We really don't need it to tell us what URL is being requested every single
time we sync events. Because logstash can't parse these, this was making it
hard to look at logstash error logs and figure out whether logstash is dropping
messages.
Validate the namespace_id part of API URL paths.

This prevents a StatementError from being raised if you try to do something
like /n/bad%20id/threads, where the namespace_id part can't be decoded to an
integer.

Fixes T321.
Optionally attach namespace to a db session to avoid unnecessary queries.

A surprising fraction of mailsync's performance cost comes from lazy-loading
a namespace instance in various places. In principle, we don't need to do this,
because the namespace object is effectively immutable, and we already know what
it has to be.

This commit adds the option to 'attach' a namespace to a database session. This
is really not an option for general use, since you must take great care not to
make wrong assumptions about which namespace your objects actually belong to.
But it lets us make significant performance gains.

Needs a corresponding EAS change to pass the right arguments around
(forthcoming).
[SCHEMA] Simplify transaction log construction.

Instead of storing an ORM-derived 'delta', only store a snapshot of the API
representation of the versioned object. This reduces the volume of data we
store, makes constructing the transaction faster, and removes a bunch of
complexity left over from when we were trying to infer syncback actions from
the transaction log.