Added directory creation to setup.sh for /var/log/inboxapp and /var/lib/inboxapp

Summary:

Test Plan: Test with fresh install, confirm that ./inbox works on the first try now

Reviewers: mg

CC:
Implement a uid/thread-polling helper for IMAP/Gmail initial sync.

Summary:
In the case that a user has e.g. a very large inbox, our current design has the
shortcoming that, during the initial sync, it won't continue to download new
messages. This is untenable for the user experience, so we're solving it by
having a parallel greenlet during this time that keeps checking for new
UIDs/threads and adding them to the top of the initial sync uid/thread download
stack.

By doing this, we have to get rid of the "chunked download" feature...  if we
batch download messages in groups, we'll introduce a delay with how fast we
download new UIDs. It's not clear to me that batching actually wins us that
much speed anyway.

I think we're going to need to introduce IDLE support at some point for
the quickest receipt of new mail, but this is better than what we've got
for now.

Test Plan: ./runtests + a bunch of manual testing that has me thinking real hard about how to do better automatic testing.

Reviewers: mg

Reviewed By: mg

CC: kav-ya, emfree

Differential Revision: https://review.inboxapp.com/D51
Add some new MX records for Google mail servers
Lint tables.base and don't drop to pdb on txn errors.
Use a gevent Group instead of a list to keep track of folder monitors.

This is cleaner and more concise and probably more efficient since we're
not running an endless loop that just yields every iteration. More on
groups at http://sdiehl.github.io/gevent-tutorial/#groups-and-pools.
Fix tests in test_api.py.
Make account.sync_lock() gevent-safe.

Summary: Thanks @emfree for pointing out this bug.

Test Plan:
./runtests, manually fired up a sync and stopped it, seems OK. I
can't reproduce the error you were getting stopping a sync on master though,
does this fix-in-theory work for you?

Reviewers: emfree

Reviewed By: emfree

CC: emfree

Differential Revision: https://review.inboxapp.com/D109
Upgrade our TLS support by using pyOpenSSL-backed requests.

requests will automatically use pyOpenSSL if it, ndg-httpsclient, and
pyasn1 are installed. pyOpenSSL is currently the best TLS implementation
for Python, and gives us features above the standard librarly ssl module
like TLS SNI support.

You'll need to re-run `pip install -r requirements.txt` and note that
you'll need the libffi headers to compile pyOpenSSL for platforms like
Linux that don't have binary dists available from PyPI (such as Linux).
(Install package `libffi-dev` on Debian/Ubuntu.)

You can verify that yours is working properly by running:

    py.test -s tests/general/network/test_sni.py

Thanks to Hynek Schlawack's PyCon 2014 talk on SSL/TLS for this. :)
Actually return contact search results highest-scoring first.

Summary:
Previously we were (1) returning results lowest-scoring first, and (2)
limiting the result count before ranking, instead of after.

Test Plan:
Still needs a regression test, but for now manually verified via
`zerorpc tcp://0.0.0.0:9999 search_contacts <account_id> <search_string>.`

Reviewers: spang, mg

Reviewed By: mg

Differential Revision: https://review.inboxapp.com/D111
Remove jsonify decorator from API.

Summary:
It's not really needed with ZeroRPC, and we weren't applying it consistently
anyways.

Test Plan: ./runtests

Reviewers: spang, mg

Reviewed By: mg

CC: mg

Differential Revision: https://review.inboxapp.com/D112
Fix and lint tests in tests/imap.

Summary:
 * Add missing imports.
 * Don't expect API method return values to be jsonified.
 * Update auth token in test DB dump.
 * Update test_actions_syncback.THREAD_ID to a value that works with the actual
   state of the test account (previously, it corresponded to a thread that
   seems to have been deleted).

Test Plan: ./runtests --all

Reviewers: spang

Reviewed By: spang

Differential Revision: https://review.inboxapp.com/D113
Use new vagrant directory name for test keydir.
use `.format()` when will I learn...
pyOpenSSL requires libffi to build.

Thanks Jacob Emmert-Aronsen for pointing this out.
[SCHEMA] several changes

Summary:
1. Log error messages when crispin throws exceptions via @retry_crispin decorator. (Previously was geventconnpool's @retry)
2. Store message.from_addr and message.sender as list of addresses. Apparently sometimes there can be more than one sender. I believe the spec allows this, even though it's uncommon. The change for this includes a migration and updated test database dump.
3. Trim block.filename to 64 characters. I'm not sure what the real limit is for this in the spec. It was failing for me with malformed messages that had some crazy huge b64 encoded data in the filename. (See the Dropbox folder.)
4. Print all objects and fields when SQLalchemy throws a DataError during commits
5.  Add some misc logging

Test Plan: Run the migration on your dev machine and get some weird-ass messages in your mail account.

Reviewers: spang, emfree, kav-ya

Reviewed By: spang

Differential Revision: https://review.inboxapp.com/D115
Drastically simplify Crispin code.

DummyCrispinClient was a hairbrained idea to save all the raw bits we
pulled off the wire to disk and be able to replay operations in a sort
of offline mode, for testing and things. In retrospect, this is not the
most straightforward way to get the benefits that we want, namely the
ability to test message parsing separately from message download, and
the caching schema was complicated enough that we never ended up
finishing it or using it for anything.

Let's rip it out and move on. As it is, it's just confusing.
Lint inbox.sqlalchemy.
Hard error-out if anyone tries to use YahooCrispinClient.

This was part of the review process, but I messed up landing the patch
(D119) and accidentally pushed an earlier version. Adding back in the
change here.
Remove extraneous add_all/commit calls.

They were moved into a try/except block by D115.
Resurrect CrispinClient.CHUNK_SIZE attribute.

Deleted in D119, but seems to be needed in update_metadata() and
update_saved_g_metadata().