tests.py now completely compiles, and reveals a few features of the metis executor we have to add

Reviewers: usmanm, lydia

Reviewed By: usmanm

Differential Revision: http://phabricator.locu.com/D4146