pass data to plugins via environment variables

Summary: Pass parameters to plugins via environment variables.

Reviewers: jacko

Reviewed By: jacko

Differential Revision: https://phabricator.buildinspace.com/D48
keep git submodules from getting stale

Summary:
Previously we were never running `git fetch` in repos that had been
downloaded as submodules. So eventually when the main repo moved forward, we
would get "not a tree" errors in the submodule. Unify the `checkout_tree` logic
in the git plugin, so that it always runs a fetch if it needs to.

Fixes https://github.com/buildinspace/peru/issues/21.

Test Plan: Add a test that fails before and passes now.

Reviewers: sean

Differential Revision: https://phabricator.buildinspace.com/D43
implement Cache.read_file()

Summary:
As we move towards allowing remote repos to define their own
`peru.yaml`, we'll need to be able to read file contents out of trees
that we have in cache, without unpacking the whole tree. This diff
creates that method.

I notice that the `_git` method in Cache is getting a little crufty. But
my ideal fix for that is replacing the whole thing with libgit2, so I'm
not worrying about it too much for now.

Test Plan: New unit test.

Reviewers: sean

Differential Revision: https://phabricator.buildinspace.com/D46
delete an unused flag

Summary:
I forgot to remove `--delete` when we changed the syntax for
`override`.

Reviewers: sean

Differential Revision: https://phabricator.buildinspace.com/D47
update comments in compat.py

Summary:
If we ever remove support for 3.2, we'll probably want to delete
`compat.indent`.

Reviewers: sean

Differential Revision: https://phabricator.buildinspace.com/D50
add excludes to Cache.import_tree()

Summary:
When we import files from local overrides, it's very important
that we don't include the `.peru` dir, because a) that could conflict
with the toplevel one and b) the user certainly doesn't want that
anyway. We've been getting that wrong until now, because we've never
tested an overridden module with imports (which would create the
`lastimports` file, otherwise `.peru` is empty).

To fix this, we need a mechanism to exclude specific paths from the
import. (We have `files` to whitelist paths, but that doesn't work for
us here.) This diff adds that new parameter. We'll fix local modules in
a subsequent diff.

We could consider exposing `excludes` to users as a rule field. I could
see legitimate use cases for that. @sean what do you think?

Test Plan:
Turns out `files` was untested at the Cache level. Add a ton
of unit tests for `files` and `excludes`.

Reviewers: sean

Subscribers: sean

Differential Revision: https://phabricator.buildinspace.com/D55
don't import .peru from local overrides

Summary:
We need to create `.peru` in local overrides, because it keeps
track of lastimports. But we don't want to import that dir into cache.
Take advantage of the new `excludes` param on `Cache.import_tree`.

Test Plan: New integration test that fails before this diff.

Reviewers: sean

Differential Revision: https://phabricator.buildinspace.com/D56
prepare remote module caching for parallel fetching

Summary:
When peru starts to do parallel fetching, it's going to be
important that a remote module can fetch itself and its dependencies at
the same time. Previously we had functions in `resolver` that fetched and
merged imports, and the remote module would fetch itself before calling
those. This diff refactors all of that.

- Create `merge.py` to provide `merge_import_trees()`. See the large
  comment in that function for more about why its structured the way it
  is. `resolver.py` is no longer aware of anything about imports; its
  only job is now to resolve targets into trees (and occasionally
  module/rule objects).
- Refactor `LocalModule` to call `merge_import_trees`. Clean up some of
  the lastimports code while we're at it.
- Make `RemoteModule` responsible for fetching its own dependencies, so
  that we can parallelize this soon. `RemoteModule.get_tree` no longer
  caches the final unified tree; it only caches the result of the fetch.
  The rest is left to `merge_import_trees`. This has the nice side
  effect that changing a module's imports won't cause it to refetch.

Test Plan: Relying on existing integration test.

Reviewers: sean

Differential Revision: https://phabricator.buildinspace.com/D57
[RFC] parallel plugin fetching

Summary:
Use Python 3.4's `asyncio` to do plugin fetching in parallel. Right
now we don't try to parallelize any build or cache operations; those are still
done in series to keep things simple. We also don't do anything fancy with the
console; fetches will still print as they go, though reups have been modified a
little to make sure their outputs don't interleave.

Before we push something like this, we'll add to `third-party` to make sure
that at least 3.3 is supported. The same `yield from` coroutines won't work in
3.2, so we may end up dropping support for that.

Test Plan: All tests pass.

Reviewers: sean

Differential Revision: https://phabricator.buildinspace.com/D58