when editing YAML, quote potential number literals

Summary:
This came up implementing reup for svn. Revisions are just numbers, so when we
edit them into YAML files, and then read them back in during a sync, they
become Python ints, and peru throws up. Have the editing code guard against
this and insert quotation marks as necessary. Plugins are still responsible for
only ever returning strings in their reup output.

Test Plan:
New unit test, and a new assert to guarantee plugins always return
strings when they reup.

Reviewers: sean

Reviewed By: sean

Differential Revision: https://phabricator.buildinspace.com/D26
update the Travis badge with the new repo url
make cache import include .gitignored files

Summary:
`git add -A` doesn't add ignored files by default. But *peru* doesn't have any
concept of ignored files. When peru imports a directory, it should import
eveything.

https://github.com/oconnor663/peru/issues/15

Test Plan: New unit test.

Reviewers: sean

Reviewed By: sean

Differential Revision: https://phabricator.buildinspace.com/D12
create shared.write_files()

Summary: This saves us from manually opening lots of files in tests.

Reviewers: sean

Reviewed By: sean

Differential Revision: https://phabricator.buildinspace.com/D15
coveralls support

Summary: Integrate with coveralls.io.

Reviewers: jacko

Reviewed By: jacko

Differential Revision: https://phabricator.buildinspace.com/D28
create the curl plugin

Summary:
This plugin downloads a single file and puts it by itself in a directory. This
is for e.g. fetching prebuilt JARs or jquery.js. Like the 'cp' plugin isn't
actually backed by `cp`, this isn't actually backed by `curl`. It uses Python's
`urllib`, which supports a similar variety of URL schemes (including `file://`,
which we use for testing). My other preferred name would've been "http", but
since the plugin is more generic than that, we need a more generic name.
Suggestions please.

Test Plan: Fetch and reup tests.

Reviewers: sean

Reviewed By: sean

Differential Revision: https://phabricator.buildinspace.com/D24
make cache export refuse to pave .gitgnored files

Summary:
Similar to D12, which fixed a .gitignore bug in import_tree(), we also have a
bug in export_tree(). This time the bug has to do with our dirty file
detection. Normally we fail an export if it would overwrite dirty files, but it
turns out `git checkout` will merrily pave over files that it thinks are
.gitignored.

The solution is to use `git reset --keep`, which apparently doesn't respect
.gitignore. That in turn runs into a tricky issue with timestamps in the index
file. See the big, sorrowful comments in `export_tree`.

https://github.com/oconnor663/peru/issues/15

Test Plan: Extend an existing unit test to include this case.

Reviewers: sean

Differential Revision: https://phabricator.buildinspace.com/D13
create Runtime, major refactor

Summary:
The `Runtime` class now contains most of the state that peru needs to pass
around, including module/rule definitions, config flags, and the cache
instance. In several cases, functions that took all these parameters separately
now just take one `Runtime`. As a result of this change, the `Resolver` no
longer needs to hold any state of its own, and all its methods have been turned
into functions.

The original motiviation for all these changes is that we wanted to make it
easier for the user to keep the entire cache dir (not just the plugins cache
dir) somewhere in his home folder. For that, we needed to pull the `tmp_dir`
functionality out of `Cache`, because peru shouldn't be creating tons of temp
directories in your home folder if your project lives somewhere else. We needed
a new class to represent the state stored in the .peru dir itself (like overrides),
which usually lives in the project even if the cache doesn't. Now `Runtime` is
this class. The `PERU_PLUGINS_CACHE` env var is gone, and only
`PERU_CACHE` remains.

Test Plan: Modified a couple integration tests.

Reviewers: sean

Differential Revision: https://phabricator.buildinspace.com/D14
fix imports in overridden modules

Summary:
We were forgetting to apply imports inside of overrides. Make `do_build` take
care of this automatically, so that it's no longer possible to forget it.

This diff exposed major bugs in our imports code. First, we were failing on
empty imports. Second, overridden modules were using the lastimports file of
the toplevel module, so overriding a module with imports would make your local
imports appear dirty. Fix this second problem by making each local_module own
its own .peru dir. The toplevel .peru dir can still be configired with
`PERU_DIR`.

Test Plan:
Extend the overrides integration tests to exercise this. Add tests for empty
imports and for overridden modules with imports.

Reviewers: sean

Differential Revision: https://phabricator.buildinspace.com/D16
walk parent directories to find peru.yaml

Summary:
We want to be able to run peru from subdirs below the root of a project. Peru
should walk up the directory hierarchy until it finds a peru.yaml file, and
interpret that as the root of the project. The core of this logic is the new
function `find_peru_file`.

Lots of code up until this point has been working under the assumption that
peru's current working directory is also the root of the project, and this
assumption is no longer true. That means several features (relative overrides,
relative module paths) don't work properly away from the root yet; followup
diffs will fix these and add tests for them.

We could do something drastic like have peru cd to the root after it's
detected, but I think this is going to cause more trouble than it saves.
Thoughts greatly appreciated on this.

Test Plan: New unit and integration tests.

Reviewers: sean

Differential Revision: https://phabricator.buildinspace.com/D17
factor the out KeyVal class out of Cache

Summary:
Our storage for overrides is very similar to the keval storage we do in Cache,
but so far we've been using a different mechanism for it. In preparation for
unifying them, separate the KeyVal class out of Cache and beef it up with a few
more functions.

Test Plan: Separate and expand the existing KeyVal unit test.

Reviewers: sean

Differential Revision: https://phabricator.buildinspace.com/D18
delete override.py

Summary:
That file was a lot of code doing very little. Replace it with the newly
exposed KeyVal class. KeyVal allows more granular access to the values stored
on disk, and it avoids keeping duplicated copies of data in memory that could
get out of sync. (Though of course, we could do that in the future for
performance if we need.) Also, it has a nice dictionary-like interface.

Test Plan: Existing override tests cover this.

Reviewers: sean

Differential Revision: https://phabricator.buildinspace.com/D19
make the override command handle relative paths correctly

Summary:
When adding overrides on the command line, users can supply relative paths
based on their current working dir. However, when users later run a build that
reads these override paths, there's no guarantee that their working dir will be
the same. We could always translate to absolute paths for storage, but that
would make it impossible to do convenient things like copy an entire workspace
of projects to a different place on disk (their absolute override paths would
no longer be valid).

To keep the advantages of relative paths, reinterpret them relative to the
project root when they're stored. And when overrides are listed out for the user,
they should again be reinterpreted relative to the users current directory.

Test Plan: A big new integration test for this.

Reviewers: sean

Differential Revision: https://phabricator.buildinspace.com/D20
always set plugins' working dir to the project root

Summary:
Plugins can receive relative paths in their fields (for example, the path to a
git repo). These paths come from the peru.yaml file, and they should always be
interpreted relative to the directory that contains that file. To make sure
that happens, explicitly set the working directory of plugins to the project
root whenever we shell out.

Test Plan: Update one of the integration tests to exercise this.

Reviewers: sean

Differential Revision: https://phabricator.buildinspace.com/D21
work_dir -> root

Summary:
We need to stop using `work_dir` to refer to the root of the projcet. It was
inspired by git's `work_tree` parameter, but now it's ambiguous with the actual
current working directory of the process. Prefer `root`.

Reviewers: sean

Differential Revision: https://phabricator.buildinspace.com/D23
avoid returning None for empty imports

Summary:
This was causing an exception for yaml files that included the
"imports" field but didn't actually put anything in it.

Test Plan: new unit test

Reviewers: sean

Differential Revision: https://phabricator.buildinspace.com/D25
add pathlib to third-party

Summary:
We need a good `glob()` function for the upcoming `files:` field. It turns out
that git's native globbing doesn't work very well; ** can't match the root
directory for example. Pathlib has an excellent glob, but it's only
standardized in 3.4. Add pathlib to third-party, for compatibility with 3.2 and
3.3. Now we can start using it instead of `os.path` throughout the codebase.

Test Plan: `scripts/validate_third_party.sh`

Reviewers: sean

Differential Revision: https://phabricator.buildinspace.com/D29
create the "plugins" field for user-defined plugins

Summary:
Extend the interfaces in plugins.py to take additional plugin paths, and plumb
this all the way up. Make sure plugins are resolved relative to the project
root (in runtime.py) and not just blindly assumed to be valid paths. This field
shouldn't be able to change the behavior of existing modules, so if the same
plugin is ever defined twice, raise an error.

The complexity of this diff raises a couple of design issues that we need to
think about:
- The plugins.py interface is getting very complicated. Should we encapsulate
  it in some kind of config object? Should it take a `Runtime`? (This would make
  it difficult to unit test.)
- Allowing users to run peru from directories other than the project root is
  creating complexity throughout the code. The kind of path massaging that we
  have to do here in runtime.py is very easy to get wrong, and very easy to
  miss in testing. We really don't want to burden every new feature with a test
  to make sure it handles funny dirs. Is there a way we could make this kind of
  testing automatic? Should we use chdir magic to make paths Just Work? (That
  would make resolving paths in peru.yaml easier, at the expense of resolving
  paths given on the command line.)

Test Plan: Lots of new unit tests and an integration test.

Reviewers: sean

Differential Revision: https://phabricator.buildinspace.com/D27
add the 'files' field to rules and modules

Summary:
For a while now we've been using `build` rules to handle file filtering, by
copying files into an out directory and then exporting it. That's pretty hacky,
and it's worse when it needs to be portable. The new `files:` field replaces
these hacks with native globbing from `pathlib`. These globs are resolved
relative to the `export` path.

Also update our peru.yaml file to take advantage of the new field.

Test Plan: New unit test for `Cache` and integration tests for `Rule`.

Reviewers: sean

Differential Revision: https://phabricator.buildinspace.com/D30
allow `peru help`

Summary: I constantly type that when I meant `peru --help`.

Test Plan: Created an EC2 cluster and ran a few hundred thousand test passes.

Reviewers: sean

Differential Revision: https://phabricator.buildinspace.com/D31