The traversal order used is preorder, not inorder; support postorder traversal

1. The term "inorder traversal" does not make sense outside of the context of binary trees.  Instead, use the term "preorder", which is actually what is done.

2. Also allow for postorder traversal (and test that it works).

Postorder stuff is needed for https://rbcommons.com/s/twitter/r/1179/

Testing Done:
./pants goal test tests/python/pants_test/graph:graph

Reviewed at https://rbcommons.com/s/twitter/r/1177/
Handle list-typed options (action='append') correctly.

Ensure that we read them from config in the right way, and apply type
conversion to the list members.

Added test cases for this.

Testing Done:
Ran the options unittests before and after the fix, to verify
that the test discriminates this.

CI passed.

Reviewed at https://rbcommons.com/s/twitter/r/1183/
convert and move about 1/2 of our docs from .rst to .md

This change starts converting our docs from sphinx-powered-rst to Pants-enhanced-md, moving files to some different directories:

src/docs/*.md  <= language-agnostic Pants-user docs

examples/src/java/com/pants/examples/*.md  <= Pants-with-Java docs

examples/src/python/example/*.md  <= Pants-with-py docs

src/python/pants/docs  <= good place for pants-dev docs

Keeping foolang docs with examples/src/foolang feels right; if we change the examples, we probably want to change the docs, too. Less-certain about where language-agnostic docs go.

Testing Done:
eyeball-compare pages gen'd from .rst, .md

Reviewed at https://rbcommons.com/s/twitter/r/1176/
'goal sitegen' tweaks inspired by eyeballing more Pants doc pages

Don't insert pilcrow anchor in the heading; instead, lay it nearby with a table. Otherwise, it looks funny if the line breaks right there.

Add a "Generated by publish_docs dist/markdown/html/README.html 2014-10-17T11\:16:07.253750" so that folks who are looking at a doc from two years ago have an inkling of the danger. And/or if they want to fix a mistake, they have a clue where the content came from.

Widened margin around the page-level table of contents.

Testing Done:
https://travis-ci.org/pantsbuild/pants/builds/38290410

Reviewed at https://rbcommons.com/s/twitter/r/1178/
Make CacheKey hash and pickle friendly

Don't need payloads on cache_keys, so removing them makes them pickle-friendly.

Testing Done:
tests/python/pants_test/cache::

Passed: https://travis-ci.org/pantsbuild/pants/builds/38538962

Reviewed at https://rbcommons.com/s/twitter/r/986/
Add test for CombinedArtifactCache's behavior

Makes sure that remote + local correctly "backfills"
local when fetching from remote.

This is just documenting/asserting current behavior.

Testing Done:
https://travis-ci.org/pantsbuild/pants/builds/38635762

Reviewed at https://rbcommons.com/s/twitter/r/982/
Add support for running a command before tests.

This depends on https://rbcommons.com/s/twitter/r/1177/

Testing Done:
Ran new test.

Reviewed at https://rbcommons.com/s/twitter/r/1179/
Fix problem parsing stuff like options: --compile-java-args='-target 7 -source 7' in pants.rc files

Testing Done:
Ran new tests

Reviewed at https://rbcommons.com/s/twitter/r/1192/
Migrate all JvmTasks to use the new options system.

The common --jvm-options, --args, --debug and --confs flags now
live in the base class, but this required migrating all the
relevant tasks at once, hence the scope of this change.

This also gets rid of manual flag-to-config fallback, as the new
options system does this automatically. However this required some
changes to section and key names in pants.ini. I'm documenting
these changes in a spreadsheet, for smooth config migration in
pants install bases.

This required breaking a couple of inconsistently named flags, e.g.,
--benchmark-caliper-args is now just --benchmark-args.

Testing Done:
CI passed.

Manually tested that new flags and config settings work.

Reviewed at https://rbcommons.com/s/twitter/r/1186/
A simple script to detect unmigrated config keys.

Future changes that modify config keys should add the migration
to this script.

Reviewed at https://rbcommons.com/s/twitter/r/1195/
Provides documentation for some of the properties and methods in target.py

The type annotations help when using an IDE like IntelliJ

Testing Done:
There should be no code changes.  running CI

Bugs closed: 707

Reviewed at https://rbcommons.com/s/twitter/r/1196/
Print a diagnostic if a BUILD file references a source file that does not exist

Testing Done:
Edited a BUILD file, added a reference to a source file that does not exist.

```
diff --git a/src/python/pants/backend/core/BUILD b/src/python/pants/backend/core/BUILD
index 2da9682..8b1e827 100644
--- a/src/python/pants/backend/core/BUILD
+++ b/src/python/pants/backend/core/BUILD
@@ -4,7 +4,7 @@

 python_library(
   name = 'plugin',
-  sources = ['register.py'],
+  sources = ['register.py', 'does-not-exist.py'],
   dependencies = [
     'src/python/pants/backend/core/targets:all',
     'src/python/pants/backend/core/tasks:all',

```

Before this change, when running

```
PANTS_DEV=1 goal test tests/python/pants_test:all
```

the build would end with an error but not give any further information:

```
Exception message: [Errno 2] No such file or directory
```

After this change, you now get a diagnostic message that gives you a hint why the test failed:

```
 ERROR] Failed to copy src/python/pants/backend/core/does-not-exist.py for library PythonLibrary(BuildFileAddress(/Users/zundel/Src/pants/src/python/pants/backend/core/BUILD, plugin))

               FAILURE

Exception message: [Errno 2] No such file or directory
```

Reviewed at https://rbcommons.com/s/twitter/r/1198/
Fix all uncompressed artifacts stored as just '.tar'

Testing Done:
https://travis-ci.org/pantsbuild/pants/builds/38738752

Reviewed at https://rbcommons.com/s/twitter/r/1197/
jvmdoc_gen: In the event of an exception, call get() on the remaining futures.

jvmdoc_gen: In the event of an exception, we want to call get() on the remaining futures because otherwise we get errors on exit when multiprocessing tries to kill them because what is dead may never die.

Ideally, we would kill them instead of waiting for them, but I don't see how to do that.

Testing Done:
./pants goal test tests/python/pants_test/tasks:jvmdoc_gen

It would be insanely difficult to write a real test for this, because the failure happens in an atexit handler.

Reviewed at https://rbcommons.com/s/twitter/r/1202/
Adds the ability to pass a filter method for ZIP extraction

One piece of the external_archive change I am working on.  I briefly looked at adding this to TAR archives and it should be possible, but I don't need it now so I didn't add it.

Testing Done:
Added new unit test.
CI running @ https://travis-ci.org/pantsbuild/pants/builds/38752080

Bugs closed: 710

Reviewed at https://rbcommons.com/s/twitter/r/1199/
Updated instructions on how to fetch code from a forked repo, how to kick of a travis-ci build from a PR

Testing Done:
ran build-support/bin/publish_docs.py and looked at the output and clicked on the links.

Bugs closed: 583, 713

Reviewed at https://rbcommons.com/s/twitter/r/1205/
Add a flexible directory re-mapper for the bundle.

Testing Done:
PANTS_DEV=1 ./pants goal test tests/python/pants_test:all
./build-support/bin/ci.sh

Bugs closed: 701

Reviewed at https://rbcommons.com/s/twitter/r/1181/
Do not crash on unicode filenames

Introduce a new function, safe_walk, which wraps os.path.walk, but
ensures that the returned values are unicode objects.  This isn't
strictly safe, in that it is possible that some paths will not be
decodeable, but that case is rare, and the only alternative is to
somehow avoid all interaction between paths and unicode objects, which
seems especially tough in the presence of unicode_literals. See e.g.
https://mail.python.org/pipermail/python-dev/2008-December/083856.html

Testing Done:
Ran new test.

Reviewed at https://rbcommons.com/s/twitter/r/1193/
Revert "Do not crash on unicode filenames"

This reverts commit 8e5131d5ec8d8be91972997620c3b8b73c6b568b.

I pushed a bogus version of this, which somehow lost the kwargs again.
Do not crash on unicode filenames

Introduce a new function, safe_walk, which wraps os.path.walk, but
ensures that the returned values are unicode objects.  This isn't
strictly safe, in that it is possible that some paths will not be
decodeable, but that case is rare, and the only alternative is to
somehow avoid all interaction between paths and unicode objects, which
seems especially tough in the presence of unicode_literals. See e.g.
https://mail.python.org/pipermail/python-dev/2008-December/083856.html

Testing Done:
Ran new test.

Reviewed at https://rbcommons.com/s/twitter/r/1193/