Move workdirs creation from __init__ to pre_execute in jvm_compile & Remove QuietTaskMixin from several tasks.

- jvm_compile creates a bunch of dirs in its __init__ method which gets wiped out by clean-all. Instead, move the dirs creation to pre_execute().
- Another issue is clean-all/clean-all-async/invalidate/ng-killall are with QuietTaskMixin, and once there is one task is QuietTaskMixin, console output will be all suppressed by reporter. This is a fairly common use of combining these goals with others like compile/publish/test/etc.

This is because we only have one reporter for all tasks. A long term solution could be that we have two reporters (or more), one for quiet or one verbose. And tasks can each use its own reporter (not sure if there is any output lineage problem if combined).

Or we can simply make these 3 tasks non quiet. Rest of the ConsoleTask(QuietTaskMixin) derived goals/tasks, their usage is generally standalone. Thus no need for changes.

Testing Done:
PANTS_DEV=1 ./pants goal test tests::

Bugs closed: 325

Reviewed at https://rbcommons.com/s/twitter/r/1173/
Add PantsRunIntegrationTest into pantsbuild.pants.testinfra package.

Testing Done:
ran `build-support/bin/release.sh -n` and verified the inclusion.

Bugs closed: 696

Reviewed at https://rbcommons.com/s/twitter/r/1185/
doc: dust off the Task Developer's Guide :-) rm the "pagerank" example

Our page about how to develop a Task described "science-era" Pants.

Removed a lot of what's there; it talked about APIs that have since become new and improved.

Removed the "pagerank" example. :-( When I started on this change, I thought I was gonna update this example and maybe move it to examples/src/... so it would get exercised and not drift back into b0rken-ness . But it's kind of a weird example--it doesn't really care about products, it doesn't do "the usual" things with deps. So I find myself deleting it... but with some misgivings, because it's cool. If someone wants to do the work to modernize it and move it to examples, I'll happily code review :-)

Replaced copy-pasted code snippets with "includes" from live tasks.
Maybe this won't drift out of date so easily from now on.
(Didn't add includes for the JVM section, just a TODO. But did remove now-obsolete sample code.)

rm'd some redundant-and-obsolete info from the Internals page, replacing w/link to Task Developer's guide. (Thanks to Mateo for spotting this!) The basic concepts were OK, but the details, yikes. Internals page had info about GroupTask, which Task Developers page lacked... so move it over.

Bugs closed: 433

Reviewed at https://rbcommons.com/s/twitter/r/1074/
Use a Session to enable connection pooling

Requests and urllib3 automatically reuse connections if using a Session, which otherwise has all the same methods as the main Requests API, making this a drop-in replacement: http://docs.python-requests.org/en/latest/user/advanced/

In my tests, this change cut 830 HEAD requests to Foursquare's internal artifact cache from 22s to 11s.

Testing Done:
Using 830 cache keys copied from a real build, this change cut the time to do HEAD requests for each from 22s to 11s.

https://travis-ci.org/pantsbuild/pants/builds/38515947

Reviewed at https://rbcommons.com/s/twitter/r/981/
Move requests-logging setting to global scope

This should happen at import, since the __init__ may have been run in a different proc, and the cache pickled, since then.

Testing Done:
https://travis-ci.org/pantsbuild/pants/builds/38522554

Reviewed at https://rbcommons.com/s/twitter/r/1022/
The traversal order used is preorder, not inorder; support postorder traversal

1. The term "inorder traversal" does not make sense outside of the context of binary trees.  Instead, use the term "preorder", which is actually what is done.

2. Also allow for postorder traversal (and test that it works).

Postorder stuff is needed for https://rbcommons.com/s/twitter/r/1179/

Testing Done:
./pants goal test tests/python/pants_test/graph:graph

Reviewed at https://rbcommons.com/s/twitter/r/1177/
Handle list-typed options (action='append') correctly.

Ensure that we read them from config in the right way, and apply type
conversion to the list members.

Added test cases for this.

Testing Done:
Ran the options unittests before and after the fix, to verify
that the test discriminates this.

CI passed.

Reviewed at https://rbcommons.com/s/twitter/r/1183/