Temporarily change stderr log level to silence log.init if --quiet

Otherwise, if disk logging is configured, it prints 'logging to /path/to/something' even when in quiet mode (https://github.com/twitter/commons/blob/master/src/python/twitter/common/log/initialize.py#L285)

Testing Done:
https://github.com/pantsbuild/pants/pull/743

https://travis-ci.org/pantsbuild/pants/builds/39412239

Reviewed at https://rbcommons.com/s/twitter/r/1243/
Add notes about JDK versions supported.

Bugs closed: https://github.com/pantsbuild/pants/issues/739

Reviewed at https://rbcommons.com/s/twitter/r/1241/
Migrate ivy_resolve options to new options system

Testing Done:
inspected output of pants goal resolve.ivy examples/src/java:: --help
ran several invocations by hand.

Bugs closed: 738

Reviewed at https://rbcommons.com/s/twitter/r/1234/
Fix the issue of seeing "Error while writing to artifact cache: an integer is required" during [cache check]

Apparently the order of args was swapped by mistake. Fixed the issue by using named param calling style and updated the arg validation which would fail tests if similar problem occurs.

Testing Done:
https://travis-ci.org/jinfeng/jinfeng-pants-fork/builds/39441973 has passed.

local tests:

PANTS_DEV=1 ./pants goal test ./tests/python/pants_test/cache/

Reviewed at https://rbcommons.com/s/twitter/r/1249/
Fix the regression introduced in https://rbcommons.com/s/twitter/r/1186/

The original code in benchmark_run.py was:

  option_group.add_option(mkflag("target"), dest="target_class", action="append",
                            help="Name of the benchmark class.")
  ...
  ...
  self.caliper_args = self.context.options.target_class
  self.caliper_args += [ str1, str2 ]
  self.caliper_args.extend([str3, str4])
  ...
  ...
  # eventually deep in the callstack, a maybe_list(caliper_args) is called.

For whatever reason, we used "append" and the target_class is a list (even though
it should be a single string argument). But anyhow, it works, since the it was an
assignment "self.caliper_args = self.context.options.target_class"

Now the change introduced in the RB, while keeping the target still a list, it
changes the line "self.caliper_args = self.context.options.target_class" to

  self.args.insert(0, self.get_options().target)

Note even though it's called "target", it's actually an list. Thus we're inserting
a list as an element into a list. Later the maybe_list failed.

- also fixed the pants.ini benchmark setting name mismatch.

Testing Done:
local benchmark run worked.

CI (with all 3 commits) is baking: https://travis-ci.org/jinfeng/jinfeng-pants-fork/builds/39519055

Reviewed at https://rbcommons.com/s/twitter/r/1254/
Migrate ensime_gen.py, reporting_server.py to new options format.

Testing Done:
manually invoking tasks w/ command line flags.
ci is running

Bugs closed: 746

Reviewed at https://rbcommons.com/s/twitter/r/1251/
fix signature in 'goal targets' and BUILD dictionary

We construct the arg signature for Targets by walking up the class inheritance chain, accumulating info. But if more than one class in the chain mentions a param, then... we were showing it more than once in the arg signature, whoops. This had other side effects: we doc'd some params we meant to hide.

With this change, the Target-class-walking code got big enough such that I separated it out into its own function.

Testing Done:
https://travis-ci.org/pantsbuild/pants/builds/39509603

Reviewed at https://rbcommons.com/s/twitter/r/1253/
RESTfulCache delegates tarfile handling to LocalArtifactCache

The motivation for this change is to allow artifacts downloaded by RESTfulCache to be directly added to the LocalArtifactCache without needing to decompress and recompress them as we do now.

At a high level, this is done by refactoring RESTfulCache such that it no longer handles creating or extracting tarfiles directly and instead delegates those operations to a LocalArtifactCache instance passed in its constructor, which may store them in the process.

This is a revision of a previous proposal (https://rbcommons.com/s/twitter/r/985/) along similar lines, though this iteration has RESTful artifact cache always passed a LocalArtifactCache at construction, to which it can always delegate all local file interaction (tarfile creation and extraction)

This simplifies RESTful cache, reducing it to only the HTTP-related functionality. This also eliminates duplication of tarfile creation and extraction logic, which now justs lives in the LocalArtifactCache.

TempLocalArtifactCache provides an alternate implementation of LocalArtifactCache, backed by tmpfiles instead of a cache_root. This can be passes to RESTfulCache to handle artifacts when running in a remote-only cache configuration.

Notes:

This change removes all usage of CombinedCache -- the only case where it was used,
composing an HTTP and local cache, is covered by the above changes.

It also removes the  "backfill" behavior from CombinedCache, which only made sense in the above setup: two remote caches should likely use rsyncd or whatever to share files rather than relying on clients building to shuttle artifacts between them and local caches can use symlinks or mounts.

Caches also now use stdlib logging instead of context.log since the latter
has references to the runtracker making it almost impossible to pickle.

Caches now use string.format instead of % for composing log messages.

Add more Sphinx style docstrings.

Testing Done:
https://travis-ci.org/pantsbuild/pants/builds/39519073 passed

Bugs closed: 737

Reviewed at https://rbcommons.com/s/twitter/r/1233/
new param for page target, links. List of hyperlinked-to targets

List of hyperlinked-to targets. We use dependencies for this now.
But circular dependencies are an error while hyperlink-cycles should be OK.

So param page(links=[...]) feeds into traversable_specs (versus traversable_dependency_specs).

While I was looking at the tests, un-commented-in some commented-out assertions that had a done TODO.

Testing Done:
https://travis-ci.org/pantsbuild/pants/builds/39520576

Bugs closed: 253, 742

Reviewed at https://rbcommons.com/s/twitter/r/1242/
Migrate some utility tasks to new-style options.

Reviewed at https://rbcommons.com/s/twitter/r/1248/