Add test for jar_publish.  This required some refactoring so that functions could be mocked.

Includes a few minor cleanups to comments, error messages, docstrings,
etc.  This is in preparation for some improvements to how the scm integration is done.

Testing Done:
Ran the new tests and the integration tests.

Reviewed at https://rbcommons.com/s/twitter/r/1082/
Added an example of how to use Antlr with Java.

Testing Done:
Added integration test and unit test.

Bugs closed: 620

Reviewed at https://rbcommons.com/s/twitter/r/1088/
Retry on failed scm push.  Also pull with rebase, to increase the odds of success.  Don't pull before tagging.

In addition, separate commit from push; this means that SVN will no longer be supportable, but that's OK because SVN users could use git-svn (which is supportable if anyone cares).

This review is meant to apply on to of https://rbcommons.com/s/twitter/r/1082/ -- when we're happy with 1082, I'll rewrite this one as necessary.

Testing Done:
Ran unit and integration tests for jar_publish. Also for git.

Reviewed at https://rbcommons.com/s/twitter/r/1083/
Fix some error reporting issues.

1. Let the new options system throw exceptions to the top in the
   usual way.  We already have error handling there.

2. Ensure that if we throw an exception to the top we report our
   status as FAILURE, not SUCCESS.

Reviewed at https://rbcommons.com/s/twitter/r/1113/
Factor in the jvm version string into the nailgun executor fingerprint

In some configurations, the path to the 'java' executable stays the same but can be manipulated to a different version via a symlink or environment variable.
This causes a mismatch between the already running nailgun server and the jvm that Pants thinks it has chosen.

Testing Done:
Manual testing to make sure that ng servier is killed if symlink to java executable is changed.

Writing a unit test for this looked prohibitivly difficult.

Bugs closed: 639

Reviewed at https://rbcommons.com/s/twitter/r/1122/
Memoize fingerprints by the FPStrategy hash

* Force __hash__ and __eq__ to be defined for FingerprintStrategy
  subclasses.
* Fix implementations of Target.{transitive_,}fingerprint and existing
  FingerprintStrategy subclasses.

Testing Done:
CI is clean at https://travis-ci.org/pantsbuild/pants/builds/37204281

Reviewed at https://rbcommons.com/s/twitter/r/1119/
get 'goal targets' using similar codepath to 'goal builddict'

'goal targets' and the thing that builds build_dictionary.html used different code. At any given time, one or the other worked OK, but never both. Most recently, 'goal targets' doesn't show "inherited" docstrings :-( Also, if you wanted CLI help for a non-target thing like 'artifact', you got none :-(

This change...
moves builddict's reflection-y code out to backend/core/tasks/reflect.py . Tweaks 'goal targets' to get its info from reflect.py. Yay, one less code path. Has the nice side effect of pulling in help for 'artifact', not a target type but useful.

Testing Done:
started up https://travis-ci.org/pantsbuild/pants/builds/37021417

Bugs closed: 584

Reviewed at https://rbcommons.com/s/twitter/r/1112/
Merge remote-tracking branch 'pantsbuild/master'
Add missing coverage deps

Testing Done:
CI is green at https://travis-ci.org/pantsbuild/pants/builds/37057783

Reviewed at https://rbcommons.com/s/twitter/r/1117/
Memoize Address.__hash__ computation.

I noticed during profiling that it is a major hotspot and a very easy
change.  This is unsurprising: addresses are dictionary keys for many of
the fundamental structures in pants.

Also hoist spec_path and target_name into private attributes exposed by properties.

Testing Done:
Travis is being flakey.  ci.sh passed and a previous run of travis was green.

Reviewed at https://rbcommons.com/s/twitter/r/1118/
Read values for a handful of options in the new way.

Provides an example for mass migration.

Also fixes an issue where we weren't "descoping" old-style options
in all the places they could appear.

Reviewed in https://rbcommons.com/s/twitter/r/1114/
sitegen task, create site with better navigation.

Still not everything we want from a basic site generator,
but some steps in the right direction:

A site-wide table of contents, an outline in which the "nodes" are pages.

For each page,
+ page-wide table of contents, a list of headings.
+ "breadcrumbs" up top, that show how/if this page fits into outline

Testing Done:
added some tests. mostly "normal" cases, didn't try to dream up corner cases.

added some non-ASCII text to some test data, inspired by recent Unicode-mangling stories on the mailing list.

started https://travis-ci.org/pantsbuild/pants/builds/36613009

Reviewed at https://rbcommons.com/s/twitter/r/1086/
Ensure that we don't create a forwarding cycle.

Testing Done:
ci.sh passed.

Reviewed at https://rbcommons.com/s/twitter/r/1124/
Filter out None scopes, to avoid trouble later.

I'm not sure why we have tasks with no set scope, but we've seen
this with custom tasks at Foursqare. Will figure that out later.

Testing Done:
Running ci.

Reviewed at https://rbcommons.com/s/twitter/r/1125/
Migrate all jvm compilation options to the new registration style.

Testing Done:
CI passed. Manual verification that flags still work.

Reviewed at https://rbcommons.com/s/twitter/r/1127/
Update help message for ./pants goal goals  (was ./pants goals)

./pants goals fails with a message:

BadSpecError: BUILD file does not exist at: /Users/zundel/Src/pants/goals/BUILD

Testing Done:
manual inspection of the help output.

Bugs closed: 644

Reviewed at https://rbcommons.com/s/twitter/r/1128/
Publishing doc fixes

Further cleanups to the publishing doc to avoid sharp edges and fix a formatting error.

Testing Done:
Eyeballed:
./build-support/bin/publish_docs.sh -o

Reviewed at https://rbcommons.com/s/twitter/r/1136/
Remove hack to add java_sources in context

Testing Done:
Yes:
https://travis-ci.org/pantsbuild/pants/builds/37349364

Maunally generated idea for
./pants goal idea testprojects/src/scala/com/pants/testproject/javasources:javasources and
./pants goal idea testprojects/src/scala/com/pants/testproject/javasources:javasources --idea-intransitive

Both have dependent java_sources targets configured in generated Project Structure.

With Eric's hack to add java_sources in ide_gen for scala_library, we no longer need the java_sources to be added to the self.context targets.
https://github.com/pantsbuild/pants/blob/master/src/python/pants/backend/jvm/tasks/ide_gen.py#L552

Reviewed at https://rbcommons.com/s/twitter/r/1130/
When a jvm doc tool (e.g. scaladoc) fails in combined mode, throw an exception.  Log runs of jvmdoc.  Eliminate unnecessary dict lookup.

Testing Done:
Ran tests.

Reviewed at https://rbcommons.com/s/twitter/r/1116/
Added documentation on the difference between a pants.pex for testing and production

Bugs closed: 648

Reviewed at https://rbcommons.com/s/twitter/r/1134/