our example repositories snuck into the build dictionary.

Our internal_backend/repositories defines a couple of example repositories that our BUILD files can refer to.

Our BUILD Dictionary builder code looks at symbols defined in our BUILD files to figure out what do doc in http://pantsbuild.github.io/build_dictionary.html

So...

Give our old @manual.builddict() decorator a new optional arg, suppress.

Mark those example repositories with manual.builddict(suppress=True)

BUILD Dictionary builder checks for suppress==True and omits those

Testing Done:
publish_docs.sh , eyeballed

https://travis-ci.org/pantsbuild/pants/builds/36815549

Reviewed at https://rbcommons.com/s/twitter/r/1101/
Protobuf and payload cleanups

Moved some of the protobuf examples around to be more consistent.
Some cleanups from payload refactoring
Added a deprecation message for the unused 'buildflags' attribute in java_protobuf_library
Removed the ability to use a jar_dependency in java_protobuf_library imports.

Testing Done:
CI is green.
Added a unit test for the java_protobuf_library target

Bugs closed: 623

Reviewed at https://rbcommons.com/s/twitter/r/1099/
Adding a list of dirs to exclude from the '::' scan.  The CmdLineSpecParser

By default passes along the workdir as the one dir to exclude.
You can add more directories relative to the build root using 'spec_excludes' in pants.ini

Testing Done:
ci is green.

I also created a bogus BUILD file under .pants.d and ran build-support/bin/ci.sh locally and the tests still passed.

Bugs closed: 618, 621

Reviewed at https://rbcommons.com/s/twitter/r/1091/
'goal builddict' : generate .rst and .html, not just .rst

before: running 'goal builddict' generates a couple of .rst files which we then give to Sphinx

after: running 'goal builddict' generates those .rst files but also a couple of .html files which we then give to sitegen.

If/when sitegen's in good shape, it'd be good to get rid of the .rst-generating part.

Most of this change is in src/python/pants/backend/core/tasks/builddictionary.py , plus new html templates.

There are also changes to some docstrings:

I replaced some sphinx-rst-isms with docutil-rst-isms (which also work with Sphinx).

And I replaced some lost docstrings which I removed with the "inherit docstrings" change because I "knew" they'd be inherited from a superclass.

And added some docstrings which have been missing for a while which I noticed because increased scrutiny grrrrr.

Testing Done:
https://travis-ci.org/pantsbuild/pants/builds/36888156

Running sphinx on the generated .rst files now generates warnings :-(
I poked around 'em for a while, but eventually gave up.

Bugs closed: 529

Reviewed at https://rbcommons.com/s/twitter/r/1079/
Plumb new-style options into the Context.

Allows code to reference option values the new way.

Renamed the overloaded self.options in Command to self.old_options/self.new_options,
to avoid confusion. Previously these collided and the only reason it worked was that
self.options happened to be reset to the right one just before using it.

Change a single flag reference to use the new path, as proof that it works.

With this change we can not only register flags the new way but also reference the option
values the new way. Once everything is ported to use these, we can switch the parser to
the new one, and the transition will be complete.

Testing Done:
ci baking.

Reviewed at https://rbcommons.com/s/twitter/r/1102/
Merge remote-tracking branch 'pantsbuild/master'

Conflicts:
	src/python/pants/goal/context.py
Switch global option registration to the new options system.

Also fix a couple of small issues that arose while testing this.

Also switch the --graph option in list_goals to be read from the
new location, to verify that this all works together.

Note that add_global_options is still used when generating the
build dictionary, so we can't get rid of it yet. We'll do so
in a pending change.

Testing Done:
Ran ci.sh. Ran manual command lines to verify that flags still work.

Reviewed at https://rbcommons.com/s/twitter/r/1109/
Add transitive jar dependencies to depmap project info goal for intellij plugin.

Add transitive jar dependencies to depmap goal for Pants Intellij plugin

When creating projects using the plugin, the depmap showed direct dependencies for 3rdparty jar libraries.
This change add the trasitive dependency jars to the depmap output.
The transitive jars are required to correctly configure intellij modules.

Testing Done:
yes.

Added integration tests but need to verify the contents.

PR created here:
https://github.com/pantsbuild/pants/pull/626
{Coverage increased (+0.04%) when pulling 09790ec on add_transitive_jar_deps into ad7d09d on master.}

Locally tested
./pants tests/python/pants_test/tasks:depmap_integration

Reviewed at https://rbcommons.com/s/twitter/r/1047/
Make sure that 'option java_package' always overrides 'package' in protobuf_gen

Found a bug where:

option java_package = "foo";
package bar;

was processed differently from

package bar;
option java_package = "foo";

For java code generation, the setting of 'option java_package' should always win.

Testing Done:
Added a unit test.

Bugs closed: 629

Reviewed at https://rbcommons.com/s/twitter/r/1108/
Add test for jar_publish.  This required some refactoring so that functions could be mocked.

Includes a few minor cleanups to comments, error messages, docstrings,
etc.  This is in preparation for some improvements to how the scm integration is done.

Testing Done:
Ran the new tests and the integration tests.

Reviewed at https://rbcommons.com/s/twitter/r/1082/
Added an example of how to use Antlr with Java.

Testing Done:
Added integration test and unit test.

Bugs closed: 620

Reviewed at https://rbcommons.com/s/twitter/r/1088/
Retry on failed scm push.  Also pull with rebase, to increase the odds of success.  Don't pull before tagging.

In addition, separate commit from push; this means that SVN will no longer be supportable, but that's OK because SVN users could use git-svn (which is supportable if anyone cares).

This review is meant to apply on to of https://rbcommons.com/s/twitter/r/1082/ -- when we're happy with 1082, I'll rewrite this one as necessary.

Testing Done:
Ran unit and integration tests for jar_publish. Also for git.

Reviewed at https://rbcommons.com/s/twitter/r/1083/
Fix some error reporting issues.

1. Let the new options system throw exceptions to the top in the
   usual way.  We already have error handling there.

2. Ensure that if we throw an exception to the top we report our
   status as FAILURE, not SUCCESS.

Reviewed at https://rbcommons.com/s/twitter/r/1113/
Factor in the jvm version string into the nailgun executor fingerprint

In some configurations, the path to the 'java' executable stays the same but can be manipulated to a different version via a symlink or environment variable.
This causes a mismatch between the already running nailgun server and the jvm that Pants thinks it has chosen.

Testing Done:
Manual testing to make sure that ng servier is killed if symlink to java executable is changed.

Writing a unit test for this looked prohibitivly difficult.

Bugs closed: 639

Reviewed at https://rbcommons.com/s/twitter/r/1122/
Memoize fingerprints by the FPStrategy hash

* Force __hash__ and __eq__ to be defined for FingerprintStrategy
  subclasses.
* Fix implementations of Target.{transitive_,}fingerprint and existing
  FingerprintStrategy subclasses.

Testing Done:
CI is clean at https://travis-ci.org/pantsbuild/pants/builds/37204281

Reviewed at https://rbcommons.com/s/twitter/r/1119/
get 'goal targets' using similar codepath to 'goal builddict'

'goal targets' and the thing that builds build_dictionary.html used different code. At any given time, one or the other worked OK, but never both. Most recently, 'goal targets' doesn't show "inherited" docstrings :-( Also, if you wanted CLI help for a non-target thing like 'artifact', you got none :-(

This change...
moves builddict's reflection-y code out to backend/core/tasks/reflect.py . Tweaks 'goal targets' to get its info from reflect.py. Yay, one less code path. Has the nice side effect of pulling in help for 'artifact', not a target type but useful.

Testing Done:
started up https://travis-ci.org/pantsbuild/pants/builds/37021417

Bugs closed: 584

Reviewed at https://rbcommons.com/s/twitter/r/1112/
Merge remote-tracking branch 'pantsbuild/master'
Add missing coverage deps

Testing Done:
CI is green at https://travis-ci.org/pantsbuild/pants/builds/37057783

Reviewed at https://rbcommons.com/s/twitter/r/1117/
Memoize Address.__hash__ computation.

I noticed during profiling that it is a major hotspot and a very easy
change.  This is unsurprising: addresses are dictionary keys for many of
the fundamental structures in pants.

Also hoist spec_path and target_name into private attributes exposed by properties.

Testing Done:
Travis is being flakey.  ci.sh passed and a previous run of travis was green.

Reviewed at https://rbcommons.com/s/twitter/r/1118/
Read values for a handful of options in the new way.

Provides an example for mass migration.

Also fixes an issue where we weren't "descoping" old-style options
in all the places they could appear.

Reviewed in https://rbcommons.com/s/twitter/r/1114/