Update help message for ./pants goal goals  (was ./pants goals)

./pants goals fails with a message:

BadSpecError: BUILD file does not exist at: /Users/zundel/Src/pants/goals/BUILD

Testing Done:
manual inspection of the help output.

Bugs closed: 644

Reviewed at https://rbcommons.com/s/twitter/r/1128/
Publishing doc fixes

Further cleanups to the publishing doc to avoid sharp edges and fix a formatting error.

Testing Done:
Eyeballed:
./build-support/bin/publish_docs.sh -o

Reviewed at https://rbcommons.com/s/twitter/r/1136/
Remove hack to add java_sources in context

Testing Done:
Yes:
https://travis-ci.org/pantsbuild/pants/builds/37349364

Maunally generated idea for
./pants goal idea testprojects/src/scala/com/pants/testproject/javasources:javasources and
./pants goal idea testprojects/src/scala/com/pants/testproject/javasources:javasources --idea-intransitive

Both have dependent java_sources targets configured in generated Project Structure.

With Eric's hack to add java_sources in ide_gen for scala_library, we no longer need the java_sources to be added to the self.context targets.
https://github.com/pantsbuild/pants/blob/master/src/python/pants/backend/jvm/tasks/ide_gen.py#L552

Reviewed at https://rbcommons.com/s/twitter/r/1130/
When a jvm doc tool (e.g. scaladoc) fails in combined mode, throw an exception.  Log runs of jvmdoc.  Eliminate unnecessary dict lookup.

Testing Done:
Ran tests.

Reviewed at https://rbcommons.com/s/twitter/r/1116/
Added documentation on the difference between a pants.pex for testing and production

Bugs closed: 648

Reviewed at https://rbcommons.com/s/twitter/r/1134/
BUILD Dictionary vs factory functions: doc the class, not the factory

Wondering why the docs for 'bundle' say 'Return a factory method that can
create bundles rooted at the parse context path.' ? This is why.

This change...
introduces another @manaul.builddict() param, factory. The builddict
goal, if it sees a symbol tagged @manual.builddict(factory=True), hopes
the factory is a class method and doc's the attached class instead of
the factory function itself.

Testing Done:
https://travis-ci.org/pantsbuild/pants/builds/37459441

Bugs closed: 580

Reviewed at https://rbcommons.com/s/twitter/r/1126/
no longer git-ignore these no-longer-generated files

We stopped generating these back in July. Surely everybody's done a 'git clean' since then. Surely.

Testing Done:
git status

Bugs closed: 371

Reviewed at https://rbcommons.com/s/twitter/r/1144/
Changes dependees task to register options using the new system

Testing Done:
ci.sh and manual testing of options.

Reviewed at https://rbcommons.com/s/twitter/r/1138/
Remove debug_args from pants.ini.  Fixes issue 650 by removing string interpolation of debug_port

Testing Done:
Added a unit test for new jvm_debug_config.py

Ran the following tests with `debug_port` unset and set in pants.ini to make sure debug port was actually overridden

./pants goal test ./examples/tests/java/com/pants/examples/hello/greet --test-junit-debug
./pants goal test ./examples/tests/java/com/pants/examples/hello/greet --test-junit-debug --test-junit-coverage --test-junit-coverage-processor=emma
./pants goal test ./examples/tests/java/com/pants/examples/hello/greet --test-junit-debug --test-junit-coverage --test-junit-coverage-processor=cobertura
./pants goal run testprojects/src/java/com/pants/testproject/unicode/main --run-jvm-run-debug
./pants goal idea ./src:: # checked the project.i* file for the debug port

I could not find examples or testprojects for goals benchmark and spec to test them out.

Bugs closed: 650, 652

Reviewed at https://rbcommons.com/s/twitter/r/1137/
list assertion for python_requirement_library's requirements

Testing Done:
Ran new tests.

Reviewed at https://rbcommons.com/s/twitter/r/1142/
Fix issue with appendable options in new options system.

Reviewed at https://rbcommons.com/s/twitter/r/1131/
Read jvm compilation options the new way.

Also fix a bug where we weren't detecting legacy inverse boolean flags correctly.

Testing Done:
CI running. Manual use of relevant flags.

Reviewed at https://rbcommons.com/s/twitter/r/1129/
A unicode-safe wrapper around shlex.split().
Temporarily move nailgun flag registration to be global.

It was causing problems during option migration. This temporarily
widens an existing abstraction leak, but once we're fully over to
the new options system this naturally goes away.

Also - ensure task options are registered in proper scope order.

Testing Done:
ci is baking.

Reviewed at https://rbcommons.com/s/twitter/r/1140/
Fix a BUILD file bug.
Make old-style flag scoping conform to the new style rules.

Specifically, a singleton task in a goal must be fully qualified
by the goal name, unless the task has the same name as the goal.

E.g., --ivy-report becomes --resolve-ivy-report.

This only affects a small number of flags, and has to happen
sooner or later anyway.

Reviewed at https://rbcommons.com/s/twitter/r/1146/
Simpler implementation of flag prefix gathering.

The previous implementation detected "leaf scopes", to avoid matching
a prefix of a longer prefix. However this was more trouble than it
was worth, and the same effect can be achieved by simply sorting
the scopes.

Reviewed in https://rbcommons.com/s/twitter/r/1147/
Merge branch 'master' into task_newoptionsreg_doclike