Refactor options bootstrapping.

Instead of free-floating functions, the bootstrapping is now
encapsulated in a bootstrapper class. This class exposes both the
bootstrap and full options.

This is necessary to support the startup sequence in pants_exe.py properly:

- To create full options we need the known scopes.
- To get the known scopes we need to load plugins and extensions.
- The plugin and extension registration code may use config.

Therefore we need a way to load the bootstrap config, then do some work,
then load the full options.

Currently we achieve this with a gross manual config-reading hack in pants_exe.py.
This change will allow us to get rid of that hack.

This change also fixes an issue where in tests the buildroot wasn't plumbed in
everywhere it should be.

Testing Done:
Unit tests pass. Integration tests pass.

Reviewed at https://rbcommons.com/s/twitter/r/1453/
Fix ./pants to ./pants goal gaols in tutorial.

Fix ./pants to ./pants goal goals

Since ./pants gives you an error saying no goals to prepare.

Reviewed at https://rbcommons.com/s/twitter/r/1427/
Always relativize the classpath where possible.

Mesos produces very long path names, and repeating these names in the classpath can exceed the max length of a single argument (which defaults to 131072 on Linux).

Testing Done:
https://travis-ci.org/pantsbuild/pants/builds/43541955

(and then ran new tests locally).

Reviewed at https://rbcommons.com/s/twitter/r/1455/
Fix gitignore path for bin to only include root bin directory
(which is created by eclipse); remove unnecessary unexclusions.

Reviewed at https://rbcommons.com/s/twitter/r/1456/
s/whre/where/ in first tutorial

Reviewed at https://rbcommons.com/s/twitter/r/1464/
Fix a bug where a request for help wasn't detected.

Testing Done:
options unittests pass.

Reviewed at https://rbcommons.com/s/twitter/r/1467/
Get rid of the Command class completely.

GoalRunner now just extends object.

The functionality previously in Command is now in GoalRunner.

Most of the functionality in pants_exe.py is now in GoalRunner too.

RIP old pants...

Testing Done:
All integration and unit tests pass.

Reviewed at https://rbcommons.com/s/twitter/r/1465/
Optimize goal changed

Make goal changed approximately O(paths + targets), instead of something much worse.

Testing Done:
Correctness: https://travis-ci.org/pantsbuild/pants/builds/43764453

Performance: tested on a megachange branch on Twitter's monorepo; previous time was on the order of hours; time with patch is 3 minutes.

Reviewed at https://rbcommons.com/s/twitter/r/1470/
Handle UTF-8 in thrift files for python

Testing Done:
Ran new tests (red/green/go)

Reviewed at https://rbcommons.com/s/twitter/r/1459/
Move GoalRunner to the bin package, together with pants_exe.py.

Gets rid of the command package.

Testing Done:
Pants builds. Dev and regular pants work.

Reviewed at https://rbcommons.com/s/twitter/r/1471/
docs: update a couple of examples that showed Pants output. old Pants output

specifically, we caught these as places that contain the string "BUILD:", and
in a refactor a while back, that string was a symptom of obsolescence.

We fixed most of those in passing, but these remain, https://github.com/pantsbuild/pants/issues/461

Testing Done:
rendered at
* http://pantsbuild.github.io/lahosken-demo/build_files.html#debugging-a-build-file
* http://pantsbuild.github.io/lahosken-demo/publish.html#does-not-provide-an-artifact

Bugs closed: 461

Reviewed at https://rbcommons.com/s/twitter/r/1401/
Cleanup after custom options bootstrapping in reflect.

The custom bootstrapping to get a symbolic buildroot for
doc generation was affecting global state and leaving a
buildroot of <buildroot> laying around for subsequent code
to pick up and unwittingly try to use.

Testing Done:
Locally this was failing before, passed after:
```console
$ PANTS_DEV=1 ./pants goal test tests/python/pants_test/tasks:reflect tests/python/pants_test/backend/python:test_builder
```

Also generated docs and confirmed the intended symbolic <buildroot> was being interpolated: http://pantsbuild.github.io/jsirois/goals_reference.html#Common

Installed python 3.2.5 after which the following failed but then passed after diff 3:
```console
$ PATH=/home/jsirois/.pyenv/versions/3.2.5/bin:$PATH pants.dev goal test tests/python/pants_test/tasks:reflect tests/python/pants_test/backend/python:test_builder
```

CI went green here: https://travis-ci.org/pantsbuild/pants/builds/43773742

Bugs closed: 861

Reviewed at https://rbcommons.com/s/twitter/r/1468/
Restore the deprecation warning for specifying 'goal' on the cmdline.

Reviewed at https://rbcommons.com/s/twitter/r/1476/
Fixup test_test_builder.py interpreter caching.

Comparing PythonInterpeters directly is currently broken for systems
that have atime turned on for the filesystem hosting the python binary.
This localizes the 'correct' equality check proposed in:
  https://rbcommons.com/s/twitter/r/1477/

Testing Done:
The same change was tested against OSX here: https://travis-ci.org/pantsbuild/pants-for-travis-osx-ci/builds/43799766

This change went green against linux CI here: https://travis-ci.org/pantsbuild/pants/builds/43803116

Bugs closed: 865

Reviewed at https://rbcommons.com/s/twitter/r/1478/
Make ./pants help and ./pants help <goal> work properly.

Make ./pants help and ./pants help [goal] work properly.

Fixes https://github.com/pantsbuild/pants/issues/866 and https://github.com/pantsbuild/pants/issues/839

Testing Done:
Options unittests pass. CI baking.

Reviewed at https://rbcommons.com/s/twitter/r/1482/
Kill pants own use of the deprecated goal command.

This pass does not go for docs, just code and scripts.

Testing Done:
Locally:
```console
$ ./build-support/bin/publish_docs.sh -o
$ ./build-support/bin/release.sh -n
```

CI went green here: https://travis-ci.org/pantsbuild/pants/builds/43877791

Bugs closed: 864

Reviewed at https://rbcommons.com/s/twitter/r/1479/
Fix missing import.

Reported by Zero Cho.

Reviewed at https://rbcommons.com/s/twitter/r/1483/
fix/refactor checkstyle

- Renaming BUILD.tools's twitter-checkstyle target to checkstyle since this is not
  twitter specific
- Adding build-support/checkstyle/* to wire up the checkstyle
- A few longer than 100-character per line java files fixes
- pants.ini, fix/change broken [checkstyle] to [compile.checkstyle]
- pants.ini remove suppression_files setting:
  There are a couple reasons for doing this:
  1) OS pants and com.puppycrawl.tools.checkstyle only supports single suppression file.
     The multiple suppression files concept is from a Twitter internal custom filter, which
     isn't applicable here.
  2) But more important, suppression isn't a required component for running checkstyle.
     Someone can perfectly author a coding_style.xml which contains no SuppressionFilter
     filter. And right now the checkstyle internally reads suppression_files list and injects
     them into the properties, which assuming the hardcoded SuppressionFilter. Doesn't make
     any sense. Anyone wants to use suppression, can still put checkstyle.suppression.file
     into properties setting under [compile.checkstyle] in OS pants, or any company specific
     pants.ini
- Adding the checkstyle task into the optional backend so when we run PANTS_DEV=1 ./pants
  in OS pants, it's added, however, it's not by default wired up in pants release. Anyone
  can still wire it up in their own wrapper (like what Twitter currently does)
- In ide_gen, currently it tries to read suppression_files from [checkstyle]. First, like
  I mentioned above, the default open source checkstyle doesn't have multiple suppression
  files concept. Second, the current setting points to an invalid file that doesn't exist
  anyway. So instead, I'm setting it to [] for now and will follow up with ide/idea owners
  later.
- Adding unit tests for checkstyle to prevent future regressions.

Testing Done:
PANTS_DEV=1 ./pants goal compile examples/src/java::
PANTS_DEV=1 ./pants goal test tests/python/pants_test/backend/jvm/tasks:checkstyle

Travis baking:
https://travis-ci.org/jinfeng/jinfeng-pants-fork/builds/43051952

Reviewed at https://rbcommons.com/s/twitter/r/1432/
Use subscripting for looking up targets in resources_by_products

And add a test for a regression that RB 1250 caused when multiple partitions
were combined with resources_by_targets.

Testing Done:
Ran tests

Reviewed at https://rbcommons.com/s/twitter/r/1380/
CmdLineProcessor uses binary class name (http://docs.oracle.com/javase/specs/jvms/se7/html/jvms-4.html#jvms-4.2.1),

https://github.com/twitter/commons/blob/master/src/java/com/twitter/common/args/apt/CmdLineProcessor.java#L653

Classes that uses CmdLineProcessor as annotation processor would fail,

"Expected classname -> path, got "com.twitter.common.args.ArgsTest$App -> ..."

This patch adds $ to the class name matching, and changed the format of compile/jvm/java/classes/deprecation_report.txt slightly so it's easier to load in testing

Testing Done:
https://travis-ci.org/pantsbuild/pants/builds/44126276 passed

Bugs closed: 868

Reviewed at https://rbcommons.com/s/twitter/r/1489/