Update "Pants Development with IntelliJ IDEA".

commit 1a6f730aa05b7135116e06b0a0faea40d4390d6b
Author: John Sirois <jsirois@twitter.com>
Date:   Tue Apr 29 00:21:31 2014 -0600

    Update "Pants Development with IntelliJ IDEA".

    This updates the docs to work with the new setup
    in pantsbuild/pants as well as refreshing the example
    verbage and screen shots to reflect current IntelliJ.

 src/python/pants/docs/images/intellij-configure-tests.png                        | Bin 0 -> 56869 bytes
 src/python/pants/docs/images/intellij-new-project-1.png                          | Bin 217258 -> 54734 bytes
 src/python/pants/docs/images/intellij-new-project-2.png                          | Bin 133244 -> 24893 bytes
 src/python/pants/docs/images/intellij-new-pythonsdk.png                          | Bin 0 -> 62720 bytes
 src/python/pants/docs/images/intellij-project-structure-modules-dependencies.png | Bin 92174 -> 0 bytes
 src/python/pants/docs/images/intellij-project-structure-modules-sources.png      | Bin 107848 -> 49187 bytes
 src/python/pants/docs/images/intellij-project-structure-project.png              | Bin 109356 -> 53495 bytes
 src/python/pants/docs/images/intellij-run.png                                    | Bin 110186 -> 55478 bytes
 src/python/pants/docs/images/intellij-select-venv.png                            | Bin 0 -> 60848 bytes
 src/python/pants/docs/intellij.rst                                               |  60 ++++++++++++++++++++++++++++----------------------
 10 files changed, 34 insertions(+), 26 deletions(-)

Testing Done:
See the test site here: http://pantsbuild.github.io/sirois-pants-ij-test/intellij.html

Note that I had to work the doc publish process a bit - but that is not yet ready for review - I'll keep it seperate.  It gets easier just like IDEA setup does with a venv for the egg deps in conf.py.

Bugs closed: 88

Reviewed at https://rbcommons.com/s/twitter/r/290/
Added protobuf sample

Bugs closed: 91

Reviewed at https://rbcommons.com/s/twitter/r/291/
Debug log the 404s we get back from the restful artifact cache. Simplifies debugging what parts we didn't find in the cache
Invalidate cache if the Java target version or Scala compiler version changes.

Bugs closed: 76

Reviewed at https://rbcommons.com/s/twitter/r/277/
Use requests instead of httplib for restful_artifact_cache.py. Preparation for enabling auth

Reviewed at https://rbcommons.com/s/twitter/r/294
Reimplement pants help.

Previously there were several different help paths,
help messages weren't uniform and in some cases were messy.

This change:

- Gets rid of the 'help' command.
- Gets rid of the 'help' goal.
- Has a single entry point to help message printing.
- Cleans up help messages. For example, we no longer print
  the list of goals unless you explicitly ask for it,
  because that list is long now.

Note that this gets rid of help for non-goal commands, but
those are going away soon anyway.

This change is in preparation for getting us off optparse and
onto our own custom cmd-line parser. We no longer use optparse's
help messages (but we do still use optparse as an implementation
detail for formatting flag help, for now).
Fix build break.
Add proper package handling for antlr4 targets

Testing Done:
I've tested it locally with grammars we use internally. I might add that before this patch, using the antlr4 compiler was broken and couldn't be used at all.

Reviewed at https://rbcommons.com/s/twitter/r/298/
Use absolute path references in the pants bootstrap script.

commit bc3d021389edf69a8db3135a2e9eafe3a2b20055
Author: John Sirois <jsirois@twitter.com>
Date:   Wed Apr 30 17:15:26 2014 -0600

    Use absolute path references in the pants bootstrap script.

    This allows pants to be run from sources in other repos.

 pants | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

Testing Done:
I could use some help from Eric.

I concocted this local test though to confirm this works:

In my fork - this branch:
$ git diff
diff --git a/src/python/pants/bin/pants_exe.py b/src/python/pants/bin/pants_exe.py
index 70ade97..b10b832 100644
--- a/src/python/pants/bin/pants_exe.py
+++ b/src/python/pants/bin/pants_exe.py
@@ -165,6 +165,7 @@ def _run():
       NailgunTask.killall(None)

 def main():
+  print('>>> Customized locally in %s, running in %s' % (__file__, os.path.realpath(os.getcwd())))
   try:
     _run()
   except KeyboardInterrupt:

Then in pantsbuild/pants master:
$ PANTS_DEV=1 ~/dev-jsirois-pants/pants goal list | head
*** Running pants in dev mode from /home/jsirois/dev-jsirois-pants/src/python/pants/bin/pants_exe.py ***
>>> Customized locally in /home/jsirois/dev-jsirois-pants/src/python/pants/bin/pants_exe.py, running in /home/jsirois/dev/3rdparty/pantsbuild-pants
3rdparty/BUILD:antlr2
3rdparty/BUILD:antlr3
3rdparty/BUILD:antlr-3.4
3rdparty/BUILD:antlr4
3rdparty/BUILD:checkstyle
3rdparty/BUILD:commons-lang
3rdparty/BUILD:finagle-core
3rdparty/BUILD:finagle-thrift
3rdparty/BUILD:guava

Bugs closed: 94

Reviewed at https://rbcommons.com/s/twitter/r/302/
Issue 90: Add Add authentication files to pants bin target
Add tests for netrc

commit a0fce480bdc12343982e652fcd820f7f0c3a8141
Author: Tejal Desai <tdesai@twitter.com>
Date:   Wed Apr 30 20:27:35 2014 -0700

    Fix typo

commit 1b9f27304b06d846935bb18c297238b24e318f2d
Author: Tejal Desai <tdesai@twitter.com>
Date:   Wed Apr 30 11:53:19 2014 -0700

    Style check

commit af6fd235938ada3ba87eb96979945894ecda797d
Author: Tejal Desai <tdesai@twitter.com>
Date:   Wed Apr 30 11:51:53 2014 -0700

    Added style check

commit d5e59f42a06bbdc21e6656bd84eece642e91494b
Author: Tejal Desai <tdesai@twitter.com>
Date:   Wed Apr 30 00:49:46 2014 -0700

    Added tests and code review comments

commit 7616a0bd97b984d442e2acb697c6d50ac48e410e
Author: Tejal Desai <tdesai@twitter.com>
Date:   Tue Apr 29 13:40:08 2014 -0700

    Add authentication files to pants bin target
Hacky fix for issue with inner classes in .proto files with 'java_mulitple_files' enabled

Testing Done:
Updated test_cache_manager.py. Also fixes the issue with inner object definitions being expected to generate .java files from protoc in our repo.

Reviewed at: https://rbcommons.com/s/twitter/r/303/
Rearrange the goal registration code.

- Now goal.py is just the goal command.
- goal/register.py registers the goals.
- Some inline task classes now have their own proper homes.
Fixup doc publishing to work under pantsbuild/pants.

commit 0efa643dfdd347e74206629ab4fa483ec9033475
Author: John Sirois <jsirois@twitter.com>
Date:   Thu May 1 14:37:52 2014 -0600

    Fixup doc publishing to work under pantsbuild/pants.

    Introduce a central driver script to replace the manual process
    and leverage the pants venv for the needed 3rdparty packages.

 3rdparty/python/dev_only_requirements.txt |  6 ++++
 build-support/bin/publish_docs.sh         | 74 ++++++++++++++++++++++++++++++++++++++++++++++++
 pants => build-support/pants_venv         | 58 +++++++++++++------------------------
 pants                                     | 77 ++------------------------------------------------
 src/python/pants/docs/conf.py             | 10 -------
 src/python/pants/docs/docs.rst            | 50 ++++++++++++--------------------
 src/python/pants/docs/publish_via_git.sh  |  2 +-
 7 files changed, 121 insertions(+), 156 deletions(-)

Testing Done:
Re-ran pants bootstrap vanilla and PANTS_DEV=1 to confirm that was not broken by the shuffle.

Then for the new stuff:

Usage
=====
$ ./build-support/bin/publish_docs.sh -h
Publishes the http://pantsbuild.github.io/ docs locally or remotely.

Usage: ./build-support/bin/publish_docs.sh (-h|-opd)
 -h           print out this help message
 -o           open the doc site locally
 -s           publish the doc site remotely
 -d           publish the site to a subdir at this path (useful for public previews)

Hermeticity
===========
# no sphinx - no pants
$ pip uninstall sphinx sphinx_rtd_theme
$ pip list | grep sphinx
$ git clean -fdx

# go
$ ./build-support/bin/publish_docs.sh -opd sirois-test-site
...
*** Running pants in dev mode from /home/jsirois/dev/3rdparty/jsirois-pantsbuild-pants/src/python/pants/bin/pants_exe.py ***

14:45:55 00:00 [main]
               (To run a reporting server: ./pants goal server)
14:45:55 00:00   [setup]
14:45:55 00:00     [bootstrap]
14:45:55 00:00     [parse]
14:45:55 00:00   [builddict]
14:45:55 00:00     [builddict]
                   Generating /home/jsirois/dev/3rdparty/jsirois-pantsbuild-pants/dist/builddict/goals_reference.rst
                   Generating /home/jsirois/dev/3rdparty/jsirois-pantsbuild-pants/dist/builddict/pants_ini_reference.rst
                   Generating /home/jsirois/dev/3rdparty/jsirois-pantsbuild-pants/dist/builddict/build_dictionary.rst
               SUCCESS
Copying '/home/jsirois/dev/3rdparty/jsirois-pantsbuild-pants/dist/builddict/build_dictionary.rst' to '/home/jsirois/dev/3rdparty/jsirois-pantsbuild-pants/src/python/pants/docs'
Copying '/home/jsirois/dev/3rdparty/jsirois-pantsbuild-pants/dist/builddict/goals_reference.rst' to '/home/jsirois/dev/3rdparty/jsirois-pantsbuild-pants/src/python/pants/docs'
Copying '/home/jsirois/dev/3rdparty/jsirois-pantsbuild-pants/dist/builddict/pants_ini_reference.rst' to '/home/jsirois/dev/3rdparty/jsirois-pantsbuild-pants/src/python/pants/docs'
Writing to file '/home/jsirois/dev/3rdparty/jsirois-pantsbuild-pants/src/python/pants/docs/targets.rst'
Writing to file '/home/jsirois/dev/3rdparty/jsirois-pantsbuild-pants/src/python/pants/docs/base.rst'
rm -rf _build/*
sphinx-build -b html -d _build/doctrees   . _build/html
Making output directory...
Running Sphinx v1.2.2
loading pickled environment... not yet created
building [html]: targets for 34 source files that are out of date
updating environment: 34 added, 0 changed, 0 removed
reading sources... [100%] with_intellij
/home/jsirois/dev/3rdparty/jsirois-pantsbuild-pants/src/python/pants/docs/JVMProjects.rst:89: WARNING: Include file u'/home/jsirois/dev/3rdparty/jsirois-pantsbuild-pants/java/com/pants/examples/pingpong/main/BUILD' not found or reading it failed
/home/jsirois/dev/3rdparty/jsirois-pantsbuild-pants/src/python/pants/docs/JVMProjects.rst:100: WARNING: Include file u'/home/jsirois/dev/3rdparty/jsirois-pantsbuild-pants/java/com/pants/examples/pingpong/handler/BUILD' not found or reading it failed
/home/jsirois/dev/3rdparty/jsirois-pantsbuild-pants/src/python/pants/docs/JVMProjects.rst:119: WARNING: Include file u'/home/jsirois/dev/3rdparty/jsirois-pantsbuild-pants/java/com/pants/examples/pingpong/handler/BUILD' not found or reading it failed
/home/jsirois/dev/3rdparty/jsirois-pantsbuild-pants/src/python/pants/docs/JVMProjects.rst:126: WARNING: Include file u'/home/jsirois/dev/3rdparty/3rdparty/jvm/com/sun/jersey/BUILD' not found or reading it failed
/home/jsirois/dev/3rdparty/jsirois-pantsbuild-pants/src/python/pants/docs/pants_ini_reference.rst:16: WARNING: malformed hyperlink target.
/home/jsirois/dev/3rdparty/jsirois-pantsbuild-pants/src/python/pants/docs/pants_ini_reference.rst:29: WARNING: malformed hyperlink target.
/home/jsirois/dev/3rdparty/jsirois-pantsbuild-pants/src/python/pants/docs/tshoot.rst:39: ERROR: Unexpected indentation.
looking for now-outdated files... none found
pickling environment... done
checking consistency... done
preparing documents... done
writing output... [100%] with_intellij
writing additional files... (83 module code pages) _modules/index genindex py-modindex search
copying images... [100%] images/intellij-project-structure-project.png
copying static files... WARNING: html_static_path entry u'/home/jsirois/dev/3rdparty/jsirois-pantsbuild-pants/src/python/pants/docs/_static' does not exist
done
copying extra files... done
dumping search index... done
dumping object inventory... done
build succeeded, 8 warnings.

Build finished. The HTML pages are in _build/html.
To abort publishing these docs to http://pantsbuild.github.io/sirois-test-site press CTRL-C, otherwise press enter to continue.Created new window in existing browser session.
To abort publishing these docs to http://pantsbuild.github.io/sirois-test-site press CTRL-C, otherwise press enter to continue.
Cloning into 'pantsbuild.github.io'...
remote: Counting objects: 4374, done.
remote: Compressing objects: 100% (706/706), done.
remote: Total 4374 (delta 3938), reused 4024 (delta 3641)
Receiving objects: 100% (4374/4374), 7.60 MiB | 684.00 KiB/s, done.
Resolving deltas: 100% (3938/3938), done.
Checking connectivity... done.
[master 2045c2b] publish by jsirois
...
 create mode 100644 sirois-test-site/with_intellij.html
Counting objects: 10, done.
Delta compression using up to 8 threads.
Compressing objects: 100% (8/8), done.
Writing objects: 100% (8/8), 50.88 KiB | 0 bytes/s, done.
Total 8 (delta 1), reused 0 (delta 0)
To git@github.com:pantsbuild/pantsbuild.github.io.git
   1ff2f17..2045c2b  master -> master
Created new window in existing browser session.

See the results here: http://pantsbuild.github.io/sirois-test-site/docs.html

Bugs closed: 98

Reviewed at https://rbcommons.com/s/twitter/r/305/
Support non-ascii class names.

See https://github.com/pantsbuild/pants/pull/101 for details.
Move rwbuf from commons into pants.

commit 39d266964e2ff39ba0ab9f309b920f474fa83954
Author: John Sirois <jsirois@twitter.com>
Date:   Mon May 5 17:08:39 2014 -0600

    Blank out the __init__.py for easy no-shenanigans checkability.

 src/python/pants/rwbuf/__init__.py | 2 --
 1 file changed, 2 deletions(-)

Testing Done:
$ ./build-support/bin/ci.sh

Bugs closed: 102

Reviewed at https://rbcommons.com/s/twitter/r/308/
Add back missing __init__.py file.
emma refactoring
works again
refactored setup_parser
narrowed Task methods sent to _JUnitRUnner and children by putting them in a namedtuple