Install all deps as eggs.

To skirt around IDE bugs like http://youtrack.jetbrains.com/issue/PY-6477.
docs: publish exists now, so don't warn of nonexistence. publish auth

While I was looking at it, removed a no-longer-useful "twitter" from
an error message
Cleaned up (ab)uses of _jvm_tool_bootstrapper.* by replacing them with the published methods from class Task.
While I was at it, I fixed some of the more egregious style violations.
Revert "Minimize the classpath pants passes to java for running tests."

commit f02492d4b37c812e10bded1efc3253dfc1620545
Author: John Sirois <jsirois@twitter.com>
Date:   Wed Apr 23 11:09:30 2014 -0600

    Revert "Minimize the classpath pants passes to java for running tests."

    This reverts commit aa4a419785e1d45e778292ddc67df90420f60185.

    This change breaks code that tries to re-formulate classpaths.
    A notable example is here:
      https://github.com/twitter/util/blob/master/util-eval/src/main/scala/com/twitter/util/Eval.scala#L317

    We'll attack this from a different angle.

 src/python/pants/ivy/ivy.py         | 30 +++++++++++++-----------------
 src/python/pants/java/BUILD         |  1 -
 src/python/pants/java/executor.py   | 41 ++++-------------------------------------
 src/python/pants/java/util.py       | 12 ++++++------
 src/python/pants/tasks/ivy_utils.py | 30 +++++++++++++++---------------
 5 files changed, 38 insertions(+), 76 deletions(-)

Testing Done:
None.

Reviewed at https://rbcommons.com/s/twitter/r/269/
Instead of always bootstrapping to eggs, only do this in dev mode.

commit 045a10f5c7d2f603712c901addc7e12096c415d3
Author: John Sirois <jsirois@twitter.com>
Date:   Wed Apr 23 13:01:45 2014 -0600

    Instead of always bootstrapping to eggs, only do this in dev mode.

    Since egg bootstrapping is just there to support IDEs - implying
    PANTS_DEV mode makes sense to be using, and, more importantly,
    since travis-ci runs choke on the generated eggs almost 100% of the
    time [1], just use eggs in dev mode.

    [1] pip fails like so:
      Running setup.py install for twitter.common.dirutil

        installing library code to build/bdist.linux-x86_64/egg
        removing 'build/bdist.linux-x86_64/egg' (and everything under it)
        Processing twitter.common.dirutil-0.1.3-py2.7.egg
        Removing /home/travis/build/pantsbuild/pants/build-support/pants_deps.venv/lib/python2.7/site-packages/twitter.common.dirutil-0.1.3-py2.7.egg
        twitter.common.dirutil 0.1.3 is already the active version in easy-install.pth

        Installed /home/travis/build/pantsbuild/pants/build-support/pants_deps.venv/lib/python2.7/site-packages/twitter.common.dirutil-0.1.3-py2.7.egg
        Processing dependencies for twitter.common.dirutil==0.1.3
        Traceback (most recent call last):
          File "<string>", line 1, in <module>
          File "/home/travis/build/pantsbuild/pants/build-support/pants_deps.venv/build/twitter.common.dirutil/setup.py", line 22, in <module>
            'zip_safe': True}
          File "/usr/lib/python2.7/distutils/core.py", line 152, in setup
            dist.run_commands()
          File "/usr/lib/python2.7/distutils/dist.py", line 953, in run_commands
            self.run_command(cmd)
          File "/usr/lib/python2.7/distutils/dist.py", line 972, in run_command
            cmd_obj.run()
          File "/home/travis/build/pantsbuild/pants/build-support/pants_deps.venv/local/lib/python2.7/site-packages/setuptools/command/install.py", line 74, in run
            self.do_egg_install()
          File "/home/travis/build/pantsbuild/pants/build-support/pants_deps.venv/local/lib/python2.7/site-packages/setuptools/command/install.py", line 97, in do_egg_install
            cmd.run()
          File "/home/travis/build/pantsbuild/pants/build-support/pants_deps.venv/local/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 358, in run
            self.easy_install(spec, not self.no_deps)
          File "/home/travis/build/pantsbuild/pants/build-support/pants_deps.venv/local/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 574, in easy_install
            return self.install_item(None, spec, tmpdir, deps, True)
          File "/home/travis/build/pantsbuild/pants/build-support/pants_deps.venv/local/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 625, in install_item
            self.process_distribution(spec, dist, deps)
          File "/home/travis/build/pantsbuild/pants/build-support/pants_deps.venv/local/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 671, in process_distribution
            [requirement], self.local_index, self.easy_install
          File "/home/travis/build/pantsbuild/pants/build-support/pants_deps.venv/local/lib/python2.7/site-packages/pkg_resources.py", line 593, in resolve
            requirements.extend(dist.requires(req.extras)[::-1])
          File "/home/travis/build/pantsbuild/pants/build-support/pants_deps.venv/local/lib/python2.7/site-packages/pkg_resources.py", line 2251, in requires
            dm = self._dep_map
          File "/home/travis/build/pantsbuild/pants/build-support/pants_deps.venv/local/lib/python2.7/site-packages/pkg_resources.py", line 2237, in _dep_map
            for extra,reqs in split_sections(self._get_metadata(name)):
          File "/home/travis/build/pantsbuild/pants/build-support/pants_deps.venv/local/lib/python2.7/site-packages/pkg_resources.py", line 2673, in split_sections
            for line in yield_lines(s):
          File "/home/travis/build/pantsbuild/pants/build-support/pants_deps.venv/local/lib/python2.7/site-packages/pkg_resources.py", line 1949, in yield_lines
            for ss in strs:
          File "/home/travis/build/pantsbuild/pants/build-support/pants_deps.venv/local/lib/python2.7/site-packages/pkg_resources.py", line 2265, in _get_metadata
            for line in self.get_metadata_lines(name):
          File "/home/travis/build/pantsbuild/pants/build-support/pants_deps.venv/local/lib/python2.7/site-packages/pkg_resources.py", line 1329, in get_metadata_lines
            return yield_lines(self.get_metadata(name))
          File "/home/travis/build/pantsbuild/pants/build-support/pants_deps.venv/local/lib/python2.7/site-packages/pkg_resources.py", line 1321, in get_metadata
            return self._get(self._fn(self.egg_info,name))
          File "/home/travis/build/pantsbuild/pants/build-support/pants_deps.venv/local/lib/python2.7/site-packages/pkg_resources.py", line 1385, in _get
            return self.loader.get_data(path)
        zipimport.ZipImportError: bad local file header in /home/travis/build/pantsbuild/pants/build-support/pants_deps.venv/lib/python2.7/site-packages/twitter.common.dirutil-0.1.3-py2.7.egg

 pants | 35 +++++++++++++++++++++++++----------
 1 file changed, 25 insertions(+), 10 deletions(-)

Bugs closed: 81

Reviewed at https://rbcommons.com/s/twitter/r/272/
Added a maximum version to Distribution().

I encountered an incompatibility when part of the build (zinc) picked
up java 1.8 in my path instead of java 1.7. This change by itself
does not fix the issue, but lays some foundation to help pick a
distribution less than a maximum rev.

Signed-off-by: John Sirois <jsirois@twitter.com>
Use cache when chroot-dumping deps with custom repos.

This makes running ./pants tests/python/pants_test:all faster.

Before: 4m30s. After: 2m30s.
Added a maximum version to Distribution().

I encountered an incompatibility when part of the build (zinc) picked
up java 1.8 in my path instead of java 1.7. This change by itself
does not fix the issue, but lays some foundation to help pick a
distribution less than a maximum rev.

Signed-off-by: John Sirois <jsirois@twitter.com>
Remove ability to configure workdir locations.

There's no need to be able to override the location of task workdirs,
and this allows us to be rid of a bunch of flags and pants.ini keys.

After this change we compute a unique workdir per task instance, based
on the phase/goal it's registered under, and plumb that through
the task's __init__(). That did require changing __init__'s signature
on all tasks, but it made for more straightforward code.

While at it, I also fixed a bunch of superclass __init__ invocations
to use super().

This change affects many files (every task at least has its
__init__() signature changed) but the changes are mostly tiny.
Get rid of some configurable outdirs.

In these tasks it makes sense to hardwire the outdir to
be the workdir. Allowing it to be customized via flags seems
like unnecessary overkill, especially when it comes to intermediate
build products like generated code.
Upgrade jmake to 1.3.8-1.

commit c9e81c08db5be5074009d9414206f355d9a90017
Author: John Sirois <jsirois@twitter.com>
Date:   Mon Apr 28 10:35:56 2014 -0600

    Upgrade jmake to 1.3.8-1.

    Also fixes old twitter-commons cache dir name.

 BUILD.tools                       | 2 +-
 build-support/ivy/ivysettings.xml | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

Testing Done:
$ ./pants goal test tests/{java,scala}::
...
10:35:03 00:12           [jmake]
                         warning: bootstrap class path not set in conjunction with -source 1.6
                         Jmake version 1.3.8-1
                         Writing project database...  Done.

10:35:05 00:14         [find-missing-dependencies]
...
10:35:15 00:24   [test]
10:35:15 00:24     [junit]
10:35:15 00:24       [bootstrap-junit]
10:35:16 00:25       [run]
                     .......
                     Time: 0.165

                     OK (7 tests)

10:35:17 00:26     [specs]
               Waiting for background workers to finish.
               SUCCESS

Bugs closed: 85

Reviewed at https://rbcommons.com/s/twitter/r/282/
Get rid of the remaining 'outdir' flags.
Get rid of conditional imports of markdown and pygments.
Ensure that zinc analysis from cache is correctly merged.

This is a probable fix for the missing-class-at-runtime bug.
Update "Pants Development with IntelliJ IDEA".

commit 1a6f730aa05b7135116e06b0a0faea40d4390d6b
Author: John Sirois <jsirois@twitter.com>
Date:   Tue Apr 29 00:21:31 2014 -0600

    Update "Pants Development with IntelliJ IDEA".

    This updates the docs to work with the new setup
    in pantsbuild/pants as well as refreshing the example
    verbage and screen shots to reflect current IntelliJ.

 src/python/pants/docs/images/intellij-configure-tests.png                        | Bin 0 -> 56869 bytes
 src/python/pants/docs/images/intellij-new-project-1.png                          | Bin 217258 -> 54734 bytes
 src/python/pants/docs/images/intellij-new-project-2.png                          | Bin 133244 -> 24893 bytes
 src/python/pants/docs/images/intellij-new-pythonsdk.png                          | Bin 0 -> 62720 bytes
 src/python/pants/docs/images/intellij-project-structure-modules-dependencies.png | Bin 92174 -> 0 bytes
 src/python/pants/docs/images/intellij-project-structure-modules-sources.png      | Bin 107848 -> 49187 bytes
 src/python/pants/docs/images/intellij-project-structure-project.png              | Bin 109356 -> 53495 bytes
 src/python/pants/docs/images/intellij-run.png                                    | Bin 110186 -> 55478 bytes
 src/python/pants/docs/images/intellij-select-venv.png                            | Bin 0 -> 60848 bytes
 src/python/pants/docs/intellij.rst                                               |  60 ++++++++++++++++++++++++++++----------------------
 10 files changed, 34 insertions(+), 26 deletions(-)

Testing Done:
See the test site here: http://pantsbuild.github.io/sirois-pants-ij-test/intellij.html

Note that I had to work the doc publish process a bit - but that is not yet ready for review - I'll keep it seperate.  It gets easier just like IDEA setup does with a venv for the egg deps in conf.py.

Bugs closed: 88

Reviewed at https://rbcommons.com/s/twitter/r/290/
Added protobuf sample

Bugs closed: 91

Reviewed at https://rbcommons.com/s/twitter/r/291/
Debug log the 404s we get back from the restful artifact cache. Simplifies debugging what parts we didn't find in the cache
Invalidate cache if the Java target version or Scala compiler version changes.

Bugs closed: 76

Reviewed at https://rbcommons.com/s/twitter/r/277/
Use requests instead of httplib for restful_artifact_cache.py. Preparation for enabling auth

Reviewed at https://rbcommons.com/s/twitter/r/294
Reimplement pants help.

Previously there were several different help paths,
help messages weren't uniform and in some cases were messy.

This change:

- Gets rid of the 'help' command.
- Gets rid of the 'help' goal.
- Has a single entry point to help message printing.
- Cleans up help messages. For example, we no longer print
  the list of goals unless you explicitly ask for it,
  because that list is long now.

Note that this gets rid of help for non-goal commands, but
those are going away soon anyway.

This change is in preparation for getting us off optparse and
onto our own custom cmd-line parser. We no longer use optparse's
help messages (but we do still use optparse as an implementation
detail for formatting flag help, for now).