Add manifest parser to AndroidTarget.

This removes the need to enter target_sdk and package variables
for every BUILD file in the package. It was always on the TODO,
but as I am prepping the codegen tasks, it seemed time. This removes
several params from the targets, docstrings are updated accordingly.

Also added BadManifestError to catch malformed or incomplete manifests.
These errors would eventually cause problems in the compilation
process, but since we are parsing the manifest here, uncaught
exceptions would appear to be pants-related problems, as opposed
to an issue with the user's Android project. So I added an exception
to catch the issue.

Testing Done:
ci.sh passed locally.
Travis passed [0].

There will be some further integration tests of this functionality when I push up a
separate patch this weekend comprised of the aapt_gen class and its test coverage.

[0] https://travis-ci.org/pantsbuild/pants/builds/29720531

Reviewed at: https://rbcommons.com/s/twitter/r/668/
Fix relativity of antlr sources

This fixes an issue where CacheValidator's source hashing was failing on
the synthetic targets generated by AntlrGen, since the assumed source
paths mismatched the actual generated paths.

- Adds an integration test for antlr4 codegen
- Uses sources_relative_to_source_root for the assumed output sources,
  and fixes the synthetic source root for the antlr output. We can't
  control antlr's output relative paths very tightly, so we can't use
  the output directory directly as a sourceroot.
Lock globs into rootdir and below

This avoids the situation where a user adds a sources=globs('../*.scala'), causing it to be silently ignored but causes issues later.

Reviewed here: https://rbcommons.com/s/twitter/r/686/
Closes: https://github.com/pantsbuild/pants/issues/348
Pluralize product_type of Task to product_types
doc: Python doc, don't use twitter/commons as example

Back when Pants' source code lived in twitter/commons, it made sense to use example code from there. Nowadays, not so much. Use Pants' own src/python/example instead.

Instead of having sample code in the doc, "include" from src/python/example; that's easier to keep correct.

Testing Done:
build-support/bin/publish_docs.sh

Reviewed at https://rbcommons.com/s/twitter/r/654/
Stop pointing twitter/commons users at the "old python" docs

Reviewed at https://rbcommons.com/s/twitter/r/693/
make build_configuration test happy on OSX, not just linux

"Looks like we need to toss in a realpath somewhere in these tests" was true

Also a couple of missing deps reared their head when I ran this test on its own

Testing Done:
./pants goal test tests/python/pants_test/base:build_configuration

./build-support/bin/ci.sh # on my mac

Linux still happy
https://travis-ci.org/pantsbuild/pants/builds/30092011

Reviewed at https://rbcommons.com/s/twitter/r/694/
Updated junit-runner to 0.0.41 to be able to use guava version > 15 exclude guava from junit-runner in BUILD.tools explicitly to show where you   need to update the guava version in case you want to use 17 or higher

Testing Done:
I've been using this in our repo to test.

PANTS_DEV=1 ./pants goal test tests/java::

Reviewed at https://rbcommons.com/s/twitter/r/691/
Cleanups to recent patch to pants goal depmap

Incorporated feedback from Patrick after submission of https://rbcommons.com/s/twitter/r/661/

Remove is_delegate attribute from AbstractTarget
Remove stray debug print message in a test

Testing Done:
ci passed at https://travis-ci.org/pantsbuild/pants/builds/30079790

Also ran pants goal depmap by hand on a dependencies() target in our repo

Bugs closed: 359

Reviewed at https://rbcommons.com/s/twitter/r/692/
Update the release docs and unify the pants semver.

This centralizes the pants version editing to
src/python/pants/version.py and firms up a manual release
process that will be dogfooded after this commit.

Outstanding work includes automation of release testing
and a scheme for editing & publishing a CHANGELOG.

Testing Done:
Ran the release test command block and that works - tail lines included in the docs itself.
Also burned a test site, see: http://pantsbuild.github.io/jsirois/release.html

Bugs closed: 357

Reviewed at https://rbcommons.com/s/twitter/r/690/
doc: committing someone else's change: not always straightforward

'./rbt patch -c 123' doesn't always work, so don't suggest
it so prominently

Testing Done:
Trying out the steps on this very commit.

Reviewed at https://rbcommons.com/s/twitter/r/699
Author: Dan Davydov <davydovdan@gmail.com>
Date:   Fri Jul 18 10:39:00 2014 -0700

    Enable Nailgun Per Task

    For discussion around this patch please see: https://groups.google.com/forum/#!searchin/pants-devel/nailgun/pants-devel/wcvYfUvhPeg/uyFfy6_IqPEJ
    Note for antlr3 / antlr4 I made their ng-killall option under a new more generic [antlr] config section. Also note that any classes that extend NailgunTask must supply
    the config_section constructor parameter (otherwise when nailgun is attempted to be used it will fail), this is intentional
    (though this would obviously be better do to during compile time rather than run time).

    Also did a bit of style/duplicate code refactoring.

    Testing Done:
    Added print statements to nailgun/non-nailgun execution paths, and tested different combinations of pants configs:
    -No use_nailgun flags (should always use nailgun)
    -use_nailgun: False in [DEFAULT] (should never use nailgun)
    -use_nailgun: False in [DEFAULT] and use_nailgun: True in [scala-compile] (nailgun was only used for scala)

    Also
    ./build-support/ci.sh

    Reviewed at https://rbcommons.com/s/twitter/r/687/
Fix pants test libraries missing dependencies

Currently on master if you don't have PANTS_DEV=1, pants_test:all target fails.

Fixed following python test targets:
1. tests/python/pants_test/test_utf8_header.py F due to
BackendConfigurationError: Failed to load the pants.backend.core.register backend: No module named goal

2.tests/python/pants_test/base/test_build_configuration.py:16: in <module>
    from pants.base.build_configuration import BuildConfiguration
/var/folders/t_/ck6c3zvj2z5cg77dqx2kh8tr0000gp/T/tmpy8_YJ_/pants/base/build_configuration.py:12: in <module>
    from pants.base.build_file_aliases import BuildFileAliases
E   ImportError: No module named build_file_aliases

3. tests/python/pants_test/java/test_executor.py:18: in <module>
    from pants.java.executor import SubprocessExecutor
/var/folders/t_/ck6c3zvj2z5cg77dqx2kh8tr0000gp/T/tmp3lxmyq/pants/java/executor.py:18: in <module>
    from pants.base.build_environment import get_buildroot
E   ImportError: No module named build_environment

4. >   ???
E   ImportError: No module named authentication.netrc_util

Testing Done:
CI passes as usual.
https://travis-ci.org/pantsbuild/pants/builds/30308887

On my branch:- Following command succeeds.
 rm pants.pex && ./pants tests/python/pants_test:all

Bugs closed: 355

Reviewed at https://rbcommons.com/s/twitter/r/685/
rm doc-generation code that made mostly-unused docs

This code worked hard to generate some documentation pages.
The Google analytics numbers point out: these pages were largely unread.
Some (but not all) of this code broke in a refactor: it generated empty docs.
Yet nobody complained.

Rather than work hard to fix and maintain this code, I'm ripping it out.
Behold the fiery debate: https://groups.google.com/forum/#!topic/pants-devel/f7V4_6kLdZs

Testing Done:
https://travis-ci.org/pantsbuild/pants/builds/30318159

Reviewed at https://rbcommons.com/s/twitter/r/704/
This is a fix for issue #367

Objects of class BinaryNotFound can not be instantiated because the constructor does not qualify the class name with its parent when it invokes the constructor of the superclass.

Testing Done:
https://travis-ci.org/pantsbuild/pants/builds/30476122

Reviewed at https://rbcommons.com/s/twitter/r/705/
Resurrected pants goal idea.

This mostly involved updating outdated references, and making utility functions to mimick functionality refactored long ago.

Source and Javadoc jars are not mapped in, and I'm not quite sure how to get that working again. I'd appreciate any input on that matter.

The code in idea_gen and ide_gen isn't very clean, because it wasn't very clean in the first place. It would be good to go back at some point and tidy it up.

Idea gen is also much more efficient than it used to be, thanks to the round manager: it now avoids compiling or jarring anything, which saves tons of time. It would still be nice to include a flag to optionally disable codegen, but I'm not sure how (un)trivial that is at this juncture.

Testing Done:
Added (passing) integration tests under tasks/test_idea_integration.py
Tests under tasks:integration pass.

Bugs closed: 349

Reviewed at https://rbcommons.com/s/twitter/r/695/
Upgrade to twitter.common.python 0.5.10.

This brings in the following bugfix:

"Update the mtime on retranslation of existing distributions.

1bff97e stopped existing distributions from being overwritten, to
prevent subtle errors. However without updating the mtime these
distributions will appear to be permanently expired wrt the ttl."
Made a small change to stop problems we were getting compiling our annotation processors.

This is related to the pants-devel discussion 'Nondeterministic Phase Execution Order'. Briefly:
when the resources phase happens to precede the compile phase, resources in .pants.d  pollute the
jmake classpath when compiling some of Square's internal annotation processors. This makes jmake
quite irate, so this code stops that from happening.

This fix is not terribly elegant, in that it doesn't address the underlying problem of resources
getting on the classpath in the first place. Perhaps this problem will go away when a formal
classpath object is created.

Testing Done:
Code which didn't compile now compiles.

Ran test tests/python/pants_test:: and nothing fails that doesn't also fail on the master branch.

Bugs closed: 373

Reviewed at https://rbcommons.com/s/twitter/r/708/
Use resources from junit_tests, in addition to from jvm_binary

https://rbcommons.com/s/twitter/r/711/
Fix transitive dependencies for GroupIterator/GroupTask

A Java target A has a transitive dep on a Scala target C via an
intermediate JarLibrary/Dependencies target B (ie, "un-flavored" from
GroupTask's perspective.) GroupIterator's call to sort_targets correctly
includes target B, which results in a topsort of 'C <- B <- A'. But
coalesce_targets drops target B, because it doesn't match the
discriminator ("is unflavored".) Because target B is ignored and
coalesce only looks at direct deps, C is allowed to be reordered before
A.

- Must not reorder past an intermediate (dropped) target in
  GroupIterator
  - Allow for more subclasses of GroupTaskTest
  - Add reproduction unit test

https://rbcommons.com/s/twitter/r/706/