doc python_req things

give python_requirements_library a docstring

tag python_requirements with @manual so it shows up in BUILD dictionary

add example to python_requirements with a "this gives you something like..."

Bugs closed: 292

Reviewed at https://rbcommons.com/s/twitter/r/629/
Upgrade default setuptools to 5.4.1

Testing Done:
CI passed.  Looking at https://pypi.python.org/pypi/setuptools , there appear to be no expected breaking changes between our current setuptools (5.0) and this one.  We want this upgrade because of the bugfix in 5.1, and the other bug fixes since then seem good to pull in.

Reviewed at https://rbcommons.com/s/twitter/r/674/
Patch around unicode dup detection failure.

I'm not super happy with this solution and I think it deserves a closer look, but since the thread for this fizzled out, this is the patch we've been running internally to unblock ourselves.  Thread for reference: https://groups.google.com/forum/#!searchin/pants-devel/unicode$20decode/pants-devel/tx4NHhxX1fI/VRP0fzzG4C4J

Testing Done:
CI passed

Reviewed at https://rbcommons.com/s/twitter/r/615/
Add manifest parser to AndroidTarget.

This removes the need to enter target_sdk and package variables
for every BUILD file in the package. It was always on the TODO,
but as I am prepping the codegen tasks, it seemed time. This removes
several params from the targets, docstrings are updated accordingly.

Also added BadManifestError to catch malformed or incomplete manifests.
These errors would eventually cause problems in the compilation
process, but since we are parsing the manifest here, uncaught
exceptions would appear to be pants-related problems, as opposed
to an issue with the user's Android project. So I added an exception
to catch the issue.

Testing Done:
ci.sh passed locally.
Travis passed [0].

There will be some further integration tests of this functionality when I push up a
separate patch this weekend comprised of the aapt_gen class and its test coverage.

[0] https://travis-ci.org/pantsbuild/pants/builds/29720531

Reviewed at: https://rbcommons.com/s/twitter/r/668/
Fix relativity of antlr sources

This fixes an issue where CacheValidator's source hashing was failing on
the synthetic targets generated by AntlrGen, since the assumed source
paths mismatched the actual generated paths.

- Adds an integration test for antlr4 codegen
- Uses sources_relative_to_source_root for the assumed output sources,
  and fixes the synthetic source root for the antlr output. We can't
  control antlr's output relative paths very tightly, so we can't use
  the output directory directly as a sourceroot.
Lock globs into rootdir and below

This avoids the situation where a user adds a sources=globs('../*.scala'), causing it to be silently ignored but causes issues later.

Reviewed here: https://rbcommons.com/s/twitter/r/686/
Closes: https://github.com/pantsbuild/pants/issues/348
Pluralize product_type of Task to product_types
doc: Python doc, don't use twitter/commons as example

Back when Pants' source code lived in twitter/commons, it made sense to use example code from there. Nowadays, not so much. Use Pants' own src/python/example instead.

Instead of having sample code in the doc, "include" from src/python/example; that's easier to keep correct.

Testing Done:
build-support/bin/publish_docs.sh

Reviewed at https://rbcommons.com/s/twitter/r/654/
Stop pointing twitter/commons users at the "old python" docs

Reviewed at https://rbcommons.com/s/twitter/r/693/
make build_configuration test happy on OSX, not just linux

"Looks like we need to toss in a realpath somewhere in these tests" was true

Also a couple of missing deps reared their head when I ran this test on its own

Testing Done:
./pants goal test tests/python/pants_test/base:build_configuration

./build-support/bin/ci.sh # on my mac

Linux still happy
https://travis-ci.org/pantsbuild/pants/builds/30092011

Reviewed at https://rbcommons.com/s/twitter/r/694/
Updated junit-runner to 0.0.41 to be able to use guava version > 15 exclude guava from junit-runner in BUILD.tools explicitly to show where you   need to update the guava version in case you want to use 17 or higher

Testing Done:
I've been using this in our repo to test.

PANTS_DEV=1 ./pants goal test tests/java::

Reviewed at https://rbcommons.com/s/twitter/r/691/
Cleanups to recent patch to pants goal depmap

Incorporated feedback from Patrick after submission of https://rbcommons.com/s/twitter/r/661/

Remove is_delegate attribute from AbstractTarget
Remove stray debug print message in a test

Testing Done:
ci passed at https://travis-ci.org/pantsbuild/pants/builds/30079790

Also ran pants goal depmap by hand on a dependencies() target in our repo

Bugs closed: 359

Reviewed at https://rbcommons.com/s/twitter/r/692/
Update the release docs and unify the pants semver.

This centralizes the pants version editing to
src/python/pants/version.py and firms up a manual release
process that will be dogfooded after this commit.

Outstanding work includes automation of release testing
and a scheme for editing & publishing a CHANGELOG.

Testing Done:
Ran the release test command block and that works - tail lines included in the docs itself.
Also burned a test site, see: http://pantsbuild.github.io/jsirois/release.html

Bugs closed: 357

Reviewed at https://rbcommons.com/s/twitter/r/690/
doc: committing someone else's change: not always straightforward

'./rbt patch -c 123' doesn't always work, so don't suggest
it so prominently

Testing Done:
Trying out the steps on this very commit.

Reviewed at https://rbcommons.com/s/twitter/r/699
Author: Dan Davydov <davydovdan@gmail.com>
Date:   Fri Jul 18 10:39:00 2014 -0700

    Enable Nailgun Per Task

    For discussion around this patch please see: https://groups.google.com/forum/#!searchin/pants-devel/nailgun/pants-devel/wcvYfUvhPeg/uyFfy6_IqPEJ
    Note for antlr3 / antlr4 I made their ng-killall option under a new more generic [antlr] config section. Also note that any classes that extend NailgunTask must supply
    the config_section constructor parameter (otherwise when nailgun is attempted to be used it will fail), this is intentional
    (though this would obviously be better do to during compile time rather than run time).

    Also did a bit of style/duplicate code refactoring.

    Testing Done:
    Added print statements to nailgun/non-nailgun execution paths, and tested different combinations of pants configs:
    -No use_nailgun flags (should always use nailgun)
    -use_nailgun: False in [DEFAULT] (should never use nailgun)
    -use_nailgun: False in [DEFAULT] and use_nailgun: True in [scala-compile] (nailgun was only used for scala)

    Also
    ./build-support/ci.sh

    Reviewed at https://rbcommons.com/s/twitter/r/687/
Fix pants test libraries missing dependencies

Currently on master if you don't have PANTS_DEV=1, pants_test:all target fails.

Fixed following python test targets:
1. tests/python/pants_test/test_utf8_header.py F due to
BackendConfigurationError: Failed to load the pants.backend.core.register backend: No module named goal

2.tests/python/pants_test/base/test_build_configuration.py:16: in <module>
    from pants.base.build_configuration import BuildConfiguration
/var/folders/t_/ck6c3zvj2z5cg77dqx2kh8tr0000gp/T/tmpy8_YJ_/pants/base/build_configuration.py:12: in <module>
    from pants.base.build_file_aliases import BuildFileAliases
E   ImportError: No module named build_file_aliases

3. tests/python/pants_test/java/test_executor.py:18: in <module>
    from pants.java.executor import SubprocessExecutor
/var/folders/t_/ck6c3zvj2z5cg77dqx2kh8tr0000gp/T/tmp3lxmyq/pants/java/executor.py:18: in <module>
    from pants.base.build_environment import get_buildroot
E   ImportError: No module named build_environment

4. >   ???
E   ImportError: No module named authentication.netrc_util

Testing Done:
CI passes as usual.
https://travis-ci.org/pantsbuild/pants/builds/30308887

On my branch:- Following command succeeds.
 rm pants.pex && ./pants tests/python/pants_test:all

Bugs closed: 355

Reviewed at https://rbcommons.com/s/twitter/r/685/