Added source root tests, updated the internal data structure to include a tree.

Added source root tests, updated the internal data structure to include a tree to speed up the find() operation.

With 1000+ source roots in our environment, this appears to shave about 5 seconds off of a no-op build.

Testing Done:
Retrofitted tests for SourceRoot.
Added new unit tests for SourceRootTree data structure

CI passed at: https://travis-ci.org/pantsbuild/pants/builds/34713735

Bugs closed: 556

Reviewed at https://rbcommons.com/s/twitter/r/1003/
Add flag to dump project info output to file.

This file option will be used in intellij plugin.

Testing Done:
https://travis-ci.org/pantsbuild/pants/builds/34542731

This is OS Pants review for https://github.com/pantsbuild/intellij-pants-plugin/issues/5

Bugs closed: https://github.com/pantsbuild/intellij-pants-plugin/issues/5

Reviewed at https://rbcommons.com/s/twitter/r/998/
doc: update 3rdparty_py to mention requirements.txt and python_requirements

python_requirement() is nice, too, but it's not your only choice

Reviewed at https://rbcommons.com/s/twitter/r/995/
fixed --depmap-project-info for scala sources with the same package_prefix

the problem was that we were using same current_target  for all collected source roots. not the proper target this root is part of.

Testing Done:
tests are passing

Reviewed at https://rbcommons.com/s/twitter/r/990/
update release.sh and release.rst to include pantsbuild.pants.testinfra

Moving on, we'll publish two packages at the same time: pantsbuild.pants
and pantsbuild.pants.testinfra to PyPi. While they're sim-released, users
of pants can choose to install pants only if they don't have any
customization to pants thus avoiding the need of writing additional test
cases to which the pantsbuild.pants.testinfra would be helpful.

Testing Done:
I'm not the owner of pantsbuild.pants package on PyPi, so all my testing are local:

build-support/bin/release.sh -t
build-support/bin/release.sh -n

tested both. For the '-n' dry-run, I've also manually injected some failures at pants, pants_testinfra build stages, as well some failures in their testing commands, to have verified the proper command chaining behavior of the bash script.

Bugs closed: 539

Reviewed at https://rbcommons.com/s/twitter/r/1004/
Adds the --<goal>-use-source-root for IDE project generation tasks.

The option will put one entry for each source_root registered that has an active path in the project.
This is useful for Maven style layouts.

Testing Done:
Added unit test and integration test.

Manually tested with:

  pants goal idea testprojects/maven_layout:: --idea-use-source-root

CI is running.

Bugs closed: 558

Reviewed at https://rbcommons.com/s/twitter/r/1006/
Skipping a doc phase should result in transitive deps being skipped as well.

When `--doc-javadoc-skip` or `--doc-scaladoc-skip` are passed to a transitive publish, products generated by those goals will not be available, and should not be depended on in the Ivy javadoc configuration that JarPublish generates. Otherwise, Ivy will fail to resolve the transitive `javadoc` configuration during publishing.

* Expand existing targets to test transitive publishing of java/scala
* Simplify to-string for jar_coordinate of pushdb entries
* Don`t request docs unless they are available for all transitive targets

Testing Done:
new test, and manual publishing with --doc-scaladoc-skip and --doc-javadoc-skip

Reviewed at https://rbcommons.com/s/twitter/r/1011/
depmap KeyError fix

Testing Done:
ci

Reviewed at https://rbcommons.com/s/twitter/r/1015/
update and narrow down our "setting up Pants is hard" message

pantsbuild.pants is is a thing. So stop saying it's not installable.

We had commented out stuff about how to install it. Un-comment it.

For a venv-y example, see twitter/commons; not pants' repo, that was only true for a brief time.

BUT, setting up Pants in your code workspace still requires a PhD in Pants-ology. So shift our scary warning over to the setup_repo page.

Reviewed at https://rbcommons.com/s/twitter/r/1007/
Fixup test deps

These were revealed broken by running in non-fast mode.

Testing Done:
$ rm -rf pants.pex dist/ && ./pants build tests/python/pants_test:all

CI is went green here: https://travis-ci.org/pantsbuild/pants/builds/34972075

Bugs closed: 369, 569

Reviewed at https://rbcommons.com/s/twitter/r/1016/
Changes the behavior of goal idea to create a subdirectory named for the project name.

IntelliJ will fail to load the first time if you change the name and try to write the project into
the same directory as an existing project.

Also, enhanced the integration check to examine the contents of the project file when launching "hello"

Testing Done:
Manually confirmed the error "Cannot load module file '/Users/zundel/Development/java/squarepants/idea/foo4568.iml':
File /Users/zundel/Development/java/squarepants/idea/foo4568.iml does not exist
Would you like to remove module 'foo4568' from the project?"

does not occur any more when you change the project name.

Bugs closed: 564, 571

Reviewed at https://rbcommons.com/s/twitter/r/1019/
doc: top page linked to 404/broken pages. so remove those links

Index had links to BUILD dictionary entries, but nothing else.
Module Index is 404.
Search page is empty

Reviewed at https://rbcommons.com/s/twitter/r/1020/
Added unit tests for registering a source root path with no targets

Testing Done:
Added a unit test requested by Patrick.   Passed CI @ https://travis-ci.org/pantsbuild/pants/builds/35208443

Bugs closed: 573

Reviewed at https://rbcommons.com/s/twitter/r/1027/
'goal builddict' Target classes "inherit" docstrings, signatures

Nowadays, our BUILD target implementations use class inheritance + some constructor idioms. But the BUILD Dictionary hasn't kept up. Thus, we have boilerplate docstrings and documented signatures like android_binary(build_type=None, *args, **kwargs)

This change...

Special-cases target handling in src/python/pants/backend/core/tasks/builddictionary.py to go up the inheritence tree, pull apart and re-assemble signatures and docstrings. Also, special-cases leaving out implementation-params we don't want in the user-facing doc (address, build_graph, payload). It doesn't have a general mechanism for leaving stuff out, though. E.g., python_binary's entry thinks it has a sources param (inherited from PythonTarget); mitigated by having a docstring for it saying "overridden: use source instead".

Removes many many redundant docstring fragments. My search wasn't scientific. Probably other removal-worthy pieces lurk.

Adds some docstrings. E.g. python_binary's "sources: overridden". Move some docstrings to superclass for easier inheriting.

Testing Done:
https://travis-ci.org/pantsbuild/pants/builds/35162328

Reviewed at https://rbcommons.com/s/twitter/r/1018/
Changed publish integration tests to use testprojects/*
Copied the examples required by the test to testprojects so they can be changed independently of examples without making those more convoluted.

Testing Done:
ci.sh

Reviewed at https://rbcommons.com/s/twitter/r/1026/
One publish/push db file per artifact

In order to reduce contention around publish/push db files this change splits up the repo wide db into one per artifact.

Reviewed at https://rbcommons.com/s/twitter/r/994/
Make a better error message when os.symlink fails during bundle.

Reviewed at https://rbcommons.com/s/twitter/r/1037/
Make SourceRoot fundamentally understand a rel_path.

Add a factory for constructing a SourceRoot from a ParseContext
as a secondary means of creation and wire this up to core's
regiaster.py.  Simplify a test as a result.

 src/python/pants/backend/core/register.py        |  2 +-
 src/python/pants/base/source_root.py             | 10 ++++++++--
 tests/python/pants_test/base/test_source_root.py |  6 +-----
 3 files changed, 10 insertions(+), 8 deletions(-)

Testing Done:
CI went green here: https://travis-ci.org/pantsbuild/pants/builds/35379833

Bugs closed: 575

Reviewed at https://rbcommons.com/s/twitter/r/1036/
Added --idea-exclude-maven-target to get exclude IntelliJ from indexing 'target' directories. If you are running maven and pants side by side, these 'target' files contain duplicates of a lot resources and other cruft you don't want to see when you search.

Testing Done:
Manual testing in the square/java repo.

Bugs closed: 570

Reviewed at https://rbcommons.com/s/twitter/r/1017/
 Added 'exclude-folders' option in pants.ini, defaulted to excluding a few dirs in .pants.d

 This is a followon to https://rbcommons.com/s/twitter/r/1017 that gets rid of the fixed list of includes in the .iml mustache template
 and allows you to override it.

Testing Done:
Updated the integration tests and inspected manually.

Reviewed at https://rbcommons.com/s/twitter/r/1023/