update doco; goal() is obsolete

Reviewed at https://rbcommons.com/s/twitter/r/931/
Moved CucumberTest into testprojects

Testing Done:
ci passed.

Bugs closed: 532

Reviewed at https://rbcommons.com/s/twitter/r/948/
Re-enable filedeps - it was using a now-gone .expand_files() API.

This maintains a hack with special knowledge of the JvmApp target and
adds one more to deal with ScalaLibrary.java_sources.

Both of these hacks have unit-test coverage to help back up future
refactors away from these hacks.

Testing Done:
$ PANTS_DEV=1 ./pants goal test tests/python/pants_test/:all

CI went green here: https://travis-ci.org/pantsbuild/pants/builds/33652015

Bugs closed: 437, 522

Reviewed at https://rbcommons.com/s/twitter/r/939/
docs: add pages w/not much content (yet), howto_plugin, howto_customize

howto_plugin: how to write a Pants plugin

howto_customize: how to write a Pants customized for your org

So far, neither of these pages has much content. But I hope we'll
be better about capturing some of this info if we have a place to put it.

Reviewed at https://rbcommons.com/s/twitter/r/938/
Fix sloppy commit from https://rbcommons.com/s/twitter/r/901/

Testing Done:
Did a test publish from a Twitter monorepo.

Reviewed at https://rbcommons.com/s/twitter/r/940/
bad troubleshooting instructions: kicking 'em when they're down :-(
But that's what we're doing:

./pants -ldebug goal goals
Invalid command: -ldebug

Reviewed at https://rbcommons.com/s/twitter/r/953/
Refactor exceptions in build_file.py and build_file_parser.py to derive   from a common baseclass and eliminate throwing IOError. Catch errors from parse errors in the BUILD file and format   context around the exception.

Followon to https://rbcommons.com/s/twitter/r/929/

Refactor exceptions in build_file.py and build_file_parser.py to derive from a common baseclass and eliminate throwing IOError.

Revive error message that prints out alternatives to an incorrect build target.

Also, Catch exceptions from parse errors in the BUILD file and print some formatted context around the line of the file that failed.

Exception message: Error parsing /Users/zundel/Src/Pants/examples/src/java/com/pants/examples/hello/greet/BUILD:
    4: java_library(name = 'greet',
    5:   dependencies = [], # A more realistic example would depend on other libs,
    6:                      # but this "hello world" is pretty simple.
*   7:   sources = globs('*.java' && foo),
                                   ^ SyntaxError: invalid syntax (<string>, line 7)

    8:   provides = artifact(org='com.pants.examples',
    9:                     name='hello-greet',
   10:                     repo='build-support/ivy:public',),
   11: )

Testing Done:
ci passed at https://travis-ci.org/pantsbuild/pants/builds/33756154

Bugs closed: 537

Reviewed at https://rbcommons.com/s/twitter/r/954/
Add tests for junit_run._JUnitRunner; fix a regrsssion in Emma/Cobertura; refactor exclusive_groups prep to base_test.

- Add test for junit_run._JUnitRunner class (specially testing against future regression in https://github.com/pantsbuild/pants/issues/508).
- Fix a regression introduced in last fix that impacted Emma and Cobertura (reported by jayeye)
- Refactor populate_exclusive_groups into base_test so tests (test_jvm_run and test_junit_run and others) can reuse it.

Testing Done:
- Travis CI: https://travis-ci.org/jinfeng/jinfeng-pants-fork - CI passed.

- Local run:

$ PANTS_DEV=1 ./pants goal test tests::

============== test session starts ===============
                     platform darwin -- Python 2.6.8 -- py-1.4.23 -- pytest-2.5.2
                     plugins: cov, timeout
                     collected 500 items
                     ...
                     ...
                     tests/python/pants_test/backend/jvm/tasks/test_junit_run.py .
                     ...
                     ...
                     tests/python/pants_test/tasks/test_jvm_run.py .
                     ...
                     ...

                      490 passed, 5 skipped, 5 xfailed in 413.71 seconds

16:45:57 07:37     [junit]
16:45:57 07:37     [specs]
               SUCCESS

Bugs closed: 508

Reviewed at https://rbcommons.com/s/twitter/r/935/
Add IntelliJ content type to project Info. Add missing resources to targets.

This is required for intelliJ pants plugin to correctly mark the source, resource, test target and test resources.

The constants are defined here in IntelliJ
http://grepcode.com/file_/repository.grepcode.com/java/ext/com.jetbrains/intellij-idea/13.0.0/com/intellij/openapi/externalSystem/model/project/ExternalSystemSourceType.java/?v=source

Testing Done:
Locally ./pants tests/python/pants_test/tasks:depmap passes

https://travis-ci.org/pantsbuild/pants/builds/33880605

Reviewed at https://rbcommons.com/s/twitter/r/964/
jar_library docstring += dependencies

eagle-eyed J.I. Ioannidis noticed that our 3rdparty JVM docs show jar_library(dependencies=) but the BUILD dictionary doesn't (at least, I think that's what's going on)

until we fix https://github.com/pantsbuild/pants/issues/142, add doc for this param by hand, like an animal

Reviewed at https://rbcommons.com/s/twitter/r/960/
add include-excerpt feature to 'goal markdown'

Extend markdown syntax so that page authors can "include" a snippet from a real source file. Inspired by the way the Pants docs use Spinx's literalinclude http://sphinx-doc.org/markup/code.html#directive-literalinclude .

Added docs for it in src/python/pants/docs/page.rst

Exercise it in examples/src/java/com/pants/examples/hello/main/README.md

Testing Done:
Added test_markdown_to_html.py with a bunch of tests for the "excerpt a few lines" functionality, which has corner cases.

https://travis-ci.org/pantsbuild/pants/builds/33925158

Reviewed at https://rbcommons.com/s/twitter/r/949/
Move the jvm_compile/java IT from unit to IT.

Noticed this one very slow test when running the unit tests via :all which is not dead yet.

Testing Done:
CI went green here: https://travis-ci.org/pantsbuild/pants/builds/34067665

Bugs closed: 542

Reviewed at https://rbcommons.com/s/twitter/r/970/
Fixup engine test deps.

I was starting to look at coveralls and ran into some missing deps on my chosen victim test target, this fixes the deps up.

Testing Done:
```console
$ PANTS_PY_COVERAGE=1 ./pants goal test tests/python/pants_test/engine
$ PANTS_PY_COVERAGE=1 ./pants goal test tests/python/pants_test/engine:test_engine
$ PANTS_PY_COVERAGE=1 ./pants goal test tests/python/pants_test/engine:test_round_engine
$ COVERALLS_REPO_TOKEN=XYZ coveralls
```

https://coveralls.io/jobs/2668487

CI went green here: https://travis-ci.org/pantsbuild/pants/builds/34040483

Bugs closed: 369, 540

Reviewed at https://rbcommons.com/s/twitter/r/968/
Add a whitelist to jvm dependency analyzer

The whitelist allows big repos to reduce dependency issues while stopping any new from being introduced. You can whitelist the projects that currently have issues and make the dependency checks fatal. All projects with current issues will still be able to build but new issues will result in breaking builds.

Also it seems like the direct dependency check would always fatal if set to fatal, fixed that.

Testing Done:
Added two integration tests.

Reviewed at https://rbcommons.com/s/twitter/r/888/
Fixed source roots in project info for IntelliJ plugin in case of ScalaLibrary with java_sources

Testing Done:
CI is passing.

Reviewed at https://rbcommons.com/s/twitter/r/955/
Fix the --ivy-report option and add integration test

I inadvertently broke this back at https://rbcommons.com/s/twitter/r/941/ by  removing the override from pants.ini
I went back and audited other bootstrap-tools references and made them absolute address specs and made sure they were surrounded in '[]'

Testing Done:
CI passed, invoking with --ivy-report manually from the command line no long throws an exception and creates a report.

Bugs closed: 545

Reviewed at https://rbcommons.com/s/twitter/r/976/
Add Coveralls code coverage badge

https://coveralls.io/r/pantsbuild/pants
predictable output path for 'goal markdown'.

Instead of outputting to .pants.d/something/... , generate to dist/markdown

(A Twitter team was smooshing Task code into a BUILD file to tweak page generation.
This here change aims to generate .html in a predictable place.
So that Twitter team (or maybe I) can scriptily modify that .html _not_ in a BUILD file, sheesh.)

Testing Done:
https://travis-ci.org/pantsbuild/pants/builds/34236236

and added a test

Bugs closed: 535

Reviewed at https://rbcommons.com/s/twitter/r/961/
Remove IntelliJ contants ClassName. Make them more generic.

This review is to patch the comments from https://rbcommons.com/s/twitter/r/964/

Testing Done:
https://travis-ci.org/pantsbuild/pants/builds/34313082

Reviewed at https://rbcommons.com/s/twitter/r/987/
Add support for doing remote publishes with an explicit snapshot name

This patch initially contributed by Ian O`Connell allows for local or remote publishing of "named snapshots". Named snapshots skip semantic versioning, and  record a "named_version" in the pushdb.

Testing Done:
additional publishing smoketest

$ yes | PANTS_DEV=1 ./pants goal publish \
  --publish-local=blah \
  --no-publish-dryrun \
  --publish-force \
  --publish-named-snapshot=436ae2dee0e2fc15a6ccfd2fcec5231b8e127a8d \
  examples/src/java/com/pants/examples/hello/greet
...

$ ls -lR blah
total 0
drwxrwx---  3 stuhood  admin  102 Aug 21 18:20 com

blah/com:
total 0
drwxrwx---  3 stuhood  admin  102 Aug 21 18:20 pants

blah/com/pants:
total 0
drwxrwx---  3 stuhood  admin  102 Aug 21 18:20 examples

blah/com/pants/examples:
total 0
drwxrwx---  3 stuhood  admin  102 Aug 21 18:20 hello-greet

blah/com/pants/examples/hello-greet:
total 0
drwxrwx---  17 stuhood  admin  578 Aug 21 18:20 436ae2dee0e2fc15a6ccfd2fcec5231b8e127a8d

blah/com/pants/examples/hello-greet/436ae2dee0e2fc15a6ccfd2fcec5231b8e127a8d:
total 184
-rw-rw----  1 stuhood  admin  34462 Aug 21 18:20 hello-greet-436ae2dee0e2fc15a6ccfd2fcec5231b8e127a8d-javadoc.jar
-rw-rw----  1 stuhood  admin     32 Aug 21 18:20 hello-greet-436ae2dee0e2fc15a6ccfd2fcec5231b8e127a8d-javadoc.jar.md5
-rw-rw----  1 stuhood  admin     40 Aug 21 18:20 hello-greet-436ae2dee0e2fc15a6ccfd2fcec5231b8e127a8d-javadoc.jar.sha1
-rw-rw----  1 stuhood  admin   1635 Aug 21 18:20 hello-greet-436ae2dee0e2fc15a6ccfd2fcec5231b8e127a8d-sources.jar
-rw-rw----  1 stuhood  admin     32 Aug 21 18:20 hello-greet-436ae2dee0e2fc15a6ccfd2fcec5231b8e127a8d-sources.jar.md5
-rw-rw----  1 stuhood  admin     40 Aug 21 18:20 hello-greet-436ae2dee0e2fc15a6ccfd2fcec5231b8e127a8d-sources.jar.sha1
-rw-rw----  1 stuhood  admin   2046 Aug 21 18:20 hello-greet-436ae2dee0e2fc15a6ccfd2fcec5231b8e127a8d.jar
-rw-rw----  1 stuhood  admin     32 Aug 21 18:20 hello-greet-436ae2dee0e2fc15a6ccfd2fcec5231b8e127a8d.jar.md5
-rw-rw----  1 stuhood  admin     40 Aug 21 18:20 hello-greet-436ae2dee0e2fc15a6ccfd2fcec5231b8e127a8d.jar.sha1
-rw-rw----  1 stuhood  admin    867 Aug 21 18:20 hello-greet-436ae2dee0e2fc15a6ccfd2fcec5231b8e127a8d.pom
-rw-rw----  1 stuhood  admin     32 Aug 21 18:20 hello-greet-436ae2dee0e2fc15a6ccfd2fcec5231b8e127a8d.pom.md5
-rw-rw----  1 stuhood  admin     40 Aug 21 18:20 hello-greet-436ae2dee0e2fc15a6ccfd2fcec5231b8e127a8d.pom.sha1
-rw-rw----  1 stuhood  admin   1009 Aug 21 18:20 ivy-436ae2dee0e2fc15a6ccfd2fcec5231b8e127a8d.xml
-rw-rw----  1 stuhood  admin     32 Aug 21 18:20 ivy-436ae2dee0e2fc15a6ccfd2fcec5231b8e127a8d.xml.md5
-rw-rw----  1 stuhood  admin     40 Aug 21 18:20 ivy-436ae2dee0e2fc15a6ccfd2fcec5231b8e127a8d.xml.sha1

Reviewed at https://rbcommons.com/s/twitter/r/923/