Turn dependency dupes into errors

We see a bunch of dupes pop up in dependency lists in our internal repos. Often they go unnoticed by whoever should be fixing them. They also make each parsing of the build graph quite noisy for the rest of our users.

This branch turns those warnings into errors. That ought to ensure they're caught early by whoever added the duplicate dependency and they won't hit master.

Testing Done:
ci.sh passes
tested against a dependency list with dupes

Reviewed at https://rbcommons.com/s/twitter/r/1332/
Clean pants script which used "build"

Clean pants script which referred to "build"

[tw-172-25-146-131 pants]$ UNSET PANTS_DEV
[tw-172-25-146-131 pants]$ build-support/python/clean.sh && ./pants
Building pants.pex to /Users/tdesai/projects/pants/pants.pex...
Pants build missing a command: None

*** Pants bootstrapping failed.
[tw-172-25-146-131 pants]$

Testing Done:
yes
./pants now works.

I can add this in CI if required.

Reviewed at https://rbcommons.com/s/twitter/r/1335/
update Pants+IntelliJ docs to say the Plugin's the new hotness, link to plugin's README

Reviewed at https://rbcommons.com/s/twitter/r/891/
Fix a nasty bug when injecting target closures in BuildGraph

Just because a target has been injected doesn't mean that its
closure (and therefore its dependency edges) have been injected.
The current code misses the case where a target was injected
(but not its closure) and then has its closure injected later.

Testing Done:
Added a test that identifies the bug, which now passes (doesn't on master):
tests/python/pants_test/graph test_inject_then_inject_closure.

CI is green: https://travis-ci.org/pantsbuild/pants/builds/41270498

Reviewed at https://rbcommons.com/s/twitter/r/1337/
Fix java_sources missing dep detection.

This seems to just be an oversight in the original implementation.
We noticed incorrect missing dependency errors (we have it set to
error rather than warn) when we started using java_sources in our codegen.
This resolved our issue internally.

The general idea is that a ScalaLibrary should be claiming its
java_sources sources for the purpose of mapping generated classfiles
back to the sources zinc used to generate them.

Testing Done:
CI is green: https://travis-ci.org/pantsbuild/pants/builds/41043159

Reviewed at https://rbcommons.com/s/twitter/r/1336/
Kill all nailgun tasks after an integration test run to keep from leaking NGServer instances.

Before this change, Travis-CI builds are failing with: Cannot create GC thread. Out of system resources.
For example:  https://travis-ci.org/pantsbuild/pants/jobs/40841557

Testing Done:
CI running at: https://travis-ci.org/pantsbuild/pants/builds/41295684

Manual test: while running devpants goal test ./tests/python/pants_test:integration

before:
ps auxww | grep NGServer | wc -l
143

after:
ps auxww | grep NGServer | wc -l
13

Bugs closed: 799

Reviewed at https://rbcommons.com/s/twitter/r/1345/
Remove legacy option dests from all non-jvm-y tasks.

Testing Done:
CI passed.

Reviewed at https://rbcommons.com/s/twitter/r/1341/
Remove legacy dests from jvm task options.

Testing Done:
CI passed.

Reviewed at https://rbcommons.com/s/twitter/r/1343/
Remove support for legacy flag dests in new options.

We no longer use it anywhere.

Testing Done:
CI passes.

Reviewed at https://rbcommons.com/s/twitter/r/1351/
doc site cross references: call them pantsmark, pantsref instead of xmark, xref

<a xmark="foo"> looks like it could be some obscure HTML thing. Switch to <a pantsmark="foo"> which is more obviously a pants thing.

Testing Done:
https://travis-ci.org/pantsbuild/pants/builds/41306457

Reviewed at https://rbcommons.com/s/twitter/r/1348/
Use awk instead of sed to filter ever Nth line out of the integration test output The sed -n "K~N" syntax is not compatible with sed on MacOS.

Testing Done:
compared the list of integration tests listed when modifying the -i N:M parameters to the script.
Here is a good CI run https://travis-ci.org/pantsbuild/pants/builds/41359083

Bugs closed: 800

Reviewed at https://rbcommons.com/s/twitter/r/1350/
fix string formatting so 'test -h' does not crash

Testing Done:
PANTS_DEV=1 ./pants goal test -h

doesn't crash, unlike

./pants goal test -h

https://travis-ci.org/pantsbuild/pants/builds/41402745

Bugs closed: 805

Reviewed at https://rbcommons.com/s/twitter/r/1353/
Add dummy readme in examples/src/../scala/hello for Intellij plugin integration tests.

We added capability to the pants plugin to index non target owning files e.g. deploy scripts or READMEs
here https://rbcommons.com/s/twitter/r/1338/

We added the for the plugins.
Integration tests for Intllij plugin

Testing Done:
yes.
https://github.com/pantsbuild/pants/commit/d0fc5d05e8fae14311bab4cb7e664fa23fb02a8c

Reviewed at https://rbcommons.com/s/twitter/r/1356/
Support for the --kill-nailguns option was inadvertently removed, this puts it back.

The kill-nailguns flag was being read through old_options.cleanup_nailguns

Also updated the nailgun timeout logic to try and pinpoint transient errors in CI if it
happens again.

This reverts the change at https://rbcommons.com/s/twitter/r/1345/ which is redundant with this one (and not as effective.)

Testing Done:
CI Passed.

Bugs closed: 803

Reviewed at https://rbcommons.com/s/twitter/r/1352/
scala_specs: mention specs lib in the docstring; mention junit works, too

because not all Scala tests are scala_tests

Bugs closed: 622

Reviewed at https://rbcommons.com/s/twitter/r/1326/
Re-upload artifacts that encountered read-errors