move the publish-extras-task doc to its own page. mention repo in register.py

moved some info around in hopes folks can find it more easily
mention repo-in-register.py

before:
setup_repo had some info about setting up publishing, but the publish page didn't link to it.
setup_repo didn't mention something new and important: setting up a repo in a plugin/register.py
publish page had a section about setting up an extras-with-publish task

after:
link to setup_repo's publish section from the publish page
mention repo-in-register.py in setup_repo
mv 'extras-with-publish task' to its own page, linked from setup_repo

Bugs closed: 602

Reviewed at https://rbcommons.com/s/twitter/r/1090/
get the name right, no stray quote mark, sheesh

Testing Done:
monkey-patched in Twitter's pants. hand-copied over. what coudl go wrong?

Bugs closed: 679

Reviewed at https://rbcommons.com/s/twitter/r/1164/
Fix issue 668: make release.sh execute packaged pants without loading internal backends during testing

The introduction of src/python/internal_backend plugins broke build-support/bin/release.sh.
Release.sh, during dry-run and real testing, fails to execute packaged pants, because it
doesn't have internal_backend packaged within but the repo root pants.ini still asks it
to load them.

The fix is to use PANTS_CONFIG_OVERRIDE during such execution with an empty plugin list.

Testing Done:
ran and passed:

- build-support/bin/release.sh -n
- build-support/bin/release.sh -t

CI passed: https://travis-ci.org/jinfeng/jinfeng-pants-fork/builds/37896390

Bugs closed: 668

Reviewed at https://rbcommons.com/s/twitter/r/1158/
A convenient way to set new-style options in tests.

Testing Done:
CI passed.

Merged this into a branch where I switched a bunch of JvmTask-related tests over to new options, and checked that this worked.

Reviewed at https://rbcommons.com/s/twitter/r/1163/
comments attempting to clear up: what goes in BUILD.tools ? what goes in 3rdparty/BUILD ?

Inspired by conversation on https://rbcommons.com/s/twitter/r/1151/ , I bet we want comments on these files helping folks to figure out which deps go where.

Reviewed at https://rbcommons.com/s/twitter/r/1155/
Fix scalastyle task, wire it up, make configs optional + refactoring and unit tests

- fix the task, which has slipped out of maintenance.
- wire it up in OS pants.
- but make it flexible so that pants_internal or other private wrapper of OS pants won't fail if they don't have scalastyle config and excludes present (making config/exclude optional)
- some refactoring so I can write unit tests easily.
- unit tests and end to end tests

Testing Done:
- CI passed: https://travis-ci.org/jinfeng/jinfeng-pants-fork/builds/37824308

- - PANTS_DEV=1 ./pants goal test tests::

- positive manual test:

$ PANTS_DEV=1 ./pants goal compile examples/src/scala/com/pants/example/hello/welcome
...
...
...
16:00:13 00:02   [compile]
16:00:13 00:02     [jvm]
...
...
16:00:22 00:11     [scalastyle]
16:00:22 00:11       [bootstrap-scalastyle]
16:00:23 00:12       [org.scalastyle.Main]
               Waiting for background workers to finish.
               SUCCESS

- negative manual test (with the examples/src/scala/com/pants/example/Welcome.scala to break the ImportGroupingChecker rule:

$ PANTS_DEV=1 ./pants goal compile examples/src/scala/com/pants/example/hello/welcome
...
...
...
16:03:42 00:00   [compile]
16:03:42 00:00     [jvm]
...
...
16:03:44 00:02     [scalastyle]
16:03:44 00:02       [org.scalastyle.Main]
                     ==== stderr ====

                     ==== stdout ====
                     error file=/Users/jinfeng/workspace/github-pants-jinfeng/examples/src/scala/com/pants/example/hello/welcome/Welcome.scala message=Imports should be grouped together  line=13 column=0
                     Processed 1 file(s)
                     Found 1 errors
                     Found 0 warnings
                     Finished in 56 ms

FAILURE: java org.scalastyle.Main ... exited non-zero (1)

               Waiting for background workers to finish.
               FAILURE

Bugs closed: 647

Reviewed at https://rbcommons.com/s/twitter/r/1145/
doc: if you're adopting Pants, and need to integrate that one weird tool

without an org-custom plugin, Pants does many amazing things. but it seems every organization has that handful of tools to integrate

Bugs closed: 105

Reviewed at https://rbcommons.com/s/twitter/r/1166/
sitegen += section links and xrefs

xmarks, xrefs:
.html files can have xmark anchors, like htmls anchor, except can be referenced from other pages.
xref is like href, but references xmarks

"pilcrow" section links: when mouse hovers over a heading, show a clickable link to that section. handy for when you want to answer a question w/a link to place in the docs.

fix to "breadcrumb" navigation: just show "ancestors", not also "older siblings"

added the start of some docs-for-docs

Testing Done:
https://travis-ci.org/pantsbuild/pants/builds/37849182

Reviewed at https://rbcommons.com/s/twitter/r/1154/
Adds a junit test integration test, including coverage options.

(Is cobertura working? I don't see any output in the cobertura reports.)

Testing Done:
ci is running.

Bugs closed: 675

Reviewed at https://rbcommons.com/s/twitter/r/1161/
(Adds missing fixes from new junit test integration test,  https://rbcommons.com/s/twitter/r/1161/)
Switch travis email notifications back to the default.

Emails now go to author (+comitter).

 .travis.yml                  | 14 ++++----------
 build-support/bin/ci-sync.sh |  5 +++--
 2 files changed, 7 insertions(+), 12 deletions(-)

Testing Done:
$ travis lint
Hooray, .travis.yml looks valid :)

And CI went green here: https://travis-ci.org/pantsbuild/pants/builds/38073663

I enabled ci for my pantsbuild/pants fork and I added a commit of this:
```console
$ cat << EOF > build-support/bin/ci.sh
#!/usr/bin/env bash
exit 1
EOF
```
That CI went red and emailed just me for both the pantsbuild/pants and jsirois/pants jobs:
  https://travis-ci.org/jsirois/pants/builds/38087136

Email to just me:
 jsirois / pants (jsirois/travis/email_simple)

Build #4 is still failing.	 38 seconds
	John Sirois	f727f99 Changeset →
 	Small workding fixes.

Bugs closed: 682

Reviewed at https://rbcommons.com/s/twitter/r/1169/
When resolving a tool dep, report back the pants.ini section with a reference that is failing.
Added a convenience method to pull a dependency list from pants.ini and   report the section/key info along to register_jvm_tool()

Testing Done:
ci is baking.

Manual Testing:

1) renamed target 3rdparty:protobuf-2.4.1 to 3rdparty:protobuf-2.4.2 and ran example protobuf compile.

``
$ ./pants goal compile examples/src/java/com/pants/examples/protobuf/distance
  ...
Exception message: protobuf-2.4.1 was not found in BUILD file /Users/zundel/Src/Pants/3rdparty/BUILD. Perhaps you meant one of:
  :checkstyle
  ...
  :util-core
  referenced from [protobuf-gen] key: javadeps in pants.ini
```

2) In BUILD.tools, I renamed '//:jar-tool to //:jar-tool2

```
$ ./pants goal bundle examples/src/java/com/pants/examples/hello/main
  ...
                     Failed to resolve target for bootstrap tool: //:jar-tool. You probably need to add this dep to your tools BUILD file(s), usually located in the root of the build.
                     See pants.ini section: [jar-tool] key: bootstrap-tools
                     Error: jar-tool was not found in BUILD file /Users/zundel/Src/Pants/BUILD. Perhaps you meant one of:
                       BUILD:readme
                       ...
                       BUILD.tools:zinc

```

Bugs closed: 676

Reviewed at https://rbcommons.com/s/twitter/r/1162/
Relabel the thrift_linter test to an integration test

test_thrift_linter is actually an integration test.  It is fast for an integration test, but slow as a unit test.

Testing Done:
ci si running

Bugs closed: 684

Reviewed at https://rbcommons.com/s/twitter/r/1171/
Refactor jar_library to be able to unwrap its list of jar_dependency objects

This change is intended to support the external_archive() sharing the same logic as java_protobuf_library() for processing
a list of addresses to JarLibrary targets.

Testing Done:
Added new unit test for new method in test_jvm_target
Also beefed up the test_jar_library unit test

CI is running

Bugs closed: 680

Reviewed at https://rbcommons.com/s/twitter/r/1165/
Missed some style comments from code reviews earlier today, trying to clean up.

Missed some style comments from code reviews yesterday, trying to clean up.

Testing Done:
CI passed.

Bugs closed: 686

Reviewed at https://rbcommons.com/s/twitter/r/1174/
Move registration of --no-pantsrc to global_options.py to resurrect it

It's a bit of a special case since it is used before the options are parsed. I left that part as is.

Testing Done:
Manual

Reviewed at https://rbcommons.com/s/twitter/r/1170/
Ensure new options values are always properly typed.

Previously values read from pants.ini were always strings.

Amend the relevant unit test to detect a regression of this.

Testing Done:
Fixed the test, then ran it before and after the fix.

ci baking.

Reviewed at https://rbcommons.com/s/twitter/r/1175/
Adding the ability to pull in a Maven artifact and extract its contents for use to
feeding into another task.

Adds target ExternalArchive, task UnpackExternalArchive, and method FromTarget