scm: add merge-base and origin url properties to git

catch exceptions and always remove remote in `mkremote`

Testing Done:
build passes: https://travis-ci.org/cheecheeo/pants/builds/31666560

This is necessary for a feature that will include the merge-base and repository url in pants-built artifacts.

Reviewed at https://rbcommons.com/s/twitter/r/834/
Simplify phase.py.

- Gets rid of clever but complicated deduping logic. Replaces it with
  dumb but simple deduping logic via a factory.

- Gets rid of a few methods that were either entirely unused or only used
  once (in which case replaced them with alternative code).

- Phases now store task names and types directly. The TaskRegistrar instance
  isn't retained after registration.

- A Phase now directly knows its depenendcies on other phases, instead of
  delegating to the union of dependencies of its TaskRegistrars.
  This makes a lot of code much simpler.

- Phase no longers uses the old 'goal' terminology.

- Move logic such as option parser setup from TaskRegistrar into Phase.

This simplifies Phase itself and a lot of code that uses it, and
paves the way for renaming Phase to Goal.

Testing Done:
CI passed: https://travis-ci.org/pantsbuild/pants/builds/32051833.

Unit and integration tests pass locally too.

Ran some manual builds, including "./pants goal goals --goals-graph".

Reviewed at https://rbcommons.com/s/twitter/r/856/
Moved some example code out from /src and /tests into /examples/src and examples/test

We discussed a new layout for example and testproject code on the list:

https://groups.google.com/forum/#!topic/pants-devel/cLsY7ksLoyM

This patch covers moving the java examples.  If we like this, I'll continue on with non-java targets and
testprojects into a new /testprojects/ top level directory.

Testing Done:
Updated tests, ran tests. CI is baking.

Bugs closed: 472

Reviewed at https://rbcommons.com/s/twitter/r/868/
git: commit date of refs

Testing Done:
ci.sh succeeds

Reviewed at https://rbcommons.com/s/twitter/r/871/
Fixup missed parameter rename.

https://rbcommons.com/s/twitter/r/871/ comitted @ 50bdab06 failed CI here: https://travis-ci.org/pantsbuild/pants/jobs/32402628
This fixes that.

Testing Done:
$ PANTS_DEV=1 ./pants goal test tests/python/pants_test/scm/::
...
23:16:08 00:00   [test]
23:16:08 00:00     [pytest]
23:16:08 00:00       [run]
                     ============== test session starts ===============
                     platform linux2 -- Python 2.6.9 -- py-1.4.23 -- pytest-2.6.1
                     plugins: cov, timeout
                     collected 4 items

                     tests/python/pants_test/scm/test_git.py ....

                     ============ 4 passed in 0.60 seconds ============

23:16:28 00:20     [junit]
23:16:28 00:20     [specs]
               SUCCESS

Bugs closed: 476

Reviewed at https://rbcommons.com/s/twitter/r/875/
Fix the builddict and add a basic ci test.

This wires up a run of build-support/bin/publish_docs.sh as part
of the ci suite to at least confirm it runs without raising.

Testing Done:
Now works:
==
./build-support/bin/publish_docs.sh

Travis CI went green here: https://travis-ci.org/pantsbuild/pants/builds/32401953

Bugs closed: 475

Reviewed at https://rbcommons.com/s/twitter/r/873/
Fixup flag parser prep for phases - actually add option groups.

Testing Done:
Now real help:
==
$ PANTS_DEV=1 ./pants goal resolve -h
*** Running pants in dev mode from /home/jsirois/dev/3rdparty/jsirois-pants/src/python/pants/bin/pants_exe.py ***

resolve: Resolve dependencies and produce dependency reports.

resolve options:
  --ng-daemons, --no-ng-daemons
                          [True] Use nailgun daemons to execute java tasks.

gen:thrift options:
  --gen-thrift-version=THRIFT_VERSION
                          Thrift compiler version.
  --gen-thrift-lang=THRIFT_GEN_LANGS
                          Force generation of thrift code for these languages.

...
ivy options:
  --ivy-override=IVY_RESOLVE_OVERRIDES
                          Specifies a jar dependency override in the form:
                          [org]#[name]=(revision|url)
                          For example, to specify 2 overrides:
                          --ivy-override=com.foo#bar=0.1.2 \
                          --ivy-override=com.baz#spam=file:///tmp/spam.jar
  --ivy-report, --no-ivy-report
                          [False] Generate an ivy resolve html report
  --ivy-open, --no-ivy-open
                          [False] Attempt to open the generated ivy resolve
                          report in a browser (implies --ivy-report).
  --ivy-outdir=IVY_RESOLVE_OUTDIR
                          Emit ivy report outputs in to this directory.
  --ivy-args=IVY_ARGS     Pass these extra args to ivy.
  --ivy-mutable-pattern=IVY_MUTABLE_PATTERN
                          If specified, all artifact revisions matching this
                          pattern will be treated as mutable unless a matching
                          artifact explicitly marks mutable as False.

Bugs closed: 479

Reviewed at https://rbcommons.com/s/twitter/r/877/
Support uninstalling a task from a phase.

Reviewed at https://rbcommons.com/s/twitter/r/879/
Fix a bug in the signature of __new__.

Also, Python special-cases __new__ so you don't need to specify
@classmethod.

Reviewed at https://rbcommons.com/s/twitter/r/880/
add test to tickle brittle reflection-y part of Goals Reference generation

adding 'goal builddict' to ci.sh was great. this here test hopes to be a lightweight way to test a reflection-y part of the code.

Testing Done:
ran test. it passed.

temporarily tweaked Phase.setup->phase.setup in src/.../builddict.py; test failed.

Reviewed at https://rbcommons.com/s/twitter/r/876/
restore flag docs to Goals Reference

builddict.py wants to know how a phase computes its OptionGroup's title. From this, it figures out that the Goals Reference entry for "compile" should show "jvm" options but not "ivy" options (available at the command line, but documented with "resolve" instead).

In the Phase refactor, this broke. Phase used "task" or "phase:task"; builddict used "phase.task", d'oh. If we generate the docs today, the Goals Reference is missing flags.

This change adds Phase.options_group_title() and uses it in Phase and in builddict.

Testing Done:
https://travis-ci.org/pantsbuild/pants/builds/32483970

Reviewed at https://rbcommons.com/s/twitter/r/885/
don't forget to update CHANGELOG with things done lately

...or remember after the fact, at least

Reviewed at https://rbcommons.com/s/twitter/r/829/
Fix so the changed goal can be run in a repo with a directory called 'build'

* Added changed goal to register.py.
* Changed goal failed on mac (case insensitive filesystem) in one of our repos. Fixed so BuildFile can be opened on a dir called build that is not a build file. Added tests for it.

Reviewed at https://rbcommons.com/s/twitter/r/872/
git: merge-base takes references

Testing Done:
ci.sh locally succeeds

travis: https://travis-ci.org/cheecheeo/pants/builds/32479798

Reviewed at https://rbcommons.com/s/twitter/r/884/
Clean up (lack of) BuildFile.from_cache uses

Also a tiny tweak to send output to stderr rather than stdout.

Testing Done:
Deployed internally at 4sq.  CI is green

Reviewed at https://rbcommons.com/s/twitter/r/895/
Change a couple of warnings to infos

Overriding aliases is not entirely uncommon, and seeing a warning on
startup every time is very noisy.

Testing Done:
CI is green

Reviewed at https://rbcommons.com/s/twitter/r/896/
Patch JvmRun to accept JvmApps

Testing Done:
Running internally at 4sq.  CI is green.

Reviewed at https://rbcommons.com/s/twitter/r/893/
Clean up two BUILD files to get rid of warnings

* In pants/bin/BUILD, remove a duplicate dep and sort the sections
* In jvm_compile/BUILD, don't pull in the analysis anonymizer
  python_binary target, as it generates a warning and doesn't need to be
  included with the JVM libraries.
* Change jvm_compile:all to dependencies()

Testing Done:
CI is clean

Reviewed at https://rbcommons.com/s/twitter/r/897/
Add python as default codegen product.

This sits alongside the existing (brittle) hack in codegen that
allows downstream goals that only work with python products
to trigger codegen before they're run.

Testing Done:
Running internally at 4sq.  CI is green

Reviewed at https://rbcommons.com/s/twitter/r/894/
doc: integrate with_emacs page

I whined at Igor until he wrote a with_emacs page
describing the Ensime integration... but I forgot to
tell him how to link it into the doc set. So fix that.

Testing Done:
./build-support/bin/publish_docs.sh -o # yep, the link is there

Reviewed at https://rbcommons.com/s/twitter/r/863/