Update config migration script with new scalac plugin options.

Reviewed at https://rbcommons.com/s/twitter/r/1218/
Ensure proper semantics when replacing all tasks in a goal.

Testing Done:
ci in flight.

Reviewed at https://rbcommons.com/s/twitter/r/1220/
Take proper care with the order of operations when replacing tasks.

This logic is fragile and should probably be removed entirely.

Testing Done:
Manually verified that this fixes foursquare.web's issues with replacing all tasks in a goal.

ci running.

Reviewed at https://rbcommons.com/s/twitter/r/1221/
Revert conversion of thrift_linter.py to new options format, left in place TODO about precedence of override in target

Testing Done:
PANTS_DEV=1 ./pants goal test tests/python/pants_test/tasks:thrift_linter_integration   passes again

Bugs closed: 727

Reviewed at https://rbcommons.com/s/twitter/r/1223/
Read arguments for thrift-linter from pants.ini.

This allows changing linter options without changing pants.

Testing Done:
./build-support/bin/ci.sh

PANTS_DEV=1 ./pants goal thrift-linter testprojects/src/thrift/com/pants/testproject:thrift-java

Bugs closed: 719

Reviewed at https://rbcommons.com/s/twitter/r/1215/
Add support for updating the environment from prep_commands.

Reviewed at https://rbcommons.com/s/twitter/r/1222/
Include traceback with failures (even without fail-fast).

Testing Done:
Ran tests.

Reviewed at https://rbcommons.com/s/twitter/r/1226/
Fix a bug where all OptionValue objects had the same underlying forwardings.

This wasn't seen earlier because in practice all tasks migrated so far had
disjoint option names.

This was exposed when I migrated two unrelated tasks that happened to have
an identically named option (--skip).

Testing Done:
CI passed.

Verified that this fixes the name collision issue that triggered its discovery in the first place.

Reviewed at https://rbcommons.com/s/twitter/r/1229/
move ide_gen.py and friends idea_gen.py and eclipse_gen.py to new options

Move ide_gen.py and friends idea_gen.py and eclipse_gen.py to new options

Testing Done:
CI is running.

Bugs closed: 736

Reviewed at https://rbcommons.com/s/twitter/r/1230/
change thrift_linter.py to use new register_options

Testing Done:
Take two. Previous change accidentally added "default=False" which was inappropriate and ended up breaking the logic that falls back to a value defined in the BUILD file. Now we explicitly default the flag to None and preserve the desired logic.

I ran the integration tests related to thrift linter, which I inadvertently broke in my last attempt at this:
  devpants goal test tests/python/pants_test/tasks:thrift_linter_integration

Reviewed at https://rbcommons.com/s/twitter/r/1225/
Fix spacing in thrift_linter.py
Allow UTF-8 characters in changelog

Testing Done:
Tests pass, except that sometimes they fail on travis because
sys.stdout.encoding.

Reviewed at https://rbcommons.com/s/twitter/r/1228/
Minimal target "tags" support

This doesn't provide any addtional end-user functionality, but has been useful as a channel for custom tasks and BUILD files to communicate addtional information about targets.

We've been rebasing this change on top of upstream pants for releases at Foursquare for a few months now and we've found it useful in implmenting downstream custom tasks (like our graph rules), which need a place to put additional information in the build graph, while staying out of the internally meaningful things like `labels`.

Testing Done:
https://travis-ci.org/pantsbuild/pants/builds/39211424

Reviewed at https://rbcommons.com/s/twitter/r/1227/
Remove java_sources as target roots for scala library in depmap project info #670

Testing Done:
ci passes

Reviewed at https://rbcommons.com/s/twitter/r/1190/
convert more .rst files to .md

convert more .rst docs to .md. With this, src/python/pants/docs/*.rst have been converted.

In some pages that were already converted, some links were "TODO" because their destinations weren't converted yet. So fix those up.

Mostly formatting changes, no content changes. except...

src/python/pants/docs/docs.rst -> src/python/pants/docs/docs.md : this page said that docs were .rst format with some details. switched to saying .md format, left out new details.

src/python/pants/docs/with_emacs.rst -> src/docs/with_emacs.md : Changed title to mention "Emacs" instead of just "Ensime".

more small tweaks in response to reviewer feedback

Testing Done:
eyeballing. To eyeball this change along with a couple of sitegen-fix reviewboards, see http://pantsbuild.github.io/sitegen-test/

grepping for some common mistakes from previous batch

Reviewed at https://rbcommons.com/s/twitter/r/1219/
git root calculation for idea goal

Testing Done:
tested in dev mode locally. ci passes.

Reviewed at https://rbcommons.com/s/twitter/r/1189/
Revert "convert more .rst files to .md"

This reverts commit b7967f52c1b70aa0c7c297633438fafbfd23164a.

This commit broke CI :-(

Submitting this reversion TBR

Reviewed at https://rbcommons.com/s/twitter/r/1235/
Some small fixes to the new options implementation.

These all relate to options whose action is 'append', and address rare corner cases.

1. Switch some invocations of copy() to deepcopy(), to ensure e.g., that
   default values of list type are not shared between the
   parser we use to generate help and the parser we use to actually
   parse args.

2. Distinguish between an empty list value in config, and no value at all.
   This ensures that "no value" doesn't trump a hard-coded default because
   we mistake it for an explicit config value.

3. Don't special-case appends to RankedValue. Instead make copy()
   return the underlying value.  argparse is the only thing that calls copy()
   on a RankedValue, and this will ensure that argparse always works with
   raw values, both in the list case and (as before) in the single value case.

4. Nits: Remove an unused import, break up an overlong line.

Testing Done:
Various manual tests of corner cases that could trigger these issues. CI running.

Reviewed at https://rbcommons.com/s/twitter/r/1236/
Migrate the filter task to new-style options.

Testing Done:
Ran all of filter's unittests. CI pending.

Reviewed at https://rbcommons.com/s/twitter/r/1232/
Add the environment's PYTHONPATH to sys.path when running dev pants.

This is helpful when running a graphical debugger: you need to ensure that
the debugger's code (pydevd in my case) is on your path so you can import
it at the tracepoint.

Reviewed at https://rbcommons.com/s/twitter/r/1237/