Validate that registered args begin with dashes, as appropriate.

Otherwise we get a cryptic error further along.

Testing Done:
CI baking. Verified that bad registration triggers these validation checks.

Reviewed at https://rbcommons.com/s/twitter/r/1211/
Migrate more tasks, including all android tasks, to new options.

Renamed --vm-options to --jvm-options, for uniformity.

Testing Done:
Manual testing.

CI baking.

Reviewed at https://rbcommons.com/s/twitter/r/1214/
Fix reading of scalac plugin info from config.

The relevant keys are now:

[compile.scala]
plugins=['foo', 'bar']
plugin-args= {
  'foo': ['bar']
}

Testing Done:
Manually verified that the args are picked up.

Reviewed at https://rbcommons.com/s/twitter/r/1217/
Configurable Compression Level for Cache Artifacts

Allow changing (or disabling) the compression level of tarball artifacts via pants.ini

Also, require passing compression level in most init/callsites rather than allowing various default args to proliferate.

Default compression level in python gzip appears to be 9, so that is the default on TarballArtifact (replacing True).
The default when reading the config is currently 5, as we suspect that we're CPU rather than IO bound, but this change will also make it easier experiment with different values to find confirm that / find the optimal value.

Testing Done:
https://travis-ci.org/pantsbuild/pants/builds/38756549
https://travis-ci.org/pantsbuild/pants/builds/38964649

Reviewed at https://rbcommons.com/s/twitter/r/1194/
Update config migration script with new scalac plugin options.

Reviewed at https://rbcommons.com/s/twitter/r/1218/
Ensure proper semantics when replacing all tasks in a goal.

Testing Done:
ci in flight.

Reviewed at https://rbcommons.com/s/twitter/r/1220/
Take proper care with the order of operations when replacing tasks.

This logic is fragile and should probably be removed entirely.

Testing Done:
Manually verified that this fixes foursquare.web's issues with replacing all tasks in a goal.

ci running.

Reviewed at https://rbcommons.com/s/twitter/r/1221/
Revert conversion of thrift_linter.py to new options format, left in place TODO about precedence of override in target

Testing Done:
PANTS_DEV=1 ./pants goal test tests/python/pants_test/tasks:thrift_linter_integration   passes again

Bugs closed: 727

Reviewed at https://rbcommons.com/s/twitter/r/1223/
Read arguments for thrift-linter from pants.ini.

This allows changing linter options without changing pants.

Testing Done:
./build-support/bin/ci.sh

PANTS_DEV=1 ./pants goal thrift-linter testprojects/src/thrift/com/pants/testproject:thrift-java

Bugs closed: 719

Reviewed at https://rbcommons.com/s/twitter/r/1215/
Add support for updating the environment from prep_commands.

Reviewed at https://rbcommons.com/s/twitter/r/1222/
Include traceback with failures (even without fail-fast).

Testing Done:
Ran tests.

Reviewed at https://rbcommons.com/s/twitter/r/1226/
Fix a bug where all OptionValue objects had the same underlying forwardings.

This wasn't seen earlier because in practice all tasks migrated so far had
disjoint option names.

This was exposed when I migrated two unrelated tasks that happened to have
an identically named option (--skip).

Testing Done:
CI passed.

Verified that this fixes the name collision issue that triggered its discovery in the first place.

Reviewed at https://rbcommons.com/s/twitter/r/1229/
move ide_gen.py and friends idea_gen.py and eclipse_gen.py to new options

Move ide_gen.py and friends idea_gen.py and eclipse_gen.py to new options

Testing Done:
CI is running.

Bugs closed: 736

Reviewed at https://rbcommons.com/s/twitter/r/1230/
change thrift_linter.py to use new register_options

Testing Done:
Take two. Previous change accidentally added "default=False" which was inappropriate and ended up breaking the logic that falls back to a value defined in the BUILD file. Now we explicitly default the flag to None and preserve the desired logic.

I ran the integration tests related to thrift linter, which I inadvertently broke in my last attempt at this:
  devpants goal test tests/python/pants_test/tasks:thrift_linter_integration

Reviewed at https://rbcommons.com/s/twitter/r/1225/
Fix spacing in thrift_linter.py
Allow UTF-8 characters in changelog

Testing Done:
Tests pass, except that sometimes they fail on travis because
sys.stdout.encoding.

Reviewed at https://rbcommons.com/s/twitter/r/1228/
Minimal target "tags" support

This doesn't provide any addtional end-user functionality, but has been useful as a channel for custom tasks and BUILD files to communicate addtional information about targets.

We've been rebasing this change on top of upstream pants for releases at Foursquare for a few months now and we've found it useful in implmenting downstream custom tasks (like our graph rules), which need a place to put additional information in the build graph, while staying out of the internally meaningful things like `labels`.

Testing Done:
https://travis-ci.org/pantsbuild/pants/builds/39211424

Reviewed at https://rbcommons.com/s/twitter/r/1227/
Remove java_sources as target roots for scala library in depmap project info #670

Testing Done:
ci passes

Reviewed at https://rbcommons.com/s/twitter/r/1190/
Adding the ability to pull in a Maven artifact and extract its contents.

Specifically intended to feed into another task's sources= attribute.

Adds the concept of a SourceSet which is referenced from a SourcesField type payload field.
Adds target JarSourceSet, task UnpackJarSourceSet, and method FromTarget.