doc: fix sphinx warning for option-less Goals

Four Goals don't have CLI options. Yay, elegance. Boo, the
CLI-option-automagic-docs template I just added assumed everything
had options; Sphinx warns for those four Goals.

So... don't assume that. Fix the warnings.

Reviewed at https://rbcommons.com/s/twitter/r/320/
Allow use of a BUILD file with a suffix in a directory with no BUILD file without a suffix.
Also allow more characters in a BUILD file suffix.

Testing Done:
Manually renamed BUILD to BUILD.gen in src/java/com/pants/examples/hello/main

Updated test_build_file.py and test_linkify.py
Travis-ci went green at https://travis-ci.org/pantsbuild/pants/builds/25567951

Bugs closed: 126

Reviewed at https://rbcommons.com/s/twitter/r/381/
Make the 'py' command work post-target-refactor.
Allow old- and new-style resources on python targets.

New-style (specs of Resurces targets) are provided via the resource_targets= arg.
Old-style (lists of files, filesets) are wrapped in a synthetic target.

This allows a smooth transition from old to new-style resources.

If and when all codebases have switched to resource_targets=, and we get rid of
old-style resource support entirely, we can repurpose the resources= arg
to mean the same as resource_targets=.
Really fix the thing I said I fixed.
get 'goal builddict' and 'tests/python/pants_test/tasks:builddict' working again

un-break/re-enable 'builddict' goal and its test

As part of this added a class function to BuildFileParser: report_registered_context()
Returns a dict of symbols defined in BUILD files. This dict is quite useful for code that spews docs about symbols defined in BUILD files.

somewhat sad, but tolerable: many target constructor sigs now look roughly like __init__(arg, kwargs):Super(...).__init__(args,kwargs) . Thus, several of our formerly-detailed function sig docs are now less informative "args, kwargs". Maybe if I dig around in pydoc code and/or other reflection-y stuff, there's some straightforward way to do something lovely that automagically accumulates a method signature up the inheritance tree. I won't try to figure that out today, though.

Testing Done:
a lot of looking at old-Pants output vs new-Pants output, reconciling differences

re-enabled a test. and got it working again.

./build-support/bin/ci.sh

Reviewed at https://rbcommons.com/s/twitter/r/362/
Make classpath components relative to the current directory to solve problem of very long classpaths in jobs running under mesos.
Add resources to examples

* Add a resources subdirectory, and add it as a dependency of the java/scala
  examples
* Confirm loading the resource from the classpath
    Fixed misleading typo in pants.ini
Set no_color=True when TERM=dumb

  reviewed at https://rbcommons.com/s/twitter/r/370/
docs: don't suggest './rbt status'. That fails. Instead, './rbt help'

For the background, see this mail thread
https://groups.google.com/forum/#!topic/pants-devel/oZvsn_LzDNU

Testing Done:
moved my cached cookie aside
ran ./rbt commands
This review posted OK.

Bugs closed: https://github.com/pantsbuild/pants/issues/136

Reviewed at https://rbcommons.com/s/twitter/r/384/
Updated cache_validation exception handler to catch all types of exceptions.

This change will catch and re-throw with the original stack trace, enhancing the exception message with the name of the pants target and location of the build file.

Was:
*** Running pants in dev mode from /Users/zundel/Src/pants/src/python/pants/bin/pants_exe.py ***

10:18:40 00:00 [main]
               See a report at: http://localhost:50651/run/pants_run_2014_05_22_10_18_40_62
10:18:40 00:00   [bootstrap]
10:18:40 00:00   [setup]
10:18:40 00:00     [parse]
10:18:40 00:00   [jvm-locate]
10:18:40 00:00   [jvm-locate]
10:18:40 00:00   [jvm-locate]
10:18:40 00:00   [jvm-locate]
10:18:40 00:00   [jvm-locate]
10:18:40 00:00   [jvm-locate]
10:18:40 00:00   [bootstrap]
10:18:40 00:00     [bootstrap-jvm-tools]
10:18:40 00:00   [gen]
10:18:40 00:00     [thrift]
10:18:40 00:00     [scrooge]
10:18:40 00:00     [protoc]

               FAILURE
Traceback (most recent call last):
  File "/Users/zundel/Src/pants/src/python/pants/bin/pants_exe.py", line 175, in <module>
    main()
  File "/Users/zundel/Src/pants/src/python/pants/bin/pants_exe.py", line 169, in main
    _run()
  File "/Users/zundel/Src/pants/src/python/pants/bin/pants_exe.py", line 150, in _run
    result = command.run(lock)
  File "/Users/zundel/Src/pants/src/python/pants/commands/goal.py", line 234, in run
    return engine.execute(context, self.phases)
  File "/Users/zundel/Src/pants/src/python/pants/engine/engine.py", line 49, in execute
    self.attempt(context, phases)
  File "/Users/zundel/Src/pants/src/python/pants/engine/group_engine.py", line 278, in attempt
    phase_executor.attempt(explain)
  File "/Users/zundel/Src/pants/src/python/pants/engine/group_engine.py", line 170, in attempt
    execute_task(goal, self._tasks_by_goal[goal], self._context.targets())
  File "/Users/zundel/Src/pants/src/python/pants/engine/group_engine.py", line 141, in execute_task
    task.execute(targets)
  File "/Users/zundel/Src/pants/src/python/pants/tasks/code_gen.py", line 97, in execute
    with self.invalidated(gentargets, invalidate_dependents=True) as invalidation_check:
  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/contextlib.py", line 17, in __enter__
    return self.gen.next()
  File "/Users/zundel/Src/pants/src/python/pants/tasks/task.py", line 195, in invalidated
    invalidation_check = cache_manager.check(targets, partition_size_hint, colors)
  File "/Users/zundel/Src/pants/src/python/pants/base/cache_manager.py", line 217, in check
    all_vts = self._sort_and_validate_targets(targets)
  File "/Users/zundel/Src/pants/src/python/pants/base/cache_manager.py", line 242, in _sort_and_validate_targets
    cache_key = self._key_for(target, transitive=self._invalidate_dependents)
  File "/Users/zundel/Src/pants/src/python/pants/base/cache_manager.py", line 262, in _key_for
    return self._cache_key_generator.key_for_target(target, transitive=transitive)
  File "/Users/zundel/Src/pants/src/python/pants/base/build_invalidator.py", line 85, in key_for_target
    target_key = target.transitive_invalidation_hash()
  File "/Users/zundel/Src/pants/src/python/pants/base/target.py", line 179, in transitive_invalidation_hash
    target_hash = self.invalidation_hash()
  File "/Users/zundel/Src/pants/src/python/pants/base/target.py", line 162, in invalidation_hash
    self._cached_invalidation_hash = self.compute_invalidation_hash()
  File "/Users/zundel/Src/pants/src/python/pants/base/target.py", line 158, in compute_invalidation_hash
    return self.payload.invalidation_hash()
  File "/Users/zundel/Src/pants/src/python/pants/base/payload.py", line 112, in invalidation_hash
    sources_hash = hash_sources(get_buildroot(), self.sources_rel_path, self.sources)
  File "/Users/zundel/Src/pants/src/python/pants/base/payload.py", line 22, in hash_sources
    with open(os.path.join(root_path, rel_path, source), 'rb') as f:
  File "/Users/zundel/Development/java/build-support/pants_dev_deps.venv/bin/../lib/python2.7/posixpath.py", line 75, in join
    if b.startswith('/'):
AttributeError: 'Fileset' object has no attribute 'startswith'

Now:
  ...

  File "/Users/zundel/Src/pants/src/python/pants/base/target.py", line 158, in compute_invalidation_hash
    return self.payload.invalidation_hash()
  File "/Users/zundel/Src/pants/src/python/pants/base/payload.py", line 112, in invalidation_hash
    sources_hash = hash_sources(get_buildroot(), self.sources_rel_path, self.sources)
  File "/Users/zundel/Src/pants/src/python/pants/base/payload.py", line 22, in hash_sources
    with open(os.path.join(root_path, rel_path, source), 'rb') as f:
  File "/Users/zundel/Development/java/build-support/pants_dev_deps.venv/bin/../lib/python2.7/posixpath.py", line 75, in join
    if b.startswith('/'):
pants.base.cache_manager.CacheValidationError: Problem validating target squarepants.hack-protos.src.main.proto.proto in squarepants/hack-protos/src/main/proto: 'Fileset' object has no attribute 'startswith'

Testing Done:
This is just a diagnostic - I hope we cache problems like this explicitly before it comes down to an unexpected problem in validation.  I updated the exception handling to help me track down which BUILD is causing a problem after the refactor... I'll track down that issue separately.

Bugs closed: 145

Reviewed at https://rbcommons.com/s/twitter/r/396/
Only run tests once each...

This was due to the regular target dep-following logic being duplicated
by legacy python_test_suite logic.

This change gets rid of python_test_suite (or rather, makes it
an alias of the 'dependencies' target, to not break existing BUILD files).

We're now back to taking ~2m20s to run all the pants tests.
Add missing dep.
Fix some broken codegen target synthesis code.

Also got rid of bogus duplicate is_synthetic concept.
Quick fix for gen_java/gen_python in codegen.  I ran into issues with running codegen because the task was trying to access the build graph during its __init__.

Testing Done:
CI passes

Reviewed at https://rbcommons.com/s/twitter/r/400/
Add non-closure injection methods to BuildGraph

This came up for optimizing an internal implementation of WhatChanged.

Testing Done:
CI passes

Reviewed at https://rbcommons.com/s/twitter/r/399/
Name codegen synth targets by target id, instead of name.

Before, we were just getting lucky that we didn't have two colliding target names for our codegen targets, but their synthetic targets should be 1:1 with their (language, build file target) tuple.

Testing Done:
CI passes

Bugs closed: https://github.com/pantsbuild/pants/pull/148

Reviewed at https://rbcommons.com/s/twitter/r/398/
doc: pants.new.rc is old, pants.rc is new

Bugs closed: https://github.com/pantsbuild/pants/issues/137

Reviewed at https://rbcommons.com/s/twitter/r/401/
Fix publishing and also add integration tests

Fix publishing errors
1. Fix no config javadoc found for targets with no depdencies.
==== stderr ====
                     Exception in thread "main" java.text.ParseException: Problem occurred while parsing ivy file: Cannot add artifact 'example.jar(javadoc)' to configuration 'javadoc' of module com.twitter#example;0.0.1-SNAPSHOT because this configuration doesn't exist! in file:/Users/tdesai/projects/pants/.pants.d/publish/publish/com.twitter/example/ivy-0.0.1-SNAPSHOT.xml

2. Add Integration tests

Testing Done:
locally:
 ./pants tests/python/pants_test/tasks:jar_publish_integratio
tests/python/pants_test/tasks/test_jar_publish_integration.py ..

============================================================================= 2 passed in 23.32 seconds =============================================================================
Error in atexit._run_exitfuncs:
Traceback (most recent call last):
  File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/atexit.py", line 24, in _run_exitfuncs
    func(*targs, **kargs)
  File "/private/var/folders/t_/ck6c3zvj2z5cg77dqx2kh8tr0000gp/T/tmpKYikI8/.deps/twitter.common.dirutil-0.3.0-py26-none-any.whl/twitter/common/dirutil/__init__.py", line 61, in _mkdtemp_atexit_cleaner
    for td in _MKDTEMP_DIRS.pop(os.getpid(), []):
AttributeError: 'NoneType' object has no attribute 'pop'
Error in sys.exitfunc:
Traceback (most recent call last):
  File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/atexit.py", line 24, in _run_exitfuncs
    func(*targs, **kargs)
  File "/private/var/folders/t_/ck6c3zvj2z5cg77dqx2kh8tr0000gp/T/tmpKYikI8/.deps/twitter.common.dirutil-0.3.0-py26-none-any.whl/twitter/common/dirutil/__init__.py", line 61, in _mkdtemp_atexit_cleaner
    for td in _MKDTEMP_DIRS.pop(os.getpid(), []):
AttributeError: 'NoneType' object has no attribute 'pop'
tests.python.pants_test.tasks.jar_publish_integration                           .....   SUCCESS
[tw-172-25-20-142 pants]$ ./pants tests/python/pants_test/tasks:jar_publish_integration
Build operating on targets: OrderedSet([PythonTests(tests/python/pants_test/tasks/BUILD:jar_publish_integration)])

Bugs closed: 120

Reviewed at https://rbcommons.com/s/twitter/r/356/