bigdatagenomics/adam — pull request #293 by nfergu (2014-07-06 14:51:08+00:00)
Title: Add aggregation and display of metrics obtained from Spark

This change adds an new command-line option called "print_metrics" which causes a listener to be registered with Spark, which accumulates metrics when each task and stage end. When a command is complete the metrics are logged to stdout.

Some notes:

- The "print_metrics" option is added in the Args4jBase class. I thought about adding it in SparkArgs, but the intention is that it can be used for non Spark-related metrics in the future.
- The metrics are currently logged to stdout. They could be logged to SLF4J instead if that's better -- let me know and I'll change it if necessary.
- I've added a dependency on this library: http://code.google.com/p/java-ascii-table/ for printing ascii tables (Apache-licensed). It's not on any public maven repos, so I've added a local maven repo at /repo which contains the JAR file and POM. This seems to work fine both with the command line and Intellij, but let me know if there are any issues.