mesos/spark — pull request #462 by squito (2013-02-11 06:06:09+00:00)
Title: Track assorted metrics for each task, report summaries to user at stage completion

Three major parts here:
1) each task sends a TaskMetrics object back to the driver, to track some of the stats that used to just get printed out (eg. run time)
2) add some stats related to shuffles
3) create a SparkListener to get notification of various events (for now w/ just a StageCompletion event).

This is not ready to get merged in right now (missing docs & tests, at least), but I think its ready for some general feedback. In particular, I dunno if there is a better way to get all of the metrics back to the driver. I had to some coupling between components that used to be more independent. (eg., a TaskContext now has a reference to a Task).

I probably don't have the right set of metrics at the moment, but with this setup, we can add a bunch more and see what sticks. If possible, I'd like to just get it merged into master first, rather than fret over the exact metrics right now.

With the metrics I have so far, & using the provided SparkListener, after each stage the user gets output like:

```
Finished stage: MappedRDD[7] at map at :18
13/02/10 09:49:22 INFO scheduler.StatsReportListener: task runtime:(count: 64, mean: 144.562500, stdev: 83.793249)
13/02/10 09:49:22 INFO scheduler.StatsReportListener:	0%	5%	10%	25%	50%	75%	90%	95%	100%
13/02/10 09:49:22 INFO scheduler.StatsReportListener:	102	104	107	112	131	148	170	174	610
13/02/10 09:49:17 INFO scheduler.StatsReportListener: shuffle bytes written:(count: 64, mean: 265079.562500, stdev: 3616.055045)
13/02/10 09:49:17 INFO scheduler.StatsReportListener:	0%	5%	10%	25%	50%	75%	90%	95%	100%
13/02/10 09:49:17 INFO scheduler.StatsReportListener:	253.8 KB	254.9 KB	255.7 KB	256.8 KB	258.6 KB	259.6 KB	261.5 KB	262.8 KB	278.6 KB
13/02/10 09:49:22 INFO scheduler.StatsReportListener: fetch wait time:(count: 64, mean: 0.765625, stdev: 1.608009)
13/02/10 09:49:22 INFO scheduler.StatsReportListener:	0%	5%	10%	25%	50%	75%	90%	95%	100%
13/02/10 09:49:22 INFO scheduler.StatsReportListener:	0	0	0	0	0	1	2	3	9
13/02/10 09:49:22 INFO scheduler.StatsReportListener: remote bytes read:(count: 64, mean: 4498011.000000, stdev: 14247.297252)
13/02/10 09:49:22 INFO scheduler.StatsReportListener:	0%	5%	10%	25%	50%	75%	90%	95%	100%
13/02/10 09:49:22 INFO scheduler.StatsReportListener:	4.2 MB	4.3 MB	4.3 MB	4.3 MB	4.3 MB	4.3 MB	4.3 MB	4.3 MB	4.3 MB
```