apache/incubator-spark — pull request #208 by atkeano (2013-11-26 01:55:25+00:00)
Title: Update call site information for DStream functions

When a spark streaming job gets scheduled the original call site information is no longer available.
In addition to this, the call site extraction treats DStream classes as user space.
It is difficult to distinguish between jobs in the web ui (eg: 'take at DStream.scala:561')
This patch will inject the original call site information where possible.

Things to note:
- call site will now be captured by DStream and reset as context on RDDs as they are generated
- RDD context origin is used if available, otherwise fallback to current approach
- NetworkInputTracker is handled as special case to maintain its source with streaming package

Note: RDD already has both an 'origin' and a 'generator'. While one of these could have been
used and/or repurposed for this.. they both appear to have a slightly different intent, so I thought
it might be better to keep this separate.