[SPARK-695] In DAGScheduler's getPreferredLocs, track set of visited partitions.

What do you think of a method like this for avoiding exponential path exploration in DAGScheduler's getPreferredLocs, per SPARK-695?

Some minor cleanups are also included.  The comment I removed in Dependency.scala seemed incorrect to me, per my commit comment:

    Remove apparently incorrect comment describing NarrowDependency.
  
    Each of a CartesianRDD's dependencies, for example, is a NarrowDependency, but child partitions of
    these dependencies may depend on shared parent partitions.  For example, the cartesian product of
    RDDs a and b containing partitions [ a_0 ] and [ b_0, b_1 ] respectively will be partitioned as
    [ ( a_0, b_0 ), ( a_0, b_1 ) ].  Each child partition depends on parent partition a_0.

I'm new to spark, scala (and even java) so a careful review may be in order.

Thanks,
Aaron
Yeah I think a substitute comment is preferable to removal, thanks for providing the proper definition.  Per your suggestion I’ll change it to:

Base class for dependencies where each partition of the child RDD depends on a small number of partitions of the parent RDD.  Narrow dependencies allow for pipelined execution.