apache/incubator-spark — pull request #74 by rxin (2013-10-18 21:41:27+00:00)
Title: Job cancellation via job group id.

This PR adds a simple API to group together a set of jobs belonging to a thread and threads spawned from it. It also allows the cancellation of all jobs in this group.

An example:

sc.setJobGroupId("job-to-kill")
sc.parallelize(1 to 10000, 2).map { i => Thread.sleep(10); i }.count()

In a separate thread:

sc.cancelJobGroup("job-to-kill")