@pwendell Wait, actually isn't that right? Correct me if I'm wrong, but once a ShuffleDependency or an RDD goes out of scope, then all associated stages can be safely removed, since to run tasks in the Stage you need this information. This suggests if we keep track of a list of stage IDs in both ShuffleDependency and RDD, then we can clean up this map (and the one in ResultTask) explicitly.

@tdas, looks like this map only concerns the master, so I don't think we need to notify the workers of stage completion.