When we create a new RDD at each step we store the RDD references in say a ArrayBuffer. After the loop exits, we call unpersist on all the older RDDs. This doesn't work very well with lazy transformations, though allReduce doesn't need to be lazy ?