andrewor14/incubator-spark — pull request #4 by aarondav (2013-12-30 05:32:13+00:00)
Title: Utilize DiskBlockManager pathway for temp file writing

This gives us a couple advantages:
- Uses spark.local.dir and randomly selects a directory/disk.
- Ensure files are deleted on normal DiskBlockManager cleanup.
- Availability of same stats as usual DiskBlockObjectWriter (currenty unused).

Also enable basic cleanup when iterator is fully drained.
Still requires cleanup for operations that fail or don't go through all elements.