The shuffle file is closed when all the map tasks are done (aster the shuffle request is received). If we need to re-run a map task afterwards, 
(1) There is no easy way to remove the results of the previous run of the tasks from the shuffle file
(2) You cannot re-open a closed shuffle file and append to it (e.g., kyro will append some tailer when closing the file) 
(3) If a fetch fails, Spark will re-run all the map tasks (for the same shuffle) whose out locations are at the failing node; so it's OK to delete their outputs.

I actually also tried throwing random exceptions during shuffle and it works fine. Again, is there a good way to include such tests?