Ignore errors when deleting the failed files

We should ignore errors when deleting the failed files.

Reason:
![Stack trace](https://lh3.googleusercontent.com/-H6TDVFcOZ80/UakOmg_bDkI/AAAAAAAAFXE/Jm6VDtHuZ9U/s0/2013-05-31_16-56-59.png)

(On a side note, this is exactly what I was talking about with isdir and then remove not being atomic)