On lazy load error, rmeove any created directories.