Do not try to delete the directory, instead delete the actual file.