Stop trying to remove stuff that is already removed