Should not remove the cache directory if that does not exists.