Don't allow caching of empty directories