allow crawling, enable file-based cache