configurable crawl interval