Set a crawling interval and flush output buffer on each page crawl