crawl log processing