Added crawling statistics and minor fixes.