removed print statements
added flush option to batch crawl job
added flush option
batch crawling and serialization to disk working