Added functionality to flush out index due to  java heap size issues