added hard commit every 100k docs