Speed up and reduce memory usage of CommitStatsRecalculator