Revert "Revert "Only do Accumulator updates when there's new stuff""

This reverts commit abf86a1a69bd145e52d920c76eaa93b442ce97d8.