Do not flush the cache after every block outside of IBD