Close open cache files as soon as possible

See #124