I suppose we should have this in a `finally` or context manager or something.