Make close idempotent

Since it might be called twice.
Also fixes broken test, since I wanted to ensure it works be calling tests.

@syncbak-infrastructure please take a look.