Be sure to close before trying to unlink.

On Windows, this is required, otherwise the unlink fails.