Dont write to an already closed file

Link to related IRC discussion log: https://url.corp.redhat.com/bce8a01