Make a note about deleting open files on Windows.
*os.File -> io.WriteCloser.
Make error messages read a little better.
Use defer for cleaning up.
Unroll the cleanup actions.
Save the possibly multiple errors coming out of cleanUp.