Don't potentially block forever in Close() (fixes #655)