Fix data race when reading conn.isClosed