Repository: edgedb/edgedb

Issue:
Misleading error log when CLI disconnects in a transaction
<!-- Please search existing issues to avoid creating duplicates. -->
- EdgeDB Version: 4aa682557093f7c4e44fefbcc1727333b59438dd
- OS Version: macOS

Steps to Reproduce:

1. Start a transaction in the CLI REPL
2. Terminate the CLI without closing the transaction

Server prints:

```
========================================= Exception occurred: cannot release connection <edb.server.pgcon.pgcon.PGConnection object at 0x11736cee0>: the connection was never acquired from the pool =========================================

1. RuntimeError: cannot release connection <edb.server.pgcon.pgcon.PGConnection object at 0x11736cee0>: the connection was never acquired from the pool

----------------------------------------------------------------------------------------------------------------- Traceback ------------------------------------------------------------------------------------------------------------------

    uvloop/cbhandles.pyx, line 69, in uvloop.loop.Handle._run
        > trimmed = Field(bool, default=False)
    uvloop/handles/basetransport.pyx, line 169, in uvloop.loop.UVBaseTransport._call_connection_lost
        > ExceptionContextList = checked.CheckedList[ExceptionContext]
    edb/server/pgcon/pgcon.pyx, line 2145, in edb.server.pgcon.pgcon.PGConnection.connection_lost
        > pinned_by.on_aborted_pgcon(self)
    edb/server/protocol/binary.pyx, line 331, in edb.server.protocol.binary.EdgeConnection.on_aborted_pgcon
        > self.server.release_pgcon(self.dbname, conn, discard=True)
    edb/server/server.py, line 521, in release_pgcon
        > self._pg_pool.release(dbname, conn, discard=discard)
    edb/server/connpool/pool.py, line 1156, in release
        > raise RuntimeError(

RuntimeError: cannot release connection <edb.server.pgcon.pgcon.PGConnection object at 0x11736cee0>: the connection was never acquired from the pool
ERROR 7468 2021-11-10T15:27:21.790 asyncio: Exception in callback UVTransport._call_connection_lost
handle: <Handle UVTransport._call_connection_lost>
INFO 7468 2021-11-10T15:27:21.796 edb.server: Connection discarded to backend database: edgedb
INFO 7743 2021-11-10T15:27:21.797 postgres: unexpected EOF on client connection with an open transaction
INFO 7468 2021-11-10T15:27:22.820 edb.server: Backend connections to database edgedb: 1 were established in at least the last 1.0 seconds.
```


Discussion:

Resolving patch:
diff --git a/edb/server/server.py b/edb/server/server.py
--- a/edb/server/server.py
+++ b/edb/server/server.py
@@ -520,7 +520,11 @@ def release_pgcon(self, dbname, conn, *, discard=False):
             if not discard:
                 logger.warning('Released an unhealthy pgcon; discard now.')
             discard = True
-        self._pg_pool.release(dbname, conn, discard=discard)
+        try:
+            self._pg_pool.release(dbname, conn, discard=discard)
+        except Exception:
+            metrics.background_errors.inc(1.0, 'release_pgcon')
+            raise
 
     async def load_sys_config(self):
         syscon = await self._acquire_sys_pgcon()


Test patch:
diff --git a/tests/test_protocol.py b/tests/test_protocol.py
--- a/tests/test_protocol.py
+++ b/tests/test_protocol.py
@@ -254,3 +254,36 @@ async def test_proto_connection_lost_cancel_query(self):
                     con.terminate()
                 for con in other_conns:
                     await con.aclose()
+
+    async def test_proto_gh3170_connection_lost_error(self):
+        async with tb.start_edgedb_server() as sd:
+            self.assertNotIn(
+                'edgedb_server_background_errors_total'
+                '{source="release_pgcon"}',
+                sd.fetch_metrics(),
+            )
+            con = await sd.connect_test_protocol()
+            try:
+                await con.send(
+                    protocol.ExecuteScript(
+                        headers=[],
+                        script='START TRANSACTION'
+                    )
+                )
+                await con.recv_match(
+                    protocol.CommandComplete,
+                    status='START TRANSACTION'
+                )
+                await con.recv_match(
+                    protocol.ReadyForCommand,
+                    transaction_state=protocol.TransactionState.IN_TRANSACTION,
+                )
+                await con.aclose()
+                self.assertNotIn(
+                    'edgedb_server_background_errors_total'
+                    '{source="release_pgcon"}',
+                    sd.fetch_metrics(),
+                )
+            except Exception:
+                await con.aclose()
+                raise