Fix leftover leaks on persist_connection=false (see #1708)