Repository: dbt-labs/dbt-core

Issue:
Lazy load connections
## Issue
The `dbt compile` command requires `gcloud` for bigquery profiles.

This is a regression from version 0.13.1, which did not require gcloud for running the compile command.

### Issue description
Command: 

`dbt compile`
dbt --debug compile --profiles-dir . --vars \"{target-path: my_path}\"; 

### Results
Error Summary:
```
Could not find command, ensure it is in the user's PATH: "gcloud"
```
[dbt_compile_error.txt](https://github.com/fishtown-analytics/dbt/files/3340481/dbt_compile_error.txt)

### System information
The output of `dbt --version`:
`Running with dbt=0.14.0-a2`

The operating system you're running on:
Debian-based Linux with kernel version 4.19.37

The python version you're using (probably the output of `python --version`)
2.7

### Steps to reproduce
*   Run `dbt compile` with a project that uses bigquery and with `gcloud` missing from the path.


Discussion:
I think this was introduced in https://github.com/fishtown-analytics/dbt/commit/e2af871a5adc1b6b0269ef900e04320550ac1bb8 with this change to the context/parser.py generate function:

```python
def generate(model, runtime_config, manifest, source_config):
    # during parsing, we don't have a connection, but we might need one, so we
    # have to acquire it.
    # In the future, it would be nice to lazily open the connection, as in some
    # projects it would be possible to parse without connecting to the db
    with get_adapter(runtime_config).connection_named(model.get('name')):
        return dbt.context.common.generate(
            model, runtime_config, manifest, source_config, Provider()
        )
```

beckjake@, would it be possible to go back to making this lazily open the connection?  I can't contact an actual bigquery client to run the compile command.
(Meant to put the ampersand in front:  @beckjake)
Thanks for the report @heisencoder. Can you tell me a little bit more about your setup here? I'm interested in your note that you can't access a BQ client when running `dbt compile` -- that feels atypical given my experience with most dbt projects out there, so definitely curious to learn more about your use case.

Additionally, I'm unable to reproduce this! I've removed `gcloud` from my path, but can't seem to make dbt fail with this type of error. Do you know if there are other specifics about your project required to make dbt fail in this way?
I'm running `dbt compile` in a hermetic, stripped-down environment that does not allow external RPCs.  There is an error message that suggests that `gcloud` needs to be in the PATH, but I suspect that the real failure is to to gcloud trying to make an external rpc and then failing.

I suspect that uninstalling gcloud could lead to an easy reproduction, although I don't know whether this will cause other unrelated failures.  Maybe trying unplugging the network cable and then running the `dbt compile` command?

In any case, I think the main issue is due to greedily creating a connection during the compile (in the generate function).  Jake's comment suggests that this could be approached via a lazy connection, although this was not implemented.

Ok, that makes sense. Fundamentally, this line of reasoning is a little divergent from how we think about model compilation. Right now, the `dbt compile` command just outputs the `select` statement that dbt will use during the model materialization in the `dbt run` step. Longer term though, I'd _love_ to make `dbt compile` output the full DDL/DML it would run at `dbt run`-time. This involves inspecting the state of the database and recording the existence (and types) of the relations that dbt intends to operate on. It just so happens that BigQuery didn't historically provide SQL-interfaces into its state, but we're planning on using things like the `information_schema`, for instance, going forwards to fetch information about the state of the database.

We could indeed lazily load connections, but I imagine that these future plans are necessarily going to involve running SQL against the warehouse in the `dbt compile` step, so that actually wouldn't be a super helpful change given your particular use case.

Can you just say a little bit more about why you're running `dbt compile` in a hermetic environment like this? Are you just validating that the project is compilable? Or are you grabbing the compiled output from the `target/` directory? Or something else?

I hesitate to recommend this, but one approach may be to connect to a locally-running postgres instance in your hermetic environment. The resulting SQL might be postgres-specific, but depending on your needs, that might be acceptable.

Edit: made this make more sense :)
What I'm trying to do with the `dbt compile` step is to just create the `manifest.json` file and then use that to link together an Airflow DAG in a similar manner to how this reference implementation uses the networkx gpickle file:  https://gist.github.com/adamhaney/916a21b0adcabef4038c38e3ac96a36f

I'm happy to have the SQL interpolation happen at run time, so in any case would want to defer actual network connects to the run phase.
Ok - that totally makes sense. It also helps me understand what https://github.com/fishtown-analytics/dbt/issues/1538 was all about too. Thanks for the additional info!

Maybe the right way to proceed here is to expose dbt's understanding of the graph after the parsing step? This seems like something we could support in a first-class way via the (eventual) Python API. dbt compiles projects in two steps:
1. parsing out the shape of the graph by evaluating `ref` and `config` function calls
2. "compiling" SQL with a fully-built out context and connections to the database

After parsing, dbt will have a complete understanding of the shape of the graph, but will not yet have "correct" SQL for each of the models. Sounds like that wouldn't be a problem given your use case.
In the short-term, I either need to revert back to version 0.13.1 or find a way to patch 0.14 to allow for lazily creating database connections.  Longer term, I'd be happy if there was either a Python API or a command-line 'parse' option to create the `manifest.json` file as a hermetic step (I have a slight preference for a Python API, since I'm already creating one myself by wrapping the dbt binary with a Python API).  Ultimately, I want to create a rich unit-testing environment that understands dependencies of each SQL file, but doesn't require creating any real database connections.  This testing environment needs to understand the DAG, which is why I need that build hermetically.

Is the lazy evaluation option still on the table?  I took out the database connection in the generate() function, but it failed on me later on.

Thanks!
I believe @beckjake implemented a new pattern for adapters in which each thread gets their own dedicated connection. I think the diff you're seeing here was a part of that change, though I'm not 100% certain.

Jake, this is a little outside of my wheelhouse: how involved do you think it is to make these adapters lazy again? Is it something that could conceivably happen for 0.14.1? Or are we better off supporting this use case via some other mechanism?
I don't love that we eagerly acquire the connection, and I am into trying to make it lazy load for 0.14.1

My thoughts were that "acquiring" a connection in the with block would mark it as ready to be acquired, and then on exit we'd only close it if it was acquired at some point (so the handle was non-`None`/state left `init`, something like that)

Thanks Jake!  Do you have a sense of when this might be ready?  I'm trying to decide whether to revert back to 0.13.1 or look into moving forward with patches against 0.14.
I think you might be best off reverting to 0.13.1 - we're wrapping up the release candidate for 0.14.0 presently, and I don't think this is a change we're going to be able to incorporate this late into the release cycle. Super happy to turn this around in short order after 0.14.0 drop though! Apologies for the inconvenience, but definitely looking to implement this change!
Sounds good -- thanks!

@beckjake Is there a tracking feature that I can follow to see progress on creating lazy connections?
@heisencoder I'll use this issue for that
Another thought on this same topic, would it make sense to address this issue by explicitly using a 'fake' version of the database?  In the context of BigQuery, this could be accomplished by creating a new `method` called something like 'fake'.  The 'fake' method would then have a fake implementation of the BigQuery interface (possibly using a MagicMock initially) that would not create any real connections, but would instead do something like log all the commands that would have been sent.  This could also be viewed as a 'dry-run' mode to allow developers to see what commands would be sent before making actual database changes.  This could be expressed as a --dry-run flag or as a new profile target (like 'fake').

Either of these approaches would make `compile` work inside my hermetic build and test environment.

Thoughts?
Hey @heisencoder - dbt requires a connection to the database in order to correctly compile SQL. If you have code like:

```sql
{{ config(materialized='incremental') }}

select * from my_big_table
{% if is_incremental() %}
  where event_time > (select max(event_time) from {{ this }}
{% endif %}
```

then the `is_incremental()` macro will require a query to the database in order to correctly compile this sql. When I think of `--dry-run`, I imagine that dbt would still make introspective queries to the database, but it would _not_ run any sort of mutative/creative queries.

If you're so inclined, you can create a [new "null" adapter](https://docs.getdbt.com/docs/building-a-new-adapter) for dbt which just implements the adapter interface and discards queries. I actually think that might be a kind of cool thing for dbt to ship with :)

Note: if you call `dbt compile` on your query, the compiled SQL is going to be pretty inaccurate. If you actually want to compile SQL with dbt, you'll need to have an open connection to the database. If you care more about building the graph with dbt, this could be a good option
Hi @drewbanin 

The main goal of compiling the SQL is to test the SQL in an isolated environment that doesn't have a database connection.  This would just be for simple syntax and schema testing, things that could be verified in a unit-test context.  It look like the is_incremental macro defaults to returning false if there is no database connection, which is probably fine for our purposes.

I'm assuming that if we're not using certain dbt power features that the compiled SQL will exactly match the executed SQL.  Are there other macros other than is_incremental that we need to avoid?

With regard to a "null" adapter, I'll probably need to create one to allow internally upgrading to 0.14.  I'll look into putting one together, but I may also need certain BigQuery features, so I'm also considering creating one that extends the existing BigQuery adapter and just replaces the connection with a fake object of some kind.

Thanks!
-Matt
The `is_incremental()` macro just proxies a call to `adapter.get_relation()` I believe. You may want to watch out for any other similar calls to `adapter` methods, but otherwise you should be in good shape.
Hi @drewbanin 

With regard to implementing a 'null' adapter, I think for my purposes I'll need to extend the existing bigquery adapter because I need the result of the `dbt compile` command to closely match that of existing bigquery adapter.  Is this something that you'd be interested in taking as an upstream eventually?
I think there could be merit to supporting a "null" adapter in dbt core, but probably not one which implements BigQuery-specific semantics. In general, we're aiming to have the output of `dbt compile` match exactly what dbt would actually execute in a `dbt run`, so I think some amount of querying/introspection is going to be required. Issue #588 does a good job of outlining the place we want to get to with dbt's approach to compilation if you're interesting in reading more about this.

I don't have a firm use-case for a generic null adapter in mind (it certainly would not output correct SQL for _any_ given database) -- maybe just as a debugging / testing sort of utility? So, feel free to do whatever fits your needs here!
Thank you Drew for your thoughts!  I'll go ahead and create an internal fake BigQuery adapter then.
This came up again today. We're going to prioritize this for an 0.15.x release - currently looking like 0.15.1.

The impetus for this is that acquiring connections greedily can be really slow. This impacts more real-time queries, like ones that can be submitted to a dbt server.

Resolving patch:
diff --git a/core/dbt/adapters/base/connections.py b/core/dbt/adapters/base/connections.py
--- a/core/dbt/adapters/base/connections.py
+++ b/core/dbt/adapters/base/connections.py
@@ -11,7 +11,7 @@
 import dbt.exceptions
 import dbt.flags
 from dbt.contracts.connection import (
-    Connection, Identifier, ConnectionState, AdapterRequiredConfig
+    Connection, Identifier, ConnectionState, AdapterRequiredConfig, LazyHandle
 )
 from dbt.adapters.base.query_headers import (
     QueryStringSetter, MacroQueryStringSetter,
@@ -61,6 +61,14 @@ def get_thread_connection(self) -> Connection:
                 )
             return self.thread_connections[key]
 
+    def set_thread_connection(self, conn):
+        key = self.get_thread_identifier()
+        if key in self.thread_connections:
+            raise dbt.exceptions.InternalException(
+                'In set_thread_connection, existing connection exists for {}'
+            )
+        self.thread_connections[key] = conn
+
     def get_if_exists(self) -> Optional[Connection]:
         key = self.get_thread_identifier()
         with self.lock:
@@ -109,8 +117,6 @@ def set_connection_name(self, name: Optional[str] = None) -> Connection:
             conn_name = name
 
         conn = self.get_if_exists()
-        thread_id_key = self.get_thread_identifier()
-
         if conn is None:
             conn = Connection(
                 type=Identifier(self.TYPE),
@@ -120,7 +126,7 @@ def set_connection_name(self, name: Optional[str] = None) -> Connection:
                 handle=None,
                 credentials=self.profile.credentials
             )
-            self.thread_connections[thread_id_key] = conn
+            self.set_thread_connection(conn)
 
         if conn.name == conn_name and conn.state == 'open':
             return conn
@@ -138,7 +144,7 @@ def set_connection_name(self, name: Optional[str] = None) -> Connection:
                 'Opening a new connection, currently in state {}'
                 .format(conn.state)
             )
-            self.open(conn)
+            conn.handle = LazyHandle(type(self))
 
         conn.name = conn_name
         return conn
diff --git a/core/dbt/adapters/base/impl.py b/core/dbt/adapters/base/impl.py
--- a/core/dbt/adapters/base/impl.py
+++ b/core/dbt/adapters/base/impl.py
@@ -238,11 +238,11 @@ def nice_connection_name(self) -> str:
     @contextmanager
     def connection_named(
         self, name: str, node: Optional[CompileResultNode] = None
-    ) -> Connection:
+    ) -> Iterator[None]:
         try:
             self.connections.query_header.set(name, node)
-            conn = self.acquire_connection(name)
-            yield conn
+            self.acquire_connection(name)
+            yield
         finally:
             self.release_connection()
             self.connections.query_header.reset()
@@ -250,9 +250,9 @@ def connection_named(
     @contextmanager
     def connection_for(
         self, node: CompileResultNode
-    ) -> Iterator[Connection]:
-        with self.connection_named(node.unique_id, node) as conn:
-            yield conn
+    ) -> Iterator[None]:
+        with self.connection_named(node.unique_id, node):
+            yield
 
     @available.parse(lambda *a, **k: ('', empty_table()))
     def execute(
diff --git a/core/dbt/contracts/connection.py b/core/dbt/contracts/connection.py
--- a/core/dbt/contracts/connection.py
+++ b/core/dbt/contracts/connection.py
@@ -2,7 +2,7 @@
 import itertools
 from dataclasses import dataclass, field
 from typing import (
-    Any, ClassVar, Dict, Tuple, Iterable, Optional, NewType, List
+    Any, ClassVar, Dict, Tuple, Iterable, Optional, NewType, List, Type
 )
 from typing_extensions import Protocol
 
@@ -12,6 +12,7 @@
 )
 
 from dbt.contracts.util import Replaceable
+from dbt.exceptions import InternalException
 from dbt.utils import translate_aliases
 
 
@@ -26,6 +27,23 @@ class ConnectionState(StrEnum):
     FAIL = 'fail'
 
 
+class ConnectionOpenerProtocol(Protocol):
+    @classmethod
+    def open(cls, connection: 'Connection') -> Any:
+        raise NotImplementedError(f'open() not implemented for {cls.__name__}')
+
+
+class LazyHandle:
+    """Opener must be a callable that takes a Connection object and opens the
+    connection, updating the handle on the Connection.
+    """
+    def __init__(self, opener: Type[ConnectionOpenerProtocol]):
+        self.opener = opener
+
+    def resolve(self, connection: 'Connection') -> Any:
+        return self.opener.open(connection)
+
+
 @dataclass(init=False)
 class Connection(ExtensibleJsonSchemaMixin, Replaceable):
     type: Identifier
@@ -62,6 +80,15 @@ def credentials(self, value):
 
     @property
     def handle(self):
+        if isinstance(self._handle, LazyHandle):
+            try:
+                # this will actually change 'self._handle'.
+                self._handle.resolve(self)
+            except RecursionError as exc:
+                raise InternalException(
+                    "A connection's open() method attempted to read the "
+                    "handle value"
+                ) from exc
         return self._handle
 
     @handle.setter


Test patch:
diff --git a/test/integration/032_concurrent_transaction_test/test_concurrent_transaction.py b/test/integration/032_concurrent_transaction_test/test_concurrent_transaction.py
--- a/test/integration/032_concurrent_transaction_test/test_concurrent_transaction.py
+++ b/test/integration/032_concurrent_transaction_test/test_concurrent_transaction.py
@@ -41,11 +41,12 @@ def project_config(self):
     def run_select_and_check(self, rel, sql):
         connection_name = '__test_{}'.format(id(threading.current_thread()))
         try:
-            with self._secret_adapter.connection_named(connection_name) as conn:
+            with self._secret_adapter.connection_named(connection_name):
+                conn = self._secret_adapter.connections.get_thread_connection()
                 res = self.run_sql_common(self.transform_sql(sql), 'one', conn)
 
             # The result is the output of f_sleep(), which is True
-            if res[0] == True:
+            if res[0]:
                 self.query_state[rel] = 'good'
             else:
                 self.query_state[rel] = 'bad'
diff --git a/test/integration/base.py b/test/integration/base.py
--- a/test/integration/base.py
+++ b/test/integration/base.py
@@ -753,7 +753,8 @@ def get_connection(self, name=None):
         if name is None:
             name = '__test'
         with patch.object(common, 'get_adapter', return_value=self.adapter):
-            with self.adapter.connection_named(name) as conn:
+            with self.adapter.connection_named(name):
+                conn = self.adapter.connections.get_thread_connection()
                 yield conn
 
     def get_relation_columns(self, relation):
diff --git a/test/rpc/util.py b/test/rpc/util.py
--- a/test/rpc/util.py
+++ b/test/rpc/util.py
@@ -519,7 +519,8 @@ def __init__(self, profiles_dir, which='run-operation', kwargs={}):
 
 
 def execute(adapter, sql):
-    with adapter.connection_named('rpc-tests') as conn:
+    with adapter.connection_named('rpc-tests'):
+        conn = adapter.connections.get_thread_connection()
         with conn.handle.cursor() as cursor:
             try:
                 cursor.execute(sql)
diff --git a/test/unit/test_bigquery_adapter.py b/test/unit/test_bigquery_adapter.py
--- a/test/unit/test_bigquery_adapter.py
+++ b/test/unit/test_bigquery_adapter.py
@@ -100,6 +100,8 @@ def test_acquire_connection_oauth_validations(self, mock_open_connection):
         except BaseException as e:
             raise
 
+        mock_open_connection.assert_not_called()
+        connection.handle
         mock_open_connection.assert_called_once()
 
     @patch('dbt.adapters.bigquery.BigQueryConnectionManager.open', return_value=_bq_conn())
@@ -115,6 +117,8 @@ def test_acquire_connection_service_account_validations(self, mock_open_connecti
         except BaseException as e:
             raise
 
+        mock_open_connection.assert_not_called()
+        connection.handle
         mock_open_connection.assert_called_once()
 
     @patch('dbt.adapters.bigquery.BigQueryConnectionManager.open', return_value=_bq_conn())
@@ -128,9 +132,8 @@ def test_acquire_connection_priority(self, mock_open_connection):
         except dbt.exceptions.ValidationException as e:
             self.fail('got ValidationException: {}'.format(str(e)))
 
-        except BaseException as e:
-            raise
-
+        mock_open_connection.assert_not_called()
+        connection.handle
         mock_open_connection.assert_called_once()
 
     def test_cancel_open_connections_empty(self):
@@ -158,8 +161,11 @@ def test_location_value(self, mock_bq, mock_auth_default):
         mock_auth_default.return_value = (creds, MagicMock())
         adapter = self.get_adapter('loc')
 
-        adapter.acquire_connection('dummy')
+        connection = adapter.acquire_connection('dummy')
         mock_client = mock_bq.Client
+
+        mock_client.assert_not_called()
+        connection.handle
         mock_client.assert_called_once_with('dbt-unit-000000', creds,
                                             location='Luna Station')
 
diff --git a/test/unit/test_postgres_adapter.py b/test/unit/test_postgres_adapter.py
--- a/test/unit/test_postgres_adapter.py
+++ b/test/unit/test_postgres_adapter.py
@@ -60,12 +60,17 @@ def test_acquire_connection_validations(self, psycopg2):
             self.fail('acquiring connection failed with unknown exception: {}'
                       .format(str(e)))
         self.assertEqual(connection.type, 'postgres')
+
+        psycopg2.connect.assert_not_called()
+        connection.handle
         psycopg2.connect.assert_called_once()
 
     @mock.patch('dbt.adapters.postgres.connections.psycopg2')
     def test_acquire_connection(self, psycopg2):
         connection = self.adapter.acquire_connection('dummy')
 
+        psycopg2.connect.assert_not_called()
+        connection.handle
         self.assertEqual(connection.state, 'open')
         self.assertNotEqual(connection.handle, None)
         psycopg2.connect.assert_called_once()
@@ -101,6 +106,8 @@ def test_cancel_open_connections_single(self):
     def test_default_keepalive(self, psycopg2):
         connection = self.adapter.acquire_connection('dummy')
 
+        psycopg2.connect.assert_not_called()
+        connection.handle
         psycopg2.connect.assert_called_once_with(
             dbname='postgres',
             user='root',
@@ -114,6 +121,8 @@ def test_changed_keepalive(self, psycopg2):
         self.config.credentials = self.config.credentials.replace(keepalives_idle=256)
         connection = self.adapter.acquire_connection('dummy')
 
+        psycopg2.connect.assert_not_called()
+        connection.handle
         psycopg2.connect.assert_called_once_with(
             dbname='postgres',
             user='root',
@@ -128,6 +137,8 @@ def test_search_path(self, psycopg2):
         self.config.credentials = self.config.credentials.replace(search_path="test")
         connection = self.adapter.acquire_connection('dummy')
 
+        psycopg2.connect.assert_not_called()
+        connection.handle
         psycopg2.connect.assert_called_once_with(
             dbname='postgres',
             user='root',
@@ -142,6 +153,8 @@ def test_schema_with_space(self, psycopg2):
         self.config.credentials = self.config.credentials.replace(search_path="test test")
         connection = self.adapter.acquire_connection('dummy')
 
+        psycopg2.connect.assert_not_called()
+        connection.handle
         psycopg2.connect.assert_called_once_with(
             dbname='postgres',
             user='root',
@@ -156,6 +169,8 @@ def test_set_zero_keepalive(self, psycopg2):
         self.config.credentials = self.config.credentials.replace(keepalives_idle=0)
         connection = self.adapter.acquire_connection('dummy')
 
+        psycopg2.connect.assert_not_called()
+        connection.handle
         psycopg2.connect.assert_called_once_with(
             dbname='postgres',
             user='root',
diff --git a/test/unit/test_redshift_adapter.py b/test/unit/test_redshift_adapter.py
--- a/test/unit/test_redshift_adapter.py
+++ b/test/unit/test_redshift_adapter.py
@@ -153,6 +153,8 @@ def test_cancel_open_connections_single(self):
     def test_default_keepalive(self, psycopg2):
         connection = self.adapter.acquire_connection('dummy')
 
+        psycopg2.connect.assert_not_called()
+        connection.handle
         psycopg2.connect.assert_called_once_with(
             dbname='redshift',
             user='root',
@@ -168,6 +170,8 @@ def test_changed_keepalive(self, psycopg2):
         self.config.credentials = self.config.credentials.replace(keepalives_idle=256)
         connection = self.adapter.acquire_connection('dummy')
 
+        psycopg2.connect.assert_not_called()
+        connection.handle
         psycopg2.connect.assert_called_once_with(
             dbname='redshift',
             user='root',
@@ -182,6 +186,8 @@ def test_search_path(self, psycopg2):
         self.config.credentials = self.config.credentials.replace(search_path="test")
         connection = self.adapter.acquire_connection('dummy')
 
+        psycopg2.connect.assert_not_called()
+        connection.handle
         psycopg2.connect.assert_called_once_with(
             dbname='redshift',
             user='root',
@@ -197,6 +203,8 @@ def test_search_path_with_space(self, psycopg2):
         self.config.credentials = self.config.credentials.replace(search_path="test test")
         connection = self.adapter.acquire_connection('dummy')
 
+        psycopg2.connect.assert_not_called()
+        connection.handle
         psycopg2.connect.assert_called_once_with(
             dbname='redshift',
             user='root',
@@ -212,6 +220,8 @@ def test_set_zero_keepalive(self, psycopg2):
         self.config.credentials = self.config.credentials.replace(keepalives_idle=0)
         connection = self.adapter.acquire_connection('dummy')
 
+        psycopg2.connect.assert_not_called()
+        connection.handle
         psycopg2.connect.assert_called_once_with(
             dbname='redshift',
             user='root',
diff --git a/test/unit/test_snowflake_adapter.py b/test/unit/test_snowflake_adapter.py
--- a/test/unit/test_snowflake_adapter.py
+++ b/test/unit/test_snowflake_adapter.py
@@ -230,7 +230,10 @@ def test_cancel_open_connections_single(self):
             add_query.assert_called_once_with('select system$abort_session(42)')
 
     def test_client_session_keep_alive_false_by_default(self):
-        self.adapter.connections.set_connection_name(name='new_connection_with_new_config')
+        conn = self.adapter.connections.set_connection_name(name='new_connection_with_new_config')
+
+        self.snowflake.assert_not_called()
+        conn.handle
         self.snowflake.assert_has_calls([
             mock.call(
                 account='test_account', autocommit=False,
@@ -243,8 +246,10 @@ def test_client_session_keep_alive_true(self):
         self.config.credentials = self.config.credentials.replace(
                                           client_session_keep_alive=True)
         self.adapter = SnowflakeAdapter(self.config)
-        self.adapter.connections.set_connection_name(name='new_connection_with_new_config')
+        conn = self.adapter.connections.set_connection_name(name='new_connection_with_new_config')
 
+        self.snowflake.assert_not_called()
+        conn.handle
         self.snowflake.assert_has_calls([
             mock.call(
                 account='test_account', autocommit=False,
@@ -258,8 +263,10 @@ def test_user_pass_authentication(self):
             password='test_password',
         )
         self.adapter = SnowflakeAdapter(self.config)
-        self.adapter.connections.set_connection_name(name='new_connection_with_new_config')
+        conn = self.adapter.connections.set_connection_name(name='new_connection_with_new_config')
 
+        self.snowflake.assert_not_called()
+        conn.handle
         self.snowflake.assert_has_calls([
             mock.call(
                 account='test_account', autocommit=False,
@@ -275,8 +282,10 @@ def test_authenticator_user_pass_authentication(self):
             authenticator='test_sso_url',
         )
         self.adapter = SnowflakeAdapter(self.config)
-        self.adapter.connections.set_connection_name(name='new_connection_with_new_config')
+        conn = self.adapter.connections.set_connection_name(name='new_connection_with_new_config')
 
+        self.snowflake.assert_not_called()
+        conn.handle
         self.snowflake.assert_has_calls([
             mock.call(
                 account='test_account', autocommit=False,
@@ -292,8 +301,10 @@ def test_authenticator_externalbrowser_authentication(self):
             authenticator='externalbrowser'
         )
         self.adapter = SnowflakeAdapter(self.config)
-        self.adapter.connections.set_connection_name(name='new_connection_with_new_config')
+        conn = self.adapter.connections.set_connection_name(name='new_connection_with_new_config')
 
+        self.snowflake.assert_not_called()
+        conn.handle
         self.snowflake.assert_has_calls([
             mock.call(
                 account='test_account', autocommit=False,
@@ -311,8 +322,10 @@ def test_authenticator_private_key_authentication(self, mock_get_private_key):
         )
 
         self.adapter = SnowflakeAdapter(self.config)
-        self.adapter.connections.set_connection_name(name='new_connection_with_new_config')
+        conn = self.adapter.connections.set_connection_name(name='new_connection_with_new_config')
 
+        self.snowflake.assert_not_called()
+        conn.handle
         self.snowflake.assert_has_calls([
             mock.call(
                 account='test_account', autocommit=False,