Repository: ibis-project/ibis

Issue:
bug (or maybe feature request), bigquery: compile() doesn't include fully qualified table ID for memtable unless the expression is executed()
### What happened?

After 7.1.0 with the BigQuery backend, we must call `execute()` before `compile()` can generate valid SQL.

```
In [1]: import ibis; import pandas as pd

In [2]: bq = ibis.bigquery.connect(project_id="swast-scratch")

In [3]: t = ibis.memtable(pd.DataFrame({"a": [1, 2, 3], "b": [4, 5, 6]}), schema=ibis.schema({"a": "int64", "b": "int64"}))

In [7]: bq.compile(t)
Out[7]: 'SELECT\n  t0.*\nFROM _ibis_pandas_memtable_egy3axr6nbbhdh2dcd423hu45m AS t0'
```

This table in the compiled SQL doesn't actually exist (I believe not anyway) and it's not until I call `execute()` that the table ID becomes fully qualified.

```
In [3]: bq.execute(t)
Out[3]: 
   a  b
0  1  4
1  2  5
2  3  6

In [4]: bq.compile(t)
Out[4]: 'SELECT\n  t0.*\nFROM swast-scratch._63cfa399614a54153cc386c27d6c0c6fdb249f9e._ibis_pandas_memtable_g7qvufcjtbcqvpqyawnqzxv46a AS t0'
```

I'd love for a "compile" that makes sure all memtables are available in the engine first. Alternatively, an "execute" mode that does everything except run the actual query.

### What version of ibis are you using?

7.1.0

### What backend(s) are you using, if any?

BigQuery

### Relevant log output

_No response_

### Code of Conduct

- [X] I agree to follow this project's Code of Conduct


Discussion:
Thanks for the report @tswast !  I think this is related to the switch over to removing the `session` and some attributes that we expect to be populated are coming up as `None`.
Probably something in here: https://github.com/ibis-project/ibis/blob/master/ibis/backends/bigquery/__init__.py#L621-L631

I can take a look later tonight or tomorrow morning.
Hmm, yeah.  So we can fix this by calling `_make_session` before we call `compile` -- is that an acceptable trade-off?

Or should we just go ahead and call ` _make_session` when we establish the connection in the first place?  (Also, we DID remove session usage, some of the method names are still in place, though)
I think `_make_session` before `compile` would make sense to me.

Resolving patch:
diff --git a/ibis/backends/bigquery/__init__.py b/ibis/backends/bigquery/__init__.py
--- a/ibis/backends/bigquery/__init__.py
+++ b/ibis/backends/bigquery/__init__.py
@@ -558,9 +558,12 @@ def table(
         return rename_partitioned_column(table_expr, bq_table, self.partition_column)
 
     def _make_session(self) -> tuple[str, str]:
-        if self._session_dataset is None:
+        if (
+            self._session_dataset is None
+            and (client := getattr(self, "client", None)) is not None
+        ):
             job_config = bq.QueryJobConfig(use_query_cache=False)
-            query = self.client.query(
+            query = client.query(
                 "SELECT 1", job_config=job_config, project=self.billing_project
             )
             query.result()
@@ -615,6 +618,7 @@ def compile(
             The output of compilation. The type of this value depends on the
             backend.
         """
+        self._make_session()
         self._define_udf_translation_rules(expr)
         sql = self.compiler.to_ast_ensure_limit(expr, limit, params=params).compile()
 


Test patch:
diff --git a/ibis/backends/bigquery/tests/system/test_client.py b/ibis/backends/bigquery/tests/system/test_client.py
--- a/ibis/backends/bigquery/tests/system/test_client.py
+++ b/ibis/backends/bigquery/tests/system/test_client.py
@@ -389,6 +389,22 @@ def test_fully_qualified_table_creation(con, project_id, dataset_id, temp_table)
     assert t.get_name() == f"{project_id}.{dataset_id}.{temp_table}"
 
 
+def test_fully_qualified_memtable_compile(project_id, dataset_id):
+    new_bq_con = ibis.bigquery.connect(project_id=project_id, dataset_id=dataset_id)
+    # New connection shouldn't have _session_dataset populated after connection
+    assert new_bq_con._session_dataset is None
+
+    t = ibis.memtable(
+        {"a": [1, 2, 3], "b": [4, 5, 6]},
+        schema=ibis.schema({"a": "int64", "b": "int64"}),
+    )
+
+    # call to compile should fill in _session_dataset
+    sql = new_bq_con.compile(t)
+    assert new_bq_con._session_dataset is not None
+    assert project_id in sql
+
+
 def test_create_table_with_options(con):
     name = gen_name("bigquery_temp_table")
     schema = ibis.schema(dict(a="int64", b="int64", c="array<string>", d="date"))