Repository: tobymao/sqlglot

Issue:
Unexpected failure when qualifying columns in BigQuery
Hey there, I am not sure if it is expected for this sort of query to fail here so please do let me know if that is the case or if I am doing something wrong. The query is able to run on Bigquery without issue. 

I have this (simplified) query here:
sql = """
WITH 
cte_1 as(
    SELECT
       a,
       b
    FROM 
        (select a from cte_1, unnest (l_1) as ul_1 with offset as t_1_col) evts
    LEFT JOIN
        (select b from cte_1, unnest (l_2) as ul_2 with offset as t_2_col) types
    ON  
        evts.t_1_col = types.t_2_col
)
select * from cte_2
"""

parsed_query = parse_one(sql, dialect="bigquery")
parsed_query = eliminate_subqueries(parsed_query)
qualified = qualify_columns(parsed_query, schema={}, infer_schema=True)

When I run this it gives me:
Traceback (most recent call last):
  File "/Users/pablobeltran/SS_REPOS_SL/kelp/scripts/validate_all_dbt_sql.py", line 278, in <module>
    qualified = qualify_columns(parsed_query, schema={}, infer_schema=True)
  File "/Users/pablobeltran/SS_REPOS_SL/kelp/kelp_dagster/.venv/lib/python3.10/site-packages/sqlglot/optimizer/qualify_columns.py", line 61, in qualify_columns
    _qualify_columns(scope, resolver)
  File "/Users/pablobeltran/SS_REPOS_SL/kelp/kelp_dagster/.venv/lib/python3.10/site-packages/sqlglot/optimizer/qualify_columns.py", line 330, in _qualify_columns
    raise OptimizeError(f"Unknown column: {column_name}")
sqlglot.errors.OptimizeError: Unknown column: t_1_col






Discussion:
Hmm think I messed something up when simplifying the query going to close for now
As mentioned in https://github.com/tobymao/sqlglot/blob/main/sqlglot/optimizer/optimizer.py#L68, you can't just run the rules in isolation or in any arbitrary order, there are certain invariants that are expected to be true and are enforced by the default rule order. The `qualify` rule in particular is very important.
> As mentioned in https://github.com/tobymao/sqlglot/blob/main/sqlglot/optimizer/optimizer.py#L68, you can't just run the rules in isolation or in any arbitrary order, there are certain invariants that are expected to be true and are enforced by the default rule order. The `qualify` rule in particular is very important.

Hey there thanks for the reply, I am still struggling with this bit of code I have correctly extracted the sql this time:
```
sql = """
WITH 
amd as (
    SELECT

        CAST(NULL as ARRAY<STRING>) uts,
        config as cj,
        Date('2023-06-20') as ds
    FROM
        project.dataset.table
),
md as (
    SELECT 
        cj,
        ds
    FROM amd
    LEFT JOIN unnest(uts) ut WITH offset off
),
funnel_lists as (
    SELECT
        ds,
        JSON_EXTRACT_STRING_ARRAY(cj, '$.some_list') el,
        JSON_EXTRACT_STRING_ARRAY(cj, '$.other_list') mt,
        cj
    FROM md
),
fr as(
    SELECT
        ets.ds,
        ets.er,
        CASE
            WHEN ts.et is not null THEN ts.et
            ELSE 'something'
        END as et
    FROM 
        (select * from funnel_lists, unnest (el) as evt with offset as er) ets
    LEFT JOIN
        (select * from funnel_lists, unnest (mt) as et with offset as tr) ts
    ON  
      ets.er = ts.tr
)
select * from fr
"""

parsed_query = parse_one(sql, dialect="bigquery")
optimized = optimize(parsed_query)
```

This time I called optimize directly (I have also tried calling qualify directly) and still see:
sqlglot.errors.OptimizeError: Unknown column: er


After digging in a bit it seems a bit related to the * in 'select * from funnel_lists, unnest (el) as evt with offset as er' as it doesnt seem to think that the * includes the 'er' column in bigquery.  Even when I specify: 

`select ds, er from funnel_lists, unnest (el) as evt with offset as er` it says that 'er' could not be resolved. Sorry if its another simple mistake I am making. 


Resolving patch:
diff --git a/sqlglot/expressions.py b/sqlglot/expressions.py
--- a/sqlglot/expressions.py
+++ b/sqlglot/expressions.py
@@ -2709,6 +2709,14 @@ class Unnest(UDTF):
         "offset": False,
     }
 
+    @property
+    def selects(self) -> t.List[Expression]:
+        columns = super().selects
+        offset = self.args.get("offset")
+        if offset:
+            columns = columns + [to_identifier("offset") if offset is True else offset]
+        return columns
+
 
 class Update(Expression):
     arg_types = {


Test patch:
diff --git a/tests/fixtures/optimizer/qualify_columns.sql b/tests/fixtures/optimizer/qualify_columns.sql
--- a/tests/fixtures/optimizer/qualify_columns.sql
+++ b/tests/fixtures/optimizer/qualify_columns.sql
@@ -441,6 +441,16 @@ SELECT t.aa AS aa FROM x AS x, UNNEST(x.a) AS t(aa);
 SELECT aa FROM x, UNNEST(a) AS aa;
 SELECT aa AS aa FROM x AS x, UNNEST(x.a) AS aa;
 
+# dialect: bigquery
+# execute: false
+select * from unnest ([1, 2]) as x with offset;
+SELECT x AS x, offset AS offset FROM UNNEST([1, 2]) AS x WITH OFFSET AS offset;
+
+# dialect: bigquery
+# execute: false
+select * from unnest ([1, 2]) as x with offset as y;
+SELECT x AS x, y AS y FROM UNNEST([1, 2]) AS x WITH OFFSET AS y;
+
 # dialect: presto
 SELECT x.a, i.b FROM x CROSS JOIN UNNEST(SPLIT(b, ',')) AS i(b);
 SELECT x.a AS a, i.b AS b FROM x AS x CROSS JOIN UNNEST(SPLIT(x.b, ',')) AS i(b);