Repository: ibis-project/ibis

Issue:
Potential timezone bug in ibis-framework[postgres]
### What happened?

In upgrading from ibis-framework<7.0 to ibis-framework==7.2 many queries I had written suddenly started returning 
unexpected datetime values.  I believe this is due to a bug in the way timezones are being handled by the postgres ibis 
backend, but I can't be sure.

### What version of ibis are you using?

7.2.0

### What backend(s) are you using, if any?

postgres

### Relevant log output


I was unable to reproduce my exact bug, but I do have code the replicates what could be a related error.

I have a complete docker set up to replicate the behavior here:
[Here is the github repo](https://github.com/robdmc/ibis_debug)

In re-reading my notebook I see a couple of comments that come across as grumbly.  I apologize in advance.  I created 
this at the end of a very long frustrating day of bug-hunting.  :)

I am deeply grateful for the work you all are doing with ibis.  I LOVE this project.  Thank you.



### Code of Conduct

- [X] I agree to follow this project's Code of Conduct


Discussion:
@robdmc Thanks for the bug report, reproducer and and the kind words :)

This looks like a bug, but I'll look into it and report back.
@robdmc I took a look at the issue here.

The cast you show:

```sql
CAST(t4.case_started AT TIME ZONE 'UTC' AS TIMESTAMPTZ) AS case_started
```

is indeed strange.

However, I am not able to reproduce this SQL using this code:

```python
import ibis


def test_timezone_from_column():
    con = ibis.postgres.connect(
        user="postgres",
        password="postgres",
        host="localhost",
        port=5432,
        database="ibis_testing",
    )

    with con.begin() as c:
        c.exec_driver_sql(
            """
            CREATE TEMPORARY TABLE cs_case (
                id SERIAL PRIMARY KEY,
                when_started TIMESTAMP WITH TIME ZONE,
                visit_id INTEGER
            );

            INSERT INTO cs_case (when_started, visit_id) VALUES
                ('2018-01-01 00:00:00+00', 1),
                ('2018-01-01 01:00:00+00', 1),
                ('2018-01-01 02:00:00+00', 1);

            CREATE TEMPORARY TABLE vis (
                id SERIAL PRIMARY KEY,
                submitted_by_id INTEGER
            );

            INSERT INTO vis (submitted_by_id) VALUES (1), (1), (2);
            """
        )

    case = con.table("cs_case")
    case = case.filter(case.id.isin([1, 2, 3]))
    case = case.select("id", "when_started", "visit_id")
    case = case.rename(case_id="id", case_started="when_started")

    # Generate an expression for a table with information I want joined
    visit = con.table("vis")
    visit = visit[["id", "submitted_by_id"]]
    visit = visit.rename(visit_id="id")

    # WEIRD THINGS HAPPEND HERE.  If I comment out this join, the "when_started" field
    # is returned with the expected timezone information.  Running this join mysteriously
    # keeps the timezones the same but shifts the underlying time values  by 5 hours.
    # This is the expected offset between UTC and America/New_York timezone.
    # THIS BEHAVIOR DOES NOT HAPPEN IN ibis-frame<7.0.0
    case = case.left_join(visit, "visit_id")

    sql = str(ibis.to_sql(case))
    assert "AT TIME ZONE 'UTC' AS TIMESTAMPTZ" not in sql
```

This test passes for me, but it should fail if the test successfully reproduces the problem.

A couple bits of information that might help me get to a reproducible failure:

1. The postgres schema of `cusomterservicecase` (intentional typo as that's what's in the notebook). You can get this information with `\d table_name` using the `psql` CLI.
2. The ibis schema of that table via `conn.table("cusomterservicecase").schema()`
I did some spelunking and found the original (ancient) PR where I added the code that slaps `AT TIME ZONE` on column accesses https://github.com/ibis-project/ibis/pull/1048.

That said, I haven't determined where the `CAST` around the `AT TIME ZONE` is coming from.
@robdmc Another thing that would be useful is the locked requirements.txt from the **working** version.
@cpcloud I can share with you the exact schemas I'm working with, but I don't think I should do that in a public forum.  I'm happy to get those to you via your preferred DM channel.  I just sent you linkedIn invite in case that is useful.
@robdmc You can also DM me on our Zulip chat at https://ibis-project.zulipchat.com/

Resolving patch:
diff --git a/ibis/backends/postgres/registry.py b/ibis/backends/postgres/registry.py
--- a/ibis/backends/postgres/registry.py
+++ b/ibis/backends/postgres/registry.py
@@ -309,11 +309,6 @@ def _table_column(t, op):
     sa_table = get_sqla_table(ctx, table)
     out_expr = get_col(sa_table, op)
 
-    if op.dtype.is_timestamp():
-        timezone = op.dtype.timezone
-        if timezone is not None:
-            out_expr = out_expr.op("AT TIME ZONE")(timezone).label(op.name)
-
     # If the column does not originate from the table set in the current SELECT
     # context, we should format as a subquery
     if t.permit_subquery and ctx.is_foreign_expr(table):


Test patch:
diff --git a/ibis/backends/postgres/tests/snapshots/test_client/test_timezone_from_column/out.sql b/ibis/backends/postgres/tests/snapshots/test_client/test_timezone_from_column/out.sql
new file mode 100644
--- /dev/null
+++ b/ibis/backends/postgres/tests/snapshots/test_client/test_timezone_from_column/out.sql
@@ -0,0 +1,15 @@
+WITH t0 AS (
+  SELECT
+    t2.id AS id,
+    t2.ts_tz AS tz,
+    t2.ts_no_tz AS no_tz
+  FROM x AS t2
+)
+SELECT
+  t0.id,
+  CAST(t0.tz AS TIMESTAMPTZ) AS tz,
+  CAST(t0.no_tz AS TIMESTAMP) AS no_tz,
+  t1.id AS id_right
+FROM t0
+LEFT OUTER JOIN y AS t1
+  ON t0.id = t1.id
\ No newline at end of file
diff --git a/ibis/backends/postgres/tests/test_client.py b/ibis/backends/postgres/tests/test_client.py
--- a/ibis/backends/postgres/tests/test_client.py
+++ b/ibis/backends/postgres/tests/test_client.py
@@ -17,6 +17,7 @@
 
 import numpy as np
 import pandas as pd
+import pandas.testing as tm
 import pytest
 from pytest import param
 
@@ -223,3 +224,51 @@ def test_insert_with_cte(con):
 def test_connect_url_with_empty_host():
     con = ibis.connect("postgres:///ibis_testing")
     assert con.con.url.host is None
+
+
+@pytest.fixture(scope="module")
+def contz(con):
+    with con.begin() as c:
+        tz = c.exec_driver_sql("SHOW TIMEZONE").scalar()
+        c.exec_driver_sql("SET TIMEZONE TO 'America/New_York'")
+
+    yield con
+
+    with con.begin() as c:
+        c.exec_driver_sql(f"SET TIMEZONE TO '{tz}'")
+
+
+def test_timezone_from_column(contz, snapshot):
+    with contz.begin() as c:
+        c.exec_driver_sql(
+            """
+            CREATE TEMPORARY TABLE x (
+                id BIGINT,
+                ts_tz TIMESTAMP WITH TIME ZONE NOT NULL,
+                ts_no_tz TIMESTAMP WITHOUT TIME ZONE NOT NULL
+            );
+
+            INSERT INTO x VALUES
+                (1, '2018-01-01 00:00:01+00', '2018-01-01 00:00:02');
+
+            CREATE TEMPORARY TABLE y AS SELECT 1::BIGINT AS id;
+            """
+        )
+
+    case = (
+        contz.table("x")
+        .rename(tz="ts_tz", no_tz="ts_no_tz")
+        .left_join(contz.table("y"), "id")
+    )
+
+    result = case.execute()
+    expected = pd.DataFrame(
+        {
+            "id": [1],
+            "tz": [pd.Timestamp("2018-01-01 00:00:01", tz="UTC")],
+            "no_tz": [pd.Timestamp("2018-01-01 00:00:02")],
+            "id_right": [1],
+        }
+    )
+    tm.assert_frame_equal(result, expected)
+    snapshot.assert_match(ibis.to_sql(case), "out.sql")