Repository: ibis-project/ibis

Issue:
bug: order_by() with non-existent column does not error
### What happened?

I would expect order_by(DNE) to fail. But instead it silently gives me back the original unsorted table:

```python
import ibis
import pandas as pd

df = pd.DataFrame({"x": [3,1,2]})
t = ibis.memtable(df)
t.order_by("bogus")
┏━━━━━━━┓
┃ x     ┃
┡━━━━━━━┩
│ int64 │
├───────┤
│     3 │
│     1 │
│     2 │
└───────┘
```


### What version of ibis are you using?

4.0.0

### What backend(s) are you using, if any?

duckdb

### Relevant log output

_No response_

### Code of Conduct

- [X] I agree to follow this project's Code of Conduct


Discussion:
Thanks for the report and the minimal reproducible example! We should be able to fix this for 4.1, slated for a week or two from now.
Looking into this, there's a simple but likely not obvious explanation for what's happening:

The ibis input rule for the `sort_keys` field of `Selection` operations is basically: "try to find a column in the child table, otherwise treat the value as an expression"

The latter part of that validation

> otherwise treat the value as an expression

causes the sort key to be the constant string `'bogus'`.

This is equivalent to writing:

```sql
SELECT x
FROM t
ORDER BY 'bogus'
```

Indeed that is what is output by `ibis.show_sql`:

```
In [6]: ibis.show_sql(t.order_by("bogus"))
SELECT
  t0.x
FROM _ibis_memtable0 AS t0
ORDER BY
  CAST('bogus' AS TEXT)
```

Since ordering by a constant value is equivalent to no `ORDER BY` there's no effect on the input relation.

I think this is probably confusing and unexpected behavior even though it is correct.

I see a at least two options:

1. Disallow strings as arguments to `order_by`. I think this is inconsistent with the rest of the places where we convert strings to columns so it's probably not a good idea.
2. Allow strings as arguments, but require any literal strings to be wrapped in `ibis.literal`. This is currently how `t.select` works, and we would make `order_by` consistent with that.

One wrinkle here is that in ibis we allow the SQL ordinal shorthand syntax for `order_by`, which allows you to refer to columns in `ORDER BY` using one-based ordinal offsets. I think we can handle this by making the `ops.SortKey.arg` rule more specific than `rlz.any` 😅.

I personally am in favor of option 2.

Thanks for this report @NickCrews, it's a nice edge case find.
Yeah, option 2 makes sense to me. For `order_by(some_int)` case are you saying you'd also make that error for an integer literal that indexed > the number of columns (this would make sense to me as well).
Yes option 2 sounds great! I think a note on `ibis.literal` would be good to include in the docstring when that fix goes in.

Resolving patch:
diff --git a/ibis/expr/rules.py b/ibis/expr/rules.py
--- a/ibis/expr/rules.py
+++ b/ibis/expr/rules.py
@@ -134,11 +134,16 @@ def sort_key_from(table_ref, key, **kwargs):
     else:
         key, order = key, True
 
-    key = one_of(
-        (function_of(table_ref), column_from(table_ref), any),
-        key,
-        **kwargs,
-    )
+    if isinstance(key, (str, int)):
+        # Actual str/int keys must refer to columns in the table, we don't want
+        # to fallback to converting them to expressions with ibis.literal
+        key = column_from(table_ref, key, **kwargs)
+    else:
+        key = one_of(
+            (function_of(table_ref), column_from(table_ref), any),
+            key,
+            **kwargs,
+        )
 
     if isinstance(order, str):
         order = order.lower()


Test patch:
diff --git a/ibis/tests/expr/test_table.py b/ibis/tests/expr/test_table.py
--- a/ibis/tests/expr/test_table.py
+++ b/ibis/tests/expr/test_table.py
@@ -454,6 +454,36 @@ def test_order_by_scalar(table, key, expected):
     assert result.op().sort_keys == (ops.SortKey(expected),)
 
 
+@pytest.mark.parametrize(
+    "key",
+    [
+        "bogus",
+        ("bogus", False),
+        ibis.desc("bogus"),
+        1000,
+        (1000, False),
+        _.bogus,
+        _.bogus.desc(),
+    ],
+)
+@pytest.mark.parametrize(
+    "expr_func",
+    [
+        param(lambda t: t, id="table"),
+        param(lambda t: t.select("a", "b"), id="selection"),
+        param(lambda t: t.group_by("a").agg(new=_.b.sum()), id="aggregation"),
+    ],
+)
+def test_order_by_nonexistent_column_errors(table, expr_func, key):
+    # `order_by` is implemented on a few different operations, we check them
+    # all in turn here.
+    expr = expr_func(table)
+    # Depending on the expression, this can raise a few different errors.
+    # For now just check that an exception is raised.
+    with pytest.raises(Exception):
+        expr.order_by(key)
+
+
 def test_slice(table):
     expr = table[:5]
     expr2 = table[:5:1]