Repository: ibis-project/ibis

Issue:
bug: inner Deferred binds to wrong thing in `Table.select(_.x.map(_.b))`
### What happened?

See https://github.com/ibis-project/ibis/pull/8552

I think what is happening:
1. ibis encounters the _.x when evaluating the .select
2. it tries to bind it to the parent expression, the Table
3. then it recurses down into the deferred to further evaluate it
4. It encounters the _.b
5. It isn't smart enough to understand that this now refers to each array element. It just binds it to that outer Table
6. raises AttributeError

This seems like it might be a doozy to solve. Would need to make the idea of a context much more nuanced, with nested scopes??

### What version of ibis are you using?

main

### What backend(s) are you using, if any?

_No response_

### Relevant log output

_No response_

### Code of Conduct

- [X] I agree to follow this project's Code of Conduct


Discussion:
I think this is pretty tricky 🤓

There's also no reason to prevent use of underscore inside of a lambda afaict.

I think we're limited in our options.
> use of underscore inside of a lambda afaict.

I added a xfail example for this in the linked PR. It fails for a different reason though I think?

> I think we're limited in our options.

Yeah I bet. In these examples, we get an error. But I think it would be possible to make a query that doesn't crash, but just insidiously gives the wrong result (ie both the table and the struct have a field called "y", we wanted to reference struct.y, but it compiled to table.y). At the least we need to ensure that this doesn't happen. I'm fine with getting rid of the nice feature if that is the only way to get this correctness.

Resolving patch:
diff --git a/ibis/expr/types/arrays.py b/ibis/expr/types/arrays.py
--- a/ibis/expr/types/arrays.py
+++ b/ibis/expr/types/arrays.py
@@ -445,15 +445,19 @@ def map(self, func: Deferred | Callable[[ir.Value], ir.Value]) -> ir.ArrayValue:
         """
         if isinstance(func, Deferred):
             name = "_"
-        else:
+            resolve = func.resolve
+        elif callable(func):
             name = next(iter(inspect.signature(func).parameters.keys()))
+            resolve = func
+        else:
+            raise TypeError(
+                f"`func` must be a Deferred or Callable, got `{type(func).__name__}`"
+            )
+
         parameter = ops.Argument(
             name=name, shape=self.op().shape, dtype=self.type().value_type
         )
-        if isinstance(func, Deferred):
-            body = func.resolve(parameter.to_expr())
-        else:
-            body = func(parameter.to_expr())
+        body = resolve(parameter.to_expr())
         return ops.ArrayMap(self, param=parameter.param, body=body).to_expr()
 
     def filter(
@@ -545,17 +549,20 @@ def filter(
         """
         if isinstance(predicate, Deferred):
             name = "_"
-        else:
+            resolve = predicate.resolve
+        elif callable(predicate):
             name = next(iter(inspect.signature(predicate).parameters.keys()))
+            resolve = predicate
+        else:
+            raise TypeError(
+                f"`predicate` must be a Deferred or Callable, got `{type(predicate).__name__}`"
+            )
         parameter = ops.Argument(
             name=name,
             shape=self.op().shape,
             dtype=self.type().value_type,
         )
-        if isinstance(predicate, Deferred):
-            body = predicate.resolve(parameter.to_expr())
-        else:
-            body = predicate(parameter.to_expr())
+        body = resolve(parameter.to_expr())
         return ops.ArrayFilter(self, param=parameter.param, body=body).to_expr()
 
     def contains(self, other: ir.Value) -> ir.BooleanValue:


Test patch:
diff --git a/ibis/tests/expr/test_value_exprs.py b/ibis/tests/expr/test_value_exprs.py
--- a/ibis/tests/expr/test_value_exprs.py
+++ b/ibis/tests/expr/test_value_exprs.py
@@ -1533,52 +1533,36 @@ def test_array_length_scalar():
     assert isinstance(expr.op(), ops.ArrayLength)
 
 
-def double_int(x):
-    return x * 2
-
-
-def double_float(x):
-    return x * 2.0
-
-
-def is_negative(x):
-    return x < 0
-
-
 def test_array_map():
     arr = ibis.array([1, 2, 3])
 
-    result_int = arr.map(double_int)
-    result_float = arr.map(double_float)
+    r1 = arr.map(_ * 2)
+    r2 = arr.map(lambda x: x * 2.0)
+    r3 = arr.map(functools.partial(lambda a, b: a + b, b=2))
 
-    assert result_int.type() == dt.Array(dt.int16)
-    assert result_float.type() == dt.Array(dt.float64)
+    assert r1.type() == dt.Array(dt.int16)
+    assert r2.type() == dt.Array(dt.float64)
+    assert r3.type() == dt.Array(dt.int16)
 
-
-def test_array_map_partial():
-    arr = ibis.array([1, 2, 3])
-
-    def add(x, y):
-        return x + y
-
-    result = arr.map(functools.partial(add, y=2))
-    assert result.type() == dt.Array(dt.int16)
+    with pytest.raises(TypeError, match="must be a Deferred or Callable"):
+        # Non-deferred expressions aren't allowed
+        arr.map(arr[0])
 
 
 def test_array_filter():
     arr = ibis.array([1, 2, 3])
-    result = arr.filter(is_negative)
-    assert result.type() == arr.type()
-
 
-def test_array_filter_partial():
-    arr = ibis.array([1, 2, 3])
+    r1 = arr.filter(lambda x: x < 0)
+    r2 = arr.filter(_ < 0)
+    r3 = arr.filter(functools.partial(lambda a, b: a == b, b=2))
 
-    def equal(x, y):
-        return x == y
+    assert r1.type() == arr.type()
+    assert r2.type() == arr.type()
+    assert r3.type() == arr.type()
 
-    result = arr.filter(functools.partial(equal, y=2))
-    assert result.type() == arr.type()
+    with pytest.raises(TypeError, match="must be a Deferred or Callable"):
+        # Non-deferred expressions aren't allowed
+        arr.filter(arr[0])
 
 
 @pytest.mark.parametrize(