internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: ibis-project/ibis

Issue:
bug: pivot_longer() doesn't support iterable of strings
### What happened?

The docstring claims this should work, but it only works with a single str or a full fledged selector. 

PS I love the names pivot_wider and pivot_longer. So much better than the melt and explode. Thanks for adding this function, this would have been a doozy for me to implement myself.

### What version of ibis are you using?

main

### What backend(s) are you using, if any?

NA

### Relevant log output

_No response_

### Code of Conduct

- [X] I agree to follow this project's Code of Conduct


Discussion:
Seems easy enough to remove this part of the docstring!

Resolving patch:
diff --git a/ibis/expr/types/relations.py b/ibis/expr/types/relations.py
--- a/ibis/expr/types/relations.py
+++ b/ibis/expr/types/relations.py
@@ -2800,7 +2800,7 @@ def cache(self) -> Table:
 
     def pivot_longer(
         self,
-        cols: str | s.Selector,
+        col: str | s.Selector,
         *,
         names_to: str | Iterable[str] = "name",
         names_pattern: str | re.Pattern = r"(.+)",
@@ -2814,8 +2814,8 @@ def pivot_longer(
 
         Parameters
         ----------
-        cols
-            String column names or selectors.
+        col
+            String column name or selector.
         names_to
             A string or iterable of strings indicating how to name the new
             pivoted columns.
@@ -3066,7 +3066,7 @@ def pivot_longer(
         """
         import ibis.selectors as s
 
-        pivot_sel = s.c(cols) if isinstance(cols, str) else cols
+        pivot_sel = s.c(col) if isinstance(col, str) else col
 
         pivot_cols = pivot_sel.expand(self)
         if not pivot_cols:


Test patch:
Repository: ibis-project/ibis

Issue:
bug: Error messages from impala are hidden behind "Operation is in ERROR_STATE"
### What happened?

Impala queries that result in an error state do not propagate the impala error message. The error is replaced with "Operation is in ERROR_STATE" in all cases, as seen here:
https://github.com/ibis-project/ibis/blob/master/ibis/backends/impala/client.py#L182


Impyla's approach does not conceal the error:
https://github.com/cloudera/impyla/blob/master/impala/hiveserver2.py#L428

I suggest a change to `ImpalaCursor._wait_synchronous` to just rely on impyla for this, similar to the following:

```python
    def _wait_synchronous(self):
        try:
            self._cursor._wait_to_finish()
        except KeyboardInterrupt:
            util.log('Canceling query')
            self.cancel()
            raise
```

Another option is to include the error text in the exception ourselves, but this does rely more on impyla internals:

```python
raise OperationalError(self._cursor._last_operation.get_state().errorMessage)
```

### What version of ibis are you using?

2.1.1 (though this appears to apply to master as well)

### What backend(s) are you using, if any?

Impala

### Relevant log output

_No response_

### Code of Conduct

- [X] I agree to follow this project's Code of Conduct


Discussion:
Hi @dankrause-nokia 👋🏻!

Thanks for the report.

I think you've got most of a pull request here. Do you mind trying to submit that? Happy to help review.

Resolving patch:
diff --git a/ibis/backends/impala/client.py b/ibis/backends/impala/client.py
--- a/ibis/backends/impala/client.py
+++ b/ibis/backends/impala/client.py
@@ -1,7 +1,6 @@
 from __future__ import annotations
 
 import contextlib
-import time
 import traceback
 from typing import TYPE_CHECKING
 
@@ -158,31 +157,8 @@ def execute(self, stmt):
 
     def _wait_synchronous(self):
         # Wait to finish, but cancel if KeyboardInterrupt
-        from impala.hiveserver2 import OperationalError
-
-        loop_start = time.time()
-
-        def _sleep_interval(start_time):
-            elapsed = time.time() - start_time
-            if elapsed < 0.05:
-                return 0.01
-            elif elapsed < 1.0:
-                return 0.05
-            elif elapsed < 10.0:
-                return 0.1
-            elif elapsed < 60.0:
-                return 0.5
-            return 1.0
-
-        cur = self._cursor
         try:
-            while True:
-                state = cur.status()
-                if self._cursor._op_state_is_error(state):
-                    raise OperationalError("Operation is in ERROR_STATE")
-                if not cur._op_state_is_executing(state):
-                    break
-                time.sleep(_sleep_interval(loop_start))
+            self._cursor._wait_to_finish()
         except KeyboardInterrupt:
             util.log('Canceling query')
             self.cancel()


Test patch:
Repository: ibis-project/ibis

Issue:
bug(duckdb): type parser doesn't respect quotes for struct field names
### What happened?

The duckdb type parser doesn't respect quotes for struct field names:

```
In [2]: import ibis.backends.duckdb.datatypes as ddt

In [3]: ddt.parse('struct("a b" int)')
...
ParseError: expected one of ')', '[a-zA-Z_][a-zA-Z_0-9]*' at 0:7
```

### What version of ibis are you using?

`master`

### What backend(s) are you using, if any?

DuckDB

### Relevant log output

_No response_

### Code of Conduct

- [X] I agree to follow this project's Code of Conduct


Discussion:

Resolving patch:
diff --git a/ibis/backends/duckdb/datatypes.py b/ibis/backends/duckdb/datatypes.py
--- a/ibis/backends/duckdb/datatypes.py
+++ b/ibis/backends/duckdb/datatypes.py
@@ -14,6 +14,7 @@
     LBRACKET,
     LPAREN,
     PRECISION,
+    RAW_STRING,
     RBRACKET,
     RPAREN,
     SCALE,
@@ -80,10 +81,12 @@ def parse(text: str, default_decimal_parameters=(18, 3)) -> dt.DataType:
         .skip(RPAREN)
     )
 
+    field = spaceless(parsy.alt(FIELD, RAW_STRING))
+
     struct = (
         spaceless_string("struct")
         .then(LPAREN)
-        .then(parsy.seq(spaceless(FIELD), ty).sep_by(COMMA).map(dt.Struct.from_tuples))
+        .then(parsy.seq(field, ty).sep_by(COMMA).map(dt.Struct.from_tuples))
         .skip(RPAREN)
     )
 


Test patch:
diff --git a/ibis/backends/duckdb/tests/test_datatypes.py b/ibis/backends/duckdb/tests/test_datatypes.py
--- a/ibis/backends/duckdb/tests/test_datatypes.py
+++ b/ibis/backends/duckdb/tests/test_datatypes.py
@@ -111,3 +111,11 @@ def test_null_dtype():
         match="DuckDB cannot yet reliably handle `null` typed columns",
     ):
         con.execute(t)
+
+
+def test_parse_quoted_struct_field():
+    import ibis.backends.duckdb.datatypes as ddt
+
+    assert ddt.parse('STRUCT("a" INT, "a b c" INT)') == dt.Struct(
+        {"a": dt.int32, "a b c": dt.int32}
+    )
Repository: ibis-project/ibis

Issue:
bug: .distinct(on="does_not_exist") does not crash
### What happened?

```python
ibis.examples.topo.fetch().distinct(on="does_not_exist")
```

I would expect this to error.

### What version of ibis are you using?

5.1.0, have not tested on main

### What backend(s) are you using, if any?

duckdb, but probably irrelevant?

### Relevant log output

_No response_

### Code of Conduct

- [X] I agree to follow this project's Code of Conduct


Discussion:
I think the problem is in the `GroupedTable` [constructor](https://github.com/ibis-project/ibis/blob/ab758bd0627c0ea9987aed65f8812c00c08b3656/ibis/expr/types/groupby.py#L70), it allows `self.by` to be empty:
```python

    def __init__(
        self, table, by, having=None, order_by=None, window=None, **expressions
    ):
        self.table = table
        self.by = list(
            itertools.chain(
                itertools.chain.from_iterable(
                    _get_group_by_key(table, v) for v in util.promote_list(by)
                ),
                (
                    expr.name(k)
                    for k, v in expressions.items()
                    for expr in _get_group_by_key(table, v)
                ),
            )
        )
        self._order_by = order_by or []
        self._having = having or []
        self._window = window
```

Resolving patch:
diff --git a/ibis/expr/types/groupby.py b/ibis/expr/types/groupby.py
--- a/ibis/expr/types/groupby.py
+++ b/ibis/expr/types/groupby.py
@@ -28,6 +28,7 @@
 from ibis.expr.deferred import Deferred
 from ibis.selectors import Selector
 from ibis.expr.types.relations import bind_expr
+import ibis.common.exceptions as com
 
 _function_types = tuple(
     filter(
@@ -79,6 +80,10 @@ def __init__(
                 ),
             )
         )
+
+        if not self.by:
+            raise com.IbisInputError("The grouping keys list is empty")
+
         self._order_by = order_by or []
         self._having = having or []
         self._window = window


Test patch:
diff --git a/ibis/tests/expr/test_table.py b/ibis/tests/expr/test_table.py
--- a/ibis/tests/expr/test_table.py
+++ b/ibis/tests/expr/test_table.py
@@ -1447,6 +1447,13 @@ def test_group_by_key_function():
     assert expr.columns == ['new_key', 'foo']
 
 
+def test_group_by_no_keys():
+    t = ibis.table([('a', 'timestamp'), ('b', 'string'), ('c', 'double')])
+
+    with pytest.raises(com.IbisInputError):
+        t.group_by(s.startswith("x")).aggregate(foo=t.c.mean())
+
+
 def test_unbound_table_name():
     t = ibis.table([('a', 'timestamp')])
     name = t.op().name
@@ -1812,3 +1819,9 @@ def test_invalid_keep_distinct():
     t = ibis.table(dict(a="int", b="string"), name="t")
     with pytest.raises(com.IbisError, match="Invalid value for `keep`:"):
         t.distinct(on="a", keep="invalid")
+
+
+def test_invalid_distinct_empty_key():
+    t = ibis.table(dict(a="int", b="string"), name="t")
+    with pytest.raises(com.IbisInputError):
+        t.distinct(on="c", keep="first")
Repository: ibis-project/ibis

Issue:
bug: `pivot_longer` doesn't work with field names containing spaces
### What happened?

when using pivot_longer, the names_to field and values_to arguments can't have a space in them

### What version of ibis are you using?

5.1.0

### What backend(s) are you using, if any?

DuckDB

### Relevant log output

```sh
>>> import ibis
>>> import ibis.selectors as s
>>> from ibis import _
>>> ibis.options.interactive = True
>>> relig_income = ibis.examples.relig_income_raw.fetch()
>>> relig_income.pivot_longer(~s.c("religion"), names_to="in come", values_to="my count")

This produces a ParserException: Parser Error: syntax error at or near "in"
LINE 2: ...e" := ?, "my count" := t1."$100-150k"), struct_pack("in come" := ?, "my count"...

This can be fixed by removing the space, but would be nice to at least fix the error message or documentation to say there should be no spaces in the names and values
```


### Code of Conduct

- [X] I agree to follow this project's Code of Conduct


Discussion:
@wleescor Welcome! 👋🏻 

Thanks for the report and the easy-to-reproduce failure!

Slated for 6.0.
The error appears to be in the `STRUCT` part of the `CAST` expression we're generating:

```
STRUCT(in come TEXT, my count BIGINT)[]
```

should be

```
STRUCT("in come" TEXT, "my count" BIGINT)[]
```

Resolving patch:
diff --git a/ibis/backends/base/sql/alchemy/datatypes.py b/ibis/backends/base/sql/alchemy/datatypes.py
--- a/ibis/backends/base/sql/alchemy/datatypes.py
+++ b/ibis/backends/base/sql/alchemy/datatypes.py
@@ -47,8 +47,9 @@ def __init__(self, fields: Mapping[str, sat.TypeEngine]) -> None:
 
 @compiles(StructType, "default")
 def compiles_struct(element, compiler, **kw):
+    quote = compiler.dialect.identifier_preparer.quote
     content = ", ".join(
-        f"{field} {compiler.process(typ, **kw)}"
+        f"{quote(field)} {compiler.process(typ, **kw)}"
         for field, typ in element.fields.items()
     )
     return f"STRUCT({content})"


Test patch:
diff --git a/ibis/backends/duckdb/tests/test_datatypes.py b/ibis/backends/duckdb/tests/test_datatypes.py
--- a/ibis/backends/duckdb/tests/test_datatypes.py
+++ b/ibis/backends/duckdb/tests/test_datatypes.py
@@ -1,8 +1,11 @@
+import duckdb_engine
 import pytest
+import sqlalchemy as sa
 import sqlglot as sg
 from packaging.version import parse as vparse
 from pytest import param
 
+import ibis.backends.base.sql.alchemy.datatypes as sat
 import ibis.common.exceptions as exc
 import ibis.expr.datatypes as dt
 from ibis.backends.duckdb.datatypes import parse
@@ -119,3 +122,12 @@ def test_parse_quoted_struct_field():
     assert ddt.parse('STRUCT("a" INT, "a b c" INT)') == dt.Struct(
         {"a": dt.int32, "a b c": dt.int32}
     )
+
+
+def test_generate_quoted_struct():
+    typ = sat.StructType(
+        {"in come": sa.TEXT(), "my count": sa.BIGINT(), "thing": sa.INT()}
+    )
+    result = typ.compile(dialect=duckdb_engine.Dialect())
+    expected = 'STRUCT("in come" TEXT, "my count" BIGINT, thing INTEGER)'
+    assert result == expected
Repository: ibis-project/ibis

Issue:
test(devtools): make custom xfail decorators detect invalid backend names
### Is your feature request related to a problem?

Not related to a problem

### Describe the solution you'd like

It would be nice if misspellings of backend names (like `orcale` instead of `oracle`) used in our backend xfail markers (`never`, `notyet`, and `notimpl`) would cause a test run to exit early with a relevant error message.

### What version of ibis are you running?

`master`

### What backend(s) are you using, if any?

N/A

### Code of Conduct

- [X] I agree to follow this project's Code of Conduct


Discussion:

Resolving patch:
diff --git a/ibis/backends/conftest.py b/ibis/backends/conftest.py
--- a/ibis/backends/conftest.py
+++ b/ibis/backends/conftest.py
@@ -121,6 +121,8 @@
     'ibis/backends/tests/test_window.py',
 ]
 
+ALL_BACKENDS = set(_get_backend_names())
+
 
 @pytest.fixture(scope='session')
 def script_directory() -> Path:
@@ -279,7 +281,6 @@ def pytest_ignore_collect(path, config):
 
 
 def pytest_collection_modifyitems(session, config, items):
-    # add the backend marker to any tests are inside "ibis/backends"
     all_backends = _get_backend_names()
     additional_markers = []
 
@@ -288,7 +289,20 @@ def pytest_collection_modifyitems(session, config, items):
     except ImportError:
         pyspark = None
 
+    unrecognized_backends = set()
     for item in items:
+        # Yell loudly if unrecognized backend in notimpl, notyet or never
+        for name in ("notimpl", "notyet", "never"):
+            for mark in item.iter_markers(name=name):
+                if backend := set(util.promote_list(mark.args[0])).difference(
+                    ALL_BACKENDS
+                ):
+                    unrecognized_backends.add(
+                        f"""Unrecognize backend(s) {backend} passed to {name} marker in
+{item.path}::{item.originalname}"""
+                    )
+
+        # add the backend marker to any tests are inside "ibis/backends"
         parts = item.path.parts
         backend = _get_backend_from_parts(parts)
         if backend is not None and backend in all_backends:
@@ -305,6 +319,7 @@ def pytest_collection_modifyitems(session, config, items):
             if not any(item.iter_markers(name="benchmark")):
                 item.add_marker(pytest.mark.core)
 
+        # skip or xfail pyspark tests that run afoul of our non-ancient stack
         for _ in item.iter_markers(name="pyspark"):
             if not isinstance(item, pytest.DoctestItem):
                 additional_markers.append(
@@ -329,6 +344,9 @@ def pytest_collection_modifyitems(session, config, items):
                     )
                 )
 
+    if unrecognized_backends:
+        raise pytest.PytestCollectionWarning("\n" + "\n".join(unrecognized_backends))
+
     for item, markers in additional_markers:
         for marker in markers:
             item.add_marker(marker)


Test patch:
Repository: ibis-project/ibis

Issue:
docs: move away from .execute() in favor of explicit methods
### What happened?

per https://github.com/ibis-project/ibis/issues/6351, opening this to track updating docs

### What version of ibis are you using?

n/a

### What backend(s) are you using, if any?

n/a

### Relevant log output

_No response_

### Code of Conduct

- [X] I agree to follow this project's Code of Conduct


Discussion:
/take

Resolving patch:
diff --git a/docs/backends/app/backend_info_app.py b/docs/backends/app/backend_info_app.py
--- a/docs/backends/app/backend_info_app.py
+++ b/docs/backends/app/backend_info_app.py
@@ -35,7 +35,7 @@ def support_matrix_df():
                 short_operation=_.full_operation.split(".")[-1],
                 operation_category=_.full_operation.split(".")[-2],
             )
-            .execute()
+            .to_pandas()
         )
 
 
@@ -75,7 +75,7 @@ def get_all_backend_categories():
         backend_info_table.select(category=_.categories.unnest())
         .distinct()
         .order_by('category')['category']
-        .execute()
+        .to_pandas()
         .tolist()
     )
 
@@ -85,7 +85,7 @@ def get_all_operation_categories():
     return (
         support_matrix_table.select(_.operation_category)
         .distinct()['operation_category']
-        .execute()
+        .to_pandas()
         .tolist()
     )
 
@@ -96,7 +96,7 @@ def get_backend_names(categories: Optional[List[str]] = None):
     if categories:
         backend_expr = backend_expr.filter(_.category.isin(categories))
     return (
-        backend_expr.select(_.backend_name).distinct().backend_name.execute().tolist()
+        backend_expr.select(_.backend_name).distinct().backend_name.to_pandas().tolist()
     )
 
 
@@ -170,7 +170,7 @@ def get_selected_operation_categories():
 table_expr = table_expr[current_backend_names + ["index"]]
 
 # Execute query
-df = table_expr.execute()
+df = table_expr.to_pandas()
 df = df.set_index('index')
 
 # Display result
diff --git a/docs/example_streamlit_app/example_streamlit_app.py b/docs/example_streamlit_app/example_streamlit_app.py
--- a/docs/example_streamlit_app/example_streamlit_app.py
+++ b/docs/example_streamlit_app/example_streamlit_app.py
@@ -28,7 +28,7 @@ def query():
         .mutate(ner=_.ner.map(lambda n: n.lower()).unnest())
         .ner.topk(max(options))
         .relabel(dict(ner="ingredient"))
-        .execute()
+        .to_pandas()
         .assign(
             emoji=lambda df: df.ingredient.map(
                 lambda emoji: f"{emojis.get(emoji, '-')}"


Test patch:
Repository: ibis-project/ibis

Issue:
bug(duckdb): re_search only matches whole string
### What happened?

A vdsql user reported (in saulpw/visidata#1855) that regex filter with a DuckDB database is not working.  This is because `ibis_col.re_search("f")` becomes `WHERE colname SIMILAR TO 'f'`, but `re_search` is expected to match the regex anywhere in the string, whereas `SIMILAR TO` only matches the whole string.

```
import ibis
con = ibis.connect('duckdb://tpch.ddb')
t = con.table('customer')
q = t.c_address.re_search('f')
qc = con.compile(q)
qstr = str(qc.compile(compile_kwargs=dict(literal_binds=True)))
print(qstr)
```

Result:

```
SELECT t0.c_address SIMILAR TO 'f' AS "RegexSearch(c_address, 'f')" 
FROM customer AS t0
```

### What version of ibis are you using?

6.0.0

### What backend(s) are you using, if any?

DuckDB

### Relevant log output

_No response_

### Code of Conduct

- [X] I agree to follow this project's Code of Conduct


Discussion:

Resolving patch:
diff --git a/ibis/backends/duckdb/registry.py b/ibis/backends/duckdb/registry.py
--- a/ibis/backends/duckdb/registry.py
+++ b/ibis/backends/duckdb/registry.py
@@ -413,7 +413,7 @@ def _try_cast(t, op):
         ops.RegexReplace: fixed_arity(
             lambda *args: sa.func.regexp_replace(*args, sa.text("'g'")), 3
         ),
-        ops.RegexSearch: fixed_arity(lambda x, y: x.op("SIMILAR TO")(y), 2),
+        ops.RegexSearch: fixed_arity(sa.func.regexp_matches, 2),
         ops.StringContains: fixed_arity(sa.func.contains, 2),
         ops.ApproxMedian: reduction(
             # without inline text, duckdb fails with


Test patch:
diff --git a/ibis/backends/tests/test_string.py b/ibis/backends/tests/test_string.py
--- a/ibis/backends/tests/test_string.py
+++ b/ibis/backends/tests/test_string.py
@@ -191,6 +191,18 @@ def test_string_col_is_unicode(alltypes, df):
                 ),
             ],
         ),
+        param(
+            lambda t: ("a" + t.string_col + "a").re_search(r"\d+"),
+            lambda t: t.string_col.str.contains(r"\d+"),
+            id="re_search_substring",
+            marks=[
+                pytest.mark.notimpl(
+                    ["datafusion", "mssql", "oracle"],
+                    raises=com.OperationNotDefinedError,
+                ),
+                pytest.mark.notimpl(["impala"], raises=AssertionError),
+            ],
+        ),
         param(
             lambda t: t.string_col.re_search(r'\d+'),
             lambda t: t.string_col.str.contains(r'\d+'),
Repository: ibis-project/ibis

Issue:
bug:  parse_sql fails for table alias("Sales as s") without subquery expression
### What happened?

Hi Team, 

I was trying to parse a sql  "ibis.parse_sql("SELECT * FROM SALES as T1",catalog=catalog)".  Actually I tried to parse the sql generated by the ibis for simple select with some group by and subsequently  tried to parse it. The expression sales as t1 when used outside subquery context its failing. Tried lot of variation but still the same result. 

### What version of ibis are you using?

6.0.0

### What backend(s) are you using, if any?

Actually tried with datafusion and duckdb

### Relevant log output

```sh
KeyError                                  Traceback (most recent call last)
Cell In[98], line 1
----> 1 ibis.parse_sql("SELECT * FROM SALES as T1",catalog=catalog, dialect="duckdb")

File ~/miniforge3/envs/llm/lib/python3.9/site-packages/ibis/expr/sql.py:278, in parse_sql(sqlstring, catalog, dialect)
    275 tree = sgo.optimize(expr, catalog.to_sqlglot(), rules=sgo.RULES)
    276 plan = sgp.Plan(tree)
--> 278 return convert(plan.root, catalog=catalog)

File ~/miniforge3/envs/llm/lib/python3.9/functools.py:888, in singledispatch.<locals>.wrapper(*args, **kw)
    884 if not args:
    885     raise TypeError(f'{funcname} requires at least '
    886                     '1 positional argument')
--> 888 return dispatch(args[0].__class__)(*args, **kw)

File ~/miniforge3/envs/llm/lib/python3.9/site-packages/ibis/expr/sql.py:75, in convert_scan(scan, catalog)
     71 @convert.register(sgp.Scan)
     72 def convert_scan(scan, catalog):
     73     catalog = catalog.overlay(scan)
---> 75     table = catalog[scan.source.alias_or_name]
     77     if scan.condition:
     78         pred = convert(scan.condition, catalog=catalog)

KeyError: 't1'
```


### Code of Conduct

- [X] I agree to follow this project's Code of Conduct


Discussion:

Resolving patch:
diff --git a/ibis/expr/sql.py b/ibis/expr/sql.py
--- a/ibis/expr/sql.py
+++ b/ibis/expr/sql.py
@@ -60,6 +60,13 @@ def to_sqlglot(self):
 
     def overlay(self, step):
         updates = {dep.name: convert(dep, catalog=self) for dep in step.dependencies}
+
+        # handle scan aliases: FROM foo AS bar
+        source = getattr(step, "source", None)
+        alias = getattr(source, "args", {}).get("alias")
+        if alias is not None and (source_name := self.get(source.name)) is not None:
+            self[alias.name] = source_name
+
         return Catalog({**self, **updates})
 
 


Test patch:
diff --git a/ibis/tests/expr/test_sql.py b/ibis/tests/expr/test_sql.py
--- a/ibis/tests/expr/test_sql.py
+++ b/ibis/tests/expr/test_sql.py
@@ -112,3 +112,9 @@ def test_parse_sql_simple_select_count():
     sql = """SELECT COUNT(first_name) FROM employee"""
     expr = ibis.parse_sql(sql, catalog)
     code = ibis.decompile(expr, format=True)  # noqa: F841
+
+
+def test_parse_sql_table_alias():
+    sql = """SELECT e.* FROM employee AS e"""
+    expr = ibis.parse_sql(sql, catalog)
+    code = ibis.decompile(expr, format=True)  # noqa: F841