Repository: python-attrs/attrs

Issue:
Creating a Subclassed instance now errors with `got an unexpected keyword argument`
We just noticed this in Airflow CI on upgrading to 25.2.0.

The error is

```
ERROR    airflow.models.dagbag.DagBag:dagbag.py:394 Failed to import: /opt/airflow/airflow/example_dags/example_dynamic_task_mapping.py
Traceback (most recent call last):
  File "/opt/airflow/airflow/models/dagbag.py", line 384, in parse
    loader.exec_module(new_module)
  File "<frozen importlib._bootstrap_external>", line 999, in exec_module
  File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
  File "/opt/airflow/airflow/example_dags/example_dynamic_task_mapping.py", line 27, in <module>
    with DAG(dag_id="example_dynamic_task_mapping", schedule=None, start_date=datetime(2022, 3, 4)) as dag:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: DAG.__init__() got an unexpected keyword argument 'dag_id'
```

(Yes, I know your view on subclassing. I don't disagree either)

Minimal repro case:

```python
import attrs

def _all_after_dag_id_to_kw_only(cls, fields: list[attrs.Attribute]):
    i = iter(fields)
    f = next(i)
    if f.name != "dag_id":
        raise RuntimeError("dag_id was not the first field")
    yield f

    for f in i:
        yield f.evolve(kw_only=True)


@attrs.define(field_transformer=_all_after_dag_id_to_kw_only)
class Base:
    dag_id: str
    has_default: str = "default"


@attrs.define()
class Subclass(Base):
    other: bool


x = Subclass(dag_id="foo", other=False)

print(repr(x))
print(attrs.__version__)
```


Discussion:
https://github.com/python-attrs/attrs/pull/1401 might have caused it
Might do, yes, but the minimal example is not failing, so it's some quirk of what we are doing in Airflow
cc @jamesmurphy-mc 
pls provide a repro and I'll see if I can do anything before leaving for my vacation tomorrow afternoon
I updated the example to include `help(Subclass.__init__)`;

On 25.1.0 we see this:
```
__init__(self, dag_id: str, other: bool, *, has_default: str = 'default') -> None
    Method generated by attrs for class Subclass.
```

On 25.2.0 we see this:
```
__init__(self, other: bool) -> None
    Method generated by attrs for class Subclass.
```
@hynek No rush at all!, we are happy to pin to !latest for now
I have got a simple reproducer. Let me try that.
cc @sscherfke for good measure
@tirkarthi I've got a reproducer now. Top post has been updated.
@ashb Thanks, I got the same one but didn't to subclass.

```python
import attrs

def _all_after_dag_id_to_kw_only(cls, fields: list[attrs.Attribute]):
    i = iter(fields)
    f = next(i)
    if f.name != "dag_id":
        raise RuntimeError("dag_id was not the first field")
    yield f

    for f in i:
        yield f.evolve(kw_only=True)


@attrs.define(repr=False, field_transformer=_all_after_dag_id_to_kw_only, slots=False)
class DAG:
    dag_id: str = attrs.field(kw_only=False, validator=attrs.validators.instance_of(str))


dag = DAG(dag_id="test")
print(dag)
```

```
(.venv) ➜  airflow git:(main) ✗ python /tmp/attrs_repro.py
<__main__.DAG object at 0x7f69c5305bd0>
(.venv) ➜  airflow git:(main) ✗ pip install attrs==25.2.0
Collecting attrs==25.2.0
  Downloading attrs-25.2.0-py3-none-any.whl.metadata (11 kB)
Downloading attrs-25.2.0-py3-none-any.whl (64 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 64.0/64.0 kB 1.5 MB/s eta 0:00:00
Installing collected packages: attrs
  Attempting uninstall: attrs
    Found existing installation: attrs 25.1.0
    Uninstalling attrs-25.1.0:
      Successfully uninstalled attrs-25.1.0
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
ydb 3.18.15 requires protobuf<5.0.0,>=3.13.0, but you have protobuf 5.29.3 which is incompatible.
Successfully installed attrs-25.2.0

[notice] A new release of pip is available: 24.0 -> 25.0.1
[notice] To update, run: pip install --upgrade pip
(.venv) ➜  airflow git:(main) ✗ python /tmp/attrs_repro.py 
Traceback (most recent call last):
  File "/tmp/attrs_repro.py", line 19, in <module>
    dag = DAG(dag_id="test")
          ^^^^^^^^^^^^^^^^^^
TypeError: DAG.__init__() got an unexpected keyword argument 'dag_id'
```
Walking up the stack to where the field transformer is called, I notice that base_attrs is an empty list, even though collect_mro is true. 
Debugging continues!
`print(inspect.signature(DAG))`

On 24.2.0 : `(dag_id: str) -> None`
On 25.2.0 : `() -> None`


Oh, this might be part of the problem, Base.__attrs_attrs__ is empty when a field transformer is used:

```python

import attrs

print(f"{attrs.__version__=}")
def _all_after_dag_id_to_kw_only(cls, fields: list[attrs.Attribute]):
    i = iter(fields)
    f = next(i)
    if f.name != "dag_id":
        raise RuntimeError("dag_id was not the first field")
    yield f

    for f in i:
        yield f.evolve(kw_only=True)


@attrs.define(field_transformer=_all_after_dag_id_to_kw_only)
class Base:
    dag_id: str
    has_default: str = "default"

print(attrs.fields(Base))
```

Prints:

```
attrs.__version__='25.2.0'
()
```
Ahhh, got it.

The issue is that the field transformer is defined as a generator, so it's exhausted once looking for the `a.init` and `a.kw_only` args, then it's empty in the uses elsewhere in `_transform_attr` 
> Ahhh, got it.

🎉

> The issue is that the field transformer is defined as a generator, so it's exhausted once looking for the `a.init` and `a.kw_only` args, then it's empty in the uses elsewhere in `_transform_attr`

Without being able to look at the code: sounds like we should be able to convert it to a tuple on first use? Quick PRs with tests welcome. ;)
Yup, that was exactly my plan, `inspect.isgenerator()` + tuple.



Patch:
diff --git a/src/attr/_make.py b/src/attr/_make.py
index 7439dc2ad..e84d9792a 100644
--- a/src/attr/_make.py
+++ b/src/attr/_make.py
@@ -448,7 +448,7 @@ def _transform_attrs(
     attrs = base_attrs + own_attrs
 
     if field_transformer is not None:
-        attrs = field_transformer(cls, attrs)
+        attrs = tuple(field_transformer(cls, attrs))
 
     # Check attr order after executing the field_transformer.
     # Mandatory vs non-mandatory attr order only matters when they are part of


Test patch:
diff --git a/tests/test_hooks.py b/tests/test_hooks.py
index 6b3420639..4dcd35240 100644
--- a/tests/test_hooks.py
+++ b/tests/test_hooks.py
@@ -217,6 +217,22 @@ class C:
         assert "CAttributes" == fields_type.__name__
         assert issubclass(fields_type, tuple)
 
+    def test_hook_generator(self):
+        """
+        Ensure that `attrs.fields` are correctly recorded when field_transformer is a generator
+
+        Regression test for #1417
+        """
+
+        def hook(cls, attribs):
+            yield from attribs
+
+        @attr.s(auto_attribs=True, field_transformer=hook)
+        class Base:
+            x: int
+
+        assert ["x"] == [a.name for a in attr.fields(Base)]
+
 
 class TestAsDictHook:
     def test_asdict(self):