bug: Nested Pipeline is not fitted (sklearn 1.8)

EDIT: The specific issue described here is actually caused by skrub, and solved in https://github.com/skrub-data/skrub/pull/1813.
However, this poses a more general question on how easy it is to abide by sklearn's conventions: skip directly to [this comment](https://github.com/scikit-learn/scikit-learn/issues/32910#issuecomment-3665534057) for that discussion.

---

### Describe the bug

With sklearn==1.8.0 and skrub==0.6.0, the following script fails:

```python
# test_failure.py

from sklearn.linear_model import Ridge
from sklearn.compose import make_column_transformer
from sklearn.pipeline import make_pipeline
from skrub import DatetimeEncoder, ToDatetime, DropCols
from skrub.datasets import fetch_employee_salaries

datasets = fetch_employee_salaries()

X, y = datasets.X, datasets.y

date_encoder = make_pipeline(
    ToDatetime(),
    DatetimeEncoder(),
    DropCols("date_first_hired_year"),
)

preprocessing = make_column_transformer(
    (date_encoder, "date_first_hired"),
)

model = make_pipeline(preprocessing, Ridge())

model.fit(X, y)
model.predict(X)
```

with stacktrace:
<details>
<summary>Stacktrace</summary>

```
/home/auguste/Desktop/work/skore/.venv/lib/python3.14/site-packages/scipy/_lib/_util.py:1233: LinAlgWarning: Ill-conditioned matrix (rcond=1.36439e-16): result may not be accurate.
  return f(*arrays, *other_args, **kwargs)
Traceback (most recent call last):
  File "/home/auguste/Desktop/work/skore/examples/use_cases/test_failure.py", line 24, in <module>
    model.predict(X)
    ~~~~~~~~~~~~~^^^
  File "/home/auguste/Desktop/work/skore/.venv/lib/python3.14/site-packages/sklearn/pipeline.py", line 745, in predict
    Xt = transform.transform(Xt)
  File "/home/auguste/Desktop/work/skore/.venv/lib/python3.14/site-packages/sklearn/utils/_set_output.py", line 316, in wrapped
    data_to_wrap = f(self, X, *args, **kwargs)
  File "/home/auguste/Desktop/work/skore/.venv/lib/python3.14/site-packages/sklearn/compose/_column_transformer.py", line 1099, in transform
    Xs = self._call_func_on_transformers(
        X,
    ...<3 lines>...
        routed_params=routed_params,
    )
  File "/home/auguste/Desktop/work/skore/.venv/lib/python3.14/site-packages/sklearn/compose/_column_transformer.py", line 901, in _call_func_on_transformers
    return Parallel(n_jobs=self.n_jobs)(jobs)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/auguste/Desktop/work/skore/.venv/lib/python3.14/site-packages/sklearn/utils/parallel.py", line 91, in __call__
    return super().__call__(iterable_with_config_and_warning_filters)
           ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/auguste/Desktop/work/skore/.venv/lib/python3.14/site-packages/joblib/parallel.py", line 1986, in __call__
    return output if self.return_generator else list(output)
                                                ~~~~^^^^^^^^
  File "/home/auguste/Desktop/work/skore/.venv/lib/python3.14/site-packages/joblib/parallel.py", line 1914, in _get_sequential_output
    res = func(*args, **kwargs)
  File "/home/auguste/Desktop/work/skore/.venv/lib/python3.14/site-packages/sklearn/utils/parallel.py", line 184, in __call__
    return self.function(*args, **kwargs)
           ~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
  File "/home/auguste/Desktop/work/skore/.venv/lib/python3.14/site-packages/sklearn/pipeline.py", line 1464, in _transform_one
    res = transformer.transform(X, **params.transform)
  File "/home/auguste/Desktop/work/skore/.venv/lib/python3.14/site-packages/sklearn/pipeline.py", line 1035, in transform
    check_is_fitted(self)
    ~~~~~~~~~~~~~~~^^^^^^
  File "/home/auguste/Desktop/work/skore/.venv/lib/python3.14/site-packages/sklearn/utils/validation.py", line 1705, in check_is_fitted
    raise NotFittedError(msg % {"name": type(estimator).__name__})
sklearn.exceptions.NotFittedError: This Pipeline instance is not fitted yet. Call 'fit' with appropriate arguments before using this estimator.
```

</details>

Indeed, if we look at the model after `.fit()` it seems some internal steps are not fitted:
```python
>>> model
Pipeline(steps=[('columntransformer',
                 ColumnTransformer(transformers=[('pipeline',
                                                  Pipeline(steps=[('todatetime',
                                                                   ToDatetime()),
                                                                  ('datetimeencoder',
                                                                   DatetimeEncoder()),
                                                                  ('dropcols',
                                                                   DropCols(cols='date_first_hired_year'))]),
                                                  'date_first_hired')])),
                ('ridge', Ridge())])
>>> model.__sklearn_is_fitted__()
True
>>> model.steps[0][1]
ColumnTransformer(transformers=[('pipeline',
                                 Pipeline(steps=[('todatetime', ToDatetime()),
                                                 ('datetimeencoder',
                                                  DatetimeEncoder()),
                                                 ('dropcols',
                                                  DropCols(cols='date_first_hired_year'))]),
                                 'date_first_hired')])
>>> model.steps[0][1].transformers[0][1].__sklearn_is_fitted__()
False
```

---

I used the debugger to dig a bit more inside `model.fit`, and I ended up in `sklearn.compose._column_transformer.py` at line 892.

<https://github.com/scikit-learn/scikit-learn/blob/1.8.0/sklearn/compose/_column_transformer.py#L892>

Notice that at this line, `trans` is *cloned* if `fitted` is False. If it's a clone of the transformer that gets fitted, I guess it makes sense that the original is not fitted?

Indeed the state when I reach this line is
```python
>>> self
ColumnTransformer(transformers=[('pipeline',
                                 Pipeline(steps=[('todatetime', ToDatetime()),
                                                 ('datetimeencoder',
                                                  DatetimeEncoder()),
                                                 ('dropcols',
                                                  DropCols(cols='date_first_hired_year'))]),
                                 'date_first_hired')])
>>> trans
Pipeline(steps=[('todatetime', ToDatetime()),
                ('datetimeencoder', DatetimeEncoder()),
                ('dropcols', DropCols(cols='date_first_hired_year'))])
>>> fitted
False
>>> func
<function _fit_transform_one at 0x7f0997d335e0>
```

---

However, if you replace the datetime pipeline with a scaling (skrub-less) pipeline, there is no error:
```python
# test_success.py

from sklearn.linear_model import Ridge
from sklearn.compose import make_column_transformer
from sklearn.pipeline import make_pipeline
from sklearn.datasets import make_classification
from sklearn.preprocessing import StandardScaler

X, y = make_classification()

scaling = make_pipeline(StandardScaler())

preprocessing = make_column_transformer((scaling, [1]))

model = make_pipeline(preprocessing, Ridge())

model.fit(X, y)
model.predict(X)
```

Likewise, the error does not happen with sklearn 1.7.

For what it's worth, this might be linked to https://github.com/scikit-learn/scikit-learn/issues/29983 and https://github.com/scikit-learn/scikit-learn/issues/8157

### Versions

```shell
System:
    python: 3.14.0 (main, Nov 19 2025, 22:48:15) [Clang 21.1.4 ]
executable: /home/auguste/Desktop/work/skore/.venv/bin/python3
   machine: Linux-6.12.59-x86_64-with-glibc2.40

Python dependencies:
      sklearn: 1.8.0
          pip: None
   setuptools: 80.9.0
        numpy: 2.3.5
        scipy: 1.16.3
       Cython: None
       pandas: 2.3.3
   matplotlib: 3.10.8
       joblib: 1.5.3
threadpoolctl: 3.6.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: openblas
    num_threads: 12
         prefix: libscipy_openblas
       filepath: /home/auguste/Desktop/work/skore/.venv/lib/python3.14/site-packages/numpy.libs/libscipy_openblas64_-fdde5778.so
        version: 0.3.30
threading_layer: pthreads
   architecture: Haswell

       user_api: blas
   internal_api: openblas
    num_threads: 12
         prefix: libscipy_openblas
       filepath: /home/auguste/Desktop/work/skore/.venv/lib/python3.14/site-packages/scipy.libs/libscipy_openblas-b75cc656.so
        version: 0.3.29.dev
threading_layer: pthreads
   architecture: Haswell

       user_api: openmp
   internal_api: openmp
    num_threads: 12
         prefix: libgomp
       filepath: /home/auguste/Desktop/work/skore/.venv/lib/python3.14/site-packages/scikit_learn.libs/libgomp-e985bcbb.so.1.0.0
        version: None
```

COMMENT:
Coming back to this after a long day of work...

First, https://github.com/skrub-data/skrub/pull/1813 is the right fix on the skrub side. This is great, thanks @auguste-probabl !

Second, to reply to @betatim 's questions:

> Does DropCols pass the common estimator checks? Does skrub use them to "certify" the estimators that want to be scikit-learn estimators?

I don't know :). Skrub should implement this, but it not that easy (aka people need to know and understand these things).

> This means there are two questions: are these common checks easy enough to discover and use?

IMHO, no. But then, it's a recurrent problem of everything in life :) : the information is often hard to find, even when it's in the middle of my desk (or worse, mailbox). I believe that we need to cater for people not finding it.


**Broader point of view** : I think that if skrub has this problem, we probably have a bunch of libraries that have it. I do think that it is worth reflecting on how to avoid that we keep breaking compatibility with a broader ecosystem

Labels: Bug, Needs Triage