Talip7/scikit-learn-issues-multilabel huggingface dataset:
Improve `FunctionTransformer` diagram representation

### Describe the workflow you want to enable

Currently, using multiple `FunctionTransformers` in a pipeline leads to an uninformative view:

```python
import pandas as pd
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import FunctionTransformer

df = pd.DataFrame([[1,2,3], [4,5,6]], columns=['one','two','three']) # sample data
def a(df): return df+1 # 1st transformer
def b(df): return df*10 # 2nd transformer

make_pipeline(FunctionTransformer(a), FunctionTransformer(b))
```

![image](https://github.com/scikit-learn/scikit-learn/assets/5570380/21382d09-82ad-4e14-8091-6e14ab9989e8)

I would like to see the name of the function being used in the visual blocks


### Describe your proposed solution

I would like to see something like this:

![image](https://github.com/scikit-learn/scikit-learn/assets/5570380/8c67e04a-6328-4afd-9ebc-a0e1da29ca57)

(or perhaps `Function(<name of function>)`  or `<name of function>()` or `FunctionTransformer_<name of function>`) 
 
A sample implementation might be look like this:

```python
from sklearn.preprocessing import FunctionTransformer
from sklearn.utils._estimator_html_repr import _VisualBlock
from functools import partial

class PrettyFunctionTransformer(FunctionTransformer):
    def _sk_visual_block_(self):
        return _VisualBlock(
            "single",
            self,
            names=self.func.func.__name__ if isinstance(self.func, partial) else self.func.__name__,
            name_details=str(self),
        )
```

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
I also like 'the harder way' better.

Two further possible improvements:
1)  switch the titles: 'FunctionTransformer' should be the caption and the function names the titles. This way the repetition is in the small font and the transformer func name in the big
2) show the partial function name. I don't think there is added value in showing that a function is a partial without showing the original function name. It's the same function but with different defaults.. we can just use the `.func.__name__`.  We use partials a lot as we create pipelines from configuration files (using hydra instantiate)

Labels: New Feature
Improve `FunctionTransformer` diagram representation

### Describe the workflow you want to enable

Currently, using multiple `FunctionTransformers` in a pipeline leads to an uninformative view:

```python
import pandas as pd
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import FunctionTransformer

df = pd.DataFrame([[1,2,3], [4,5,6]], columns=['one','two','three']) # sample data
def a(df): return df+1 # 1st transformer
def b(df): return df*10 # 2nd transformer

make_pipeline(FunctionTransformer(a), FunctionTransformer(b))
```

![image](https://github.com/scikit-learn/scikit-learn/assets/5570380/21382d09-82ad-4e14-8091-6e14ab9989e8)

I would like to see the name of the function being used in the visual blocks


### Describe your proposed solution

I would like to see something like this:

![image](https://github.com/scikit-learn/scikit-learn/assets/5570380/8c67e04a-6328-4afd-9ebc-a0e1da29ca57)

(or perhaps `Function(<name of function>)`  or `<name of function>()` or `FunctionTransformer_<name of function>`) 
 
A sample implementation might be look like this:

```python
from sklearn.preprocessing import FunctionTransformer
from sklearn.utils._estimator_html_repr import _VisualBlock
from functools import partial

class PrettyFunctionTransformer(FunctionTransformer):
    def _sk_visual_block_(self):
        return _VisualBlock(
            "single",
            self,
            names=self.func.func.__name__ if isinstance(self.func, partial) else self.func.__name__,
            name_details=str(self),
        )
```

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
1. Okay I'll take a look when I've got time.
2. Maybe; I can go with this implementation in the PR and see if people think it's worth the complexity or if there are other ways.

Labels: New Feature
Improve `FunctionTransformer` diagram representation

### Describe the workflow you want to enable

Currently, using multiple `FunctionTransformers` in a pipeline leads to an uninformative view:

```python
import pandas as pd
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import FunctionTransformer

df = pd.DataFrame([[1,2,3], [4,5,6]], columns=['one','two','three']) # sample data
def a(df): return df+1 # 1st transformer
def b(df): return df*10 # 2nd transformer

make_pipeline(FunctionTransformer(a), FunctionTransformer(b))
```

![image](https://github.com/scikit-learn/scikit-learn/assets/5570380/21382d09-82ad-4e14-8091-6e14ab9989e8)

I would like to see the name of the function being used in the visual blocks


### Describe your proposed solution

I would like to see something like this:

![image](https://github.com/scikit-learn/scikit-learn/assets/5570380/8c67e04a-6328-4afd-9ebc-a0e1da29ca57)

(or perhaps `Function(<name of function>)`  or `<name of function>()` or `FunctionTransformer_<name of function>`) 
 
A sample implementation might be look like this:

```python
from sklearn.preprocessing import FunctionTransformer
from sklearn.utils._estimator_html_repr import _VisualBlock
from functools import partial

class PrettyFunctionTransformer(FunctionTransformer):
    def _sk_visual_block_(self):
        return _VisualBlock(
            "single",
            self,
            names=self.func.func.__name__ if isinstance(self.func, partial) else self.func.__name__,
            name_details=str(self),
        )
```

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
Sorry that I made a few mistakes in my earlier comments:

- Nested `partial`s work without recursion
- `np.vectorize` does have `__name__` on latest versions

Hence I think we do not need to care about that many corner cases. Having a special case for `partial` and having a fallback sounds good to me.

Labels: New Feature
DOC Investigate scipy-doctest for more convenient doctests

I learned about [scipy-doctest](https://github.com/scipy/scipy_doctest) recent release in the [Scientific Python Discourse announcement](https://discuss.scientific-python.org/t/ann-scipy-doctest-package/1181). Apparently, scipy-doctest has been used internally in numpy and scipy for doctests for some time. In particular it allows floating point comparisons.

After a bit of work from us setting everything up, it would allow to have a few sprint / first good issues.

There is quite a few places where we used the doctest ellipsis, the quick and dirty following regexp finds 595 lines:
```
git grep -P '\d+\.\.\.' | wc -l
```

If you are not sure what I am talking about, this is the `...` for doctest in rst for docstrings e.g. the last line of this snippet:
```py
>>> from sklearn import svm, datasets
>>> from sklearn.model_selection import cross_val_score
>>> X, y = datasets.load_iris(return_X_y=True)
>>> clf = svm.SVC(random_state=0)
>>> cross_val_score(clf, X, y, cv=5, scoring='recall_macro')
array([0.96..., 0.96..., 0.96..., 0.93..., 1.        ])
```

An example of a doctest with a spurious failure recently: https://github.com/scikit-learn/scikit-learn/pull/29140#issuecomment-2139904739

If you are wondering about the difference to [pytest-doctestplus](https://github.com/scientific-python/pytest-doctestplus) look at [this](https://github.com/scipy/scipy_doctest?tab=readme-ov-file#prior-art-and-related-work). This does seem a bit unfortunate to have `scipy/scipy_doctest` and `scientific-python/pytest-doctestplus` but oh well (full disclosure I did not have time to look into the history) ...

COMMENT:
If there's still interest in scipy-doctest, feel free to ping me

Labels: Documentation, Enhancement, RFC
DOC Investigate scipy-doctest for more convenient doctests

I learned about [scipy-doctest](https://github.com/scipy/scipy_doctest) recent release in the [Scientific Python Discourse announcement](https://discuss.scientific-python.org/t/ann-scipy-doctest-package/1181). Apparently, scipy-doctest has been used internally in numpy and scipy for doctests for some time. In particular it allows floating point comparisons.

After a bit of work from us setting everything up, it would allow to have a few sprint / first good issues.

There is quite a few places where we used the doctest ellipsis, the quick and dirty following regexp finds 595 lines:
```
git grep -P '\d+\.\.\.' | wc -l
```

If you are not sure what I am talking about, this is the `...` for doctest in rst for docstrings e.g. the last line of this snippet:
```py
>>> from sklearn import svm, datasets
>>> from sklearn.model_selection import cross_val_score
>>> X, y = datasets.load_iris(return_X_y=True)
>>> clf = svm.SVC(random_state=0)
>>> cross_val_score(clf, X, y, cv=5, scoring='recall_macro')
array([0.96..., 0.96..., 0.96..., 0.93..., 1.        ])
```

An example of a doctest with a spurious failure recently: https://github.com/scikit-learn/scikit-learn/pull/29140#issuecomment-2139904739

If you are wondering about the difference to [pytest-doctestplus](https://github.com/scientific-python/pytest-doctestplus) look at [this](https://github.com/scipy/scipy_doctest?tab=readme-ov-file#prior-art-and-related-work). This does seem a bit unfortunate to have `scipy/scipy_doctest` and `scientific-python/pytest-doctestplus` but oh well (full disclosure I did not have time to look into the history) ...

COMMENT:
Thanks! There is still some interest, I did have a quick go at one point but failed to make it work in the amount of time I had :sweat_smile: 

Maybe you can comment on the general approach? In particular, I am guessing that you can still run the previous doctests unchanged with scipy-doctest?

Otherwise I guess I (or someone else) would need to look at how numpy or scipy is doing it and adapt it for scikit-learn (for example we already have some logic to skip all doctests for numpy<2 or if matplotlib is not installed):

https://github.com/numpy/numpy/blob/8e6914d1d1586248aac518082d632839767eab91/numpy/conftest.py#L153-L232

https://github.com/scipy/scipy/blob/2fde036ec404512d695568a4259ba16f6c9156b5/scipy/conftest.py#L395-L551

After a quick look it looks like the main things are:
- warnings control (probably because warnings are turned into errors)
- managing a list of tests to skip or xfail
- ignore some tests during collection

Labels: Documentation, Enhancement, RFC
DOC Investigate scipy-doctest for more convenient doctests

I learned about [scipy-doctest](https://github.com/scipy/scipy_doctest) recent release in the [Scientific Python Discourse announcement](https://discuss.scientific-python.org/t/ann-scipy-doctest-package/1181). Apparently, scipy-doctest has been used internally in numpy and scipy for doctests for some time. In particular it allows floating point comparisons.

After a bit of work from us setting everything up, it would allow to have a few sprint / first good issues.

There is quite a few places where we used the doctest ellipsis, the quick and dirty following regexp finds 595 lines:
```
git grep -P '\d+\.\.\.' | wc -l
```

If you are not sure what I am talking about, this is the `...` for doctest in rst for docstrings e.g. the last line of this snippet:
```py
>>> from sklearn import svm, datasets
>>> from sklearn.model_selection import cross_val_score
>>> X, y = datasets.load_iris(return_X_y=True)
>>> clf = svm.SVC(random_state=0)
>>> cross_val_score(clf, X, y, cv=5, scoring='recall_macro')
array([0.96..., 0.96..., 0.96..., 0.93..., 1.        ])
```

An example of a doctest with a spurious failure recently: https://github.com/scikit-learn/scikit-learn/pull/29140#issuecomment-2139904739

If you are wondering about the difference to [pytest-doctestplus](https://github.com/scientific-python/pytest-doctestplus) look at [this](https://github.com/scipy/scipy_doctest?tab=readme-ov-file#prior-art-and-related-work). This does seem a bit unfortunate to have `scipy/scipy_doctest` and `scientific-python/pytest-doctestplus` but oh well (full disclosure I did not have time to look into the history) ...

COMMENT:
Also, while we're on "general approach" topic: the setup in both scipy and numpy is to maintain a clear separation between doctests and actual unit testing. For one, `--doctest-modules` only runs doctests, not regular unit tests.

Labels: Documentation, Enhancement, RFC
DOC Investigate scipy-doctest for more convenient doctests

I learned about [scipy-doctest](https://github.com/scipy/scipy_doctest) recent release in the [Scientific Python Discourse announcement](https://discuss.scientific-python.org/t/ann-scipy-doctest-package/1181). Apparently, scipy-doctest has been used internally in numpy and scipy for doctests for some time. In particular it allows floating point comparisons.

After a bit of work from us setting everything up, it would allow to have a few sprint / first good issues.

There is quite a few places where we used the doctest ellipsis, the quick and dirty following regexp finds 595 lines:
```
git grep -P '\d+\.\.\.' | wc -l
```

If you are not sure what I am talking about, this is the `...` for doctest in rst for docstrings e.g. the last line of this snippet:
```py
>>> from sklearn import svm, datasets
>>> from sklearn.model_selection import cross_val_score
>>> X, y = datasets.load_iris(return_X_y=True)
>>> clf = svm.SVC(random_state=0)
>>> cross_val_score(clf, X, y, cv=5, scoring='recall_macro')
array([0.96..., 0.96..., 0.96..., 0.93..., 1.        ])
```

An example of a doctest with a spurious failure recently: https://github.com/scikit-learn/scikit-learn/pull/29140#issuecomment-2139904739

If you are wondering about the difference to [pytest-doctestplus](https://github.com/scientific-python/pytest-doctestplus) look at [this](https://github.com/scipy/scipy_doctest?tab=readme-ov-file#prior-art-and-related-work). This does seem a bit unfortunate to have `scipy/scipy_doctest` and `scientific-python/pytest-doctestplus` but oh well (full disclosure I did not have time to look into the history) ...

COMMENT:
Thanks a lot for the details, I need to take a closer look and I'll definitely ping you if I get stuck.

Labels: Documentation, Enhancement, RFC
DOC Investigate scipy-doctest for more convenient doctests

I learned about [scipy-doctest](https://github.com/scipy/scipy_doctest) recent release in the [Scientific Python Discourse announcement](https://discuss.scientific-python.org/t/ann-scipy-doctest-package/1181). Apparently, scipy-doctest has been used internally in numpy and scipy for doctests for some time. In particular it allows floating point comparisons.

After a bit of work from us setting everything up, it would allow to have a few sprint / first good issues.

There is quite a few places where we used the doctest ellipsis, the quick and dirty following regexp finds 595 lines:
```
git grep -P '\d+\.\.\.' | wc -l
```

If you are not sure what I am talking about, this is the `...` for doctest in rst for docstrings e.g. the last line of this snippet:
```py
>>> from sklearn import svm, datasets
>>> from sklearn.model_selection import cross_val_score
>>> X, y = datasets.load_iris(return_X_y=True)
>>> clf = svm.SVC(random_state=0)
>>> cross_val_score(clf, X, y, cv=5, scoring='recall_macro')
array([0.96..., 0.96..., 0.96..., 0.93..., 1.        ])
```

An example of a doctest with a spurious failure recently: https://github.com/scikit-learn/scikit-learn/pull/29140#issuecomment-2139904739

If you are wondering about the difference to [pytest-doctestplus](https://github.com/scientific-python/pytest-doctestplus) look at [this](https://github.com/scipy/scipy_doctest?tab=readme-ov-file#prior-art-and-related-work). This does seem a bit unfortunate to have `scipy/scipy_doctest` and `scientific-python/pytest-doctestplus` but oh well (full disclosure I did not have time to look into the history) ...

COMMENT:
Turns out it was slightly easier than anticipated, I opened #30496 which seems to work well enough and do not require any changes to our doctests. For now, it seems like we don't need as many tweaks as numpy or scipy, which is good.

The next step would be to open a meta-issue to invite contributors clean things up, one file at a time to remove `...` in doctests by relying on built-in scipy-doctest floating point comparison

Labels: Documentation, Enhancement, RFC
DOC Investigate scipy-doctest for more convenient doctests

I learned about [scipy-doctest](https://github.com/scipy/scipy_doctest) recent release in the [Scientific Python Discourse announcement](https://discuss.scientific-python.org/t/ann-scipy-doctest-package/1181). Apparently, scipy-doctest has been used internally in numpy and scipy for doctests for some time. In particular it allows floating point comparisons.

After a bit of work from us setting everything up, it would allow to have a few sprint / first good issues.

There is quite a few places where we used the doctest ellipsis, the quick and dirty following regexp finds 595 lines:
```
git grep -P '\d+\.\.\.' | wc -l
```

If you are not sure what I am talking about, this is the `...` for doctest in rst for docstrings e.g. the last line of this snippet:
```py
>>> from sklearn import svm, datasets
>>> from sklearn.model_selection import cross_val_score
>>> X, y = datasets.load_iris(return_X_y=True)
>>> clf = svm.SVC(random_state=0)
>>> cross_val_score(clf, X, y, cv=5, scoring='recall_macro')
array([0.96..., 0.96..., 0.96..., 0.93..., 1.        ])
```

An example of a doctest with a spurious failure recently: https://github.com/scikit-learn/scikit-learn/pull/29140#issuecomment-2139904739

If you are wondering about the difference to [pytest-doctestplus](https://github.com/scientific-python/pytest-doctestplus) look at [this](https://github.com/scipy/scipy_doctest?tab=readme-ov-file#prior-art-and-related-work). This does seem a bit unfortunate to have `scipy/scipy_doctest` and `scientific-python/pytest-doctestplus` but oh well (full disclosure I did not have time to look into the history) ...

COMMENT:
Yay, great!

Re removing `...` --- most likely tangentially related if at all --- https://github.com/scipy/scipy_doctest/issues/147 was thinking about actually supporting doctest.ellipsis with numpy 2.x output format.

Labels: Documentation, Enhancement, RFC
DOC Investigate scipy-doctest for more convenient doctests

I learned about [scipy-doctest](https://github.com/scipy/scipy_doctest) recent release in the [Scientific Python Discourse announcement](https://discuss.scientific-python.org/t/ann-scipy-doctest-package/1181). Apparently, scipy-doctest has been used internally in numpy and scipy for doctests for some time. In particular it allows floating point comparisons.

After a bit of work from us setting everything up, it would allow to have a few sprint / first good issues.

There is quite a few places where we used the doctest ellipsis, the quick and dirty following regexp finds 595 lines:
```
git grep -P '\d+\.\.\.' | wc -l
```

If you are not sure what I am talking about, this is the `...` for doctest in rst for docstrings e.g. the last line of this snippet:
```py
>>> from sklearn import svm, datasets
>>> from sklearn.model_selection import cross_val_score
>>> X, y = datasets.load_iris(return_X_y=True)
>>> clf = svm.SVC(random_state=0)
>>> cross_val_score(clf, X, y, cv=5, scoring='recall_macro')
array([0.96..., 0.96..., 0.96..., 0.93..., 1.        ])
```

An example of a doctest with a spurious failure recently: https://github.com/scikit-learn/scikit-learn/pull/29140#issuecomment-2139904739

If you are wondering about the difference to [pytest-doctestplus](https://github.com/scientific-python/pytest-doctestplus) look at [this](https://github.com/scipy/scipy_doctest?tab=readme-ov-file#prior-art-and-related-work). This does seem a bit unfortunate to have `scipy/scipy_doctest` and `scientific-python/pytest-doctestplus` but oh well (full disclosure I did not have time to look into the history) ...

COMMENT:
> The next step would be to open a meta-issue to invite contributors clean things up, one file at a time to remove ... in doctests by relying on built-in scipy-doctest floating point comparison

I doubt that's needed. Sometimes `...` actually improves readability: when we know that a cross-validated score value is only statistically meaningful up to the 3 first digits, there is no point in displaying more digits in the output of the example.

Labels: Documentation, Enhancement, RFC
DOC Investigate scipy-doctest for more convenient doctests

I learned about [scipy-doctest](https://github.com/scipy/scipy_doctest) recent release in the [Scientific Python Discourse announcement](https://discuss.scientific-python.org/t/ann-scipy-doctest-package/1181). Apparently, scipy-doctest has been used internally in numpy and scipy for doctests for some time. In particular it allows floating point comparisons.

After a bit of work from us setting everything up, it would allow to have a few sprint / first good issues.

There is quite a few places where we used the doctest ellipsis, the quick and dirty following regexp finds 595 lines:
```
git grep -P '\d+\.\.\.' | wc -l
```

If you are not sure what I am talking about, this is the `...` for doctest in rst for docstrings e.g. the last line of this snippet:
```py
>>> from sklearn import svm, datasets
>>> from sklearn.model_selection import cross_val_score
>>> X, y = datasets.load_iris(return_X_y=True)
>>> clf = svm.SVC(random_state=0)
>>> cross_val_score(clf, X, y, cv=5, scoring='recall_macro')
array([0.96..., 0.96..., 0.96..., 0.93..., 1.        ])
```

An example of a doctest with a spurious failure recently: https://github.com/scikit-learn/scikit-learn/pull/29140#issuecomment-2139904739

If you are wondering about the difference to [pytest-doctestplus](https://github.com/scientific-python/pytest-doctestplus) look at [this](https://github.com/scipy/scipy_doctest?tab=readme-ov-file#prior-art-and-related-work). This does seem a bit unfortunate to have `scipy/scipy_doctest` and `scientific-python/pytest-doctestplus` but oh well (full disclosure I did not have time to look into the history) ...

COMMENT:
> I doubt that's needed. Sometimes ... actually improves readability: when we know that a cross-validated score value is only statistically meaningful up to the 3 first digits

Not sure I follow you, do you find this (without the explicit `...`):
```rst
>> cross_val_score(model, X, y).mean()
0.123
```

to be less readable than (with the explicit `...`):
```rst
>> cross_val_score(model, X, y).mean()
0.123...
```

scipy-doctest allows you to use the first option (without `...`) and the test will pass [^1] . `doctest` in contrast is more finicky and we then need `...` all over the place to make it happy.

[^1]: there is a tweakable default rtol (default is 1e-3 IIRC)

Labels: Documentation, Enhancement, RFC