Talip7/scikit-learn-issues-multilabel huggingface dataset:
Add More Data to the RidgeCV, LassoCV, and ElasticNetCV Path

### Describe the workflow you want to enable

Currently, the mse_path_ is available from the above models, which lets you inspect/plot the mse for all folds, alphas, and l1_ratios for elasticnet for instance. It would be very nice to record not only the mse in this way, but also the coefficients and possibly the in-sample/validation score.

### Describe your proposed solution

Add variables that include the coefficients and maybe the score.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
These classes already have a `coef_` attribute. Not sure what you mean by score beyond mse

Labels: New Feature, spam, Needs Triage
`scipy.minimize(method=’L-BFGS-B’)` deprecation warning for `iprint` and `disp` arguments

### Describe the bug

When upgrading to scipy 1.16, fitting a LogisticRegression raises a deprecation warning:

```
DeprecationWarning: scipy.optimize: The `disp` and `iprint` options of the L-BFGS-B solver are deprecated and will be removed in SciPy 1.18.0.
```

The [documentation page of scipy.minimize](https://docs.scipy.org/doc/scipy-1.16.0/reference/optimize.minimize-lbfgsb.html#optimize-minimize-lbfgsb) mentions this double deprecation.

### Steps/Code to Reproduce

`python -Wd`
```python
>>> from sklearn.linear_model import LogisticRegression
>>> import numpy as np
>>> X = np.array([[1], [0]])
>>> y = np.array([1, 0])
>>> LogisticRegression().fit(X, y)
DeprecationWarning: scipy.optimize: The `disp` and `iprint` options of the L-BFGS-B solver are deprecated and will be removed in SciPy 1.18.0.
  opt_res = optimize.minimize(
```

### Expected Results

No deprecation warning

### Actual Results

See above

### Versions

```shell
System:
    python: 3.12.4 (main, Jul 25 2024, 22:11:22) [Clang 18.1.8 ]
executable: /Users/vincentmaladiere/dev/inria/skrub/.venv/bin/python
   machine: macOS-14.0-arm64-arm-64bit

Python dependencies:
      sklearn: 1.7.0
          pip: None
   setuptools: 80.9.0
        numpy: 2.3.1
        scipy: 1.16.0
       Cython: None
       pandas: 2.3.1
   matplotlib: 3.10.3
       joblib: 1.5.1
threadpoolctl: 3.6.0

Built with OpenMP: True

threadpoolctl info:
       user_api: openmp
   internal_api: openmp
    num_threads: 8
         prefix: libomp
       filepath: /Users/vincentmaladiere/dev/inria/skrub/.venv/lib/python3.12/site-packages/sklearn/.dylibs/libomp.dylib
        version: None
```

COMMENT:
Hi @Vincent-Maladiere,
@lesteve has already taken care of this in [31642](https://github.com/scikit-learn/scikit-learn/pull/31642).

I will close because it's completed. Thanks though :)

Labels: Bug
Making the extension contract stable through version upgrades

### Describe the workflow you want to enable

Currently, every time `scikit-learn` releases a new minor version - e.g., 1.5.0, 1.6.0, 1.7.0 - compliant extensions, e.g., custom transformers, classifiers, etc, break - specifically, referring to the API conformance as tested through `check_estimator` or `parametrize_with_checks`.

These repeated breakages in the "extender contract" contrast the stability of the usage contract, which is stable and professionally managed.

For a package like `scikit-learn` which means to be a standard not just for ML algorithms but also an API standard that everyone uses, this is not a good state to be in - "do not break user code" is the maxim that gets broken for power users writing extensions.

Of course maintaining downwards compatibility is not always possible, but nothing should break without a proper warning.

### Describe your proposed solution

The main reason imo why this keeps happening is that `scikit-learn` is not using a proper pattern that ensures stability of the extension contract - and also no secondary deprecation patterns in relation to it.

A simple pattern that could improve a lot would be the "template pattern", in a specific form to separate likely changing parts such as the boilerplate (e.g., `validate_data` vs `_validate_data` and such) from the extension locus.
Reference: https://refactoring.guru/design-patterns/template-method

Examples of how this can be used to improve stability:

* `sktime`, for a different API, has a separation between `fit` calling an internal `_fit`, where change-prone boilerplate is sandwiched between a stable user contract (`fit`) and a stable extender contract (`_fit`); similarly `predict` and `_predict`
* `feature-engine` overrides the `BaseTransformer` `scikit-learn` extension contract with a similar pattern using `super()` calls in `fit` etc.

In particular the `fit`/`_fit` pairing that combines strategy and template pattern can be introduced easily via pure internal refactoring - I would go with that, and also add a layer of tests that checks the stability of the extension contract.

As a side effect, this would likely reduce the amount of copy-pasted boilerplate among the core estimators in `scikit-learn` proper as well.

### Describe alternatives you've considered, if relevant

The obvious alternative is "do nothing" - which will lead to packages with extensions producing their own version of mitigating patterns for the unstable status quo.

The problem with this is that the extending packages are mutually incompatible with their extender pattern. It would be better if there is a single stable extension pattern prescribed - or at least recommended - by `scikit-learn`.

### Additional context

_No response_

COMMENT:
PS: happy to contribute a refactor which is completely internal and downwards compatible to the template pattern if the core devs can agree that this is desired.

Labels: New Feature, Developer API
Making the extension contract stable through version upgrades

### Describe the workflow you want to enable

Currently, every time `scikit-learn` releases a new minor version - e.g., 1.5.0, 1.6.0, 1.7.0 - compliant extensions, e.g., custom transformers, classifiers, etc, break - specifically, referring to the API conformance as tested through `check_estimator` or `parametrize_with_checks`.

These repeated breakages in the "extender contract" contrast the stability of the usage contract, which is stable and professionally managed.

For a package like `scikit-learn` which means to be a standard not just for ML algorithms but also an API standard that everyone uses, this is not a good state to be in - "do not break user code" is the maxim that gets broken for power users writing extensions.

Of course maintaining downwards compatibility is not always possible, but nothing should break without a proper warning.

### Describe your proposed solution

The main reason imo why this keeps happening is that `scikit-learn` is not using a proper pattern that ensures stability of the extension contract - and also no secondary deprecation patterns in relation to it.

A simple pattern that could improve a lot would be the "template pattern", in a specific form to separate likely changing parts such as the boilerplate (e.g., `validate_data` vs `_validate_data` and such) from the extension locus.
Reference: https://refactoring.guru/design-patterns/template-method

Examples of how this can be used to improve stability:

* `sktime`, for a different API, has a separation between `fit` calling an internal `_fit`, where change-prone boilerplate is sandwiched between a stable user contract (`fit`) and a stable extender contract (`_fit`); similarly `predict` and `_predict`
* `feature-engine` overrides the `BaseTransformer` `scikit-learn` extension contract with a similar pattern using `super()` calls in `fit` etc.

In particular the `fit`/`_fit` pairing that combines strategy and template pattern can be introduced easily via pure internal refactoring - I would go with that, and also add a layer of tests that checks the stability of the extension contract.

As a side effect, this would likely reduce the amount of copy-pasted boilerplate among the core estimators in `scikit-learn` proper as well.

### Describe alternatives you've considered, if relevant

The obvious alternative is "do nothing" - which will lead to packages with extensions producing their own version of mitigating patterns for the unstable status quo.

The problem with this is that the extending packages are mutually incompatible with their extender pattern. It would be better if there is a single stable extension pattern prescribed - or at least recommended - by `scikit-learn`.

### Additional context

_No response_

COMMENT:
> and then propagated through the whole project (which is a lot).

Due to the current extension contract overriding `fit` etc, it can be propagated incrementally, similar to the array API changes - it does not need to be done in a single PR.

Labels: New Feature, Developer API
Making the extension contract stable through version upgrades

### Describe the workflow you want to enable

Currently, every time `scikit-learn` releases a new minor version - e.g., 1.5.0, 1.6.0, 1.7.0 - compliant extensions, e.g., custom transformers, classifiers, etc, break - specifically, referring to the API conformance as tested through `check_estimator` or `parametrize_with_checks`.

These repeated breakages in the "extender contract" contrast the stability of the usage contract, which is stable and professionally managed.

For a package like `scikit-learn` which means to be a standard not just for ML algorithms but also an API standard that everyone uses, this is not a good state to be in - "do not break user code" is the maxim that gets broken for power users writing extensions.

Of course maintaining downwards compatibility is not always possible, but nothing should break without a proper warning.

### Describe your proposed solution

The main reason imo why this keeps happening is that `scikit-learn` is not using a proper pattern that ensures stability of the extension contract - and also no secondary deprecation patterns in relation to it.

A simple pattern that could improve a lot would be the "template pattern", in a specific form to separate likely changing parts such as the boilerplate (e.g., `validate_data` vs `_validate_data` and such) from the extension locus.
Reference: https://refactoring.guru/design-patterns/template-method

Examples of how this can be used to improve stability:

* `sktime`, for a different API, has a separation between `fit` calling an internal `_fit`, where change-prone boilerplate is sandwiched between a stable user contract (`fit`) and a stable extender contract (`_fit`); similarly `predict` and `_predict`
* `feature-engine` overrides the `BaseTransformer` `scikit-learn` extension contract with a similar pattern using `super()` calls in `fit` etc.

In particular the `fit`/`_fit` pairing that combines strategy and template pattern can be introduced easily via pure internal refactoring - I would go with that, and also add a layer of tests that checks the stability of the extension contract.

As a side effect, this would likely reduce the amount of copy-pasted boilerplate among the core estimators in `scikit-learn` proper as well.

### Describe alternatives you've considered, if relevant

The obvious alternative is "do nothing" - which will lead to packages with extensions producing their own version of mitigating patterns for the unstable status quo.

The problem with this is that the extending packages are mutually incompatible with their extender pattern. It would be better if there is a single stable extension pattern prescribed - or at least recommended - by `scikit-learn`.

### Additional context

_No response_

COMMENT:
Developer API has been a long project, and we'd welcome contributions. But it might be challenging not having all the context and historical reasons for decisions.

However, the perspective of extension developers is invaluable in guiding us forward.

Labels: New Feature, Developer API
Making the extension contract stable through version upgrades

### Describe the workflow you want to enable

Currently, every time `scikit-learn` releases a new minor version - e.g., 1.5.0, 1.6.0, 1.7.0 - compliant extensions, e.g., custom transformers, classifiers, etc, break - specifically, referring to the API conformance as tested through `check_estimator` or `parametrize_with_checks`.

These repeated breakages in the "extender contract" contrast the stability of the usage contract, which is stable and professionally managed.

For a package like `scikit-learn` which means to be a standard not just for ML algorithms but also an API standard that everyone uses, this is not a good state to be in - "do not break user code" is the maxim that gets broken for power users writing extensions.

Of course maintaining downwards compatibility is not always possible, but nothing should break without a proper warning.

### Describe your proposed solution

The main reason imo why this keeps happening is that `scikit-learn` is not using a proper pattern that ensures stability of the extension contract - and also no secondary deprecation patterns in relation to it.

A simple pattern that could improve a lot would be the "template pattern", in a specific form to separate likely changing parts such as the boilerplate (e.g., `validate_data` vs `_validate_data` and such) from the extension locus.
Reference: https://refactoring.guru/design-patterns/template-method

Examples of how this can be used to improve stability:

* `sktime`, for a different API, has a separation between `fit` calling an internal `_fit`, where change-prone boilerplate is sandwiched between a stable user contract (`fit`) and a stable extender contract (`_fit`); similarly `predict` and `_predict`
* `feature-engine` overrides the `BaseTransformer` `scikit-learn` extension contract with a similar pattern using `super()` calls in `fit` etc.

In particular the `fit`/`_fit` pairing that combines strategy and template pattern can be introduced easily via pure internal refactoring - I would go with that, and also add a layer of tests that checks the stability of the extension contract.

As a side effect, this would likely reduce the amount of copy-pasted boilerplate among the core estimators in `scikit-learn` proper as well.

### Describe alternatives you've considered, if relevant

The obvious alternative is "do nothing" - which will lead to packages with extensions producing their own version of mitigating patterns for the unstable status quo.

The problem with this is that the extending packages are mutually incompatible with their extender pattern. It would be better if there is a single stable extension pattern prescribed - or at least recommended - by `scikit-learn`.

### Additional context

_No response_

COMMENT:
Another point which we've talked on and off has been using protocols to define input/output APIs. That one might be a good place to work too.

Labels: New Feature, Developer API
Making the extension contract stable through version upgrades

### Describe the workflow you want to enable

Currently, every time `scikit-learn` releases a new minor version - e.g., 1.5.0, 1.6.0, 1.7.0 - compliant extensions, e.g., custom transformers, classifiers, etc, break - specifically, referring to the API conformance as tested through `check_estimator` or `parametrize_with_checks`.

These repeated breakages in the "extender contract" contrast the stability of the usage contract, which is stable and professionally managed.

For a package like `scikit-learn` which means to be a standard not just for ML algorithms but also an API standard that everyone uses, this is not a good state to be in - "do not break user code" is the maxim that gets broken for power users writing extensions.

Of course maintaining downwards compatibility is not always possible, but nothing should break without a proper warning.

### Describe your proposed solution

The main reason imo why this keeps happening is that `scikit-learn` is not using a proper pattern that ensures stability of the extension contract - and also no secondary deprecation patterns in relation to it.

A simple pattern that could improve a lot would be the "template pattern", in a specific form to separate likely changing parts such as the boilerplate (e.g., `validate_data` vs `_validate_data` and such) from the extension locus.
Reference: https://refactoring.guru/design-patterns/template-method

Examples of how this can be used to improve stability:

* `sktime`, for a different API, has a separation between `fit` calling an internal `_fit`, where change-prone boilerplate is sandwiched between a stable user contract (`fit`) and a stable extender contract (`_fit`); similarly `predict` and `_predict`
* `feature-engine` overrides the `BaseTransformer` `scikit-learn` extension contract with a similar pattern using `super()` calls in `fit` etc.

In particular the `fit`/`_fit` pairing that combines strategy and template pattern can be introduced easily via pure internal refactoring - I would go with that, and also add a layer of tests that checks the stability of the extension contract.

As a side effect, this would likely reduce the amount of copy-pasted boilerplate among the core estimators in `scikit-learn` proper as well.

### Describe alternatives you've considered, if relevant

The obvious alternative is "do nothing" - which will lead to packages with extensions producing their own version of mitigating patterns for the unstable status quo.

The problem with this is that the extending packages are mutually incompatible with their extender pattern. It would be better if there is a single stable extension pattern prescribed - or at least recommended - by `scikit-learn`.

### Additional context

_No response_

COMMENT:
Opened issue here: https://github.com/scikit-learn/scikit-learn/issues/31912

Labels: New Feature, Developer API
Confusion around coef_ and intercept_ for Polynomial Ridge Regression inside a Pipeline

### Describe the issue linked to the documentation

When using a Pipeline with PolynomialFeatures and Ridge, it's unclear in the documentation how to extract the actual model coefficients and intercept to reproduce the regression equation manually (outside scikit-learn).

For example, when fitting a polynomial regression with:

make_pipeline(PolynomialFeatures(degree=3), Ridge())

Most users wrongly assume that coef_[0] is the intercept, which it is not. This behavior is not explained clearly in the Ridge or Pipeline documentation and led to confusion even after reading the docs and searching online.

This is a common use case — for example, when exporting trained models to plain Python, Java, or C++.

### Suggest a potential alternative/fix

### ✅ Suggested Fix


The coefficients returned by `.coef_` include the weight for the constant basis function (created by `PolynomialFeatures`), but the actual y-intercept is stored separately in `.intercept_`. This makes it unclear how to reconstruct an equation like:

y = a·x³ + b·x² + c·x + d

### Suggested Fix:

1. In the `Ridge`, `Pipeline`, and/or `PolynomialFeatures` documentation, add a clear explanation that:
   - `PolynomialFeatures(degree=n)` creates features `[1, x, x², ..., xⁿ]`
   - The intercept is **not** included in `.coef_`, but is returned separately as `.intercept_`
   - The first element of `.coef_` corresponds to the coefficient of the constant term `1`, not the model intercept

2. Provided a code snippet that reconstructs the polynomial using both:

```python
coefs = model.named_steps['ridge'].coef_
intercept = model.named_steps['ridge'].intercept_
```

This change would help students and developers trying to reproduce the regression manually in another language or platform.

COMMENT:
Hi! 👋 I'm a beginner looking to contribute to open source, and this issue looks like a great place to start. I'd love to work on clarifying the documentation.

Is anyone already working on this? If not, may I take it up?

Thanks! 🙌

Labels: Documentation, spam, Needs Triage
Confusion around coef_ and intercept_ for Polynomial Ridge Regression inside a Pipeline

### Describe the issue linked to the documentation

When using a Pipeline with PolynomialFeatures and Ridge, it's unclear in the documentation how to extract the actual model coefficients and intercept to reproduce the regression equation manually (outside scikit-learn).

For example, when fitting a polynomial regression with:

make_pipeline(PolynomialFeatures(degree=3), Ridge())

Most users wrongly assume that coef_[0] is the intercept, which it is not. This behavior is not explained clearly in the Ridge or Pipeline documentation and led to confusion even after reading the docs and searching online.

This is a common use case — for example, when exporting trained models to plain Python, Java, or C++.

### Suggest a potential alternative/fix

### ✅ Suggested Fix


The coefficients returned by `.coef_` include the weight for the constant basis function (created by `PolynomialFeatures`), but the actual y-intercept is stored separately in `.intercept_`. This makes it unclear how to reconstruct an equation like:

y = a·x³ + b·x² + c·x + d

### Suggested Fix:

1. In the `Ridge`, `Pipeline`, and/or `PolynomialFeatures` documentation, add a clear explanation that:
   - `PolynomialFeatures(degree=n)` creates features `[1, x, x², ..., xⁿ]`
   - The intercept is **not** included in `.coef_`, but is returned separately as `.intercept_`
   - The first element of `.coef_` corresponds to the coefficient of the constant term `1`, not the model intercept

2. Provided a code snippet that reconstructs the polynomial using both:

```python
coefs = model.named_steps['ridge'].coef_
intercept = model.named_steps['ridge'].intercept_
```

This change would help students and developers trying to reproduce the regression manually in another language or platform.

COMMENT:
Hi! 👋 I’ve already created a PR for this issue as my first open-source contribution to scikit-learn — here it is: #31727. Thanks for your interest! 😊

Labels: Documentation, spam, Needs Triage
Confusion around coef_ and intercept_ for Polynomial Ridge Regression inside a Pipeline

### Describe the issue linked to the documentation

When using a Pipeline with PolynomialFeatures and Ridge, it's unclear in the documentation how to extract the actual model coefficients and intercept to reproduce the regression equation manually (outside scikit-learn).

For example, when fitting a polynomial regression with:

make_pipeline(PolynomialFeatures(degree=3), Ridge())

Most users wrongly assume that coef_[0] is the intercept, which it is not. This behavior is not explained clearly in the Ridge or Pipeline documentation and led to confusion even after reading the docs and searching online.

This is a common use case — for example, when exporting trained models to plain Python, Java, or C++.

### Suggest a potential alternative/fix

### ✅ Suggested Fix


The coefficients returned by `.coef_` include the weight for the constant basis function (created by `PolynomialFeatures`), but the actual y-intercept is stored separately in `.intercept_`. This makes it unclear how to reconstruct an equation like:

y = a·x³ + b·x² + c·x + d

### Suggested Fix:

1. In the `Ridge`, `Pipeline`, and/or `PolynomialFeatures` documentation, add a clear explanation that:
   - `PolynomialFeatures(degree=n)` creates features `[1, x, x², ..., xⁿ]`
   - The intercept is **not** included in `.coef_`, but is returned separately as `.intercept_`
   - The first element of `.coef_` corresponds to the coefficient of the constant term `1`, not the model intercept

2. Provided a code snippet that reconstructs the polynomial using both:

```python
coefs = model.named_steps['ridge'].coef_
intercept = model.named_steps['ridge'].intercept_
```

This change would help students and developers trying to reproduce the regression manually in another language or platform.

COMMENT:
Wait, is this a re-statement of issue #31719 that was opened two days ago by @MikeB2019x with @virchan 's answer on it?

That's not okay, @Sanjaykumar030.

Labels: Documentation, spam, Needs Triage