Talip7/scikit-learn-issues-multilabel huggingface dataset:
LinearRegression in User Guide seems to have $w_0=0$

From the documentation of https://scikit-learn.org/stable/modules/linear_model.html#ordinary-least-squares, one could understand that LinearRegression fits a model where the intercept $w_0$ is absent, that is, where $w_0=0$.

To prevent this misinterpretation, I suggest rephrasing (i) "LinearRegression fits a linear model with coefficients $w = (w_1, ..., w_p)" as "LinearRegression fits a linear model with coefficients $w = (w_0, w_1, ..., w_p)" AND (ii) "and will store the coefficients of the linear model in its coef_ member" as "and will store the coefficients of the linear model in its coef_ and intercept_ members"

COMMENT:
Is this issue still open? I'd like to work on it.

Labels: Documentation, module:linear_model
Link to the latest Developer's guide rather than to stable.

### Describe the issue linked to the documentation

The scikit-learn website links to the stable version of the [Developer's guide](https://scikit-learn.org/stable/developers/index.html), while [CONTRIBUTING.md](https://github.com/scikit-learn/scikit-learn/blob/main/CONTRIBUTING.md) links to the development version (latest) of the same guide.
This is inconsistent and makes contributors miss recent modification about how to contribute, eg, about github workflow modifications.

### Suggest a potential alternative/fix

I think that the "Development" item in the "More" menu on the homepage should rather point to the [latest](https://scikit-learn.org/dev/developers/contributing.html) version of the documentation.

COMMENT:
If we go with this, then I think it is best to open a new page when we link to the dev version of the developer guide. If we go directly without a new page, a user would be on the dev page, which has unreleased API, etc.

The alternative is to be proactive in merging documentation changes into the release branch so that the `stable` site updates.

**Edit**: Grammar correction: "If we go with this, ..."

Labels: Documentation
Link to the latest Developer's guide rather than to stable.

### Describe the issue linked to the documentation

The scikit-learn website links to the stable version of the [Developer's guide](https://scikit-learn.org/stable/developers/index.html), while [CONTRIBUTING.md](https://github.com/scikit-learn/scikit-learn/blob/main/CONTRIBUTING.md) links to the development version (latest) of the same guide.
This is inconsistent and makes contributors miss recent modification about how to contribute, eg, about github workflow modifications.

### Suggest a potential alternative/fix

I think that the "Development" item in the "More" menu on the homepage should rather point to the [latest](https://scikit-learn.org/dev/developers/contributing.html) version of the documentation.

COMMENT:
> If we go this open, then I think it is best to open a new page when we link to the dev version of the developer guide. If we go directly without a new page, a user would be on the dev page, which has unreleased API, etc.

Please do not add a new page. This will just add more confusion.
The dev version is already linked in CONTRIBUTING.md, a big banner is warning that the dev version is displayed.
The Contributor guide should be the updated list of actions a contributor should be aware of in contributing to main, where the last changes already apply.
Two small examples:
- the "take" -> "\take" change,
- the bump of Black version.

Labels: Documentation
Link to the latest Developer's guide rather than to stable.

### Describe the issue linked to the documentation

The scikit-learn website links to the stable version of the [Developer's guide](https://scikit-learn.org/stable/developers/index.html), while [CONTRIBUTING.md](https://github.com/scikit-learn/scikit-learn/blob/main/CONTRIBUTING.md) links to the development version (latest) of the same guide.
This is inconsistent and makes contributors miss recent modification about how to contribute, eg, about github workflow modifications.

### Suggest a potential alternative/fix

I think that the "Development" item in the "More" menu on the homepage should rather point to the [latest](https://scikit-learn.org/dev/developers/contributing.html) version of the documentation.

COMMENT:
> Please do not add a new page. This will just add more confusion.

I suggested to **open** a new page. In other words, use the "blank" target in the anchor. For example, this opens a new page to the scikit-learn repo:

```html
<a href="https://github.com/scikit-learn/scikit-learn" target="_blank">Scikit-learn Repo</a>
```

Labels: Documentation
Link to the latest Developer's guide rather than to stable.

### Describe the issue linked to the documentation

The scikit-learn website links to the stable version of the [Developer's guide](https://scikit-learn.org/stable/developers/index.html), while [CONTRIBUTING.md](https://github.com/scikit-learn/scikit-learn/blob/main/CONTRIBUTING.md) links to the development version (latest) of the same guide.
This is inconsistent and makes contributors miss recent modification about how to contribute, eg, about github workflow modifications.

### Suggest a potential alternative/fix

I think that the "Development" item in the "More" menu on the homepage should rather point to the [latest](https://scikit-learn.org/dev/developers/contributing.html) version of the documentation.

COMMENT:
> I suggested to **open** a new page. In other words, use the "blank" target in the anchor. For example, this opens a new page to the scikit-learn repo:

Thanks for clarifying. I don't see the advantage, but if you feel more comfortable with that I have no preferences.

Labels: Documentation
Link to the latest Developer's guide rather than to stable.

### Describe the issue linked to the documentation

The scikit-learn website links to the stable version of the [Developer's guide](https://scikit-learn.org/stable/developers/index.html), while [CONTRIBUTING.md](https://github.com/scikit-learn/scikit-learn/blob/main/CONTRIBUTING.md) links to the development version (latest) of the same guide.
This is inconsistent and makes contributors miss recent modification about how to contribute, eg, about github workflow modifications.

### Suggest a potential alternative/fix

I think that the "Development" item in the "More" menu on the homepage should rather point to the [latest](https://scikit-learn.org/dev/developers/contributing.html) version of the documentation.

COMMENT:
Now that I am clicking around, I am okay with linking directly to the development page as you originally suggested.

Labels: Documentation
Link to the latest Developer's guide rather than to stable.

### Describe the issue linked to the documentation

The scikit-learn website links to the stable version of the [Developer's guide](https://scikit-learn.org/stable/developers/index.html), while [CONTRIBUTING.md](https://github.com/scikit-learn/scikit-learn/blob/main/CONTRIBUTING.md) links to the development version (latest) of the same guide.
This is inconsistent and makes contributors miss recent modification about how to contribute, eg, about github workflow modifications.

### Suggest a potential alternative/fix

I think that the "Development" item in the "More" menu on the homepage should rather point to the [latest](https://scikit-learn.org/dev/developers/contributing.html) version of the documentation.

COMMENT:
I opened #22550 to link to the latest developer guide directly as suggested. I think the big red banner is enough to see that it is the development page.

Labels: Documentation
RidgeCV replacement for normalize=True gives different results

### Describe the bug

Running RidgeCV with `normalize=True` results in a Future Warning that provides the new preferred method for normalizing prior to fitting a regularized regression:
FutureWarning: 'normalize' was deprecated in version 1.0 and will be removed in 1.2.
If you wish to scale the data, use Pipeline with a StandardScaler in a preprocessing stage. To reproduce the previous behavior:
```python
from sklearn.pipeline import make_pipeline

model = make_pipeline(StandardScaler(with_mean=False), _RidgeGCV())
```

However, when I try to implement this, I get very different results than with normalize=True, making me think perhaps I didn't know what normalize was doing in the first place.

### Steps/Code to Reproduce

```python
from sklearn.datasets import make_regression
from sklearn import linear_model
from sklearn.pipeline import make_pipeline

X, y, w = make_regression(
    n_samples=10, n_features=10, coef=True, random_state=1, bias=3.5
)

reg = linear_model.RidgeCV(normalize=True)

reg.fit(X, y)
print('old method: ', reg.coef_, reg.intercept_, reg.alpha_, reg.predict(X))

model = make_pipeline(StandardScaler(with_mean=False), linear_model.RidgeCV())
model.fit(X, y)
print('new method: ',model['ridgecv'].coef_, model['ridgecv'].intercept_, model['ridgecv'].alpha_, model.predict(X))
```

### Expected Results

I would expect these to have the same result.

### Actual Results

```
old method:  [ 31.71587415 -22.24842109  16.39309719  21.21120971   6.0422928
  55.09416648  12.35520845 -31.34002266  41.30725923  71.19405147] 20.03577681177929 0.1 [  25.21745712   53.95958306  -68.09400539 -201.61697665   48.18804729
   -2.41519428   30.21811352  -25.23316533  108.24806911   27.85799418]
```
```
new method:  [ 64.42944029 -12.21304369  20.65494721  18.33833625   5.31778008
  50.76979275   9.707485   -16.44095044  51.70245172  23.1077787 ] 30.278259703366963 0.1 [  42.1051797    59.36215179  -82.18834535 -215.193882     55.814325
   -1.73335152   30.27829894  -40.63725969  108.43678068   40.08602508]
```
### Versions

```shell
System:
    python: 3.7.9 (default, Aug 31 2020, 12:42:55)  [GCC 7.3.0]
executable: /home1/djhalp/.conda/envs/cml/bin/python
   machine: Linux-3.10.0-693.17.1.el7.x86_64-x86_64-with-centos-7.4.1708-Core

Python dependencies:
          pip: 21.2.2
   setuptools: 58.0.4
      sklearn: 1.0.2
        numpy: 1.20.3
        scipy: 1.7.1
       Cython: 0.29.23
       pandas: 1.3.5
   matplotlib: 3.5.0
       joblib: 1.1.0
threadpoolctl: 2.2.0

Built with OpenMP: True
```

COMMENT:
Interesting, thanks for looking into this! Is the reason for this change written up anywhere? I didn't realize that normalize was changing the amount of regularization based on sample size.

Labels: Bug, module:linear_model
RidgeCV replacement for normalize=True gives different results

### Describe the bug

Running RidgeCV with `normalize=True` results in a Future Warning that provides the new preferred method for normalizing prior to fitting a regularized regression:
FutureWarning: 'normalize' was deprecated in version 1.0 and will be removed in 1.2.
If you wish to scale the data, use Pipeline with a StandardScaler in a preprocessing stage. To reproduce the previous behavior:
```python
from sklearn.pipeline import make_pipeline

model = make_pipeline(StandardScaler(with_mean=False), _RidgeGCV())
```

However, when I try to implement this, I get very different results than with normalize=True, making me think perhaps I didn't know what normalize was doing in the first place.

### Steps/Code to Reproduce

```python
from sklearn.datasets import make_regression
from sklearn import linear_model
from sklearn.pipeline import make_pipeline

X, y, w = make_regression(
    n_samples=10, n_features=10, coef=True, random_state=1, bias=3.5
)

reg = linear_model.RidgeCV(normalize=True)

reg.fit(X, y)
print('old method: ', reg.coef_, reg.intercept_, reg.alpha_, reg.predict(X))

model = make_pipeline(StandardScaler(with_mean=False), linear_model.RidgeCV())
model.fit(X, y)
print('new method: ',model['ridgecv'].coef_, model['ridgecv'].intercept_, model['ridgecv'].alpha_, model.predict(X))
```

### Expected Results

I would expect these to have the same result.

### Actual Results

```
old method:  [ 31.71587415 -22.24842109  16.39309719  21.21120971   6.0422928
  55.09416648  12.35520845 -31.34002266  41.30725923  71.19405147] 20.03577681177929 0.1 [  25.21745712   53.95958306  -68.09400539 -201.61697665   48.18804729
   -2.41519428   30.21811352  -25.23316533  108.24806911   27.85799418]
```
```
new method:  [ 64.42944029 -12.21304369  20.65494721  18.33833625   5.31778008
  50.76979275   9.707485   -16.44095044  51.70245172  23.1077787 ] 30.278259703366963 0.1 [  42.1051797    59.36215179  -82.18834535 -215.193882     55.814325
   -1.73335152   30.27829894  -40.63725969  108.43678068   40.08602508]
```
### Versions

```shell
System:
    python: 3.7.9 (default, Aug 31 2020, 12:42:55)  [GCC 7.3.0]
executable: /home1/djhalp/.conda/envs/cml/bin/python
   machine: Linux-3.10.0-693.17.1.el7.x86_64-x86_64-with-centos-7.4.1708-Core

Python dependencies:
          pip: 21.2.2
   setuptools: 58.0.4
      sklearn: 1.0.2
        numpy: 1.20.3
        scipy: 1.7.1
       Cython: 0.29.23
       pandas: 1.3.5
   matplotlib: 3.5.0
       joblib: 1.1.0
threadpoolctl: 2.2.0

Built with OpenMP: True
```

COMMENT:
ping @agramfort maybe you'll explain more clearly, I wasn't part of the 'normalize' deprecation.

Labels: Bug, module:linear_model
RidgeCV replacement for normalize=True gives different results

### Describe the bug

Running RidgeCV with `normalize=True` results in a Future Warning that provides the new preferred method for normalizing prior to fitting a regularized regression:
FutureWarning: 'normalize' was deprecated in version 1.0 and will be removed in 1.2.
If you wish to scale the data, use Pipeline with a StandardScaler in a preprocessing stage. To reproduce the previous behavior:
```python
from sklearn.pipeline import make_pipeline

model = make_pipeline(StandardScaler(with_mean=False), _RidgeGCV())
```

However, when I try to implement this, I get very different results than with normalize=True, making me think perhaps I didn't know what normalize was doing in the first place.

### Steps/Code to Reproduce

```python
from sklearn.datasets import make_regression
from sklearn import linear_model
from sklearn.pipeline import make_pipeline

X, y, w = make_regression(
    n_samples=10, n_features=10, coef=True, random_state=1, bias=3.5
)

reg = linear_model.RidgeCV(normalize=True)

reg.fit(X, y)
print('old method: ', reg.coef_, reg.intercept_, reg.alpha_, reg.predict(X))

model = make_pipeline(StandardScaler(with_mean=False), linear_model.RidgeCV())
model.fit(X, y)
print('new method: ',model['ridgecv'].coef_, model['ridgecv'].intercept_, model['ridgecv'].alpha_, model.predict(X))
```

### Expected Results

I would expect these to have the same result.

### Actual Results

```
old method:  [ 31.71587415 -22.24842109  16.39309719  21.21120971   6.0422928
  55.09416648  12.35520845 -31.34002266  41.30725923  71.19405147] 20.03577681177929 0.1 [  25.21745712   53.95958306  -68.09400539 -201.61697665   48.18804729
   -2.41519428   30.21811352  -25.23316533  108.24806911   27.85799418]
```
```
new method:  [ 64.42944029 -12.21304369  20.65494721  18.33833625   5.31778008
  50.76979275   9.707485   -16.44095044  51.70245172  23.1077787 ] 30.278259703366963 0.1 [  42.1051797    59.36215179  -82.18834535 -215.193882     55.814325
   -1.73335152   30.27829894  -40.63725969  108.43678068   40.08602508]
```
### Versions

```shell
System:
    python: 3.7.9 (default, Aug 31 2020, 12:42:55)  [GCC 7.3.0]
executable: /home1/djhalp/.conda/envs/cml/bin/python
   machine: Linux-3.10.0-693.17.1.el7.x86_64-x86_64-with-centos-7.4.1708-Core

Python dependencies:
          pip: 21.2.2
   setuptools: 58.0.4
      sklearn: 1.0.2
        numpy: 1.20.3
        scipy: 1.7.1
       Cython: 0.29.23
       pandas: 1.3.5
   matplotlib: 3.5.0
       joblib: 1.1.0
threadpoolctl: 2.2.0

Built with OpenMP: True
```

COMMENT:
thanks @ogrisel !

> Message ID: <scikit-learn/scikit-learn/issue/22540/issue_event/6120152008@
> github.com>
>

Labels: Bug, module:linear_model
RidgeCV replacement for normalize=True gives different results

### Describe the bug

Running RidgeCV with `normalize=True` results in a Future Warning that provides the new preferred method for normalizing prior to fitting a regularized regression:
FutureWarning: 'normalize' was deprecated in version 1.0 and will be removed in 1.2.
If you wish to scale the data, use Pipeline with a StandardScaler in a preprocessing stage. To reproduce the previous behavior:
```python
from sklearn.pipeline import make_pipeline

model = make_pipeline(StandardScaler(with_mean=False), _RidgeGCV())
```

However, when I try to implement this, I get very different results than with normalize=True, making me think perhaps I didn't know what normalize was doing in the first place.

### Steps/Code to Reproduce

```python
from sklearn.datasets import make_regression
from sklearn import linear_model
from sklearn.pipeline import make_pipeline

X, y, w = make_regression(
    n_samples=10, n_features=10, coef=True, random_state=1, bias=3.5
)

reg = linear_model.RidgeCV(normalize=True)

reg.fit(X, y)
print('old method: ', reg.coef_, reg.intercept_, reg.alpha_, reg.predict(X))

model = make_pipeline(StandardScaler(with_mean=False), linear_model.RidgeCV())
model.fit(X, y)
print('new method: ',model['ridgecv'].coef_, model['ridgecv'].intercept_, model['ridgecv'].alpha_, model.predict(X))
```

### Expected Results

I would expect these to have the same result.

### Actual Results

```
old method:  [ 31.71587415 -22.24842109  16.39309719  21.21120971   6.0422928
  55.09416648  12.35520845 -31.34002266  41.30725923  71.19405147] 20.03577681177929 0.1 [  25.21745712   53.95958306  -68.09400539 -201.61697665   48.18804729
   -2.41519428   30.21811352  -25.23316533  108.24806911   27.85799418]
```
```
new method:  [ 64.42944029 -12.21304369  20.65494721  18.33833625   5.31778008
  50.76979275   9.707485   -16.44095044  51.70245172  23.1077787 ] 30.278259703366963 0.1 [  42.1051797    59.36215179  -82.18834535 -215.193882     55.814325
   -1.73335152   30.27829894  -40.63725969  108.43678068   40.08602508]
```
### Versions

```shell
System:
    python: 3.7.9 (default, Aug 31 2020, 12:42:55)  [GCC 7.3.0]
executable: /home1/djhalp/.conda/envs/cml/bin/python
   machine: Linux-3.10.0-693.17.1.el7.x86_64-x86_64-with-centos-7.4.1708-Core

Python dependencies:
          pip: 21.2.2
   setuptools: 58.0.4
      sklearn: 1.0.2
        numpy: 1.20.3
        scipy: 1.7.1
       Cython: 0.29.23
       pandas: 1.3.5
   matplotlib: 3.5.0
       joblib: 1.1.0
threadpoolctl: 2.2.0

Built with OpenMP: True
```

COMMENT:
After talking with @jeremiedbb I realized that the warning message for `RidgeCV` is missing the critical information about the scaling of alpha that only appears in the deprecation warning message of `Ridge`. We should indeed fix the deprecation warning message of `RidgeCV` to be consistent with the message of `Ridge`.

Labels: Bug, module:linear_model
Implementation of XRAY solver for NMF per Kumar 2013

### Describe the workflow you want to enable

I was inspired by this talk [https://www.youtube.com/watch?v=kSfwY68gQ9I](url) to attempt an implementation of the XRAY solver for NMF per Kumar 2013: [https://arxiv.org/pdf/1210.1190.pdf](https://arxiv.org/pdf/1210.1190.pdf) and [https://arxiv.org/pdf/1312.7167.pdf](https://arxiv.org/pdf/1312.7167.pdf)

The talk and the paper make some pretty extraordinary claims about uniqueness, speed, and convergence - subject to certain assumptions about the matrix to be factorized.  

Here is the issue - the paper meets the criteria for inclusion, except for the number of citations, which is at 166 according to Google Scholar.  I would be happy to make a PR, but before I expend the effort I'd like a general sense from the maintainers whether the PR is likely to survive *assuming* the paper's claims pan out and the XRAY solver is a significant advancement for sklearn's NMF implementation.

I have reviewed open issues and cannot find any other proposals for implementing the XRAY solver.   I have, however, found a huge open PR for MinibatchNMF, issue #13308.  Presumably once this is released (milestoned for 1.1), the XRAY solver would be supported in that implementation as well.  I mention this only because I worry about that PR dogpiling on top of whatever work I would also do, causing headaches all around.

### Describe your proposed solution

I propose XRAY as an additional solver within the current NMF implementation, alongside cd and mu.  If there is a test for the separability or near-separability assumption, that may take the form of a warning or exception if the condition is not met.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
Hi, ai is it does not pass the strict inclusion criteria. However it could be seen just as an alternative solver for which the rules can be a bit bent. Still it seems that it only works when the matrix to decompose satisfies a bunch of specific hypothesis and I'm not sure this kind of very specific task fits the scikit-learn roadmap.

@TomDLT, @GaelVaroquaux Are you aware of this solver ?

Labels: New Feature, module:decomposition
Implementation of XRAY solver for NMF per Kumar 2013

### Describe the workflow you want to enable

I was inspired by this talk [https://www.youtube.com/watch?v=kSfwY68gQ9I](url) to attempt an implementation of the XRAY solver for NMF per Kumar 2013: [https://arxiv.org/pdf/1210.1190.pdf](https://arxiv.org/pdf/1210.1190.pdf) and [https://arxiv.org/pdf/1312.7167.pdf](https://arxiv.org/pdf/1312.7167.pdf)

The talk and the paper make some pretty extraordinary claims about uniqueness, speed, and convergence - subject to certain assumptions about the matrix to be factorized.  

Here is the issue - the paper meets the criteria for inclusion, except for the number of citations, which is at 166 according to Google Scholar.  I would be happy to make a PR, but before I expend the effort I'd like a general sense from the maintainers whether the PR is likely to survive *assuming* the paper's claims pan out and the XRAY solver is a significant advancement for sklearn's NMF implementation.

I have reviewed open issues and cannot find any other proposals for implementing the XRAY solver.   I have, however, found a huge open PR for MinibatchNMF, issue #13308.  Presumably once this is released (milestoned for 1.1), the XRAY solver would be supported in that implementation as well.  I mention this only because I worry about that PR dogpiling on top of whatever work I would also do, causing headaches all around.

### Describe your proposed solution

I propose XRAY as an additional solver within the current NMF implementation, alongside cd and mu.  If there is a test for the separability or near-separability assumption, that may take the form of a warning or exception if the condition is not met.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
The solver seems rather specific to near-separable cases, and I don't know any way to test that assumption beforehand. Is there an available implementation that we could test to see how it performs in practice ?

In terms of roadmap, I would rather focus our efforts in finishing the mini-batch solver (#16948) first.

Labels: New Feature, module:decomposition