Talip7/scikit-learn-issues-multilabel huggingface dataset:
correct SAGA weight update with prox

It seems that saga weights are being updated here with saga bit after the prox operator has been applied to SAG update
https://github.com/scikit-learn/scikit-learn/blob/8610e14f8a9acd488253444b8e551fb3e0d60ef7/sklearn/linear_model/_sag_fast.pyx.tp#L430

Should the prox operator be applied again here after SAGA weight update? 
Looks like it is what is happeng in lightning [implementation](https://github.com/scikit-learn-contrib/lightning/blob/dbbe833963280e675c124bbd5caadfcb13d89bd7/lightning/impl/sag_fast.pyx#L441)

COMMENT:
This is normal, although the code could be more self-explanatory:
- In this code, the gradient step is not applied every time the gradient is computed. Indeed, if the data is sparse, the random sample `X_i` will change the averaged gradient only on features `j` where `X_ij != 0`. In some cases, the average gradient on feature `j` might change only after `k` random samples with no change. In these cases, instead of applying `k` times the same gradient step on feature `j`, we apply the gradient step only once, scaled by `k`. This is called the "Just-in-time update" (or [`lagged_update`](https://github.com/scikit-learn/scikit-learn/blob/8610e14f8a9acd488253444b8e551fb3e0d60ef7/sklearn/linear_model/_sag_fast.pyx.tp#L587) in the code).
- So the gradient step is performed in `lagged_update`. This function also applies the prox operator after the weights update. Note that because the lagged update actually performs `k` gradient steps, we need to apply `prox(gradient_step(prox(gradient_step(...))))`, which kind of defeats the purpose of the lagged update, but this can be avoided sometimes with a [trick](https://github.com/scikit-learn/scikit-learn/blob/8610e14f8a9acd488253444b8e551fb3e0d60ef7/sklearn/linear_model/_sag_fast.pyx.tp#L631-L634) actually taken from lightning.
- For SAGA, there are also [these lines](https://github.com/scikit-learn/scikit-learn/blob/8610e14f8a9acd488253444b8e551fb3e0d60ef7/sklearn/linear_model/_sag_fast.pyx.tp#L429-L432), which change the weights outside of the lagged update because it is a term in SAGA that does not depend on the average gradient. The prox is applied later, in the following lagged update, before computing the following prediction.

- On top of that, the SAG/A update scales down all the weights at each iteration. So to avoid updating all the weights at each iteration, there is a trick to factor out the scaling down, and to only apply it in the lagged update. When the factored-out scale is too small, it is [reset](https://github.com/scikit-learn/scikit-learn/blob/8610e14f8a9acd488253444b8e551fb3e0d60ef7/sklearn/linear_model/_sag_fast.pyx.tp#L548-L564) for numerical stability, and this reset requires applying all remaining lagged updates. 
- This scale reset is also [performed](https://github.com/scikit-learn/scikit-learn/blob/8610e14f8a9acd488253444b8e551fb3e0d60ef7/sklearn/linear_model/_sag_fast.pyx.tp#L495-L503) every `n_samples` iterations before each convergence check, so when the algorithm stops, we are sure that there is no remaining lagged updates.

Labels: help wanted
correct SAGA weight update with prox

It seems that saga weights are being updated here with saga bit after the prox operator has been applied to SAG update
https://github.com/scikit-learn/scikit-learn/blob/8610e14f8a9acd488253444b8e551fb3e0d60ef7/sklearn/linear_model/_sag_fast.pyx.tp#L430

Should the prox operator be applied again here after SAGA weight update? 
Looks like it is what is happeng in lightning [implementation](https://github.com/scikit-learn-contrib/lightning/blob/dbbe833963280e675c124bbd5caadfcb13d89bd7/lightning/impl/sag_fast.pyx#L441)

COMMENT:
@NikZak does this answer your concern?

Labels: help wanted
correct SAGA weight update with prox

It seems that saga weights are being updated here with saga bit after the prox operator has been applied to SAG update
https://github.com/scikit-learn/scikit-learn/blob/8610e14f8a9acd488253444b8e551fb3e0d60ef7/sklearn/linear_model/_sag_fast.pyx.tp#L430

Should the prox operator be applied again here after SAGA weight update? 
Looks like it is what is happeng in lightning [implementation](https://github.com/scikit-learn-contrib/lightning/blob/dbbe833963280e675c124bbd5caadfcb13d89bd7/lightning/impl/sag_fast.pyx#L441)

COMMENT:
Maybe someone could open a PR to add some inline comments with either a copy of @TomDLT's comment or a link to it.

Labels: help wanted
suggestion to move (copy?) binder button for example to top or right side bar

### Describe the issue linked to the documentation

Firstly, thanks for having binder links for examples!

In the examples such as https://scikit-learn.org/stable/auto_examples/release_highlights/plot_release_highlights_1_1_0.html#sphx-glr-auto-examples-release-highlights-plot-release-highlights-1-1-0-py the binder button is as the bottom of the page

<img width="1469" alt="Screen Shot 2022-10-15 at 11 16 30 PM" src="https://user-images.githubusercontent.com/17162724/196016269-ddf3ddd4-7dbe-4825-ac60-f9c538a6c410.png">

It can sometimes take a while to scroll to get to the button. It would great if it could be accessed quicker with the binder link added to the top or to the right side bar.

### Suggest a potential alternative/fix

I would find it beneficial if the button was at the top (as well as the bottom?) or on the right side bar. This would make it quicker for me to run an example. If the example is long then it can take a bit of scrolling to get to the binder button.

I've mocked up some proposals below. I'm think the button under the title ("A demo on Spectal Biclustering algorithm") would look like a binder button badge as the top of a README e.g. the shield.io format https://mybinder.readthedocs.io/en/latest/howto/badges.html

It's also worth noting that the box in the right isn't accessible and I was able to scroll up to access it

I understand I don't want the badge/link to stick out like a sore thumb and not distract people are simple interested in reading though the examples.

![Screen Shot 2022-10-15 at 11 24 46 PM2](https://user-images.githubusercontent.com/17162724/196016750-134556fb-5839-42ff-8fd2-04e80df58752.png)

COMMENT:
I am not sure we can easily do it. The other links are automatically generated by `sphinx-gallery` and we cannot move them without making some ugly tweaks. It makes a lot of sense to have the binder links next to those download links.

I think that the current UX is a trade-off that we came up with.

@lesteve would probably recall the design choice. If it confirms my thoughts, I think we better keep it as-is and await a broader website redesign that could come at some point.

Labels: Documentation, Needs Decision
suggestion to move (copy?) binder button for example to top or right side bar

### Describe the issue linked to the documentation

Firstly, thanks for having binder links for examples!

In the examples such as https://scikit-learn.org/stable/auto_examples/release_highlights/plot_release_highlights_1_1_0.html#sphx-glr-auto-examples-release-highlights-plot-release-highlights-1-1-0-py the binder button is as the bottom of the page

<img width="1469" alt="Screen Shot 2022-10-15 at 11 16 30 PM" src="https://user-images.githubusercontent.com/17162724/196016269-ddf3ddd4-7dbe-4825-ac60-f9c538a6c410.png">

It can sometimes take a while to scroll to get to the button. It would great if it could be accessed quicker with the binder link added to the top or to the right side bar.

### Suggest a potential alternative/fix

I would find it beneficial if the button was at the top (as well as the bottom?) or on the right side bar. This would make it quicker for me to run an example. If the example is long then it can take a bit of scrolling to get to the binder button.

I've mocked up some proposals below. I'm think the button under the title ("A demo on Spectal Biclustering algorithm") would look like a binder button badge as the top of a README e.g. the shield.io format https://mybinder.readthedocs.io/en/latest/howto/badges.html

It's also worth noting that the box in the right isn't accessible and I was able to scroll up to access it

I understand I don't want the badge/link to stick out like a sore thumb and not distract people are simple interested in reading though the examples.

![Screen Shot 2022-10-15 at 11 24 46 PM2](https://user-images.githubusercontent.com/17162724/196016750-134556fb-5839-42ff-8fd2-04e80df58752.png)

COMMENT:
I think I have seen some other repo where the binder button was floating on the right, but I can't find which one right now. Maybe this has already been discussed in a scikit-learn issue or in the sphinx-gallery repo.

Someone with CSS skills may be able to improve the situation, not 100% sure.

> It can sometimes take a while to scroll to get to the button.

A work-around (not ideal I know) is to press the `End` button (maybe another key depending on your browser) to go to the bottom of the page in a single keystroke.

Labels: Documentation, Needs Decision
suggestion to move (copy?) binder button for example to top or right side bar

### Describe the issue linked to the documentation

Firstly, thanks for having binder links for examples!

In the examples such as https://scikit-learn.org/stable/auto_examples/release_highlights/plot_release_highlights_1_1_0.html#sphx-glr-auto-examples-release-highlights-plot-release-highlights-1-1-0-py the binder button is as the bottom of the page

<img width="1469" alt="Screen Shot 2022-10-15 at 11 16 30 PM" src="https://user-images.githubusercontent.com/17162724/196016269-ddf3ddd4-7dbe-4825-ac60-f9c538a6c410.png">

It can sometimes take a while to scroll to get to the button. It would great if it could be accessed quicker with the binder link added to the top or to the right side bar.

### Suggest a potential alternative/fix

I would find it beneficial if the button was at the top (as well as the bottom?) or on the right side bar. This would make it quicker for me to run an example. If the example is long then it can take a bit of scrolling to get to the binder button.

I've mocked up some proposals below. I'm think the button under the title ("A demo on Spectal Biclustering algorithm") would look like a binder button badge as the top of a README e.g. the shield.io format https://mybinder.readthedocs.io/en/latest/howto/badges.html

It's also worth noting that the box in the right isn't accessible and I was able to scroll up to access it

I understand I don't want the badge/link to stick out like a sore thumb and not distract people are simple interested in reading though the examples.

![Screen Shot 2022-10-15 at 11 24 46 PM2](https://user-images.githubusercontent.com/17162724/196016750-134556fb-5839-42ff-8fd2-04e80df58752.png)

COMMENT:
> A work-around (not ideal I know) is to press the End button (maybe another key depending on your browser) to go to the bottom of the page in a single keystroke.

Actually, we already have the note on the top (or left depending on the screen size) stating "Click here" to go down.

<img width="955" alt="image" src="https://user-images.githubusercontent.com/7454015/196403395-52b12c80-54f1-45f7-9e51-99d62c49c17a.png">

Labels: Documentation, Needs Decision
suggestion to move (copy?) binder button for example to top or right side bar

### Describe the issue linked to the documentation

Firstly, thanks for having binder links for examples!

In the examples such as https://scikit-learn.org/stable/auto_examples/release_highlights/plot_release_highlights_1_1_0.html#sphx-glr-auto-examples-release-highlights-plot-release-highlights-1-1-0-py the binder button is as the bottom of the page

<img width="1469" alt="Screen Shot 2022-10-15 at 11 16 30 PM" src="https://user-images.githubusercontent.com/17162724/196016269-ddf3ddd4-7dbe-4825-ac60-f9c538a6c410.png">

It can sometimes take a while to scroll to get to the button. It would great if it could be accessed quicker with the binder link added to the top or to the right side bar.

### Suggest a potential alternative/fix

I would find it beneficial if the button was at the top (as well as the bottom?) or on the right side bar. This would make it quicker for me to run an example. If the example is long then it can take a bit of scrolling to get to the binder button.

I've mocked up some proposals below. I'm think the button under the title ("A demo on Spectal Biclustering algorithm") would look like a binder button badge as the top of a README e.g. the shield.io format https://mybinder.readthedocs.io/en/latest/howto/badges.html

It's also worth noting that the box in the right isn't accessible and I was able to scroll up to access it

I understand I don't want the badge/link to stick out like a sore thumb and not distract people are simple interested in reading though the examples.

![Screen Shot 2022-10-15 at 11 24 46 PM2](https://user-images.githubusercontent.com/17162724/196016750-134556fb-5839-42ff-8fd2-04e80df58752.png)

COMMENT:
I can not see the note floating on the right with a wide screen, looks like `margin-top` is too small. 

Also looks like one reason for this is that .sphx-glr-download-link-note CSS is overriden by a higher priority rule.

Labels: Documentation, Needs Decision
suggestion to move (copy?) binder button for example to top or right side bar

### Describe the issue linked to the documentation

Firstly, thanks for having binder links for examples!

In the examples such as https://scikit-learn.org/stable/auto_examples/release_highlights/plot_release_highlights_1_1_0.html#sphx-glr-auto-examples-release-highlights-plot-release-highlights-1-1-0-py the binder button is as the bottom of the page

<img width="1469" alt="Screen Shot 2022-10-15 at 11 16 30 PM" src="https://user-images.githubusercontent.com/17162724/196016269-ddf3ddd4-7dbe-4825-ac60-f9c538a6c410.png">

It can sometimes take a while to scroll to get to the button. It would great if it could be accessed quicker with the binder link added to the top or to the right side bar.

### Suggest a potential alternative/fix

I would find it beneficial if the button was at the top (as well as the bottom?) or on the right side bar. This would make it quicker for me to run an example. If the example is long then it can take a bit of scrolling to get to the binder button.

I've mocked up some proposals below. I'm think the button under the title ("A demo on Spectal Biclustering algorithm") would look like a binder button badge as the top of a README e.g. the shield.io format https://mybinder.readthedocs.io/en/latest/howto/badges.html

It's also worth noting that the box in the right isn't accessible and I was able to scroll up to access it

I understand I don't want the badge/link to stick out like a sore thumb and not distract people are simple interested in reading though the examples.

![Screen Shot 2022-10-15 at 11 24 46 PM2](https://user-images.githubusercontent.com/17162724/196016750-134556fb-5839-42ff-8fd2-04e80df58752.png)

COMMENT:
The text of the bar made me think "if I click a download will start, no that isn't what I want". What do people think about changing it to something like "[Go to the end]() to download the full example code or to run ..."? It would fix two pet peeves: "here" as clickable link instead of describing what will happen and that it makes me think it will download a file. If people don't hate it, I'll open a PR.

Labels: Documentation, Needs Decision
suggestion to move (copy?) binder button for example to top or right side bar

### Describe the issue linked to the documentation

Firstly, thanks for having binder links for examples!

In the examples such as https://scikit-learn.org/stable/auto_examples/release_highlights/plot_release_highlights_1_1_0.html#sphx-glr-auto-examples-release-highlights-plot-release-highlights-1-1-0-py the binder button is as the bottom of the page

<img width="1469" alt="Screen Shot 2022-10-15 at 11 16 30 PM" src="https://user-images.githubusercontent.com/17162724/196016269-ddf3ddd4-7dbe-4825-ac60-f9c538a6c410.png">

It can sometimes take a while to scroll to get to the button. It would great if it could be accessed quicker with the binder link added to the top or to the right side bar.

### Suggest a potential alternative/fix

I would find it beneficial if the button was at the top (as well as the bottom?) or on the right side bar. This would make it quicker for me to run an example. If the example is long then it can take a bit of scrolling to get to the binder button.

I've mocked up some proposals below. I'm think the button under the title ("A demo on Spectal Biclustering algorithm") would look like a binder button badge as the top of a README e.g. the shield.io format https://mybinder.readthedocs.io/en/latest/howto/badges.html

It's also worth noting that the box in the right isn't accessible and I was able to scroll up to access it

I understand I don't want the badge/link to stick out like a sore thumb and not distract people are simple interested in reading though the examples.

![Screen Shot 2022-10-15 at 11 24 46 PM2](https://user-images.githubusercontent.com/17162724/196016750-134556fb-5839-42ff-8fd2-04e80df58752.png)

COMMENT:
I am fine with the change but very likely changing the text of the note needs to be done in sphinx-gallery.

Labels: Documentation, Needs Decision
suggestion to move (copy?) binder button for example to top or right side bar

### Describe the issue linked to the documentation

Firstly, thanks for having binder links for examples!

In the examples such as https://scikit-learn.org/stable/auto_examples/release_highlights/plot_release_highlights_1_1_0.html#sphx-glr-auto-examples-release-highlights-plot-release-highlights-1-1-0-py the binder button is as the bottom of the page

<img width="1469" alt="Screen Shot 2022-10-15 at 11 16 30 PM" src="https://user-images.githubusercontent.com/17162724/196016269-ddf3ddd4-7dbe-4825-ac60-f9c538a6c410.png">

It can sometimes take a while to scroll to get to the button. It would great if it could be accessed quicker with the binder link added to the top or to the right side bar.

### Suggest a potential alternative/fix

I would find it beneficial if the button was at the top (as well as the bottom?) or on the right side bar. This would make it quicker for me to run an example. If the example is long then it can take a bit of scrolling to get to the binder button.

I've mocked up some proposals below. I'm think the button under the title ("A demo on Spectal Biclustering algorithm") would look like a binder button badge as the top of a README e.g. the shield.io format https://mybinder.readthedocs.io/en/latest/howto/badges.html

It's also worth noting that the box in the right isn't accessible and I was able to scroll up to access it

I understand I don't want the badge/link to stick out like a sore thumb and not distract people are simple interested in reading though the examples.

![Screen Shot 2022-10-15 at 11 24 46 PM2](https://user-images.githubusercontent.com/17162724/196016750-134556fb-5839-42ff-8fd2-04e80df58752.png)

COMMENT:
I'm closing this issue because it should be handled in sphinx-gallery. With the migration to `pydata-sphinx-theme`, we are likely to have a better integration.

The related issue upstream is the following: https://github.com/sphinx-gallery/sphinx-gallery/issues/1258

Labels: Documentation, Needs Decision
Can I show the chart in the browser ?

### Describe the workflow you want to enable

<img width="350" alt="image" src="https://user-images.githubusercontent.com/14190605/195821119-5117dd2e-cc12-4024-8f9b-e87f49cd34f8.png">

Hi! Can I show the chart in the browser ? Thanks

### Describe your proposed solution

LOL

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

``

COMMENT:
Closing this one since this does not seem to be about scikit-learn.

Labels: New Feature, Needs Triage
Update min joblib to 1.2.0

### Describe the workflow you want to enable

As of 3 days ago, Joblib 1.2.0 has [significant improvements](https://github.com/joblib/joblib/blob/master/CHANGES.rst) over the Joblib 1.1.1 that sklearn currently supports

I suggest that sklearn should increase joblib version in required deps

### Describe your proposed solution

Modify `JOBLIB_MIN_VERSION` in `sklearn/_min_dependencies` to  `JOBLIB_MIN_VERSION = "1.2.0"`


https://github.com/scikit-learn/scikit-learn/blob/5b45d1fa7faad53db1a0fac9b15d23cb22375031/sklearn/_min_dependencies.py#L18



### Describe alternatives you've considered, if relevant

N/A.  No backwards compatibility issue in this upgrade.

### Additional context

I'm happy to make the PR with change.

COMMENT:
As noted in https://github.com/scikit-learn/scikit-learn/issues/24401#issuecomment-1263733450, we can not update to 1.2 right away. We tend to be conservative with bumping minimum versions, so it is easier for scikit-learn to be compatible with other libraries (that can have their own constraints).

That being said, scikit-learn is tested and works with joblib 1.2. A user can update to joblib 1.2 and get the benefits from that version.

Labels: New Feature, Needs Triage
Default value for solving an SVM in primal and dual should be determined automatically

### Describe the workflow you want to enable

Currently the default value for LinearSVC and LinearSVR with regards to solving the problem in dual is True. This means that for users that are unfamiliar with support-vector machines and do not know the importance of this parameter they could be using te framework inefficiently. 

I would propose for the default value to be None and a simple check to occur. If n_samples > n_features it's set to dual and otherwise it is set to primal. 

### Describe your proposed solution

This is a quick and easy fix and would only require a few lines of code in the fit method:

```
        if self.dual != None:
            pass
        else:
            if X.shape[0] < X.shape[1]:
                self.dual = True
            else:
                self.dual = False
```


### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
@agramfort I don't know enough about the potential side-effect of solving the SVM problem in the primal. Would having an `"auto"` option indeed be better to switch automatically from dual to primal depending on the shape of `X`?

Labels: Enhancement, module:svm
Default value for solving an SVM in primal and dual should be determined automatically

### Describe the workflow you want to enable

Currently the default value for LinearSVC and LinearSVR with regards to solving the problem in dual is True. This means that for users that are unfamiliar with support-vector machines and do not know the importance of this parameter they could be using te framework inefficiently. 

I would propose for the default value to be None and a simple check to occur. If n_samples > n_features it's set to dual and otherwise it is set to primal. 

### Describe your proposed solution

This is a quick and easy fix and would only require a few lines of code in the fit method:

```
        if self.dual != None:
            pass
        else:
            if X.shape[0] < X.shape[1]:
                self.dual = True
            else:
                self.dual = False
```


### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
indeed. Dual should be faster when n_samples < n_features. This would deserve a tiny benchmark though.

Labels: Enhancement, module:svm