Talip7/scikit-learn-issues-multilabel huggingface dataset:
Extra plots in partial dependence plots

### Describe the workflow you want to enable

As discussed in #19410, there has been interest in including additional visualizations along with the partial dependence visualizations. Extra plots would aid in the interpretation of partial dependence plots. It would be low overhead for the user to specify "hist" as an argument and have the feature distribution plotted in the same figure as the partial dependence plot.  This issue only addresses the suggestion to improve partial dependence plots, not ICE plots.

### Describe your proposed solution

#27388 introduces three new parameters to the `from_estimator` method in the `PartialDependenceDisplay` class:

* `extra_plots`: A string or list of strings specifying what type of extra plot to include in the partial dependence display.
* `extra_plots_kw`: A dictionary where the keys should match the plot type specified in `extra_plots` and the values are kwarg dictionaries for each plot type.
* `y`: Only used in one-way partial dependence plots when the extra plot is a scatter plot. 

Their defaults are all `None`, which does not affect the current PDP display behavior.
The following image can be generated by simply adding: `extra_plots=["boxplot", "hist"]` to the `from_estimator` call.

![image](https://github.com/scikit-learn/scikit-learn/assets/9151717/03483a24-a238-4594-be7f-49ae5437eac3)


### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
I like the histogram option but I am not convinced that that the boxplot is that helpful: in particular, it wouldn't summarize well multimodal marginal distributions. Maybe a half-violinplot would be a better alternative (which is basically a a KDE-smoothed histogram).

Labels: New Feature
Extra plots in partial dependence plots

### Describe the workflow you want to enable

As discussed in #19410, there has been interest in including additional visualizations along with the partial dependence visualizations. Extra plots would aid in the interpretation of partial dependence plots. It would be low overhead for the user to specify "hist" as an argument and have the feature distribution plotted in the same figure as the partial dependence plot.  This issue only addresses the suggestion to improve partial dependence plots, not ICE plots.

### Describe your proposed solution

#27388 introduces three new parameters to the `from_estimator` method in the `PartialDependenceDisplay` class:

* `extra_plots`: A string or list of strings specifying what type of extra plot to include in the partial dependence display.
* `extra_plots_kw`: A dictionary where the keys should match the plot type specified in `extra_plots` and the values are kwarg dictionaries for each plot type.
* `y`: Only used in one-way partial dependence plots when the extra plot is a scatter plot. 

Their defaults are all `None`, which does not affect the current PDP display behavior.
The following image can be generated by simply adding: `extra_plots=["boxplot", "hist"]` to the `from_estimator` call.

![image](https://github.com/scikit-learn/scikit-learn/assets/9151717/03483a24-a238-4594-be7f-49ae5437eac3)


### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
For the latter suggestion we should first fix #26658 but I am myself running out of bandwidth :)

Labels: New Feature
Extra plots in partial dependence plots

### Describe the workflow you want to enable

As discussed in #19410, there has been interest in including additional visualizations along with the partial dependence visualizations. Extra plots would aid in the interpretation of partial dependence plots. It would be low overhead for the user to specify "hist" as an argument and have the feature distribution plotted in the same figure as the partial dependence plot.  This issue only addresses the suggestion to improve partial dependence plots, not ICE plots.

### Describe your proposed solution

#27388 introduces three new parameters to the `from_estimator` method in the `PartialDependenceDisplay` class:

* `extra_plots`: A string or list of strings specifying what type of extra plot to include in the partial dependence display.
* `extra_plots_kw`: A dictionary where the keys should match the plot type specified in `extra_plots` and the values are kwarg dictionaries for each plot type.
* `y`: Only used in one-way partial dependence plots when the extra plot is a scatter plot. 

Their defaults are all `None`, which does not affect the current PDP display behavior.
The following image can be generated by simply adding: `extra_plots=["boxplot", "hist"]` to the `from_estimator` call.

![image](https://github.com/scikit-learn/scikit-learn/assets/9151717/03483a24-a238-4594-be7f-49ae5437eac3)


### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
I also prefer the histogram. I'm happy to update the PR to limit the scope to histogram-only.

Labels: New Feature
Add new estimator checks for sparse arrays to ensure compatibility for third-party libraries

In #27090, we make changes in our tests to check that our estimators are compatible with sparse arrays. However, it does not intend to write common tests through new checks available in `estimator_checks.py`.

We should implement new checks that are testing the same as the sparse matrix tests to ensure that third-party library will be compatible with such arrays. However, we want to be extra-nice: we should also create a new tag, e.g. `X_types: {..., "sparse_arrays", "sparse matrix"}` and run the tests only when the types is explicitly supported.

The documentation states that we are not currently using the `"sparse"` key, so we should be able to introduce new keys without any regression.

COMMENT:
I'd like to ask a question. What is the reason you plan to duplicate the checks concerned with sparse data generation instead of testing both types in the same check? Or rather, why would a depending library that develops a scikit-learn compatible estimator want to filter the checks, so that only part of the compatibility is tested?

Labels: module:test-suite
Add new estimator checks for sparse arrays to ensure compatibility for third-party libraries

In #27090, we make changes in our tests to check that our estimators are compatible with sparse arrays. However, it does not intend to write common tests through new checks available in `estimator_checks.py`.

We should implement new checks that are testing the same as the sparse matrix tests to ensure that third-party library will be compatible with such arrays. However, we want to be extra-nice: we should also create a new tag, e.g. `X_types: {..., "sparse_arrays", "sparse matrix"}` and run the tests only when the types is explicitly supported.

The documentation states that we are not currently using the `"sparse"` key, so we should be able to introduce new keys without any regression.

COMMENT:
At the long term, I could foresee that we drop support for "sparse matrix" because SciPy will drop the support. So it will be easy to maintain by just removing the test corresponding instead of modifying within the test some behaviour.

> so that only part of the compatibility is tested?

It is fine that a new library decide to only support sparse arrays if they assume that sparse matrices will be obsolete. So I think this is OK to allow them to adapt the check here.

At the end it all boils down to the fact that sparse matrix will be discontinued and sparse array will be the container that has to be supported.

Labels: module:test-suite
Add new estimator checks for sparse arrays to ensure compatibility for third-party libraries

In #27090, we make changes in our tests to check that our estimators are compatible with sparse arrays. However, it does not intend to write common tests through new checks available in `estimator_checks.py`.

We should implement new checks that are testing the same as the sparse matrix tests to ensure that third-party library will be compatible with such arrays. However, we want to be extra-nice: we should also create a new tag, e.g. `X_types: {..., "sparse_arrays", "sparse matrix"}` and run the tests only when the types is explicitly supported.

The documentation states that we are not currently using the `"sparse"` key, so we should be able to introduce new keys without any regression.

COMMENT:
I see, thank you @glemaitre. :)

If this is something you think I could do, I'd like to try.

Labels: module:test-suite
Add new estimator checks for sparse arrays to ensure compatibility for third-party libraries

In #27090, we make changes in our tests to check that our estimators are compatible with sparse arrays. However, it does not intend to write common tests through new checks available in `estimator_checks.py`.

We should implement new checks that are testing the same as the sparse matrix tests to ensure that third-party library will be compatible with such arrays. However, we want to be extra-nice: we should also create a new tag, e.g. `X_types: {..., "sparse_arrays", "sparse matrix"}` and run the tests only when the types is explicitly supported.

The documentation states that we are not currently using the `"sparse"` key, so we should be able to introduce new keys without any regression.

COMMENT:
Yes it will allow you to explore the `estimator_checks` and the common tests. They are far to be straightforward but this is an interesting piece of code to know about :).

Labels: module:test-suite
Add new estimator checks for sparse arrays to ensure compatibility for third-party libraries

In #27090, we make changes in our tests to check that our estimators are compatible with sparse arrays. However, it does not intend to write common tests through new checks available in `estimator_checks.py`.

We should implement new checks that are testing the same as the sparse matrix tests to ensure that third-party library will be compatible with such arrays. However, we want to be extra-nice: we should also create a new tag, e.g. `X_types: {..., "sparse_arrays", "sparse matrix"}` and run the tests only when the types is explicitly supported.

The documentation states that we are not currently using the `"sparse"` key, so we should be able to introduce new keys without any regression.

COMMENT:
Nice, thank you. It will take me a few days before I can start, but I'm looking forward. :)

Labels: module:test-suite
Add new estimator checks for sparse arrays to ensure compatibility for third-party libraries

In #27090, we make changes in our tests to check that our estimators are compatible with sparse arrays. However, it does not intend to write common tests through new checks available in `estimator_checks.py`.

We should implement new checks that are testing the same as the sparse matrix tests to ensure that third-party library will be compatible with such arrays. However, we want to be extra-nice: we should also create a new tag, e.g. `X_types: {..., "sparse_arrays", "sparse matrix"}` and run the tests only when the types is explicitly supported.

The documentation states that we are not currently using the `"sparse"` key, so we should be able to introduce new keys without any regression.

COMMENT:
I'm closing this issue because the PR of @StefanieSenger already implement the tests. We could later improve with a better handling of the estimator tags but this is another story.

Labels: module:test-suite
Model Persistence doc page could provide clearer actionable recommendations

### Describe the issue linked to the documentation

The [Model Persistence page](https://github.com/scikit-learn/scikit-learn/blob/main/doc/model_persistence.rst) currently discusses many options (pickling, `skops`, ONNX and PMML, but it does it sequentially (first discusses one, then the other, and so on) without a clear narration. I think most people would not know what they should do after reading that page.

I understand these pages are not supposed to be tutorials or blogposts, but only high-level directions on the path to take, but still I think we could have a better page.

Doing MLOps consulting work, I've found most Data Scientists have big misconceptions about what they should do in order to correctly and safely save and then load scikit-learn models. In particular, most Data Scientists I've met think pinning the scikit-learn version is enough to ensure compatibility. The documentation is clear in this regard, but maybe dances around too much between options to make the point something people will remember.

### Suggest a potential alternative/fix

I think there's a couple of changes that could be done in order to improve the narrative and make sure the basic points come across to everyone, while still discussing all available options as discussed now.

1. List all alternatives near the beginning of the document.
2. _Start_ by discussing the fact that you need to pin all transitive scikit-learn dependencies to be able to safely load a pickle, which is the format most people will use, and probably gets you 80% of the benefit you will get from more complicated recommendations.
2. Do more to compare the presented alternatives (what does skops lack that would make one want to use ONNX?)
3. Summarize the most important points at the end of the document.

I can provide a PR if you agree these changes are desirable.

COMMENT:
Those are all excellent suggestions in my opinion! Looking forward to reviewing the PR :)

Labels: Documentation
Model Persistence doc page could provide clearer actionable recommendations

### Describe the issue linked to the documentation

The [Model Persistence page](https://github.com/scikit-learn/scikit-learn/blob/main/doc/model_persistence.rst) currently discusses many options (pickling, `skops`, ONNX and PMML, but it does it sequentially (first discusses one, then the other, and so on) without a clear narration. I think most people would not know what they should do after reading that page.

I understand these pages are not supposed to be tutorials or blogposts, but only high-level directions on the path to take, but still I think we could have a better page.

Doing MLOps consulting work, I've found most Data Scientists have big misconceptions about what they should do in order to correctly and safely save and then load scikit-learn models. In particular, most Data Scientists I've met think pinning the scikit-learn version is enough to ensure compatibility. The documentation is clear in this regard, but maybe dances around too much between options to make the point something people will remember.

### Suggest a potential alternative/fix

I think there's a couple of changes that could be done in order to improve the narrative and make sure the basic points come across to everyone, while still discussing all available options as discussed now.

1. List all alternatives near the beginning of the document.
2. _Start_ by discussing the fact that you need to pin all transitive scikit-learn dependencies to be able to safely load a pickle, which is the format most people will use, and probably gets you 80% of the benefit you will get from more complicated recommendations.
2. Do more to compare the presented alternatives (what does skops lack that would make one want to use ONNX?)
3. Summarize the most important points at the end of the document.

I can provide a PR if you agree these changes are desirable.

COMMENT:
Hey @IanTayler, I am interested in working towards a contribution for this. May I take up this issue and submit a PR for it?

Labels: Documentation
Model Persistence doc page could provide clearer actionable recommendations

### Describe the issue linked to the documentation

The [Model Persistence page](https://github.com/scikit-learn/scikit-learn/blob/main/doc/model_persistence.rst) currently discusses many options (pickling, `skops`, ONNX and PMML, but it does it sequentially (first discusses one, then the other, and so on) without a clear narration. I think most people would not know what they should do after reading that page.

I understand these pages are not supposed to be tutorials or blogposts, but only high-level directions on the path to take, but still I think we could have a better page.

Doing MLOps consulting work, I've found most Data Scientists have big misconceptions about what they should do in order to correctly and safely save and then load scikit-learn models. In particular, most Data Scientists I've met think pinning the scikit-learn version is enough to ensure compatibility. The documentation is clear in this regard, but maybe dances around too much between options to make the point something people will remember.

### Suggest a potential alternative/fix

I think there's a couple of changes that could be done in order to improve the narrative and make sure the basic points come across to everyone, while still discussing all available options as discussed now.

1. List all alternatives near the beginning of the document.
2. _Start_ by discussing the fact that you need to pin all transitive scikit-learn dependencies to be able to safely load a pickle, which is the format most people will use, and probably gets you 80% of the benefit you will get from more complicated recommendations.
2. Do more to compare the presented alternatives (what does skops lack that would make one want to use ONNX?)
3. Summarize the most important points at the end of the document.

I can provide a PR if you agree these changes are desirable.

COMMENT:
@rprkh if you feel confident that you understand the pros and cons and tradeoffs between those different ways of persistence, then yes, otherwise I was planning to open a PR.

Labels: Documentation
Model Persistence doc page could provide clearer actionable recommendations

### Describe the issue linked to the documentation

The [Model Persistence page](https://github.com/scikit-learn/scikit-learn/blob/main/doc/model_persistence.rst) currently discusses many options (pickling, `skops`, ONNX and PMML, but it does it sequentially (first discusses one, then the other, and so on) without a clear narration. I think most people would not know what they should do after reading that page.

I understand these pages are not supposed to be tutorials or blogposts, but only high-level directions on the path to take, but still I think we could have a better page.

Doing MLOps consulting work, I've found most Data Scientists have big misconceptions about what they should do in order to correctly and safely save and then load scikit-learn models. In particular, most Data Scientists I've met think pinning the scikit-learn version is enough to ensure compatibility. The documentation is clear in this regard, but maybe dances around too much between options to make the point something people will remember.

### Suggest a potential alternative/fix

I think there's a couple of changes that could be done in order to improve the narrative and make sure the basic points come across to everyone, while still discussing all available options as discussed now.

1. List all alternatives near the beginning of the document.
2. _Start_ by discussing the fact that you need to pin all transitive scikit-learn dependencies to be able to safely load a pickle, which is the format most people will use, and probably gets you 80% of the benefit you will get from more complicated recommendations.
2. Do more to compare the presented alternatives (what does skops lack that would make one want to use ONNX?)
3. Summarize the most important points at the end of the document.

I can provide a PR if you agree these changes are desirable.

COMMENT:
I am familiar with the different methods for model persistance and some of the pros/cons. I'm trying to understand some of the nuances before I make the PR. I can submit the PR this weekend, if that's ok?

Labels: Documentation
Model Persistence doc page could provide clearer actionable recommendations

### Describe the issue linked to the documentation

The [Model Persistence page](https://github.com/scikit-learn/scikit-learn/blob/main/doc/model_persistence.rst) currently discusses many options (pickling, `skops`, ONNX and PMML, but it does it sequentially (first discusses one, then the other, and so on) without a clear narration. I think most people would not know what they should do after reading that page.

I understand these pages are not supposed to be tutorials or blogposts, but only high-level directions on the path to take, but still I think we could have a better page.

Doing MLOps consulting work, I've found most Data Scientists have big misconceptions about what they should do in order to correctly and safely save and then load scikit-learn models. In particular, most Data Scientists I've met think pinning the scikit-learn version is enough to ensure compatibility. The documentation is clear in this regard, but maybe dances around too much between options to make the point something people will remember.

### Suggest a potential alternative/fix

I think there's a couple of changes that could be done in order to improve the narrative and make sure the basic points come across to everyone, while still discussing all available options as discussed now.

1. List all alternatives near the beginning of the document.
2. _Start_ by discussing the fact that you need to pin all transitive scikit-learn dependencies to be able to safely load a pickle, which is the format most people will use, and probably gets you 80% of the benefit you will get from more complicated recommendations.
2. Do more to compare the presented alternatives (what does skops lack that would make one want to use ONNX?)
3. Summarize the most important points at the end of the document.

I can provide a PR if you agree these changes are desirable.

COMMENT:
@rprkh Of course! It'd be great to see these improvements done to the documentation. I could not quite find the time to do it myself. Would gladly cede the spot to let someone else do it.

Labels: Documentation
GrideSearchCV() has Issue with LSSVM () classification

### Describe the workflow you want to enable

Hi, 
I tried to use `GrideSearchCV()` with `LSSVM()` but could not do that , please could you help ?
 The code : 
The code which i used is from Github romolo code.
[https://github.com/RomuloDrumond/LSSVM](https://www.researchgate.net/deref/https%3A%2F%2Fgithub.com%2FRomuloDrumond%2FLSSVM?_tp=eyJjb250ZXh0Ijp7ImZpcnN0UGFnZSI6Il9kaXJlY3QiLCJwYWdlIjoiX2RpcmVjdCJ9fQ)

```python
lssvc = LSSVC(gamma=1, kernel='rbf', sigma=.5) # Class instantiation
lssvc.fit(X_tr_norm, y_train) # Fitting the model
y_pred = lssvc.predict(X_ts_norm) # Making predictions with the trained model
acc = accuracy_score(dummie2multilabel(y_test), dummie2multilabel(y_pred)) # Calculate Accuracy
print('acc_test = ', acc, '\n')
```

Then tried to do `GridesearchCV()` :

```python
parameters = {'kernel':('rbf'),
'gamma':[0.001, 0.01, 0.1, 1.0, 10.0, 100.0, 1000.0],
'sigma':[0.001, 0.01, 0.1, 1.0, 10.0, 100.0, 1000.0]}
lssvm = LSSVC()
LSSVC= GridSearchCV(lssvm, parameters)
lssvc.fit(X_tr_norm, y_train) # Fitting the model
y_pred = lssvc.predict(X_ts_norm) # Making predictions with the trained model
acc = accuracy_score(dummie2multilabel(y_test), dummie2multilabel(y_pred)) # Calculate Accuracy
print('acc_test = ', acc, '\n')
```

There are other implementaion for LSSVM but non of them worked with GrideSearchCV() since not in the original sklearn ?

### Describe your proposed solution

Adding LSSVM to GridesearchCV() 

### Describe alternatives you've considered, if relevant

No alternatives 

### Additional context

_No response_

COMMENT:
Can you provide text of an error?

Labels: New Feature, Needs Triage