Talip7/scikit-learn-issues-multilabel huggingface dataset:
Enable exporting trained models to text files to be able to import later

### Describe the workflow you want to enable

```python
from sklearn.datasets import fetch_california_housing
from sklearn.ensemble import RandomForestRegressor

# make data
X,y = fetch_california_housing(return_X_y=True)

# instantiate Random-Forest and fit it
rf_model = RandomForestRegressor(min_samples_leaf=5, random_state=0, n_jobs=-1)
rf_model.fit(X, y)

# export model to a text file, inspired by https://xgboost.readthedocs.io/en/release_3.0.0/python/python_api.html#xgboost.XGBRegressor.save_model
rf_model.save_model("model.json")


#################### in a new python environment
from sklearn.ensemble import RandomForestRegressor
rf_model2 = RandomForestRegressor(min_samples_leaf=5, random_state=0, n_jobs=-1)
rf_model2.load_model("model.json")
```

### Describe your proposed solution

The current recommended way I believe is to export fit (or trained) models is to serialize them using joblib, which depends on python version, joblib version and scikit-learn version too, and I presume this may lead to issues with OS and CPU architecture as well (windows or  GNU Linux and x86 or ARM64).

So, I request a way to export model's trained weights (or other relevant things like bins & trees for RandomForestRegressor) for it be to loaded from any scikit-learn version or python version or operating system. This is just how the big packages like [xgboost](https://xgboost.readthedocs.io/en/release_3.0.0/python/python_api.html#xgboost.XGBRegressor.save_model) and pytorch ([using `state_dict`](https://pytorch.org/docs/stable/generated/torch.nn.Module.html#torch.nn.Module.load_state_dict)) and hence transformers,  handle things. 

This would enable to change environments and platforms easily without having to train model for the new package-versions, architecture and OS again or every time an update in them is required.

### Describe alternatives you've considered, if relevant

There is no alternative to everything that scikit-learn offers as of now.

### Additional context

This has been provied above.

COMMENT:
I think there's nothing to be done here. Closing, but very happy to see if anybody wants to sponsor the efforts.

Labels: New Feature, Needs Decision - Include Feature
Enable exporting trained models to text files to be able to import later

### Describe the workflow you want to enable

```python
from sklearn.datasets import fetch_california_housing
from sklearn.ensemble import RandomForestRegressor

# make data
X,y = fetch_california_housing(return_X_y=True)

# instantiate Random-Forest and fit it
rf_model = RandomForestRegressor(min_samples_leaf=5, random_state=0, n_jobs=-1)
rf_model.fit(X, y)

# export model to a text file, inspired by https://xgboost.readthedocs.io/en/release_3.0.0/python/python_api.html#xgboost.XGBRegressor.save_model
rf_model.save_model("model.json")


#################### in a new python environment
from sklearn.ensemble import RandomForestRegressor
rf_model2 = RandomForestRegressor(min_samples_leaf=5, random_state=0, n_jobs=-1)
rf_model2.load_model("model.json")
```

### Describe your proposed solution

The current recommended way I believe is to export fit (or trained) models is to serialize them using joblib, which depends on python version, joblib version and scikit-learn version too, and I presume this may lead to issues with OS and CPU architecture as well (windows or  GNU Linux and x86 or ARM64).

So, I request a way to export model's trained weights (or other relevant things like bins & trees for RandomForestRegressor) for it be to loaded from any scikit-learn version or python version or operating system. This is just how the big packages like [xgboost](https://xgboost.readthedocs.io/en/release_3.0.0/python/python_api.html#xgboost.XGBRegressor.save_model) and pytorch ([using `state_dict`](https://pytorch.org/docs/stable/generated/torch.nn.Module.html#torch.nn.Module.load_state_dict)) and hence transformers,  handle things. 

This would enable to change environments and platforms easily without having to train model for the new package-versions, architecture and OS again or every time an update in them is required.

### Describe alternatives you've considered, if relevant

There is no alternative to everything that scikit-learn offers as of now.

### Additional context

This has been provied above.

COMMENT:
@ogrisel I think that the core of the problem is that you cannot persist a model in say v1.4 of scikit-learn and then load the model into a scikit-learn v1.6 environment without scary warnings. This seems unreasonable for such a widely used project.

For most models it should be reasonably straight forward to create robust persistence functions for getting the model parameters (or state) as a python dict and for restoring the model from a python dict. The user can then choose whatever binary or text format they prefer for saving the model parameters.

This approach would make it easier to use models across minor version upgrades of scikit-learn, but still allowing for breaking changes at major versions.

I am not saying that you or @adrinjalali should implement this, I believe that the open source community would step up and do the implementation given a plan/approach for what needs to be done.

Labels: New Feature, Needs Decision - Include Feature
Enable exporting trained models to text files to be able to import later

### Describe the workflow you want to enable

```python
from sklearn.datasets import fetch_california_housing
from sklearn.ensemble import RandomForestRegressor

# make data
X,y = fetch_california_housing(return_X_y=True)

# instantiate Random-Forest and fit it
rf_model = RandomForestRegressor(min_samples_leaf=5, random_state=0, n_jobs=-1)
rf_model.fit(X, y)

# export model to a text file, inspired by https://xgboost.readthedocs.io/en/release_3.0.0/python/python_api.html#xgboost.XGBRegressor.save_model
rf_model.save_model("model.json")


#################### in a new python environment
from sklearn.ensemble import RandomForestRegressor
rf_model2 = RandomForestRegressor(min_samples_leaf=5, random_state=0, n_jobs=-1)
rf_model2.load_model("model.json")
```

### Describe your proposed solution

The current recommended way I believe is to export fit (or trained) models is to serialize them using joblib, which depends on python version, joblib version and scikit-learn version too, and I presume this may lead to issues with OS and CPU architecture as well (windows or  GNU Linux and x86 or ARM64).

So, I request a way to export model's trained weights (or other relevant things like bins & trees for RandomForestRegressor) for it be to loaded from any scikit-learn version or python version or operating system. This is just how the big packages like [xgboost](https://xgboost.readthedocs.io/en/release_3.0.0/python/python_api.html#xgboost.XGBRegressor.save_model) and pytorch ([using `state_dict`](https://pytorch.org/docs/stable/generated/torch.nn.Module.html#torch.nn.Module.load_state_dict)) and hence transformers,  handle things. 

This would enable to change environments and platforms easily without having to train model for the new package-versions, architecture and OS again or every time an update in them is required.

### Describe alternatives you've considered, if relevant

There is no alternative to everything that scikit-learn offers as of now.

### Additional context

This has been provied above.

COMMENT:
Thanks everyone for this discussion. At probabl (probabl.ai) we are currently exploring a dedicated initiative to make persisted scikit-learn models forward-compatible across versions. 

Our idea is to keep the infrastructure and migration logic open, while offering enterprise-grade continuity guarantees.

If this resonates, let’s discuss @stdkoehler, @naveenmarthala, @jcbsv ! I would like to know better about your needs, working env...I am available at francois@probabl.ai

Labels: New Feature, Needs Decision - Include Feature
Add sankey style confusion matrix visualization

### Describe the workflow you want to enable

Confusion matrices can be displayed as a colored matrix using the [ConfusionMatrixDisplay](https://scikit-learn.org/stable/modules/generated/sklearn.metrics.ConfusionMatrixDisplay.html#sklearn.metrics.ConfusionMatrixDisplay) class.

![confusion_matrix](https://scikit-learn.org/stable/_images/sphx_glr_plot_label_propagation_digits_001.png)

However color scaling is hard to interpret, and spatial cues are easier to help interpret quantities than color variations. 
The number represented in each box represent absolute ones, while one may be interested in either row based, column based, or complete matrix sum based normalization. 
Plus it gets even harder to read for multiclass classification.  

### Describe your proposed solution

I would propose introducing sankey like plots to visualize the confusion matrix data, using a `ConfusionMatrixSankeyDisplay` class.
The qualitative information is displayed by different colors (easy to interpret), while actual amounts in each cell are represented by the size of the flows (easier to interpret quantitatively than colorscale variations).
On the left size one can see the number of occurrence of each label in the ground truth data (confusion matrix row marginals). On the right side the number of occurrence of each label in the predictions (confusion matrix column marginals). Each flow represents both row-normalized (left side) and column-normalized (right side) at the same time, and could be labeled with the actual absolute number of examples in each confusion matrix cell.
Interpretation is straightforward even in the multiclass case.

There exist a matplotlib based implementation doing almost exactly this in the small [pySankeyBeta](https://github.com/Pierre-Sassoulas/pySankey) package. Here is a sample from its readme:
![sankey](https://github.com/Pierre-Sassoulas/pySankey/raw/main/.github/img/fruits.png)

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
Thanks for opening this feature request. Here is some personal feedback:

- I agree that a Sankey diagram is helps to visualize confusion between observed and predicted class membership, possibly easier to read than a confusion matrix, especially if classes are not balanced.
- [pySankeyBeta](https://github.com/Pierre-Sassoulas/pySankey) is GPLv3 which is incompatible with the scikit-learn license. So if we decide to include such a feature in scikit-learn with cannot directly reuse this code without an explicit authorization of the copyright holder of `pySankeyBeta`.

Let's see what other maintainers think about this proposal.

Labels: New Feature, Needs Decision - Include Feature
Add sankey style confusion matrix visualization

### Describe the workflow you want to enable

Confusion matrices can be displayed as a colored matrix using the [ConfusionMatrixDisplay](https://scikit-learn.org/stable/modules/generated/sklearn.metrics.ConfusionMatrixDisplay.html#sklearn.metrics.ConfusionMatrixDisplay) class.

![confusion_matrix](https://scikit-learn.org/stable/_images/sphx_glr_plot_label_propagation_digits_001.png)

However color scaling is hard to interpret, and spatial cues are easier to help interpret quantities than color variations. 
The number represented in each box represent absolute ones, while one may be interested in either row based, column based, or complete matrix sum based normalization. 
Plus it gets even harder to read for multiclass classification.  

### Describe your proposed solution

I would propose introducing sankey like plots to visualize the confusion matrix data, using a `ConfusionMatrixSankeyDisplay` class.
The qualitative information is displayed by different colors (easy to interpret), while actual amounts in each cell are represented by the size of the flows (easier to interpret quantitatively than colorscale variations).
On the left size one can see the number of occurrence of each label in the ground truth data (confusion matrix row marginals). On the right side the number of occurrence of each label in the predictions (confusion matrix column marginals). Each flow represents both row-normalized (left side) and column-normalized (right side) at the same time, and could be labeled with the actual absolute number of examples in each confusion matrix cell.
Interpretation is straightforward even in the multiclass case.

There exist a matplotlib based implementation doing almost exactly this in the small [pySankeyBeta](https://github.com/Pierre-Sassoulas/pySankey) package. Here is a sample from its readme:
![sankey](https://github.com/Pierre-Sassoulas/pySankey/raw/main/.github/img/fruits.png)

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
Indeed I did not think about the license issue, good point! This might indeed be an issue since pySankeyBeta is a fork of a unmaintained package, and the original author's last activity on GH was in 2020...
There might be a way to rewrite something simpler using [matplotlib's Sankey class](https://matplotlib.org/stable/api/sankey_api.html), but I find the documentation a bit obscure.

Labels: New Feature, Needs Decision - Include Feature
Add sankey style confusion matrix visualization

### Describe the workflow you want to enable

Confusion matrices can be displayed as a colored matrix using the [ConfusionMatrixDisplay](https://scikit-learn.org/stable/modules/generated/sklearn.metrics.ConfusionMatrixDisplay.html#sklearn.metrics.ConfusionMatrixDisplay) class.

![confusion_matrix](https://scikit-learn.org/stable/_images/sphx_glr_plot_label_propagation_digits_001.png)

However color scaling is hard to interpret, and spatial cues are easier to help interpret quantities than color variations. 
The number represented in each box represent absolute ones, while one may be interested in either row based, column based, or complete matrix sum based normalization. 
Plus it gets even harder to read for multiclass classification.  

### Describe your proposed solution

I would propose introducing sankey like plots to visualize the confusion matrix data, using a `ConfusionMatrixSankeyDisplay` class.
The qualitative information is displayed by different colors (easy to interpret), while actual amounts in each cell are represented by the size of the flows (easier to interpret quantitatively than colorscale variations).
On the left size one can see the number of occurrence of each label in the ground truth data (confusion matrix row marginals). On the right side the number of occurrence of each label in the predictions (confusion matrix column marginals). Each flow represents both row-normalized (left side) and column-normalized (right side) at the same time, and could be labeled with the actual absolute number of examples in each confusion matrix cell.
Interpretation is straightforward even in the multiclass case.

There exist a matplotlib based implementation doing almost exactly this in the small [pySankeyBeta](https://github.com/Pierre-Sassoulas/pySankey) package. Here is a sample from its readme:
![sankey](https://github.com/Pierre-Sassoulas/pySankey/raw/main/.github/img/fruits.png)

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
Hello! I'd like to work on this. I'll propose a BSD-compatible implementation approach for ConfusionMatrixSankeyDisplay.

Labels: New Feature, Needs Decision - Include Feature
Add sankey style confusion matrix visualization

### Describe the workflow you want to enable

Confusion matrices can be displayed as a colored matrix using the [ConfusionMatrixDisplay](https://scikit-learn.org/stable/modules/generated/sklearn.metrics.ConfusionMatrixDisplay.html#sklearn.metrics.ConfusionMatrixDisplay) class.

![confusion_matrix](https://scikit-learn.org/stable/_images/sphx_glr_plot_label_propagation_digits_001.png)

However color scaling is hard to interpret, and spatial cues are easier to help interpret quantities than color variations. 
The number represented in each box represent absolute ones, while one may be interested in either row based, column based, or complete matrix sum based normalization. 
Plus it gets even harder to read for multiclass classification.  

### Describe your proposed solution

I would propose introducing sankey like plots to visualize the confusion matrix data, using a `ConfusionMatrixSankeyDisplay` class.
The qualitative information is displayed by different colors (easy to interpret), while actual amounts in each cell are represented by the size of the flows (easier to interpret quantitatively than colorscale variations).
On the left size one can see the number of occurrence of each label in the ground truth data (confusion matrix row marginals). On the right side the number of occurrence of each label in the predictions (confusion matrix column marginals). Each flow represents both row-normalized (left side) and column-normalized (right side) at the same time, and could be labeled with the actual absolute number of examples in each confusion matrix cell.
Interpretation is straightforward even in the multiclass case.

There exist a matplotlib based implementation doing almost exactly this in the small [pySankeyBeta](https://github.com/Pierre-Sassoulas/pySankey) package. Here is a sample from its readme:
![sankey](https://github.com/Pierre-Sassoulas/pySankey/raw/main/.github/img/fruits.png)

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
@wndu123 it may be worth waiting for other maintainers to chime in about what they think of adding this feature before you open a PR (#31255).

Note you should reference the relevant issue when opening a PR and give the PR a informative name.
You can find our guide to contributing here: https://scikit-learn.org/dev/developers/contributing.html

Labels: New Feature, Needs Decision - Include Feature
Add sankey style confusion matrix visualization

### Describe the workflow you want to enable

Confusion matrices can be displayed as a colored matrix using the [ConfusionMatrixDisplay](https://scikit-learn.org/stable/modules/generated/sklearn.metrics.ConfusionMatrixDisplay.html#sklearn.metrics.ConfusionMatrixDisplay) class.

![confusion_matrix](https://scikit-learn.org/stable/_images/sphx_glr_plot_label_propagation_digits_001.png)

However color scaling is hard to interpret, and spatial cues are easier to help interpret quantities than color variations. 
The number represented in each box represent absolute ones, while one may be interested in either row based, column based, or complete matrix sum based normalization. 
Plus it gets even harder to read for multiclass classification.  

### Describe your proposed solution

I would propose introducing sankey like plots to visualize the confusion matrix data, using a `ConfusionMatrixSankeyDisplay` class.
The qualitative information is displayed by different colors (easy to interpret), while actual amounts in each cell are represented by the size of the flows (easier to interpret quantitatively than colorscale variations).
On the left size one can see the number of occurrence of each label in the ground truth data (confusion matrix row marginals). On the right side the number of occurrence of each label in the predictions (confusion matrix column marginals). Each flow represents both row-normalized (left side) and column-normalized (right side) at the same time, and could be labeled with the actual absolute number of examples in each confusion matrix cell.
Interpretation is straightforward even in the multiclass case.

There exist a matplotlib based implementation doing almost exactly this in the small [pySankeyBeta](https://github.com/Pierre-Sassoulas/pySankey) package. Here is a sample from its readme:
![sankey](https://github.com/Pierre-Sassoulas/pySankey/raw/main/.github/img/fruits.png)

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
While I really like good visualisations, I am -1 on this one.
With the plot library of your choice, it should be straightforward to produce a Sankey diagram with the result of `confusion_matrix(..)`.

Labels: New Feature, Needs Decision - Include Feature
Add sankey style confusion matrix visualization

### Describe the workflow you want to enable

Confusion matrices can be displayed as a colored matrix using the [ConfusionMatrixDisplay](https://scikit-learn.org/stable/modules/generated/sklearn.metrics.ConfusionMatrixDisplay.html#sklearn.metrics.ConfusionMatrixDisplay) class.

![confusion_matrix](https://scikit-learn.org/stable/_images/sphx_glr_plot_label_propagation_digits_001.png)

However color scaling is hard to interpret, and spatial cues are easier to help interpret quantities than color variations. 
The number represented in each box represent absolute ones, while one may be interested in either row based, column based, or complete matrix sum based normalization. 
Plus it gets even harder to read for multiclass classification.  

### Describe your proposed solution

I would propose introducing sankey like plots to visualize the confusion matrix data, using a `ConfusionMatrixSankeyDisplay` class.
The qualitative information is displayed by different colors (easy to interpret), while actual amounts in each cell are represented by the size of the flows (easier to interpret quantitatively than colorscale variations).
On the left size one can see the number of occurrence of each label in the ground truth data (confusion matrix row marginals). On the right side the number of occurrence of each label in the predictions (confusion matrix column marginals). Each flow represents both row-normalized (left side) and column-normalized (right side) at the same time, and could be labeled with the actual absolute number of examples in each confusion matrix cell.
Interpretation is straightforward even in the multiclass case.

There exist a matplotlib based implementation doing almost exactly this in the small [pySankeyBeta](https://github.com/Pierre-Sassoulas/pySankey) package. Here is a sample from its readme:
![sankey](https://github.com/Pierre-Sassoulas/pySankey/raw/main/.github/img/fruits.png)

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
I think it would be nice to have this as part of the visualisations in scikit-learn. My main thinking for why is that I've looked at many confusion matrix displays but never thought that a different way of displaying them would be nice. And once I saw the sankey version I thought "wow, why isn't this the default?? And how did I go so long without learning that it exists?"

Labels: New Feature, Needs Decision - Include Feature