Talip7/scikit-learn-issues-multilabel huggingface dataset:
Two different versions for weighted lorenz curve calculation in the examples

### Describe the issue linked to the documentation

There are 2 definitions of (weighted) `lorenz_curve()` functions [here](https://scikit-learn.org/stable/auto_examples/linear_model/plot_tweedie_regression_insurance_claims.html) and [here](https://scikit-learn.org/stable/auto_examples/linear_model/plot_poisson_regression_non_normal_loss.html)

The difference is in the X coordinates that these functions returns. Both return X coordinates between 0 and 1, but the first example returns **equally spaced** X coordinates:
```python
cumulated_samples = np.linspace(0, 1, len(cumulated_claim_amount))
```
and the second example return **un-equally spaced** X coordinates (spaced using the samples weights):
```python
cumulated_exposure = np.cumsum(ranked_exposure)
cumulated_exposure /= cumulated_exposure[-1]
```

### Suggest a potential alternative/fix

_No response_

COMMENT:
Indeed this looks like a bug. Someone would need to do a quick literature review to find the correct way to compute this curve when there are per-individual weights / exposure.

EDIT: I have the feeling that the cumsum variant is the correct one but I am not 100% sure.

Labels: Bug, Documentation
Two different versions for weighted lorenz curve calculation in the examples

### Describe the issue linked to the documentation

There are 2 definitions of (weighted) `lorenz_curve()` functions [here](https://scikit-learn.org/stable/auto_examples/linear_model/plot_tweedie_regression_insurance_claims.html) and [here](https://scikit-learn.org/stable/auto_examples/linear_model/plot_poisson_regression_non_normal_loss.html)

The difference is in the X coordinates that these functions returns. Both return X coordinates between 0 and 1, but the first example returns **equally spaced** X coordinates:
```python
cumulated_samples = np.linspace(0, 1, len(cumulated_claim_amount))
```
and the second example return **un-equally spaced** X coordinates (spaced using the samples weights):
```python
cumulated_exposure = np.cumsum(ranked_exposure)
cumulated_exposure /= cumulated_exposure[-1]
```

### Suggest a potential alternative/fix

_No response_

COMMENT:
> Indeed this looks like a bug. Someone would need to do a quick literature review to find the correct way to compute this curve when there are per-individual weights / exposure.
> 
> EDIT: I have the feeling that the cumsum variant is the correct one but I am not 100% sure.

I think you're right, the horizontal axis is the cumulative sum of probability Pr(Y=y). Here it corresponds to the individual weight/exposure.
We can use linspace when all claims are equally probable with probabilities 1/n.
Source: [https://en.wikipedia.org/wiki/Lorenz_curve#:~:text=For%20a%20discrete,to%20n%3A](url)

Labels: Bug, Documentation
Two different versions for weighted lorenz curve calculation in the examples

### Describe the issue linked to the documentation

There are 2 definitions of (weighted) `lorenz_curve()` functions [here](https://scikit-learn.org/stable/auto_examples/linear_model/plot_tweedie_regression_insurance_claims.html) and [here](https://scikit-learn.org/stable/auto_examples/linear_model/plot_poisson_regression_non_normal_loss.html)

The difference is in the X coordinates that these functions returns. Both return X coordinates between 0 and 1, but the first example returns **equally spaced** X coordinates:
```python
cumulated_samples = np.linspace(0, 1, len(cumulated_claim_amount))
```
and the second example return **un-equally spaced** X coordinates (spaced using the samples weights):
```python
cumulated_exposure = np.cumsum(ranked_exposure)
cumulated_exposure /= cumulated_exposure[-1]
```

### Suggest a potential alternative/fix

_No response_

COMMENT:
One version is unweighted, the other takes into account sample weights. So neither is wrong.

See also https://github.com/scikit-learn/scikit-learn/pull/21320#issuecomment-1983962648 and note that wikipedia is not a reliable source on those quantities.

Labels: Bug, Documentation
Two different versions for weighted lorenz curve calculation in the examples

### Describe the issue linked to the documentation

There are 2 definitions of (weighted) `lorenz_curve()` functions [here](https://scikit-learn.org/stable/auto_examples/linear_model/plot_tweedie_regression_insurance_claims.html) and [here](https://scikit-learn.org/stable/auto_examples/linear_model/plot_poisson_regression_non_normal_loss.html)

The difference is in the X coordinates that these functions returns. Both return X coordinates between 0 and 1, but the first example returns **equally spaced** X coordinates:
```python
cumulated_samples = np.linspace(0, 1, len(cumulated_claim_amount))
```
and the second example return **un-equally spaced** X coordinates (spaced using the samples weights):
```python
cumulated_exposure = np.cumsum(ranked_exposure)
cumulated_exposure /= cumulated_exposure[-1]
```

### Suggest a potential alternative/fix

_No response_

COMMENT:
I'm not expert but something I find strange in the [unweighted version](https://scikit-learn.org/stable/auto_examples/linear_model/plot_tweedie_regression_insurance_claims.html) is that it does use the weights (exposure) but differently:\
`cumulated_claim_amount = np.cumsum(ranked_pure_premium * ranked_exposure)`\
In a more intuitive terms - it use the weights to make the bar higher but not wider (each bar has the same width but the height of each bar is affected by the weight where in the [weighted version](https://scikit-learn.org/stable/auto_examples/linear_model/plot_poisson_regression_non_normal_loss.html) the weight affects both the height and the width of each bar)

Labels: Bug, Documentation
Two different versions for weighted lorenz curve calculation in the examples

### Describe the issue linked to the documentation

There are 2 definitions of (weighted) `lorenz_curve()` functions [here](https://scikit-learn.org/stable/auto_examples/linear_model/plot_tweedie_regression_insurance_claims.html) and [here](https://scikit-learn.org/stable/auto_examples/linear_model/plot_poisson_regression_non_normal_loss.html)

The difference is in the X coordinates that these functions returns. Both return X coordinates between 0 and 1, but the first example returns **equally spaced** X coordinates:
```python
cumulated_samples = np.linspace(0, 1, len(cumulated_claim_amount))
```
and the second example return **un-equally spaced** X coordinates (spaced using the samples weights):
```python
cumulated_exposure = np.cumsum(ranked_exposure)
cumulated_exposure /= cumulated_exposure[-1]
```

### Suggest a potential alternative/fix

_No response_

COMMENT:
Indeed I agree it's weird to only use the weights on y axis and not on the x axis.

I would intuitively expect the following weighing semantics to hold:

- setting weights to 0 to a fraction of the observations should be equivalent to trimming those data points;
- setting weights to 2 to a fraction of the observations (while keeping the other to unit weights) should be equivalent to duplicating those data points.

We can empirically check when those properties hold: the curves for the weighted vs trimmed/duplicated but unweighted data should match exactly.

Labels: Bug, Documentation
Two different versions for weighted lorenz curve calculation in the examples

### Describe the issue linked to the documentation

There are 2 definitions of (weighted) `lorenz_curve()` functions [here](https://scikit-learn.org/stable/auto_examples/linear_model/plot_tweedie_regression_insurance_claims.html) and [here](https://scikit-learn.org/stable/auto_examples/linear_model/plot_poisson_regression_non_normal_loss.html)

The difference is in the X coordinates that these functions returns. Both return X coordinates between 0 and 1, but the first example returns **equally spaced** X coordinates:
```python
cumulated_samples = np.linspace(0, 1, len(cumulated_claim_amount))
```
and the second example return **un-equally spaced** X coordinates (spaced using the samples weights):
```python
cumulated_exposure = np.cumsum(ranked_exposure)
cumulated_exposure /= cumulated_exposure[-1]
```

### Suggest a potential alternative/fix

_No response_

COMMENT:
@m-maggi sorry I had not seen your reply.

Could you try to do the simple experiment:

```python
import numpy as np
rng = np.random.default_rng(0)
n = 30
y_true = rng.uniform(low=0, high=10, size=n)
y_pred = y_true * rng.uniform(low=0.9, high=1.1, size=n)
exposure = rng.integers(low=0, high=10, size=n)
```

And then compute the curves for:

- the exposure-repeated data:

```python
y_true_repeated = y_true.repeat(sample_weight)
y_pred_repeated = y_pred.repeat(sample_weight)
sample_weight = None  # or equivalently: np.ones(n)
```

vs

- exposure-weighted data:

```python
y_true_weighted = y_true
y_pred_weighted = y_pred
sample_weight = exposure
```

Labels: Bug, Documentation
Two different versions for weighted lorenz curve calculation in the examples

### Describe the issue linked to the documentation

There are 2 definitions of (weighted) `lorenz_curve()` functions [here](https://scikit-learn.org/stable/auto_examples/linear_model/plot_tweedie_regression_insurance_claims.html) and [here](https://scikit-learn.org/stable/auto_examples/linear_model/plot_poisson_regression_non_normal_loss.html)

The difference is in the X coordinates that these functions returns. Both return X coordinates between 0 and 1, but the first example returns **equally spaced** X coordinates:
```python
cumulated_samples = np.linspace(0, 1, len(cumulated_claim_amount))
```
and the second example return **un-equally spaced** X coordinates (spaced using the samples weights):
```python
cumulated_exposure = np.cumsum(ranked_exposure)
cumulated_exposure /= cumulated_exposure[-1]
```

### Suggest a potential alternative/fix

_No response_

COMMENT:
But feel free to open a PR to fix the wrong example in any case.

Labels: Bug, Documentation
Problem of get_params attribute in skleran0.20.3

### Describe the bug

Greetings
I'm using Windows 7 and Orange 3.20.1 with sklearn 0.20.3 . I have generated some optimised ann models without knowing that this version of sklearn seems not to support get_params attribute and I have faced difficulties in extracting the weights and biases of the trained models. After exploring the internet I found that the latest sklearn 1.4 covers this property. Now due to the fact that generated models are not backward compatible considering this versions. I would be pleased if someone can guide me through determining the weights and the biases of the models trained in sklearn 0.20.3 without getting into trouble of training thousands of models again in sklearn 1.4. Great thanks

### Steps/Code to Reproduce

import pickle
import sklearn
# Load the model from a file
model=pickle.load(open("d:\\1.pkcls","rb"))
#params = model.get_params()

params = model.get_params()


### Expected Results

weights and biases of the model

### Actual Results

Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "<string>", line 9, in <module>
AttributeError: 'SklModelClassification' object has no attribute 'get_params'

### Versions

```shell
0.20.3
```

COMMENT:
That's a very old unsupported version. Please use the latest 1.4.1 release and open an issue if the issue persists.

Labels: Bug, Needs Triage
⚠️ CI failed on Ubuntu_Jammy_Jellyfish.pymin_conda_forge_openblas_ubuntu_2204 ⚠️

**CI is still failing on [Ubuntu_Jammy_Jellyfish.pymin_conda_forge_openblas_ubuntu_2204](https://dev.azure.com/scikit-learn/scikit-learn/_build/results?buildId=64462&view=logs&j=f71949a9-f9d9-549e-cf45-2e99c7b412d1)** (Feb 26, 2024)
- test_estimators[RegressorChain(base_estimator=Ridge())-check_estimator_sparse_array]

COMMENT:
Addressed in #28524

Labels: Needs Triage
Create estimators for inference only

### Describe the workflow you want to enable

Allow a trained estimator to be converted into a form suitable only for predict/transform type operations and not fitting. In many cases, the estimator could be made more compact or performant as part of this transformation. 

For instance, feature selection steps in a pipeline may rely on complex models during training, but at inference they simply drop unused features. When deploying the model, conversion of the feature selection step to a simpler form could save memory and model load time. 

### Describe your proposed solution

Add a new method to BaseEstimator `prep_for_inference(self)` which returns a model which retains all predict/transform methods but does not necessarily support fitting. By default it would return `self`. Estimators and transformers could override this as necessary. Pipeline would convert each step. 

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
Alternatively, this functionality could be supported on just feature selection and pipelines. This would require a PreselectedFeatures class that all feature selection classes could convert into.

Labels: New Feature
Create estimators for inference only

### Describe the workflow you want to enable

Allow a trained estimator to be converted into a form suitable only for predict/transform type operations and not fitting. In many cases, the estimator could be made more compact or performant as part of this transformation. 

For instance, feature selection steps in a pipeline may rely on complex models during training, but at inference they simply drop unused features. When deploying the model, conversion of the feature selection step to a simpler form could save memory and model load time. 

### Describe your proposed solution

Add a new method to BaseEstimator `prep_for_inference(self)` which returns a model which retains all predict/transform methods but does not necessarily support fitting. By default it would return `self`. Estimators and transformers could override this as necessary. Pipeline would convert each step. 

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
Did you have a look at `sklearn-onnx` when the idea is only to have the inference part?

Labels: New Feature
Create estimators for inference only

### Describe the workflow you want to enable

Allow a trained estimator to be converted into a form suitable only for predict/transform type operations and not fitting. In many cases, the estimator could be made more compact or performant as part of this transformation. 

For instance, feature selection steps in a pipeline may rely on complex models during training, but at inference they simply drop unused features. When deploying the model, conversion of the feature selection step to a simpler form could save memory and model load time. 

### Describe your proposed solution

Add a new method to BaseEstimator `prep_for_inference(self)` which returns a model which retains all predict/transform methods but does not necessarily support fitting. By default it would return `self`. Estimators and transformers could override this as necessary. Pipeline would convert each step. 

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
cc @GaelVaroquaux since you were mentioning exactly this the other day.

Labels: New Feature
Create estimators for inference only

### Describe the workflow you want to enable

Allow a trained estimator to be converted into a form suitable only for predict/transform type operations and not fitting. In many cases, the estimator could be made more compact or performant as part of this transformation. 

For instance, feature selection steps in a pipeline may rely on complex models during training, but at inference they simply drop unused features. When deploying the model, conversion of the feature selection step to a simpler form could save memory and model load time. 

### Describe your proposed solution

Add a new method to BaseEstimator `prep_for_inference(self)` which returns a model which retains all predict/transform methods but does not necessarily support fitting. By default it would return `self`. Estimators and transformers could override this as necessary. Pipeline would convert each step. 

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
@adrinjalali @GaelVaroquaux curious what your thoughts were. My intuition is to use onnx or ... completely change the scikit-learn API and have the fitted model be a different class than the fitting algorithm ;)

Labels: New Feature
Create estimators for inference only

### Describe the workflow you want to enable

Allow a trained estimator to be converted into a form suitable only for predict/transform type operations and not fitting. In many cases, the estimator could be made more compact or performant as part of this transformation. 

For instance, feature selection steps in a pipeline may rely on complex models during training, but at inference they simply drop unused features. When deploying the model, conversion of the feature selection step to a simpler form could save memory and model load time. 

### Describe your proposed solution

Add a new method to BaseEstimator `prep_for_inference(self)` which returns a model which retains all predict/transform methods but does not necessarily support fitting. By default it would return `self`. Estimators and transformers could override this as necessary. Pipeline would convert each step. 

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
> @adrinjalali @GaelVaroquaux curious what your thoughts were.

Facilitate restoring predictors from storage, including across versions. Consider for instance linear models for regression. The prediction function is something really simple that is easy to keep stable across time. On the other hand, for the fitting algorithm, it's much harder to promise that options won't change, or that a fitting procedure will give the same result on the same data.

> My intuition is to use onnx

I would say we should consider it optionally, but one of the factors of success of scikit-learn historically has been that it requires very little that is not installed on every data scientist's computer.


> or ... completely change the scikit-learn API and have the fitted model be a different class than the fitting algorithm ;)

I think that we want to go this way. Another factor of success of scikit-learn is that it exposes a very simple surface to users, with little to learn or understand.

Labels: New Feature
Create estimators for inference only

### Describe the workflow you want to enable

Allow a trained estimator to be converted into a form suitable only for predict/transform type operations and not fitting. In many cases, the estimator could be made more compact or performant as part of this transformation. 

For instance, feature selection steps in a pipeline may rely on complex models during training, but at inference they simply drop unused features. When deploying the model, conversion of the feature selection step to a simpler form could save memory and model load time. 

### Describe your proposed solution

Add a new method to BaseEstimator `prep_for_inference(self)` which returns a model which retains all predict/transform methods but does not necessarily support fitting. By default it would return `self`. Estimators and transformers could override this as necessary. Pipeline would convert each step. 

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
This can also be a separate package (to iterate faster also).

We could have a `scikit-learn-predictor` kind of thing, where we get predictors from our classes. Testing is also not that hard, we test if the output of the predictor is the same as the sklearn native class. However, this does seem A LOT like ONNX, with the benefit of it being in python and lightweight.

Labels: New Feature
Create estimators for inference only

### Describe the workflow you want to enable

Allow a trained estimator to be converted into a form suitable only for predict/transform type operations and not fitting. In many cases, the estimator could be made more compact or performant as part of this transformation. 

For instance, feature selection steps in a pipeline may rely on complex models during training, but at inference they simply drop unused features. When deploying the model, conversion of the feature selection step to a simpler form could save memory and model load time. 

### Describe your proposed solution

Add a new method to BaseEstimator `prep_for_inference(self)` which returns a model which retains all predict/transform methods but does not necessarily support fitting. By default it would return `self`. Estimators and transformers could override this as necessary. Pipeline would convert each step. 

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
Having a "predictor only" solution to the problem of "I trained a model in vX and now want to use it on vX+1" would be cool. Especially because there is currently no good answer to the problem and it crops up semi regularly on the issue tracker.

Labels: New Feature
Create estimators for inference only

### Describe the workflow you want to enable

Allow a trained estimator to be converted into a form suitable only for predict/transform type operations and not fitting. In many cases, the estimator could be made more compact or performant as part of this transformation. 

For instance, feature selection steps in a pipeline may rely on complex models during training, but at inference they simply drop unused features. When deploying the model, conversion of the feature selection step to a simpler form could save memory and model load time. 

### Describe your proposed solution

Add a new method to BaseEstimator `prep_for_inference(self)` which returns a model which retains all predict/transform methods but does not necessarily support fitting. By default it would return `self`. Estimators and transformers could override this as necessary. Pipeline would convert each step. 

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
Hi! I'm new here! I've also ran into this. I think it's a common thing for voting ensembles and stacking ensembles.

sklearn.ensemble accepts prefit estimators. I think it makes sense for the user to have a way of specifying whether an estimator has been fitted. Upon closer inspection, it seems that this issue has been getting a lot of work here: https://github.com/scikit-learn/scikit-learn/pull/29705

Now that FrozenEstimator is merged I think this issue can be closed?

Labels: New Feature