Talip7/scikit-learn-issues-multilabel huggingface dataset:
partial_dependence should respect sample weights

### Describe the workflow you want to enable

Currently, the inspect.partial_dependence funtions calculate arithmetic averages over predictions. For models fitted with sample weights, this is between suboptimal and wrong.

### Describe your proposed solution

Add new argument "sample_weight = None". If vector of right length, replace arithmetic average of predictions by weighted averages.

Note that this does not affect the calculation of ICE curves, just the aggregate.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
@vitaliset Not yet started! I would be super happy if you could dig into this. 

I think there are two ways to calculate PDPs. For the model agnostic logic, we would probably need to replace `np.mean()` by `np.average()`. There is a second (fast exact) way for trees based on Friedman's algorithm. Here, I am not sure if the change is straightforward, but in the worst case we would just say "The fast exact PDP for trees does not support case weights".

Labels: New Feature, module:inspection
partial_dependence should respect sample weights

### Describe the workflow you want to enable

Currently, the inspect.partial_dependence funtions calculate arithmetic averages over predictions. For models fitted with sample weights, this is between suboptimal and wrong.

### Describe your proposed solution

Add new argument "sample_weight = None". If vector of right length, replace arithmetic average of predictions by weighted averages.

Note that this does not affect the calculation of ICE curves, just the aggregate.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
Fantastic research! 

Additional possible tests for "brute": PDP unweighted is the same as PDP with all weights 1.0. Same for all weights 2.0.

My feeling is : the "recurse" approach for trees should respect sample weights of the training data when tracking split weights. I cannot explain why the two tree-methods are different. Should we open an issue for clarification?

Labels: New Feature, module:inspection
partial_dependence should respect sample weights

### Describe the workflow you want to enable

Currently, the inspect.partial_dependence funtions calculate arithmetic averages over predictions. For models fitted with sample weights, this is between suboptimal and wrong.

### Describe your proposed solution

Add new argument "sample_weight = None". If vector of right length, replace arithmetic average of predictions by weighted averages.

Note that this does not affect the calculation of ICE curves, just the aggregate.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
> The algorithms based on the CART implementation use the sample_weight of the .fit method through the weighted_n_node_samples attribute (code above).
> While the estimators of HistGradientBoosting doesn't.

That's correct - this is something that is explicitly not supported in the HGBDT trees yet:

https://github.com/scikit-learn/scikit-learn/blob/205f3b76ef8500c90c346c6c6fb6f4e589368278/sklearn/ensemble/_hist_gradient_boosting/gradient_boosting.py#L1142-L1148

(See https://github.com/scikit-learn/scikit-learn/pull/14696#issuecomment-548295813 for historical decision / context)

I thought there was an open issue for that, but it looks like there isn't. Feel free to open one!

Labels: New Feature, module:inspection
Logisitic Regression: Questions

### Describe the workflow you want to enable

Hi all,

I found an exact fomrula on my own playing with maths for the logisitc regression.

I do not know if it is implemented yet for logisitc regression in sklearn.

Sorry if it is

### Describe your proposed solution

https://medium.com/@playe.nicolas/logistic-regression-the-global-minimum-142af772e4f1

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
Moreover, this method can use a part of the code from Linear Regression:
- more speed
- no random state
- better generalization (best results)

Labels: New Feature, Needs Triage
Logisitic Regression: Questions

### Describe the workflow you want to enable

Hi all,

I found an exact fomrula on my own playing with maths for the logisitc regression.

I do not know if it is implemented yet for logisitc regression in sklearn.

Sorry if it is

### Describe your proposed solution

https://medium.com/@playe.nicolas/logistic-regression-the-global-minimum-142af772e4f1

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
Our LogisticRegression class is quite a bit more general than what you have in your notes. You can have a look at the docs here: https://scikit-learn.org/dev/modules/generated/sklearn.linear_model.LogisticRegression.html#sklearn.linear_model.LogisticRegression and here: https://scikit-learn.org/dev/modules/linear_model.html#logistic-regression

Labels: New Feature, Needs Triage
Logisitic Regression: Questions

### Describe the workflow you want to enable

Hi all,

I found an exact fomrula on my own playing with maths for the logisitc regression.

I do not know if it is implemented yet for logisitc regression in sklearn.

Sorry if it is

### Describe your proposed solution

https://medium.com/@playe.nicolas/logistic-regression-the-global-minimum-142af772e4f1

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
Hello,
Thanks for your comments.

As you can see:
https://medium.com/@playe.nicolas/linear-regression-global-minimum-a-new-approch-fcf43366ad0b
https://medium.com/@playe.nicolas/logistic-regression-the-global-minimum-142af772e4f1

You'll see that the 2 solutions are very close.
So if you consider the variable change Y => log ( (Y-€) / (1 -Y-€)) where Y € {0,1} and € >0 very small, you'll have a regression problem like a linear regression. If you want then to put regularization, with just the variable change, by applying regularisation on your linear regression code (LinearRegression, Ridge, Lasso, and ElascticNet) you will have the best results.

I tried this solution on several datasets and it works very well.
Your LinearRegression code is very fast. So by doing just this variable change, you'll have the fastest way to compute linear regression with the best generalisation (dividing by 2 the time compute on fit).

If you want I can share you my codes and I can do a pull request to show you:
- how easy it is to code it
- how fast it is to compute the fit
- how better are the results

Labels: New Feature, Needs Triage
Make automatic validation for all scikit-learn public functions

PR #22722 introduced a decorator to validate the parameters of functions. We now need to use it for all functions where it is applicable.

Please open one PR per function. The title of the PR must mention which function it's dealing with. We recommend using the following pattern for titles:

```
MAINT Parameters validation for <function>
```

where `<function>` is a placeholder to be replaced with the function you chose.

The description of the PR must begin with `Towards #24862` so that this issue and the PR are mutually crossed-linked.

### Steps

1. Chose a public function that is documented in https://scikit-learn.org/dev/modules/classes.html. Check in the source code if the function contains some manual parameter validation (i.e. you should see some `if` condition and error raising). In case there is no validation in the function, you can report it in the issue where we will decide whether or not to skip the function.

2. To validate the function, you need to decorate it with the decorator `sklearn.utils._param_validation.validate_params`. **Do not rely only on the docstring of the estimator to define it**: although it can help, it's important to primarily rely on the implementation to find the valid values because the docstring might not be completely accurate. The decorator take a Python dictionary as input where each key corresponds to a parameter name and the value corresponds to the associate constraints. You can find an example for `kmeans_plusplus` below https://github.com/scikit-learn/scikit-learn/blob/2e481f114169396660f0051eee1bcf6bcddfd556/sklearn/cluster/_kmeans.py#L63-L74 You can also get more details regarding the constraints by looking at the different Estimators validation previously implemented (cf. the `_parameter_constraints` attribute). 
3. All existing simple param validation can now be removed. (simple means that does not depend on the input data or that does not depend on the value of another parameter for instance).
4. Tests that check error messages from simple param validation can also be removed (carefully: we need to keep the tests checking for more complex param validation !).

5. Finally, add the function to the list of the common param validation test https://github.com/scikit-learn/scikit-learn/blob/2e481f114169396660f0051eee1bcf6bcddfd556/sklearn/tests/test_public_functions.py#L11-L13
      and make sure the test passes: `pytest -vl sklearn/tests/test_public_functions.py`

### Functions already updated:

See "details" in section 1 

Be aware that you can see an up-to-date list at the following link: https://github.com/scikit-learn/scikit-learn/blob/main/sklearn/tests/test_public_functions.py#L132

COMMENT:
Hi can I work on sklearn.cluster.spectral_clustering

Labels: Sprint, good first issue, Meta-issue, Validation
Make automatic validation for all scikit-learn public functions

PR #22722 introduced a decorator to validate the parameters of functions. We now need to use it for all functions where it is applicable.

Please open one PR per function. The title of the PR must mention which function it's dealing with. We recommend using the following pattern for titles:

```
MAINT Parameters validation for <function>
```

where `<function>` is a placeholder to be replaced with the function you chose.

The description of the PR must begin with `Towards #24862` so that this issue and the PR are mutually crossed-linked.

### Steps

1. Chose a public function that is documented in https://scikit-learn.org/dev/modules/classes.html. Check in the source code if the function contains some manual parameter validation (i.e. you should see some `if` condition and error raising). In case there is no validation in the function, you can report it in the issue where we will decide whether or not to skip the function.

2. To validate the function, you need to decorate it with the decorator `sklearn.utils._param_validation.validate_params`. **Do not rely only on the docstring of the estimator to define it**: although it can help, it's important to primarily rely on the implementation to find the valid values because the docstring might not be completely accurate. The decorator take a Python dictionary as input where each key corresponds to a parameter name and the value corresponds to the associate constraints. You can find an example for `kmeans_plusplus` below https://github.com/scikit-learn/scikit-learn/blob/2e481f114169396660f0051eee1bcf6bcddfd556/sklearn/cluster/_kmeans.py#L63-L74 You can also get more details regarding the constraints by looking at the different Estimators validation previously implemented (cf. the `_parameter_constraints` attribute). 
3. All existing simple param validation can now be removed. (simple means that does not depend on the input data or that does not depend on the value of another parameter for instance).
4. Tests that check error messages from simple param validation can also be removed (carefully: we need to keep the tests checking for more complex param validation !).

5. Finally, add the function to the list of the common param validation test https://github.com/scikit-learn/scikit-learn/blob/2e481f114169396660f0051eee1bcf6bcddfd556/sklearn/tests/test_public_functions.py#L11-L13
      and make sure the test passes: `pytest -vl sklearn/tests/test_public_functions.py`

### Functions already updated:

See "details" in section 1 

Be aware that you can see an up-to-date list at the following link: https://github.com/scikit-learn/scikit-learn/blob/main/sklearn/tests/test_public_functions.py#L132

COMMENT:
Hi ! Can I try to do the sklearn.cluster.affinity_propagation function ?

Labels: Sprint, good first issue, Meta-issue, Validation
Make automatic validation for all scikit-learn public functions

PR #22722 introduced a decorator to validate the parameters of functions. We now need to use it for all functions where it is applicable.

Please open one PR per function. The title of the PR must mention which function it's dealing with. We recommend using the following pattern for titles:

```
MAINT Parameters validation for <function>
```

where `<function>` is a placeholder to be replaced with the function you chose.

The description of the PR must begin with `Towards #24862` so that this issue and the PR are mutually crossed-linked.

### Steps

1. Chose a public function that is documented in https://scikit-learn.org/dev/modules/classes.html. Check in the source code if the function contains some manual parameter validation (i.e. you should see some `if` condition and error raising). In case there is no validation in the function, you can report it in the issue where we will decide whether or not to skip the function.

2. To validate the function, you need to decorate it with the decorator `sklearn.utils._param_validation.validate_params`. **Do not rely only on the docstring of the estimator to define it**: although it can help, it's important to primarily rely on the implementation to find the valid values because the docstring might not be completely accurate. The decorator take a Python dictionary as input where each key corresponds to a parameter name and the value corresponds to the associate constraints. You can find an example for `kmeans_plusplus` below https://github.com/scikit-learn/scikit-learn/blob/2e481f114169396660f0051eee1bcf6bcddfd556/sklearn/cluster/_kmeans.py#L63-L74 You can also get more details regarding the constraints by looking at the different Estimators validation previously implemented (cf. the `_parameter_constraints` attribute). 
3. All existing simple param validation can now be removed. (simple means that does not depend on the input data or that does not depend on the value of another parameter for instance).
4. Tests that check error messages from simple param validation can also be removed (carefully: we need to keep the tests checking for more complex param validation !).

5. Finally, add the function to the list of the common param validation test https://github.com/scikit-learn/scikit-learn/blob/2e481f114169396660f0051eee1bcf6bcddfd556/sklearn/tests/test_public_functions.py#L11-L13
      and make sure the test passes: `pytest -vl sklearn/tests/test_public_functions.py`

### Functions already updated:

See "details" in section 1 

Be aware that you can see an up-to-date list at the following link: https://github.com/scikit-learn/scikit-learn/blob/main/sklearn/tests/test_public_functions.py#L132

COMMENT:
Yes sure, no need to ask. Just commenting that you are working on it is enough :)

Labels: Sprint, good first issue, Meta-issue, Validation
Make automatic validation for all scikit-learn public functions

PR #22722 introduced a decorator to validate the parameters of functions. We now need to use it for all functions where it is applicable.

Please open one PR per function. The title of the PR must mention which function it's dealing with. We recommend using the following pattern for titles:

```
MAINT Parameters validation for <function>
```

where `<function>` is a placeholder to be replaced with the function you chose.

The description of the PR must begin with `Towards #24862` so that this issue and the PR are mutually crossed-linked.

### Steps

1. Chose a public function that is documented in https://scikit-learn.org/dev/modules/classes.html. Check in the source code if the function contains some manual parameter validation (i.e. you should see some `if` condition and error raising). In case there is no validation in the function, you can report it in the issue where we will decide whether or not to skip the function.

2. To validate the function, you need to decorate it with the decorator `sklearn.utils._param_validation.validate_params`. **Do not rely only on the docstring of the estimator to define it**: although it can help, it's important to primarily rely on the implementation to find the valid values because the docstring might not be completely accurate. The decorator take a Python dictionary as input where each key corresponds to a parameter name and the value corresponds to the associate constraints. You can find an example for `kmeans_plusplus` below https://github.com/scikit-learn/scikit-learn/blob/2e481f114169396660f0051eee1bcf6bcddfd556/sklearn/cluster/_kmeans.py#L63-L74 You can also get more details regarding the constraints by looking at the different Estimators validation previously implemented (cf. the `_parameter_constraints` attribute). 
3. All existing simple param validation can now be removed. (simple means that does not depend on the input data or that does not depend on the value of another parameter for instance).
4. Tests that check error messages from simple param validation can also be removed (carefully: we need to keep the tests checking for more complex param validation !).

5. Finally, add the function to the list of the common param validation test https://github.com/scikit-learn/scikit-learn/blob/2e481f114169396660f0051eee1bcf6bcddfd556/sklearn/tests/test_public_functions.py#L11-L13
      and make sure the test passes: `pytest -vl sklearn/tests/test_public_functions.py`

### Functions already updated:

See "details" in section 1 

Be aware that you can see an up-to-date list at the following link: https://github.com/scikit-learn/scikit-learn/blob/main/sklearn/tests/test_public_functions.py#L132

COMMENT:
Hi. I can work on sklearn.cluster.mean_shift

Labels: Sprint, good first issue, Meta-issue, Validation
Make automatic validation for all scikit-learn public functions

PR #22722 introduced a decorator to validate the parameters of functions. We now need to use it for all functions where it is applicable.

Please open one PR per function. The title of the PR must mention which function it's dealing with. We recommend using the following pattern for titles:

```
MAINT Parameters validation for <function>
```

where `<function>` is a placeholder to be replaced with the function you chose.

The description of the PR must begin with `Towards #24862` so that this issue and the PR are mutually crossed-linked.

### Steps

1. Chose a public function that is documented in https://scikit-learn.org/dev/modules/classes.html. Check in the source code if the function contains some manual parameter validation (i.e. you should see some `if` condition and error raising). In case there is no validation in the function, you can report it in the issue where we will decide whether or not to skip the function.

2. To validate the function, you need to decorate it with the decorator `sklearn.utils._param_validation.validate_params`. **Do not rely only on the docstring of the estimator to define it**: although it can help, it's important to primarily rely on the implementation to find the valid values because the docstring might not be completely accurate. The decorator take a Python dictionary as input where each key corresponds to a parameter name and the value corresponds to the associate constraints. You can find an example for `kmeans_plusplus` below https://github.com/scikit-learn/scikit-learn/blob/2e481f114169396660f0051eee1bcf6bcddfd556/sklearn/cluster/_kmeans.py#L63-L74 You can also get more details regarding the constraints by looking at the different Estimators validation previously implemented (cf. the `_parameter_constraints` attribute). 
3. All existing simple param validation can now be removed. (simple means that does not depend on the input data or that does not depend on the value of another parameter for instance).
4. Tests that check error messages from simple param validation can also be removed (carefully: we need to keep the tests checking for more complex param validation !).

5. Finally, add the function to the list of the common param validation test https://github.com/scikit-learn/scikit-learn/blob/2e481f114169396660f0051eee1bcf6bcddfd556/sklearn/tests/test_public_functions.py#L11-L13
      and make sure the test passes: `pytest -vl sklearn/tests/test_public_functions.py`

### Functions already updated:

See "details" in section 1 

Be aware that you can see an up-to-date list at the following link: https://github.com/scikit-learn/scikit-learn/blob/main/sklearn/tests/test_public_functions.py#L132

COMMENT:
I can work on `cluster.dbscan`

Labels: Sprint, good first issue, Meta-issue, Validation
Make automatic validation for all scikit-learn public functions

PR #22722 introduced a decorator to validate the parameters of functions. We now need to use it for all functions where it is applicable.

Please open one PR per function. The title of the PR must mention which function it's dealing with. We recommend using the following pattern for titles:

```
MAINT Parameters validation for <function>
```

where `<function>` is a placeholder to be replaced with the function you chose.

The description of the PR must begin with `Towards #24862` so that this issue and the PR are mutually crossed-linked.

### Steps

1. Chose a public function that is documented in https://scikit-learn.org/dev/modules/classes.html. Check in the source code if the function contains some manual parameter validation (i.e. you should see some `if` condition and error raising). In case there is no validation in the function, you can report it in the issue where we will decide whether or not to skip the function.

2. To validate the function, you need to decorate it with the decorator `sklearn.utils._param_validation.validate_params`. **Do not rely only on the docstring of the estimator to define it**: although it can help, it's important to primarily rely on the implementation to find the valid values because the docstring might not be completely accurate. The decorator take a Python dictionary as input where each key corresponds to a parameter name and the value corresponds to the associate constraints. You can find an example for `kmeans_plusplus` below https://github.com/scikit-learn/scikit-learn/blob/2e481f114169396660f0051eee1bcf6bcddfd556/sklearn/cluster/_kmeans.py#L63-L74 You can also get more details regarding the constraints by looking at the different Estimators validation previously implemented (cf. the `_parameter_constraints` attribute). 
3. All existing simple param validation can now be removed. (simple means that does not depend on the input data or that does not depend on the value of another parameter for instance).
4. Tests that check error messages from simple param validation can also be removed (carefully: we need to keep the tests checking for more complex param validation !).

5. Finally, add the function to the list of the common param validation test https://github.com/scikit-learn/scikit-learn/blob/2e481f114169396660f0051eee1bcf6bcddfd556/sklearn/tests/test_public_functions.py#L11-L13
      and make sure the test passes: `pytest -vl sklearn/tests/test_public_functions.py`

### Functions already updated:

See "details" in section 1 

Be aware that you can see an up-to-date list at the following link: https://github.com/scikit-learn/scikit-learn/blob/main/sklearn/tests/test_public_functions.py#L132

COMMENT:
hi! I am working on ward_tree

Labels: Sprint, good first issue, Meta-issue, Validation