Talip7/scikit-learn-issues-multilabel huggingface dataset:
Handle all-zeros cases for multioutput metrics

### Describe the workflow you want to enable

For multioutput problems, all-zero label columns (or in general constant label columns) can sometimes happen, for example when using cross-validation. Most metrics (e.g. precision, recall, F1, AUPRC/average recall) return 0.0 with a warning, AUROC throws an error.

This is particularly common in chemoinformatics, where we have many heavily imbalanced multioutput problems. Getting all-zeros column or two among 50-100 targets almost always happens. Currently, I write wrappers in each project, manually looping through columns and ignoring those with constant targets, which is 1) code duplication 2) clearly a missing feature from the built-in metrics.

### Describe your proposed solution

add an option for the multioutput case for metrics whether to ignore the label columns with constant value, e.g. `constant_value: str = "use"`, with possible values "use" or "ignore". If all columns are constant, throw an error. If only some are constant, ignore them and aggregate the value only from those columns that had at least 2 classes.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
I also wonder if this stems from errors like division by zero, or other reasons why some metric would be invalid, in which case, I wonder if this actually lies under https://github.com/scikit-learn/scikit-learn/issues/29048 and if `replace_undefined_by` would cover the issue here.

cc @glemaitre @StefanieSenger

Labels: New Feature, Needs Decision
Handle all-zeros cases for multioutput metrics

### Describe the workflow you want to enable

For multioutput problems, all-zero label columns (or in general constant label columns) can sometimes happen, for example when using cross-validation. Most metrics (e.g. precision, recall, F1, AUPRC/average recall) return 0.0 with a warning, AUROC throws an error.

This is particularly common in chemoinformatics, where we have many heavily imbalanced multioutput problems. Getting all-zeros column or two among 50-100 targets almost always happens. Currently, I write wrappers in each project, manually looping through columns and ignoring those with constant targets, which is 1) code duplication 2) clearly a missing feature from the built-in metrics.

### Describe your proposed solution

add an option for the multioutput case for metrics whether to ignore the label columns with constant value, e.g. `constant_value: str = "use"`, with possible values "use" or "ignore". If all columns are constant, throw an error. If only some are constant, ignore them and aggregate the value only from those columns that had at least 2 classes.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
It might be caused by division by zero. And it would be good to have a reproducible in order to check if it is really caused by it.

If we allow users to set return values with `replace_undefined_by`, then it could be any value that falls into the usual space of that score. But still it would appear as a constant then in this setting. So I am not sure if this would help in such a case.

Labels: New Feature, Needs Decision
Allow missing values in multioutput metrics

### Describe the workflow you want to enable

In many multioutput problems, for example in chemoinformatics, there are missing values in target values, because only some properties are actually measured. Currently, scikit-learn requires all values to be present, leading to a lot of duplicated code across molecular property prediction projects, see e.g. [Open Graph Benchmark evaluation code](https://github.com/snap-stanford/ogb/blob/master/ogb/graphproppred/evaluate.py#L168). Such evaluators are peppered with lines like:
```
# ignore nan values
is_labeled = y_true[:,i] == y_true[:,i]
```
Another example from my recent paper [is available here](https://github.com/j-adamczyk/MOLTOP/blob/master/utils.py#L19):
```
    for i in range(y_pred.shape[1]):
        mask = ~np.isnan(y_test[:, i])
        y_test_i = y_test[mask, i]
        y_pred_i = y_pred[mask, i]
```
This is irrirating, since basically people are forced to manually reimplement the multioutput case, filtering out NaN values

### Describe your proposed solution

I see two possible solutions:
1. Always use NaN-aware functions instead, e.g. `np.nanmean()`, and possibly print a warning if NaN values are detected. This would be a behavior similar to transformers, e.g. `MinMaxScaler` ([list from the docs](https://scikit-learn.org/stable/modules/impute.html#estimators-that-handle-nan-values))
2. Add a parameter whether to ignore NaN values, e.g. `missing_behavior: str = "raise"` with possible values "raise" and "ignore". The default value of "raise" would exactly follow the existing behavior, while "ignore" would simply omit NaN values

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
Thanks for the detailed issue. Currently we don't have any plans to add support for missing values regarding target variables.

I wonder if implementing a meta-metric and a meta-estimator in your case which would filter out missing values from target and pass it along to the metric / estimator would be a solution?

That seems easy to implement and should remove duplicate code from your side. Something like a `_PassthroughScorer` but with filtering for missing values.

Closing this as out of scope for now, unless other @scikit-learn/core-devs think differently.

Labels: New Feature, Needs Triage
An inconsistency between the document of `LogisticRegression` and code implementation

### Describe the issue linked to the documentation

Hi,

I may find a potential condition missing in [`sklearn.linear_model.LogisticRegression`](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html). 

As mentioned in the document of parameter `dual`:
> **dual: bool, default=False**
Dual (constrained) or primal (regularized) formulation. **Dual formulation is only implemented for l2 penalty with liblinear solver.** Prefer dual=False when n_samples > n_features.

Corresponding part found in the source code:
```python
def _check_solver(solver, penalty, dual):
    if solver not in ["liblinear", "saga"] and penalty not in ("l2", None):
        raise ValueError(
            f"Solver {solver} supports only 'l2' or None penalties, got {penalty} "
            "penalty."
        )
    if solver != "liblinear" and dual:
        raise ValueError(f"Solver {solver} supports only dual=False, got dual={dual}")

    if penalty == "elasticnet" and solver != "saga":
        raise ValueError(
            f"Only 'saga' solver supports elasticnet penalty, got solver={solver}."
        )

    if solver == "liblinear" and penalty is None:
        raise ValueError("penalty=None is not supported for the liblinear solver")

    return solver
```
Apprarently, the relationship between `dual` and `solver` are checked **without** the condition of l2 `penalty`.
(`solver == 'liblinear'` && `penalty == 'l1'` && `dual = True`) can still pass the `_check_solver` function.

Could you check it?

### Suggest a potential alternative/fix

Perhaps we can modify the judgment condition.

COMMENT:
This seems AI generated, very similar to https://github.com/scikit-learn/scikit-learn/issues/29440

Other issues by the same author seem also AI generated: #29463, #29464

@ParsifalXu we don't appreciate AI generated content (issue or PR). If you find issues and would like to fix, please open PRs for them. Otherwise these issues so far seem low impact and take more energy from maintainers than they bring value. Not banning you from the org since you opened https://github.com/scikit-learn/scikit-learn/pull/29468, but more AI generated content might ban you from the org.

Labels: Documentation, spam, Needs Triage
An inconsistency between the document of `LogisticRegression` and code implementation

### Describe the issue linked to the documentation

Hi,

I may find a potential condition missing in [`sklearn.linear_model.LogisticRegression`](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html). 

As mentioned in the document of parameter `dual`:
> **dual: bool, default=False**
Dual (constrained) or primal (regularized) formulation. **Dual formulation is only implemented for l2 penalty with liblinear solver.** Prefer dual=False when n_samples > n_features.

Corresponding part found in the source code:
```python
def _check_solver(solver, penalty, dual):
    if solver not in ["liblinear", "saga"] and penalty not in ("l2", None):
        raise ValueError(
            f"Solver {solver} supports only 'l2' or None penalties, got {penalty} "
            "penalty."
        )
    if solver != "liblinear" and dual:
        raise ValueError(f"Solver {solver} supports only dual=False, got dual={dual}")

    if penalty == "elasticnet" and solver != "saga":
        raise ValueError(
            f"Only 'saga' solver supports elasticnet penalty, got solver={solver}."
        )

    if solver == "liblinear" and penalty is None:
        raise ValueError("penalty=None is not supported for the liblinear solver")

    return solver
```
Apprarently, the relationship between `dual` and `solver` are checked **without** the condition of l2 `penalty`.
(`solver == 'liblinear'` && `penalty == 'l1'` && `dual = True`) can still pass the `_check_solver` function.

Could you check it?

### Suggest a potential alternative/fix

Perhaps we can modify the judgment condition.

COMMENT:
Hi @adrinjalali , 

I sincerely apologize for any extra work this may have caused. However, I want to clarify that this content is not generated by AI. I am a beginner trying to learn how to use your great tool well. I truly appreciate your efforts in developing this fantastic tool and I am simply trying to do some contributions, even if they are just minor documentation improvements. 

To better understand how to submit an issue, I reviewed some previous issues and tried to learn how to describe the questions or issues as clear as possible and hoped that you can make a quick fix or confirm (I can submit PRs too). Maybe the similarity in text might lead you to believe this was AI-generated ( **I typed every single word by myself!** ). So if possible, please remove the `spam` label. If you think this issue is a problem, you can just confirm it and I will try to make a PR to improve the document, if not, please just let me know and close the issue. 

Once again, I apologize for any inconvenience this has caused you and other contributors. I have always believed that good documentation and good code each other.  If there are better ways to provide documentation improvements, please let me know.

Labels: Documentation, spam, Needs Triage
An inconsistency between the document of `LogisticRegression` and code implementation

### Describe the issue linked to the documentation

Hi,

I may find a potential condition missing in [`sklearn.linear_model.LogisticRegression`](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html). 

As mentioned in the document of parameter `dual`:
> **dual: bool, default=False**
Dual (constrained) or primal (regularized) formulation. **Dual formulation is only implemented for l2 penalty with liblinear solver.** Prefer dual=False when n_samples > n_features.

Corresponding part found in the source code:
```python
def _check_solver(solver, penalty, dual):
    if solver not in ["liblinear", "saga"] and penalty not in ("l2", None):
        raise ValueError(
            f"Solver {solver} supports only 'l2' or None penalties, got {penalty} "
            "penalty."
        )
    if solver != "liblinear" and dual:
        raise ValueError(f"Solver {solver} supports only dual=False, got dual={dual}")

    if penalty == "elasticnet" and solver != "saga":
        raise ValueError(
            f"Only 'saga' solver supports elasticnet penalty, got solver={solver}."
        )

    if solver == "liblinear" and penalty is None:
        raise ValueError("penalty=None is not supported for the liblinear solver")

    return solver
```
Apprarently, the relationship between `dual` and `solver` are checked **without** the condition of l2 `penalty`.
(`solver == 'liblinear'` && `penalty == 'l1'` && `dual = True`) can still pass the `_check_solver` function.

Could you check it?

### Suggest a potential alternative/fix

Perhaps we can modify the judgment condition.

COMMENT:
Feel free to open PRs in these cases with proper tests if needed, and fixes to documentation.

Labels: Documentation, spam, Needs Triage
Add "ensure_positive" to check_array for non-negative value validation

### Describe the workflow you want to enable

Adding an option to `ensure_positive` to the `sklearn.utils.validation.check_array` function. 
Currently, to ensure that an input array contains only positive values `check_non_negative` is used. Most users then either use the `check_non_negative` right after `check_array`, or create a custom `check_X` function that contains both checks. 

Additionally, the new [estimator_checks](https://github.com/scikit-learn/scikit-learn/blob/d79cb58c464f0b54bf0f0286c725d2df837574d0/sklearn/utils/estimator_checks.py), in the case of the `"requires_positive_X": True` tag, require a `ValueError` to be raised if negative values are found in X. This will also help simplify making a new estimator more compliant easier, just with using `check_array`, as a large number of users already do.

### Describe your proposed solution

Adding an option to `ensure_positive` to the `sklearn.utils.validation.check_array` function, that contains the `check_non_negative` functionality. 

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
> Most users then either use the check_non_negative right after check_array

Most `check_non_negative` calls are like that in scikit-learn. I'm in favor to add this option directly in `check_array`.

Labels: New Feature
Add "ensure_positive" to check_array for non-negative value validation

### Describe the workflow you want to enable

Adding an option to `ensure_positive` to the `sklearn.utils.validation.check_array` function. 
Currently, to ensure that an input array contains only positive values `check_non_negative` is used. Most users then either use the `check_non_negative` right after `check_array`, or create a custom `check_X` function that contains both checks. 

Additionally, the new [estimator_checks](https://github.com/scikit-learn/scikit-learn/blob/d79cb58c464f0b54bf0f0286c725d2df837574d0/sklearn/utils/estimator_checks.py), in the case of the `"requires_positive_X": True` tag, require a `ValueError` to be raised if negative values are found in X. This will also help simplify making a new estimator more compliant easier, just with using `check_array`, as a large number of users already do.

### Describe your proposed solution

Adding an option to `ensure_positive` to the `sklearn.utils.validation.check_array` function, that contains the `check_non_negative` functionality. 

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
> > Most users then either use the check_non_negative right after check_array
> 
> Most `check_non_negative` calls are like that in scikit-learn. I'm in favor to add this option directly in `check_array`.

Thanks for the comment @jeremiedbb! I am happy to work on a PR during the next few days.

Labels: New Feature
In gaussian_process/kernels.py, the Tanimoto kernel would be welcome

### Describe the workflow you want to enable

Here is a formula:
x*y / (||x||^2 + ||y||^2 - x*Y)


### Describe your proposed solution

In the context of Gaussian Process Regression, maybe this should be multiplied by the variance,
so the formula becomes:
v * (x*y / (||x||^2 + ||y||^2 - x*Y))

### Describe alternatives you've considered, if relevant

Implement a new kernel myself, but since implementation of a kernel requires much more than just
a K method (evaluate the kernel), I find this way too dangerous.


### Additional context

_No response_

COMMENT:
Thanks for the report. But from your description it's not clear to me what you're intending to change. It seems to me writing a custom kernel is exactly designed for these cases.

So closing as out of scope, but happy to reopen if more context is given.

Labels: New Feature, Needs Triage
In gaussian_process/kernels.py, the Tanimoto kernel would be welcome

### Describe the workflow you want to enable

Here is a formula:
x*y / (||x||^2 + ||y||^2 - x*Y)


### Describe your proposed solution

In the context of Gaussian Process Regression, maybe this should be multiplied by the variance,
so the formula becomes:
v * (x*y / (||x||^2 + ||y||^2 - x*Y))

### Describe alternatives you've considered, if relevant

Implement a new kernel myself, but since implementation of a kernel requires much more than just
a K method (evaluate the kernel), I find this way too dangerous.


### Additional context

_No response_

COMMENT:
Could a scikit-learn hacker provide an implementation of the Tanimoto kernel?
Since scikit-learn's kernel template requires much more than just implementing the K method (i.e. score a pair of instances), this is super annoying for potential users who are not numerical maths researchers familiar w/ the scikit-learn codebase.

Labels: New Feature, Needs Triage
In gaussian_process/kernels.py, the Tanimoto kernel would be welcome

### Describe the workflow you want to enable

Here is a formula:
x*y / (||x||^2 + ||y||^2 - x*Y)


### Describe your proposed solution

In the context of Gaussian Process Regression, maybe this should be multiplied by the variance,
so the formula becomes:
v * (x*y / (||x||^2 + ||y||^2 - x*Y))

### Describe alternatives you've considered, if relevant

Implement a new kernel myself, but since implementation of a kernel requires much more than just
a K method (evaluate the kernel), I find this way too dangerous.


### Additional context

_No response_

COMMENT:
The Tanimoto kernel is for instances which are bitstrings.
The more general version of this kernel is called the Jaccard index (cf. https://en.wikipedia.org/wiki/Jaccard_index).
The Jaccard index can handle bitstrings but also vectors of positive integers, so it might be potentially useful to more scikit-learn users.

Labels: New Feature, Needs Triage
In gaussian_process/kernels.py, the Tanimoto kernel would be welcome

### Describe the workflow you want to enable

Here is a formula:
x*y / (||x||^2 + ||y||^2 - x*Y)


### Describe your proposed solution

In the context of Gaussian Process Regression, maybe this should be multiplied by the variance,
so the formula becomes:
v * (x*y / (||x||^2 + ||y||^2 - x*Y))

### Describe alternatives you've considered, if relevant

Implement a new kernel myself, but since implementation of a kernel requires much more than just
a K method (evaluate the kernel), I find this way too dangerous.


### Additional context

_No response_

COMMENT:
Please reopen this issue.

Labels: New Feature, Needs Triage
In gaussian_process/kernels.py, the Tanimoto kernel would be welcome

### Describe the workflow you want to enable

Here is a formula:
x*y / (||x||^2 + ||y||^2 - x*Y)


### Describe your proposed solution

In the context of Gaussian Process Regression, maybe this should be multiplied by the variance,
so the formula becomes:
v * (x*y / (||x||^2 + ||y||^2 - x*Y))

### Describe alternatives you've considered, if relevant

Implement a new kernel myself, but since implementation of a kernel requires much more than just
a K method (evaluate the kernel), I find this way too dangerous.


### Additional context

_No response_

COMMENT:
There are some interesting kernels implemented in there:
https://github.com/AustinT/sklearn_tanimoto_gp/blob/main/sklearn_tanimoto_gp/kernels.py

Labels: New Feature, Needs Triage
pruning trees

### Describe the workflow you want to enable

I would like a more general `prune_tree` function which would allow the user to specify criteria on pruning a DecisionTree _posthoc_, i.e. without refitting it.
Criteria could be minimum leaf samples per class, min variance, etc...
Ideally, this function could be applied to each tree in a random forest as well.

### Describe your proposed solution

A recursive function that works its way up from the leaves and prunes the tree.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
Is this related to #18680?

Do you have a more concrete example of something that you would like to implement that is not possible right now?

Labels: New Feature, module:tree, cython
pruning trees

### Describe the workflow you want to enable

I would like a more general `prune_tree` function which would allow the user to specify criteria on pruning a DecisionTree _posthoc_, i.e. without refitting it.
Criteria could be minimum leaf samples per class, min variance, etc...
Ideally, this function could be applied to each tree in a random forest as well.

### Describe your proposed solution

A recursive function that works its way up from the leaves and prunes the tree.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
A little bit embarrassing and thanks for reminding me of my own post ! (I had searched github issues for pruning trees and did not find that one).
And no, my concern is still identical to the one from 4 years ago: I am looking for a flexible customizable function to prune trees.
Thanks,

Labels: New Feature, module:tree, cython
I want to import kerasregressor I have tried this from scikeras.wrappers import KerasClassifier , but I am getting the error  cannot import name '_deprecate_Xt_in_inverse_transform' from 'sklearn.utils.deprecation' (C:\Users\hp\AppData\Local\Programs\Python\Python311\Lib\site-packages\sklearn\utils\deprecation.py)

### Describe the bug

I want to import kerasregressor I have tried this from scikeras.wrappers import KerasClassifier , but I am getting the error  cannot import name '_deprecate_Xt_in_inverse_transform' from 'sklearn.utils.deprecation' (C:\Users\hp\AppData\Local\Programs\Python\Python311\Lib\site-packages\sklearn\utils\deprecation.py)

### Steps/Code to Reproduce

I want to import kerasregressor I have tried this from scikeras.wrappers import KerasClassifier , but I am getting the error  cannot import name '_deprecate_Xt_in_inverse_transform' from 'sklearn.utils.deprecation' (C:\Users\hp\AppData\Local\Programs\Python\Python311\Lib\site-packages\sklearn\utils\deprecation.py)

### Expected Results

I want to import kerasregressor I have tried this from scikeras.wrappers import KerasClassifier , but I am getting the error  cannot import name '_deprecate_Xt_in_inverse_transform' from 'sklearn.utils.deprecation' (C:\Users\hp\AppData\Local\Programs\Python\Python311\Lib\site-packages\sklearn\utils\deprecation.py)

### Actual Results

I want to import kerasregressor I have tried this from scikeras.wrappers import KerasClassifier , but I am getting the error  cannot import name '_deprecate_Xt_in_inverse_transform' from 'sklearn.utils.deprecation' (C:\Users\hp\AppData\Local\Programs\Python\Python311\Lib\site-packages\sklearn\utils\deprecation.py)

### Versions

```shell
I want to import kerasregressor I have tried this from scikeras.wrappers import KerasClassifier , but I am getting the error  cannot import name '_deprecate_Xt_in_inverse_transform' from 'sklearn.utils.deprecation' (C:\Users\hp\AppData\Local\Programs\Python\Python311\Lib\site-packages\sklearn\utils\deprecation.py)
```

COMMENT:
Please file an issue with keras, they need to fix this on their side.

Labels: Bug, Needs Triage