Talip7/scikit-learn-issues-multilabel huggingface dataset:
UnboundTransform implementing log and logit transforms

### Describe the workflow you want to enable

Most classifiers and regressors expected unbounded input. Bounded input typically comes in the forms (a, infty) and (a, b) with the important special cases (0, infty) for radii, counts, and other things that are always non-negative, and (0, 1) for probabilities and fractions. One needs a way to transform bounded data to the unbounded domain that works with all classifiers and regressors.

### Describe your proposed solution

A new transform should be implemented to convert bounded domains into the unbounded domain, tentatively called `UnboundTransform`. The transform can optionally be chained by a PowerTransform to make the output more gaussian. The `UnboundTransform` will apply scale and shift options and the log transform or the logit transform to convert half-bounded and fully bounded data, respectively.

Because bounds are difficult to estimate from a distribution, the user should pass the bounds on construction.

### Describe alternatives you've considered, if relevant

* The Yeo-Johnson power transform does not make a bounded variable unbounded.
* The QuantileTransform overfits on small samples and its tail behavior is unpredictable. It is also computationally expensive.
* The FunctionTransform is a generic solution and able to perform this task, but the user has to provide the transforms. Since making data unbounded is such a common thing, having a specialized transform for this case is justified, to avoid letting users reinvert the wheel many times.

### Additional context

I am interested in implementing the transforms and submitting a PR. I previously contributed to numpy, scipy, and matplotlib, and am the maintainer of several OSS libraries for data analysis here on Github used in (astro)particle physics.

COMMENT:
Hi @HDembinski 

> The FunctionTransform is a generic solution and able to perform this task, but the user has to provide the transforms. Since making data unbounded is such a common thing, having a specialized transform for this case is justified, to avoid letting users reinvert the wheel many times.

I think that the `FunctionTransformer` is perfectly adapted to this task. And has the flexibility to suit most use-cases. There are many ways to transform data from a bounded domain to an unbounded domain. A dedicated transformer would need to allow this flexibility and thus would look a lot like the `FunctionTransformer`.

Labels: New Feature, Needs Decision - Close
UnboundTransform implementing log and logit transforms

### Describe the workflow you want to enable

Most classifiers and regressors expected unbounded input. Bounded input typically comes in the forms (a, infty) and (a, b) with the important special cases (0, infty) for radii, counts, and other things that are always non-negative, and (0, 1) for probabilities and fractions. One needs a way to transform bounded data to the unbounded domain that works with all classifiers and regressors.

### Describe your proposed solution

A new transform should be implemented to convert bounded domains into the unbounded domain, tentatively called `UnboundTransform`. The transform can optionally be chained by a PowerTransform to make the output more gaussian. The `UnboundTransform` will apply scale and shift options and the log transform or the logit transform to convert half-bounded and fully bounded data, respectively.

Because bounds are difficult to estimate from a distribution, the user should pass the bounds on construction.

### Describe alternatives you've considered, if relevant

* The Yeo-Johnson power transform does not make a bounded variable unbounded.
* The QuantileTransform overfits on small samples and its tail behavior is unpredictable. It is also computationally expensive.
* The FunctionTransform is a generic solution and able to perform this task, but the user has to provide the transforms. Since making data unbounded is such a common thing, having a specialized transform for this case is justified, to avoid letting users reinvert the wheel many times.

### Additional context

I am interested in implementing the transforms and submitting a PR. I previously contributed to numpy, scipy, and matplotlib, and am the maintainer of several OSS libraries for data analysis here on Github used in (astro)particle physics.

COMMENT:
That is poor decision making on your part. There are not many ways, there are two established standard ways, the ones I mentioned. I assumed that scikit learn cares about the UX of their users, given that this is one of the few advantages it has over other libraries.

Labels: New Feature, Needs Decision - Close
UnboundTransform implementing log and logit transforms

### Describe the workflow you want to enable

Most classifiers and regressors expected unbounded input. Bounded input typically comes in the forms (a, infty) and (a, b) with the important special cases (0, infty) for radii, counts, and other things that are always non-negative, and (0, 1) for probabilities and fractions. One needs a way to transform bounded data to the unbounded domain that works with all classifiers and regressors.

### Describe your proposed solution

A new transform should be implemented to convert bounded domains into the unbounded domain, tentatively called `UnboundTransform`. The transform can optionally be chained by a PowerTransform to make the output more gaussian. The `UnboundTransform` will apply scale and shift options and the log transform or the logit transform to convert half-bounded and fully bounded data, respectively.

Because bounds are difficult to estimate from a distribution, the user should pass the bounds on construction.

### Describe alternatives you've considered, if relevant

* The Yeo-Johnson power transform does not make a bounded variable unbounded.
* The QuantileTransform overfits on small samples and its tail behavior is unpredictable. It is also computationally expensive.
* The FunctionTransform is a generic solution and able to perform this task, but the user has to provide the transforms. Since making data unbounded is such a common thing, having a specialized transform for this case is justified, to avoid letting users reinvert the wheel many times.

### Additional context

I am interested in implementing the transforms and submitting a PR. I previously contributed to numpy, scipy, and matplotlib, and am the maintainer of several OSS libraries for data analysis here on Github used in (astro)particle physics.

COMMENT:
First of all, please use a less aggressive language. Then, it's not decision making, just my own opinion. Nothing is decided yet. Others may think otherwise and decide to include the feature.

Labels: New Feature, Needs Decision - Close
UnboundTransform implementing log and logit transforms

### Describe the workflow you want to enable

Most classifiers and regressors expected unbounded input. Bounded input typically comes in the forms (a, infty) and (a, b) with the important special cases (0, infty) for radii, counts, and other things that are always non-negative, and (0, 1) for probabilities and fractions. One needs a way to transform bounded data to the unbounded domain that works with all classifiers and regressors.

### Describe your proposed solution

A new transform should be implemented to convert bounded domains into the unbounded domain, tentatively called `UnboundTransform`. The transform can optionally be chained by a PowerTransform to make the output more gaussian. The `UnboundTransform` will apply scale and shift options and the log transform or the logit transform to convert half-bounded and fully bounded data, respectively.

Because bounds are difficult to estimate from a distribution, the user should pass the bounds on construction.

### Describe alternatives you've considered, if relevant

* The Yeo-Johnson power transform does not make a bounded variable unbounded.
* The QuantileTransform overfits on small samples and its tail behavior is unpredictable. It is also computationally expensive.
* The FunctionTransform is a generic solution and able to perform this task, but the user has to provide the transforms. Since making data unbounded is such a common thing, having a specialized transform for this case is justified, to avoid letting users reinvert the wheel many times.

### Additional context

I am interested in implementing the transforms and submitting a PR. I previously contributed to numpy, scipy, and matplotlib, and am the maintainer of several OSS libraries for data analysis here on Github used in (astro)particle physics.

COMMENT:
There are too many options, and a lot of the 'transforms' are application dependent. In fact, any CDF of a distribution over your unbounded domain is a valid `transform`. And there are plenty of these well-known parametric distributions on $[0, \infty)$ or $(-\infty, \infty)$. Using a distribution should 'uniformize' your transformed values if the true data is distributed in a manner similar to the chosen distribution. But it doesn't mean it's the best choice for _fitting a model_.

So even if we restrict ourselves just to parametric CDFs of well-known distributions - there are too many options to choose from, and no easy guideline for how to choose them. I don't think that scikit-learn is the right place to add those. I even believe that a dedicated small library, that is `pip install`able, is a much better option.

Labels: New Feature, Needs Decision - Close
Documenting return array types

Since we are introducing Array API compatibility we are discussing that some functions (especially in the metrics section) would not return the input array type, but a numpy array. 

How would we document that, so that users know what they get as a return type?

We have started to discuss this [here](https://github.com/scikit-learn/scikit-learn/pull/30440#discussion_r1897937388), [here](https://github.com/scikit-learn/scikit-learn/pull/30562#issuecomment-2565380749) and [here](https://github.com/scikit-learn/scikit-learn/pull/30562#pullrequestreview-2537038557) (and possibly in other places), but this discussion a bit scattered and in this issue I am trying to bring this together.

I would think we need to find a standard way of how to talk about **return types in the docstrings**.

- use the terms `ndarray` and `array` (or something more eye-catching) for the input array type
- from the docstring, link to a dedicated section in the glossary, explaining the differences between `ndarray` and `array` and which are the implications for the users

What are the general feelings about that?
@ogrisel @OmarManzoor @adrinjalali (I don't want to bother you by tagging, but it would be interesting to hear the takes of betatim, thomasjpfan, lesteve and jeremiedbb as well if they are interested :sweat_smile:)

COMMENT:
Personally, I haven't fully understood why scikit-learn maintainers prefer to keep numpy return types for functions with small array outputs. 

It doesn't seem computationally costly to convert small arrays into the input array type, whereas returning them as numpy arrays complicates the matter. 

This is because we would need to discuss and decide on array return types individually, which could potentially lead to confusion for users, especially if they miss parts of the documentation where we explain this.

**This is costly in a different way:**

- It slows down the overall Array API transition.
- It will annoy users.
- It could keep us busy with clarifications and decisions on return types and then again to maintain this.

I am sharing my thoughts here primarily to get some direction from my senior colleagues on how to pick and document the return type in #30562, rather than to start a broader discussion.

Labels: Documentation, RFC, Array API
Documenting return array types

Since we are introducing Array API compatibility we are discussing that some functions (especially in the metrics section) would not return the input array type, but a numpy array. 

How would we document that, so that users know what they get as a return type?

We have started to discuss this [here](https://github.com/scikit-learn/scikit-learn/pull/30440#discussion_r1897937388), [here](https://github.com/scikit-learn/scikit-learn/pull/30562#issuecomment-2565380749) and [here](https://github.com/scikit-learn/scikit-learn/pull/30562#pullrequestreview-2537038557) (and possibly in other places), but this discussion a bit scattered and in this issue I am trying to bring this together.

I would think we need to find a standard way of how to talk about **return types in the docstrings**.

- use the terms `ndarray` and `array` (or something more eye-catching) for the input array type
- from the docstring, link to a dedicated section in the glossary, explaining the differences between `ndarray` and `array` and which are the implications for the users

What are the general feelings about that?
@ogrisel @OmarManzoor @adrinjalali (I don't want to bother you by tagging, but it would be interesting to hear the takes of betatim, thomasjpfan, lesteve and jeremiedbb as well if they are interested :sweat_smile:)

COMMENT:
If this is with reference to the documentation part, the return types in the doc strings are retained intentionally, considering that the Array api is experimental. Please refer to this set of comments https://github.com/scikit-learn/scikit-learn/pull/27113#discussion_r1604511376

Labels: Documentation, RFC, Array API
Documenting return array types

Since we are introducing Array API compatibility we are discussing that some functions (especially in the metrics section) would not return the input array type, but a numpy array. 

How would we document that, so that users know what they get as a return type?

We have started to discuss this [here](https://github.com/scikit-learn/scikit-learn/pull/30440#discussion_r1897937388), [here](https://github.com/scikit-learn/scikit-learn/pull/30562#issuecomment-2565380749) and [here](https://github.com/scikit-learn/scikit-learn/pull/30562#pullrequestreview-2537038557) (and possibly in other places), but this discussion a bit scattered and in this issue I am trying to bring this together.

I would think we need to find a standard way of how to talk about **return types in the docstrings**.

- use the terms `ndarray` and `array` (or something more eye-catching) for the input array type
- from the docstring, link to a dedicated section in the glossary, explaining the differences between `ndarray` and `array` and which are the implications for the users

What are the general feelings about that?
@ogrisel @OmarManzoor @adrinjalali (I don't want to bother you by tagging, but it would be interesting to hear the takes of betatim, thomasjpfan, lesteve and jeremiedbb as well if they are interested :sweat_smile:)

COMMENT:
Thanks for the link, @OmarManzoor. It makes sense to not touch the docstrings while Array API is still experimental.

Labels: Documentation, RFC, Array API
scikit-learn 1.6: Elliptic Envelope Fails with More Features than Samples

### Describe the bug

When using the EllipticEnvelope class in scikit-learn 1.6, the model raises an error when the number of features exceeds the number of samples in the input dataset. This issue occurs even when the data is preprocessed (e.g., scaled with StandardScaler) and is independent of the contamination or support fraction settings.

This behavior differs from previous versions of scikit-learn, where the EllipticEnvelope was able to handle cases with more features than samples without raising errors.

### Steps/Code to Reproduce

```python
import numpy as np
from sklearn.covariance import EllipticEnvelope
from sklearn.preprocessing import StandardScaler

# Generate data with more features than samples
X = np.random.rand(5, 10)  # 5 samples, 10 features

# Preprocess the data
scaler = StandardScaler()
X_scaled = scaler.fit_transform(X)

# Initialize Elliptic Envelope
model = EllipticEnvelope(contamination=0.1)

# Attempt to fit the model
try:
    model.fit(X_scaled)
    print("EllipticEnvelope successfully fitted.")
except Exception as e:
    print("Error encountered:", e)
```


### Expected Results

Results with scikit-learn version < 1.6
```python
UserWarning: The covariance matrix associated to your dataset is not full rank
  warnings.warn(
EllipticEnvelope successfully fitted.
```


### Actual Results

Results with scikit-learn version >= 1.6
```python
UserWarning: The covariance matrix associated to your dataset is not full rank
  warnings.warn(
Error encountered: kth(=7) out of bounds (5)
```

### Versions

```shell
Versions

System:
    python: 3.10.11 (tags/v3.10.11:7d4cc5a, Apr  5 2023, 00:38:17) [MSC v.1929 64 bit (AMD64)]
    executable: <redacted>
    machine: Windows-10-10.0.19045-SP0

Python dependencies:
      sklearn: 1.6.0
          pip: 24.3.1
   setuptools: 75.6.0
        numpy: 2.0.2
        scipy: 1.13.1
       Cython: None
       pandas: 2.2.3
   matplotlib: 3.9.2
       joblib: 1.4.2
threadpoolctl: 3.5.0

Built with OpenMP: True

threadpoolctl info:
       user_api: openmp
   internal_api: openmp
    num_threads: 4
         prefix: vcomp
       filepath: <redacted>
        version: None

       user_api: blas
   internal_api: openblas
    num_threads: 4
         prefix: libscipy_openblas
       filepath: <redacted>
        version: 0.3.27
threading_layer: pthreads
   architecture: Zen

       user_api: blas
   internal_api: openblas
    num_threads: 4
         prefix: libopenblas
       filepath: <redacted>
        version: 0.3.27
threading_layer: pthreads
   architecture: Zen
```

COMMENT:
#30483 should fix this, I think.

Labels: Bug, Regression
Bad color choice in Prediction Intervals for Gradient Boosting Regression

### Describe the issue linked to the documentation

The first plot in the example [Prediction Intervals for Gradient Boosting Regression](https://scikit-learn.org/stable/auto_examples/ensemble/plot_gradient_boosting_quantile.html#fitting-non-linear-quantile-and-least-squares-regressors) is plotting mean and median using orange and red (I think? maybe it's even the exact same color) which makes it near-impossible to differentiate, at least on my device.

![sphx_glr_plot_gradient_boosting_quantile_001](https://github.com/user-attachments/assets/4b4fbbbe-e042-46ac-bd54-895d72c3d9a1)

### Suggest a potential alternative/fix

EDIT: As Julian points out, the color actually is being set explicitly, my bad.

In the code, no color is being set explicitly (the relevant lines are 109 and 110 [here](https://github.com/scikit-learn/scikit-learn/blob/main/examples/ensemble/plot_gradient_boosting_quantile.py#L109)).
Honestly, I think explicitly setting more or less any other color for either the mean or the median would work better, like purple or even yellow (due to the blue shading in the background).

COMMENT:
@MischaDy it seems that both [Line 109]( https://github.com/scikit-learn/scikit-learn/blob/5691f2672a4d5bc0ce36629aec58d8a4076e5e99/examples/ensemble/plot_gradient_boosting_quantile.py#L109) and [Line 110](https://github.com/scikit-learn/scikit-learn/blob/5691f2672a4d5bc0ce36629aec58d8a4076e5e99/examples/ensemble/plot_gradient_boosting_quantile.py#L110) explicitly set both the plots for the predicted mean and median to red. Yes, I do agree with @MischaDy that literally any other colour would be useful to distinguish them here.

Labels: Documentation, Needs Triage
Bad color choice in Prediction Intervals for Gradient Boosting Regression

### Describe the issue linked to the documentation

The first plot in the example [Prediction Intervals for Gradient Boosting Regression](https://scikit-learn.org/stable/auto_examples/ensemble/plot_gradient_boosting_quantile.html#fitting-non-linear-quantile-and-least-squares-regressors) is plotting mean and median using orange and red (I think? maybe it's even the exact same color) which makes it near-impossible to differentiate, at least on my device.

![sphx_glr_plot_gradient_boosting_quantile_001](https://github.com/user-attachments/assets/4b4fbbbe-e042-46ac-bd54-895d72c3d9a1)

### Suggest a potential alternative/fix

EDIT: As Julian points out, the color actually is being set explicitly, my bad.

In the code, no color is being set explicitly (the relevant lines are 109 and 110 [here](https://github.com/scikit-learn/scikit-learn/blob/main/examples/ensemble/plot_gradient_boosting_quantile.py#L109)).
Honestly, I think explicitly setting more or less any other color for either the mean or the median would work better, like purple or even yellow (due to the blue shading in the background).

COMMENT:
I'd like to work on this issue. I plan to change one of the line colors to make the mean and median lines more distinguishable.

Labels: Documentation, Needs Triage
Validate estimators argument of VotingClassifier

### Describe the workflow you want to enable

`VotingClassifier` takes as input `estimators`, which is expected to be `list of (str, estimator) tuples`. 

However, if one accidentially puts in a list of estimators instead of a list of `tuples(str, estim)` or a single estimator, no warning/exception is thrown and one only finds out during runtime/fitting with the obscure error 
```
AttributeError: 'RandomForestClassifier' object has no attribute 'estimators_'` 
or 
`AttributeError: 'RandomForestClassifier' is not iterable'
```

As seen in some stackoverflow questions, this problem seems to occur to users other than me as well, e.g.

https://stackoverflow.com/questions/47291590/fitting-votingclassifier

https://stackoverflow.com/questions/74461779/sklearn-votingclassifier-is-throwing-an-issue-about-argument-not-being-iterable

### Describe your proposed solution

Implement validation of the estimators argument within the `__init__`, e.g., 

```
# check that estimators is a list or tuple
if not instance(estimators, (tuple, list)):
    .,..
for item in estimators:
    if not isinstance(item, (tuple, list)) or not len(item)==2:
           # error: need str/classifier
    assert isinstance(item[0], str)
    assert isinstance(item[1], BaseEstimator)
```

### Describe alternatives you've considered, if relevant

Alternatively, we could also improve the documentation to highlight this case.

### Additional context

_No response_

COMMENT:
Thank you for the issue! Scikit-learn's estimators perform validation is in `fit`, so we can validate there. Note, that we can not really check for `BaseEstimator` because we support duck typing. i.e. A third-party estimator implementation is not required to subclass `BaseEstimator`.

I opened https://github.com/scikit-learn/scikit-learn/pull/30649 to fix the issue.

Labels: New Feature