Talip7/scikit-learn-issues-multilabel huggingface dataset:
HuberRegressor failed with ABNORMAL_TERMINATION_IN_LNSRCH on simple dataset

### Describe the bug

sklearn linear_model HuberRegressor fails with a very simple dataset. 

The code below fails with ValueError: HuberRegressor convergence failed: l-BFGS-b solver terminated with ABNORMAL_TERMINATION_IN_LNSRCH



### Steps/Code to Reproduce

```python
import numpy as np
import sklearn
from sklearn.linear_model import HuberRegressor

X = np.array([10, 11, 28]).reshape(-1, 1)
y = np.log(np.array([5000., 5000., 5000.]))
sample_weights = np.array([2., 2., 2.])
h = HuberRegressor().fit(X, y, sample_weights)
```

### Expected Results

no ValueError thrown

### Actual Results

```pytb
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[2], line 10
      8 y = np.log(np.array([5000., 5000., 5000.]))
      9 sample_weights = np.array([2., 2., 2.])
---> 10 h = HuberRegressor().fit(X, y, sample_weights)

File ~\Anaconda3\lib\site-packages\sklearn\base.py:1151, in _fit_context.<locals>.decorator.<locals>.wrapper(estimator, *args, **kwargs)
   1144     estimator._validate_params()
   1146 with config_context(
   1147     skip_parameter_validation=(
   1148         prefer_skip_nested_validation or global_skip_validation
   1149     )
   1150 ):
-> 1151     return fit_method(estimator, *args, **kwargs)

File ~\Anaconda3\lib\site-packages\sklearn\linear_model\_huber.py:338, in HuberRegressor.fit(self, X, y, sample_weight)
    335 parameters = opt_res.x
    337 if opt_res.status == 2:
--> 338     raise ValueError(
    339         "HuberRegressor convergence failed: l-BFGS-b solver terminated with %s"
    340         % opt_res.message
    341     )
    342 self.n_iter_ = _check_optimize_result("lbfgs", opt_res, self.max_iter)
    343 self.scale_ = parameters[-1]

ValueError: HuberRegressor convergence failed: l-BFGS-b solver terminated with ABNORMAL_TERMINATION_IN_LNSRCH
```

### Versions

```shell
System:
    python: 3.9.18 (main, Sep 11 2023, 13:30:38) [MSC v.1916 64 bit (AMD64)]
executable: C:\Users\[username]\Anaconda3\python.exe
   machine: Windows-10-10.0.19044-SP0

Python dependencies:
      sklearn: 1.3.0
          pip: 23.3
   setuptools: 68.0.0
        numpy: 1.24.3
        scipy: 1.11.3
       Cython: None
       pandas: 2.1.1
   matplotlib: 3.8.0
       joblib: 1.2.0
threadpoolctl: 2.2.0

Built with OpenMP: True

threadpoolctl info:
       filepath: C:\Users\[username]\Anaconda3\Library\bin\mkl_rt.2.dll
         prefix: mkl_rt
       user_api: blas
   internal_api: mkl
        version: 2023.1-Product
    num_threads: 4
threading_layer: intel

       filepath: C:\Users\[username]\Anaconda3\vcomp140.dll
         prefix: vcomp
       user_api: openmp
   internal_api: openmp
        version: None
    num_threads: 8
```

COMMENT:
> Are there are other instances of such a warning in sklearn so that the warning could be adapted?

It looks that we raise such warning when it happens in `LogisticRegression`.

Labels: help wanted
DOC: incorrect rendering for d(\\cdot, \\cdot)

### Describe the issue linked to the documentation

Documentation for [ExpSineSquared gaussian process kernel](https://scikit-learn.org/dev/modules/generated/sklearn.gaussian_process.kernels.ExpSineSquared.html) displays d(\\cdot, \\cdot) as d(cdot, cdot)

![Screenshot 2023-11-13 08 59 48](https://github.com/scikit-learn/scikit-learn/assets/23830955/d74866e2-5da0-40cb-9623-2d8192fddbfc)

however, it is supposed to be rendered as d(., .) , see e.g. [RBF kernel](https://scikit-learn.org/dev/modules/generated/sklearn.gaussian_process.kernels.RBF.html).

![Screenshot 2023-11-13 09 01 22](https://github.com/scikit-learn/scikit-learn/assets/23830955/6e9701cf-f58a-4892-8b2b-7e542e92f1d5)

both stable and dev versions of the documentation have this bug


### Suggest a potential alternative/fix

_No response_

COMMENT:
Surprising but I think that we need to change `\\cdot` by `\cdot` in the docstring.
@partev do you wish to make a pull request to fix this issue?

Labels: Documentation
DOC: incorrect rendering for d(\\cdot, \\cdot)

### Describe the issue linked to the documentation

Documentation for [ExpSineSquared gaussian process kernel](https://scikit-learn.org/dev/modules/generated/sklearn.gaussian_process.kernels.ExpSineSquared.html) displays d(\\cdot, \\cdot) as d(cdot, cdot)

![Screenshot 2023-11-13 08 59 48](https://github.com/scikit-learn/scikit-learn/assets/23830955/d74866e2-5da0-40cb-9623-2d8192fddbfc)

however, it is supposed to be rendered as d(., .) , see e.g. [RBF kernel](https://scikit-learn.org/dev/modules/generated/sklearn.gaussian_process.kernels.RBF.html).

![Screenshot 2023-11-13 09 01 22](https://github.com/scikit-learn/scikit-learn/assets/23830955/6e9701cf-f58a-4892-8b2b-7e542e92f1d5)

both stable and dev versions of the documentation have this bug


### Suggest a potential alternative/fix

_No response_

COMMENT:
it is caused by using a raw string in documentation:
r"""Exp-Sine-Squared kernel (aka periodic kernel).

so there are two ways to fix it remove the raw string or \\cdot -> \cdot, which one do you think is more appropriate?

Labels: Documentation
DOC: incorrect rendering for d(\\cdot, \\cdot)

### Describe the issue linked to the documentation

Documentation for [ExpSineSquared gaussian process kernel](https://scikit-learn.org/dev/modules/generated/sklearn.gaussian_process.kernels.ExpSineSquared.html) displays d(\\cdot, \\cdot) as d(cdot, cdot)

![Screenshot 2023-11-13 08 59 48](https://github.com/scikit-learn/scikit-learn/assets/23830955/d74866e2-5da0-40cb-9623-2d8192fddbfc)

however, it is supposed to be rendered as d(., .) , see e.g. [RBF kernel](https://scikit-learn.org/dev/modules/generated/sklearn.gaussian_process.kernels.RBF.html).

![Screenshot 2023-11-13 09 01 22](https://github.com/scikit-learn/scikit-learn/assets/23830955/6e9701cf-f58a-4892-8b2b-7e542e92f1d5)

both stable and dev versions of the documentation have this bug


### Suggest a potential alternative/fix

_No response_

COMMENT:
As mentioned before, a single `\` is the way to go.

Labels: Documentation
Is the MSE Criterion of DecisionTreeRegressor right ?

### Discussed in https://github.com/scikit-learn/scikit-learn/discussions/27765

<div type='discussions-op-text'>

<sup>Originally posted by **IceCapriccio** November 13, 2023</sup>
```py
    cdef int init(
        self,
        const DOUBLE_t[:, ::1] y,
        const DOUBLE_t[:] sample_weight,
        double weighted_n_samples,
        const SIZE_t[:] sample_indices,
        SIZE_t start,
        SIZE_t end,
    ) except -1 nogil:
        """Initialize the criterion.

        This initializes the criterion at node sample_indices[start:end] and children
        sample_indices[start:start] and sample_indices[start:end].
        """
        # Initialize fields
        self.y = y
        self.sample_weight = sample_weight
        self.sample_indices = sample_indices
        self.start = start
        self.end = end
        self.n_node_samples = end - start
        self.weighted_n_samples = weighted_n_samples
        self.weighted_n_node_samples = 0.

        cdef SIZE_t i
        cdef SIZE_t p
        cdef SIZE_t k
        cdef DOUBLE_t y_ik
        cdef DOUBLE_t w_y_ik
        cdef DOUBLE_t w = 1.0
        self.sq_sum_total = 0.0
        memset(&self.sum_total[0], 0, self.n_outputs * sizeof(double))

        for p in range(start, end):
            i = sample_indices[p]

            if sample_weight is not None:
                w = sample_weight[i]

            for k in range(self.n_outputs):
                y_ik = self.y[i, k]
                w_y_ik = w * y_ik
                self.sum_total[k] += w_y_ik
                self.sq_sum_total += w_y_ik * y_ik

            self.weighted_n_node_samples += w

        # Reset to pos=start
        self.reset()
        return 0
```
These source code is [https://github.com/scikit-learn/scikit-learn/blob/main/sklearn/tree/_criterion.pyx](https://github.com/scikit-learn/scikit-learn/blob/main/sklearn/tree/_criterion.pyx) start at line 860
`self.sq_sum_total` is calculated by `w_y_ik * y_ik`, without square.
 But in MSE bellow, `self.sq_sum_total` is used to calculate MSE:
```py
    cdef double node_impurity(self) noexcept nogil:
        """Evaluate the impurity of the current node.

        Evaluate the MSE criterion as impurity of the current node,
        i.e. the impurity of sample_indices[start:end]. The smaller the impurity the
        better.
        """
        cdef double impurity
        cdef SIZE_t k

        impurity = self.sq_sum_total / self.weighted_n_node_samples
        for k in range(self.n_outputs):
            impurity -= (self.sum_total[k] / self.weighted_n_node_samples)**2.0

        return impurity / self.n_outputs
```
So is the node impurity calculated in  MSE right?</div>

COMMENT:
> self.sq_sum_total is calculated by w_y_ik * y_ik, without square

What so you mean `y_ik` is multiply with itself so it is squared.

The place that we use `**2`, the reason is that we use the sum that is not squared so we need to additionally square it.

Labels: Needs Triage
Is the MSE Criterion of DecisionTreeRegressor right ?

### Discussed in https://github.com/scikit-learn/scikit-learn/discussions/27765

<div type='discussions-op-text'>

<sup>Originally posted by **IceCapriccio** November 13, 2023</sup>
```py
    cdef int init(
        self,
        const DOUBLE_t[:, ::1] y,
        const DOUBLE_t[:] sample_weight,
        double weighted_n_samples,
        const SIZE_t[:] sample_indices,
        SIZE_t start,
        SIZE_t end,
    ) except -1 nogil:
        """Initialize the criterion.

        This initializes the criterion at node sample_indices[start:end] and children
        sample_indices[start:start] and sample_indices[start:end].
        """
        # Initialize fields
        self.y = y
        self.sample_weight = sample_weight
        self.sample_indices = sample_indices
        self.start = start
        self.end = end
        self.n_node_samples = end - start
        self.weighted_n_samples = weighted_n_samples
        self.weighted_n_node_samples = 0.

        cdef SIZE_t i
        cdef SIZE_t p
        cdef SIZE_t k
        cdef DOUBLE_t y_ik
        cdef DOUBLE_t w_y_ik
        cdef DOUBLE_t w = 1.0
        self.sq_sum_total = 0.0
        memset(&self.sum_total[0], 0, self.n_outputs * sizeof(double))

        for p in range(start, end):
            i = sample_indices[p]

            if sample_weight is not None:
                w = sample_weight[i]

            for k in range(self.n_outputs):
                y_ik = self.y[i, k]
                w_y_ik = w * y_ik
                self.sum_total[k] += w_y_ik
                self.sq_sum_total += w_y_ik * y_ik

            self.weighted_n_node_samples += w

        # Reset to pos=start
        self.reset()
        return 0
```
These source code is [https://github.com/scikit-learn/scikit-learn/blob/main/sklearn/tree/_criterion.pyx](https://github.com/scikit-learn/scikit-learn/blob/main/sklearn/tree/_criterion.pyx) start at line 860
`self.sq_sum_total` is calculated by `w_y_ik * y_ik`, without square.
 But in MSE bellow, `self.sq_sum_total` is used to calculate MSE:
```py
    cdef double node_impurity(self) noexcept nogil:
        """Evaluate the impurity of the current node.

        Evaluate the MSE criterion as impurity of the current node,
        i.e. the impurity of sample_indices[start:end]. The smaller the impurity the
        better.
        """
        cdef double impurity
        cdef SIZE_t k

        impurity = self.sq_sum_total / self.weighted_n_node_samples
        for k in range(self.n_outputs):
            impurity -= (self.sum_total[k] / self.weighted_n_node_samples)**2.0

        return impurity / self.n_outputs
```
So is the node impurity calculated in  MSE right?</div>

COMMENT:
```python
w_y_ik = w * y_ik
self.sq_sum_total += w_y_ik * y_ik
```
So this is correct. BTW, you can also have a look at the tests to be even more sure about correctnes.

Labels: Needs Triage
conda.core.link:_execute(945): An error occurred while installing package 'defaults::scikit-learn-1.2.0-py39hd77b12b_0'

![d34ea4783a1551919fb2058dab2cc29](https://github.com/scikit-learn/scikit-learn/assets/143175589/fa0f38f1-80b7-457d-8728-b2d0fc5db930)
Does anyone know how to deal with this error? I’m using the Windows system, miniconda. I’ve tried all versions, but the error is the same.

COMMENT:
Could you try to install from the conda-forge channel as documented in the documentation:

https://scikit-learn.org/stable/install.html

Maybe there is something fishy with the version from the main channel provided by Anaconda.

Labels: Needs Triage
conda.core.link:_execute(945): An error occurred while installing package 'defaults::scikit-learn-1.2.0-py39hd77b12b_0'

![d34ea4783a1551919fb2058dab2cc29](https://github.com/scikit-learn/scikit-learn/assets/143175589/fa0f38f1-80b7-457d-8728-b2d0fc5db930)
Does anyone know how to deal with this error? I’m using the Windows system, miniconda. I’ve tried all versions, but the error is the same.

COMMENT:
thank you，i tried `conda create -n sklearn-env -c conda-forge scikit-learn`，and it did created an environment with scikit-learn,But this also leads to many version conflicts with other dependencies, such as TensorFlow.So, for those who encounter the same problem, please be aware. When creating, remember to specify the version of scikit-learn. The default version uses Python 3.12, which is not compatible with TensorFlow.

Labels: Needs Triage
conda.core.link:_execute(945): An error occurred while installing package 'defaults::scikit-learn-1.2.0-py39hd77b12b_0'

![d34ea4783a1551919fb2058dab2cc29](https://github.com/scikit-learn/scikit-learn/assets/143175589/fa0f38f1-80b7-457d-8728-b2d0fc5db930)
Does anyone know how to deal with this error? I’m using the Windows system, miniconda. I’ve tried all versions, but the error is the same.

COMMENT:
Hello, I am also experiencing the same problem, do you have a good solution now?

Labels: Needs Triage
help support

### Describe the bug

i'm using pycaret with scikit-learn but it only uses scikit-learn 1.2.2.
could anyone send a pull-request with changes to support scikit-learn 1.3 ?
https://github.com/pycaret/pycaret/

### Steps/Code to Reproduce

.

### Expected Results

.

### Actual Results

.

### Versions

```shell
1.2.2
```

COMMENT:
I don't think that we have the maintenance workforce to maintain all packages depending on scikit-learn. The thing that we try to do is to ensure that we don't make any breaking changes on the public API that would impact downstream packages.

We currently manually check it and we could have a better system for that.

I am closing the issue while anyone from the community is more than welcome to contribute to `pycaret`. Also if `pycaret` developer observe that we are breaking some public interface that we should not, we will be more than happy to correct it.

Labels: Bug, Needs Triage
Groups in BaggingRegressor

### Describe the workflow you want to enable

It would be nice if we could control how random indices are created with groups in BaggingRegressor. As far as I can tell, BaggingRegressor will select sub samples u.a.r. with/without replacement.

It would be nice if we could group the data points, and have BaggingRegressor select sub samples of (optional) _groups_ u.a.r. with/without replacement.

An example of this paradigm already exists in sklearn with cross validation (e.g. KFold vs. GroupKFold)

### Describe your proposed solution

One idea is to pass in a vector `groups` of size `len(X)` mapping each data point to its group. Then, `_generate_indices` can respect groups if they exist, or default to its existing behavior if they do not.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
If we start to use `groups` information to sample, then we are not taking bootstrap samples and thus this is not anymore a `BaggingRegressor` or `BaggingClassifier`.

I am wondering if we should not instead stick to the theory behind those estimator.

In the case of the `KFold` and other splitter, we don't have such constraint.

@neil-kulkarni are you aware of any literature that are modifying the bagging when making the sampling and how they call these estimators?

Labels: New Feature, Needs Decision
Groups in BaggingRegressor

### Describe the workflow you want to enable

It would be nice if we could control how random indices are created with groups in BaggingRegressor. As far as I can tell, BaggingRegressor will select sub samples u.a.r. with/without replacement.

It would be nice if we could group the data points, and have BaggingRegressor select sub samples of (optional) _groups_ u.a.r. with/without replacement.

An example of this paradigm already exists in sklearn with cross validation (e.g. KFold vs. GroupKFold)

### Describe your proposed solution

One idea is to pass in a vector `groups` of size `len(X)` mapping each data point to its group. Then, `_generate_indices` can respect groups if they exist, or default to its existing behavior if they do not.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
@glemaitre Thanks for getting back to me. While indeed the standard approach to bagging is to sample the data u.a.r., I don’t think it’s a large departure to sample instead groups u.a.r. In particular, I don’t think sampling groups instead of data points would mean that this is no longer a `BaggingRegressor` or `BaggingClassifier`. Open to different opinions on this.

A brief look online suggests that there are a number of studies that suggest alternative methods of sampling when doing bagging:
* Subbagging: https://arxiv.org/abs/2103.00631
* Neighborhood sampling: https://www.sciencedirect.com/science/article/abs/pii/S0925231214012296

The method I propose above is indeed distinct from these articles. However, it gives users a new knob to turn if they have the prior that their data points can be grouped in some meaningful way (e.g. by date on which the data was sampled).

Labels: New Feature, Needs Decision
Groups in BaggingRegressor

### Describe the workflow you want to enable

It would be nice if we could control how random indices are created with groups in BaggingRegressor. As far as I can tell, BaggingRegressor will select sub samples u.a.r. with/without replacement.

It would be nice if we could group the data points, and have BaggingRegressor select sub samples of (optional) _groups_ u.a.r. with/without replacement.

An example of this paradigm already exists in sklearn with cross validation (e.g. KFold vs. GroupKFold)

### Describe your proposed solution

One idea is to pass in a vector `groups` of size `len(X)` mapping each data point to its group. Then, `_generate_indices` can respect groups if they exist, or default to its existing behavior if they do not.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
One potential way for this would be not to support it natively, but rather accept a sampler object with its own required metadata. However, the closest object we have to the sampler is a splitter, and although passing a splitter like object would work, it's not really what it's doing or it should be doing at all. The plus side of it would be he most common usecase stays simple, and more advanced ones become a bit more complicated which is okay I think. WDYT @glemaitre

Labels: New Feature, Needs Decision
Groups in BaggingRegressor

### Describe the workflow you want to enable

It would be nice if we could control how random indices are created with groups in BaggingRegressor. As far as I can tell, BaggingRegressor will select sub samples u.a.r. with/without replacement.

It would be nice if we could group the data points, and have BaggingRegressor select sub samples of (optional) _groups_ u.a.r. with/without replacement.

An example of this paradigm already exists in sklearn with cross validation (e.g. KFold vs. GroupKFold)

### Describe your proposed solution

One idea is to pass in a vector `groups` of size `len(X)` mapping each data point to its group. Then, `_generate_indices` can respect groups if they exist, or default to its existing behavior if they do not.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
> One potential way for this would be not to support it natively, but rather accept a sampler object with its own required metadata.

This make me think about the strategy used in `imbalanced-learn` to implement the `BalancedBaggingClassifier`: https://imbalanced-learn.org/stable/references/generated/imblearn.ensemble.BalancedBaggingClassifier.html

Labels: New Feature, Needs Decision
Make an instance of ColumnTransformer pass the common test

### Describe the bug

Not sure if `check_estimator` is appropriate to use with transformers, but `ColumnTransformer` inherits from `BaseEstimator`, so I'm assuming so.

I have a custom transformer that inherits from `ColumnTransformer`, and I'm trying to use `check_estimator` to make sure I am doing so properly. However, `ColumnTransformer` itself fails on multiple tests; I'm assuming many are because creating a non-trivial `ColumnTransformer` involves specifying required feature names, which seems to go against some of the tests in `check_estimator`.

Is there a different best practice to test a custom subclass of `ColumnTransformer` and/or transformers in general?

### Steps/Code to Reproduce

```python
from sklearn.utils.estimator_checks import parametrize_with_checks
from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import Normalizer

ct = ColumnTransformer(
    [
        ("norm1", Normalizer(norm="l1"), [0, 1]),
        ("norm2", Normalizer(norm="l1"), slice(2, 4)),
    ],
    remainder="passthrough",
)


@parametrize_with_checks([ct])
def test_sklearn_checks(estimator, check):
    check(estimator)
```

### Expected Results

All tests pass.

### Actual Results

[Test Results —.pdf](https://github.com/scikit-learn/scikit-learn/files/13285011/Test.Results.pdf)


### Versions

```shell
System:
    python: 3.11.6 | packaged by conda-forge | (main, Oct  3 2023, 10:37:07) [Clang 15.0.7 ]
executable: /my_env/bin/python
   machine: macOS-13.4-arm64-arm-64bit
Python dependencies:
      sklearn: 1.3.1
          pip: 23.3.1
   setuptools: 68.2.2
        numpy: 1.26.0
        scipy: 1.11.3
       Cython: None
       pandas: 2.1.1
   matplotlib: 3.8.0
       joblib: 1.3.2
threadpoolctl: 3.2.0
Built with OpenMP: True
threadpoolctl info:
       user_api: blas
   internal_api: openblas
    num_threads: 8
         prefix: libopenblas
       filepath: /my_env/lib/libopenblas.0.dylib
        version: 0.3.24
threading_layer: openmp
   architecture: VORTEX
       user_api: openmp
   internal_api: openmp
    num_threads: 8
         prefix: libomp
       filepath: /my_env/lib/libomp.dylib
        version: None
```

COMMENT:
Can you please the traceback directly in the issue tracker.

Labels: Enhancement