Talip7/scikit-learn-issues-multilabel huggingface dataset:
The data type of input data for LinearRegression class will affect the results

### Describe the bug

Our team just used the class sklearn.linear_model.LinearRegression to do multi-linear regression. And, we found out that the same data, which means the values are identical for each element, with different data formats will return different coefficients. The array we use are all integer values but saved as float32 and float64, separately. However, the array in float64 as input get us the expected result, but float32 doesn't.

The code below is a simple reproducible example.

### Steps/Code to Reproduce

```python
In [1]: from sklearn.linear_model import LinearRegression
    ...: import numpy as np
    ...: rng = np.random.default_rng(0)
    ...: x_1_array = rng.integers(low=1000, high=10000, size=10_000_000)
    ...: x_2_array = rng.integers(low=1000, high=10000, size=10_000_000)
    ...: x_3_array = rng.integers(low=1000, high=10000, size=10_000_000)
    ...: y_array = 3 * x_1_array + x_2_array - 5 * x_3_array
    ...: intercept_data = rng.integers(low=-3, high=3)
    ...: print(f'True intercept {intercept_data}')
    ...: y_array += intercept_data
    ...: X_train = np.stack([x_1_array,x_2_array,x_3_array],axis=-1)
    ...: y_train = y_array
    ...: linreg = LinearRegression()
    ...: model = linreg.fit(X_train.astype(np.float64), y_train)
    ...: print(f'Float 64 Intercept: {linreg.intercept_}')
    ...: print(f'Float 64 Coefficient: {linreg.coef_}')
    ...: model = linreg.fit(X_train.astype(np.float32), y_train)
    ...: print(f'Float 32 Intercept: {linreg.intercept_}')
    ...: print(f'Float 32 Coefficient: {linreg.coef_}')
```

### Expected Results

![image](https://user-images.githubusercontent.com/51532833/168492124-d98f9663-4899-47ee-83b2-4952462c37a6.png)

### Actual Results

![image](https://user-images.githubusercontent.com/51532833/168492136-4e0f4100-9e48-4f58-aa2a-97e826f06c61.png)


### Versions

```shell
System:
    python: 3.7.9 (v3.7.9:13c94747c7, Aug 15 2020, 01:31:08)  [Clang 6.0 (clang-600.0.57)]
executable: /Users/lazy/PycharmProjects/projects/bin/python
   machine: Darwin-21.4.0-x86_64-i386-64bit

Python dependencies:
          pip: 21.3
   setuptools: 58.2.0
      sklearn: 1.0.2
        numpy: 1.21.4
        scipy: 1.7.2
       Cython: None
       pandas: 1.3.4
   matplotlib: 3.4.3
       joblib: 1.1.0
threadpoolctl: 3.1.0

Built with OpenMP: True
```

COMMENT:
The problematic line of code is the following:

https://github.com/scikit-learn/scikit-learn/blob/16625450b58f555dc3955d223f0c3b64a5686984/sklearn/linear_model/_base.py#L392

This is due to some accumulation of numerical error I assume. The coefficients obtain via `lsqt` are less precise with some numerical rounding that becomes larger when `n` increases. When we compute `np.dot(X_offset, self.coef_.T)` on a very large dataset, we add up these numerical errors and we end up with a wrong intercept.

Labels: Bug, module:linear_model, float32
The data type of input data for LinearRegression class will affect the results

### Describe the bug

Our team just used the class sklearn.linear_model.LinearRegression to do multi-linear regression. And, we found out that the same data, which means the values are identical for each element, with different data formats will return different coefficients. The array we use are all integer values but saved as float32 and float64, separately. However, the array in float64 as input get us the expected result, but float32 doesn't.

The code below is a simple reproducible example.

### Steps/Code to Reproduce

```python
In [1]: from sklearn.linear_model import LinearRegression
    ...: import numpy as np
    ...: rng = np.random.default_rng(0)
    ...: x_1_array = rng.integers(low=1000, high=10000, size=10_000_000)
    ...: x_2_array = rng.integers(low=1000, high=10000, size=10_000_000)
    ...: x_3_array = rng.integers(low=1000, high=10000, size=10_000_000)
    ...: y_array = 3 * x_1_array + x_2_array - 5 * x_3_array
    ...: intercept_data = rng.integers(low=-3, high=3)
    ...: print(f'True intercept {intercept_data}')
    ...: y_array += intercept_data
    ...: X_train = np.stack([x_1_array,x_2_array,x_3_array],axis=-1)
    ...: y_train = y_array
    ...: linreg = LinearRegression()
    ...: model = linreg.fit(X_train.astype(np.float64), y_train)
    ...: print(f'Float 64 Intercept: {linreg.intercept_}')
    ...: print(f'Float 64 Coefficient: {linreg.coef_}')
    ...: model = linreg.fit(X_train.astype(np.float32), y_train)
    ...: print(f'Float 32 Intercept: {linreg.intercept_}')
    ...: print(f'Float 32 Coefficient: {linreg.coef_}')
```

### Expected Results

![image](https://user-images.githubusercontent.com/51532833/168492124-d98f9663-4899-47ee-83b2-4952462c37a6.png)

### Actual Results

![image](https://user-images.githubusercontent.com/51532833/168492136-4e0f4100-9e48-4f58-aa2a-97e826f06c61.png)


### Versions

```shell
System:
    python: 3.7.9 (v3.7.9:13c94747c7, Aug 15 2020, 01:31:08)  [Clang 6.0 (clang-600.0.57)]
executable: /Users/lazy/PycharmProjects/projects/bin/python
   machine: Darwin-21.4.0-x86_64-i386-64bit

Python dependencies:
          pip: 21.3
   setuptools: 58.2.0
      sklearn: 1.0.2
        numpy: 1.21.4
        scipy: 1.7.2
       Cython: None
       pandas: 1.3.4
   matplotlib: 3.4.3
       joblib: 1.1.0
threadpoolctl: 3.1.0

Built with OpenMP: True
```

COMMENT:
As a workaround, I suspect that using a `StandardScaler` as preprocessor in a pipeline should help fix the problem by operating closer to 0.0 where the float32 representation is more precise.

Labels: Bug, module:linear_model, float32
The data type of input data for LinearRegression class will affect the results

### Describe the bug

Our team just used the class sklearn.linear_model.LinearRegression to do multi-linear regression. And, we found out that the same data, which means the values are identical for each element, with different data formats will return different coefficients. The array we use are all integer values but saved as float32 and float64, separately. However, the array in float64 as input get us the expected result, but float32 doesn't.

The code below is a simple reproducible example.

### Steps/Code to Reproduce

```python
In [1]: from sklearn.linear_model import LinearRegression
    ...: import numpy as np
    ...: rng = np.random.default_rng(0)
    ...: x_1_array = rng.integers(low=1000, high=10000, size=10_000_000)
    ...: x_2_array = rng.integers(low=1000, high=10000, size=10_000_000)
    ...: x_3_array = rng.integers(low=1000, high=10000, size=10_000_000)
    ...: y_array = 3 * x_1_array + x_2_array - 5 * x_3_array
    ...: intercept_data = rng.integers(low=-3, high=3)
    ...: print(f'True intercept {intercept_data}')
    ...: y_array += intercept_data
    ...: X_train = np.stack([x_1_array,x_2_array,x_3_array],axis=-1)
    ...: y_train = y_array
    ...: linreg = LinearRegression()
    ...: model = linreg.fit(X_train.astype(np.float64), y_train)
    ...: print(f'Float 64 Intercept: {linreg.intercept_}')
    ...: print(f'Float 64 Coefficient: {linreg.coef_}')
    ...: model = linreg.fit(X_train.astype(np.float32), y_train)
    ...: print(f'Float 32 Intercept: {linreg.intercept_}')
    ...: print(f'Float 32 Coefficient: {linreg.coef_}')
```

### Expected Results

![image](https://user-images.githubusercontent.com/51532833/168492124-d98f9663-4899-47ee-83b2-4952462c37a6.png)

### Actual Results

![image](https://user-images.githubusercontent.com/51532833/168492136-4e0f4100-9e48-4f58-aa2a-97e826f06c61.png)


### Versions

```shell
System:
    python: 3.7.9 (v3.7.9:13c94747c7, Aug 15 2020, 01:31:08)  [Clang 6.0 (clang-600.0.57)]
executable: /Users/lazy/PycharmProjects/projects/bin/python
   machine: Darwin-21.4.0-x86_64-i386-64bit

Python dependencies:
          pip: 21.3
   setuptools: 58.2.0
      sklearn: 1.0.2
        numpy: 1.21.4
        scipy: 1.7.2
       Cython: None
       pandas: 1.3.4
   matplotlib: 3.4.3
       joblib: 1.1.0
threadpoolctl: 3.1.0

Built with OpenMP: True
```

COMMENT:
<del>The output would variable `y` might also need rescaling, possibly using a `sklearn.compose.TransformedTargetRegressor`.</del>

Edit: no sure this is needed to fix this specific problem.

Labels: Bug, module:linear_model, float32
The data type of input data for LinearRegression class will affect the results

### Describe the bug

Our team just used the class sklearn.linear_model.LinearRegression to do multi-linear regression. And, we found out that the same data, which means the values are identical for each element, with different data formats will return different coefficients. The array we use are all integer values but saved as float32 and float64, separately. However, the array in float64 as input get us the expected result, but float32 doesn't.

The code below is a simple reproducible example.

### Steps/Code to Reproduce

```python
In [1]: from sklearn.linear_model import LinearRegression
    ...: import numpy as np
    ...: rng = np.random.default_rng(0)
    ...: x_1_array = rng.integers(low=1000, high=10000, size=10_000_000)
    ...: x_2_array = rng.integers(low=1000, high=10000, size=10_000_000)
    ...: x_3_array = rng.integers(low=1000, high=10000, size=10_000_000)
    ...: y_array = 3 * x_1_array + x_2_array - 5 * x_3_array
    ...: intercept_data = rng.integers(low=-3, high=3)
    ...: print(f'True intercept {intercept_data}')
    ...: y_array += intercept_data
    ...: X_train = np.stack([x_1_array,x_2_array,x_3_array],axis=-1)
    ...: y_train = y_array
    ...: linreg = LinearRegression()
    ...: model = linreg.fit(X_train.astype(np.float64), y_train)
    ...: print(f'Float 64 Intercept: {linreg.intercept_}')
    ...: print(f'Float 64 Coefficient: {linreg.coef_}')
    ...: model = linreg.fit(X_train.astype(np.float32), y_train)
    ...: print(f'Float 32 Intercept: {linreg.intercept_}')
    ...: print(f'Float 32 Coefficient: {linreg.coef_}')
```

### Expected Results

![image](https://user-images.githubusercontent.com/51532833/168492124-d98f9663-4899-47ee-83b2-4952462c37a6.png)

### Actual Results

![image](https://user-images.githubusercontent.com/51532833/168492136-4e0f4100-9e48-4f58-aa2a-97e826f06c61.png)


### Versions

```shell
System:
    python: 3.7.9 (v3.7.9:13c94747c7, Aug 15 2020, 01:31:08)  [Clang 6.0 (clang-600.0.57)]
executable: /Users/lazy/PycharmProjects/projects/bin/python
   machine: Darwin-21.4.0-x86_64-i386-64bit

Python dependencies:
          pip: 21.3
   setuptools: 58.2.0
      sklearn: 1.0.2
        numpy: 1.21.4
        scipy: 1.7.2
       Cython: None
       pandas: 1.3.4
   matplotlib: 3.4.3
       joblib: 1.1.0
threadpoolctl: 3.1.0

Built with OpenMP: True
```

COMMENT:
The intercept is much better but then you have a large variation of the weights. Taking the previous example and applying a standard scaling.

```python
Float 64 Intercept: -5498.377062199998
Float 64 Coefficient: [  7793.46267406   2597.7576475  -12992.64703011]
Float 32 Intercept: -5498.37841796875
Float 32 Coefficient: [  7870.14     2623.2705 -13117.827 ]
```

Labels: Bug, module:linear_model, float32
The data type of input data for LinearRegression class will affect the results

### Describe the bug

Our team just used the class sklearn.linear_model.LinearRegression to do multi-linear regression. And, we found out that the same data, which means the values are identical for each element, with different data formats will return different coefficients. The array we use are all integer values but saved as float32 and float64, separately. However, the array in float64 as input get us the expected result, but float32 doesn't.

The code below is a simple reproducible example.

### Steps/Code to Reproduce

```python
In [1]: from sklearn.linear_model import LinearRegression
    ...: import numpy as np
    ...: rng = np.random.default_rng(0)
    ...: x_1_array = rng.integers(low=1000, high=10000, size=10_000_000)
    ...: x_2_array = rng.integers(low=1000, high=10000, size=10_000_000)
    ...: x_3_array = rng.integers(low=1000, high=10000, size=10_000_000)
    ...: y_array = 3 * x_1_array + x_2_array - 5 * x_3_array
    ...: intercept_data = rng.integers(low=-3, high=3)
    ...: print(f'True intercept {intercept_data}')
    ...: y_array += intercept_data
    ...: X_train = np.stack([x_1_array,x_2_array,x_3_array],axis=-1)
    ...: y_train = y_array
    ...: linreg = LinearRegression()
    ...: model = linreg.fit(X_train.astype(np.float64), y_train)
    ...: print(f'Float 64 Intercept: {linreg.intercept_}')
    ...: print(f'Float 64 Coefficient: {linreg.coef_}')
    ...: model = linreg.fit(X_train.astype(np.float32), y_train)
    ...: print(f'Float 32 Intercept: {linreg.intercept_}')
    ...: print(f'Float 32 Coefficient: {linreg.coef_}')
```

### Expected Results

![image](https://user-images.githubusercontent.com/51532833/168492124-d98f9663-4899-47ee-83b2-4952462c37a6.png)

### Actual Results

![image](https://user-images.githubusercontent.com/51532833/168492136-4e0f4100-9e48-4f58-aa2a-97e826f06c61.png)


### Versions

```shell
System:
    python: 3.7.9 (v3.7.9:13c94747c7, Aug 15 2020, 01:31:08)  [Clang 6.0 (clang-600.0.57)]
executable: /Users/lazy/PycharmProjects/projects/bin/python
   machine: Darwin-21.4.0-x86_64-i386-64bit

Python dependencies:
          pip: 21.3
   setuptools: 58.2.0
      sklearn: 1.0.2
        numpy: 1.21.4
        scipy: 1.7.2
       Cython: None
       pandas: 1.3.4
   matplotlib: 3.4.3
       joblib: 1.1.0
threadpoolctl: 3.1.0

Built with OpenMP: True
```

COMMENT:
Hello, I am a beginner in contributions and I am interested in this issue. Is it still available ?

Labels: Bug, module:linear_model, float32
The data type of input data for LinearRegression class will affect the results

### Describe the bug

Our team just used the class sklearn.linear_model.LinearRegression to do multi-linear regression. And, we found out that the same data, which means the values are identical for each element, with different data formats will return different coefficients. The array we use are all integer values but saved as float32 and float64, separately. However, the array in float64 as input get us the expected result, but float32 doesn't.

The code below is a simple reproducible example.

### Steps/Code to Reproduce

```python
In [1]: from sklearn.linear_model import LinearRegression
    ...: import numpy as np
    ...: rng = np.random.default_rng(0)
    ...: x_1_array = rng.integers(low=1000, high=10000, size=10_000_000)
    ...: x_2_array = rng.integers(low=1000, high=10000, size=10_000_000)
    ...: x_3_array = rng.integers(low=1000, high=10000, size=10_000_000)
    ...: y_array = 3 * x_1_array + x_2_array - 5 * x_3_array
    ...: intercept_data = rng.integers(low=-3, high=3)
    ...: print(f'True intercept {intercept_data}')
    ...: y_array += intercept_data
    ...: X_train = np.stack([x_1_array,x_2_array,x_3_array],axis=-1)
    ...: y_train = y_array
    ...: linreg = LinearRegression()
    ...: model = linreg.fit(X_train.astype(np.float64), y_train)
    ...: print(f'Float 64 Intercept: {linreg.intercept_}')
    ...: print(f'Float 64 Coefficient: {linreg.coef_}')
    ...: model = linreg.fit(X_train.astype(np.float32), y_train)
    ...: print(f'Float 32 Intercept: {linreg.intercept_}')
    ...: print(f'Float 32 Coefficient: {linreg.coef_}')
```

### Expected Results

![image](https://user-images.githubusercontent.com/51532833/168492124-d98f9663-4899-47ee-83b2-4952462c37a6.png)

### Actual Results

![image](https://user-images.githubusercontent.com/51532833/168492136-4e0f4100-9e48-4f58-aa2a-97e826f06c61.png)


### Versions

```shell
System:
    python: 3.7.9 (v3.7.9:13c94747c7, Aug 15 2020, 01:31:08)  [Clang 6.0 (clang-600.0.57)]
executable: /Users/lazy/PycharmProjects/projects/bin/python
   machine: Darwin-21.4.0-x86_64-i386-64bit

Python dependencies:
          pip: 21.3
   setuptools: 58.2.0
      sklearn: 1.0.2
        numpy: 1.21.4
        scipy: 1.7.2
       Cython: None
       pandas: 1.3.4
   matplotlib: 3.4.3
       joblib: 1.1.0
threadpoolctl: 3.1.0

Built with OpenMP: True
```

COMMENT:
I am going to close this one, because I don't think this is a bug per se. Using float32 instead of float64 will lead to different results, it may lead to wildly different results in some cases. 

I don't think the scikit-learn doc is the right place to discuss this kind of generic floating point issues.

Labels: Bug, module:linear_model, float32
SGDRegressor documentation refers to non-existent loss metric

### Describe the issue linked to the documentation

The documentation refers to "SGDRegressor(loss='squared_error') in various places throughout, including an example in paragraph 3 and the descriptions of acceptable values in section 1.5.2.  However, testing and review shows that 'squared_error' is not an option and the correct value should be 'squared_loss'.  This is using 0.24.1, the latest I can download via pip.  The help function also mentions that 'squared_loss' is the default value, which is not mentioned in section 1.5.2.


https://scikit-learn.org/stable/modules/sgd.html#:~:text=Stochastic%20Gradient%20Descent%20(SGD)%20is,Vector%20Machines%20and%20Logistic%20Regression.


### Suggest a potential alternative/fix

Change 'squared_error' in help docs to 'squared_loss'

COMMENT:
Please refer to the deprecation of `squared_loss` in favor of `squared_error` in 1.0: 

```
Deprecated since version 1.0: The loss ‘squared_loss’ was deprecated in v1.0 and will be removed in version 1.2. Use loss='squared_error' which is equivalent.
```

The latest version on PyPI is scikit-learn 1.1:
https://pypi.org/project/scikit-learn/

If you are using a previous release of scikit-learn, you can check the associated documentation version:
https://scikit-learn.org/dev/versions.html

Labels: Documentation, Needs Triage
SGDRegressor documentation refers to non-existent loss metric

### Describe the issue linked to the documentation

The documentation refers to "SGDRegressor(loss='squared_error') in various places throughout, including an example in paragraph 3 and the descriptions of acceptable values in section 1.5.2.  However, testing and review shows that 'squared_error' is not an option and the correct value should be 'squared_loss'.  This is using 0.24.1, the latest I can download via pip.  The help function also mentions that 'squared_loss' is the default value, which is not mentioned in section 1.5.2.


https://scikit-learn.org/stable/modules/sgd.html#:~:text=Stochastic%20Gradient%20Descent%20(SGD)%20is,Vector%20Machines%20and%20Logistic%20Regression.


### Suggest a potential alternative/fix

Change 'squared_error' in help docs to 'squared_loss'

COMMENT:
Hi- this was an error on my part with updating SKL, sorry about the confusion!

Labels: Documentation, Needs Triage
ImportError: cannot import name '_joblib_parallel_args' from 'sklearn.utils.fixes' (/Users/anaconda3/lib/python3.9/site-packages/sklearn/utils/fixes.py)

---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
/var/folders/1g/yyb24n4s4l381d6slhybxdt80000gn/T/ipykernel_49688/4032187472.py in <module>
      8 
      9 from sksurv.preprocessing import OneHotEncoder
---> 10 from sksurv.ensemble import RandomSurvivalForest

/Users/anaconda3/lib/python3.9/site-packages/sksurv/ensemble/__init__.py in <module>
      1 from .boosting import ComponentwiseGradientBoostingSurvivalAnalysis, GradientBoostingSurvivalAnalysis  # noqa: F401
----> 2 from .forest import ExtraSurvivalTrees, RandomSurvivalForest  # noqa: F401

/Users/anaconda3/lib/python3.9/site-packages/sksurv/ensemble/forest.py in <module>
     15 )
     16 from sklearn.tree._tree import DTYPE
---> 17 from sklearn.utils.fixes import _joblib_parallel_args
     18 from sklearn.utils.validation import check_array, check_is_fitted, check_random_state
     19 

ImportError: cannot import name '_joblib_parallel_args' from 'sklearn.utils.fixes' (/Users/anaconda3/lib/python3.9/site-packages/sklearn/utils/fixes.py)

COMMENT:
Scikit-learn has removed the private function `_joblib_parallel_args` in 1.1. I recommend opening an issue in https://github.com/sebp/scikit-survival so they know about error.

If you need to use `sksurv` now, I recommend pinning scikit-learn to 1.0 for now until `scikit-survival` is updated.

Labels: Needs Triage
sklearn.metrics.coverage_error wrong error message for 1D array

### Describe the bug

Let y_true and y_score be numpy arrays of shape (K,),
when you try to run the "sklearn.metrics.coverage_error" as explained in the documentation
it returns "binary type not supported" error, but this is not the case at all, the metric works with,
binary, the problem is it expects matrices of shape (1, K) if there is only one sample,
Please either fix the function so that it can work with 1D array or please fix the error message.

### Steps/Code to Reproduce

```python
coverage = np.mean([coverage_error(discrete_labels[i], discrete_scores[i]) - 1 for i in range(N)])
```
### Expected Results

No error or, can not use 1D array error

### Actual Results

```python-traceback
   raise ValueError("{0} format is not supported".format(y_type))
ValueError: binary format is not supported
```

### Versions

```shell
>>> import sklearn; sklearn.show_versions()

System:
    python: 3.9.7 | packaged by conda-forge | (default, Sep 29 2021, 19:20:46)  [GCC 9.4.0]
executable: /home/delivahit/anaconda3/envs/bugra/bin/python
   machine: Linux-5.4.0-107-generic-x86_64-with-glibc2.27

Python dependencies:
          pip: 21.2.4
   setuptools: 58.0.4
      sklearn: 1.0.2
        numpy: 1.21.2
        scipy: 1.7.2
       Cython: None
       pandas: 1.3.4
   matplotlib: 3.5.0
       joblib: 1.1.0
threadpoolctl: 3.0.0

Built with OpenMP: True
```

COMMENT:
> Please either fix the function so that it can work with 1D array or please fix the error message.

I agree we should fix the error messages. I think it comes from the fact that the 2 calls `check_array(., ensure_2d=False)` at the beginning of `coverage_error` should be changes to `ensure_2d=True` instead. Some tests might need to be updated accordingly and we would need a new test to check that the right error message is raised when passing a 1d array as input.

Labels: Bug, Easy, help wanted, module:metrics
KMeans processing n_init sequentially!!

Hi,

I was looking into KMeans code and found that the following can be parallelized. For example, each work in `for loop` can be processed independently. I expect this to reduce the runtime. Please check.

https://github.com/scikit-learn/scikit-learn/blob/84f8409dc5c485729649c5332e66fd5602549b50/sklearn/cluster/_kmeans.py#L1406-L1438

COMMENT:
You can have a look at the following blog post explaining the current optimization and to see where the parallelization is currently done in the code: https://scikit-learn.fondation-inria.fr/implementing-a-faster-kmeans-in-scikit-learn-0-23/

The associated PR was the following: https://github.com/scikit-learn/scikit-learn/pull/11950

Sometimes optimum parallelization is not as easy as it looks.

Labels: Needs Triage
⚠️ CI failed on Linux_Nightly_ICC.pylatest_conda_forge_mkl ⚠️

**CI Failed on [Linux_Nightly_ICC.pylatest_conda_forge_mkl](https://dev.azure.com/scikit-learn/scikit-learn/_build/results?buildId=42057&view=logs&j=8628a494-79d0-53fa-274c-1b00464f7121)**
Unable to find junit file. Please see link for details.

COMMENT:
It crashes at install time:

```
E: Failed to fetch https://apt.repos.intel.com/oneapi/dists/all/main/binary-amd64/Packages.bz2  File has unexpected size (141938 != 141743). Mirror sync in progress? [IP: 104.124.147.131 443]
```

maybe it's a transient problem on the apt repo that is concurrently being updated?

Labels: Needs Triage