Talip7/scikit-learn-issues-multilabel huggingface dataset:
QuantileRegressor unable to allocate memory for large datasets

### Describe the bug

I am using sklearn.linear_model.QuantileRegressor for a dataset with ~2.9 million datapoints. 
When I use it as follows, I get a memory error.

`MemoryError: Unable to allocate 61.6 TiB for an array with shape (2909376, 2909376) and data type float64`

When I do the same with statsmodel library, I don't encounter any issues. 

Would this be a known limitation of the sklarn implementation? What would be the biggest size of dataframe it can take? Or is it a bug? 

I was choosing to do it in sklearn rather than statsmodel because the sklearn API provides ability to add regularization which the other library doesn't. 

### Steps/Code to Reproduce

```python
from sklearn.linear_model import QuantileRegressor
reg = QuantileRegressor(quantile=0.8).fit(X, y)
```

### Expected Results

The data is fit without errors. 

### Actual Results
```
---------------------------------------------------------------------------
MemoryError                               Traceback (most recent call last)
/tmp/ipykernel_53907/1165686720.py in <module>
      1 from sklearn.linear_model import QuantileRegressor
      2 
----> 3 reg = QuantileRegressor(quantile=0.8).fit(X, y)
      4 print_analysis(reg, X, y, flyte_type)

/anaconda3/lib/python3.7/site-packages/sklearn/linear_model/_quantile.py in fit(self, X, y, sample_weight)
    239                     -np.ones((n_mask, 1)),
    240                     -X[mask],
--> 241                     np.eye(n_mask),
    242                     -np.eye(n_mask),
    243                 ],

/anaconda3/lib/python3.7/site-packages/numpy/lib/twodim_base.py in eye(N, M, k, dtype, order)
    197     if M is None:
    198         M = N
--> 199     m = zeros((N, M), dtype=dtype, order=order)
    200     if k >= M:
    201         return m

MemoryError: Unable to allocate 61.6 TiB for an array with shape (2909376, 2909376) and data type float64
```

### Versions

```shell
System:
    python: 3.7.7 (default, Mar 26 2020, 15:48:22)  [GCC 7.3.0]
executable: /anaconda3/bin/python
   machine: Linux-4.14.68-4.14.3-amd64-3adf3675665129fa-x86_64-with-debian-stretch-sid

Python dependencies:
          pip: 21.2.2
   setuptools: 58.0.4
      sklearn: 1.0.1
        numpy: 1.19.2
        scipy: 1.5.2
       Cython: 0.29.25
       pandas: 1.2.4
   matplotlib: 3.5.0
       joblib: 1.1.0
threadpoolctl: 2.2.0

Built with OpenMP: True
```

COMMENT:
@bharatkrishna Sparse data support for `QuantileRegression` comes in the upcoming release, see https://scikit-learn.org/dev/whats_new/v1.1.html.
In the meantime, you could install a nightly build, see https://scikit-learn.org/dev/developers/advanced_installation.html?highlight=nightly#installing-nightly-builds.

The original issue is not a bug of scikit-learn, but a known limitation. Therefore, I'll close.

Labels: Bug, module:linear_model
Calculation of alphas in ElasticNetCV doesn't use sample_weight

### Describe the bug

In ElasticNetCV, the first and largest value of `alpha`, call it `alpha_max`, should be just large enough to force all of the coefficients to become zero. The existing code works correctly when `sample_weight` is not specified. However, the computation of `alpha_max` does not take into account `sample_weight`.

### Steps/Code to Reproduce

```python
import numpy as np
from sklearn.linear_model import ElasticNet, ElasticNetCV

X = np.array([[3, 1], [2, 5], [5, 3], [1, 4]])
beta = np.array([1, 1])
y = X @ beta
w = np.array([10, 1, 10, 1])

# Fit ElasticNetCV just to get the .alphas_ attribute
enetCV = ElasticNetCV(cv=2)
enetCV.fit(X, y, sample_weight=w)

# The coefficient of ElasticNet fitted at alpha_max should be [0.  0.].
alpha_max = enetCV.alphas_[0]
enet = ElasticNet(alpha=alpha_max)
enet.fit(X, y, sample_weight=w)
print(enet.coef_)  # [0.1970807  0.19708023]
```

### Expected Results

If the correct value of `alpha_max` is computed, then `enet.coef_` should be right at the cusp of zero, such that any smaller value of `alpha` makes it nonzero:
```python
def get_alpha_max(X, y, w, l1_ratio=0.5):
    wn = w / w.sum()
    Xn = X - np.dot(wn, X)
    yn = (y - np.dot(wn, y)) * wn
    return np.max(np.abs(yn @ Xn)) / l1_ratio


enet = ElasticNet(alpha=get_alpha_max(X, y, w))
enet.fit(X, y, sample_weight=w)
print(enet.coef_)  # [6.70427878e-17 6.70427878e-17]
```

### Actual Results

`enet.coef_` is `[0.1970807  0.19708023]`.

### Versions

```shell
System:
    python: 3.9.7 (default, Sep 16 2021, 13:09:58)  [GCC 7.5.0]
executable: /home/jhopfens/.conda/envs/jhop39/bin/python
   machine: Linux-3.10.0-1160.53.1.el7.x86_64-x86_64-with-glibc2.17

Python dependencies:
          pip: 21.2.4
   setuptools: 58.0.4
      sklearn: 1.0.2
        numpy: 1.21.5
        scipy: 1.7.2
       Cython: 0.29.24
       pandas: 1.3.5
   matplotlib: 3.5.1
       joblib: 1.1.0
threadpoolctl: 3.0.0

Built with OpenMP: True
```

COMMENT:
I agree, the max `alpha` should give zero coefficients.
Do you want to give it a try and open a PR?

The *change policy* might be tricky as it corresponds to changing a model (once fixed, `ElasticNetCV.fit` will give different results than before) and there is no meaningful deprecation path, IMO.
I consider it a bug fix such that no deprecation cycle is needed.

Labels: Bug, module:linear_model
Calculation of alphas in ElasticNetCV doesn't use sample_weight

### Describe the bug

In ElasticNetCV, the first and largest value of `alpha`, call it `alpha_max`, should be just large enough to force all of the coefficients to become zero. The existing code works correctly when `sample_weight` is not specified. However, the computation of `alpha_max` does not take into account `sample_weight`.

### Steps/Code to Reproduce

```python
import numpy as np
from sklearn.linear_model import ElasticNet, ElasticNetCV

X = np.array([[3, 1], [2, 5], [5, 3], [1, 4]])
beta = np.array([1, 1])
y = X @ beta
w = np.array([10, 1, 10, 1])

# Fit ElasticNetCV just to get the .alphas_ attribute
enetCV = ElasticNetCV(cv=2)
enetCV.fit(X, y, sample_weight=w)

# The coefficient of ElasticNet fitted at alpha_max should be [0.  0.].
alpha_max = enetCV.alphas_[0]
enet = ElasticNet(alpha=alpha_max)
enet.fit(X, y, sample_weight=w)
print(enet.coef_)  # [0.1970807  0.19708023]
```

### Expected Results

If the correct value of `alpha_max` is computed, then `enet.coef_` should be right at the cusp of zero, such that any smaller value of `alpha` makes it nonzero:
```python
def get_alpha_max(X, y, w, l1_ratio=0.5):
    wn = w / w.sum()
    Xn = X - np.dot(wn, X)
    yn = (y - np.dot(wn, y)) * wn
    return np.max(np.abs(yn @ Xn)) / l1_ratio


enet = ElasticNet(alpha=get_alpha_max(X, y, w))
enet.fit(X, y, sample_weight=w)
print(enet.coef_)  # [6.70427878e-17 6.70427878e-17]
```

### Actual Results

`enet.coef_` is `[0.1970807  0.19708023]`.

### Versions

```shell
System:
    python: 3.9.7 (default, Sep 16 2021, 13:09:58)  [GCC 7.5.0]
executable: /home/jhopfens/.conda/envs/jhop39/bin/python
   machine: Linux-3.10.0-1160.53.1.el7.x86_64-x86_64-with-glibc2.17

Python dependencies:
          pip: 21.2.4
   setuptools: 58.0.4
      sklearn: 1.0.2
        numpy: 1.21.5
        scipy: 1.7.2
       Cython: 0.29.24
       pandas: 1.3.5
   matplotlib: 3.5.1
       joblib: 1.1.0
threadpoolctl: 3.0.0

Built with OpenMP: True
```

COMMENT:
> I consider it a bug fix such that no deprecation cycle is needed.

I agree with that

Labels: Bug, module:linear_model
Calculation of alphas in ElasticNetCV doesn't use sample_weight

### Describe the bug

In ElasticNetCV, the first and largest value of `alpha`, call it `alpha_max`, should be just large enough to force all of the coefficients to become zero. The existing code works correctly when `sample_weight` is not specified. However, the computation of `alpha_max` does not take into account `sample_weight`.

### Steps/Code to Reproduce

```python
import numpy as np
from sklearn.linear_model import ElasticNet, ElasticNetCV

X = np.array([[3, 1], [2, 5], [5, 3], [1, 4]])
beta = np.array([1, 1])
y = X @ beta
w = np.array([10, 1, 10, 1])

# Fit ElasticNetCV just to get the .alphas_ attribute
enetCV = ElasticNetCV(cv=2)
enetCV.fit(X, y, sample_weight=w)

# The coefficient of ElasticNet fitted at alpha_max should be [0.  0.].
alpha_max = enetCV.alphas_[0]
enet = ElasticNet(alpha=alpha_max)
enet.fit(X, y, sample_weight=w)
print(enet.coef_)  # [0.1970807  0.19708023]
```

### Expected Results

If the correct value of `alpha_max` is computed, then `enet.coef_` should be right at the cusp of zero, such that any smaller value of `alpha` makes it nonzero:
```python
def get_alpha_max(X, y, w, l1_ratio=0.5):
    wn = w / w.sum()
    Xn = X - np.dot(wn, X)
    yn = (y - np.dot(wn, y)) * wn
    return np.max(np.abs(yn @ Xn)) / l1_ratio


enet = ElasticNet(alpha=get_alpha_max(X, y, w))
enet.fit(X, y, sample_weight=w)
print(enet.coef_)  # [6.70427878e-17 6.70427878e-17]
```

### Actual Results

`enet.coef_` is `[0.1970807  0.19708023]`.

### Versions

```shell
System:
    python: 3.9.7 (default, Sep 16 2021, 13:09:58)  [GCC 7.5.0]
executable: /home/jhopfens/.conda/envs/jhop39/bin/python
   machine: Linux-3.10.0-1160.53.1.el7.x86_64-x86_64-with-glibc2.17

Python dependencies:
          pip: 21.2.4
   setuptools: 58.0.4
      sklearn: 1.0.2
        numpy: 1.21.5
        scipy: 1.7.2
       Cython: 0.29.24
       pandas: 1.3.5
   matplotlib: 3.5.1
       joblib: 1.1.0
threadpoolctl: 3.0.0

Built with OpenMP: True
```

COMMENT:
> I could simply modify the function sklearn.linear_model._coordinate_descent._alpha_grid, but I'm not sure this would be optimal. I would simply be repeating the calculation of multiplying by sqrt(sample_weight) which is already being done when we fit ElasticNet. Personally, here is how I would approach the problem:

I think it's better to first fix the issue, if there's the possibility of a simple fix as it seems here, with a good non-regression test like you showed in your issue. Then we can think about making some small refactorings in a separate PR.

Labels: Bug, module:linear_model
UserWarning is thrown when calling `HistGradientBoostingRegressor.fit` while specifying `scoring` argument

### Describe the bug

`UserWarning: X does not have valid feature names` is thrown when calling `HistGradientBoostingRegressor.fit` using `pandas.DataFrame`.

The regressor was constructed by specifying `scoring` argument.
Also since the sample size is large, seems like `early_stopping` is automatically enabled in this case.

If I recall correctly, this warning was not thrown on version before 1.0.

Reproducible on Colab:
https://colab.research.google.com/drive/1Qsg9b8L3gsDBWXQPvHbWrqGiuH49E6Xm?usp=sharing

### Steps/Code to Reproduce

```python3
import pandas as pd
import numpy as np
import sklearn
from sklearn import ensemble

size = 99999
X = pd.DataFrame(dict(
    a=np.random.rand(size), 
    b=np.random.rand(size),
))
y = np.random.rand(size)

est = ensemble.HistGradientBoostingRegressor(
    scoring='neg_mean_absolute_error',
)
est.fit(X, y)
```

### Expected Results

no warning thrown

### Actual Results

```
UserWarning: X does not have valid feature names, but HistGradientBoostingRegressor was fitted with feature names
```

### Versions

```shell
System:
    python: 3.7.12 (default, Jan 15 2022, 18:48:18)  [GCC 7.5.0]
executable: /usr/bin/python3
   machine: Linux-5.4.144+-x86_64-with-Ubuntu-18.04-bionic

Python dependencies:
          pip: 21.1.3
   setuptools: 57.4.0
      sklearn: 1.0.2
        numpy: 1.21.5
        scipy: 1.4.1
       Cython: 0.29.28
       pandas: 1.3.5
   matplotlib: 3.2.2
       joblib: 1.1.0
threadpoolctl: 3.1.0

Built with OpenMP: True
```

COMMENT:
Thank you for opening the bug report! I agree this is a bug with `HistGradientBoosting*` and opened https://github.com/scikit-learn/scikit-learn/pull/22908 to resolve the issue.

Labels: Bug
KMeans support for complex numbers

### Describe the workflow you want to enable

It would be nice with K-means clustering (`KMeans`) for complex-valued vectors. Today complex-valued vectors in the `fit` method throw a `ValueError`.

### Describe your proposed solution

Each element of a complex vector is associated with two axes (a real and an imaginary) in the parameter space on which to perform clustering. A way to "fake" this behavior today, is to re-arrange the complex input vector of length N to a real vector of length 2N, where the real parts comes first and then the imaginary parts.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
Supporting complex numbers is outside of the scope of scikit-learn.

Labels: New Feature, Needs Triage
SGDRegressor intercept exactly around half of the value it should be

### Describe the bug

It seems that the recent sklearn.SGDRegressor produces intercepts half of the real values. 

### Steps/Code to Reproduce

import numpy as np
from sklearn.linear_model import SGDRegressor, LinearRegression
x = np.random.randn(100,1)
y = 10 + 7 * x 
X = np.c_[np.ones(100), x]
sgd = SGDRegressor(max_iter=1000, eta0=.1, tol=1e-3)
lin = LinearRegression()
sgd.fit(X, y)
lin.fit(X,y)
print('Intercept and coef from LinReg:', [lin.intercept_[0], lin.coef_[0][1]]) #  10, 7
print('Intercept and coef from SGDReg:', sgd.coef_)                                     # ~5, 7 ?!


### Expected Results

[10, 7] is expected but [5, 7] is given

### Actual Results

Intercept and coef from SGDReg: [4.99492237 6.99910504]

### Versions

```shell
System:
    python: 3.9.7 (default, Sep 16 2021, 08:50:36)  [Clang 10.0.0 ]
executable: /opt/miniconda3/bin/python
   machine: macOS-10.16-x86_64-i386-64bit

Python dependencies:
          pip: 21.2.4
   setuptools: 58.0.4
      sklearn: 1.0.2
        numpy: 1.21.2
        scipy: 1.7.3
       Cython: None
       pandas: 1.4.1
   matplotlib: 3.5.1
       joblib: 1.1.0
threadpoolctl: 2.2.0

Built with OpenMP: True
```

COMMENT:
`LinearRegression`'s solver places all the weight on the constant feature. On the other hand, `SGDRegressor` splits the weight evenly between the constant feature's coefficient and the intercept term:

```python
import numpy as np
from sklearn.linear_model import SGDRegressor, LinearRegression

x = np.random.randn(100,1)
y = 10 + 7 * x
X = np.c_[np.ones(100), x]
sgd = SGDRegressor(max_iter=1000, eta0=.1, tol=1e-3)
sgd.fit(X, y)

sgd.coef_, sgd.intercept_
# (array([4.99551843, 6.99945584]), array([5.00437337]))

sgd.coef_[0] + sgd.intercept_[0]
# 9.999
```

With a constant feature, one can pass in `fit_intercept=False` to put all the weight on the constant feature's coefficient:

```python
import numpy as np
from sklearn.linear_model import SGDRegressor, LinearRegression

x = np.random.randn(100,1)
y = 10 + 7 * x
X = np.c_[np.ones(100), x]
sgd = SGDRegressor(max_iter=1000, eta0=.1, tol=1e-3, fit_intercept=False)
sgd.fit(X, y)

sgd.coef_, sgd.intercept_
# (array([9.9990354 , 6.99934434]), array([0.]))
```

Labels: Bug, Needs Triage
Encapsulate nearest neighbor code for `sort_sparse_row_values` in a function

### Describe the issue linked to the documentation

The [nearest neighbor code](https://github.com/scikit-learn/scikit-learn/blob/fb71eeff5a40399648d63258239079f5db7dcd76/sklearn/neighbors/_base.py#L206) often raises an `EfficiencyWarning`, particularly when using [DBSCAN](https://scikit-learn.org/stable/modules/generated/sklearn.cluster.DBSCAN.html) on sparse precomputed distance matrices. There's no proper documentation to resolve this warning, probably because there is no clean fix. You'd need to encapsulate [the code to sort the rows](https://github.com/scikit-learn/scikit-learn/blob/fb71eeff5a40399648d63258239079f5db7dcd76/sklearn/neighbors/_base.py#L206) in a function, and call that. Besides the downside of needing to maintain this low level code in one's own code base, while hoping it will stay compatible with scikit-learn's code, it's also an eye soar and something you can't document cleanly as a fix.

### Suggest a potential alternative/fix

Encapsulate the code as a function and document the `EfficiencyWarning` and its fix running the function on the matrix [here](https://scikit-learn.org/stable/modules/generated/sklearn.cluster.DBSCAN.html).

<!-- langauge: Python -->

    def sort_sparse_row_values(graph):
      # graph = graph.copy()
      row_nnz = np.diff(graph.indptr)
      if row_nnz.max() == row_nnz.min():
        n_samples = graph.shape[0]
        distances = graph.data.reshape(n_samples, -1)
    
        order = np.argsort(distances, kind="mergesort")
        order += np.arange(n_samples)[:, None] * row_nnz[0]
        order = order.ravel()
        graph.data = graph.data[order]
        graph.indices = graph.indices[order]
      else:
        for start, stop in zip(graph.indptr, graph.indptr[1:]):
          order = np.argsort(graph.data[start:stop], kind="mergesort")
          graph.data[start:stop] = graph.data[start:stop][order]
          graph.indices[start:stop] = graph.indices[start:stop][order]
      return graph

COMMENT:
Adding a public function to sort the graph by it's data seems reasonable to me to avoid the warning.

@TomDLT What do you think of this proposal?

Labels: New Feature, Documentation, module:neighbors
Encapsulate nearest neighbor code for `sort_sparse_row_values` in a function

### Describe the issue linked to the documentation

The [nearest neighbor code](https://github.com/scikit-learn/scikit-learn/blob/fb71eeff5a40399648d63258239079f5db7dcd76/sklearn/neighbors/_base.py#L206) often raises an `EfficiencyWarning`, particularly when using [DBSCAN](https://scikit-learn.org/stable/modules/generated/sklearn.cluster.DBSCAN.html) on sparse precomputed distance matrices. There's no proper documentation to resolve this warning, probably because there is no clean fix. You'd need to encapsulate [the code to sort the rows](https://github.com/scikit-learn/scikit-learn/blob/fb71eeff5a40399648d63258239079f5db7dcd76/sklearn/neighbors/_base.py#L206) in a function, and call that. Besides the downside of needing to maintain this low level code in one's own code base, while hoping it will stay compatible with scikit-learn's code, it's also an eye soar and something you can't document cleanly as a fix.

### Suggest a potential alternative/fix

Encapsulate the code as a function and document the `EfficiencyWarning` and its fix running the function on the matrix [here](https://scikit-learn.org/stable/modules/generated/sklearn.cluster.DBSCAN.html).

<!-- langauge: Python -->

    def sort_sparse_row_values(graph):
      # graph = graph.copy()
      row_nnz = np.diff(graph.indptr)
      if row_nnz.max() == row_nnz.min():
        n_samples = graph.shape[0]
        distances = graph.data.reshape(n_samples, -1)
    
        order = np.argsort(distances, kind="mergesort")
        order += np.arange(n_samples)[:, None] * row_nnz[0]
        order = order.ravel()
        graph.data = graph.data[order]
        graph.indices = graph.indices[order]
      else:
        for start, stop in zip(graph.indptr, graph.indptr[1:]):
          order = np.argsort(graph.data[start:stop], kind="mergesort")
          graph.data[start:stop] = graph.data[start:stop][order]
          graph.indices[start:stop] = graph.indices[start:stop][order]
      return graph

COMMENT:
It makes a lot of sense to have a public function for this. See #23139

Labels: New Feature, Documentation, module:neighbors
RuntimeError: "Cannot clone object ..." when cloning an estimator that copies parameters in either __init__ or get_params

### Describe the bug

Context: https://github.com/scikit-learn/scikit-learn/issues/15722#issuecomment-893942972

Calling `clone()` on a BaseEstimator that copies parameters results in a RuntimeError, even if the parameters are otherwise equal (`estimator.param == clone(estimator).param` returns True but `estimator.param is clone(estimator.param)` returns False).

Either the documentation has an issue in that this is an unspecified requirement for `clone()` to work (and BaseEstimator `__init__()` and `get_params()` documentation should say that parameters must always be the same object), or the equality check in `clone()` is too strict and should be loosened.

### Steps/Code to Reproduce

```python
from sklearn.base import BaseEstimator, clone

class TestEstimator(BaseEstimator):
    def __init__(self, my_dict):
        self.my_dict = my_dict.copy()

some_dict = {'foo': 'bar'}
estimator = TestEstimator(some_dict)

clone(estimator)      # raises RuntimeError: Cannot clone object TestEstimator(my_dict={'foo': 'bar'}), as the constructor either does not set or modifies parameter my_dict
```

### Expected Results

Calling `clone(estimator)` results in a new TestEstimator where the following assertions are true:

```python
from sklearn.base import BaseEstimator, clone

class TestEstimator(BaseEstimator):
    def __init__(self, my_dict):
        self.my_dict = my_dict.copy()

some_dict = {'foo': 'bar'}
estimator = TestEstimator(some_dict)

new_estimator = clone(estimator)

assert estimator is not new_estimator
assert estimator.some_dict == new_estimator.some_dict        #  this isn't strictly necessary, but if clone() is going to assert equality then this seems like the right kind of check
assert estimator.some_dict is not new_estimator.some_dict

# no RuntimeError or AssertionError should be raised after running this snippet
```

### Actual Results

```python
>>> clone(estimator)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/wchill/miniconda/envs/test38/lib/python3.8/site-packages/sklearn/base.py", line 95, in clone
    raise RuntimeError(
RuntimeError: Cannot clone object TestEstimator(my_dict={'foo': 'bar'}), as the constructor either does not set or modifies parameter my_dict
```

### Versions

```shell
>>> import sklearn; sklearn.show_versions()

System:
    python: 3.8.12 (default, Oct 12 2021, 06:23:56)  [Clang 10.0.0 ]
executable: /Users/wchill/miniconda/envs/test38/bin/python
   machine: macOS-10.16-x86_64-i386-64bit

Python dependencies:
          pip: 21.2.4
   setuptools: 58.0.4
      sklearn: 1.0.2
        numpy: 1.22.3
        scipy: 1.8.0
       Cython: None
       pandas: None
   matplotlib: None
       joblib: 1.1.0
threadpoolctl: 3.1.0

Built with OpenMP: True
```

COMMENT:
I think this would be okay.

Labels: Bug