Talip7/scikit-learn-issues-multilabel huggingface dataset:
R squared is missing in Scikit learn

### Describe the bug

For the calculation of R squared, you need to determine the Correlation coefficient, and then you need to square the result. 

Is not that R2 score with negative values... 

### Steps/Code to Reproduce

from sklearn.metrics import r2_score
y_true = [1, 2, 3]
y_pred = [3, 2, 1]
r2_score(y_true, y_pred)


### Expected Results

The results is -3!!!!

R2 should be 1.

you have a perfect correlation coeficient of -1.

The correct R2 score should be available! Or, all Scikit reputation is in stake. Not even the basics are correct... I can not trust a more complex fuction, when a basic test fails!

### Actual Results

-3.0

### Versions

```shell
import sklearn; sklearn.show_versions()
```

COMMENT:
> Well, this says alot about scikit-learn.
What is the correlation of 1 , 2 , 3 with 3 , 2 , 1?
scikit-learn answer -3
LOL

Please be respectful ([code of conduct](https://github.com/scikit-learn/scikit-learn/blob/main/CODE_OF_CONDUCT.md)) with the person that kindly takes the time to explain you what you're missing.

As explained, there are several definitions of R². The most general one can yield negative values. I suggest that you read the wikipedia page https://en.wikipedia.org/wiki/Coefficient_of_determination, there might be a thing or 2 that you were not aware about.

Labels: Bug, Needs Triage
QuantileRegressor unable to allocate memory for large datasets

### Describe the bug

I am using sklearn.linear_model.QuantileRegressor for a dataset with ~2.9 million datapoints. 
When I use it as follows, I get a memory error.

`MemoryError: Unable to allocate 61.6 TiB for an array with shape (2909376, 2909376) and data type float64`

When I do the same with statsmodel library, I don't encounter any issues. 

Would this be a known limitation of the sklarn implementation? What would be the biggest size of dataframe it can take? Or is it a bug? 

I was choosing to do it in sklearn rather than statsmodel because the sklearn API provides ability to add regularization which the other library doesn't. 

### Steps/Code to Reproduce

```python
from sklearn.linear_model import QuantileRegressor
reg = QuantileRegressor(quantile=0.8).fit(X, y)
```

### Expected Results

The data is fit without errors. 

### Actual Results
```
---------------------------------------------------------------------------
MemoryError                               Traceback (most recent call last)
/tmp/ipykernel_53907/1165686720.py in <module>
      1 from sklearn.linear_model import QuantileRegressor
      2 
----> 3 reg = QuantileRegressor(quantile=0.8).fit(X, y)
      4 print_analysis(reg, X, y, flyte_type)

/anaconda3/lib/python3.7/site-packages/sklearn/linear_model/_quantile.py in fit(self, X, y, sample_weight)
    239                     -np.ones((n_mask, 1)),
    240                     -X[mask],
--> 241                     np.eye(n_mask),
    242                     -np.eye(n_mask),
    243                 ],

/anaconda3/lib/python3.7/site-packages/numpy/lib/twodim_base.py in eye(N, M, k, dtype, order)
    197     if M is None:
    198         M = N
--> 199     m = zeros((N, M), dtype=dtype, order=order)
    200     if k >= M:
    201         return m

MemoryError: Unable to allocate 61.6 TiB for an array with shape (2909376, 2909376) and data type float64
```

### Versions

```shell
System:
    python: 3.7.7 (default, Mar 26 2020, 15:48:22)  [GCC 7.3.0]
executable: /anaconda3/bin/python
   machine: Linux-4.14.68-4.14.3-amd64-3adf3675665129fa-x86_64-with-debian-stretch-sid

Python dependencies:
          pip: 21.2.2
   setuptools: 58.0.4
      sklearn: 1.0.1
        numpy: 1.19.2
        scipy: 1.5.2
       Cython: 0.29.25
       pandas: 1.2.4
   matplotlib: 3.5.0
       joblib: 1.1.0
threadpoolctl: 2.2.0

Built with OpenMP: True
```

COMMENT:
Could you convert your data into `sparse` format and see if it would be more memory efficient when building these matrices.

Labels: Bug, module:linear_model
QuantileRegressor unable to allocate memory for large datasets

### Describe the bug

I am using sklearn.linear_model.QuantileRegressor for a dataset with ~2.9 million datapoints. 
When I use it as follows, I get a memory error.

`MemoryError: Unable to allocate 61.6 TiB for an array with shape (2909376, 2909376) and data type float64`

When I do the same with statsmodel library, I don't encounter any issues. 

Would this be a known limitation of the sklarn implementation? What would be the biggest size of dataframe it can take? Or is it a bug? 

I was choosing to do it in sklearn rather than statsmodel because the sklearn API provides ability to add regularization which the other library doesn't. 

### Steps/Code to Reproduce

```python
from sklearn.linear_model import QuantileRegressor
reg = QuantileRegressor(quantile=0.8).fit(X, y)
```

### Expected Results

The data is fit without errors. 

### Actual Results
```
---------------------------------------------------------------------------
MemoryError                               Traceback (most recent call last)
/tmp/ipykernel_53907/1165686720.py in <module>
      1 from sklearn.linear_model import QuantileRegressor
      2 
----> 3 reg = QuantileRegressor(quantile=0.8).fit(X, y)
      4 print_analysis(reg, X, y, flyte_type)

/anaconda3/lib/python3.7/site-packages/sklearn/linear_model/_quantile.py in fit(self, X, y, sample_weight)
    239                     -np.ones((n_mask, 1)),
    240                     -X[mask],
--> 241                     np.eye(n_mask),
    242                     -np.eye(n_mask),
    243                 ],

/anaconda3/lib/python3.7/site-packages/numpy/lib/twodim_base.py in eye(N, M, k, dtype, order)
    197     if M is None:
    198         M = N
--> 199     m = zeros((N, M), dtype=dtype, order=order)
    200     if k >= M:
    201         return m

MemoryError: Unable to allocate 61.6 TiB for an array with shape (2909376, 2909376) and data type float64
```

### Versions

```shell
System:
    python: 3.7.7 (default, Mar 26 2020, 15:48:22)  [GCC 7.3.0]
executable: /anaconda3/bin/python
   machine: Linux-4.14.68-4.14.3-amd64-3adf3675665129fa-x86_64-with-debian-stretch-sid

Python dependencies:
          pip: 21.2.2
   setuptools: 58.0.4
      sklearn: 1.0.1
        numpy: 1.19.2
        scipy: 1.5.2
       Cython: 0.29.25
       pandas: 1.2.4
   matplotlib: 3.5.0
       joblib: 1.1.0
threadpoolctl: 2.2.0

Built with OpenMP: True
```

COMMENT:
The estimator does not use the same approach in scikit-learn and statsmodels: in scikit-learn we used a linear programming solver while statsmodels uses iterative reweighted least squares.

@agramfort @lorentzenchr what are the pros/cons of linear programming vs. IRLS?

Labels: Bug, module:linear_model
QuantileRegressor unable to allocate memory for large datasets

### Describe the bug

I am using sklearn.linear_model.QuantileRegressor for a dataset with ~2.9 million datapoints. 
When I use it as follows, I get a memory error.

`MemoryError: Unable to allocate 61.6 TiB for an array with shape (2909376, 2909376) and data type float64`

When I do the same with statsmodel library, I don't encounter any issues. 

Would this be a known limitation of the sklarn implementation? What would be the biggest size of dataframe it can take? Or is it a bug? 

I was choosing to do it in sklearn rather than statsmodel because the sklearn API provides ability to add regularization which the other library doesn't. 

### Steps/Code to Reproduce

```python
from sklearn.linear_model import QuantileRegressor
reg = QuantileRegressor(quantile=0.8).fit(X, y)
```

### Expected Results

The data is fit without errors. 

### Actual Results
```
---------------------------------------------------------------------------
MemoryError                               Traceback (most recent call last)
/tmp/ipykernel_53907/1165686720.py in <module>
      1 from sklearn.linear_model import QuantileRegressor
      2 
----> 3 reg = QuantileRegressor(quantile=0.8).fit(X, y)
      4 print_analysis(reg, X, y, flyte_type)

/anaconda3/lib/python3.7/site-packages/sklearn/linear_model/_quantile.py in fit(self, X, y, sample_weight)
    239                     -np.ones((n_mask, 1)),
    240                     -X[mask],
--> 241                     np.eye(n_mask),
    242                     -np.eye(n_mask),
    243                 ],

/anaconda3/lib/python3.7/site-packages/numpy/lib/twodim_base.py in eye(N, M, k, dtype, order)
    197     if M is None:
    198         M = N
--> 199     m = zeros((N, M), dtype=dtype, order=order)
    200     if k >= M:
    201         return m

MemoryError: Unable to allocate 61.6 TiB for an array with shape (2909376, 2909376) and data type float64
```

### Versions

```shell
System:
    python: 3.7.7 (default, Mar 26 2020, 15:48:22)  [GCC 7.3.0]
executable: /anaconda3/bin/python
   machine: Linux-4.14.68-4.14.3-amd64-3adf3675665129fa-x86_64-with-debian-stretch-sid

Python dependencies:
          pip: 21.2.2
   setuptools: 58.0.4
      sklearn: 1.0.1
        numpy: 1.19.2
        scipy: 1.5.2
       Cython: 0.29.25
       pandas: 1.2.4
   matplotlib: 3.5.0
       joblib: 1.1.0
threadpoolctl: 2.2.0

Built with OpenMP: True
```

COMMENT:
> Would this be a known limitation of the sklarn implementation?

For the time being: Yes.
As @glemaitre pointed out, if your data is sparse, it might help to work with sparse data only. Also, with newer scipy versions, you might want to use `QuantileRegressor(solver="highs")`.

> what are the pros/cons of linear programming vs. IRLS?

Linear quantile regression is a harder problem than OLS. The underlying optimization problem is a linear programming problem. As it is a non-smooth problem, any approach using gradients (and hessians) need to deal with this in some way which typical leads to less precise results.

Labels: Bug, module:linear_model
QuantileRegressor unable to allocate memory for large datasets

### Describe the bug

I am using sklearn.linear_model.QuantileRegressor for a dataset with ~2.9 million datapoints. 
When I use it as follows, I get a memory error.

`MemoryError: Unable to allocate 61.6 TiB for an array with shape (2909376, 2909376) and data type float64`

When I do the same with statsmodel library, I don't encounter any issues. 

Would this be a known limitation of the sklarn implementation? What would be the biggest size of dataframe it can take? Or is it a bug? 

I was choosing to do it in sklearn rather than statsmodel because the sklearn API provides ability to add regularization which the other library doesn't. 

### Steps/Code to Reproduce

```python
from sklearn.linear_model import QuantileRegressor
reg = QuantileRegressor(quantile=0.8).fit(X, y)
```

### Expected Results

The data is fit without errors. 

### Actual Results
```
---------------------------------------------------------------------------
MemoryError                               Traceback (most recent call last)
/tmp/ipykernel_53907/1165686720.py in <module>
      1 from sklearn.linear_model import QuantileRegressor
      2 
----> 3 reg = QuantileRegressor(quantile=0.8).fit(X, y)
      4 print_analysis(reg, X, y, flyte_type)

/anaconda3/lib/python3.7/site-packages/sklearn/linear_model/_quantile.py in fit(self, X, y, sample_weight)
    239                     -np.ones((n_mask, 1)),
    240                     -X[mask],
--> 241                     np.eye(n_mask),
    242                     -np.eye(n_mask),
    243                 ],

/anaconda3/lib/python3.7/site-packages/numpy/lib/twodim_base.py in eye(N, M, k, dtype, order)
    197     if M is None:
    198         M = N
--> 199     m = zeros((N, M), dtype=dtype, order=order)
    200     if k >= M:
    201         return m

MemoryError: Unable to allocate 61.6 TiB for an array with shape (2909376, 2909376) and data type float64
```

### Versions

```shell
System:
    python: 3.7.7 (default, Mar 26 2020, 15:48:22)  [GCC 7.3.0]
executable: /anaconda3/bin/python
   machine: Linux-4.14.68-4.14.3-amd64-3adf3675665129fa-x86_64-with-debian-stretch-sid

Python dependencies:
          pip: 21.2.2
   setuptools: 58.0.4
      sklearn: 1.0.1
        numpy: 1.19.2
        scipy: 1.5.2
       Cython: 0.29.25
       pandas: 1.2.4
   matplotlib: 3.5.0
       joblib: 1.1.0
threadpoolctl: 2.2.0

Built with OpenMP: True
```

COMMENT:
I just wrote a proof of concept IRLS solver at:

https://github.com/benchopt/benchmark_quantile_regression/pull/4

you can see the benchmark result at:

https://benchopt.github.io/results/benchmark_quantile_regression_benchmark_quantile_regression_benchopt_run_2022-03-26_15h44m40.html

Labels: Bug, module:linear_model
QuantileRegressor unable to allocate memory for large datasets

### Describe the bug

I am using sklearn.linear_model.QuantileRegressor for a dataset with ~2.9 million datapoints. 
When I use it as follows, I get a memory error.

`MemoryError: Unable to allocate 61.6 TiB for an array with shape (2909376, 2909376) and data type float64`

When I do the same with statsmodel library, I don't encounter any issues. 

Would this be a known limitation of the sklarn implementation? What would be the biggest size of dataframe it can take? Or is it a bug? 

I was choosing to do it in sklearn rather than statsmodel because the sklearn API provides ability to add regularization which the other library doesn't. 

### Steps/Code to Reproduce

```python
from sklearn.linear_model import QuantileRegressor
reg = QuantileRegressor(quantile=0.8).fit(X, y)
```

### Expected Results

The data is fit without errors. 

### Actual Results
```
---------------------------------------------------------------------------
MemoryError                               Traceback (most recent call last)
/tmp/ipykernel_53907/1165686720.py in <module>
      1 from sklearn.linear_model import QuantileRegressor
      2 
----> 3 reg = QuantileRegressor(quantile=0.8).fit(X, y)
      4 print_analysis(reg, X, y, flyte_type)

/anaconda3/lib/python3.7/site-packages/sklearn/linear_model/_quantile.py in fit(self, X, y, sample_weight)
    239                     -np.ones((n_mask, 1)),
    240                     -X[mask],
--> 241                     np.eye(n_mask),
    242                     -np.eye(n_mask),
    243                 ],

/anaconda3/lib/python3.7/site-packages/numpy/lib/twodim_base.py in eye(N, M, k, dtype, order)
    197     if M is None:
    198         M = N
--> 199     m = zeros((N, M), dtype=dtype, order=order)
    200     if k >= M:
    201         return m

MemoryError: Unable to allocate 61.6 TiB for an array with shape (2909376, 2909376) and data type float64
```

### Versions

```shell
System:
    python: 3.7.7 (default, Mar 26 2020, 15:48:22)  [GCC 7.3.0]
executable: /anaconda3/bin/python
   machine: Linux-4.14.68-4.14.3-amd64-3adf3675665129fa-x86_64-with-debian-stretch-sid

Python dependencies:
          pip: 21.2.2
   setuptools: 58.0.4
      sklearn: 1.0.1
        numpy: 1.19.2
        scipy: 1.5.2
       Cython: 0.29.25
       pandas: 1.2.4
   matplotlib: 3.5.0
       joblib: 1.1.0
threadpoolctl: 2.2.0

Built with OpenMP: True
```

COMMENT:
> As @glemaitre pointed out, if your data is sparse, it might help to work with sparse data only. 
My data has multiple columns. 
```python
X.shape
(2903970, 12)
```
I wouldn't say my data has too many 0s but it could have quite some number of them.

> Also, with newer scipy versions, you might want to use QuantileRegressor(solver="highs").

I upgraded to scipy to 1.7.3 (since solver="highs" needed >=1.6.0) and I still get the same Unable to allocate memory error.

Labels: Bug, module:linear_model
QuantileRegressor unable to allocate memory for large datasets

### Describe the bug

I am using sklearn.linear_model.QuantileRegressor for a dataset with ~2.9 million datapoints. 
When I use it as follows, I get a memory error.

`MemoryError: Unable to allocate 61.6 TiB for an array with shape (2909376, 2909376) and data type float64`

When I do the same with statsmodel library, I don't encounter any issues. 

Would this be a known limitation of the sklarn implementation? What would be the biggest size of dataframe it can take? Or is it a bug? 

I was choosing to do it in sklearn rather than statsmodel because the sklearn API provides ability to add regularization which the other library doesn't. 

### Steps/Code to Reproduce

```python
from sklearn.linear_model import QuantileRegressor
reg = QuantileRegressor(quantile=0.8).fit(X, y)
```

### Expected Results

The data is fit without errors. 

### Actual Results
```
---------------------------------------------------------------------------
MemoryError                               Traceback (most recent call last)
/tmp/ipykernel_53907/1165686720.py in <module>
      1 from sklearn.linear_model import QuantileRegressor
      2 
----> 3 reg = QuantileRegressor(quantile=0.8).fit(X, y)
      4 print_analysis(reg, X, y, flyte_type)

/anaconda3/lib/python3.7/site-packages/sklearn/linear_model/_quantile.py in fit(self, X, y, sample_weight)
    239                     -np.ones((n_mask, 1)),
    240                     -X[mask],
--> 241                     np.eye(n_mask),
    242                     -np.eye(n_mask),
    243                 ],

/anaconda3/lib/python3.7/site-packages/numpy/lib/twodim_base.py in eye(N, M, k, dtype, order)
    197     if M is None:
    198         M = N
--> 199     m = zeros((N, M), dtype=dtype, order=order)
    200     if k >= M:
    201         return m

MemoryError: Unable to allocate 61.6 TiB for an array with shape (2909376, 2909376) and data type float64
```

### Versions

```shell
System:
    python: 3.7.7 (default, Mar 26 2020, 15:48:22)  [GCC 7.3.0]
executable: /anaconda3/bin/python
   machine: Linux-4.14.68-4.14.3-amd64-3adf3675665129fa-x86_64-with-debian-stretch-sid

Python dependencies:
          pip: 21.2.2
   setuptools: 58.0.4
      sklearn: 1.0.1
        numpy: 1.19.2
        scipy: 1.5.2
       Cython: 0.29.25
       pandas: 1.2.4
   matplotlib: 3.5.0
       joblib: 1.1.0
threadpoolctl: 2.2.0

Built with OpenMP: True
```

COMMENT:
```python
from scipy import sparse

X_sparse = sparse.csc_matrix(X)
```
and then fit on `X_sparse` instead of `X`. This is only a hunch that is likely to fail, too - but worth a try.

Labels: Bug, module:linear_model
QuantileRegressor unable to allocate memory for large datasets

### Describe the bug

I am using sklearn.linear_model.QuantileRegressor for a dataset with ~2.9 million datapoints. 
When I use it as follows, I get a memory error.

`MemoryError: Unable to allocate 61.6 TiB for an array with shape (2909376, 2909376) and data type float64`

When I do the same with statsmodel library, I don't encounter any issues. 

Would this be a known limitation of the sklarn implementation? What would be the biggest size of dataframe it can take? Or is it a bug? 

I was choosing to do it in sklearn rather than statsmodel because the sklearn API provides ability to add regularization which the other library doesn't. 

### Steps/Code to Reproduce

```python
from sklearn.linear_model import QuantileRegressor
reg = QuantileRegressor(quantile=0.8).fit(X, y)
```

### Expected Results

The data is fit without errors. 

### Actual Results
```
---------------------------------------------------------------------------
MemoryError                               Traceback (most recent call last)
/tmp/ipykernel_53907/1165686720.py in <module>
      1 from sklearn.linear_model import QuantileRegressor
      2 
----> 3 reg = QuantileRegressor(quantile=0.8).fit(X, y)
      4 print_analysis(reg, X, y, flyte_type)

/anaconda3/lib/python3.7/site-packages/sklearn/linear_model/_quantile.py in fit(self, X, y, sample_weight)
    239                     -np.ones((n_mask, 1)),
    240                     -X[mask],
--> 241                     np.eye(n_mask),
    242                     -np.eye(n_mask),
    243                 ],

/anaconda3/lib/python3.7/site-packages/numpy/lib/twodim_base.py in eye(N, M, k, dtype, order)
    197     if M is None:
    198         M = N
--> 199     m = zeros((N, M), dtype=dtype, order=order)
    200     if k >= M:
    201         return m

MemoryError: Unable to allocate 61.6 TiB for an array with shape (2909376, 2909376) and data type float64
```

### Versions

```shell
System:
    python: 3.7.7 (default, Mar 26 2020, 15:48:22)  [GCC 7.3.0]
executable: /anaconda3/bin/python
   machine: Linux-4.14.68-4.14.3-amd64-3adf3675665129fa-x86_64-with-debian-stretch-sid

Python dependencies:
          pip: 21.2.2
   setuptools: 58.0.4
      sklearn: 1.0.1
        numpy: 1.19.2
        scipy: 1.5.2
       Cython: 0.29.25
       pandas: 1.2.4
   matplotlib: 3.5.0
       joblib: 1.1.0
threadpoolctl: 2.2.0

Built with OpenMP: True
```

COMMENT:
Tried it and failed with the same TypeError:
```python
X_sparse = sparse.csc_matrix(X)
qr = QuantileRegressor(quantile=0.5, alpha=0.1, solver="highs")
qr.fit(X_sparse, y)
```

`TypeError: A sparse matrix was passed, but dense data is required. Use X.toarray() to convert to a dense numpy array.`

Labels: Bug, module:linear_model