Talip7/scikit-learn-issues-multilabel huggingface dataset:
ENH: use the sparse-sparse backend for computing pairwise distance

First reported in: https://github.com/scikit-learn-contrib/imbalanced-learn/issues/1056

We have a regression in `kneighbors` with sparse matrix from 1.1.X to 1.3.X.
A code sample to reproduce:

```python
# %%
import sklearn
sklearn.__version__

# %%
import numpy as np
from scipy import sparse
from sklearn.neighbors import KNeighborsRegressor

n_samples, n_features = 1_000, 10_000
X = sparse.random(n_samples, n_features, density=0.01, format="csr", random_state=0)
rng = np.random.default_rng(0)
y = rng.integers(0, 2, size=n_samples)
knn = KNeighborsRegressor(n_neighbors=5).fit(X, y)

# %%
%%timeit
knn.kneighbors(X, return_distance=False)
```

#### 1.1.X

```
21.5 ms ± 217 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)
```

#### `main`

```
1.16 s ± 9.87 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
```

#### Small benchmark

![image](https://github.com/scikit-learn/scikit-learn/assets/7454015/5abe644f-5c05-4124-ab43-dd0a1cdd3e58)

<details>

```python
# %%
import sklearn
sklearn.__version__

# %%
import time
from collections import defaultdict
import numpy as np
from scipy import sparse
from sklearn.neighbors import KNeighborsRegressor

n_samples, n_features = 1_000, [500, 1_000, 2_500, 5_000, 7_500, 10_000, 25_000, 50_000]

results = defaultdict(list)
for nf in n_features:
    X = sparse.random(n_samples, nf, density=0.01, format="csr", random_state=0)
    rng = np.random.default_rng(0)
    y = rng.integers(0, 2, size=n_samples)
    knn = KNeighborsRegressor(n_neighbors=5).fit(X, y)
    start = time.time()
    knn.kneighbors(X, return_distance=False)
    elapsed_time = time.time() - start
    results["version"].append(sklearn.__version__)
    results["n_features"].append(nf)
    results["elapsed_time"].append(elapsed_time)

# %%
import pandas as pd

results = pd.DataFrame(results)
results.to_csv(f"bench_{sklearn.__version__}.csv", index=False)
```

```python
# %%
import pandas as pd

results_main = pd.read_csv(f"bench_1.5.dev0.csv")
results_1_1 = pd.read_csv(f"bench_1.1.3.csv")
results = pd.concat([results_main, results_1_1], axis=0)
results

# %%
import seaborn as sns

sns.set_context("talk")

# %%
sns.relplot(
    data=results,
    x="n_features",
    y="elapsed_time",
    hue="version",
    kind="line",
    height=6,
    aspect=1.5,
    legend=True,
)
sns.despine(offset=10, trim=True)
```

</details>

#### Debug profiling

My first investigation look like we are spending all our time in the following function:

https://github.com/scikit-learn/scikit-learn/blob/5c7e831306e0a087c2b6af6913fa5b3c402f6d67/sklearn/metrics/_pairwise_distances_reduction/_middle_term_computer.pyx.tp#L39-L69

But I'm a bit rusty with profiling native code. I need a bit more time to investigate.

COMMENT:
A quick fix is to change the definition of `is_usable_for` to exclude this case.

Labels: Enhancement
ENH: use the sparse-sparse backend for computing pairwise distance

First reported in: https://github.com/scikit-learn-contrib/imbalanced-learn/issues/1056

We have a regression in `kneighbors` with sparse matrix from 1.1.X to 1.3.X.
A code sample to reproduce:

```python
# %%
import sklearn
sklearn.__version__

# %%
import numpy as np
from scipy import sparse
from sklearn.neighbors import KNeighborsRegressor

n_samples, n_features = 1_000, 10_000
X = sparse.random(n_samples, n_features, density=0.01, format="csr", random_state=0)
rng = np.random.default_rng(0)
y = rng.integers(0, 2, size=n_samples)
knn = KNeighborsRegressor(n_neighbors=5).fit(X, y)

# %%
%%timeit
knn.kneighbors(X, return_distance=False)
```

#### 1.1.X

```
21.5 ms ± 217 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)
```

#### `main`

```
1.16 s ± 9.87 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
```

#### Small benchmark

![image](https://github.com/scikit-learn/scikit-learn/assets/7454015/5abe644f-5c05-4124-ab43-dd0a1cdd3e58)

<details>

```python
# %%
import sklearn
sklearn.__version__

# %%
import time
from collections import defaultdict
import numpy as np
from scipy import sparse
from sklearn.neighbors import KNeighborsRegressor

n_samples, n_features = 1_000, [500, 1_000, 2_500, 5_000, 7_500, 10_000, 25_000, 50_000]

results = defaultdict(list)
for nf in n_features:
    X = sparse.random(n_samples, nf, density=0.01, format="csr", random_state=0)
    rng = np.random.default_rng(0)
    y = rng.integers(0, 2, size=n_samples)
    knn = KNeighborsRegressor(n_neighbors=5).fit(X, y)
    start = time.time()
    knn.kneighbors(X, return_distance=False)
    elapsed_time = time.time() - start
    results["version"].append(sklearn.__version__)
    results["n_features"].append(nf)
    results["elapsed_time"].append(elapsed_time)

# %%
import pandas as pd

results = pd.DataFrame(results)
results.to_csv(f"bench_{sklearn.__version__}.csv", index=False)
```

```python
# %%
import pandas as pd

results_main = pd.read_csv(f"bench_1.5.dev0.csv")
results_1_1 = pd.read_csv(f"bench_1.1.3.csv")
results = pd.concat([results_main, results_1_1], axis=0)
results

# %%
import seaborn as sns

sns.set_context("talk")

# %%
sns.relplot(
    data=results,
    x="n_features",
    y="elapsed_time",
    hue="version",
    kind="line",
    height=6,
    aspect=1.5,
    legend=True,
)
sns.despine(offset=10, trim=True)
```

</details>

#### Debug profiling

My first investigation look like we are spending all our time in the following function:

https://github.com/scikit-learn/scikit-learn/blob/5c7e831306e0a087c2b6af6913fa5b3c402f6d67/sklearn/metrics/_pairwise_distances_reduction/_middle_term_computer.pyx.tp#L39-L69

But I'm a bit rusty with profiling native code. I need a bit more time to investigate.

COMMENT:
[`csr_matmat`](https://github.com/scipy/scipy/blob/23eee16a3acf654ae254fbd3a9a57975fa3272e0/scipy/sparse/sparsetools/csr.h#L608) from SciPy was called before and likely has a better algorithm.

Labels: Enhancement
Exception in LogisticRegressionCV

### Describe the bug

The code provided below raises ValueError. I guess that the problem is that minor classes may not be included in **train** or **val** sets for some folds during internal cross-validation, even with stratified split. This produces errors with some metrics other than default (accuracy).

One solution may be setting log-proba to -inf for classes not present in the train set, as well as providing label argument. How can I fix this in the most simple way?

### Steps/Code to Reproduce

```
import numpy as np
from sklearn.linear_model import LogisticRegressionCV
X = np.zeros((10, 1))
y = [1, 1, 1, 1, 1, 2, 2, 2, 2, 3]
logreg = LogisticRegressionCV(cv=5, scoring='neg_log_loss')
logreg.fit(X, y)
```

### Expected Results

No exception thrown

### Actual Results

ValueError: y_true and y_pred contain different number of classes 2, 3. Please provide the true labels explicitly through the labels argument. Classes found in y_true: [0 1]

### Versions

```shell
System:
    python: 3.10.8 (main, Nov  2 2023, 15:57:09) [GCC 9.4.0]
executable: /data/osedukhin/tabular-models/venv/bin/python
   machine: Linux-5.4.0-123-generic-x86_64-with-glibc2.31

Python dependencies:
      sklearn: 1.4.0
          pip: 22.2.2
   setuptools: 63.2.0
        numpy: 1.26.2
        scipy: 1.11.3
       Cython: None
       pandas: 2.1.3
   matplotlib: 3.8.1
       joblib: 1.3.2
threadpoolctl: 3.2.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: openblas
    num_threads: 32
         prefix: libopenblas
       filepath: /data/osedukhin/tabular-models/venv/lib/python3.10/site-packages/numpy.libs/libopenblas64_p-r0-0cf96a72.3.23.dev.so
        version: 0.3.23.dev
threading_layer: pthreads
   architecture: SkylakeX

       user_api: openmp
   internal_api: openmp
    num_threads: 32
         prefix: libgomp
       filepath: /data/osedukhin/tabular-models/venv/lib/python3.10/site-packages/scikit_learn.libs/libgomp-a34b3233.so.1.0.0
        version: None

       user_api: blas
   internal_api: openblas
    num_threads: 32
         prefix: libopenblas
       filepath: /data/osedukhin/tabular-models/venv/lib/python3.10/site-packages/scipy.libs/libopenblasp-r0-23e5df77.3.21.dev.so
        version: 0.3.21.dev
threading_layer: pthreads
   architecture: SkylakeX
```

COMMENT:
Found the same issue some years ago: https://github.com/scikit-learn/scikit-learn/issues/15389

Labels: Bug
Exception in LogisticRegressionCV

### Describe the bug

The code provided below raises ValueError. I guess that the problem is that minor classes may not be included in **train** or **val** sets for some folds during internal cross-validation, even with stratified split. This produces errors with some metrics other than default (accuracy).

One solution may be setting log-proba to -inf for classes not present in the train set, as well as providing label argument. How can I fix this in the most simple way?

### Steps/Code to Reproduce

```
import numpy as np
from sklearn.linear_model import LogisticRegressionCV
X = np.zeros((10, 1))
y = [1, 1, 1, 1, 1, 2, 2, 2, 2, 3]
logreg = LogisticRegressionCV(cv=5, scoring='neg_log_loss')
logreg.fit(X, y)
```

### Expected Results

No exception thrown

### Actual Results

ValueError: y_true and y_pred contain different number of classes 2, 3. Please provide the true labels explicitly through the labels argument. Classes found in y_true: [0 1]

### Versions

```shell
System:
    python: 3.10.8 (main, Nov  2 2023, 15:57:09) [GCC 9.4.0]
executable: /data/osedukhin/tabular-models/venv/bin/python
   machine: Linux-5.4.0-123-generic-x86_64-with-glibc2.31

Python dependencies:
      sklearn: 1.4.0
          pip: 22.2.2
   setuptools: 63.2.0
        numpy: 1.26.2
        scipy: 1.11.3
       Cython: None
       pandas: 2.1.3
   matplotlib: 3.8.1
       joblib: 1.3.2
threadpoolctl: 3.2.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: openblas
    num_threads: 32
         prefix: libopenblas
       filepath: /data/osedukhin/tabular-models/venv/lib/python3.10/site-packages/numpy.libs/libopenblas64_p-r0-0cf96a72.3.23.dev.so
        version: 0.3.23.dev
threading_layer: pthreads
   architecture: SkylakeX

       user_api: openmp
   internal_api: openmp
    num_threads: 32
         prefix: libgomp
       filepath: /data/osedukhin/tabular-models/venv/lib/python3.10/site-packages/scikit_learn.libs/libgomp-a34b3233.so.1.0.0
        version: None

       user_api: blas
   internal_api: openblas
    num_threads: 32
         prefix: libopenblas
       filepath: /data/osedukhin/tabular-models/venv/lib/python3.10/site-packages/scipy.libs/libopenblasp-r0-23e5df77.3.21.dev.so
        version: 0.3.21.dev
threading_layer: pthreads
   architecture: SkylakeX
```

COMMENT:
@glemaitre mentioned that this can be fixed with:

```
scoring = make_scorer(
      log_loss, greater_is_better=False, labels=labels, needs_proba=True
)
```

Does labels order matter? For example, if y = [2, 2, 1, 1, 0, 0], then will passing labels=[0, 1, 2] and labels=[2, 1, 0] work correctly?

Labels: Bug
Exception in LogisticRegressionCV

### Describe the bug

The code provided below raises ValueError. I guess that the problem is that minor classes may not be included in **train** or **val** sets for some folds during internal cross-validation, even with stratified split. This produces errors with some metrics other than default (accuracy).

One solution may be setting log-proba to -inf for classes not present in the train set, as well as providing label argument. How can I fix this in the most simple way?

### Steps/Code to Reproduce

```
import numpy as np
from sklearn.linear_model import LogisticRegressionCV
X = np.zeros((10, 1))
y = [1, 1, 1, 1, 1, 2, 2, 2, 2, 3]
logreg = LogisticRegressionCV(cv=5, scoring='neg_log_loss')
logreg.fit(X, y)
```

### Expected Results

No exception thrown

### Actual Results

ValueError: y_true and y_pred contain different number of classes 2, 3. Please provide the true labels explicitly through the labels argument. Classes found in y_true: [0 1]

### Versions

```shell
System:
    python: 3.10.8 (main, Nov  2 2023, 15:57:09) [GCC 9.4.0]
executable: /data/osedukhin/tabular-models/venv/bin/python
   machine: Linux-5.4.0-123-generic-x86_64-with-glibc2.31

Python dependencies:
      sklearn: 1.4.0
          pip: 22.2.2
   setuptools: 63.2.0
        numpy: 1.26.2
        scipy: 1.11.3
       Cython: None
       pandas: 2.1.3
   matplotlib: 3.8.1
       joblib: 1.3.2
threadpoolctl: 3.2.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: openblas
    num_threads: 32
         prefix: libopenblas
       filepath: /data/osedukhin/tabular-models/venv/lib/python3.10/site-packages/numpy.libs/libopenblas64_p-r0-0cf96a72.3.23.dev.so
        version: 0.3.23.dev
threading_layer: pthreads
   architecture: SkylakeX

       user_api: openmp
   internal_api: openmp
    num_threads: 32
         prefix: libgomp
       filepath: /data/osedukhin/tabular-models/venv/lib/python3.10/site-packages/scikit_learn.libs/libgomp-a34b3233.so.1.0.0
        version: None

       user_api: blas
   internal_api: openblas
    num_threads: 32
         prefix: libopenblas
       filepath: /data/osedukhin/tabular-models/venv/lib/python3.10/site-packages/scipy.libs/libopenblasp-r0-23e5df77.3.21.dev.so
        version: 0.3.21.dev
threading_layer: pthreads
   architecture: SkylakeX
```

COMMENT:
log_loss documentation says that "The labels in y_pred are assumed to be ordered alphabetically, as done by LabelBinarizer". Does this mean lexicographical sorting, even for integer labels? For example, is [0, 10, 9] a correct sorting?

Labels: Bug
Exception in LogisticRegressionCV

### Describe the bug

The code provided below raises ValueError. I guess that the problem is that minor classes may not be included in **train** or **val** sets for some folds during internal cross-validation, even with stratified split. This produces errors with some metrics other than default (accuracy).

One solution may be setting log-proba to -inf for classes not present in the train set, as well as providing label argument. How can I fix this in the most simple way?

### Steps/Code to Reproduce

```
import numpy as np
from sklearn.linear_model import LogisticRegressionCV
X = np.zeros((10, 1))
y = [1, 1, 1, 1, 1, 2, 2, 2, 2, 3]
logreg = LogisticRegressionCV(cv=5, scoring='neg_log_loss')
logreg.fit(X, y)
```

### Expected Results

No exception thrown

### Actual Results

ValueError: y_true and y_pred contain different number of classes 2, 3. Please provide the true labels explicitly through the labels argument. Classes found in y_true: [0 1]

### Versions

```shell
System:
    python: 3.10.8 (main, Nov  2 2023, 15:57:09) [GCC 9.4.0]
executable: /data/osedukhin/tabular-models/venv/bin/python
   machine: Linux-5.4.0-123-generic-x86_64-with-glibc2.31

Python dependencies:
      sklearn: 1.4.0
          pip: 22.2.2
   setuptools: 63.2.0
        numpy: 1.26.2
        scipy: 1.11.3
       Cython: None
       pandas: 2.1.3
   matplotlib: 3.8.1
       joblib: 1.3.2
threadpoolctl: 3.2.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: openblas
    num_threads: 32
         prefix: libopenblas
       filepath: /data/osedukhin/tabular-models/venv/lib/python3.10/site-packages/numpy.libs/libopenblas64_p-r0-0cf96a72.3.23.dev.so
        version: 0.3.23.dev
threading_layer: pthreads
   architecture: SkylakeX

       user_api: openmp
   internal_api: openmp
    num_threads: 32
         prefix: libgomp
       filepath: /data/osedukhin/tabular-models/venv/lib/python3.10/site-packages/scikit_learn.libs/libgomp-a34b3233.so.1.0.0
        version: None

       user_api: blas
   internal_api: openblas
    num_threads: 32
         prefix: libopenblas
       filepath: /data/osedukhin/tabular-models/venv/lib/python3.10/site-packages/scipy.libs/libopenblasp-r0-23e5df77.3.21.dev.so
        version: 0.3.21.dev
threading_layer: pthreads
   architecture: SkylakeX
```

COMMENT:
@glemaitre is it possible to append zero output probabilities for classes not present in the train?

Labels: Bug
Exception in LogisticRegressionCV

### Describe the bug

The code provided below raises ValueError. I guess that the problem is that minor classes may not be included in **train** or **val** sets for some folds during internal cross-validation, even with stratified split. This produces errors with some metrics other than default (accuracy).

One solution may be setting log-proba to -inf for classes not present in the train set, as well as providing label argument. How can I fix this in the most simple way?

### Steps/Code to Reproduce

```
import numpy as np
from sklearn.linear_model import LogisticRegressionCV
X = np.zeros((10, 1))
y = [1, 1, 1, 1, 1, 2, 2, 2, 2, 3]
logreg = LogisticRegressionCV(cv=5, scoring='neg_log_loss')
logreg.fit(X, y)
```

### Expected Results

No exception thrown

### Actual Results

ValueError: y_true and y_pred contain different number of classes 2, 3. Please provide the true labels explicitly through the labels argument. Classes found in y_true: [0 1]

### Versions

```shell
System:
    python: 3.10.8 (main, Nov  2 2023, 15:57:09) [GCC 9.4.0]
executable: /data/osedukhin/tabular-models/venv/bin/python
   machine: Linux-5.4.0-123-generic-x86_64-with-glibc2.31

Python dependencies:
      sklearn: 1.4.0
          pip: 22.2.2
   setuptools: 63.2.0
        numpy: 1.26.2
        scipy: 1.11.3
       Cython: None
       pandas: 2.1.3
   matplotlib: 3.8.1
       joblib: 1.3.2
threadpoolctl: 3.2.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: openblas
    num_threads: 32
         prefix: libopenblas
       filepath: /data/osedukhin/tabular-models/venv/lib/python3.10/site-packages/numpy.libs/libopenblas64_p-r0-0cf96a72.3.23.dev.so
        version: 0.3.23.dev
threading_layer: pthreads
   architecture: SkylakeX

       user_api: openmp
   internal_api: openmp
    num_threads: 32
         prefix: libgomp
       filepath: /data/osedukhin/tabular-models/venv/lib/python3.10/site-packages/scikit_learn.libs/libgomp-a34b3233.so.1.0.0
        version: None

       user_api: blas
   internal_api: openblas
    num_threads: 32
         prefix: libopenblas
       filepath: /data/osedukhin/tabular-models/venv/lib/python3.10/site-packages/scipy.libs/libopenblasp-r0-23e5df77.3.21.dev.so
        version: 0.3.21.dev
threading_layer: pthreads
   architecture: SkylakeX
```

COMMENT:
I don't see an easy way to do so without being a hack.

Labels: Bug
Exception in LogisticRegressionCV

### Describe the bug

The code provided below raises ValueError. I guess that the problem is that minor classes may not be included in **train** or **val** sets for some folds during internal cross-validation, even with stratified split. This produces errors with some metrics other than default (accuracy).

One solution may be setting log-proba to -inf for classes not present in the train set, as well as providing label argument. How can I fix this in the most simple way?

### Steps/Code to Reproduce

```
import numpy as np
from sklearn.linear_model import LogisticRegressionCV
X = np.zeros((10, 1))
y = [1, 1, 1, 1, 1, 2, 2, 2, 2, 3]
logreg = LogisticRegressionCV(cv=5, scoring='neg_log_loss')
logreg.fit(X, y)
```

### Expected Results

No exception thrown

### Actual Results

ValueError: y_true and y_pred contain different number of classes 2, 3. Please provide the true labels explicitly through the labels argument. Classes found in y_true: [0 1]

### Versions

```shell
System:
    python: 3.10.8 (main, Nov  2 2023, 15:57:09) [GCC 9.4.0]
executable: /data/osedukhin/tabular-models/venv/bin/python
   machine: Linux-5.4.0-123-generic-x86_64-with-glibc2.31

Python dependencies:
      sklearn: 1.4.0
          pip: 22.2.2
   setuptools: 63.2.0
        numpy: 1.26.2
        scipy: 1.11.3
       Cython: None
       pandas: 2.1.3
   matplotlib: 3.8.1
       joblib: 1.3.2
threadpoolctl: 3.2.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: openblas
    num_threads: 32
         prefix: libopenblas
       filepath: /data/osedukhin/tabular-models/venv/lib/python3.10/site-packages/numpy.libs/libopenblas64_p-r0-0cf96a72.3.23.dev.so
        version: 0.3.23.dev
threading_layer: pthreads
   architecture: SkylakeX

       user_api: openmp
   internal_api: openmp
    num_threads: 32
         prefix: libgomp
       filepath: /data/osedukhin/tabular-models/venv/lib/python3.10/site-packages/scikit_learn.libs/libgomp-a34b3233.so.1.0.0
        version: None

       user_api: blas
   internal_api: openblas
    num_threads: 32
         prefix: libopenblas
       filepath: /data/osedukhin/tabular-models/venv/lib/python3.10/site-packages/scipy.libs/libopenblasp-r0-23e5df77.3.21.dev.so
        version: 0.3.21.dev
threading_layer: pthreads
   architecture: SkylakeX
```

COMMENT:
I close as it is a duplicate of #15389.

Labels: Bug