Talip7/scikit-learn-issues-multilabel huggingface dataset:
TimeSeriesSplit Train Size formula correction

### Describe the issue linked to the documentation

The documentation states:
> The training set has size `i * n_samples // (n_splits + 1) + n_samples % (n_splits + 1)` in the `i`th split, with a test set of size `n_samples // (n_splits + 1)` by default, where `n_samples` is the number of samples.

The equation for the training set looks like it is flawed. Given a split with:

```python
test_set_size = 10
n_splits = X.shape[0] // test_set_size
max_train_size = 10000

cv = TimeSeriesSplit(n_splits=n_splits, test_size=test_size, max_train_size=max_train_size)

pairs
for i, (train_index, test_index) in enumerate(cv.split(X)):
    pairs.append([i, len(train_index), len(test_index)])
```

The previous code generates a list similar to this:
```
[[0, 7, 10],
[1, 17, 10],
[2, 27, 10],
[3, 37, 10],
...
[996, 9967, 10],
[997, 9977, 10],
[998, 9987, 10],
[999, 9997, 10],
[1000, 10000, 10],
[1001, 10000, 10],
...
[3177, 10000, 10],
[3178, 10000, 10]]
```

I tried using the formula in the documentation to determine the size of the training set at a given iteration:
```python
def get_train_size(i, n_samples, n_splits):
    return i * n_samples // (n_splits + 1) + n_samples % (n_splits + 1)

get_train_size(0, 31797, 3179)  # 3177, which is incorrect, it should be 7!


def get_train_size(i, n_samples, n_splits, max_train_size=float("inf")):
    start_value = n_samples % n_splits

    increment = n_samples // n_splits

    return int(min(start_value + i * increment, max_train_size))

get_train_size(0, 31797, 3179)  # 7, correct
get_train_size(999, 31797, 3179)  # 9997, correct
get_train_size(1000, 31797, 3179)  # 10007, correct
get_train_size(1000, 31797, 3179, 10000)  # 10000, correct
```

### Suggest a potential alternative/fix

So the correction to the formula should be that the training set size is equal to:
`i * (n_samples // n_splits) + n_samples % n_splits`
instead of:
`i * n_samples // (n_splits + 1) + n_samples % (n_splits + 1)`

COMMENT:
Will do, thanks! :)

Labels: Documentation
TimeSeriesSplit Train Size formula correction

### Describe the issue linked to the documentation

The documentation states:
> The training set has size `i * n_samples // (n_splits + 1) + n_samples % (n_splits + 1)` in the `i`th split, with a test set of size `n_samples // (n_splits + 1)` by default, where `n_samples` is the number of samples.

The equation for the training set looks like it is flawed. Given a split with:

```python
test_set_size = 10
n_splits = X.shape[0] // test_set_size
max_train_size = 10000

cv = TimeSeriesSplit(n_splits=n_splits, test_size=test_size, max_train_size=max_train_size)

pairs
for i, (train_index, test_index) in enumerate(cv.split(X)):
    pairs.append([i, len(train_index), len(test_index)])
```

The previous code generates a list similar to this:
```
[[0, 7, 10],
[1, 17, 10],
[2, 27, 10],
[3, 37, 10],
...
[996, 9967, 10],
[997, 9977, 10],
[998, 9987, 10],
[999, 9997, 10],
[1000, 10000, 10],
[1001, 10000, 10],
...
[3177, 10000, 10],
[3178, 10000, 10]]
```

I tried using the formula in the documentation to determine the size of the training set at a given iteration:
```python
def get_train_size(i, n_samples, n_splits):
    return i * n_samples // (n_splits + 1) + n_samples % (n_splits + 1)

get_train_size(0, 31797, 3179)  # 3177, which is incorrect, it should be 7!


def get_train_size(i, n_samples, n_splits, max_train_size=float("inf")):
    start_value = n_samples % n_splits

    increment = n_samples // n_splits

    return int(min(start_value + i * increment, max_train_size))

get_train_size(0, 31797, 3179)  # 7, correct
get_train_size(999, 31797, 3179)  # 9997, correct
get_train_size(1000, 31797, 3179)  # 10007, correct
get_train_size(1000, 31797, 3179, 10000)  # 10000, correct
```

### Suggest a potential alternative/fix

So the correction to the formula should be that the training set size is equal to:
`i * (n_samples // n_splits) + n_samples % n_splits`
instead of:
`i * n_samples // (n_splits + 1) + n_samples % (n_splits + 1)`

COMMENT:
Hey @jeremiedbb, can you review the PR? I repurposed it.

Thanks! :)

Labels: Documentation
TreeRegressors with MSE Criterion do not correctly handle missing-values

### Describe the bug

I found this bug when analyzing the PR/issue from #28295 and working on #27966.

Essentially, this bug is only found in `RegressorCriterion` because there one handles additionally the square of the `y` variables encoded in the variable `sq_sum_total`. However, this does not handle correctly when missing values are sent to the left node, even though the `sum_left` and `sum_right` are handled correctly.

The source of this issue lies in the `children_impurity` [function](https://github.com/scikit-learn/scikit-learn/blob/6e918a1f5fac2b13a9b387710980820734243bb9/sklearn/tree/_criterion.pyx#L1153-L1161).

The following fix should be valid:

```python
        for p in range(start, pos):
            i = sample_indices[p]

            if sample_weight is not None:
                w = sample_weight[i]

            for k in range(self.n_outputs):
                y_ik = self.y[i, k]
                sq_sum_left += w * y_ik * y_ik
     
        # added lines of code necessary to compute correct missing value 
        if self.missing_go_to_left:
            for p in range(end_non_missing, self.end):
                i = sample_indices[p]
                if sample_weight is not None:
                    w = sample_weight[i]

                for k in range(self.n_outputs):
                    y_ik = self.y[i, k]
                    sq_sum_left += w * y_ik * y_ik
        
        # continue with the rest of the function
        sq_sum_right = self.sq_sum_total - sq_sum_left
```

However, this issue is quite deep because it not only affects MSE, but also any Regression Criterion that requires an additional number to compute the relevant Criterion. 

I am happy to submit a PR with some relevant tests. This is quite an interesting issue because it doesn't seem to "affect" performance of `DecisionTreeRegressor` in unit-tests, but intuitively I would've expected it to. It's possible that we might even see a performance increase in MSE. Note that the fix will also help in the PR https://github.com/scikit-learn/scikit-learn/pull/27966 because I surmise it will improve performance of ExtaTrees

### Steps/Code to Reproduce

```python
# With this dataset, the missing values will always be sent to the left child
# at the first split. The leaf will be pure.
X = np.array([np.nan, 2, np.nan, 4, 5, 6]).reshape(-1, 1)
# X = np.array([np.nan, 2, 3, 4, 5, 6]).reshape(-1, 1)
y = np.arange(6)

tree = DecisionTreeRegressor(random_state=2, max_depth=1).fit(X, y)

from sklearn.tree import plot_tree
dtree = plot_tree(tree)
```


### Expected Results

![image](https://github.com/scikit-learn/scikit-learn/assets/3460267/54600cc6-f622-4acc-87e2-8dea33cd1b73)


### Actual Results

![image](https://github.com/scikit-learn/scikit-learn/assets/3460267/0749cda5-6c3c-4d18-a680-e2783d3d06fa)


### Versions

```shell
1.5.dev0
```

COMMENT:
Will put up a PR today to address this issue

Labels: Bug
Request to update "Choosing the Right Estimator" Graphic (scikit-learn algorithm cheat sheet)

### Describe the issue linked to the documentation

As seen here:
https://scikit-learn.org/stable/tutorial/machine_learning_map/index.html

One of the "tough luck" paths that go through the clustering section appear to say this is the case when there are >10k samples. 

### Suggest a potential alternative/fix

However, with modern computational hardware, and the optimized implementation of DBSCAN in Scikit-learn, it appears that it may be helpful to recommend DBSCAN as a possible solution for datasets containing <100K or even <1M datapoints for clustering in reasonable amounts of time on CPU.

COMMENT:
Indeed, we should update this map. We add IRL discussion with @ArturoAmorQ and @GaelVaroquaux regarding this topic. The map is missing new estimators. We could think also about more dynamic breakdown when zooming on the map. Anyway this a good suggestion, we need to come up with a plan to execute it properly.

Labels: Documentation
Request to update "Choosing the Right Estimator" Graphic (scikit-learn algorithm cheat sheet)

### Describe the issue linked to the documentation

As seen here:
https://scikit-learn.org/stable/tutorial/machine_learning_map/index.html

One of the "tough luck" paths that go through the clustering section appear to say this is the case when there are >10k samples. 

### Suggest a potential alternative/fix

However, with modern computational hardware, and the optimized implementation of DBSCAN in Scikit-learn, it appears that it may be helpful to recommend DBSCAN as a possible solution for datasets containing <100K or even <1M datapoints for clustering in reasonable amounts of time on CPU.

COMMENT:
@glemaitre is there a way I can help with this issue?

Labels: Documentation
Errors in Iterative Imputer Column Naming with scikit-learn 1.4 Integration

### Describe the bug

When attempting to support scikit-learn 1.4 in Pycaret, several bugs arose. Despite efforts to address them, including creating patches in Pycaret, issues persist, particularly regarding iterative imputation and related functionalities. The errors manifest as discrepancies in column names generated by functions, leading to failures in tests. These issues hinder seamless integration and usage of scikit-learn 1.4 within Pycaret.

### Steps/Code to Reproduce

Attempt integration of scikit-learn 1.4 with Pycaret.
Execute tests, particularly those involving iterative imputation.
Observe the ValueError messages indicating inconsistencies in column names.

### Expected Results

Successful integration without errors or discrepancies in column names, ensuring smooth functionality of Pycaret with scikit-learn 1.4.

### Actual Results

Failures in tests due to inconsistencies in column names generated by iterative imputer functions. Errors such as ValueError: The output generated by func have different column names than the one generated by the method get_feature_names_out occur, impeding proper execution of tests and integration efforts.
### Versions

```shell
scikit-learn 1.4
```

### More Details
https://github.com/pycaret/pycaret/pull/3857

COMMENT:
@adrinjalali @glemaitre, could you please check what's happening in the tests of my PR? It seems that the issue is related to get_feature_names_out on the scikit-learn side. Thank you in advance for your assistance!

Labels: Bug, Needs Triage
Errors in Iterative Imputer Column Naming with scikit-learn 1.4 Integration

### Describe the bug

When attempting to support scikit-learn 1.4 in Pycaret, several bugs arose. Despite efforts to address them, including creating patches in Pycaret, issues persist, particularly regarding iterative imputation and related functionalities. The errors manifest as discrepancies in column names generated by functions, leading to failures in tests. These issues hinder seamless integration and usage of scikit-learn 1.4 within Pycaret.

### Steps/Code to Reproduce

Attempt integration of scikit-learn 1.4 with Pycaret.
Execute tests, particularly those involving iterative imputation.
Observe the ValueError messages indicating inconsistencies in column names.

### Expected Results

Successful integration without errors or discrepancies in column names, ensuring smooth functionality of Pycaret with scikit-learn 1.4.

### Actual Results

Failures in tests due to inconsistencies in column names generated by iterative imputer functions. Errors such as ValueError: The output generated by func have different column names than the one generated by the method get_feature_names_out occur, impeding proper execution of tests and integration efforts.
### Versions

```shell
scikit-learn 1.4
```

### More Details
https://github.com/pycaret/pycaret/pull/3857

COMMENT:
duplicate https://github.com/scikit-learn/scikit-learn/issues/28232

Labels: Bug, Needs Triage
SimpleImputer silently cast fill values to integer when the input is of integer type

### Describe the bug

Fitting the SimpleImputer on an integer array silently cast the float `fill_value` values to integer. If `fill_value` is nan, nothing is imputed but a warning is raise:
`RuntimeWarning: invalid value encountered in cast
  multiarray.copyto(a, fill_value, casting='unsafe')`

### Steps/Code to Reproduce

```
from sklearn.impute import SimpleImputer
import numpy as np
a = np.array([0, 0, 0, 1]).reshape(-1, 1)
si = SimpleImputer(missing_values=0,
                    fill_value=3.2,
                    strategy="constant").set_output()

si.fit_transform(a)
```

```
from sklearn.impute import SimpleImputer
import numpy as np
a = np.array([0, 0, 0, 1]).reshape(-1, 1)
si = SimpleImputer(missing_values=0,
                    fill_value=np.nan,
                    strategy="constant").set_output()

si.fit_transform(a)
```

### Expected Results

Either raising an error (or a warning for the first case), or casting the array to float.
```
array([[3.2],
       [3.2],
       [3.2],
       [1.]])
```

```
array([[nan],
       [nan],
       [nan],
       [1.]])
```

### Actual Results

No warning and output:
```
array([[3],
       [3],
       [3],
       [1]])
```
Warning: `RuntimeWarning: invalid value encountered in cast
  multiarray.copyto(a, fill_value, casting='unsafe')` and output:
```
array([[0],
       [0],
       [0],
       [1]])
```

### Versions

```shell
System:
    python: 3.9.18 | packaged by conda-forge | (main, Aug 30 2023, 03:53:08)  [Clang 15.0.7 ]
executable: /Users/leo/mambaforge/envs/tabular-benchmark/bin/python
   machine: macOS-12.6.5-arm64-arm-64bit

Python dependencies:
      sklearn: 1.2.2
          pip: 23.3.1
   setuptools: 68.2.2
        numpy: 1.26.2
        scipy: 1.10.1
       Cython: None
       pandas: 2.1.4
   matplotlib: 3.8.0
       joblib: 1.2.0
threadpoolctl: 3.2.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: openblas
    num_threads: 8
         prefix: libopenblas
       filepath: /Users/leo/mambaforge/envs/tabular-benchmark/lib/python3.9/site-packages/numpy/.dylibs/libopenblas64_.0.dylib
        version: 0.3.23.dev
threading_layer: pthreads
   architecture: armv8

       user_api: blas
   internal_api: openblas
    num_threads: 8
         prefix: libopenblas
       filepath: /Users/leo/mambaforge/envs/tabular-benchmark/lib/libopenblas.0.dylib
        version: 0.3.24
threading_layer: openmp
   architecture: VORTEX

       user_api: openmp
   internal_api: openmp
    num_threads: 8
         prefix: libomp
       filepath: /Users/leo/mambaforge/envs/tabular-benchmark/lib/libomp.dylib
        version: None

       user_api: openmp
   internal_api: openmp
    num_threads: 8
         prefix: libomp
       filepath: /Users/leo/mambaforge/envs/tabular-benchmark/lib/python3.9/site-packages/xgboost/.dylibs/libomp.dylib
        version: None
```

COMMENT:
Probably we can use https://numpy.org/doc/stable/reference/generated/numpy.promote_types.html as casting rule.

Labels: Bug
MAINT remove old git branches

Can we delete old branches?
- ~~https://github.com/scikit-learn/scikit-learn/tree/feature/PairwiseDistances @jjerphan~~
- https://github.com/scikit-learn/scikit-learn/tree/debian

COMMENT:
> https://github.com/scikit-learn/scikit-learn/tree/feature/PairwiseDistances @jjerphan

`feature/PairwiseDistances` was targeted by https://github.com/scikit-learn/scikit-learn/pull/25561 which is superseded by https://github.com/scikit-learn/scikit-learn/pull/26983 which targets `main`. `feature/PairwiseDistances` can be deleted.

Labels: Needs Triage
MAINT remove old git branches

Can we delete old branches?
- ~~https://github.com/scikit-learn/scikit-learn/tree/feature/PairwiseDistances @jjerphan~~
- https://github.com/scikit-learn/scikit-learn/tree/debian

COMMENT:
Seems somebody also deleted `debian`, so closing this. Thanks @lorentzenchr

Labels: Needs Triage
⚠️ CI failed on Wheel builder ⚠️

**CI is still failing on [Wheel builder](https://github.com/scikit-learn/scikit-learn/actions/runs/7720901536)** (Jan 31, 2024)

COMMENT:
This seems legit since it starts to break in https://github.com/scikit-learn/scikit-learn/pull/28303

There is something going on with Python 3.9

Labels: Needs Triage
⚠️ CI failed on Wheel builder ⚠️

**CI is still failing on [Wheel builder](https://github.com/scikit-learn/scikit-learn/actions/runs/7720901536)** (Jan 31, 2024)

COMMENT:
I have some local fixes for pytest 8. Right now the situation is a bit complicated with lock-files update (mostly pandas Pyarrow dependency warning + some others pandas 2.2 warnings + pytest 8). I'll open a PR for pytest 8, once https://github.com/scikit-learn/scikit-learn/pull/28305 is merged.

Labels: Needs Triage
⚠️ CI failed on Wheel builder ⚠️

**CI is still failing on [Wheel builder](https://github.com/scikit-learn/scikit-learn/actions/runs/7720901536)** (Jan 31, 2024)

COMMENT:
## CI is no longer failing! ✅

[Successful run](https://github.com/scikit-learn/scikit-learn/actions/runs/7735950076) on Feb 01, 2024

Labels: Needs Triage
Trees are doing too many split with missing values

In the following example:

```python
import numpy as np
import sklearn
from sklearn.datasets import load_iris
from sklearn.tree import DecisionTreeClassifier
from sklearn.model_selection import train_test_split

X, y = load_iris(as_frame=True, return_X_y=True)

rng = np.random.RandomState(42)
X_missing = X.copy()
mask = rng.binomial(n=np.array([1, 1, 1, 1]).reshape(1, -1),
                    p=(X['petal length (cm)'] / 8).values.reshape(-1, 1)).astype(bool)
X_missing[mask] = np.NaN
indices = np.array([  2,  81,  39,  97,  91,  38,  46,  31, 101,  13,  89,  82, 100,
        42,  69,  27,  81,  16,  73,  74,  51,  47, 107,  17,  75, 110,
        20,  15, 104,  57,  26,  15,  75,  79,  35,  77,  90,  51,  46,
        13,  94,  91,  23,   8,  93,  93,  73,  77,  12,  13,  74, 109,
       110,  24,  10,  23, 104,  27,  92,  52,  20, 109,   8,   8,  28,
        27,  35,  12,  12,   7,  43,   0,  30,  31,  78,  12,  24, 105,
        50,   0,  73,  12, 102, 105,  13,  31,   1,  69,  11,  32,  75,
        90, 106,  94,  60,  56,  35,  17,  62,  85,  81,  39,  80,  16,
        63,   6,  80,  84,   3,   3,  76,  78], dtype=np.int32)
X_train, X_test, y_train, y_test = train_test_split(X_missing, y, random_state=13)
seed = 1857819720
clf = DecisionTreeClassifier(max_depth=None, max_features="sqrt", random_state=seed).fit(X_train.iloc[indices], y_train.iloc[indices])
```

we get the following tree::

![image](https://github.com/scikit-learn/scikit-learn/assets/7454015/def0ca84-fb2c-4022-8b9c-1b8a8b178033)

The path #12/#14 is weird. Indeed, we should have some missing values in #14 but we are still able to split based on `np.inf` that is not possible. So there is something fishy to investigate there.

COMMENT:
Actually this is fine. This is not along the same feature, my bad.

Labels: Bug
NeighborhoodComponentsAnalysis (NCA) sets incorrect `_n_features_out` value which makes `.transform()` fail if `transform_output="pandas"`.

### Describe the bug

`NeighborhoodComponentsAnalysis.transform()` fails with the following error whenever `transform_output` is set to "pandas":
```python-traceback
ValueError: Shape of passed values is (100, 2), indices imply (100, 20)
```

### Steps/Code to Reproduce

```python
import numpy as np
from sklearn.datasets import make_classification
from sklearn.neighbors import NeighborhoodComponentsAnalysis

random_state = np.random.RandomState(42)
X, y = make_classification(random_state=random_state)
nca = NeighborhoodComponentsAnalysis(n_components=2, random_state=random_state)
nca.set_output(transform="pandas")
nca.fit_transform(X, y)
```

### Expected Results

`NeighborhoodComponentsAnalysis.transform()` is expected to return a dataframe with `n_components` columns.

### Actual Results

The error lies in this line here:

https://github.com/scikit-learn/scikit-learn/blob/cb836be0ff8347ccb0ab722760df68d07485101e/sklearn/neighbors/_nca.py#L326

since `.components_` has the shape `(n_components, n_features)`, so this line should be, for example

    self._n_features_out = self.components_.shape[0]

because `._n_features_out` ultimately needs to correspond to `n_components` (only then the correct number of column labels will be produced, that is 2 instead of 20).

Instead, the bug could be fixed by copying the pattern found in `PCA` where [this is a property](https://github.com/scikit-learn/scikit-learn/blob/cb836be0ff8347ccb0ab722760df68d07485101e/sklearn/decomposition/_base.py#L190C5-L193C41) which would replace the faulty line of code:

    @property
    def _n_features_out(self):
        """Number of transformed output features."""
        return self.components_.shape[0]

### Versions

```shell
sklearn: 1.4.0
```

COMMENT:
I can reproduce.

Labels: Bug
NeighborhoodComponentsAnalysis (NCA) sets incorrect `_n_features_out` value which makes `.transform()` fail if `transform_output="pandas"`.

### Describe the bug

`NeighborhoodComponentsAnalysis.transform()` fails with the following error whenever `transform_output` is set to "pandas":
```python-traceback
ValueError: Shape of passed values is (100, 2), indices imply (100, 20)
```

### Steps/Code to Reproduce

```python
import numpy as np
from sklearn.datasets import make_classification
from sklearn.neighbors import NeighborhoodComponentsAnalysis

random_state = np.random.RandomState(42)
X, y = make_classification(random_state=random_state)
nca = NeighborhoodComponentsAnalysis(n_components=2, random_state=random_state)
nca.set_output(transform="pandas")
nca.fit_transform(X, y)
```

### Expected Results

`NeighborhoodComponentsAnalysis.transform()` is expected to return a dataframe with `n_components` columns.

### Actual Results

The error lies in this line here:

https://github.com/scikit-learn/scikit-learn/blob/cb836be0ff8347ccb0ab722760df68d07485101e/sklearn/neighbors/_nca.py#L326

since `.components_` has the shape `(n_components, n_features)`, so this line should be, for example

    self._n_features_out = self.components_.shape[0]

because `._n_features_out` ultimately needs to correspond to `n_components` (only then the correct number of column labels will be produced, that is 2 instead of 20).

Instead, the bug could be fixed by copying the pattern found in `PCA` where [this is a property](https://github.com/scikit-learn/scikit-learn/blob/cb836be0ff8347ccb0ab722760df68d07485101e/sklearn/decomposition/_base.py#L190C5-L193C41) which would replace the faulty line of code:

    @property
    def _n_features_out(self):
        """Number of transformed output features."""
        return self.components_.shape[0]

### Versions

```shell
sklearn: 1.4.0
```

COMMENT:
@johentsch Do you want to make a patch and a pull-request?

Labels: Bug