Talip7/scikit-learn-issues-multilabel huggingface dataset:
Random Forest Neighbors

### Describe the workflow you want to enable

Random Regression forests allow for an interpretation where the values at a new point  `x`  is a weighted linear combination of the values observed in the dataset 
<img width="188" alt="image" src="https://user-images.githubusercontent.com/61052993/186520968-88e824ed-77c1-4bf8-b7be-9885c288778e.png">
I think it would be helpful if one could obtain these weights for using the sklearn random forest for defining (for example) local neighbourhoods.

### Describe your proposed solution

I think there should be a method which would allow one to access these neighbourhoods.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

If there is any interest I could look deeper into this issue and work to get a PR on it.

COMMENT:
I am also interested in this feature. Random forests just have so much room for improvement!

Just for context some thoughts i had: to enable something like this, what would be helpful for exposing this is the concept of a leaf node in the Cython code. Right now the leaf nodes are just "nodes" which in principle can be a "split node".

This is also more generally useful for enabling querying of leaf nodes that store more than just simple count data. (I.e. quantile forests)

I mention this in https://hackmd.io/TFmRQfFHSGucBt_Pk2gOWA

Labels: New Feature
Random Forest Neighbors

### Describe the workflow you want to enable

Random Regression forests allow for an interpretation where the values at a new point  `x`  is a weighted linear combination of the values observed in the dataset 
<img width="188" alt="image" src="https://user-images.githubusercontent.com/61052993/186520968-88e824ed-77c1-4bf8-b7be-9885c288778e.png">
I think it would be helpful if one could obtain these weights for using the sklearn random forest for defining (for example) local neighbourhoods.

### Describe your proposed solution

I think there should be a method which would allow one to access these neighbourhoods.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

If there is any interest I could look deeper into this issue and work to get a PR on it.

COMMENT:
I think at this point @velezbeltran while the generalized tree paper has many citations, the specific use-case you are advocating for is still very recent and not very widely used, which makes it difficult to adopt into the main library due to the innate maintainability cost. A better solution may be to instead find a way to implement this in a separate package under `scikit-learn-contrib`

Labels: New Feature
Random Forest Neighbors

### Describe the workflow you want to enable

Random Regression forests allow for an interpretation where the values at a new point  `x`  is a weighted linear combination of the values observed in the dataset 
<img width="188" alt="image" src="https://user-images.githubusercontent.com/61052993/186520968-88e824ed-77c1-4bf8-b7be-9885c288778e.png">
I think it would be helpful if one could obtain these weights for using the sklearn random forest for defining (for example) local neighbourhoods.

### Describe your proposed solution

I think there should be a method which would allow one to access these neighbourhoods.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

If there is any interest I could look deeper into this issue and work to get a PR on it.

COMMENT:
Sounds good :)

Labels: New Feature
adjusted_mutual_info_score takes a long time with lists containing many unique values

### Describe the bug

The runtime of `adjusted_mutual_info_score` jumps significantly when we have large amounts of unique values in the two lists. Hovering around 6k total unique values (ie 2 columns of 3k unique values) keeps the runtime around a minute, but when we increase the number of unique values, the runtime shoots up.

### Steps/Code to Reproduce

```
import pandas as pd
from sklearn.metrics import adjusted_mutual_info_score as ams
import time

df = pd.DataFrame({'a': [x % 8000 for x in range(1000000)],
                   'b': [x % 7000 for x in range(1000000)]})

start = time.time()
mi = ams(df['a'], df['b'])
end = time.time()
end - start
```

### Expected Results

small or linear increase in runtime as we increase the number of unique values.

### Actual Results

Large increase in runtime 

2 rows of 6k unique values: 598s
2 rows of 8k unique values: 889s
2 rows of 10k unique values: 1118s

### Versions

```shell
System:
    python: 3.8.0 (default, Aug 17 2020, 18:01:34)  [Clang 11.0.3 (clang-1103.0.32.62)]
executable: /Users/bryan.chen/.pyenv/versions/woodwork/bin/python
   machine: macOS-10.16-x86_64-i386-64bit

Python dependencies:
          pip: 22.2.2
   setuptools: 41.2.0
      sklearn: 1.0.2
        numpy: 1.21.2
        scipy: 1.7.1
       Cython: 0.29.28
       pandas: 1.4.3
   matplotlib: 3.5.1
       joblib: 1.0.1
threadpoolctl: 2.2.0

Built with OpenMP: True
```

COMMENT:
Thank you for opening the issue. One would need to profile the implementation to see what the slow parts are to move forward on this issue.

Labels: module:metrics, Needs Investigation
adjusted_mutual_info_score takes a long time with lists containing many unique values

### Describe the bug

The runtime of `adjusted_mutual_info_score` jumps significantly when we have large amounts of unique values in the two lists. Hovering around 6k total unique values (ie 2 columns of 3k unique values) keeps the runtime around a minute, but when we increase the number of unique values, the runtime shoots up.

### Steps/Code to Reproduce

```
import pandas as pd
from sklearn.metrics import adjusted_mutual_info_score as ams
import time

df = pd.DataFrame({'a': [x % 8000 for x in range(1000000)],
                   'b': [x % 7000 for x in range(1000000)]})

start = time.time()
mi = ams(df['a'], df['b'])
end = time.time()
end - start
```

### Expected Results

small or linear increase in runtime as we increase the number of unique values.

### Actual Results

Large increase in runtime 

2 rows of 6k unique values: 598s
2 rows of 8k unique values: 889s
2 rows of 10k unique values: 1118s

### Versions

```shell
System:
    python: 3.8.0 (default, Aug 17 2020, 18:01:34)  [Clang 11.0.3 (clang-1103.0.32.62)]
executable: /Users/bryan.chen/.pyenv/versions/woodwork/bin/python
   machine: macOS-10.16-x86_64-i386-64bit

Python dependencies:
          pip: 22.2.2
   setuptools: 41.2.0
      sklearn: 1.0.2
        numpy: 1.21.2
        scipy: 1.7.1
       Cython: 0.29.28
       pandas: 1.4.3
   matplotlib: 3.5.1
       joblib: 1.0.1
threadpoolctl: 2.2.0

Built with OpenMP: True
```

COMMENT:
Performed a high-level analysis and found that in a sample of 6000 records, 99% of the time is taken by `sklearn.metrics.cluster._expected_mutual_info_fast.expected_mutual_information`

```
System:
    python: 3.9.7 (default, Sep 16 2021, 08:50:36)  [Clang 10.0.0 ]
executable: /Users/kshitijmathur/opt/anaconda3/bin/python
   machine: macOS-10.16-x86_64-i386-64bit

Python dependencies:
      sklearn: 1.1.3
          pip: 22.2.2
   setuptools: 58.0.4
        numpy: 1.22.4
        scipy: 1.7.1
       Cython: 0.29.24
       pandas: 1.4.3
   matplotlib: 3.4.3
       joblib: 1.1.0
threadpoolctl: 2.2.0

Built with OpenMP: True
```

Labels: module:metrics, Needs Investigation
adjusted_mutual_info_score takes a long time with lists containing many unique values

### Describe the bug

The runtime of `adjusted_mutual_info_score` jumps significantly when we have large amounts of unique values in the two lists. Hovering around 6k total unique values (ie 2 columns of 3k unique values) keeps the runtime around a minute, but when we increase the number of unique values, the runtime shoots up.

### Steps/Code to Reproduce

```
import pandas as pd
from sklearn.metrics import adjusted_mutual_info_score as ams
import time

df = pd.DataFrame({'a': [x % 8000 for x in range(1000000)],
                   'b': [x % 7000 for x in range(1000000)]})

start = time.time()
mi = ams(df['a'], df['b'])
end = time.time()
end - start
```

### Expected Results

small or linear increase in runtime as we increase the number of unique values.

### Actual Results

Large increase in runtime 

2 rows of 6k unique values: 598s
2 rows of 8k unique values: 889s
2 rows of 10k unique values: 1118s

### Versions

```shell
System:
    python: 3.8.0 (default, Aug 17 2020, 18:01:34)  [Clang 11.0.3 (clang-1103.0.32.62)]
executable: /Users/bryan.chen/.pyenv/versions/woodwork/bin/python
   machine: macOS-10.16-x86_64-i386-64bit

Python dependencies:
          pip: 22.2.2
   setuptools: 41.2.0
      sklearn: 1.0.2
        numpy: 1.21.2
        scipy: 1.7.1
       Cython: 0.29.28
       pandas: 1.4.3
   matplotlib: 3.5.1
       joblib: 1.0.1
threadpoolctl: 2.2.0

Built with OpenMP: True
```

COMMENT:
almost entire performance degradation is coming from this line of code
`start = np.array([[v - N + w for w in b] for v in a], dtype='int')` 
https://github.com/scikit-learn/scikit-learn/blob/main/sklearn/metrics/cluster/_expected_mutual_info_fast.pyx#L53

For 6000 records, 
above line takes 125 seconds
The for nested loop at the end of Cython function takes ~25 seconds

Labels: module:metrics, Needs Investigation
adjusted_mutual_info_score takes a long time with lists containing many unique values

### Describe the bug

The runtime of `adjusted_mutual_info_score` jumps significantly when we have large amounts of unique values in the two lists. Hovering around 6k total unique values (ie 2 columns of 3k unique values) keeps the runtime around a minute, but when we increase the number of unique values, the runtime shoots up.

### Steps/Code to Reproduce

```
import pandas as pd
from sklearn.metrics import adjusted_mutual_info_score as ams
import time

df = pd.DataFrame({'a': [x % 8000 for x in range(1000000)],
                   'b': [x % 7000 for x in range(1000000)]})

start = time.time()
mi = ams(df['a'], df['b'])
end = time.time()
end - start
```

### Expected Results

small or linear increase in runtime as we increase the number of unique values.

### Actual Results

Large increase in runtime 

2 rows of 6k unique values: 598s
2 rows of 8k unique values: 889s
2 rows of 10k unique values: 1118s

### Versions

```shell
System:
    python: 3.8.0 (default, Aug 17 2020, 18:01:34)  [Clang 11.0.3 (clang-1103.0.32.62)]
executable: /Users/bryan.chen/.pyenv/versions/woodwork/bin/python
   machine: macOS-10.16-x86_64-i386-64bit

Python dependencies:
          pip: 22.2.2
   setuptools: 41.2.0
      sklearn: 1.0.2
        numpy: 1.21.2
        scipy: 1.7.1
       Cython: 0.29.28
       pandas: 1.4.3
   matplotlib: 3.5.1
       joblib: 1.0.1
threadpoolctl: 2.2.0

Built with OpenMP: True
```

COMMENT:
So to me, since the time complexity of above line of code is O(N^2), then we can't actually expect linear increases in runtime. 
I'm not an expert in Cython, but I will try to see if something can be done about this.

Labels: module:metrics, Needs Investigation
Docstring typo 📜

https://github.com/scikit-learn/scikit-learn/blob/36958fb240fbe435673a9e3c52e769f01f36bec0/sklearn/feature_extraction/text.py#L421

Typo makes docstring unclear and ambiguous:
```py
  def build_analyzer(self):
      """Return a callable to process input data.
      The callable handles that handles preprocessing, tokenization, and
      n-grams generation.
      ...
      """
```

I suggest:
```py
  def build_analyzer(self):
      """Return a callable to process input data.
      The callable handles preprocessing, tokenization, and
      n-grams generation.
      ...
      """
```

COMMENT:
Thanks for reporting. Do you want to open a PR to solve this issue?

Labels: Documentation
Docstring typo 📜

https://github.com/scikit-learn/scikit-learn/blob/36958fb240fbe435673a9e3c52e769f01f36bec0/sklearn/feature_extraction/text.py#L421

Typo makes docstring unclear and ambiguous:
```py
  def build_analyzer(self):
      """Return a callable to process input data.
      The callable handles that handles preprocessing, tokenization, and
      n-grams generation.
      ...
      """
```

I suggest:
```py
  def build_analyzer(self):
      """Return a callable to process input data.
      The callable handles preprocessing, tokenization, and
      n-grams generation.
      ...
      """
```

COMMENT:
Sure, I got this 😆

Labels: Documentation
Docstring typo 📜

https://github.com/scikit-learn/scikit-learn/blob/36958fb240fbe435673a9e3c52e769f01f36bec0/sklearn/feature_extraction/text.py#L421

Typo makes docstring unclear and ambiguous:
```py
  def build_analyzer(self):
      """Return a callable to process input data.
      The callable handles that handles preprocessing, tokenization, and
      n-grams generation.
      ...
      """
```

I suggest:
```py
  def build_analyzer(self):
      """Return a callable to process input data.
      The callable handles preprocessing, tokenization, and
      n-grams generation.
      ...
      """
```

COMMENT:
Hi @3lLobo, I have assigned this issue to you. However, you have opened a pull request to your own main branch, and a number of old commits are in there.
You might have a look at the [contributing documentation](https://scikit-learn.org/dev/developers/contributing.html#how-to-contribute): the steps to open a pull request to scikit-learn are described in details.... and if you spot any inconsistency there, you are welcome to open an issue about that! Thanks for contributing!

Labels: Documentation
Docstring typo 📜

https://github.com/scikit-learn/scikit-learn/blob/36958fb240fbe435673a9e3c52e769f01f36bec0/sklearn/feature_extraction/text.py#L421

Typo makes docstring unclear and ambiguous:
```py
  def build_analyzer(self):
      """Return a callable to process input data.
      The callable handles that handles preprocessing, tokenization, and
      n-grams generation.
      ...
      """
```

I suggest:
```py
  def build_analyzer(self):
      """Return a callable to process input data.
      The callable handles preprocessing, tokenization, and
      n-grams generation.
      ...
      """
```

COMMENT:
Hi @cmarmo ,
Yes I noticed all the obsolete changes in the pr, thus kept it on my local branch.
Tomorrow I will make time to correct the pr and open it in origin :)

Labels: Documentation
Docstring typo 📜

https://github.com/scikit-learn/scikit-learn/blob/36958fb240fbe435673a9e3c52e769f01f36bec0/sklearn/feature_extraction/text.py#L421

Typo makes docstring unclear and ambiguous:
```py
  def build_analyzer(self):
      """Return a callable to process input data.
      The callable handles that handles preprocessing, tokenization, and
      n-grams generation.
      ...
      """
```

I suggest:
```py
  def build_analyzer(self):
      """Return a callable to process input data.
      The callable handles preprocessing, tokenization, and
      n-grams generation.
      ...
      """
```

COMMENT:
@cmarmo fine this way? PR #24309

Labels: Documentation
Issue with class template with 2 semi columns instead of a single one

### Describe the issue linked to the documentation

In the documentation, it appears that the section has 2 semi-columns instead of a single one:

![image](https://user-images.githubusercontent.com/7454015/186420230-489d6e13-9355-4e36-b89a-600bb6879842.png)

Not sure where it comes from.

### Suggest a potential alternative/fix

_No response_

COMMENT:
Maybe https://github.com/sphinx-doc/sphinx/issues/10594?

Labels: Documentation, Needs Triage
Issue with class template with 2 semi columns instead of a single one

### Describe the issue linked to the documentation

In the documentation, it appears that the section has 2 semi-columns instead of a single one:

![image](https://user-images.githubusercontent.com/7454015/186420230-489d6e13-9355-4e36-b89a-600bb6879842.png)

Not sure where it comes from.

### Suggest a potential alternative/fix

_No response_

COMMENT:
We also have this in `doc/themes/scikit-learn-modern/static/css/theme.css` which may need to be removed?

```
dl.field-list > dt:after {
  content: ":";
}
```

Labels: Documentation, Needs Triage
Issue with class template with 2 semi columns instead of a single one

### Describe the issue linked to the documentation

In the documentation, it appears that the section has 2 semi-columns instead of a single one:

![image](https://user-images.githubusercontent.com/7454015/186420230-489d6e13-9355-4e36-b89a-600bb6879842.png)

Not sure where it comes from.

### Suggest a potential alternative/fix

_No response_

COMMENT:
Removing the CSS works as long as docutils is also updated to a version greater than 0.18.

Labels: Documentation, Needs Triage
Issue with class template with 2 semi columns instead of a single one

### Describe the issue linked to the documentation

In the documentation, it appears that the section has 2 semi-columns instead of a single one:

![image](https://user-images.githubusercontent.com/7454015/186420230-489d6e13-9355-4e36-b89a-600bb6879842.png)

Not sure where it comes from.

### Suggest a potential alternative/fix

_No response_

COMMENT:
I opened #24252.

We are using sphinx 5.1.1 (which has the fix above depending on docutils version) so it should be fine I think.

For reference, we are also using docutils 0.19 > 0.18.

Labels: Documentation, Needs Triage
Add RepeatedStratifiedGroupKFold

### Describe the workflow you want to enable

Building off conversation #13621 and work already done in #18649, I'd like to add an implementation of `RepeatedStratifiedGroupKFold`.

### Describe your proposed solution

See the implementation in #24227. Then `RepeatedStratifiedGroupKFold` could be used similar to below:

```python
  >>> import numpy as np
  >>> from sklearn.model_selection import RepeatedStratifiedGroupKFold
  >>> X = np.random.randn(10, 1)
  >>> y = np.array([0, 0, 0, 0, 0, 1, 1, 1, 1, 1])
  >>> groups = np.array([1, 1, 2, 2, 2, 3, 4, 4, 5, 5])
  >>> rsgkf = RepeatedStratifiedGroupKFold(n_splits=3, n_repeats=2, random_state=42)
  >>> for train_idxs, test_idxs in rsgkf.split(X, y, groups):
  ...     # print the group assignment for the train/test indices
  ...     print("TRAIN:", groups[train_idxs], "TEST:", groups[test_idxs])
  ...     X_train, X_test = X[train_idxs], X[test_idxs]
  ...     y_train, y_test = y[train_idxs], y[test_idxs]
  TRAIN: [2 2 2 4 4 5 5] TEST: [1 1 3]
  TRAIN: [1 1 3 4 4 5 5] TEST: [2 2 2]
  TRAIN: [1 1 2 2 2 3] TEST: [4 4 5 5]
  TRAIN: [1 1 4 4 5 5] TEST: [2 2 2 3]
  TRAIN: [2 2 2 3 4 4 5 5] TEST: [1 1]
  TRAIN: [1 1 2 2 2 3] TEST: [4 4 5 5]
```

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
From the triaging meeting, we think this would be a good inclusion. Although, there are already many splitter classes in API: https://scikit-learn.org/stable/modules/classes.html#splitter-classes, adding a new splitter for "repeated" is the current status quo.

Note that it may take some time for a maintainer to review your PR.

Labels: New Feature, module:model_selection
TimeSeriesSplit add skip parameter

### Describe the workflow you want to enable

Dear sklearn community, I want to make an hour ahead forecast on a timeseries of at least a year with an interval of 5 minutes. To do CV, I can use TimeSeriesSplit with test_size = 12. I want to do this for many different times of the day, so I could use n_splits = 24 and have a split for each hour of the day. Then I want this for multiple days, so I could do n_splits = 24 * nr_days. My problem is that I want to randomly test a few hours a day over many days of the year, to see how my forecast does on different periods of the year. To then make n_splits = 24 * 365 is too many splits.

### Describe your proposed solution

I propose a `skip` parameter, that allows skipping a number of samples before making the next split. This can be used to reduce the number of splits, while still allowing to cover a large time-period with splits.

A few examples:
Data:
| index      | foo |
|------------|-----|
| 2022-01-01 | 1   |
| 2022-01-02 | 2   |
| 2022-01-03 | 3   |
| 2022-01-04 | 4   |
| 2022-01-05 | 5   |
| 2022-01-06 | 6   |
| 2022-01-07 | 7   |
| 2022-01-08 | 8   |
| 2022-01-09 | 9   |
| 2022-01-10 | 10 |

test_size=1, max_train_size=1, n_splits=3, gap=1, skip=1

| train | test |
|-------|-----|
| 8     | 10    |
| 6     | 8      |
| 4     | 6      |

test_size=1, max_train_size=1,   n_splits=3, gap=1, skip=2
| train | test |
|-------|-----|
| 8     | 10   |
| 5     | 7    |
| 2     | 4    |

test_size=2, max_train_size=1, n_splits=3, gap=1, skip=2
not enough data

test_size=2, max_train_size=1, n_splits=2, gap=1, skip=2
| train | test  |
|-------|------|
| 7     | 9, 10 |
| 3     | 5, 6  |

test_size=2, max_train_size=1, n_splits=2, gap=1, skip=2
| train | test  |
|-------|------|
| 7     | 9, 10 |
| 3     | 5, 6  |

test_size=2, n_splits=2, gap=2, skip=2
| train               | test  |
|----------------|------|
| 1, 2                | 5, 6  |
| 1, 2, 3, 4, 5, 6 | 9, 10 |

### Describe alternatives you've considered, if relevant

I could generate all those splits, then implement a wrapper in my own codebase that does the skipping.

### Additional context

I already have this solution implemented locally. The important changes to the function `TimeSeriesSplit.split()` are:

```
        if n_samples - gap - ((test_size + skip) * n_splits) < 0:
            raise ValueError(
                f"Too many splits={n_splits} for number of samples"
                f"={n_samples} with test_size={test_size}, gap={gap} and skip={skip}."
            )
```

```
test_starts = range(n_samples - n_splits * (test_size + skip) + skip, n_samples, test_size + skip)
```

I'd be happy to make a PR if you think this can be included. Thank you for your time and consideration!

COMMENT:
Would it not be possible to get the behaviour only setting `gap`? If I understand properly, `skip` is skipping sometimes between splits which looks exactly like what `gap` is intended to.

Labels: New Feature, module:model_selection, Needs Decision - Include Feature
⚠️ CI failed on Linux_Docker.debian_atlas_32bit ⚠️

**CI is still failing on [Linux_Docker.debian_atlas_32bit](https://dev.azure.com/scikit-learn/scikit-learn/_build/results?buildId=45919&view=logs&j=aabdcdc3-bb64-5414-b357-ed024fe8659e)** (Aug 25, 2022)
- test_fastica_simple[27-float32-True]
- test_fastica_simple[27-float32-False]

COMMENT:
This will be fixed by #24244

Labels: Build / CI