Talip7/scikit-learn-issues-multilabel huggingface dataset:
Random Tree Regressor crash the jupyter when fit to the data with bool columns

### Describe the bug

Got jupyter crash while trying to apply Random Forest Regression to data with about 1.5m rows and float, int and bool types.
Jupyter crashes every time even after rebooting the nootebook and PC.

### Steps/Code to Reproduce

```python
from sklearn.datasets import make_regression
from sklearn.ensemble import RandomForestRegressor
X, y, *_ = make_regression(500_000, 10, )
X[:,9]=True
model = RandomForestRegressor(n_estimators=10, random_state=42, max_depth=2)
model.fit(X, y)
```

### Expected Results

Some results

### Actual Results

Jupyter kernel crashes without additional information

### Versions

```shell
System:
    python: 3.10.2 (tags/v3.10.2:a58ebcc, Jan 17 2022, 14:12:15) [MSC v.1929 64 bit (AMD64)]
executable: C:\...\venv\Scripts\python.exe
   machine: Windows-10-10.0.19044-SP0

Python dependencies:
      sklearn: 1.1.0
          pip: 22.2.2
   setuptools: 65.3.0
        numpy: 1.23.3
        scipy: 1.8.0
       Cython: None
       pandas: 1.5.0
   matplotlib: 3.5.2
       joblib: 1.1.0
threadpoolctl: 3.1.0
```

COMMENT:
`RandomForest` with `max_depth=2` probably doesn't give you a good result anyway. I'd say this is a memory allocation issue due to the size of your data, and it'd be nice if you could try `HistGradientBoosting` instead, and report the results. For now I'm closing this as it seems the crash is due to memory allocation error.

Labels: Bug, Needs Triage
Addition of Feature That Detects And Treats Outliers As Per How The User Wishes

### Describe the workflow you want to enable

# Workflow
## Class
There Will Be A Class In sklearn.preprocessing whose instance will be created

## Function 
A Function Will Be There Like fit_transform() Where User Can Pass pandas DataFrame As Argument And The The DataFrame Will Be Treated And Returned

### Describe your proposed solution

# Outlier Detection
Outliers can be detected by Z-Score or Interquartile Range

# Outlier Treatment
They can be either removed or imputed

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
Please check our inclusion criteria. From this description it's not clear what you want to include. We don't easily include new models in scikit-learn. 

https://scikit-learn.org/stable/faq.html#what-are-the-inclusion-criteria-for-new-algorithms

You're more than welcome to have a third party package with your suggested model.

Closing, and happy to re-open if citations and new information is provided.

Labels: New Feature, Needs Triage
sklearn.metrics  jaccard_score issue in 0,1 binary classification

when using the from sklearn.metrics import jaccard_score to compare two binary classes labeled as 0 and 1.
the sample that are both 0 won't be recognized as in the same class. 
only samples that are both 1 will be recognized. 

for example, consider a dataframe with 2 columns 
```
             column 0,column1
row 0         1,1
row 1          0,0 
```
and compute the jaccard_score(column 0,column1) the row1 won't be considered as in the same overlap as row 0. 
i tested this by +1 to this datraframe and this solved the issue. 

an illustration of the calculation using the following code which compute the same output as the jaccard_score

```py
def jaccard_binary(x,y):
    """A function for finding the similarity between two binary vectors"""
    intersection = np.logical_and(x, y)
    union = np.logical_or(x, y)
    similarity = intersection.sum() / float(union.sum())
    return intersection.sum(),union.sum(), similarity
```

it will return false on the intersection for row1.

COMMENT:
Hello, I followed your explanation to test the issue. I don't see any issue here since sklearn jaccard_score and the code snippet you provided are returning same output 1.0. Similarity calculation between the col1 and col2 seems to be fine for me. What does it mean by returning false on the intersection for row1?

```
from sklearn.metrics import jaccard_score
import numpy as np
col1 = np.array([1,0])
col2 = np.array([1, 0])
jaccard_score(col1,col2)
#output --> 1.0

def jaccard_binary(x,y):
    """A function for finding the similarity between two binary vectors"""
    intersection = np.logical_and(x, y)
    union = np.logical_or(x, y)
    similarity = intersection.sum() / float(union.sum())
    return intersection.sum(),union.sum(), similarity

jaccard_binary(col1,col2)
#output --> (1, 1, 1.0)

```

Labels: Needs Triage
sklearn.metrics  jaccard_score issue in 0,1 binary classification

when using the from sklearn.metrics import jaccard_score to compare two binary classes labeled as 0 and 1.
the sample that are both 0 won't be recognized as in the same class. 
only samples that are both 1 will be recognized. 

for example, consider a dataframe with 2 columns 
```
             column 0,column1
row 0         1,1
row 1          0,0 
```
and compute the jaccard_score(column 0,column1) the row1 won't be considered as in the same overlap as row 0. 
i tested this by +1 to this datraframe and this solved the issue. 

an illustration of the calculation using the following code which compute the same output as the jaccard_score

```py
def jaccard_binary(x,y):
    """A function for finding the similarity between two binary vectors"""
    intersection = np.logical_and(x, y)
    union = np.logical_or(x, y)
    similarity = intersection.sum() / float(union.sum())
    return intersection.sum(),union.sum(), similarity
```

it will return false on the intersection for row1.

COMMENT:
consider this example: 
```
from sklearn.metrics import jaccard_score
y_true = np.array([0,0,1,1])
y_pred = np.array([0,0,0,1])
print(y_true, y_pred)
jaccard_score(y_true, y_pred)
```
output: 
[0 0 1 1] [0 0 0 1]
0.5

and 
```
y_true = y_true+1
y_pred = y_pred+1
print(y_true, y_pred)
jaccard_score(y_true, y_pred)
```
output: 
[1 1 2 2] [1 1 1 2]
0.6666666666666666

These two examples are essentially the same and should have the same jaccard score. (i can be wrong) 
I think what is happening is that when the labels are binary and labeled as 0 and 1, the instance where two of them are 0 are not recognized as intersection correctly and jaccard is counting the intersection divided by union.

Labels: Needs Triage
sklearn.metrics  jaccard_score issue in 0,1 binary classification

when using the from sklearn.metrics import jaccard_score to compare two binary classes labeled as 0 and 1.
the sample that are both 0 won't be recognized as in the same class. 
only samples that are both 1 will be recognized. 

for example, consider a dataframe with 2 columns 
```
             column 0,column1
row 0         1,1
row 1          0,0 
```
and compute the jaccard_score(column 0,column1) the row1 won't be considered as in the same overlap as row 0. 
i tested this by +1 to this datraframe and this solved the issue. 

an illustration of the calculation using the following code which compute the same output as the jaccard_score

```py
def jaccard_binary(x,y):
    """A function for finding the similarity between two binary vectors"""
    intersection = np.logical_and(x, y)
    union = np.logical_or(x, y)
    similarity = intersection.sum() / float(union.sum())
    return intersection.sum(),union.sum(), similarity
```

it will return false on the intersection for row1.

COMMENT:
Jaccard score for classification is defined as 
TP/(TP + FP + FN).

You are using `jaccard_score` with default parameters in both cases (this includes `average='binary'` and `pos_label='1'`). In the first case: TP = 1, FP = 1, FN = 0 -> 0.5. In the second case: TP = 2, FP = 1, FN = 0 -> 0.6(6). If you change `pos_label` to '2' in the second case, you should get 0.5.

Labels: Needs Triage
RFC Suggesting HistGradientBoosting in RandomForest and GradientBoosting pages

Right now we have this in the GradientBoosting API page:

> [sklearn.ensemble.HistGradientBoostingClassifier](https://scikit-learn.org/dev/modules/generated/sklearn.ensemble.HistGradientBoostingClassifier.html#sklearn.ensemble.HistGradientBoostingClassifier) is a much faster variant of this algorithm for intermediate datasets (n_samples >= 10_000).

After LogisticRegression and LinearRegression, RandomForest is the third most visited page among models we have. And I think we'd agree that in most cases users can probably be better off using HGBT models instead. Right now users compare random forests with xgboost, while they could be using HGBT.

So my question is, should we add a message on forest/tree pages regarding HGBT, and have the statement a bit bolder than just saying "it's probably faster"?

@amueller had done quite a bit of analysis on this, maybe we could link to those?

COMMENT:
We do have a more visible note for the [non-histogram GBDT User Guide page](https://scikit-learn.org/dev/modules/ensemble.html#gradient-tree-boosting), but it might be worth making it more obvious on the estimators' docstrings as well, I agree.

As for RandomForest,I think the consensus is that GBDTs are a bit harder to tune, but they can get you that extra accuracy against RFs. Whether this makes it a good enough reason to recommend HGBDT instead,  I don't have a strong opinion. If we do, the recommendation should probably be more nuanced than for the GBDT vs HGBDT case (which is more of a no-brainer).

Labels: Documentation, RFC
RFC Suggesting HistGradientBoosting in RandomForest and GradientBoosting pages

Right now we have this in the GradientBoosting API page:

> [sklearn.ensemble.HistGradientBoostingClassifier](https://scikit-learn.org/dev/modules/generated/sklearn.ensemble.HistGradientBoostingClassifier.html#sklearn.ensemble.HistGradientBoostingClassifier) is a much faster variant of this algorithm for intermediate datasets (n_samples >= 10_000).

After LogisticRegression and LinearRegression, RandomForest is the third most visited page among models we have. And I think we'd agree that in most cases users can probably be better off using HGBT models instead. Right now users compare random forests with xgboost, while they could be using HGBT.

So my question is, should we add a message on forest/tree pages regarding HGBT, and have the statement a bit bolder than just saying "it's probably faster"?

@amueller had done quite a bit of analysis on this, maybe we could link to those?

COMMENT:
I can work on this, but I would say there are 3 approaches that we can possibly combine:

1- adding a short message in the docstring of RF to maximize visibility;
2- adding a long statement on the user guide on RF;
3- adding an example similar to [this exercise in the MOOC](https://inria.github.io/scikit-learn-mooc/python_scripts/ensemble_sol_03.html) to compare HGBDT with RF and possibly link it to the docstring (in combination with option 1).

Labels: Documentation, RFC
RFC Suggesting HistGradientBoosting in RandomForest and GradientBoosting pages

Right now we have this in the GradientBoosting API page:

> [sklearn.ensemble.HistGradientBoostingClassifier](https://scikit-learn.org/dev/modules/generated/sklearn.ensemble.HistGradientBoostingClassifier.html#sklearn.ensemble.HistGradientBoostingClassifier) is a much faster variant of this algorithm for intermediate datasets (n_samples >= 10_000).

After LogisticRegression and LinearRegression, RandomForest is the third most visited page among models we have. And I think we'd agree that in most cases users can probably be better off using HGBT models instead. Right now users compare random forests with xgboost, while they could be using HGBT.

So my question is, should we add a message on forest/tree pages regarding HGBT, and have the statement a bit bolder than just saying "it's probably faster"?

@amueller had done quite a bit of analysis on this, maybe we could link to those?

COMMENT:
Sounds good to me @ArturoAmorQ

Labels: Documentation, RFC
RFC Suggesting HistGradientBoosting in RandomForest and GradientBoosting pages

Right now we have this in the GradientBoosting API page:

> [sklearn.ensemble.HistGradientBoostingClassifier](https://scikit-learn.org/dev/modules/generated/sklearn.ensemble.HistGradientBoostingClassifier.html#sklearn.ensemble.HistGradientBoostingClassifier) is a much faster variant of this algorithm for intermediate datasets (n_samples >= 10_000).

After LogisticRegression and LinearRegression, RandomForest is the third most visited page among models we have. And I think we'd agree that in most cases users can probably be better off using HGBT models instead. Right now users compare random forests with xgboost, while they could be using HGBT.

So my question is, should we add a message on forest/tree pages regarding HGBT, and have the statement a bit bolder than just saying "it's probably faster"?

@amueller had done quite a bit of analysis on this, maybe we could link to those?

COMMENT:
@adrinjalali , @ArturoAmorQ 
I have tried fixing this in one of my PR's. Please review https://github.com/scikit-learn/scikit-learn/pull/26313. I'm a beginner, and this is my first commit, so do let me know in case of any suggestions

Labels: Documentation, RFC
RFC Suggesting HistGradientBoosting in RandomForest and GradientBoosting pages

Right now we have this in the GradientBoosting API page:

> [sklearn.ensemble.HistGradientBoostingClassifier](https://scikit-learn.org/dev/modules/generated/sklearn.ensemble.HistGradientBoostingClassifier.html#sklearn.ensemble.HistGradientBoostingClassifier) is a much faster variant of this algorithm for intermediate datasets (n_samples >= 10_000).

After LogisticRegression and LinearRegression, RandomForest is the third most visited page among models we have. And I think we'd agree that in most cases users can probably be better off using HGBT models instead. Right now users compare random forests with xgboost, while they could be using HGBT.

So my question is, should we add a message on forest/tree pages regarding HGBT, and have the statement a bit bolder than just saying "it's probably faster"?

@amueller had done quite a bit of analysis on this, maybe we could link to those?

COMMENT:
> After LogisticRegression and LinearRegression, RandomForest is the third most visited page among models we have. And I think we'd agree that in most cases users can probably be better off using HGBT models instead. Right now users compare random forests with xgboost, while they could be using HGBT.

I'm not so sure that we should recommend HGBT in every (second) other estimator. We could give some advice on when to use which estimator in the user guide.
If people compare rf vs xgboost, it may be because our rf does not have many competitors and xgboost ist still an outstanding boosted tree implementation.

Labels: Documentation, RFC
RFC Suggesting HistGradientBoosting in RandomForest and GradientBoosting pages

Right now we have this in the GradientBoosting API page:

> [sklearn.ensemble.HistGradientBoostingClassifier](https://scikit-learn.org/dev/modules/generated/sklearn.ensemble.HistGradientBoostingClassifier.html#sklearn.ensemble.HistGradientBoostingClassifier) is a much faster variant of this algorithm for intermediate datasets (n_samples >= 10_000).

After LogisticRegression and LinearRegression, RandomForest is the third most visited page among models we have. And I think we'd agree that in most cases users can probably be better off using HGBT models instead. Right now users compare random forests with xgboost, while they could be using HGBT.

So my question is, should we add a message on forest/tree pages regarding HGBT, and have the statement a bit bolder than just saying "it's probably faster"?

@amueller had done quite a bit of analysis on this, maybe we could link to those?

COMMENT:
So my point is: Recommend HGBT, yes, but not in the docstring of other estimators.

Labels: Documentation, RFC
RFC Suggesting HistGradientBoosting in RandomForest and GradientBoosting pages

Right now we have this in the GradientBoosting API page:

> [sklearn.ensemble.HistGradientBoostingClassifier](https://scikit-learn.org/dev/modules/generated/sklearn.ensemble.HistGradientBoostingClassifier.html#sklearn.ensemble.HistGradientBoostingClassifier) is a much faster variant of this algorithm for intermediate datasets (n_samples >= 10_000).

After LogisticRegression and LinearRegression, RandomForest is the third most visited page among models we have. And I think we'd agree that in most cases users can probably be better off using HGBT models instead. Right now users compare random forests with xgboost, while they could be using HGBT.

So my question is, should we add a message on forest/tree pages regarding HGBT, and have the statement a bit bolder than just saying "it's probably faster"?

@amueller had done quite a bit of analysis on this, maybe we could link to those?

COMMENT:
> So my point is: Recommend HGBT, yes, but not in the docstring of other estimators.

@lorentzenchr what do you think of adding HGBT in the "see also" section of random forests?

Labels: Documentation, RFC
RFC Suggesting HistGradientBoosting in RandomForest and GradientBoosting pages

Right now we have this in the GradientBoosting API page:

> [sklearn.ensemble.HistGradientBoostingClassifier](https://scikit-learn.org/dev/modules/generated/sklearn.ensemble.HistGradientBoostingClassifier.html#sklearn.ensemble.HistGradientBoostingClassifier) is a much faster variant of this algorithm for intermediate datasets (n_samples >= 10_000).

After LogisticRegression and LinearRegression, RandomForest is the third most visited page among models we have. And I think we'd agree that in most cases users can probably be better off using HGBT models instead. Right now users compare random forests with xgboost, while they could be using HGBT.

So my question is, should we add a message on forest/tree pages regarding HGBT, and have the statement a bit bolder than just saying "it's probably faster"?

@amueller had done quite a bit of analysis on this, maybe we could link to those?

COMMENT:
> what do you think of adding HGBT in the "see also" section of random forests?

That‘s fine as it is another ensemble method/estimator.

Labels: Documentation, RFC
RFC Suggesting HistGradientBoosting in RandomForest and GradientBoosting pages

Right now we have this in the GradientBoosting API page:

> [sklearn.ensemble.HistGradientBoostingClassifier](https://scikit-learn.org/dev/modules/generated/sklearn.ensemble.HistGradientBoostingClassifier.html#sklearn.ensemble.HistGradientBoostingClassifier) is a much faster variant of this algorithm for intermediate datasets (n_samples >= 10_000).

After LogisticRegression and LinearRegression, RandomForest is the third most visited page among models we have. And I think we'd agree that in most cases users can probably be better off using HGBT models instead. Right now users compare random forests with xgboost, while they could be using HGBT.

So my question is, should we add a message on forest/tree pages regarding HGBT, and have the statement a bit bolder than just saying "it's probably faster"?

@amueller had done quite a bit of analysis on this, maybe we could link to those?

COMMENT:
@ArturoAmorQ Can I work on this?

Labels: Documentation, RFC
RFC Suggesting HistGradientBoosting in RandomForest and GradientBoosting pages

Right now we have this in the GradientBoosting API page:

> [sklearn.ensemble.HistGradientBoostingClassifier](https://scikit-learn.org/dev/modules/generated/sklearn.ensemble.HistGradientBoostingClassifier.html#sklearn.ensemble.HistGradientBoostingClassifier) is a much faster variant of this algorithm for intermediate datasets (n_samples >= 10_000).

After LogisticRegression and LinearRegression, RandomForest is the third most visited page among models we have. And I think we'd agree that in most cases users can probably be better off using HGBT models instead. Right now users compare random forests with xgboost, while they could be using HGBT.

So my question is, should we add a message on forest/tree pages regarding HGBT, and have the statement a bit bolder than just saying "it's probably faster"?

@amueller had done quite a bit of analysis on this, maybe we could link to those?

COMMENT:
> Can I work on this?

@amay1212 I submitted PRs addressing the points 1 and 3 from [my previous comment](https://github.com/scikit-learn/scikit-learn/issues/26220#issuecomment-1521480796).
Feel free to address point 2 if you want to :)

Labels: Documentation, RFC
RFC Suggesting HistGradientBoosting in RandomForest and GradientBoosting pages

Right now we have this in the GradientBoosting API page:

> [sklearn.ensemble.HistGradientBoostingClassifier](https://scikit-learn.org/dev/modules/generated/sklearn.ensemble.HistGradientBoostingClassifier.html#sklearn.ensemble.HistGradientBoostingClassifier) is a much faster variant of this algorithm for intermediate datasets (n_samples >= 10_000).

After LogisticRegression and LinearRegression, RandomForest is the third most visited page among models we have. And I think we'd agree that in most cases users can probably be better off using HGBT models instead. Right now users compare random forests with xgboost, while they could be using HGBT.

So my question is, should we add a message on forest/tree pages regarding HGBT, and have the statement a bit bolder than just saying "it's probably faster"?

@amueller had done quite a bit of analysis on this, maybe we could link to those?

COMMENT:
@ArturoAmorQ 
I have made the changes. Let me know in case of any concerns or suggestions.

Labels: Documentation, RFC
RFC Suggesting HistGradientBoosting in RandomForest and GradientBoosting pages

Right now we have this in the GradientBoosting API page:

> [sklearn.ensemble.HistGradientBoostingClassifier](https://scikit-learn.org/dev/modules/generated/sklearn.ensemble.HistGradientBoostingClassifier.html#sklearn.ensemble.HistGradientBoostingClassifier) is a much faster variant of this algorithm for intermediate datasets (n_samples >= 10_000).

After LogisticRegression and LinearRegression, RandomForest is the third most visited page among models we have. And I think we'd agree that in most cases users can probably be better off using HGBT models instead. Right now users compare random forests with xgboost, while they could be using HGBT.

So my question is, should we add a message on forest/tree pages regarding HGBT, and have the statement a bit bolder than just saying "it's probably faster"?

@amueller had done quite a bit of analysis on this, maybe we could link to those?

COMMENT:
@lorentzenchr the issue is, from the user's perspective this is what happens:

they know of linear models, so they try logistic regression or linear regression for their data, and then they're like, but random forests are also cool, let's try those, and then they know of xgboost.

Here the issue is discoverability of HGBT, and right now the docs are not really good at recommending HGBT to users, and to most users out there, the only thing they need is HGBT. If we recommend in the right places that they could be using it, more users would stay here instead of going to xgboost, even though we're not on a 1-1 feature parity with xgboost.

Labels: Documentation, RFC