Talip7/scikit-learn-issues-multilabel huggingface dataset:
Grouping of infrequent categories in 𝗢𝗻𝗲𝗛𝗼𝘁𝗘𝗻𝗰𝗼𝗱𝗲𝗿

Dear all,

This feature is very similar to the one presented in this paper:
https://ieeexplore.ieee.org/document/8851888

which states the following paragraph:
The goal of the PCP transform is to substantially reduce the input memory and processing requirements while keeping the most relevant levels. Fig. 2 exemplifies the effect of this transform, by presenting the 1,000 highest frequency values (decreasing order) for the user city attribute for the TEST traffic data (which contains a total of 10,690 levels). For this attribute and when P = 10%, PCP selects only the most frequent 688 levels (dashed vertical line in Fig. 2), merging the other 10,002 infrequent levels into the "Others" label. 

<img width="883" alt="Captura de ecrã 2022-05-17, às 16 13 10" src="https://user-images.githubusercontent.com/39952471/168860640-dbe54b09-2771-4c9f-a0d9-17d4058bf29b.png">

Just asking, did the authors took notice of this paper? It can also improve this feature (instead of using a fixed number of categories it could be using a percentage categorical values to keep). 


Thanks for the attention!

COMMENT:
> Is there any open-access pre-print for this article?

I can email it for you :)
Just give me the details :)

Labels: New Feature, module:preprocessing, Needs Decision - Include Feature
Grouping of infrequent categories in 𝗢𝗻𝗲𝗛𝗼𝘁𝗘𝗻𝗰𝗼𝗱𝗲𝗿

Dear all,

This feature is very similar to the one presented in this paper:
https://ieeexplore.ieee.org/document/8851888

which states the following paragraph:
The goal of the PCP transform is to substantially reduce the input memory and processing requirements while keeping the most relevant levels. Fig. 2 exemplifies the effect of this transform, by presenting the 1,000 highest frequency values (decreasing order) for the user city attribute for the TEST traffic data (which contains a total of 10,690 levels). For this attribute and when P = 10%, PCP selects only the most frequent 688 levels (dashed vertical line in Fig. 2), merging the other 10,002 infrequent levels into the "Others" label. 

<img width="883" alt="Captura de ecrã 2022-05-17, às 16 13 10" src="https://user-images.githubusercontent.com/39952471/168860640-dbe54b09-2771-4c9f-a0d9-17d4058bf29b.png">

Just asking, did the authors took notice of this paper? It can also improve this feature (instead of using a fixed number of categories it could be using a percentage categorical values to keep). 


Thanks for the attention!

COMMENT:
> I can email it for you :)

It would be best to have a public version available to anyone intending to triage this issue to make a decision. I assume that there is no issue with making a pre-print available publicly in terms of copyrights.

Labels: New Feature, module:preprocessing, Needs Decision - Include Feature
Grouping of infrequent categories in 𝗢𝗻𝗲𝗛𝗼𝘁𝗘𝗻𝗰𝗼𝗱𝗲𝗿

Dear all,

This feature is very similar to the one presented in this paper:
https://ieeexplore.ieee.org/document/8851888

which states the following paragraph:
The goal of the PCP transform is to substantially reduce the input memory and processing requirements while keeping the most relevant levels. Fig. 2 exemplifies the effect of this transform, by presenting the 1,000 highest frequency values (decreasing order) for the user city attribute for the TEST traffic data (which contains a total of 10,690 levels). For this attribute and when P = 10%, PCP selects only the most frequent 688 levels (dashed vertical line in Fig. 2), merging the other 10,002 infrequent levels into the "Others" label. 

<img width="883" alt="Captura de ecrã 2022-05-17, às 16 13 10" src="https://user-images.githubusercontent.com/39952471/168860640-dbe54b09-2771-4c9f-a0d9-17d4058bf29b.png">

Just asking, did the authors took notice of this paper? It can also improve this feature (instead of using a fixed number of categories it could be using a percentage categorical values to keep). 


Thanks for the attention!

COMMENT:
> > I can email it for you :)
> 
> It would be best to have a public version available to anyone intending to triage this issue to make a decision. I assume that there is no issue with making a pre-print available publicly in terms of copyrights.

No problem! I have a link for the document here:
https://www.researchgate.net/publication/336162125_Using_Deep_Learning_for_Mobile_Marketing_User_Conversion_Prediction
I can also attach the preprint version if needed

Labels: New Feature, module:preprocessing, Needs Decision - Include Feature
Grouping of infrequent categories in 𝗢𝗻𝗲𝗛𝗼𝘁𝗘𝗻𝗰𝗼𝗱𝗲𝗿

Dear all,

This feature is very similar to the one presented in this paper:
https://ieeexplore.ieee.org/document/8851888

which states the following paragraph:
The goal of the PCP transform is to substantially reduce the input memory and processing requirements while keeping the most relevant levels. Fig. 2 exemplifies the effect of this transform, by presenting the 1,000 highest frequency values (decreasing order) for the user city attribute for the TEST traffic data (which contains a total of 10,690 levels). For this attribute and when P = 10%, PCP selects only the most frequent 688 levels (dashed vertical line in Fig. 2), merging the other 10,002 infrequent levels into the "Others" label. 

<img width="883" alt="Captura de ecrã 2022-05-17, às 16 13 10" src="https://user-images.githubusercontent.com/39952471/168860640-dbe54b09-2771-4c9f-a0d9-17d4058bf29b.png">

Just asking, did the authors took notice of this paper? It can also improve this feature (instead of using a fixed number of categories it could be using a percentage categorical values to keep). 


Thanks for the attention!

COMMENT:
@glemaitre 
Regarding the Categorical Prunned (CP) Transformation and Inverse Document Frequency (IDF) Transformation you can also check the paper (open access) in the following link: https://repositorium.sdum.uminho.pt/handle/1822/61586

The CP method fixes the number of categorical values of fixed amount (while the PCP is a percentage based amount)
and the IDF for categorical data.
This CP is similar to what you have implemented 😄  and the IDF can be a good addition for the package as well as the PCP version.

Labels: New Feature, module:preprocessing, Needs Decision - Include Feature
Grouping of infrequent categories in 𝗢𝗻𝗲𝗛𝗼𝘁𝗘𝗻𝗰𝗼𝗱𝗲𝗿

Dear all,

This feature is very similar to the one presented in this paper:
https://ieeexplore.ieee.org/document/8851888

which states the following paragraph:
The goal of the PCP transform is to substantially reduce the input memory and processing requirements while keeping the most relevant levels. Fig. 2 exemplifies the effect of this transform, by presenting the 1,000 highest frequency values (decreasing order) for the user city attribute for the TEST traffic data (which contains a total of 10,690 levels). For this attribute and when P = 10%, PCP selects only the most frequent 688 levels (dashed vertical line in Fig. 2), merging the other 10,002 infrequent levels into the "Others" label. 

<img width="883" alt="Captura de ecrã 2022-05-17, às 16 13 10" src="https://user-images.githubusercontent.com/39952471/168860640-dbe54b09-2771-4c9f-a0d9-17d4058bf29b.png">

Just asking, did the authors took notice of this paper? It can also improve this feature (instead of using a fixed number of categories it could be using a percentage categorical values to keep). 


Thanks for the attention!

COMMENT:
Hi all,

I don't know if it will help but I created a capsule in code ocean where you can run these categorical transformations 
https://doi.org/10.24433/CO.8941601.v1
I hope it helps :)

Labels: New Feature, module:preprocessing, Needs Decision - Include Feature
Grouping of infrequent categories in 𝗢𝗻𝗲𝗛𝗼𝘁𝗘𝗻𝗰𝗼𝗱𝗲𝗿

Dear all,

This feature is very similar to the one presented in this paper:
https://ieeexplore.ieee.org/document/8851888

which states the following paragraph:
The goal of the PCP transform is to substantially reduce the input memory and processing requirements while keeping the most relevant levels. Fig. 2 exemplifies the effect of this transform, by presenting the 1,000 highest frequency values (decreasing order) for the user city attribute for the TEST traffic data (which contains a total of 10,690 levels). For this attribute and when P = 10%, PCP selects only the most frequent 688 levels (dashed vertical line in Fig. 2), merging the other 10,002 infrequent levels into the "Others" label. 

<img width="883" alt="Captura de ecrã 2022-05-17, às 16 13 10" src="https://user-images.githubusercontent.com/39952471/168860640-dbe54b09-2771-4c9f-a0d9-17d4058bf29b.png">

Just asking, did the authors took notice of this paper? It can also improve this feature (instead of using a fixed number of categories it could be using a percentage categorical values to keep). 


Thanks for the attention!

COMMENT:
https://www.sciencedirect.com/science/article/pii/S2665963822000720
If you want more information regarding these methods suggested you can check this open access paper @lesteve @glemaitre @thomasjpfan

Labels: New Feature, module:preprocessing, Needs Decision - Include Feature
Grouping of infrequent categories in 𝗢𝗻𝗲𝗛𝗼𝘁𝗘𝗻𝗰𝗼𝗱𝗲𝗿

Dear all,

This feature is very similar to the one presented in this paper:
https://ieeexplore.ieee.org/document/8851888

which states the following paragraph:
The goal of the PCP transform is to substantially reduce the input memory and processing requirements while keeping the most relevant levels. Fig. 2 exemplifies the effect of this transform, by presenting the 1,000 highest frequency values (decreasing order) for the user city attribute for the TEST traffic data (which contains a total of 10,690 levels). For this attribute and when P = 10%, PCP selects only the most frequent 688 levels (dashed vertical line in Fig. 2), merging the other 10,002 infrequent levels into the "Others" label. 

<img width="883" alt="Captura de ecrã 2022-05-17, às 16 13 10" src="https://user-images.githubusercontent.com/39952471/168860640-dbe54b09-2771-4c9f-a0d9-17d4058bf29b.png">

Just asking, did the authors took notice of this paper? It can also improve this feature (instead of using a fixed number of categories it could be using a percentage categorical values to keep). 


Thanks for the attention!

COMMENT:
Any update regarding this?

Labels: New Feature, module:preprocessing, Needs Decision - Include Feature
Store the OOB Loss for `GradientBoostingClassifier`

### Describe the workflow you want to enable

Currently the only OOB-related performance metric we store on `GradientBoostingClassifier` is `oob_improvement_`, which is an array of OOB loss decreases per iteration. However, it would also be useful to track the *actual OOB loss values* for each iteration. This can be used as an estimate of the generalization error, and might bypass the need for cross validation in some cases. This would also help it integrate into my #23391 framework.

### Describe your proposed solution

I propose we add a new attribute: `oob_score_` (or alternatively `oob_loss_`) to `GradientBoostingClassifier`. This would only be set in cases where `subsample < 1`. It would be updated in each iteration. Currently we are *already calculating this*, we just throw it away:
 https://github.com/scikit-learn/scikit-learn/blob/32f9deaaf27c7ae56898222be9d820ba0fd1054f/sklearn/ensemble/_gb.py#L758-L768

### Describe alternatives you've considered, if relevant

You might think that the cumsum of the `oob_improvement_` would give us the loss values, and this is almost true, except for the fact that we need the OOB loss for the first iteration, which isn't stored anywhere. So this doesn't solve the issue.

### Additional context

_No response_

COMMENT:
I agree that `oob_score_` would be good to have.

Labels: New Feature, module:ensemble
Store the OOB Loss for `GradientBoostingClassifier`

### Describe the workflow you want to enable

Currently the only OOB-related performance metric we store on `GradientBoostingClassifier` is `oob_improvement_`, which is an array of OOB loss decreases per iteration. However, it would also be useful to track the *actual OOB loss values* for each iteration. This can be used as an estimate of the generalization error, and might bypass the need for cross validation in some cases. This would also help it integrate into my #23391 framework.

### Describe your proposed solution

I propose we add a new attribute: `oob_score_` (or alternatively `oob_loss_`) to `GradientBoostingClassifier`. This would only be set in cases where `subsample < 1`. It would be updated in each iteration. Currently we are *already calculating this*, we just throw it away:
 https://github.com/scikit-learn/scikit-learn/blob/32f9deaaf27c7ae56898222be9d820ba0fd1054f/sklearn/ensemble/_gb.py#L758-L768

### Describe alternatives you've considered, if relevant

You might think that the cumsum of the `oob_improvement_` would give us the loss values, and this is almost true, except for the fact that we need the OOB loss for the first iteration, which isn't stored anywhere. So this doesn't solve the issue.

### Additional context

_No response_

COMMENT:
I am fine with storing:

```
oob_scores_  # with the full history
oob_score_  # as a convenience for the last element of the previous array.
```

Labels: New Feature, module:ensemble
Store the OOB Loss for `GradientBoostingClassifier`

### Describe the workflow you want to enable

Currently the only OOB-related performance metric we store on `GradientBoostingClassifier` is `oob_improvement_`, which is an array of OOB loss decreases per iteration. However, it would also be useful to track the *actual OOB loss values* for each iteration. This can be used as an estimate of the generalization error, and might bypass the need for cross validation in some cases. This would also help it integrate into my #23391 framework.

### Describe your proposed solution

I propose we add a new attribute: `oob_score_` (or alternatively `oob_loss_`) to `GradientBoostingClassifier`. This would only be set in cases where `subsample < 1`. It would be updated in each iteration. Currently we are *already calculating this*, we just throw it away:
 https://github.com/scikit-learn/scikit-learn/blob/32f9deaaf27c7ae56898222be9d820ba0fd1054f/sklearn/ensemble/_gb.py#L758-L768

### Describe alternatives you've considered, if relevant

You might think that the cumsum of the `oob_improvement_` would give us the loss values, and this is almost true, except for the fact that we need the OOB loss for the first iteration, which isn't stored anywhere. So this doesn't solve the issue.

### Additional context

_No response_

COMMENT:
`oob_scores_` can actually be recomputed from the improvements and the final `oob_score_`.

Labels: New Feature, module:ensemble
Store the OOB Loss for `GradientBoostingClassifier`

### Describe the workflow you want to enable

Currently the only OOB-related performance metric we store on `GradientBoostingClassifier` is `oob_improvement_`, which is an array of OOB loss decreases per iteration. However, it would also be useful to track the *actual OOB loss values* for each iteration. This can be used as an estimate of the generalization error, and might bypass the need for cross validation in some cases. This would also help it integrate into my #23391 framework.

### Describe your proposed solution

I propose we add a new attribute: `oob_score_` (or alternatively `oob_loss_`) to `GradientBoostingClassifier`. This would only be set in cases where `subsample < 1`. It would be updated in each iteration. Currently we are *already calculating this*, we just throw it away:
 https://github.com/scikit-learn/scikit-learn/blob/32f9deaaf27c7ae56898222be9d820ba0fd1054f/sklearn/ensemble/_gb.py#L758-L768

### Describe alternatives you've considered, if relevant

You might think that the cumsum of the `oob_improvement_` would give us the loss values, and this is almost true, except for the fact that we need the OOB loss for the first iteration, which isn't stored anywhere. So this doesn't solve the issue.

### Additional context

_No response_

COMMENT:
Hi, I am new to contributing to open source projects. Can I take this up?

Labels: New Feature, module:ensemble
Store the OOB Loss for `GradientBoostingClassifier`

### Describe the workflow you want to enable

Currently the only OOB-related performance metric we store on `GradientBoostingClassifier` is `oob_improvement_`, which is an array of OOB loss decreases per iteration. However, it would also be useful to track the *actual OOB loss values* for each iteration. This can be used as an estimate of the generalization error, and might bypass the need for cross validation in some cases. This would also help it integrate into my #23391 framework.

### Describe your proposed solution

I propose we add a new attribute: `oob_score_` (or alternatively `oob_loss_`) to `GradientBoostingClassifier`. This would only be set in cases where `subsample < 1`. It would be updated in each iteration. Currently we are *already calculating this*, we just throw it away:
 https://github.com/scikit-learn/scikit-learn/blob/32f9deaaf27c7ae56898222be9d820ba0fd1054f/sklearn/ensemble/_gb.py#L758-L768

### Describe alternatives you've considered, if relevant

You might think that the cumsum of the `oob_improvement_` would give us the loss values, and this is almost true, except for the fact that we need the OOB loss for the first iteration, which isn't stored anywhere. So this doesn't solve the issue.

### Additional context

_No response_

COMMENT:
I would like to work on this.

Labels: New Feature, module:ensemble
Store the OOB Loss for `GradientBoostingClassifier`

### Describe the workflow you want to enable

Currently the only OOB-related performance metric we store on `GradientBoostingClassifier` is `oob_improvement_`, which is an array of OOB loss decreases per iteration. However, it would also be useful to track the *actual OOB loss values* for each iteration. This can be used as an estimate of the generalization error, and might bypass the need for cross validation in some cases. This would also help it integrate into my #23391 framework.

### Describe your proposed solution

I propose we add a new attribute: `oob_score_` (or alternatively `oob_loss_`) to `GradientBoostingClassifier`. This would only be set in cases where `subsample < 1`. It would be updated in each iteration. Currently we are *already calculating this*, we just throw it away:
 https://github.com/scikit-learn/scikit-learn/blob/32f9deaaf27c7ae56898222be9d820ba0fd1054f/sklearn/ensemble/_gb.py#L758-L768

### Describe alternatives you've considered, if relevant

You might think that the cumsum of the `oob_improvement_` would give us the loss values, and this is almost true, except for the fact that we need the OOB loss for the first iteration, which isn't stored anywhere. So this doesn't solve the issue.

### Additional context

_No response_

COMMENT:
/take

Labels: New Feature, module:ensemble
`DecisionTreeClassifier` became slower in v1.1 when fitting encoded variables

### Describe the bug

The evaluation of a pipeline that encodes categorical data with v1.1 takes around 8 times longer than using v1.0.2

### Steps/Code to Reproduce

```python
import numpy as np
import pandas as pd
from time import time
from sklearn.tree import DecisionTreeClassifier
from sklearn.model_selection import cross_val_score
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import OrdinalEncoder
from sklearn.compose import make_column_transformer, make_column_selector

rng = np.random.RandomState(0)
n_samples, n_features = 50_000, 2
X = pd.DataFrame(rng.randn(n_samples, n_features))
X[2] = np.random.choice(
    ["male", "female", "other"], size=n_samples, p=[0.49, 0.49, 0.02]
)
X[3] = np.random.choice(
    ["jan", "feb", "mar", "apr", "may", "jun",
     "jul", "aug", "sep", "oct", "nov", "dec"],
    size=n_samples,
)
y = np.random.choice(
    [0, 1, 2], size=n_samples, p=[0.01, 0.49, 0.5]
)

preprocessor = make_column_transformer(
    (OrdinalEncoder(), make_column_selector(dtype_include=object)),
    remainder="passthrough"
)
X_transformed = preprocessor.fit_transform(X)

t0 = time()
DecisionTreeClassifier().fit(X_transformed, y)
duration = time() - t0
duration
```

### Expected Results

~450ms

### Actual Results

3s

### Versions

```shell
System:
    python: 3.9.5 | packaged by conda-forge | (default, Jun 19 2021, 00:32:32)  [GCC 9.3.0]
executable: /home/arturoamor/miniforge3/envs/scikit-learn-course/bin/python
   machine: Linux-5.14.0-1036-oem-x86_64-with-glibc2.31

Python dependencies:
      sklearn: 1.1.0
          pip: 21.1.3
   setuptools: 49.6.0.post20210108
        numpy: 1.21.0
        scipy: 1.7.0
       Cython: None
       pandas: 1.3.0
   matplotlib: 3.4.2
       joblib: 1.0.1
threadpoolctl: 2.1.0

Built with OpenMP: True

threadpoolctl info:
       filepath: /home/arturoamor/miniforge3/envs/scikit-learn-course/lib/python3.9/site-packages/scikit_learn.libs/libgomp-a34b3233.so.1.0.0
         prefix: libgomp
       user_api: openmp
   internal_api: openmp
        version: None
    num_threads: 8

       filepath: /home/arturoamor/miniforge3/envs/scikit-learn-course/lib/libopenblasp-r0.3.15.so
         prefix: libopenblas
       user_api: blas
   internal_api: openblas
        version: 0.3.15
    num_threads: 8
threading_layer: pthreads
```

COMMENT:
A git bisect points to https://github.com/scikit-learn/scikit-learn/pull/22868 (use simultaneous sort in tree splitter).

Labels: Bug, Blocker, Regression, High Priority