Talip7/scikit-learn-issues-multilabel huggingface dataset:
adding uncertainty quantifier in the "predict" function for DecisionTreeRegressor

### Describe the workflow you want to enable

The "n_node_samples" in "tree_" attribute tracks the number of samples in the leafs. But there should be a easier way of using this quantity for general users. 

I hope the following feature can be added: when calling the "predict" function, there is an option that allows the user to **directly** output number of training samples used for each prediction. This helps quantify the uncertainty in the prediction. 

This is not far from what scikit-learn already has, but it would be helpful for general users. 

### Describe your proposed solution

This should be fairly easier for scikit-learn developers. 

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
@wangt1anyu  please assign me this task

Labels: New Feature, Needs Info
adding uncertainty quantifier in the "predict" function for DecisionTreeRegressor

### Describe the workflow you want to enable

The "n_node_samples" in "tree_" attribute tracks the number of samples in the leafs. But there should be a easier way of using this quantity for general users. 

I hope the following feature can be added: when calling the "predict" function, there is an option that allows the user to **directly** output number of training samples used for each prediction. This helps quantify the uncertainty in the prediction. 

This is not far from what scikit-learn already has, but it would be helpful for general users. 

### Describe your proposed solution

This should be fairly easier for scikit-learn developers. 

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
@AmeyaVerma Hello Ameya, Thanks! How should I assign it to you?

Labels: New Feature, Needs Info
adding uncertainty quantifier in the "predict" function for DecisionTreeRegressor

### Describe the workflow you want to enable

The "n_node_samples" in "tree_" attribute tracks the number of samples in the leafs. But there should be a easier way of using this quantity for general users. 

I hope the following feature can be added: when calling the "predict" function, there is an option that allows the user to **directly** output number of training samples used for each prediction. This helps quantify the uncertainty in the prediction. 

This is not far from what scikit-learn already has, but it would be helpful for general users. 

### Describe your proposed solution

This should be fairly easier for scikit-learn developers. 

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
@wangt1anyu  there's this feature in github to assign the tasks. You can do that so I can start working on it

Labels: New Feature, Needs Info
adding uncertainty quantifier in the "predict" function for DecisionTreeRegressor

### Describe the workflow you want to enable

The "n_node_samples" in "tree_" attribute tracks the number of samples in the leafs. But there should be a easier way of using this quantity for general users. 

I hope the following feature can be added: when calling the "predict" function, there is an option that allows the user to **directly** output number of training samples used for each prediction. This helps quantify the uncertainty in the prediction. 

This is not far from what scikit-learn already has, but it would be helpful for general users. 

### Describe your proposed solution

This should be fairly easier for scikit-learn developers. 

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
@AmeyaVerma Seems that I don't have enough permissions to assign an issue to someone. I guess we'll need to leave this to the core development team.

Labels: New Feature, Needs Info
adding uncertainty quantifier in the "predict" function for DecisionTreeRegressor

### Describe the workflow you want to enable

The "n_node_samples" in "tree_" attribute tracks the number of samples in the leafs. But there should be a easier way of using this quantity for general users. 

I hope the following feature can be added: when calling the "predict" function, there is an option that allows the user to **directly** output number of training samples used for each prediction. This helps quantify the uncertainty in the prediction. 

This is not far from what scikit-learn already has, but it would be helpful for general users. 

### Describe your proposed solution

This should be fairly easier for scikit-learn developers. 

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
@wangt1anyu do you have a reference in the machine learning or statistical literature? What mathemtical formula do you have in mind to achieve this?

Note that there exists a model agnostic way to quantify uncertainty of regression models in MAPIE named "CQR" but the base models need to be quantile regressors (using the quantile/pinball loss function), rather than expected value regressors (using the mse or Poisson loss functions).

- https://mapie.readthedocs.io/en/latest/examples_regression/4-tutorials/plot_cqr_tutorial.html


In scikit-learn, all decision tree models (including https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.HistGradientBoostingRegressor.html) have the option to quantile regression and there is also https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.QuantileRegressor.html as a linear model.

Labels: New Feature, Needs Info
adding uncertainty quantifier in the "predict" function for DecisionTreeRegressor

### Describe the workflow you want to enable

The "n_node_samples" in "tree_" attribute tracks the number of samples in the leafs. But there should be a easier way of using this quantity for general users. 

I hope the following feature can be added: when calling the "predict" function, there is an option that allows the user to **directly** output number of training samples used for each prediction. This helps quantify the uncertainty in the prediction. 

This is not far from what scikit-learn already has, but it would be helpful for general users. 

### Describe your proposed solution

This should be fairly easier for scikit-learn developers. 

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
Hello, 


This is our paper: 


Tianyu Wang, Weicheng Ye, Dawei Geng, and Cynthia Rudin. 2020. Towards Practical Lipschitz Bandits. In Proceedings of the 2020 ACM-IMS on Foundations of Data Science Conference (FODS '20). Association for Computing Machinery, New York, NY, USA, 129–138. https://doi.org/10.1145/3412815.3416885 . 


The most critical sub-step is outputting the number of samples in the leafs. Over the last three years, I got several emails asking for an software package for the algorithm in the paper. It would be great if the official package supports this function.


Best regards,
Tianyu



-----原始邮件-----
发件人:"Olivier Grisel" ***@***.***>
发送时间:2023-10-06 23:18:19 (星期五)
收件人: scikit-learn/scikit-learn ***@***.***>
抄送: "Tianyu Wang" ***@***.***>, Mention ***@***.***>
主题: Re: [scikit-learn/scikit-learn] adding uncertainty quantifier in the "predict" function for DecisionTreeRegressor (Issue #27507)






@wangt1anyu do you have a reference in the machine learning or statistical literature? What mathemtical formula do you have in mind to achieve this?

Note that there exists a model agnostic way to quantify uncertainty of regression models in MAPIE named "CQR" but the base models need to be quantile regressors (using the quantile/pinball loss function), rather than expected value regressors (using the mse or Poisson loss functions).

https://mapie.readthedocs.io/en/latest/examples_regression/4-tutorials/plot_cqr_tutorial.html

In scikit-learn, all decision tree models (including https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.HistGradientBoostingRegressor.html) have the option to quantile regression and there is also https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.QuantileRegressor.html as a linear model.

—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you were mentioned.Message ID: ***@***.***>

Labels: New Feature, Needs Info
adding uncertainty quantifier in the "predict" function for DecisionTreeRegressor

### Describe the workflow you want to enable

The "n_node_samples" in "tree_" attribute tracks the number of samples in the leafs. But there should be a easier way of using this quantity for general users. 

I hope the following feature can be added: when calling the "predict" function, there is an option that allows the user to **directly** output number of training samples used for each prediction. This helps quantify the uncertainty in the prediction. 

This is not far from what scikit-learn already has, but it would be helpful for general users. 

### Describe your proposed solution

This should be fairly easier for scikit-learn developers. 

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
https://github.com/scikit-learn/scikit-learn/issues/27504#issuecomment-1742115799 already answers how to easily access the number of training samples per leaf. In principle, the `apply` gives you full control.

For easier uncertainty estimation, we would also need to store the variance in addition to the leaf values. That would increase the memory footprint.

I‘ll close. But feel free to continue the discussion.

Labels: New Feature, Needs Info
Impact of class weights in LogisticRegression

### Describe the issue linked to the documentation

The impact of class weights and the exact objective function with (all kinds of) weights for `LogisticRegression` should be mentioned in the user guide. Importantly, the scale of weights interact with the (anti-) penalty strength `C`.

Proof that this is confusing: #27455

### Suggest a potential alternative/fix

_No response_

COMMENT:
Can I try to do this?

Labels: Documentation
Returning number of samples in leaf nodes in decision trees.

### Describe the workflow you want to enable

In the paper "Towards Practical Lipschitz Bandits" by Wang, Ye, Geng and Rudin (https://dl.acm.org/doi/10.1145/3412815.3416885), the authors used a modified version of the DecisionTreeRegressor in their algorithm. More specifically, they used number of samples in the leaf nodes to quantify the uncertainty level of the predictions. It seems that the method in this paper is a bit useful. 

It would be great if "DecisionTreeRegressor" object in the official scikit-learn package could output number of samples in the leaf nodes. 

### Describe your proposed solution

When fitting the DecisionTreeRegressor, in addition to record mean of the leaf nodes, also record number of samples used to compute the mean. When output a prediction, in addition to output of mean of the corresponding leaf node, also output the number of training samples used in the prediction. 

I'm a coauthor of the paper "Towards Practical Lipschitz Bandits", and have a local copy of our solution when writing the paper. I just forked the master branch of the official scikit-learn repo, and replace the folder "sklearn" by our local copy. This fork is at: https://github.com/wangt1anyu/scikit-learn. Our code is a bit old, and may not be optimized in some aspects. But maybe it's useful as a reference. 

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
This is already tracked in the underlying `tree_` attribute, as `n_node_samples`. See https://scikit-learn.org/stable/auto_examples/tree/plot_unveil_tree_structure.html#sphx-glr-auto-examples-tree-plot-unveil-tree-structure-py

Labels: New Feature, Needs Triage
Returning number of samples in leaf nodes in decision trees.

### Describe the workflow you want to enable

In the paper "Towards Practical Lipschitz Bandits" by Wang, Ye, Geng and Rudin (https://dl.acm.org/doi/10.1145/3412815.3416885), the authors used a modified version of the DecisionTreeRegressor in their algorithm. More specifically, they used number of samples in the leaf nodes to quantify the uncertainty level of the predictions. It seems that the method in this paper is a bit useful. 

It would be great if "DecisionTreeRegressor" object in the official scikit-learn package could output number of samples in the leaf nodes. 

### Describe your proposed solution

When fitting the DecisionTreeRegressor, in addition to record mean of the leaf nodes, also record number of samples used to compute the mean. When output a prediction, in addition to output of mean of the corresponding leaf node, also output the number of training samples used in the prediction. 

I'm a coauthor of the paper "Towards Practical Lipschitz Bandits", and have a local copy of our solution when writing the paper. I just forked the master branch of the official scikit-learn repo, and replace the folder "sklearn" by our local copy. This fork is at: https://github.com/wangt1anyu/scikit-learn. Our code is a bit old, and may not be optimized in some aspects. But maybe it's useful as a reference. 

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
Hello, 




I believe you misunderstood my request. There is no quick way of associating each test sample to the number of samples used to predict this test label. 




Best regards,

Tianyu 



-----原始邮件-----
发件人:"Ben Reiniger" ***@***.***>
发送时间:2023-09-30 23:19:10 (星期六)
收件人: scikit-learn/scikit-learn ***@***.***>
抄送: "Tianyu Wang" ***@***.***>, Author ***@***.***>
主题: Re: [scikit-learn/scikit-learn] Returning number of samples in leaf nodes in decision trees. (Issue #27504)






This is already tracked in the underlying tree_ attribute, as n_node_samples. See https://scikit-learn.org/stable/auto_examples/tree/plot_unveil_tree_structure.html#sphx-glr-auto-examples-tree-plot-unveil-tree-structure-py

—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you authored the thread.Message ID: ***@***.***>

Labels: New Feature, Needs Triage
Returning number of samples in leaf nodes in decision trees.

### Describe the workflow you want to enable

In the paper "Towards Practical Lipschitz Bandits" by Wang, Ye, Geng and Rudin (https://dl.acm.org/doi/10.1145/3412815.3416885), the authors used a modified version of the DecisionTreeRegressor in their algorithm. More specifically, they used number of samples in the leaf nodes to quantify the uncertainty level of the predictions. It seems that the method in this paper is a bit useful. 

It would be great if "DecisionTreeRegressor" object in the official scikit-learn package could output number of samples in the leaf nodes. 

### Describe your proposed solution

When fitting the DecisionTreeRegressor, in addition to record mean of the leaf nodes, also record number of samples used to compute the mean. When output a prediction, in addition to output of mean of the corresponding leaf node, also output the number of training samples used in the prediction. 

I'm a coauthor of the paper "Towards Practical Lipschitz Bandits", and have a local copy of our solution when writing the paper. I just forked the master branch of the official scikit-learn repo, and replace the folder "sklearn" by our local copy. This fork is at: https://github.com/wangt1anyu/scikit-learn. Our code is a bit old, and may not be optimized in some aspects. But maybe it's useful as a reference. 

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
Indeed I failed to address the mapping between a row and that number of samples in the leaves.  You can use `apply` for that though, or `decision_path` if you want some more information.  E.g.

```python
clf = DecisionTreeClassifier(max_depth=3).fit(X, y)
leaf = clf.apply(X.iloc[5:6]).item()
clf.tree_.n_node_samples[leaf]
```

Whether it's worth it to expose a more direct way to get this information is up to the core devs, and I see you've spun off a new Issue, #27507.

Labels: New Feature, Needs Triage
Returning number of samples in leaf nodes in decision trees.

### Describe the workflow you want to enable

In the paper "Towards Practical Lipschitz Bandits" by Wang, Ye, Geng and Rudin (https://dl.acm.org/doi/10.1145/3412815.3416885), the authors used a modified version of the DecisionTreeRegressor in their algorithm. More specifically, they used number of samples in the leaf nodes to quantify the uncertainty level of the predictions. It seems that the method in this paper is a bit useful. 

It would be great if "DecisionTreeRegressor" object in the official scikit-learn package could output number of samples in the leaf nodes. 

### Describe your proposed solution

When fitting the DecisionTreeRegressor, in addition to record mean of the leaf nodes, also record number of samples used to compute the mean. When output a prediction, in addition to output of mean of the corresponding leaf node, also output the number of training samples used in the prediction. 

I'm a coauthor of the paper "Towards Practical Lipschitz Bandits", and have a local copy of our solution when writing the paper. I just forked the master branch of the official scikit-learn repo, and replace the folder "sklearn" by our local copy. This fork is at: https://github.com/wangt1anyu/scikit-learn. Our code is a bit old, and may not be optimized in some aspects. But maybe it's useful as a reference. 

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
Hello Ben, Sorry for my slow response, and thanks for the solution! I opened up that other issue, since I thought this one was closed and no longer alive... Sorry, I'm not very familiar with the GitHub issuing process. Best regards,Tianyu 
        
  
    ---- Replied Message ----
  
  
    
      
        
         From 
        
        
            Ben ***@***.***>
            
        
      
      
        
         Date 
        
        
        10/1/2023 23:29
        
      
      
        
         To 
        
        
         
          
            ***@***.***>
            
          
        
      
      
        
         Cc 
        
        
          
            Tianyu ***@***.***>
            ,
          
          
            ***@***.***>
            
          
        
      
      
        
         Subject 
        
        
              Re: [scikit-learn/scikit-learn] Returning number of samples in leaf nodes in decision trees. (Issue #27504)
        
      
    
  

Indeed I failed to address the mapping between a row and that number of samples in the leaves.  You can use apply for that though, or decision_path if you want some more information.  E.g.
clf = DecisionTreeClassifier(max_depth=3).fit(X, y)
leaf = clf.apply(X.iloc[5:6]).item()
clf.tree_.n_node_samples[leaf]
Whether it's worth it to expose a more direct way to get this information is up to the core devs, and I see you've spun off a new Issue, #27507.

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID: ***@***.***>

Labels: New Feature, Needs Triage
Returning number of samples in leaf nodes in decision trees.

### Describe the workflow you want to enable

In the paper "Towards Practical Lipschitz Bandits" by Wang, Ye, Geng and Rudin (https://dl.acm.org/doi/10.1145/3412815.3416885), the authors used a modified version of the DecisionTreeRegressor in their algorithm. More specifically, they used number of samples in the leaf nodes to quantify the uncertainty level of the predictions. It seems that the method in this paper is a bit useful. 

It would be great if "DecisionTreeRegressor" object in the official scikit-learn package could output number of samples in the leaf nodes. 

### Describe your proposed solution

When fitting the DecisionTreeRegressor, in addition to record mean of the leaf nodes, also record number of samples used to compute the mean. When output a prediction, in addition to output of mean of the corresponding leaf node, also output the number of training samples used in the prediction. 

I'm a coauthor of the paper "Towards Practical Lipschitz Bandits", and have a local copy of our solution when writing the paper. I just forked the master branch of the official scikit-learn repo, and replace the folder "sklearn" by our local copy. This fork is at: https://github.com/wangt1anyu/scikit-learn. Our code is a bit old, and may not be optimized in some aspects. But maybe it's useful as a reference. 

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
If this one is solved, it should be closed then.

Labels: New Feature, Needs Triage
`check_array` error on Pandas series is confusing

### Describe the bug

I don't know if this is a bug or a feature request.

When inputing a Pandas or Polars series for estimators or transformers accepting only 2D arrays, `check_array()` raises the following error:
```
ValueError: Expected 2D array, got 1D array instead:
array=[1. 2. 3.].
Reshape your data either using array.reshape(-1, 1) if your data has a single feature or array.reshape(1, -1) if it contains a single sample.
```
This is fine for arrays but for Pandas or Polars series this is confusing since using `reshape` will raise an error.

### Steps/Code to Reproduce

```python
import pandas as pd
from sklearn.preprocessing import StandardScaler

df = pd.DataFrame(dict(a=[1, 2, 3], b=["a", "b", "c"]))
StandardScaler().fit_transform(df["a"])
```

### Expected Results

An adapted error message to inform on what to do with a series and not an array.

```
ValueError: Expected a dataframe, got series instead:
0    1
1    2
2    3
Name: a, dtype: int64.
Pass a dataframe instead of a series with df[[column_name]] instead of df[column_name].
```

### Actual Results

```
ValueError: Expected 2D array, got 1D array instead:
array=[1. 2. 3.].
Reshape your data either using array.reshape(-1, 1) if your data has a single feature or array.reshape(1, -1) if it contains a single sample.
```

### Versions

```shell
System:
    python: 3.10.11 | packaged by conda-forge | (main, May 10 2023, 19:07:22) [Clang 14.0.6 ]
executable: /Users/vincentmaladiere/mambaforge/envs/skrub/bin/python3.10
   machine: macOS-11.7.9-x86_64-i386-64bit

Python dependencies:
      sklearn: 1.3.0
          pip: 23.1.2
   setuptools: 67.7.2
        numpy: 1.25.2
        scipy: 1.10.1
       Cython: None
       pandas: 2.1.0rc0
   matplotlib: 3.7.1
       joblib: 1.2.0
threadpoolctl: 3.1.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: openblas
         prefix: libopenblas
       filepath: /Users/vincentmaladiere/mambaforge/envs/skrub/lib/python3.10/site-packages/numpy/.dylibs/libopenblas64_.0.dylib
        version: 0.3.23.dev
threading_layer: pthreads
   architecture: Haswell
    num_threads: 4

       user_api: openmp
   internal_api: openmp
         prefix: libomp
       filepath: /Users/vincentmaladiere/mambaforge/envs/skrub/lib/python3.10/site-packages/sklearn/.dylibs/libomp.dylib
        version: None
    num_threads: 8

       user_api: blas
   internal_api: openblas
         prefix: libopenblas
       filepath: /Users/vincentmaladiere/mambaforge/envs/skrub/lib/python3.10/site-packages/scipy/.dylibs/libopenblas.0.dylib
        version: 0.3.18
threading_layer: pthreads
   architecture: SkylakeX
    num_threads: 4
```

COMMENT:
can i try this issue

Labels: Enhancement