Talip7/scikit-learn-issues-multilabel huggingface dataset:
Segmentation fault occurs when KernelPCA is loaded after torchvision

### Describe the bug

If KernelPCA is loaded after torchvision, then I get a segmentation fault. If I load them in the opposite order then I get no segmentaton fault.

### Steps/Code to Reproduce

```
import torchvision
from sklearn.decomposition import KernelPCA

import numpy as np

transformer = KernelPCA(n_components=7, kernel='rbf')
X_transformed = transformer.fit_transform(np.zeros([100,100]))
```

### Expected Results

No segmentation fault

### Actual Results

Segmentation fault (core dumped)

### Versions

```shell
System:
    python: 3.8.10 (default, Mar 15 2022, 12:22:08)  [GCC 9.4.0]
executable: /usr/bin/python3
   machine: Linux-5.4.0-77-generic-x86_64-with-glibc2.29

Python dependencies:
          pip: 22.1
   setuptools: 45.2.0
      sklearn: 0.24.2
        numpy: 1.22.4
        scipy: 1.4.1
       Cython: 0.29.23
       pandas: 1.4.3
   matplotlib: 3.5.2
       joblib: 1.0.1
threadpoolctl: 2.1.0

Built with OpenMP: True


torchvision version: 0.10.0+cu102
```

COMMENT:
It might be because `torch` tweeks the dynamic libraries that it wants to load internally... Honestly I don't know much about those internals. I think I remember being told that `import torch` before `import torchvision` may help. @LorenzoLuzi what versions of pytorch and torchvision are you using? You can paste the output of:

```
wget https://raw.githubusercontent.com/pytorch/pytorch/master/torch/utils/collect_env.py
# For security purposes, please check the contents of collect_env.py before running it.
python collect_env.py
```


(also obligatory: the snippet above doesn't segfault on my env)

Labels: Bug, module:decomposition
Segmentation fault occurs when KernelPCA is loaded after torchvision

### Describe the bug

If KernelPCA is loaded after torchvision, then I get a segmentation fault. If I load them in the opposite order then I get no segmentaton fault.

### Steps/Code to Reproduce

```
import torchvision
from sklearn.decomposition import KernelPCA

import numpy as np

transformer = KernelPCA(n_components=7, kernel='rbf')
X_transformed = transformer.fit_transform(np.zeros([100,100]))
```

### Expected Results

No segmentation fault

### Actual Results

Segmentation fault (core dumped)

### Versions

```shell
System:
    python: 3.8.10 (default, Mar 15 2022, 12:22:08)  [GCC 9.4.0]
executable: /usr/bin/python3
   machine: Linux-5.4.0-77-generic-x86_64-with-glibc2.29

Python dependencies:
          pip: 22.1
   setuptools: 45.2.0
      sklearn: 0.24.2
        numpy: 1.22.4
        scipy: 1.4.1
       Cython: 0.29.23
       pandas: 1.4.3
   matplotlib: 3.5.2
       joblib: 1.0.1
threadpoolctl: 2.1.0

Built with OpenMP: True


torchvision version: 0.10.0+cu102
```

COMMENT:
Sure this is the output:

```
Collecting environment information...
PyTorch version: 1.11.0+cu102
Is debug build: False
CUDA used to build PyTorch: 10.2
ROCM used to build PyTorch: N/A

OS: Ubuntu 20.04.4 LTS (x86_64)
GCC version: (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0
Clang version: Could not collect
CMake version: Could not collect
Libc version: glibc-2.31

Python version: 3.8.10 (default, Mar 15 2022, 12:22:08)  [GCC 9.4.0] (64-bit runtime)
Python platform: Linux-5.4.0-77-generic-x86_64-with-glibc2.29
Is CUDA available: True
CUDA runtime version: 11.2.67
CUDA_MODULE_LOADING set to: 
GPU models and configuration: 
GPU 0: NVIDIA TITAN X (Pascal)
GPU 1: NVIDIA TITAN X (Pascal)
GPU 2: NVIDIA TITAN X (Pascal)
GPU 3: NVIDIA TITAN X (Pascal)
GPU 4: NVIDIA TITAN X (Pascal)
GPU 5: NVIDIA TITAN X (Pascal)
GPU 6: NVIDIA TITAN X (Pascal)
GPU 7: NVIDIA TITAN X (Pascal)

Nvidia driver version: 465.19.01
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] numpy==1.22.4
[pip3] pytorch-lightning==1.6.4
[pip3] pytorch-pretrained-gans==0.0.1
[pip3] stylegan2-pytorch==1.8.6
[pip3] torch==1.11.0
[pip3] torchmetrics==0.5.1
[pip3] torchvision==0.10.0
[pip3] torchviz==0.0.2
[pip3] vector-quantize-pytorch==0.2.2
[conda] No relevant packages
```

Labels: Bug, module:decomposition
Segmentation fault occurs when KernelPCA is loaded after torchvision

### Describe the bug

If KernelPCA is loaded after torchvision, then I get a segmentation fault. If I load them in the opposite order then I get no segmentaton fault.

### Steps/Code to Reproduce

```
import torchvision
from sklearn.decomposition import KernelPCA

import numpy as np

transformer = KernelPCA(n_components=7, kernel='rbf')
X_transformed = transformer.fit_transform(np.zeros([100,100]))
```

### Expected Results

No segmentation fault

### Actual Results

Segmentation fault (core dumped)

### Versions

```shell
System:
    python: 3.8.10 (default, Mar 15 2022, 12:22:08)  [GCC 9.4.0]
executable: /usr/bin/python3
   machine: Linux-5.4.0-77-generic-x86_64-with-glibc2.29

Python dependencies:
          pip: 22.1
   setuptools: 45.2.0
      sklearn: 0.24.2
        numpy: 1.22.4
        scipy: 1.4.1
       Cython: 0.29.23
       pandas: 1.4.3
   matplotlib: 3.5.2
       joblib: 1.0.1
threadpoolctl: 2.1.0

Built with OpenMP: True


torchvision version: 0.10.0+cu102
```

COMMENT:
```
[pip3] torch==1.11.0
[pip3] torchvision==0.10.0
```

The 0.10 version of torchvision is only relly compatible with torch 1.9 (see our [version table](https://github.com/pytorch/vision#installation)).
You may want to simply upgrade to torch==1.13 and torchvision==0.14 as well.

HTH

Labels: Bug, module:decomposition
ENH partial_dependece plot for HistGradientBoosting estimator fitted with `sample_weight`

### Describe the workflow you want to enable

As partial dependence of a model at a point [is defined as an expectation](https://scikit-learn.org/stable/modules/partial_dependence.html#mathematical-definition), it should respect `sample_weight` if someone wishes to use it (for instance, when you know your `X` does not follow the distribution you are interested in).

#25209 tries to solve this for `method='brute'` when you have new `X`. For older tree-based models trained with sample_weights, [`method='recursion'`](https://nicolas-hug.com/blog/pdps) keeps track of the training `sample_weight` and calculates the `partial_dependece` with that into consideration.

But, as discussed during the implementation of `sample_weight` on the HistGradientBoosting estimators (https://github.com/scikit-learn/scikit-learn/pull/14696#issuecomment-548295813), these models stores an attribute `_fitted_with_sw` and when `partial_dependece` with recursion is asked, it throws an error:

https://github.com/scikit-learn/scikit-learn/blob/205f3b76ef8500c90c346c6c6fb6f4e589368278/sklearn/ensemble/_hist_gradient_boosting/gradient_boosting.py#L1142-L1148


### Describe your proposed solution

As discussed in https://github.com/scikit-learn/scikit-learn/issues/24872#issuecomment-1352354690, the big difference between other tree-based algorithms and HistGradientBoosting is that HistGradientBoosting does not save the `weighted_n_node_samples` when building the tree.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
It is not clear to me why it would be hard to keep track of `weighted_n_node_samples` as intuitively the tree should be using it during loss/criterion calculations at each node. But on the original PR for implemeting `sample_weight` on HistGradientBoosting, @adrinjalali points out that:
> adding sample weight support for pdp requires adding a weighted_count to node_struct. It'll be a non-trivial change

@adrinjalali, would you mind explaining why do you find it non-trivial to do? I would love to try to solve this issue if I can get a little push. :D

Labels: New Feature, module:ensemble
ENH partial_dependece plot for HistGradientBoosting estimator fitted with `sample_weight`

### Describe the workflow you want to enable

As partial dependence of a model at a point [is defined as an expectation](https://scikit-learn.org/stable/modules/partial_dependence.html#mathematical-definition), it should respect `sample_weight` if someone wishes to use it (for instance, when you know your `X` does not follow the distribution you are interested in).

#25209 tries to solve this for `method='brute'` when you have new `X`. For older tree-based models trained with sample_weights, [`method='recursion'`](https://nicolas-hug.com/blog/pdps) keeps track of the training `sample_weight` and calculates the `partial_dependece` with that into consideration.

But, as discussed during the implementation of `sample_weight` on the HistGradientBoosting estimators (https://github.com/scikit-learn/scikit-learn/pull/14696#issuecomment-548295813), these models stores an attribute `_fitted_with_sw` and when `partial_dependece` with recursion is asked, it throws an error:

https://github.com/scikit-learn/scikit-learn/blob/205f3b76ef8500c90c346c6c6fb6f4e589368278/sklearn/ensemble/_hist_gradient_boosting/gradient_boosting.py#L1142-L1148


### Describe your proposed solution

As discussed in https://github.com/scikit-learn/scikit-learn/issues/24872#issuecomment-1352354690, the big difference between other tree-based algorithms and HistGradientBoosting is that HistGradientBoosting does not save the `weighted_n_node_samples` when building the tree.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
IIRC it's because it'd increase the memory footprint of the model, as well as slowing the model down. I'm not opposed to adding them, but adding them in #14696 would have meant requiring more benchmarks, and more tests, which wasn't ideal since that PR took quite a while and quite an effort to finish in the first place.

It's certainly doable as a separate PR, and happy if you're up for pushing a PR for it, but be warned that it might be a non-trivial PR 😁

Labels: New Feature, module:ensemble
ENH partial_dependece plot for HistGradientBoosting estimator fitted with `sample_weight`

### Describe the workflow you want to enable

As partial dependence of a model at a point [is defined as an expectation](https://scikit-learn.org/stable/modules/partial_dependence.html#mathematical-definition), it should respect `sample_weight` if someone wishes to use it (for instance, when you know your `X` does not follow the distribution you are interested in).

#25209 tries to solve this for `method='brute'` when you have new `X`. For older tree-based models trained with sample_weights, [`method='recursion'`](https://nicolas-hug.com/blog/pdps) keeps track of the training `sample_weight` and calculates the `partial_dependece` with that into consideration.

But, as discussed during the implementation of `sample_weight` on the HistGradientBoosting estimators (https://github.com/scikit-learn/scikit-learn/pull/14696#issuecomment-548295813), these models stores an attribute `_fitted_with_sw` and when `partial_dependece` with recursion is asked, it throws an error:

https://github.com/scikit-learn/scikit-learn/blob/205f3b76ef8500c90c346c6c6fb6f4e589368278/sklearn/ensemble/_hist_gradient_boosting/gradient_boosting.py#L1142-L1148


### Describe your proposed solution

As discussed in https://github.com/scikit-learn/scikit-learn/issues/24872#issuecomment-1352354690, the big difference between other tree-based algorithms and HistGradientBoosting is that HistGradientBoosting does not save the `weighted_n_node_samples` when building the tree.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
@vitaliset, have you started working on this? I would like to take this on as well.

Labels: New Feature, module:ensemble
ENH partial_dependece plot for HistGradientBoosting estimator fitted with `sample_weight`

### Describe the workflow you want to enable

As partial dependence of a model at a point [is defined as an expectation](https://scikit-learn.org/stable/modules/partial_dependence.html#mathematical-definition), it should respect `sample_weight` if someone wishes to use it (for instance, when you know your `X` does not follow the distribution you are interested in).

#25209 tries to solve this for `method='brute'` when you have new `X`. For older tree-based models trained with sample_weights, [`method='recursion'`](https://nicolas-hug.com/blog/pdps) keeps track of the training `sample_weight` and calculates the `partial_dependece` with that into consideration.

But, as discussed during the implementation of `sample_weight` on the HistGradientBoosting estimators (https://github.com/scikit-learn/scikit-learn/pull/14696#issuecomment-548295813), these models stores an attribute `_fitted_with_sw` and when `partial_dependece` with recursion is asked, it throws an error:

https://github.com/scikit-learn/scikit-learn/blob/205f3b76ef8500c90c346c6c6fb6f4e589368278/sklearn/ensemble/_hist_gradient_boosting/gradient_boosting.py#L1142-L1148


### Describe your proposed solution

As discussed in https://github.com/scikit-learn/scikit-learn/issues/24872#issuecomment-1352354690, the big difference between other tree-based algorithms and HistGradientBoosting is that HistGradientBoosting does not save the `weighted_n_node_samples` when building the tree.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
Feel free to jump in and take it @Andrew-Wang-IB45. I will not be able to start it in the next few weeks and would be super happy if you could come with something in the meantime!

Please take a look at the issues I mentioned in the description of this one (especially 24872) for discussions related to the solution.

From my point of view, the solution involves two parts (maybe we should have two sequential PRs for easier reviews):

1 - Update the training function of the base estimator of HistGradientBoosting, so it keeps track of the `weighted_n_node_samples` (just like the other tree-based models do).
- For this, we should be *benchmarking performance and memory footprint* as HistGradientBoosting should be an efficient estimator (it will probably still be, but we need to check if we are not messing it up).

2 - Update the `compute_partial_dependence` function of the base estimators of HistGradientBoosting, so it looks at the new `weighted_n_node_samples` instead of the count.

What do you think? Does this approach make sense to you?  

Please reach out and ask for help if you need it! :)

Labels: New Feature, module:ensemble
ENH partial_dependece plot for HistGradientBoosting estimator fitted with `sample_weight`

### Describe the workflow you want to enable

As partial dependence of a model at a point [is defined as an expectation](https://scikit-learn.org/stable/modules/partial_dependence.html#mathematical-definition), it should respect `sample_weight` if someone wishes to use it (for instance, when you know your `X` does not follow the distribution you are interested in).

#25209 tries to solve this for `method='brute'` when you have new `X`. For older tree-based models trained with sample_weights, [`method='recursion'`](https://nicolas-hug.com/blog/pdps) keeps track of the training `sample_weight` and calculates the `partial_dependece` with that into consideration.

But, as discussed during the implementation of `sample_weight` on the HistGradientBoosting estimators (https://github.com/scikit-learn/scikit-learn/pull/14696#issuecomment-548295813), these models stores an attribute `_fitted_with_sw` and when `partial_dependece` with recursion is asked, it throws an error:

https://github.com/scikit-learn/scikit-learn/blob/205f3b76ef8500c90c346c6c6fb6f4e589368278/sklearn/ensemble/_hist_gradient_boosting/gradient_boosting.py#L1142-L1148


### Describe your proposed solution

As discussed in https://github.com/scikit-learn/scikit-learn/issues/24872#issuecomment-1352354690, the big difference between other tree-based algorithms and HistGradientBoosting is that HistGradientBoosting does not save the `weighted_n_node_samples` when building the tree.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
Sounds good, @vitaliset. I will review the issues and pull requests that you have brought up. Your approach makes sense and I will follow that when implementing the solution. I'll let you know how things are coming along and when I need help. Thanks!

Labels: New Feature, module:ensemble
ENH partial_dependece plot for HistGradientBoosting estimator fitted with `sample_weight`

### Describe the workflow you want to enable

As partial dependence of a model at a point [is defined as an expectation](https://scikit-learn.org/stable/modules/partial_dependence.html#mathematical-definition), it should respect `sample_weight` if someone wishes to use it (for instance, when you know your `X` does not follow the distribution you are interested in).

#25209 tries to solve this for `method='brute'` when you have new `X`. For older tree-based models trained with sample_weights, [`method='recursion'`](https://nicolas-hug.com/blog/pdps) keeps track of the training `sample_weight` and calculates the `partial_dependece` with that into consideration.

But, as discussed during the implementation of `sample_weight` on the HistGradientBoosting estimators (https://github.com/scikit-learn/scikit-learn/pull/14696#issuecomment-548295813), these models stores an attribute `_fitted_with_sw` and when `partial_dependece` with recursion is asked, it throws an error:

https://github.com/scikit-learn/scikit-learn/blob/205f3b76ef8500c90c346c6c6fb6f4e589368278/sklearn/ensemble/_hist_gradient_boosting/gradient_boosting.py#L1142-L1148


### Describe your proposed solution

As discussed in https://github.com/scikit-learn/scikit-learn/issues/24872#issuecomment-1352354690, the big difference between other tree-based algorithms and HistGradientBoosting is that HistGradientBoosting does not save the `weighted_n_node_samples` when building the tree.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
I edited the last comment as the test I suggested does not make sense: when training with `sample_weight`, the model will be different from the one trained without it... So the leaf values for each model should be different, and even though the aggregate sum is averaging out according to `sample_weight`, the values we get will be different per se.

Nonetheless, some small tests should be adapted once the fix done. For instance:
https://github.com/scikit-learn/scikit-learn/blob/c1cfc4d4f36f9c00413e20d0ef85bed208a502ca/sklearn/inspection/tests/test_partial_dependence.py#L661
https://github.com/scikit-learn/scikit-learn/blob/c1cfc4d4f36f9c00413e20d0ef85bed208a502ca/sklearn/inspection/tests/test_partial_dependence.py#L638

Labels: New Feature, module:ensemble
ENH partial_dependece plot for HistGradientBoosting estimator fitted with `sample_weight`

### Describe the workflow you want to enable

As partial dependence of a model at a point [is defined as an expectation](https://scikit-learn.org/stable/modules/partial_dependence.html#mathematical-definition), it should respect `sample_weight` if someone wishes to use it (for instance, when you know your `X` does not follow the distribution you are interested in).

#25209 tries to solve this for `method='brute'` when you have new `X`. For older tree-based models trained with sample_weights, [`method='recursion'`](https://nicolas-hug.com/blog/pdps) keeps track of the training `sample_weight` and calculates the `partial_dependece` with that into consideration.

But, as discussed during the implementation of `sample_weight` on the HistGradientBoosting estimators (https://github.com/scikit-learn/scikit-learn/pull/14696#issuecomment-548295813), these models stores an attribute `_fitted_with_sw` and when `partial_dependece` with recursion is asked, it throws an error:

https://github.com/scikit-learn/scikit-learn/blob/205f3b76ef8500c90c346c6c6fb6f4e589368278/sklearn/ensemble/_hist_gradient_boosting/gradient_boosting.py#L1142-L1148


### Describe your proposed solution

As discussed in https://github.com/scikit-learn/scikit-learn/issues/24872#issuecomment-1352354690, the big difference between other tree-based algorithms and HistGradientBoosting is that HistGradientBoosting does not save the `weighted_n_node_samples` when building the tree.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
Hi, thanks for your feedback and suggestions! I will get to it and keep you updated.

Labels: New Feature, module:ensemble
ENH partial_dependece plot for HistGradientBoosting estimator fitted with `sample_weight`

### Describe the workflow you want to enable

As partial dependence of a model at a point [is defined as an expectation](https://scikit-learn.org/stable/modules/partial_dependence.html#mathematical-definition), it should respect `sample_weight` if someone wishes to use it (for instance, when you know your `X` does not follow the distribution you are interested in).

#25209 tries to solve this for `method='brute'` when you have new `X`. For older tree-based models trained with sample_weights, [`method='recursion'`](https://nicolas-hug.com/blog/pdps) keeps track of the training `sample_weight` and calculates the `partial_dependece` with that into consideration.

But, as discussed during the implementation of `sample_weight` on the HistGradientBoosting estimators (https://github.com/scikit-learn/scikit-learn/pull/14696#issuecomment-548295813), these models stores an attribute `_fitted_with_sw` and when `partial_dependece` with recursion is asked, it throws an error:

https://github.com/scikit-learn/scikit-learn/blob/205f3b76ef8500c90c346c6c6fb6f4e589368278/sklearn/ensemble/_hist_gradient_boosting/gradient_boosting.py#L1142-L1148


### Describe your proposed solution

As discussed in https://github.com/scikit-learn/scikit-learn/issues/24872#issuecomment-1352354690, the big difference between other tree-based algorithms and HistGradientBoosting is that HistGradientBoosting does not save the `weighted_n_node_samples` when building the tree.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
Hi @vitaliset, I have updated the BaseHistGradientBoosting fit function and the Grower class to keep track of `weighted_n_node_samples` using the `sample_weight` passed in. I noticed that for the BaseDecisionTree, one of its attributes came from the parameter `min_weight_fraction_leaf`, which is used to calculate the `min_weight_leaf`, which itself is used to determine if a node is a leaf, which are respectively https://github.com/scikit-learn/scikit-learn/blob/bf03a6354670414695ef483de8187135aeec6cbd/sklearn/tree/_classes.py#L143 https://github.com/scikit-learn/scikit-learn/blob/bf03a6354670414695ef483de8187135aeec6cbd/sklearn/tree/_classes.py#L316-L320 https://github.com/scikit-learn/scikit-learn/blob/bf03a6354670414695ef483de8187135aeec6cbd/sklearn/tree/_tree.pyx#L221-L224
While I think that is is good to include this in the tree for the Grower, doing so would require changing the signature of the init function for BaseHistGradientBoosting. I am wondering if we should change it now for completeness or leave it as is to avoid changing the public interface too much.

Regardless, I will do some tests to check for its correctness and performance. Do you have a small example of the expected outputs of the BaseHistGradientBoosting with and without sample weights?

Labels: New Feature, module:ensemble