Talip7/scikit-learn-issues-multilabel huggingface dataset:
Add support for bools in `SimpleImputer`

### Describe the workflow you want to enable

Suppose you wanna impute a bool array. Because it has NaNs, it's gonna be of dtype float and work fine:

```pycon
>>> np.asarray([[True, False, np.nan]]).dtype
dtype('float64')
```

However, now suppose that the value you pass actually has no NaNs (e.g., because the current data happens to have no NaNs), so there's nothing to impute. In this case, the array is of type bool:

```pycon
>>> np.asarray([[True, False]]).dtype
dtype('bool')
```

Which makes the imputation fail because `bool` isn't supported:

```python
import numpy as np
from sklearn.impute import SimpleImputer

SimpleImputer(strategy="most_frequent").fit(np.asarray([[True, False]]))
```

This previous code generates an exception:

```
Traceback (most recent call last):
  # ...
  File ".../lib/python3.10/site-packages/sklearn/utils/_set_output.py", line 140, in wrapped
    data_to_wrap = f(self, X, *args, **kwargs)
  File ".../lib/python3.10/site-packages/sklearn/base.py", line 878, in fit_transform
    return self.fit(X, **fit_params).transform(X)
  File ".../lib/python3.10/site-packages/sklearn/impute/_base.py", line 390, in fit
    X = self._validate_input(X, in_fit=True)
  File ".../lib/python3.10/site-packages/sklearn/impute/_base.py", line 352, in _validate_input
    raise ValueError(
ValueError: SimpleImputer does not support data with dtype bool. Please provide either a numeric array (with a floating point or integer dtype) or categorical data represented either as an array with integer dtype or an array of string values with an object type.
```

This is because `np.asarray([[True, False]]).dtype.kind` is "b", which is not in {"i", "u", "f", "O"} and so it fails.

### Describe your proposed solution

My solution is to support `bool`. I believe it shouldn't be a great effort.

### Describe alternatives you've considered, if relevant

An alternative solution is for me to create a custom imputer class that copy-pastes the behavior I want from `SimpleImputer` and makes it work for bools.

### Additional context

_No response_

COMMENT:
This is not an easy issue, and we'd need to support extension arrays as well (like arrays with `pd.NA` and `boolean` dtype). This should also be fixed in all imputers.

Labels: New Feature, Moderate, help wanted
Add support for bools in `SimpleImputer`

### Describe the workflow you want to enable

Suppose you wanna impute a bool array. Because it has NaNs, it's gonna be of dtype float and work fine:

```pycon
>>> np.asarray([[True, False, np.nan]]).dtype
dtype('float64')
```

However, now suppose that the value you pass actually has no NaNs (e.g., because the current data happens to have no NaNs), so there's nothing to impute. In this case, the array is of type bool:

```pycon
>>> np.asarray([[True, False]]).dtype
dtype('bool')
```

Which makes the imputation fail because `bool` isn't supported:

```python
import numpy as np
from sklearn.impute import SimpleImputer

SimpleImputer(strategy="most_frequent").fit(np.asarray([[True, False]]))
```

This previous code generates an exception:

```
Traceback (most recent call last):
  # ...
  File ".../lib/python3.10/site-packages/sklearn/utils/_set_output.py", line 140, in wrapped
    data_to_wrap = f(self, X, *args, **kwargs)
  File ".../lib/python3.10/site-packages/sklearn/base.py", line 878, in fit_transform
    return self.fit(X, **fit_params).transform(X)
  File ".../lib/python3.10/site-packages/sklearn/impute/_base.py", line 390, in fit
    X = self._validate_input(X, in_fit=True)
  File ".../lib/python3.10/site-packages/sklearn/impute/_base.py", line 352, in _validate_input
    raise ValueError(
ValueError: SimpleImputer does not support data with dtype bool. Please provide either a numeric array (with a floating point or integer dtype) or categorical data represented either as an array with integer dtype or an array of string values with an object type.
```

This is because `np.asarray([[True, False]]).dtype.kind` is "b", which is not in {"i", "u", "f", "O"} and so it fails.

### Describe your proposed solution

My solution is to support `bool`. I believe it shouldn't be a great effort.

### Describe alternatives you've considered, if relevant

An alternative solution is for me to create a custom imputer class that copy-pastes the behavior I want from `SimpleImputer` and makes it work for bools.

### Additional context

_No response_

COMMENT:
I think that bools that had NaNs were working fine for me; the problem was when they did not have them.

Labels: New Feature, Moderate, help wanted
Add support for bools in `SimpleImputer`

### Describe the workflow you want to enable

Suppose you wanna impute a bool array. Because it has NaNs, it's gonna be of dtype float and work fine:

```pycon
>>> np.asarray([[True, False, np.nan]]).dtype
dtype('float64')
```

However, now suppose that the value you pass actually has no NaNs (e.g., because the current data happens to have no NaNs), so there's nothing to impute. In this case, the array is of type bool:

```pycon
>>> np.asarray([[True, False]]).dtype
dtype('bool')
```

Which makes the imputation fail because `bool` isn't supported:

```python
import numpy as np
from sklearn.impute import SimpleImputer

SimpleImputer(strategy="most_frequent").fit(np.asarray([[True, False]]))
```

This previous code generates an exception:

```
Traceback (most recent call last):
  # ...
  File ".../lib/python3.10/site-packages/sklearn/utils/_set_output.py", line 140, in wrapped
    data_to_wrap = f(self, X, *args, **kwargs)
  File ".../lib/python3.10/site-packages/sklearn/base.py", line 878, in fit_transform
    return self.fit(X, **fit_params).transform(X)
  File ".../lib/python3.10/site-packages/sklearn/impute/_base.py", line 390, in fit
    X = self._validate_input(X, in_fit=True)
  File ".../lib/python3.10/site-packages/sklearn/impute/_base.py", line 352, in _validate_input
    raise ValueError(
ValueError: SimpleImputer does not support data with dtype bool. Please provide either a numeric array (with a floating point or integer dtype) or categorical data represented either as an array with integer dtype or an array of string values with an object type.
```

This is because `np.asarray([[True, False]]).dtype.kind` is "b", which is not in {"i", "u", "f", "O"} and so it fails.

### Describe your proposed solution

My solution is to support `bool`. I believe it shouldn't be a great effort.

### Describe alternatives you've considered, if relevant

An alternative solution is for me to create a custom imputer class that copy-pastes the behavior I want from `SimpleImputer` and makes it work for bools.

### Additional context

_No response_

COMMENT:
Yes, that's because if there's a nan, the whole array is converted into a float64 array, and then everything works fine. Things also work fine if you convert them to dtype=object.

Labels: New Feature, Moderate, help wanted
Add support for bools in `SimpleImputer`

### Describe the workflow you want to enable

Suppose you wanna impute a bool array. Because it has NaNs, it's gonna be of dtype float and work fine:

```pycon
>>> np.asarray([[True, False, np.nan]]).dtype
dtype('float64')
```

However, now suppose that the value you pass actually has no NaNs (e.g., because the current data happens to have no NaNs), so there's nothing to impute. In this case, the array is of type bool:

```pycon
>>> np.asarray([[True, False]]).dtype
dtype('bool')
```

Which makes the imputation fail because `bool` isn't supported:

```python
import numpy as np
from sklearn.impute import SimpleImputer

SimpleImputer(strategy="most_frequent").fit(np.asarray([[True, False]]))
```

This previous code generates an exception:

```
Traceback (most recent call last):
  # ...
  File ".../lib/python3.10/site-packages/sklearn/utils/_set_output.py", line 140, in wrapped
    data_to_wrap = f(self, X, *args, **kwargs)
  File ".../lib/python3.10/site-packages/sklearn/base.py", line 878, in fit_transform
    return self.fit(X, **fit_params).transform(X)
  File ".../lib/python3.10/site-packages/sklearn/impute/_base.py", line 390, in fit
    X = self._validate_input(X, in_fit=True)
  File ".../lib/python3.10/site-packages/sklearn/impute/_base.py", line 352, in _validate_input
    raise ValueError(
ValueError: SimpleImputer does not support data with dtype bool. Please provide either a numeric array (with a floating point or integer dtype) or categorical data represented either as an array with integer dtype or an array of string values with an object type.
```

This is because `np.asarray([[True, False]]).dtype.kind` is "b", which is not in {"i", "u", "f", "O"} and so it fails.

### Describe your proposed solution

My solution is to support `bool`. I believe it shouldn't be a great effort.

### Describe alternatives you've considered, if relevant

An alternative solution is for me to create a custom imputer class that copy-pastes the behavior I want from `SimpleImputer` and makes it work for bools.

### Additional context

_No response_

COMMENT:
Only if the strategy selected is "most_frequent" or "constant" should we permit imputing boolean arrays. We must validate the fill_value for the "constant" strategy to see if it is a boolean or not, and if it is, we must fill the NA values with the appropriate boolean value. The outcome is evident in the case of the "most_frequent" method. The array's most prevalent boolean value must be used to swap out the NA values for them. Also, the fill_value should be set to False by default.

To implement this modification, I would like to submit a pull request.

Labels: New Feature, Moderate, help wanted
Add support for bools in `SimpleImputer`

### Describe the workflow you want to enable

Suppose you wanna impute a bool array. Because it has NaNs, it's gonna be of dtype float and work fine:

```pycon
>>> np.asarray([[True, False, np.nan]]).dtype
dtype('float64')
```

However, now suppose that the value you pass actually has no NaNs (e.g., because the current data happens to have no NaNs), so there's nothing to impute. In this case, the array is of type bool:

```pycon
>>> np.asarray([[True, False]]).dtype
dtype('bool')
```

Which makes the imputation fail because `bool` isn't supported:

```python
import numpy as np
from sklearn.impute import SimpleImputer

SimpleImputer(strategy="most_frequent").fit(np.asarray([[True, False]]))
```

This previous code generates an exception:

```
Traceback (most recent call last):
  # ...
  File ".../lib/python3.10/site-packages/sklearn/utils/_set_output.py", line 140, in wrapped
    data_to_wrap = f(self, X, *args, **kwargs)
  File ".../lib/python3.10/site-packages/sklearn/base.py", line 878, in fit_transform
    return self.fit(X, **fit_params).transform(X)
  File ".../lib/python3.10/site-packages/sklearn/impute/_base.py", line 390, in fit
    X = self._validate_input(X, in_fit=True)
  File ".../lib/python3.10/site-packages/sklearn/impute/_base.py", line 352, in _validate_input
    raise ValueError(
ValueError: SimpleImputer does not support data with dtype bool. Please provide either a numeric array (with a floating point or integer dtype) or categorical data represented either as an array with integer dtype or an array of string values with an object type.
```

This is because `np.asarray([[True, False]]).dtype.kind` is "b", which is not in {"i", "u", "f", "O"} and so it fails.

### Describe your proposed solution

My solution is to support `bool`. I believe it shouldn't be a great effort.

### Describe alternatives you've considered, if relevant

An alternative solution is for me to create a custom imputer class that copy-pastes the behavior I want from `SimpleImputer` and makes it work for bools.

### Additional context

_No response_

COMMENT:
Could we possibly do something like this?


```
        if X.dtype.kind == 'b':
            if self.strategy == 'most_frequent':
                # For most_frequent strategy, impute with the most frequent boolean value
                most_frequent_bool = np.bincount(X.astype(int)).argmax()
                self.fill_value = bool(most_frequent_bool)
            elif self.strategy == 'constant':
                # For constant strategy, use the provided fill_value if it's boolean
                if isinstance(self.fill_value, bool):
                    self.fill_value = bool(self.fill_value)
                else:
                    raise ValueError("fill_value should be a boolean for constant strategy.")
```

Labels: New Feature, Moderate, help wanted
Add support for bools in `SimpleImputer`

### Describe the workflow you want to enable

Suppose you wanna impute a bool array. Because it has NaNs, it's gonna be of dtype float and work fine:

```pycon
>>> np.asarray([[True, False, np.nan]]).dtype
dtype('float64')
```

However, now suppose that the value you pass actually has no NaNs (e.g., because the current data happens to have no NaNs), so there's nothing to impute. In this case, the array is of type bool:

```pycon
>>> np.asarray([[True, False]]).dtype
dtype('bool')
```

Which makes the imputation fail because `bool` isn't supported:

```python
import numpy as np
from sklearn.impute import SimpleImputer

SimpleImputer(strategy="most_frequent").fit(np.asarray([[True, False]]))
```

This previous code generates an exception:

```
Traceback (most recent call last):
  # ...
  File ".../lib/python3.10/site-packages/sklearn/utils/_set_output.py", line 140, in wrapped
    data_to_wrap = f(self, X, *args, **kwargs)
  File ".../lib/python3.10/site-packages/sklearn/base.py", line 878, in fit_transform
    return self.fit(X, **fit_params).transform(X)
  File ".../lib/python3.10/site-packages/sklearn/impute/_base.py", line 390, in fit
    X = self._validate_input(X, in_fit=True)
  File ".../lib/python3.10/site-packages/sklearn/impute/_base.py", line 352, in _validate_input
    raise ValueError(
ValueError: SimpleImputer does not support data with dtype bool. Please provide either a numeric array (with a floating point or integer dtype) or categorical data represented either as an array with integer dtype or an array of string values with an object type.
```

This is because `np.asarray([[True, False]]).dtype.kind` is "b", which is not in {"i", "u", "f", "O"} and so it fails.

### Describe your proposed solution

My solution is to support `bool`. I believe it shouldn't be a great effort.

### Describe alternatives you've considered, if relevant

An alternative solution is for me to create a custom imputer class that copy-pastes the behavior I want from `SimpleImputer` and makes it work for bools.

### Additional context

_No response_

COMMENT:
@adrinjalali I have opened #31730 to support bools in `SimpleImputer`. Please take a look when you can and let me know if there are further changes needed.

Labels: New Feature, Moderate, help wanted
KernelPCA inverse transform behaves unexpectly.

### Describe the bug

Hi!
we wanted to use inverse transform in kernel PCA. There is a parameter gamma, which is defined as 1/num_features if gamma=None. However if gamma is provided as gamma=1/num_features, the resultant inverse transform results is different. Why is that? How the parameter gamma is related to inverse transform?

### Steps/Code to Reproduce

```
from sklearn.datasets import load_iris
from sklearn.decomposition import KernelPCA

#load iris dataset (features only)
data = load_iris().data

#number of features in the dataset
n_features = data.shape[1]

#run KernelPCA with default value of gamma (1/n_features)
pca1 = KernelPCA(n_components = 2, random_state = 7, fit_inverse_transform = True, kernel = 'rbf')
pca1.fit(data)

#set the value of gamma to 1/n_features
pca2 = KernelPCA(n_components = 2, gamma = 1/n_features, random_state = 7, fit_inverse_transform = True, kernel = 'rbf')
pca2.fit(data)

#eigenvalues and eigenvectors are the same
print((pca1.eigenvalues_ == pca2.eigenvalues_).all())
print((pca1.eigenvectors_ == pca2.eigenvectors_).all())

#projections of components are the same
latent1 = pca1.transform(data)
latent2 = pca2.transform(data)
print((latent1 == latent2).all())

#but reconstructed data from these projections are different
reconstructed_data1 = pca1.inverse_transform(latent1)
reconstructed_data2 = pca2.inverse_transform(latent2)
print((reconstructed_data1 == reconstructed_data2).all())
```

### Expected Results

True
True
True
True

### Actual Results

True
True
True
False

### Versions

```shell
System:
    python: 3.10.6 (main, Mar 10 2023, 10:55:28) [GCC 11.3.0]
executable: /usr/bin/python3
   machine: Linux-5.15.0-70-generic-x86_64-with-glibc2.35

Python dependencies:
      sklearn: 1.2.2
          pip: 22.0.2
   setuptools: 59.6.0
        numpy: 1.21.5
        scipy: 1.8.0
       Cython: None
       pandas: 2.0.0
   matplotlib: 3.7.1
       joblib: 1.2.0
threadpoolctl: 3.1.0

Built with OpenMP: True

threadpoolctl info:
       user_api: openmp
   internal_api: openmp
         prefix: libgomp
       filepath: /home/petr/.local/lib/python3.10/site-packages/scikit_learn.libs/libgomp-a34b3233.so.1.0.0
        version: None
    num_threads: 8

       user_api: blas
   internal_api: openblas
         prefix: libopenblas
       filepath: /usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.20.so
        version: 0.3.20
threading_layer: pthreads
   architecture: Haswell
    num_threads: 8
```

COMMENT:
From what I recall and what the documentation says, the `inverse_transform` is a learnt approximation: https://scikit-learn.org/stable/modules/generated/sklearn.decomposition.KernelPCA.html#sklearn.decomposition.KernelPCA.inverse_transform

It is not surprising to not recover the original data.

Labels: Bug, module:decomposition
KernelPCA inverse transform behaves unexpectly.

### Describe the bug

Hi!
we wanted to use inverse transform in kernel PCA. There is a parameter gamma, which is defined as 1/num_features if gamma=None. However if gamma is provided as gamma=1/num_features, the resultant inverse transform results is different. Why is that? How the parameter gamma is related to inverse transform?

### Steps/Code to Reproduce

```
from sklearn.datasets import load_iris
from sklearn.decomposition import KernelPCA

#load iris dataset (features only)
data = load_iris().data

#number of features in the dataset
n_features = data.shape[1]

#run KernelPCA with default value of gamma (1/n_features)
pca1 = KernelPCA(n_components = 2, random_state = 7, fit_inverse_transform = True, kernel = 'rbf')
pca1.fit(data)

#set the value of gamma to 1/n_features
pca2 = KernelPCA(n_components = 2, gamma = 1/n_features, random_state = 7, fit_inverse_transform = True, kernel = 'rbf')
pca2.fit(data)

#eigenvalues and eigenvectors are the same
print((pca1.eigenvalues_ == pca2.eigenvalues_).all())
print((pca1.eigenvectors_ == pca2.eigenvectors_).all())

#projections of components are the same
latent1 = pca1.transform(data)
latent2 = pca2.transform(data)
print((latent1 == latent2).all())

#but reconstructed data from these projections are different
reconstructed_data1 = pca1.inverse_transform(latent1)
reconstructed_data2 = pca2.inverse_transform(latent2)
print((reconstructed_data1 == reconstructed_data2).all())
```

### Expected Results

True
True
True
True

### Actual Results

True
True
True
False

### Versions

```shell
System:
    python: 3.10.6 (main, Mar 10 2023, 10:55:28) [GCC 11.3.0]
executable: /usr/bin/python3
   machine: Linux-5.15.0-70-generic-x86_64-with-glibc2.35

Python dependencies:
      sklearn: 1.2.2
          pip: 22.0.2
   setuptools: 59.6.0
        numpy: 1.21.5
        scipy: 1.8.0
       Cython: None
       pandas: 2.0.0
   matplotlib: 3.7.1
       joblib: 1.2.0
threadpoolctl: 3.1.0

Built with OpenMP: True

threadpoolctl info:
       user_api: openmp
   internal_api: openmp
         prefix: libgomp
       filepath: /home/petr/.local/lib/python3.10/site-packages/scikit_learn.libs/libgomp-a34b3233.so.1.0.0
        version: None
    num_threads: 8

       user_api: blas
   internal_api: openblas
         prefix: libopenblas
       filepath: /usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.20.so
        version: 0.3.20
threading_layer: pthreads
   architecture: Haswell
    num_threads: 8
```

COMMENT:
> It is not surprising to not recover the original data.

Yes but here it's not a comparison against the original data, it's a comparison between 2 run with the (supposed) same parameters.

Actually I think there's a bug here. 

- When `gamma` is None, its set to `1/n_feature` each time the kernel is called, not once at the beginning of fit. It means that when the kernel is called at `inverse_transform`, the number of features is different and hence gamma is different.
- When gamma is set, the same value is used at fit and at inverse_transform

Labels: Bug, module:decomposition
KernelPCA inverse transform behaves unexpectly.

### Describe the bug

Hi!
we wanted to use inverse transform in kernel PCA. There is a parameter gamma, which is defined as 1/num_features if gamma=None. However if gamma is provided as gamma=1/num_features, the resultant inverse transform results is different. Why is that? How the parameter gamma is related to inverse transform?

### Steps/Code to Reproduce

```
from sklearn.datasets import load_iris
from sklearn.decomposition import KernelPCA

#load iris dataset (features only)
data = load_iris().data

#number of features in the dataset
n_features = data.shape[1]

#run KernelPCA with default value of gamma (1/n_features)
pca1 = KernelPCA(n_components = 2, random_state = 7, fit_inverse_transform = True, kernel = 'rbf')
pca1.fit(data)

#set the value of gamma to 1/n_features
pca2 = KernelPCA(n_components = 2, gamma = 1/n_features, random_state = 7, fit_inverse_transform = True, kernel = 'rbf')
pca2.fit(data)

#eigenvalues and eigenvectors are the same
print((pca1.eigenvalues_ == pca2.eigenvalues_).all())
print((pca1.eigenvectors_ == pca2.eigenvectors_).all())

#projections of components are the same
latent1 = pca1.transform(data)
latent2 = pca2.transform(data)
print((latent1 == latent2).all())

#but reconstructed data from these projections are different
reconstructed_data1 = pca1.inverse_transform(latent1)
reconstructed_data2 = pca2.inverse_transform(latent2)
print((reconstructed_data1 == reconstructed_data2).all())
```

### Expected Results

True
True
True
True

### Actual Results

True
True
True
False

### Versions

```shell
System:
    python: 3.10.6 (main, Mar 10 2023, 10:55:28) [GCC 11.3.0]
executable: /usr/bin/python3
   machine: Linux-5.15.0-70-generic-x86_64-with-glibc2.35

Python dependencies:
      sklearn: 1.2.2
          pip: 22.0.2
   setuptools: 59.6.0
        numpy: 1.21.5
        scipy: 1.8.0
       Cython: None
       pandas: 2.0.0
   matplotlib: 3.7.1
       joblib: 1.2.0
threadpoolctl: 3.1.0

Built with OpenMP: True

threadpoolctl info:
       user_api: openmp
   internal_api: openmp
         prefix: libgomp
       filepath: /home/petr/.local/lib/python3.10/site-packages/scikit_learn.libs/libgomp-a34b3233.so.1.0.0
        version: None
    num_threads: 8

       user_api: blas
   internal_api: openblas
         prefix: libopenblas
       filepath: /usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.20.so
        version: 0.3.20
threading_layer: pthreads
   architecture: Haswell
    num_threads: 8
```

COMMENT:
Hi @jeremiedbb, what about adding a line
```python
self._gamma = 1 / X.shape[1] if self.gamma is None else self.gamma
```
right after validating `X` in the `fit` method, and then in `_get_kernel`, we use `self._gamma` rather than `self.gamma`? This indeed solves this specific issue and passes `pytest sklearn/decomposition/tests`. If you think this is a valid approach and is neat enough, I'm happy to make a PR for this.

Labels: Bug, module:decomposition