Talip7/scikit-learn-issues-multilabel huggingface dataset:
KernelPCA inverse transform behaves unexpectly.

### Describe the bug

Hi!
we wanted to use inverse transform in kernel PCA. There is a parameter gamma, which is defined as 1/num_features if gamma=None. However if gamma is provided as gamma=1/num_features, the resultant inverse transform results is different. Why is that? How the parameter gamma is related to inverse transform?

### Steps/Code to Reproduce

```
from sklearn.datasets import load_iris
from sklearn.decomposition import KernelPCA

#load iris dataset (features only)
data = load_iris().data

#number of features in the dataset
n_features = data.shape[1]

#run KernelPCA with default value of gamma (1/n_features)
pca1 = KernelPCA(n_components = 2, random_state = 7, fit_inverse_transform = True, kernel = 'rbf')
pca1.fit(data)

#set the value of gamma to 1/n_features
pca2 = KernelPCA(n_components = 2, gamma = 1/n_features, random_state = 7, fit_inverse_transform = True, kernel = 'rbf')
pca2.fit(data)

#eigenvalues and eigenvectors are the same
print((pca1.eigenvalues_ == pca2.eigenvalues_).all())
print((pca1.eigenvectors_ == pca2.eigenvectors_).all())

#projections of components are the same
latent1 = pca1.transform(data)
latent2 = pca2.transform(data)
print((latent1 == latent2).all())

#but reconstructed data from these projections are different
reconstructed_data1 = pca1.inverse_transform(latent1)
reconstructed_data2 = pca2.inverse_transform(latent2)
print((reconstructed_data1 == reconstructed_data2).all())
```

### Expected Results

True
True
True
True

### Actual Results

True
True
True
False

### Versions

```shell
System:
    python: 3.10.6 (main, Mar 10 2023, 10:55:28) [GCC 11.3.0]
executable: /usr/bin/python3
   machine: Linux-5.15.0-70-generic-x86_64-with-glibc2.35

Python dependencies:
      sklearn: 1.2.2
          pip: 22.0.2
   setuptools: 59.6.0
        numpy: 1.21.5
        scipy: 1.8.0
       Cython: None
       pandas: 2.0.0
   matplotlib: 3.7.1
       joblib: 1.2.0
threadpoolctl: 3.1.0

Built with OpenMP: True

threadpoolctl info:
       user_api: openmp
   internal_api: openmp
         prefix: libgomp
       filepath: /home/petr/.local/lib/python3.10/site-packages/scikit_learn.libs/libgomp-a34b3233.so.1.0.0
        version: None
    num_threads: 8

       user_api: blas
   internal_api: openblas
         prefix: libopenblas
       filepath: /usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.20.so
        version: 0.3.20
threading_layer: pthreads
   architecture: Haswell
    num_threads: 8
```

COMMENT:
Since there is no response, I've opened a PR as above.

Labels: Bug, module:decomposition
Support `max_bins > 255` in Hist-GBDT estimators and categorical features with high cardinality

As originally sketched in https://github.com/scikit-learn/scikit-learn/pull/26268#issuecomment-1520504489 there might be a way to enable support for arbitrary high values of `max_bins` for both categorical and numerical features. This may not be super critical for numerical features, but this would **enable categorical features of arbitrary cardinality**, which is desirable.

The rough idea is to internally map an input categorical feature into **multiple **binned** features** (probably `num_categories // 255 + 1` features) and to update the `Splitter` and the predictors to treat that group of features as a single feature.

COMMENT:
The number of bins are hardcoded in:
- Histograms as 2d-array of shape (n features, n bins)
- X binned as 2d-array of dtype=uint8
- Bitsets for categorical features as C array[8] of type uint32 (8*32=256)

An alternative to allow for more than 256 bins is therefore
- Histogram as 1d array with positions where a feature starts (and ends). This is to save memory a lot (and maybe cache hits).
- X binned uint8 and a 2nd larger X binned, e.g. uint16, for features that need it, and a structure that bundles both together to a unified API.
- A second extended bitset, similar to the existing one, doubled size and a structure that bundles both together to a unified APi.

Labels: New Feature, module:ensemble
Support `max_bins > 255` in Hist-GBDT estimators and categorical features with high cardinality

As originally sketched in https://github.com/scikit-learn/scikit-learn/pull/26268#issuecomment-1520504489 there might be a way to enable support for arbitrary high values of `max_bins` for both categorical and numerical features. This may not be super critical for numerical features, but this would **enable categorical features of arbitrary cardinality**, which is desirable.

The rough idea is to internally map an input categorical feature into **multiple **binned** features** (probably `num_categories // 255 + 1` features) and to update the `Splitter` and the predictors to treat that group of features as a single feature.

COMMENT:
Updating the underlying data-structure will lead to a different memory footprint, and likely different perf as well. Sounds more risky to me, but if you implement it and benchmark indicate no regression, then why not.

Labels: New Feature, module:ensemble
Support `max_bins > 255` in Hist-GBDT estimators and categorical features with high cardinality

As originally sketched in https://github.com/scikit-learn/scikit-learn/pull/26268#issuecomment-1520504489 there might be a way to enable support for arbitrary high values of `max_bins` for both categorical and numerical features. This may not be super critical for numerical features, but this would **enable categorical features of arbitrary cardinality**, which is desirable.

The rough idea is to internally map an input categorical feature into **multiple **binned** features** (probably `num_categories // 255 + 1` features) and to update the `Splitter` and the predictors to treat that group of features as a single feature.

COMMENT:
> Updating the underlying data-structure will lead to a different memory footprint, and likely different perf as well. Sounds more risky to me, but if you implement it and benchmark indicate no regression, then why not.

In memory, the histograms look the same: a contiguous array of hist_struct. The only difference is that we currently might have quite some unused bins.

Labels: New Feature, module:ensemble
Support `max_bins > 255` in Hist-GBDT estimators and categorical features with high cardinality

As originally sketched in https://github.com/scikit-learn/scikit-learn/pull/26268#issuecomment-1520504489 there might be a way to enable support for arbitrary high values of `max_bins` for both categorical and numerical features. This may not be super critical for numerical features, but this would **enable categorical features of arbitrary cardinality**, which is desirable.

The rough idea is to internally map an input categorical feature into **multiple **binned** features** (probably `num_categories // 255 + 1` features) and to update the `Splitter` and the predictors to treat that group of features as a single feature.

COMMENT:
I'm was more thinking of `X_binned` rather than about the histograms.

> The only difference is that we currently might have quite some unused bins

I assume that using a larger dtype is only going to worsen that problem? (Is that an actual pb in practice?)

Labels: New Feature, module:ensemble
Unhelpful error message when running a classifier on a regression outcome

### Describe the bug

When running a classifier on a regression outcome, we get a really unhelpful error message:


### Steps/Code to Reproduce

```python
In [2]: from sklearn import linear_model, datasets

In [3]: X, y = datasets.make_regression()

In [4]: model = linear_model.LogisticRegression()

In [5]: model.fit(X, y)
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-5-6d10fe8973eb> in <module>
----> 1 model.fit(X, y)

~/.local/lib/python3.10/site-packages/sklearn/linear_model/_logistic.py in fit(self, X, y, sample_weight)
   1202             accept_large_sparse=solver not in ["liblinear", "sag", "saga"],
   1203         )
-> 1204         check_classification_targets(y)
   1205         self.classes_ = np.unique(y)
   1206 

~/.local/lib/python3.10/site-packages/sklearn/utils/multiclass.py in check_classification_targets(y)
    205         "multilabel-sequences",
    206     ]:
--> 207         raise ValueError("Unknown label type: %r" % y_type)
    208 
    209 

ValueError: Unknown label type: 'continuous'
```

### Expected Results

An error message that puts the user on the right track.

### Actual Results

We could add the following sentence to the error message "Maybe you are trying to fit a classifier (discrete classes) on a regression target (continuous values)".

### Versions

```shell
Present on main.

In [6]: import sklearn; sklearn.show_versions()

System:
    python: 3.10.7 (main, Mar 10 2023, 10:47:39) [GCC 12.2.0]
executable: /usr/bin/python3
   machine: Linux-5.19.0-38-generic-x86_64-with-glibc2.36

Python dependencies:
      sklearn: 1.3.dev0
          pip: 22.2
   setuptools: 59.6.0
        numpy: 1.21.5
        scipy: 1.8.1
       Cython: 0.29.34
       pandas: 1.3.5
   matplotlib: 3.5.2
       joblib: 1.1.0
threadpoolctl: 3.1.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: openblas
         prefix: libopenblas
       filepath: /usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.20.so
        version: 0.3.20
threading_layer: pthreads
   architecture: Haswell
    num_threads: 8

       user_api: openmp
   internal_api: openmp
         prefix: libgomp
       filepath: /usr/lib/x86_64-linux-gnu/libgomp.so.1.0.0
        version: None
    num_threads: 8
```
```

COMMENT:
I will take this up and open a PR ASAP.

Labels: Bug, Easy
sklearn.tree.export_text failing when feature_names supplied

folks, I'm not sure why this works for
```py
import sklearn.tree
print(my_feature_names)
['0' '0 trump' '0 trump versus' ... 'zur' 'zur ckhalten' 'zur ckhalten muss']

tree.export_graphviz(clf, out_file=None, max_depth=4, feature_names=my_feature_names)
```
but not for 

```py
import sklearn.tree
print(my_feature_names)
['0' '0 trump' '0 trump versus' ... 'zur' 'zur ckhalten' 'zur ckhalten muss']

tree.export_text(clf, max_depth=4, feature_names=my_feature_names)

Traceback (most recent call last):
  File "./sample-python-projects/machine-learning/HW1_Q2a.py", line 72, in <module>
    print(tree.export_text(clf, max_depth=4, feature_names=my_feature_names))
  File "C:\Users\sam\python\lib\site-packages\sklearn\tree\_export.py", line 1016, in export_text
    if feature_names:
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
```

Can anyone help?

COMMENT:
Could you please post a minimal reproducible? (something we can copy paste in its entirety to produce the issue).

Labels: Documentation, Enhancement
sklearn.tree.export_text failing when feature_names supplied

folks, I'm not sure why this works for
```py
import sklearn.tree
print(my_feature_names)
['0' '0 trump' '0 trump versus' ... 'zur' 'zur ckhalten' 'zur ckhalten muss']

tree.export_graphviz(clf, out_file=None, max_depth=4, feature_names=my_feature_names)
```
but not for 

```py
import sklearn.tree
print(my_feature_names)
['0' '0 trump' '0 trump versus' ... 'zur' 'zur ckhalten' 'zur ckhalten muss']

tree.export_text(clf, max_depth=4, feature_names=my_feature_names)

Traceback (most recent call last):
  File "./sample-python-projects/machine-learning/HW1_Q2a.py", line 72, in <module>
    print(tree.export_text(clf, max_depth=4, feature_names=my_feature_names))
  File "C:\Users\sam\python\lib\site-packages\sklearn\tree\_export.py", line 1016, in export_text
    if feature_names:
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
```

Can anyone help?

COMMENT:
@NickKanellos From the error message, it seems that the feature names you passed in is an array, but as [documented](https://scikit-learn.org/stable/modules/generated/sklearn.tree.export_graphviz.html), `feature_names` must either be a list of strings or `None`.

> feature_nameslist of str, default=None
Names of each of the features. If None, generic names will be used (“x[0]”, “x[1]”, …).

Labels: Documentation, Enhancement
sklearn.tree.export_text failing when feature_names supplied

folks, I'm not sure why this works for
```py
import sklearn.tree
print(my_feature_names)
['0' '0 trump' '0 trump versus' ... 'zur' 'zur ckhalten' 'zur ckhalten muss']

tree.export_graphviz(clf, out_file=None, max_depth=4, feature_names=my_feature_names)
```
but not for 

```py
import sklearn.tree
print(my_feature_names)
['0' '0 trump' '0 trump versus' ... 'zur' 'zur ckhalten' 'zur ckhalten muss']

tree.export_text(clf, max_depth=4, feature_names=my_feature_names)

Traceback (most recent call last):
  File "./sample-python-projects/machine-learning/HW1_Q2a.py", line 72, in <module>
    print(tree.export_text(clf, max_depth=4, feature_names=my_feature_names))
  File "C:\Users\sam\python\lib\site-packages\sklearn\tree\_export.py", line 1016, in export_text
    if feature_names:
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
```

Can anyone help?

COMMENT:
we're not really doing anything which definitely requires feature names to be a list, I'm happy with a fix which would accept an array like, and convert it to a list or an array. @Charlie-XIAO would you have time to submit a PR for your suggestions?

Labels: Documentation, Enhancement
sklearn.tree.export_text failing when feature_names supplied

folks, I'm not sure why this works for
```py
import sklearn.tree
print(my_feature_names)
['0' '0 trump' '0 trump versus' ... 'zur' 'zur ckhalten' 'zur ckhalten muss']

tree.export_graphviz(clf, out_file=None, max_depth=4, feature_names=my_feature_names)
```
but not for 

```py
import sklearn.tree
print(my_feature_names)
['0' '0 trump' '0 trump versus' ... 'zur' 'zur ckhalten' 'zur ckhalten muss']

tree.export_text(clf, max_depth=4, feature_names=my_feature_names)

Traceback (most recent call last):
  File "./sample-python-projects/machine-learning/HW1_Q2a.py", line 72, in <module>
    print(tree.export_text(clf, max_depth=4, feature_names=my_feature_names))
  File "C:\Users\sam\python\lib\site-packages\sklearn\tree\_export.py", line 1016, in export_text
    if feature_names:
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
```

Can anyone help?

COMMENT:
@adrinjalali Sure, I will do it right now.

Labels: Documentation, Enhancement
sklearn.tree.export_text failing when feature_names supplied

folks, I'm not sure why this works for
```py
import sklearn.tree
print(my_feature_names)
['0' '0 trump' '0 trump versus' ... 'zur' 'zur ckhalten' 'zur ckhalten muss']

tree.export_graphviz(clf, out_file=None, max_depth=4, feature_names=my_feature_names)
```
but not for 

```py
import sklearn.tree
print(my_feature_names)
['0' '0 trump' '0 trump versus' ... 'zur' 'zur ckhalten' 'zur ckhalten muss']

tree.export_text(clf, max_depth=4, feature_names=my_feature_names)

Traceback (most recent call last):
  File "./sample-python-projects/machine-learning/HW1_Q2a.py", line 72, in <module>
    print(tree.export_text(clf, max_depth=4, feature_names=my_feature_names))
  File "C:\Users\sam\python\lib\site-packages\sklearn\tree\_export.py", line 1016, in export_text
    if feature_names:
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
```

Can anyone help?

COMMENT:
/take

Labels: Documentation, Enhancement
sklearn.tree.export_text failing when feature_names supplied

folks, I'm not sure why this works for
```py
import sklearn.tree
print(my_feature_names)
['0' '0 trump' '0 trump versus' ... 'zur' 'zur ckhalten' 'zur ckhalten muss']

tree.export_graphviz(clf, out_file=None, max_depth=4, feature_names=my_feature_names)
```
but not for 

```py
import sklearn.tree
print(my_feature_names)
['0' '0 trump' '0 trump versus' ... 'zur' 'zur ckhalten' 'zur ckhalten muss']

tree.export_text(clf, max_depth=4, feature_names=my_feature_names)

Traceback (most recent call last):
  File "./sample-python-projects/machine-learning/HW1_Q2a.py", line 72, in <module>
    print(tree.export_text(clf, max_depth=4, feature_names=my_feature_names))
  File "C:\Users\sam\python\lib\site-packages\sklearn\tree\_export.py", line 1016, in export_text
    if feature_names:
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
```

Can anyone help?

COMMENT:
The error is caused by the fact that the feature_names parameter to the export_text method is an array of strings. When the export_text method is called, it checks to see if the feature_names parameter is set. If it is set, the method will try to convert the array of strings to a boolean value. However, this is not possible because an array of strings cannot be converted to a boolean value.

To fix the error, you can either set the feature_names parameter to None or you can convert the array of strings to a list. Here is an example of how to fix the error:

Python
import sklearn.tree

# Set the feature_names parameter to None.
tree.export_text(clf, max_depth=4, feature_names=None)

# Convert the array of strings to a list.
feature_names = list(my_feature_names)
tree.export_text(clf, max_depth=4, feature_names=feature_names)

Labels: Documentation, Enhancement
sklearn.tree.export_text failing when feature_names supplied

folks, I'm not sure why this works for
```py
import sklearn.tree
print(my_feature_names)
['0' '0 trump' '0 trump versus' ... 'zur' 'zur ckhalten' 'zur ckhalten muss']

tree.export_graphviz(clf, out_file=None, max_depth=4, feature_names=my_feature_names)
```
but not for 

```py
import sklearn.tree
print(my_feature_names)
['0' '0 trump' '0 trump versus' ... 'zur' 'zur ckhalten' 'zur ckhalten muss']

tree.export_text(clf, max_depth=4, feature_names=my_feature_names)

Traceback (most recent call last):
  File "./sample-python-projects/machine-learning/HW1_Q2a.py", line 72, in <module>
    print(tree.export_text(clf, max_depth=4, feature_names=my_feature_names))
  File "C:\Users\sam\python\lib\site-packages\sklearn\tree\_export.py", line 1016, in export_text
    if feature_names:
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
```

Can anyone help?

COMMENT:
@nazish12345 Thanks for your advice, but as you can see above, I've already opened a PR so that both `export_text` and `export_graphviz` will be able to accept array-like.

Labels: Documentation, Enhancement
Add sample_weight parameter to OneHotEncoder.fit(...)

### Describe the workflow you want to enable

I have a dataset with huge number of duplicates - so in order to speed-up learning process I prefer to remove these duplicates and pass `sample_weight=duplicate_counts` to the `fit(..)` method of estimator. I override the `fit(..)` method of estimator and remove duplicates in this wrapper.

But If I use `OneHotEncoder` as pre-processing step, it produces sparse output - so the input to `estimator.fit(..)` is sparse. My dataset is large, so I strongly want to use sparse output of `OneHotEncoder`.

And effective duplicate removal in sparse input is not a trivial task - it must use sorting or hashing.  So I want to perform duplicate removal and `sample_weight` calculation before one hot encoding.

But I use the `min_frequency` parameter of `OneHotEncoder` - and want level frequencies to be calculated with the use of `sample_weight`. 

### Describe your proposed solution

Add `sample_weight` parameter to `OneHotEncoder.fit(..)` function and calculate level frequencies using `sample_weight` parameter.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
Hi @ArseniuML, I was uncertain whether the scikit-learn API would support this particular feature (having `sample_weight` in a pre-process). However, it appears that it does indeed allow for it, as demonstrated by the [fit method in `sklearn.preprocessing.StandardScaler`](https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.StandardScaler.html#sklearn.preprocessing.StandardScaler.fit).

If you don't mind, I would love to submit a pull request implementing this feature once a maintainer removes the "Needs Triage" label. :)

Labels: New Feature
Add sample_weight parameter to OneHotEncoder.fit(...)

### Describe the workflow you want to enable

I have a dataset with huge number of duplicates - so in order to speed-up learning process I prefer to remove these duplicates and pass `sample_weight=duplicate_counts` to the `fit(..)` method of estimator. I override the `fit(..)` method of estimator and remove duplicates in this wrapper.

But If I use `OneHotEncoder` as pre-processing step, it produces sparse output - so the input to `estimator.fit(..)` is sparse. My dataset is large, so I strongly want to use sparse output of `OneHotEncoder`.

And effective duplicate removal in sparse input is not a trivial task - it must use sorting or hashing.  So I want to perform duplicate removal and `sample_weight` calculation before one hot encoding.

But I use the `min_frequency` parameter of `OneHotEncoder` - and want level frequencies to be calculated with the use of `sample_weight`. 

### Describe your proposed solution

Add `sample_weight` parameter to `OneHotEncoder.fit(..)` function and calculate level frequencies using `sample_weight` parameter.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
This sounds like a useful addition. At the very least it would be useful to have a PR that attempts to implement this and what kind of difficulties come up while doing that.

Labels: New Feature
Add sample_weight parameter to OneHotEncoder.fit(...)

### Describe the workflow you want to enable

I have a dataset with huge number of duplicates - so in order to speed-up learning process I prefer to remove these duplicates and pass `sample_weight=duplicate_counts` to the `fit(..)` method of estimator. I override the `fit(..)` method of estimator and remove duplicates in this wrapper.

But If I use `OneHotEncoder` as pre-processing step, it produces sparse output - so the input to `estimator.fit(..)` is sparse. My dataset is large, so I strongly want to use sparse output of `OneHotEncoder`.

And effective duplicate removal in sparse input is not a trivial task - it must use sorting or hashing.  So I want to perform duplicate removal and `sample_weight` calculation before one hot encoding.

But I use the `min_frequency` parameter of `OneHotEncoder` - and want level frequencies to be calculated with the use of `sample_weight`. 

### Describe your proposed solution

Add `sample_weight` parameter to `OneHotEncoder.fit(..)` function and calculate level frequencies using `sample_weight` parameter.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
Hello @betatim! As you seem to be interested, kindly letting you know that I believe my PR is in good shape for a review now. :)

Labels: New Feature