Talip7/scikit-learn-issues-multilabel huggingface dataset:
set_output API do not preserve original dtypes for pandas

### Describe the bug

Following issue #24182,

When using the set_output with expected output to be a pandas' data frame, while converting tougher columns with different dtypes the output does not preserve the original dtype but the "common type" by numpy.

Possible workaround is to create transformer for each column, but it doesn't feel like the right approach.

In collaboration with @BenEfrati

### Steps/Code to Reproduce

```python
import numpy as np
import pandas as pd
from sklearn.impute import SimpleImputer



X = pd.DataFrame({
                  "age": [1, 2, 4],
                  "float": [1.5, 2.5, None],
                  })
X["age"] = X["age"].astype(np.uint8)
X.dtypes

# age        uint8
# float    float64
# dtype: object

simple_imputer = SimpleImputer(strategy='constant', fill_value=1).set_output(transform="pandas")
X_trans_partial = simple_imputer.fit_transform(X)
X_trans_partial.dtypes

# age      float64
# float    float64
# dtype: object
```

### Expected Results

We'd expect the age column to keep being an unsigned 8-bit integer.

### Actual Results

the age column is now a 64-bit floating point.

### Versions

```shell
System:
    python: 3.9.12 (main, Apr  4 2022, 05:22:27) [MSC v.1916 64 bit (AMD64)]
executable: C:\New folder\venv\Scripts\python.exe
   machine: Windows-10-10.0.19044-SP0

Python dependencies:
      sklearn: 1.2.1
          pip: 23.0
   setuptools: 58.1.0
        numpy: 1.24.2
        scipy: 1.10.0
       Cython: None
       pandas: 1.5.3
   matplotlib: None
       joblib: 1.2.0
threadpoolctl: 3.1.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: openblas
         prefix: libopenblas
       filepath: C:\New folder\venv\Lib\site-packages\numpy\.libs\libopenblas64__v0.3.21-gcc_10_3_0.dll
        version: 0.3.21
threading_layer: pthreads
   architecture: Haswell
    num_threads: 12

       user_api: openmp
   internal_api: openmp
         prefix: vcomp
       filepath: C:\New folder\venv\Lib\site-packages\sklearn\.libs\vcomp140.dll
        version: None
    num_threads: 12

       user_api: blas
   internal_api: openblas
         prefix: libopenblas
       filepath: C:\New folder\venv\Lib\site-packages\scipy.libs\libopenblas-802f9ed1179cb9c9b03d67ff79f48187.dll
        version: 0.3.18
threading_layer: pthreads
   architecture: Haswell
    num_threads: 12
```

COMMENT:
@thomasjpfan 

which version of Sklearn are you using ? 

<img width="1184" alt="Screenshot 2023-02-13 at 11 37 06" src="https://user-images.githubusercontent.com/42499767/218435673-927fb3a6-6d27-4bfb-b75a-953b45c4b0d6.png">

```python
simple_imputer = SimpleImputer().set_output(transform="pandas")
```
<img width="1045" alt="Screenshot 2023-02-13 at 11 37 23" src="https://user-images.githubusercontent.com/42499767/218435722-7fd5d891-d2b3-4fb5-9b12-68ff51e34dcd.png">

Labels: Bug, module:impute, Needs Decision - Include Feature
set_output API do not preserve original dtypes for pandas

### Describe the bug

Following issue #24182,

When using the set_output with expected output to be a pandas' data frame, while converting tougher columns with different dtypes the output does not preserve the original dtype but the "common type" by numpy.

Possible workaround is to create transformer for each column, but it doesn't feel like the right approach.

In collaboration with @BenEfrati

### Steps/Code to Reproduce

```python
import numpy as np
import pandas as pd
from sklearn.impute import SimpleImputer



X = pd.DataFrame({
                  "age": [1, 2, 4],
                  "float": [1.5, 2.5, None],
                  })
X["age"] = X["age"].astype(np.uint8)
X.dtypes

# age        uint8
# float    float64
# dtype: object

simple_imputer = SimpleImputer(strategy='constant', fill_value=1).set_output(transform="pandas")
X_trans_partial = simple_imputer.fit_transform(X)
X_trans_partial.dtypes

# age      float64
# float    float64
# dtype: object
```

### Expected Results

We'd expect the age column to keep being an unsigned 8-bit integer.

### Actual Results

the age column is now a 64-bit floating point.

### Versions

```shell
System:
    python: 3.9.12 (main, Apr  4 2022, 05:22:27) [MSC v.1916 64 bit (AMD64)]
executable: C:\New folder\venv\Scripts\python.exe
   machine: Windows-10-10.0.19044-SP0

Python dependencies:
      sklearn: 1.2.1
          pip: 23.0
   setuptools: 58.1.0
        numpy: 1.24.2
        scipy: 1.10.0
       Cython: None
       pandas: 1.5.3
   matplotlib: None
       joblib: 1.2.0
threadpoolctl: 3.1.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: openblas
         prefix: libopenblas
       filepath: C:\New folder\venv\Lib\site-packages\numpy\.libs\libopenblas64__v0.3.21-gcc_10_3_0.dll
        version: 0.3.21
threading_layer: pthreads
   architecture: Haswell
    num_threads: 12

       user_api: openmp
   internal_api: openmp
         prefix: vcomp
       filepath: C:\New folder\venv\Lib\site-packages\sklearn\.libs\vcomp140.dll
        version: None
    num_threads: 12

       user_api: blas
   internal_api: openblas
         prefix: libopenblas
       filepath: C:\New folder\venv\Lib\site-packages\scipy.libs\libopenblas-802f9ed1179cb9c9b03d67ff79f48187.dll
        version: 0.3.18
threading_layer: pthreads
   architecture: Haswell
    num_threads: 12
```

COMMENT:
@davydw you need sklearn: 1.2.1
see python dependencies in the first comment.

Labels: Bug, module:impute, Needs Decision - Include Feature
set_output API do not preserve original dtypes for pandas

### Describe the bug

Following issue #24182,

When using the set_output with expected output to be a pandas' data frame, while converting tougher columns with different dtypes the output does not preserve the original dtype but the "common type" by numpy.

Possible workaround is to create transformer for each column, but it doesn't feel like the right approach.

In collaboration with @BenEfrati

### Steps/Code to Reproduce

```python
import numpy as np
import pandas as pd
from sklearn.impute import SimpleImputer



X = pd.DataFrame({
                  "age": [1, 2, 4],
                  "float": [1.5, 2.5, None],
                  })
X["age"] = X["age"].astype(np.uint8)
X.dtypes

# age        uint8
# float    float64
# dtype: object

simple_imputer = SimpleImputer(strategy='constant', fill_value=1).set_output(transform="pandas")
X_trans_partial = simple_imputer.fit_transform(X)
X_trans_partial.dtypes

# age      float64
# float    float64
# dtype: object
```

### Expected Results

We'd expect the age column to keep being an unsigned 8-bit integer.

### Actual Results

the age column is now a 64-bit floating point.

### Versions

```shell
System:
    python: 3.9.12 (main, Apr  4 2022, 05:22:27) [MSC v.1916 64 bit (AMD64)]
executable: C:\New folder\venv\Scripts\python.exe
   machine: Windows-10-10.0.19044-SP0

Python dependencies:
      sklearn: 1.2.1
          pip: 23.0
   setuptools: 58.1.0
        numpy: 1.24.2
        scipy: 1.10.0
       Cython: None
       pandas: 1.5.3
   matplotlib: None
       joblib: 1.2.0
threadpoolctl: 3.1.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: openblas
         prefix: libopenblas
       filepath: C:\New folder\venv\Lib\site-packages\numpy\.libs\libopenblas64__v0.3.21-gcc_10_3_0.dll
        version: 0.3.21
threading_layer: pthreads
   architecture: Haswell
    num_threads: 12

       user_api: openmp
   internal_api: openmp
         prefix: vcomp
       filepath: C:\New folder\venv\Lib\site-packages\sklearn\.libs\vcomp140.dll
        version: None
    num_threads: 12

       user_api: blas
   internal_api: openblas
         prefix: libopenblas
       filepath: C:\New folder\venv\Lib\site-packages\scipy.libs\libopenblas-802f9ed1179cb9c9b03d67ff79f48187.dll
        version: 0.3.18
threading_layer: pthreads
   architecture: Haswell
    num_threads: 12
```

COMMENT:
<img width="1125" alt="Screenshot 2023-02-14 at 10 30 39" src="https://user-images.githubusercontent.com/42499767/218695119-e4ac534f-c377-4546-94cc-c7b85e976a08.png">

Thanks @idanmoradarthas 

* I used `set_output(trasnform="pandas")` on my `SimpleImputer` to preserve the columns'names. Hence I was able to apply a `ColumnTransformer` afterwards
* I also used `set_output(trasnform="pandas")` after the `ColumnTransformer` and I no longer need to do:

```python
X_preprocessed = pd.DataFrame(X, columns = preprocessor.get_feature_names_out()) 
```

like I used to do.

Super convenient 👏

Labels: Bug, module:impute, Needs Decision - Include Feature
Implement beta calibration

### Describe the workflow you want to enable

It would be nice to implement beta calibration as an additional option in CalibratedClassifierCV.

### Describe your proposed solution

Use the implementation provided in https://github.com/betacal/python (MIT license).

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

Beta calibration seems like a generally superior alternative to logistic / Platt calibration. See here for further details, paper references, etc.: https://betacal.github.io/ It was proposed in 2017 in two papers by Meelis Kull, Telmo de Menezes e Silva Filho and Peter Flach.

COMMENT:
While the [inclusion criteria](https://scikit-learn.org/stable/faq.html#what-are-the-inclusion-criteria-for-new-algorithms) are, strictly speaking, not (yet) fulfilled, I think that this approach adds value. There are, however, 2 critical questions

- Where to stop: There are infinitely many distributions on [0, 1] and all of them could be used for calibration.
- https://github.com/betacal/python already provides the proposed functionality. If we decide to implement it, we should at least contact them.

Labels: New Feature, module:calibration, Needs Decision - Include Feature
Implement beta calibration

### Describe the workflow you want to enable

It would be nice to implement beta calibration as an additional option in CalibratedClassifierCV.

### Describe your proposed solution

Use the implementation provided in https://github.com/betacal/python (MIT license).

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

Beta calibration seems like a generally superior alternative to logistic / Platt calibration. See here for further details, paper references, etc.: https://betacal.github.io/ It was proposed in 2017 in two papers by Meelis Kull, Telmo de Menezes e Silva Filho and Peter Flach.

COMMENT:
Ah, sorry - I wasn't aware of the inclusion criteria; those seem sensible. As you probably also saw, the two beta calibration papers together are currently slightly below 200 citations.

Regarding 1 (where to stop): don't the inclusion criteria precisely say where to stop? I.e., calibration approaches should be i) relatively widely used/cited, and ii) provide clear benefits over existing approaches? Beta calibration seems to check both marks, since it (as I understand it) subsumes logistic regression and is clearly superior in many cases, including the important case of an already well-calibrated model. They have quite extensive experiments in their papers showing the superiority.

The only other calibration methods that might check these boxes and that I can immediately think of would be [BBQ](https://people.cs.pitt.edu/~milos/research/AAAI_Calibration.pdf) and maybe [this more recent method](https://proceedings.neurips.cc/paper/2019/file/f8c0c968632845cd133308b1a494967f-Paper.pdf), so I think it's not like you'd be opening the floodgates to dozens of calibration methods. (I might be completely wrong about that, though...)

Regarding 2: Sure, that makes sense. Should I contact them and ask whether they would like for it to be included here?

One other thing I just noticed: beta calibration is only specified for the binary classification case. (They do have [an equally well-cited extension for the multivariate case](https://arxiv.org/pdf/1910.12656.pdf), and [code is also available for that](https://github.com/dirichletcal/dirichlet_python), but that would probably be a separate discussion.) Both current methods in CalibratedClassifierCV apparently also support the multivariate case. (Cool!) Would it be a problem to add a method that only works in the binary case?

In any case it's your call, of course, and I also understand if you consider this too experimental / niche.

To give some background / motivation for this, I was basically just looking for a simple but reasonable calibration method in the low-data regime where isotonic regression is not applicable. Beta calibration seemed an attractive option for this, while LR has significant limitations.

Labels: New Feature, module:calibration, Needs Decision - Include Feature
Implement beta calibration

### Describe the workflow you want to enable

It would be nice to implement beta calibration as an additional option in CalibratedClassifierCV.

### Describe your proposed solution

Use the implementation provided in https://github.com/betacal/python (MIT license).

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

Beta calibration seems like a generally superior alternative to logistic / Platt calibration. See here for further details, paper references, etc.: https://betacal.github.io/ It was proposed in 2017 in two papers by Meelis Kull, Telmo de Menezes e Silva Filho and Peter Flach.

COMMENT:
@scikit-learn/core-devs This meeds a decision.

Citation-wise, it is a bit on the boundary. I‘m personally inclined to include it as it is statistically sound and „the next best/simplest“ method after the logistic/Platt calibration.

> Sigmoid should be applied only on classification models with infinite support

@perellonieto Could you explain what you mean? And how it differs from beta calibration?

As you said, multiclass calibration is best discussed in a new separate issue.

Labels: New Feature, module:calibration, Needs Decision - Include Feature
Implement beta calibration

### Describe the workflow you want to enable

It would be nice to implement beta calibration as an additional option in CalibratedClassifierCV.

### Describe your proposed solution

Use the implementation provided in https://github.com/betacal/python (MIT license).

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

Beta calibration seems like a generally superior alternative to logistic / Platt calibration. See here for further details, paper references, etc.: https://betacal.github.io/ It was proposed in 2017 in two papers by Meelis Kull, Telmo de Menezes e Silva Filho and Peter Flach.

COMMENT:
Is a sklibration package probabl?

Labels: New Feature, module:calibration, Needs Decision - Include Feature
Implement beta calibration

### Describe the workflow you want to enable

It would be nice to implement beta calibration as an additional option in CalibratedClassifierCV.

### Describe your proposed solution

Use the implementation provided in https://github.com/betacal/python (MIT license).

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

Beta calibration seems like a generally superior alternative to logistic / Platt calibration. See here for further details, paper references, etc.: https://betacal.github.io/ It was proposed in 2017 in two papers by Meelis Kull, Telmo de Menezes e Silva Filho and Peter Flach.

COMMENT:
My feeling: as someone having studied how calibration is used and impacts applications, I fear that we might be getting into diminishing returns.

Labels: New Feature, module:calibration, Needs Decision - Include Feature
Implement beta calibration

### Describe the workflow you want to enable

It would be nice to implement beta calibration as an additional option in CalibratedClassifierCV.

### Describe your proposed solution

Use the implementation provided in https://github.com/betacal/python (MIT license).

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

Beta calibration seems like a generally superior alternative to logistic / Platt calibration. See here for further details, paper references, etc.: https://betacal.github.io/ It was proposed in 2017 in two papers by Meelis Kull, Telmo de Menezes e Silva Filho and Peter Flach.

COMMENT:
@GaelVaroquaux Could you expand a bit? Do you argue against inclusion?

Labels: New Feature, module:calibration, Needs Decision - Include Feature
Implement beta calibration

### Describe the workflow you want to enable

It would be nice to implement beta calibration as an additional option in CalibratedClassifierCV.

### Describe your proposed solution

Use the implementation provided in https://github.com/betacal/python (MIT license).

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

Beta calibration seems like a generally superior alternative to logistic / Platt calibration. See here for further details, paper references, etc.: https://betacal.github.io/ It was proposed in 2017 in two papers by Meelis Kull, Telmo de Menezes e Silva Filho and Peter Flach.

COMMENT:
> @GaelVaroquaux Could you expand a bit?

In most cases, what matters for an application is to define well the utility associated with predictions and potential decisions. Knowing this utility enables making educated choices on the pipeline. Calibration is only an intermediate step here and a proxy on the final benefits (an incomplete one). The main calibration methods often work well enough here and to go further what matters is to understand downstream logic.

My feelings are echoed in the analysis of business logics and data science at booking.com (published here https://dl.acm.org/doi/10.1145/3292500.3330744 ), where an interesting tidebit is the mention that performance on internal benchmark correlated weakly with business value.

> Do you argue against inclusion?

I do. Each line of code that we add to scikit-learn leads to a maintenance cost and makes the library harder to move forward (eg supporting more compute backends). I believe that beta calibration will provide limited value to most users and we should focus on other things.

In addition, I worry about the problem of crowding in the library: the forest hiding the tree. Having a small number of well documented and well explained choices probably helps users asking the right questions.

Labels: New Feature, module:calibration, Needs Decision - Include Feature
documentation of k-means param n_init isn't worded nicely for people unfamiliar with the implementation

### Describe the issue linked to the documentation

Currently the doc says:

> When n_init='auto', the number of runs will be 10 if using init='random', and 1 if using init='kmeans++'.

in https://scikit-learn.org/stable/modules/generated/sklearn.cluster.KMeans.html,

and

>  When `n_init='auto'`, the number of runs will be 10 if using
> `init='random'`, and 1 if using `init='kmeans++'`.

in https://github.com/scikit-learn/scikit-learn/blob/f7e5f412ddba4f30c871749515bbc0393378aa15/sklearn/cluster/_kmeans.py.

Careful readers will make sense of it, but I'm sure we can do better for hasty readers / people unfamiliar with the implementation (because n_init and init look almost identical).

### Suggest a potential alternative/fix

Suggestion:

When `n_init='auto'`, the number of runs depends on the value of `init`:
10 if using`init='random'`, 1 if using `init='kmeans++'`

COMMENT:
Can I fix this issue

Labels: Documentation
documentation of k-means param n_init isn't worded nicely for people unfamiliar with the implementation

### Describe the issue linked to the documentation

Currently the doc says:

> When n_init='auto', the number of runs will be 10 if using init='random', and 1 if using init='kmeans++'.

in https://scikit-learn.org/stable/modules/generated/sklearn.cluster.KMeans.html,

and

>  When `n_init='auto'`, the number of runs will be 10 if using
> `init='random'`, and 1 if using `init='kmeans++'`.

in https://github.com/scikit-learn/scikit-learn/blob/f7e5f412ddba4f30c871749515bbc0393378aa15/sklearn/cluster/_kmeans.py.

Careful readers will make sense of it, but I'm sure we can do better for hasty readers / people unfamiliar with the implementation (because n_init and init look almost identical).

### Suggest a potential alternative/fix

Suggestion:

When `n_init='auto'`, the number of runs depends on the value of `init`:
10 if using`init='random'`, 1 if using `init='kmeans++'`

COMMENT:
Sounds like a good improvement. @sanjail3 sure thing. Please make a PR with the change.

Labels: Documentation
documentation of k-means param n_init isn't worded nicely for people unfamiliar with the implementation

### Describe the issue linked to the documentation

Currently the doc says:

> When n_init='auto', the number of runs will be 10 if using init='random', and 1 if using init='kmeans++'.

in https://scikit-learn.org/stable/modules/generated/sklearn.cluster.KMeans.html,

and

>  When `n_init='auto'`, the number of runs will be 10 if using
> `init='random'`, and 1 if using `init='kmeans++'`.

in https://github.com/scikit-learn/scikit-learn/blob/f7e5f412ddba4f30c871749515bbc0393378aa15/sklearn/cluster/_kmeans.py.

Careful readers will make sense of it, but I'm sure we can do better for hasty readers / people unfamiliar with the implementation (because n_init and init look almost identical).

### Suggest a potential alternative/fix

Suggestion:

When `n_init='auto'`, the number of runs depends on the value of `init`:
10 if using`init='random'`, 1 if using `init='kmeans++'`

COMMENT:
I changed the documentation can you check on to this

Labels: Documentation
quantum kernel with scikit -learn

### Describe the workflow you want to enable

I have designed a quantum kernel function with Pennylane quantum simulator. When i want to use Gaussian process for classification in combination with the quantum kernel i encountered this problem: 

```py
AttributeError: 'function' object has no attribute 'requires_vector_input'
```

this is my kernel function:

``` py
import pennylane as qml


n_qubits = 3

dev_kernel = qml.device("default.qubit", wires=n_qubits)


projector = np.zeros((2**n_qubits, 2**n_qubits))
projector[0, 0] = 1

@qml.qnode(dev_kernel)
def kernel(x1, x2):
    """The quantum kernel."""
    qml.templates.MottonenStatePreparation(x1, wires=range(n_qubits))
    qml.adjoint(qml.templates.MottonenStatePreparation(x2, wires=range(n_qubits)))
    return qml.expval(qml.Hermitian(projector, wires=range(n_qubits)))



def kernel_matrix(A, B):
    """Compute the matrix whose entries are the kernel
       evaluated on pairwise data from sets A and B."""
    return np.array([[kernel(a, b) for b in B] for a in A])
```

But i can use this kernel function with SVM class of scikit-learn without doing any changes with this command:

``` py
svm = SVC(kernel=kernel_matrix, class_weight='balanced').fit(X_train, y_train)
```

**but with this** 

```py
gp = GaussianProcessClassifier(kernel=kernel_matrix).fit(X_train, y_train)

AttributeError: 'function' object has no attribute 'requires_vector_input'
```

### Describe your proposed solution

i think it would be great, if we could have quantum kernel functions for gaussian process like for svm.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
projector = np.zeros((pow(2,n_qubits), pow(2,n_qubits)))

Labels: New Feature, Needs Triage