Talip7/scikit-learn-issues-multilabel huggingface dataset:
KMeans initialization does not use sample weights

### Describe the bug

Clustering by KMeans does not weight the input data.

### Steps/Code to Reproduce

```py
import numpy as np
from sklearn.cluster import KMeans
x = np.array([1, 1, 5, 5, 100, 100])
w = 10**np.array([8.,8,8,8,-8,-8]) # large weights for 1 and 5, small weights for 100
x=x.reshape(-1,1)# reshape to a 2-dimensional array requested for KMeans
centers_with_weight = KMeans(n_clusters=2, random_state=0,n_init=10).fit(x,sample_weight=w).cluster_centers_
centers_no_weight = KMeans(n_clusters=2, random_state=0,n_init=10).fit(x).cluster_centers_
```

### Expected Results

centers_with_weight=[[1.],[5.]]
centers_no_weight=[[100.],[3.]]

### Actual Results

centers_with_weight=[[100.],[3.]]
centers_no_weight=[[100.],[3.]]

### Versions

```shell
System:
    python: 3.10.4 (tags/v3.10.4:9d38120, Mar 23 2022, 23:13:41) [MSC v.1929 64 bit (AMD64)]
executable: E:\WPy64-31040\python-3.10.4.amd64\python.exe
   machine: Windows-10-10.0.19045-SP0

Python dependencies:
      sklearn: 1.2.1
          pip: 22.3.1
   setuptools: 62.1.0
        numpy: 1.23.3
        scipy: 1.8.1
       Cython: 0.29.28
       pandas: 1.4.2
   matplotlib: 3.5.1
       joblib: 1.2.0
threadpoolctl: 3.1.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: openblas
         prefix: libopenblas
       filepath: E:\WPy64-31040\python-3.10.4.amd64\Lib\site-packages\numpy\.libs\libopenblas.FB5AE2TYXYH2IJRDKGDGQ3XBKLKTF43H.gfortran-win_amd64.dll
        version: 0.3.20
threading_layer: pthreads
   architecture: Haswell
    num_threads: 12

       user_api: blas
   internal_api: openblas
         prefix: libopenblas
       filepath: E:\WPy64-31040\python-3.10.4.amd64\Lib\site-packages\scipy\.libs\libopenblas.XWYDX2IKJW2NMTWSFYNGFUWKQU3LYTCZ.gfortran-win_amd64.dll
        version: 0.3.17
threading_layer: pthreads
   architecture: Haswell
    num_threads: 12

       user_api: openmp
   internal_api: openmp
         prefix: vcomp
       filepath: E:\WPy64-31040\python-3.10.4.amd64\Lib\site-packages\sklearn\.libs\vcomp140.dll
        version: None
    num_threads: 12
```

COMMENT:
There is a fix proposed, you can follow its progress in #25752. Once the pull-request is merged, the fix will be available in the next public release, or by installing the development version.

Labels: Bug, module:cluster
⚠️ CI failed on Wheel builder ⚠️

**CI failed on [Wheel builder](https://github.com/scikit-learn/scikit-learn/actions/runs/4021364073)** (Jan 27, 2023)

COMMENT:
## CI is no longer failing! ✅

[Successful run](https://github.com/scikit-learn/scikit-learn/actions/runs/4021364073) on Jan 27, 2023

Labels: Needs Triage
Enable feature selectors to pass pandas DataFrame to estimator

### Describe the workflow you want to enable

When running SequentialFeatureSelector (or, presumably, other feature selection methods) with a pandas DataFrame input, the reduced-feature input is passed to the estimator as a numpy array. This seems inconsistent with the other transformers that successfully maintain the pandas indices.

My actual use case is the following: Select the best k features that complement a set of fixed features. I was going to implement it like this: (happy to take suggestions for alternatives).

```python
import pandas as pd
import numpy as np
import sklearn
from sklearn.feature_selection import SequentialFeatureSelector
from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import FunctionTransformer
from sklearn.linear_model import LinearRegression
from sklearn.pipeline import Pipeline

X=pd.DataFrame(np.random.rand(10,3))
y=pd.Series(np.random.rand(10))

fixed_features=pd.DataFrame(np.random.rand(X.shape[0],2))
appender= FunctionTransformer(lambda X:pd.concat([X,fixed_features.reindex(X.index)],axis=1))
X.shape,appender.fit_transform(X).shape
```

    ((10, 3), (10, 5))


```python
LR_with_fixed=LR=Pipeline(steps=[
    ('append',appender),
    ('LR',LinearRegression())
])
SFS_with_fixed = SequentialFeatureSelector(
    LR_with_fixed,n_features_to_select=1, direction="forward"
)
_=SFS_with_fixed.fit(X,y)
```

    ValueError: 
    All the 5 fits failed.
    .....
      File "/tmp/ipykernel_275/897617596.py", line 2, in <lambda>
        appender= FunctionTransformer(lambda X:pd.concat([X,fixed_features.reindex(X.index)],axis=1))
    AttributeError: 'numpy.ndarray' object has no attribute 'index'



### Describe your proposed solution

I believe that it should be possible to pass the reduced matrix to the estimator as a pandas DataFrame, and this would be consistent with the way other Transformers work currently.

### Describe alternatives you've considered, if relevant

An alternative that would suffice for my particular use case is to directly pass a set of fixed features as a parameter to the feature selector.

### Additional context

_No response_

COMMENT:
Knowing the job of the `SequentialFeatureSelector`, I assume that we could delegate the validation to the inner model. It would avoid the issue.

Labels: Bug
Enable feature selectors to pass pandas DataFrame to estimator

### Describe the workflow you want to enable

When running SequentialFeatureSelector (or, presumably, other feature selection methods) with a pandas DataFrame input, the reduced-feature input is passed to the estimator as a numpy array. This seems inconsistent with the other transformers that successfully maintain the pandas indices.

My actual use case is the following: Select the best k features that complement a set of fixed features. I was going to implement it like this: (happy to take suggestions for alternatives).

```python
import pandas as pd
import numpy as np
import sklearn
from sklearn.feature_selection import SequentialFeatureSelector
from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import FunctionTransformer
from sklearn.linear_model import LinearRegression
from sklearn.pipeline import Pipeline

X=pd.DataFrame(np.random.rand(10,3))
y=pd.Series(np.random.rand(10))

fixed_features=pd.DataFrame(np.random.rand(X.shape[0],2))
appender= FunctionTransformer(lambda X:pd.concat([X,fixed_features.reindex(X.index)],axis=1))
X.shape,appender.fit_transform(X).shape
```

    ((10, 3), (10, 5))


```python
LR_with_fixed=LR=Pipeline(steps=[
    ('append',appender),
    ('LR',LinearRegression())
])
SFS_with_fixed = SequentialFeatureSelector(
    LR_with_fixed,n_features_to_select=1, direction="forward"
)
_=SFS_with_fixed.fit(X,y)
```

    ValueError: 
    All the 5 fits failed.
    .....
      File "/tmp/ipykernel_275/897617596.py", line 2, in <lambda>
        appender= FunctionTransformer(lambda X:pd.concat([X,fixed_features.reindex(X.index)],axis=1))
    AttributeError: 'numpy.ndarray' object has no attribute 'index'



### Describe your proposed solution

I believe that it should be possible to pass the reduced matrix to the estimator as a pandas DataFrame, and this would be consistent with the way other Transformers work currently.

### Describe alternatives you've considered, if relevant

An alternative that would suffice for my particular use case is to directly pass a set of fixed features as a parameter to the feature selector.

### Additional context

_No response_

COMMENT:
> Maybe I'm not following, but I was trying to say that SequentialFeatureSelector doesn't pass all the input to the estimator.

`SequentialFeatureSelector` will call `_validate_data` which will convert the dataframe into a NumPy array. Then, it will make the algorithmic part and thus pass a piece of the NumPy array to the underlying estimator (i.e. your LR pipeline).

Labels: Bug
Enable feature selectors to pass pandas DataFrame to estimator

### Describe the workflow you want to enable

When running SequentialFeatureSelector (or, presumably, other feature selection methods) with a pandas DataFrame input, the reduced-feature input is passed to the estimator as a numpy array. This seems inconsistent with the other transformers that successfully maintain the pandas indices.

My actual use case is the following: Select the best k features that complement a set of fixed features. I was going to implement it like this: (happy to take suggestions for alternatives).

```python
import pandas as pd
import numpy as np
import sklearn
from sklearn.feature_selection import SequentialFeatureSelector
from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import FunctionTransformer
from sklearn.linear_model import LinearRegression
from sklearn.pipeline import Pipeline

X=pd.DataFrame(np.random.rand(10,3))
y=pd.Series(np.random.rand(10))

fixed_features=pd.DataFrame(np.random.rand(X.shape[0],2))
appender= FunctionTransformer(lambda X:pd.concat([X,fixed_features.reindex(X.index)],axis=1))
X.shape,appender.fit_transform(X).shape
```

    ((10, 3), (10, 5))


```python
LR_with_fixed=LR=Pipeline(steps=[
    ('append',appender),
    ('LR',LinearRegression())
])
SFS_with_fixed = SequentialFeatureSelector(
    LR_with_fixed,n_features_to_select=1, direction="forward"
)
_=SFS_with_fixed.fit(X,y)
```

    ValueError: 
    All the 5 fits failed.
    .....
      File "/tmp/ipykernel_275/897617596.py", line 2, in <lambda>
        appender= FunctionTransformer(lambda X:pd.concat([X,fixed_features.reindex(X.index)],axis=1))
    AttributeError: 'numpy.ndarray' object has no attribute 'index'



### Describe your proposed solution

I believe that it should be possible to pass the reduced matrix to the estimator as a pandas DataFrame, and this would be consistent with the way other Transformers work currently.

### Describe alternatives you've considered, if relevant

An alternative that would suffice for my particular use case is to directly pass a set of fixed features as a parameter to the feature selector.

### Additional context

_No response_

COMMENT:
can I work on this issue?

Labels: Bug
Enable feature selectors to pass pandas DataFrame to estimator

### Describe the workflow you want to enable

When running SequentialFeatureSelector (or, presumably, other feature selection methods) with a pandas DataFrame input, the reduced-feature input is passed to the estimator as a numpy array. This seems inconsistent with the other transformers that successfully maintain the pandas indices.

My actual use case is the following: Select the best k features that complement a set of fixed features. I was going to implement it like this: (happy to take suggestions for alternatives).

```python
import pandas as pd
import numpy as np
import sklearn
from sklearn.feature_selection import SequentialFeatureSelector
from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import FunctionTransformer
from sklearn.linear_model import LinearRegression
from sklearn.pipeline import Pipeline

X=pd.DataFrame(np.random.rand(10,3))
y=pd.Series(np.random.rand(10))

fixed_features=pd.DataFrame(np.random.rand(X.shape[0],2))
appender= FunctionTransformer(lambda X:pd.concat([X,fixed_features.reindex(X.index)],axis=1))
X.shape,appender.fit_transform(X).shape
```

    ((10, 3), (10, 5))


```python
LR_with_fixed=LR=Pipeline(steps=[
    ('append',appender),
    ('LR',LinearRegression())
])
SFS_with_fixed = SequentialFeatureSelector(
    LR_with_fixed,n_features_to_select=1, direction="forward"
)
_=SFS_with_fixed.fit(X,y)
```

    ValueError: 
    All the 5 fits failed.
    .....
      File "/tmp/ipykernel_275/897617596.py", line 2, in <lambda>
        appender= FunctionTransformer(lambda X:pd.concat([X,fixed_features.reindex(X.index)],axis=1))
    AttributeError: 'numpy.ndarray' object has no attribute 'index'



### Describe your proposed solution

I believe that it should be possible to pass the reduced matrix to the estimator as a pandas DataFrame, and this would be consistent with the way other Transformers work currently.

### Describe alternatives you've considered, if relevant

An alternative that would suffice for my particular use case is to directly pass a set of fixed features as a parameter to the feature selector.

### Additional context

_No response_

COMMENT:
@glemaitre I Delegated the validation to the inner model, and I got this results: 
```Python 
SFS = SequentialFeatureSelector(
    LR,n_features_to_select=1, direction="forward",cv=2
)
SFS.fit(X,y)
```
```
Input is <class 'pandas.core.frame.DataFrame'> with shape (5, 1)
Input is <class 'pandas.core.frame.DataFrame'> with shape (5, 1)
Input is <class 'pandas.core.frame.DataFrame'> with shape (5, 1)
Input is <class 'pandas.core.frame.DataFrame'> with shape (5, 1)
Input is <class 'pandas.core.frame.DataFrame'> with shape (5, 1)
Input is <class 'pandas.core.frame.DataFrame'> with shape (5, 1)
```

Labels: Bug
Enable feature selectors to pass pandas DataFrame to estimator

### Describe the workflow you want to enable

When running SequentialFeatureSelector (or, presumably, other feature selection methods) with a pandas DataFrame input, the reduced-feature input is passed to the estimator as a numpy array. This seems inconsistent with the other transformers that successfully maintain the pandas indices.

My actual use case is the following: Select the best k features that complement a set of fixed features. I was going to implement it like this: (happy to take suggestions for alternatives).

```python
import pandas as pd
import numpy as np
import sklearn
from sklearn.feature_selection import SequentialFeatureSelector
from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import FunctionTransformer
from sklearn.linear_model import LinearRegression
from sklearn.pipeline import Pipeline

X=pd.DataFrame(np.random.rand(10,3))
y=pd.Series(np.random.rand(10))

fixed_features=pd.DataFrame(np.random.rand(X.shape[0],2))
appender= FunctionTransformer(lambda X:pd.concat([X,fixed_features.reindex(X.index)],axis=1))
X.shape,appender.fit_transform(X).shape
```

    ((10, 3), (10, 5))


```python
LR_with_fixed=LR=Pipeline(steps=[
    ('append',appender),
    ('LR',LinearRegression())
])
SFS_with_fixed = SequentialFeatureSelector(
    LR_with_fixed,n_features_to_select=1, direction="forward"
)
_=SFS_with_fixed.fit(X,y)
```

    ValueError: 
    All the 5 fits failed.
    .....
      File "/tmp/ipykernel_275/897617596.py", line 2, in <lambda>
        appender= FunctionTransformer(lambda X:pd.concat([X,fixed_features.reindex(X.index)],axis=1))
    AttributeError: 'numpy.ndarray' object has no attribute 'index'



### Describe your proposed solution

I believe that it should be possible to pass the reduced matrix to the estimator as a pandas DataFrame, and this would be consistent with the way other Transformers work currently.

### Describe alternatives you've considered, if relevant

An alternative that would suffice for my particular use case is to directly pass a set of fixed features as a parameter to the feature selector.

### Additional context

_No response_

COMMENT:
> In the particular case of this issue, I could see the argument that what is happening is "the right thing". It follows the "rule" of scikit-learn estimators will convert input to a numpy array (at the first point of contact). I can also see the argument that it is annoying that it works this way.

In meta-estimator, I can think of 2 groups: some will require "advanced" algebra operations and then we need to go NumPy, and others will delegate the "advanced" part to sub-estimators and I think that they can delay the conversion.

For instance, this should avoid the [following proposal](https://github.com/scikit-learn/scikit-learn/pull/25102) to cast dtype because we should handle it with the native container type.

Labels: Bug
Enable feature selectors to pass pandas DataFrame to estimator

### Describe the workflow you want to enable

When running SequentialFeatureSelector (or, presumably, other feature selection methods) with a pandas DataFrame input, the reduced-feature input is passed to the estimator as a numpy array. This seems inconsistent with the other transformers that successfully maintain the pandas indices.

My actual use case is the following: Select the best k features that complement a set of fixed features. I was going to implement it like this: (happy to take suggestions for alternatives).

```python
import pandas as pd
import numpy as np
import sklearn
from sklearn.feature_selection import SequentialFeatureSelector
from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import FunctionTransformer
from sklearn.linear_model import LinearRegression
from sklearn.pipeline import Pipeline

X=pd.DataFrame(np.random.rand(10,3))
y=pd.Series(np.random.rand(10))

fixed_features=pd.DataFrame(np.random.rand(X.shape[0],2))
appender= FunctionTransformer(lambda X:pd.concat([X,fixed_features.reindex(X.index)],axis=1))
X.shape,appender.fit_transform(X).shape
```

    ((10, 3), (10, 5))


```python
LR_with_fixed=LR=Pipeline(steps=[
    ('append',appender),
    ('LR',LinearRegression())
])
SFS_with_fixed = SequentialFeatureSelector(
    LR_with_fixed,n_features_to_select=1, direction="forward"
)
_=SFS_with_fixed.fit(X,y)
```

    ValueError: 
    All the 5 fits failed.
    .....
      File "/tmp/ipykernel_275/897617596.py", line 2, in <lambda>
        appender= FunctionTransformer(lambda X:pd.concat([X,fixed_features.reindex(X.index)],axis=1))
    AttributeError: 'numpy.ndarray' object has no attribute 'index'



### Describe your proposed solution

I believe that it should be possible to pass the reduced matrix to the estimator as a pandas DataFrame, and this would be consistent with the way other Transformers work currently.

### Describe alternatives you've considered, if relevant

An alternative that would suffice for my particular use case is to directly pass a set of fixed features as a parameter to the feature selector.

### Additional context

_No response_

COMMENT:
> This means I think we should try and come up with a simple rule, so that it is easy to predict what will happen, even if you dislike what happens, to guide the discussion and code changes.

I agree that we should document what we should do. I don't know if we will think of all potential cases thought.

Labels: Bug
Enable feature selectors to pass pandas DataFrame to estimator

### Describe the workflow you want to enable

When running SequentialFeatureSelector (or, presumably, other feature selection methods) with a pandas DataFrame input, the reduced-feature input is passed to the estimator as a numpy array. This seems inconsistent with the other transformers that successfully maintain the pandas indices.

My actual use case is the following: Select the best k features that complement a set of fixed features. I was going to implement it like this: (happy to take suggestions for alternatives).

```python
import pandas as pd
import numpy as np
import sklearn
from sklearn.feature_selection import SequentialFeatureSelector
from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import FunctionTransformer
from sklearn.linear_model import LinearRegression
from sklearn.pipeline import Pipeline

X=pd.DataFrame(np.random.rand(10,3))
y=pd.Series(np.random.rand(10))

fixed_features=pd.DataFrame(np.random.rand(X.shape[0],2))
appender= FunctionTransformer(lambda X:pd.concat([X,fixed_features.reindex(X.index)],axis=1))
X.shape,appender.fit_transform(X).shape
```

    ((10, 3), (10, 5))


```python
LR_with_fixed=LR=Pipeline(steps=[
    ('append',appender),
    ('LR',LinearRegression())
])
SFS_with_fixed = SequentialFeatureSelector(
    LR_with_fixed,n_features_to_select=1, direction="forward"
)
_=SFS_with_fixed.fit(X,y)
```

    ValueError: 
    All the 5 fits failed.
    .....
      File "/tmp/ipykernel_275/897617596.py", line 2, in <lambda>
        appender= FunctionTransformer(lambda X:pd.concat([X,fixed_features.reindex(X.index)],axis=1))
    AttributeError: 'numpy.ndarray' object has no attribute 'index'



### Describe your proposed solution

I believe that it should be possible to pass the reduced matrix to the estimator as a pandas DataFrame, and this would be consistent with the way other Transformers work currently.

### Describe alternatives you've considered, if relevant

An alternative that would suffice for my particular use case is to directly pass a set of fixed features as a parameter to the feature selector.

### Additional context

_No response_

COMMENT:
@glemaitre  is there anything I can help you with?

Labels: Bug
Enable feature selectors to pass pandas DataFrame to estimator

### Describe the workflow you want to enable

When running SequentialFeatureSelector (or, presumably, other feature selection methods) with a pandas DataFrame input, the reduced-feature input is passed to the estimator as a numpy array. This seems inconsistent with the other transformers that successfully maintain the pandas indices.

My actual use case is the following: Select the best k features that complement a set of fixed features. I was going to implement it like this: (happy to take suggestions for alternatives).

```python
import pandas as pd
import numpy as np
import sklearn
from sklearn.feature_selection import SequentialFeatureSelector
from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import FunctionTransformer
from sklearn.linear_model import LinearRegression
from sklearn.pipeline import Pipeline

X=pd.DataFrame(np.random.rand(10,3))
y=pd.Series(np.random.rand(10))

fixed_features=pd.DataFrame(np.random.rand(X.shape[0],2))
appender= FunctionTransformer(lambda X:pd.concat([X,fixed_features.reindex(X.index)],axis=1))
X.shape,appender.fit_transform(X).shape
```

    ((10, 3), (10, 5))


```python
LR_with_fixed=LR=Pipeline(steps=[
    ('append',appender),
    ('LR',LinearRegression())
])
SFS_with_fixed = SequentialFeatureSelector(
    LR_with_fixed,n_features_to_select=1, direction="forward"
)
_=SFS_with_fixed.fit(X,y)
```

    ValueError: 
    All the 5 fits failed.
    .....
      File "/tmp/ipykernel_275/897617596.py", line 2, in <lambda>
        appender= FunctionTransformer(lambda X:pd.concat([X,fixed_features.reindex(X.index)],axis=1))
    AttributeError: 'numpy.ndarray' object has no attribute 'index'



### Describe your proposed solution

I believe that it should be possible to pass the reduced matrix to the estimator as a pandas DataFrame, and this would be consistent with the way other Transformers work currently.

### Describe alternatives you've considered, if relevant

An alternative that would suffice for my particular use case is to directly pass a set of fixed features as a parameter to the feature selector.

### Additional context

_No response_

COMMENT:
I would be in favor of having at least a PR to check how costly it is to make the meta-estimator more lenient regarding the validation and see if we break some other tests.

I think that we should open a separate PR or issue to discuss the point raised by @betatim

Labels: Bug