Talip7/scikit-learn-issues-multilabel huggingface dataset:
Implement class that removes features with NaNs in sklearn.feature_selection module

### Describe the workflow you want to enable

Currently, there seems to be no way in Sklearn to remove features with NaNs (only to impute missing values). As the number of NaNs increases, imputation becomes less trustworthy, so I implemented my own class to remove features with a large number of NaNs. The inspiration for this were the drug variables from the Human Connectome project. Many of the 1200 subjects simply did not take any hard drugs, so we have a large number of NaNs in the those variables. 

### Describe your proposed solution

Implement a new class in `sklearn.feature_selection`. Something like:

```python
class NaNRemover(SelectorMixin,BaseEstimator):
    '''Feature selector that removes all features with missing values
    greater than provided threshold'''
    
    def __init__(self,threshold=0.1):
        self.threshold = threshold
        
    def fit(self,X,y=None):
        
        X = self._validate_data(
            X,
            accept_sparse=("csr","csc"),
            dtype=np.float64,
            force_all_finite="allow-nan"
            )
        
        n_samples = X.shape[0]
        
        self.threshold = round(self.threshold * n_samples)
        self.nans_per_feature = np.count_nonzero(np.isnan(X),axis=0)
        
        return self
    
    def _get_support_mask(self):
        return self.nans_per_feature < self.threshold
    
    def _more_tags(self):
        return {"allow_nan": True}
```

With the default value, this will drop all features where 10% of the samples have NaNs (thus becoming more 'liberal' with increasing `threshold`, i.e. allowing more rows to have NaNs. E.g. a `threshold=1.0` would only drop features, were all samples have NaNs (thus having a similar logic as `VarianceThreshold`, because all values are the same). Of course one could also think about implementing it with the opposite logic (lower threshold --> more liberal).

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
So it seems like we have consensus that this shouldn't be a transformer. Closing, will re-open if more arguments in favor of inclusion poor in.

Labels: New Feature, Needs Decision - Include Feature
Implement class that removes features with NaNs in sklearn.feature_selection module

### Describe the workflow you want to enable

Currently, there seems to be no way in Sklearn to remove features with NaNs (only to impute missing values). As the number of NaNs increases, imputation becomes less trustworthy, so I implemented my own class to remove features with a large number of NaNs. The inspiration for this were the drug variables from the Human Connectome project. Many of the 1200 subjects simply did not take any hard drugs, so we have a large number of NaNs in the those variables. 

### Describe your proposed solution

Implement a new class in `sklearn.feature_selection`. Something like:

```python
class NaNRemover(SelectorMixin,BaseEstimator):
    '''Feature selector that removes all features with missing values
    greater than provided threshold'''
    
    def __init__(self,threshold=0.1):
        self.threshold = threshold
        
    def fit(self,X,y=None):
        
        X = self._validate_data(
            X,
            accept_sparse=("csr","csc"),
            dtype=np.float64,
            force_all_finite="allow-nan"
            )
        
        n_samples = X.shape[0]
        
        self.threshold = round(self.threshold * n_samples)
        self.nans_per_feature = np.count_nonzero(np.isnan(X),axis=0)
        
        return self
    
    def _get_support_mask(self):
        return self.nans_per_feature < self.threshold
    
    def _more_tags(self):
        return {"allow_nan": True}
```

With the default value, this will drop all features where 10% of the samples have NaNs (thus becoming more 'liberal' with increasing `threshold`, i.e. allowing more rows to have NaNs. E.g. a `threshold=1.0` would only drop features, were all samples have NaNs (thus having a similar logic as `VarianceThreshold`, because all values are the same). Of course one could also think about implementing it with the opposite logic (lower threshold --> more liberal).

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
I might approach the scikit-lego peepz and agree with your comments.

Labels: New Feature, Needs Decision - Include Feature
Implement class that removes features with NaNs in sklearn.feature_selection module

### Describe the workflow you want to enable

Currently, there seems to be no way in Sklearn to remove features with NaNs (only to impute missing values). As the number of NaNs increases, imputation becomes less trustworthy, so I implemented my own class to remove features with a large number of NaNs. The inspiration for this were the drug variables from the Human Connectome project. Many of the 1200 subjects simply did not take any hard drugs, so we have a large number of NaNs in the those variables. 

### Describe your proposed solution

Implement a new class in `sklearn.feature_selection`. Something like:

```python
class NaNRemover(SelectorMixin,BaseEstimator):
    '''Feature selector that removes all features with missing values
    greater than provided threshold'''
    
    def __init__(self,threshold=0.1):
        self.threshold = threshold
        
    def fit(self,X,y=None):
        
        X = self._validate_data(
            X,
            accept_sparse=("csr","csc"),
            dtype=np.float64,
            force_all_finite="allow-nan"
            )
        
        n_samples = X.shape[0]
        
        self.threshold = round(self.threshold * n_samples)
        self.nans_per_feature = np.count_nonzero(np.isnan(X),axis=0)
        
        return self
    
    def _get_support_mask(self):
        return self.nans_per_feature < self.threshold
    
    def _more_tags(self):
        return {"allow_nan": True}
```

With the default value, this will drop all features where 10% of the samples have NaNs (thus becoming more 'liberal' with increasing `threshold`, i.e. allowing more rows to have NaNs. E.g. a `threshold=1.0` would only drop features, were all samples have NaNs (thus having a similar logic as `VarianceThreshold`, because all values are the same). Of course one could also think about implementing it with the opposite logic (lower threshold --> more liberal).

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
Looks like this is now implemented in the feature_engine package:

https://feature-engine.trainindata.com/en/latest/user_guide/imputation/DropMissingData.html

Labels: New Feature, Needs Decision - Include Feature
Implement class that removes features with NaNs in sklearn.feature_selection module

### Describe the workflow you want to enable

Currently, there seems to be no way in Sklearn to remove features with NaNs (only to impute missing values). As the number of NaNs increases, imputation becomes less trustworthy, so I implemented my own class to remove features with a large number of NaNs. The inspiration for this were the drug variables from the Human Connectome project. Many of the 1200 subjects simply did not take any hard drugs, so we have a large number of NaNs in the those variables. 

### Describe your proposed solution

Implement a new class in `sklearn.feature_selection`. Something like:

```python
class NaNRemover(SelectorMixin,BaseEstimator):
    '''Feature selector that removes all features with missing values
    greater than provided threshold'''
    
    def __init__(self,threshold=0.1):
        self.threshold = threshold
        
    def fit(self,X,y=None):
        
        X = self._validate_data(
            X,
            accept_sparse=("csr","csc"),
            dtype=np.float64,
            force_all_finite="allow-nan"
            )
        
        n_samples = X.shape[0]
        
        self.threshold = round(self.threshold * n_samples)
        self.nans_per_feature = np.count_nonzero(np.isnan(X),axis=0)
        
        return self
    
    def _get_support_mask(self):
        return self.nans_per_feature < self.threshold
    
    def _more_tags(self):
        return {"allow_nan": True}
```

With the default value, this will drop all features where 10% of the samples have NaNs (thus becoming more 'liberal' with increasing `threshold`, i.e. allowing more rows to have NaNs. E.g. a `threshold=1.0` would only drop features, were all samples have NaNs (thus having a similar logic as `VarianceThreshold`, because all values are the same). Of course one could also think about implementing it with the opposite logic (lower threshold --> more liberal).

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
You can also use skrub's Cleaner: https://skrub-data.org/stable/reference/generated/skrub.Cleaner.html, with the `drop_null_fraction` argument

Labels: New Feature, Needs Decision - Include Feature
TypeError: '<=' not supported between instances of 'str' and 'int' when using fit_predict

I am trying to utilize silhouette analysis on KMeans clustering in order to determine how to choose the optimal number of clusters in a given dataset.
I tried the example code provided on [scikit-learn](https://scikit-learn.org/stable/auto_examples/cluster/plot_kmeans_silhouette_analysis.html#) but it seems to produce an error whenever I try to run it.

Traceback:
```
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Input In [14], in <cell line: 24>()
     37 # Initialize the clusterer with n_clusters value and a random generator
     38 # seed of 10 for reproducibility.
     39 clusterer = KMeans(n_clusters=n_clusters, n_init="auto", random_state=10)
---> 40 cluster_labels = clusterer.fit_predict(X)
     42 # The silhouette_score gives the average value for all the samples.
     43 # This gives a perspective into the density and separation of the formed
     44 # clusters
     45 silhouette_avg = silhouette_score(X, cluster_labels)

File ~\Anaconda3\lib\site-packages\sklearn\cluster\_kmeans.py:1255, in KMeans.fit_predict(self, X, y, sample_weight)
   1232 def fit_predict(self, X, y=None, sample_weight=None):
   1233     """Compute cluster centers and predict cluster index for each sample.
   1234 
   1235     Convenience method; equivalent to calling fit(X) followed by
   (...)
   1253         Index of the cluster each sample belongs to.
   1254     """
-> 1255     return self.fit(X, sample_weight=sample_weight).labels_

File ~\Anaconda3\lib\site-packages\sklearn\cluster\_kmeans.py:1146, in KMeans.fit(self, X, y, sample_weight)
   1112 """Compute k-means clustering.
   1113 
   1114 Parameters
   (...)
   1135     Fitted estimator.
   1136 """
   1137 X = self._validate_data(
   1138     X,
   1139     accept_sparse="csr",
   (...)
   1143     accept_large_sparse=False,
   1144 )
-> 1146 self._check_params(X)
   1147 random_state = check_random_state(self.random_state)
   1148 sample_weight = _check_sample_weight(sample_weight, X, dtype=X.dtype)

File ~\Anaconda3\lib\site-packages\sklearn\cluster\_kmeans.py:937, in KMeans._check_params(self, X)
    935 def _check_params(self, X):
    936     # n_init
--> 937     if self.n_init <= 0:
    938         raise ValueError(f"n_init should be > 0, got {self.n_init} instead.")
    939     self._n_init = self.n_init

TypeError: '<=' not supported between instances of 'str' and 'int'
```

Is there something I am missing here? I iterated through the sample data and did not see any strings present in `X`. I copied and ran the exact same code as present on the website.

Thank you!

COMMENT:
Ok so I tried with the old version 1.1 code and it seemed to have worked. I'm going to assume that there might have been some changed in 1.2 that have not been updated on the webpage.
I'm going to close the question since working with the 1.1 code seems to be working just fine.
Thank you!

Labels: Question
TypeError: '<=' not supported between instances of 'str' and 'int' when using fit_predict

I am trying to utilize silhouette analysis on KMeans clustering in order to determine how to choose the optimal number of clusters in a given dataset.
I tried the example code provided on [scikit-learn](https://scikit-learn.org/stable/auto_examples/cluster/plot_kmeans_silhouette_analysis.html#) but it seems to produce an error whenever I try to run it.

Traceback:
```
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Input In [14], in <cell line: 24>()
     37 # Initialize the clusterer with n_clusters value and a random generator
     38 # seed of 10 for reproducibility.
     39 clusterer = KMeans(n_clusters=n_clusters, n_init="auto", random_state=10)
---> 40 cluster_labels = clusterer.fit_predict(X)
     42 # The silhouette_score gives the average value for all the samples.
     43 # This gives a perspective into the density and separation of the formed
     44 # clusters
     45 silhouette_avg = silhouette_score(X, cluster_labels)

File ~\Anaconda3\lib\site-packages\sklearn\cluster\_kmeans.py:1255, in KMeans.fit_predict(self, X, y, sample_weight)
   1232 def fit_predict(self, X, y=None, sample_weight=None):
   1233     """Compute cluster centers and predict cluster index for each sample.
   1234 
   1235     Convenience method; equivalent to calling fit(X) followed by
   (...)
   1253         Index of the cluster each sample belongs to.
   1254     """
-> 1255     return self.fit(X, sample_weight=sample_weight).labels_

File ~\Anaconda3\lib\site-packages\sklearn\cluster\_kmeans.py:1146, in KMeans.fit(self, X, y, sample_weight)
   1112 """Compute k-means clustering.
   1113 
   1114 Parameters
   (...)
   1135     Fitted estimator.
   1136 """
   1137 X = self._validate_data(
   1138     X,
   1139     accept_sparse="csr",
   (...)
   1143     accept_large_sparse=False,
   1144 )
-> 1146 self._check_params(X)
   1147 random_state = check_random_state(self.random_state)
   1148 sample_weight = _check_sample_weight(sample_weight, X, dtype=X.dtype)

File ~\Anaconda3\lib\site-packages\sklearn\cluster\_kmeans.py:937, in KMeans._check_params(self, X)
    935 def _check_params(self, X):
    936     # n_init
--> 937     if self.n_init <= 0:
    938         raise ValueError(f"n_init should be > 0, got {self.n_init} instead.")
    939     self._n_init = self.n_init

TypeError: '<=' not supported between instances of 'str' and 'int'
```

Is there something I am missing here? I iterated through the sample data and did not see any strings present in `X`. I copied and ran the exact same code as present on the website.

Thank you!

COMMENT:
same problem with me while checking scikit learn k-mean quantization example

Labels: Question
TypeError: '<=' not supported between instances of 'str' and 'int' when using fit_predict

I am trying to utilize silhouette analysis on KMeans clustering in order to determine how to choose the optimal number of clusters in a given dataset.
I tried the example code provided on [scikit-learn](https://scikit-learn.org/stable/auto_examples/cluster/plot_kmeans_silhouette_analysis.html#) but it seems to produce an error whenever I try to run it.

Traceback:
```
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Input In [14], in <cell line: 24>()
     37 # Initialize the clusterer with n_clusters value and a random generator
     38 # seed of 10 for reproducibility.
     39 clusterer = KMeans(n_clusters=n_clusters, n_init="auto", random_state=10)
---> 40 cluster_labels = clusterer.fit_predict(X)
     42 # The silhouette_score gives the average value for all the samples.
     43 # This gives a perspective into the density and separation of the formed
     44 # clusters
     45 silhouette_avg = silhouette_score(X, cluster_labels)

File ~\Anaconda3\lib\site-packages\sklearn\cluster\_kmeans.py:1255, in KMeans.fit_predict(self, X, y, sample_weight)
   1232 def fit_predict(self, X, y=None, sample_weight=None):
   1233     """Compute cluster centers and predict cluster index for each sample.
   1234 
   1235     Convenience method; equivalent to calling fit(X) followed by
   (...)
   1253         Index of the cluster each sample belongs to.
   1254     """
-> 1255     return self.fit(X, sample_weight=sample_weight).labels_

File ~\Anaconda3\lib\site-packages\sklearn\cluster\_kmeans.py:1146, in KMeans.fit(self, X, y, sample_weight)
   1112 """Compute k-means clustering.
   1113 
   1114 Parameters
   (...)
   1135     Fitted estimator.
   1136 """
   1137 X = self._validate_data(
   1138     X,
   1139     accept_sparse="csr",
   (...)
   1143     accept_large_sparse=False,
   1144 )
-> 1146 self._check_params(X)
   1147 random_state = check_random_state(self.random_state)
   1148 sample_weight = _check_sample_weight(sample_weight, X, dtype=X.dtype)

File ~\Anaconda3\lib\site-packages\sklearn\cluster\_kmeans.py:937, in KMeans._check_params(self, X)
    935 def _check_params(self, X):
    936     # n_init
--> 937     if self.n_init <= 0:
    938         raise ValueError(f"n_init should be > 0, got {self.n_init} instead.")
    939     self._n_init = self.n_init

TypeError: '<=' not supported between instances of 'str' and 'int'
```

Is there something I am missing here? I iterated through the sample data and did not see any strings present in `X`. I copied and ran the exact same code as present on the website.

Thank you!

COMMENT:
Thanks for quick reply, I have updated my Scikit learn and the problem is resolved.

Labels: Question
ValueError: dimension mismatch for Logistic Regression.

### Discussed in https://github.com/scikit-learn/scikit-learn/discussions/25625

<div type='discussions-op-text'>

<sup>Originally posted by **samarthpatel1289** February 16, 2023</sup>
I was looking for solution to this. Followed the sklearn documentaion not sure what i am missing. 

```python
    #retrieving datasets from openml
    dataset_openml = openml.datasets.get_dataset(43360)
    df, _, _, _ = dataset_openml.get_data(dataset_format="dataframe")

    #changing ratings into likes and dislike. >3 = 1 else -1
    df = df[df['rating'] != 3]
    df['sentiment'] = df['rating'].apply(lambda rating : +1 if rating > 3 else -1)
    df['text'] = df['text'].apply(remove_punctuation)

    index = df.index
    df['random_number'] = np.random.randn(len(index))
    train = df[df['random_number'] <= 0.8]
    test = df[df['random_number'] > 0.8]

    vectorizer = CountVectorizer(token_pattern=r'\b\w+\b')
    train_matrix = vectorizer.fit_transform(train['text'])
    test_matrix = vectorizer.transform(test['text'])

    x_train = train_matrix
    x_test = test_matrix
    y_train = train['sentiment']
    y_test = test['sentiment']

    lr = LogisticRegression()
    #setting initial parameters for FL
    set_initial_params(lr)
    
    lr.predict_proba(x_test)
```

```
def set_initial_params(model: LogisticRegression):
    n_classes = 2
    n_features = 1
    model.classes_ = np.array([-1, 1])

    model.coef_ = np.zeros((1, n_features))
    if model.fit_intercept:
        model.intercept_ = np.zeros((1,))
```


```
Traceback (most recent call last):
  File "path/to/file", line 91, in <module>
    execute()
  File "path/to/file", line 36, in execute
    lr.predict_proba(x_test)
  File "path/to/file/.pyenv/versions/design/lib/python3.9/site-packages/sklearn/linear_model/_logistic.py", line 1372, in predict_proba
    return super()._predict_proba_lr(X)
  File "path/to/file/.pyenv/versions/design/lib/python3.9/site-packages/sklearn/linear_model/_base.py", line 434, in _predict_proba_lr
    prob = self.decision_function(X)
  File "path/to/file/.pyenv/versions/design/lib/python3.9/site-packages/sklearn/linear_model/_base.py", line 401, in decision_function
    scores = safe_sparse_dot(X, self.coef_.T, dense_output=True) + self.intercept_
  File "path/to/file/.pyenv/versions/design/lib/python3.9/site-packages/sklearn/utils/extmath.py", line 189, in safe_sparse_dot
    ret = a @ b
  File "path/to/file/.pyenv/versions/design/lib/python3.9/site-packages/scipy/sparse/_base.py", line 630, in __matmul__
    return self._mul_dispatch(other)
  File "path/to/file/.pyenv/versions/design/lib/python3.9/site-packages/scipy/sparse/_base.py", line 559, in _mul_dispatch
    raise ValueError('dimension mismatch')
ValueError: dimension mismatch
```
</div>

COMMENT:
Since you are setting yourself the parameters to be used during `predict`, I assume that you have the wrong shape for those. Try to fit the data and check the shape of the `coef_` and `intercept_`.

Since there is not a `fit(X, y).predict(X)` here, this is not a bug.

Labels: Needs Triage