Repository: scikit-learn/scikit-learn
Pull request: ENH Warn in SelectKBest when k > n_features
<!--
Thanks for contributing a pull request! Please ensure you have taken a look at
the contribution guidelines: https://github.com/scikit-learn/scikit-learn/blob/main/CONTRIBUTING.md
-->

#### Reference Issues/PRs
<!--
Example: Fixes #1234. See also #3456.
Please use keywords (e.g., Fixes) to create link to the issues or pull requests
you resolved, so that they will automatically be closed when your pull request
is merged. See https://github.com/blog/1506-closing-issues-via-pull-requests
-->
Closes https://github.com/scikit-learn/scikit-learn/issues/27540


#### What does this implement/fix? Explain your changes.
This PR converts the error in `SelectKBest` into a warning.


<!--
Please be aware that we are a loose team of volunteers so patience is
necessary; assistance handling other issues is very welcome. We value
all user contributions, no matter how minor they are. If we are slow to
review, either the pull request needs some benchmarking, tinkering,
convincing, etc. or more likely the reviewers are simply busy. In either
case, we ask for your understanding during the review process.
For more information, see our FAQ on this topic:
http://scikit-learn.org/dev/faq.html#why-is-my-pull-request-not-getting-any-attention.

Thanks for contributing!
-->


Problem statement:
SelectKBest shouldn't raise if k > n_samples
### Describe the workflow you want to enable

Let's say I want to build a logistic regression model with at most 50 features. I could do that with something like this: 
``make_pipeline(ColumnTransformer(...OneHotEncoder(), remainder="passthrough"), SelectKBest(k=50), LogisticRegression())``
but whether that estimator succeeds depends on the number of levels in the categorical variables, which is a bit strange. It would be great if ``SelectKBest`` wouldn't error, or at least optionally wouldn't error.

### Describe your proposed solution

I'm undecided between adding an option or changing the behavior. The easy thing would be to add an option but adding an option to disable an error is a bit strange. So ... a deprecation cycle for the error? that either means waiting for the deprecation to go through or adding a temporary variable and then deprecating that.

If someone expects the estimator to create exactly 50 features, then the error is reasonable, but I'm not sure if that's a good interpretation of what ``SelectKBest`` does?

### Describe alternatives you've considered, if relevant

In a pipeline like the above, there's no work-around as far as I can see.

### Additional context

_No response_

Hints from issue discussion:
I would not be opposed to raise a warning instead of the error mentioning that all available features are returned.
That would work for me, as the warning would be expected I could filter it out here.
To make sure I understand the case properly: you have a dataset where the number of features passed to `SelectKBest` is <=50? And it happens because you one hot encode a feature so the number of featurse ends up depending on the number of unique values in that feature.

Having a warning in the case where `SelectKBest` selects all the features seems reasonable. 

Can you link me to the code where the exception is raised? I was looking at https://github.com/scikit-learn/scikit-learn/blob/d99b728b3a7952b2111cf5e0cb5d14f92c6f3a80/sklearn/feature_selection/_base.py#L24 and can only find where a warning is raised if no features are selected. So there is precedent for using a warning.

Alternatively, some code that reproduces the problem?

Commit message: DOC Adds whats new

Patch to review:
diff --git a/doc/whats_new/v1.4.rst b/doc/whats_new/v1.4.rst
index f3f40fd26e27a..9f3f4cdf022b2 100644
--- a/doc/whats_new/v1.4.rst
+++ b/doc/whats_new/v1.4.rst
@@ -345,6 +345,11 @@ Changelog
   not check for nans during input validation.
   :pr:`21807` by `Thomas Fan`_.
 
+- |API| :class:`feature_selection.SelectKBest` and
+  :class:`feature_selection.GenericUnivariateSelect` with `mode='k_best'
+  now shows a warning when `k` is greater than the number of features.
+  :pr:`27841` by `Thomas Fan`_.
+
 :mod:`sklearn.inspection`
 .........................
 
diff --git a/sklearn/feature_selection/_univariate_selection.py b/sklearn/feature_selection/_univariate_selection.py
index f06889fb3b140..a2f3d89dc27a0 100644
--- a/sklearn/feature_selection/_univariate_selection.py
+++ b/sklearn/feature_selection/_univariate_selection.py
@@ -703,9 +703,9 @@ def __init__(self, score_func=f_classif, *, k=10):
 
     def _check_params(self, X, y):
         if not isinstance(self.k, str) and self.k > X.shape[1]:
-            raise ValueError(
-                f"k should be <= n_features = {X.shape[1]}; "
-                f"got {self.k}. Use k='all' to return all features."
+            warnings.warn(
+                f"k={self.k} is greater than n_features={X.shape[1]}. "
+                "All the features will be returned."
             )
 
     def _get_support_mask(self):
diff --git a/sklearn/feature_selection/tests/test_feature_select.py b/sklearn/feature_selection/tests/test_feature_select.py
index 8ad70d3c6f554..b7f527c5aa9d3 100644
--- a/sklearn/feature_selection/tests/test_feature_select.py
+++ b/sklearn/feature_selection/tests/test_feature_select.py
@@ -831,9 +831,10 @@ def test_invalid_k():
     X = [[0, 1, 0], [0, -1, -1], [0, 0.5, 0.5]]
     y = [1, 0, 1]
 
-    with pytest.raises(ValueError):
+    msg = "k=4 is greater than n_features=3. All the features will be returned."
+    with pytest.warns(UserWarning, match=msg):
         SelectKBest(k=4).fit(X, y)
-    with pytest.raises(ValueError):
+    with pytest.warns(UserWarning, match=msg):
         GenericUnivariateSelect(mode="k_best", param=4).fit(X, y)
 
 


Code review comments:
diff_hunk: @@ -345,6 +345,11 @@ Changelog
   not check for nans during input validation.
   :pr:`21807` by `Thomas Fan`_.
 
+- |API| :class:`feature_selection.SelectKBest` and
+  :class:`feature_selection.GenericUnivariateSelect` with `mode='k_best'
path: doc/whats_new/v1.4.rst
text: @user1:
```suggestion
  :class:`feature_selection.GenericUnivariateSelect` with `mode='k_best'`
```