Repository: scikit-learn/scikit-learn
Pull request: ENH Allow n_components = n_features in TruncatedSVD
<!--
Thanks for contributing a pull request! Please ensure you have taken a look at
the contribution guidelines: https://github.com/scikit-learn/scikit-learn/blob/main/CONTRIBUTING.md
-->

#### Reference Issues/PRs
<!--
Example: Fixes #1234. See also #3456.
Please use keywords (e.g., Fixes) to create link to the issues or pull requests
you resolved, so that they will automatically be closed when your pull request
is merged. See https://github.com/blog/1506-closing-issues-via-pull-requests
-->
Fixes #22135

#### What does this implement/fix? Explain your changes.

Update `sklearn.decomposition.TruncatedSVD` to allow `n_components == min(n_samples, n_features)` if `algorithm == 'randomized'`.


`sklearn.decomposition.PCA` allows me to use `svd_solver = 'randomized'` with `n_components == min(n_samples, n_features)` (in fact, this is the *default*!). However, if I use `sklearn.decomposition.TruncatedSVD` with the same data and `algorithm == 'randomized'` I get an error asking me to reset `n_components` to `min(n_samples, n_features) - 1`.

This seems odd to me.  If the randomized algorithm works fine for PCA with `n_components == min(n_samples, n_features)`, why not allow this same case for TruncatedSVD? The PCA class even says `Notice that this class does not support sparse input. See TruncatedSVD for an alternative with sparse data`, which implies that you can use `TruncatedSVD` to achieve PCA on sparse  input, but if you want to use `n_components == min(n_samples, n_features)`, this isn't actually the case.


#### Any other comments?


<!--
Please be aware that we are a loose team of volunteers so patience is
necessary; assistance handling other issues is very welcome. We value
all user contributions, no matter how minor they are. If we are slow to
review, either the pull request needs some benchmarking, tinkering,
convincing, etc. or more likely the reviewers are simply busy. In either
case, we ask for your understanding during the review process.
For more information, see our FAQ on this topic:
http://scikit-learn.org/dev/faq.html#why-is-my-pull-request-not-getting-any-attention.

Thanks for contributing!
-->


Problem statement:
Allow `n_components == min(n_samples, n_features)` in TruncatedSVD
### Describe the workflow you want to enable

`sklearn.decomposition.PCA` allows me to use `svd_solver = 'randomized'` with `n_components == min(n_samples, n_features)` (in fact, this is the *default*!)

However, if I use `sklearn.decomposition.TruncatedSVD` with the same data and `algorithm == 'randomized'` I get an error asking me to reset `n_components` to `min(n_samples, n_features) - 1`.

This seems odd to me.  If the randomized algorithm works fine for PCA with `n_components == min(n_samples, n_features)`, why not allow this same case for TruncatedSVD?

The PCA class even says `Notice that this class does not support sparse input. See TruncatedSVD for an alternative with sparse data`, which implies that you can use `TruncatedSVD` to achieve PCA on sparse  input, but if you want to use `n_components == min(n_samples, n_features)`, this isn't actually the case.

### Describe your proposed solution

Updated `sklearn.decomposition.TruncatedSVD` to allow `n_components == min(n_samples, n_features)` if `algorithm == 'randomized'`.

### Describe alternatives you've considered, if relevant

Adding a vector of zeros to my data to trick `TruncatedSVD` into allowing me to run SVD with the number of components I want.

### Additional context

I have a sparse dataset with many rows and 2 columns.  I would like to do something like PCA on this data, but I can't, as PCA requires dense input and I do not want to convert my data to dense.

I would like to use `TruncatedSVD` as an alternative to PCA on my sparse dataset (as proposed directly in the documentation!), but I can't without dropping one of the dimensions, which I don't want to do.

Hints from issue discussion:
I do not see any issue with supporting `n_components=min(n_samples, n_features)` with `algorithm="randomized"`. At a glance, we would need to remove the following and update tests/docs:

https://github.com/scikit-learn/scikit-learn/blob/26e2c38a961a27a9e53ce7814bf27f840510b237/sklearn/decomposition/_truncated_svd.py#L206-L209
@thomasjpfan I could make a PR— I may need some help finding the correct tests / docs to update
@zachmayer Feel free to open a PR for this issue.

Commit message: update tests and docs

Patch to review:
diff --git a/sklearn/decomposition/_truncated_svd.py b/sklearn/decomposition/_truncated_svd.py
index 01d79f742302f..1dcd695bb8e2b 100644
--- a/sklearn/decomposition/_truncated_svd.py
+++ b/sklearn/decomposition/_truncated_svd.py
@@ -44,7 +44,8 @@ class TruncatedSVD(_ClassNamePrefixFeaturesOutMixin, TransformerMixin, BaseEstim
     ----------
     n_components : int, default=2
         Desired dimensionality of output data.
-        Must be strictly less than the number of features.
+        If algorithm='arpack', must be strictly less than the number of features.
+        If algorithm='randomized', must be less than or equal to the number of features.
         The default value is useful for visualisation. For LSA, a value of
         100 is recommended.
 
@@ -203,9 +204,9 @@ def fit_transform(self, X, y=None):
         elif self.algorithm == "randomized":
             k = self.n_components
             n_features = X.shape[1]
-            if k >= n_features:
+            if k > n_features:
                 raise ValueError(
-                    "n_components must be < n_features; got %d >= %d" % (k, n_features)
+                    "n_components must be <= n_features; got %d > %d" % (k, n_features)
                 )
             U, Sigma, VT = randomized_svd(
                 X, self.n_components, n_iter=self.n_iter, random_state=random_state
diff --git a/sklearn/decomposition/tests/test_truncated_svd.py b/sklearn/decomposition/tests/test_truncated_svd.py
index f227585f4ccf7..5a6293a799c6a 100644
--- a/sklearn/decomposition/tests/test_truncated_svd.py
+++ b/sklearn/decomposition/tests/test_truncated_svd.py
@@ -52,8 +52,11 @@ def test_too_many_components(algorithm, X_sparse):
     n_features = X_sparse.shape[1]
     for n_components in (n_features, n_features + 1):
         tsvd = TruncatedSVD(n_components=n_components, algorithm=algorithm)
-        with pytest.raises(ValueError):
+        if n_components == n_features and algorithm == "randomized":
             tsvd.fit(X_sparse)
+        else:
+            with pytest.raises(ValueError):
+                tsvd.fit(X_sparse)
 
 
 @pytest.mark.parametrize("fmt", ("array", "csr", "csc", "coo", "lil"))


Code review comments:
diff_hunk: @@ -203,9 +204,9 @@ def fit_transform(self, X, y=None):
         elif self.algorithm == "randomized":
             k = self.n_components
             n_features = X.shape[1]
-            if k >= n_features:
+            if k > n_features:
                 raise ValueError(
-                    "n_components must be < n_features; got %d >= %d" % (k, n_features)
+                    "n_components must be <= n_features; got %d > %d" % (k, n_features)
path: sklearn/decomposition/_truncated_svd.py
text: @user1:
While we are here, I think we can use `check_scalar`:

```python
from ..utils.validation import check_scalar

            check_scalar(
                k,
                "n_components",
                target_type=numbers.Integral,
                min_val=1,
                max_val=n_features,
            )
```

@author:
Do I need to import `numbers` to get `numbers.Integral`?

@user1:
Yes it's from `numbers`.

@author:
I added the numbers import.  I updated the code, thank you for the feedback!

diff_hunk: @@ -52,8 +52,11 @@ def test_too_many_components(algorithm, X_sparse):
     n_features = X_sparse.shape[1]
     for n_components in (n_features, n_features + 1):
         tsvd = TruncatedSVD(n_components=n_components, algorithm=algorithm)
-        with pytest.raises(ValueError):
+        if n_components == n_features and algorithm == "randomized":
             tsvd.fit(X_sparse)
+        else:
+            with pytest.raises(ValueError):
+                tsvd.fit(X_sparse)
path: sklearn/decomposition/tests/test_truncated_svd.py
text: @user2:
Since this test is call "test_too_many_components" I think it is geared toward errors. We can refactor the test as follows:

```python
@user1.mark.parametrize(
    "algorithm, n_components",
    [
        ("arpack", 55),
        ("arpack", 56),
        ("randomized", 56),
    ],
)
def test_too_many_components(X_sparse, algorithm, n_components):
    tsvd = TruncatedSVD(n_components=n_components, algorithm=algorithm)
    with pytest.raises(ValueError):
        tsvd.fit(X_sparse)
```

@author:
Done.  Thank you for the feedback!