Repository: scverse/scanpy

Issue:
scanpy.pp.pca fails on small datasets
<!-- Please give a clear and concise description of what the bug is: -->
scanpy.pp.pca fails if n_samples < 50 < n_features

<!-- Put a minimal reproducible example that reproduces the bug in the code block below: -->
```python
import numpy as np
import scanpy as sc
import anndata

adata = anndata.AnnData(np.random.normal(0, 1, (40, 100)))
sc.pp.pca(adata)
```

<!-- Put your Error output in this code block (if applicable, else delete the block): -->
```pytb
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/scottgigante/.local/lib/python3.8/site-packages/scanpy/preprocessing/_simple.py", line 531, in pca
    X_pca = pca_.fit_transform(X)
  File "/usr/lib/python3.8/site-packages/sklearn/decomposition/_pca.py", line 369, in fit_transform
    U, S, V = self._fit(X)
  File "/usr/lib/python3.8/site-packages/sklearn/decomposition/_pca.py", line 418, in _fit
    return self._fit_truncated(X, n_components, self._fit_svd_solver)
  File "/usr/lib/python3.8/site-packages/sklearn/decomposition/_pca.py", line 497, in _fit_truncated
    raise ValueError("n_components=%r must be between 1 and "
ValueError: n_components=50 must be between 1 and min(n_samples, n_features)=40 with svd_solver='arpack'
```

#### Versions:
<!-- Output of scanpy.logging.print_versions() -->
> scanpy==1.2.3.dev1409+g7ca201d.d20200112 anndata==0.6.22.post1 umap==0.3.10 numpy==1.18.0 scipy==1.4.1 pandas==0.25.3 scikit-learn==0.22 statsmodels==0.11.0rc1 python-igraph==0.7.1 louvain==0.6.1



Discussion:
That's not a bug, that's a feature ;). You can only compute as many PCs as the minimum number of dimensions in n_samples and n_features. Do you feel as though the error message is unclear on this? I feel as though changing the default to match the setting can be dangerous as may not recall how many PCs were used then.
Three things:

1. If that is a feature, then this is a bug (runs without error):

   ```python
   import numpy as np
   import scanpy as sc
   import anndata

   adata = anndata.AnnData(np.random.normal(0, 1, (40, 10)))
   sc.pp.pca(adata)
   ```

2. Defaults should work without tuning.
3. > I feel as though changing the default to match the setting can be dangerous as may not recall how many PCs were used then.

   Given that I'm running `sc.pp.pca` without setting `n_comps`, I contend that the average user does not remember what the default value is. It would make more sense in both cases (`n_features <= n_comps` and `n_samples <= n_comps`) to throw a warning and set n_comps to the maximum allowable value.
Hmm... you are right, that should also create an error by the above logic.

It does look like this check is done for the case that `adata.n_vars < n_comps` here:
https://github.com/theislab/scanpy/blob/be1a0555252cfd97b9d00f51dc5fbab462588da0/scanpy/preprocessing/_simple.py#L472-L477

I'm not sure why that wasn't also done for `n_obs`. @Koncopd you made this fix at the time... any reason for not also checking `adata.n_obs` in the same way? Could quickly add a check for `adata.n_obs` unless there is a reason not to @ivirshup, @flying-sheep?
I'm thinking `n_comps` should default to `None`, and we'd define behaviour like:

```python
if n_comps is None:
    min_dim = min(adata.n_vars, adata.n_obs)
    if 50 >= min_dim:
        n_comps = min_dim - 1
    else:
        n_comps = 50
```

and we let sklearn throw an error if the user specified a number of components that doesn't work.
It's not only about adata.n_vars, don't forget about use_highly_variable option e. g. sc.pp.highly_variable_genes(adata, n_top_genes=49).
Thanks for the catch! It should be `min_dim = min(*X.shape)` where `X` is the selected representation.

Resolving patch:
diff --git a/scanpy/preprocessing/_simple.py b/scanpy/preprocessing/_simple.py
--- a/scanpy/preprocessing/_simple.py
+++ b/scanpy/preprocessing/_simple.py
@@ -369,7 +369,7 @@ def sqrt(
 
 def pca(
     data: Union[AnnData, np.ndarray, spmatrix],
-    n_comps: int = N_PCS,
+    n_comps: Optional[int] = None,
     zero_center: Optional[bool] = True,
     svd_solver: str = 'arpack',
     random_state: Optional[Union[int, RandomState]] = 0,
@@ -392,7 +392,8 @@ def pca(
         The (annotated) data matrix of shape `n_obs` × `n_vars`.
         Rows correspond to cells and columns to genes.
     n_comps
-        Number of principal components to compute.
+        Number of principal components to compute. Defaults to 50, or 1 - minimum
+        dimension size of selected representation.
     zero_center
         If `True`, compute standard PCA from covariance matrix.
         If `False`, omit zero-centering variables
@@ -457,8 +458,7 @@ def pca(
         logg.info(
             'Note that scikit-learn\'s randomized PCA might not be exactly '
             'reproducible across different computational platforms. For exact '
-            'reproducibility, choose `svd_solver=\'arpack\'.` This will likely '
-            'become the Scanpy default in the future.'
+            'reproducibility, choose `svd_solver=\'arpack\'.`'
         )
 
     data_is_AnnData = isinstance(data, AnnData)
@@ -467,15 +467,6 @@ def pca(
     else:
         adata = AnnData(data)
 
-    start = logg.info(f'computing PCA with n_comps = {n_comps}')
-
-    if adata.n_vars < n_comps:
-        n_comps = adata.n_vars - 1
-        logg.debug(
-            f'reducing number of computed PCs to {n_comps} '
-            f'as dim of data is only {adata.n_vars}'
-        )
-
     if use_highly_variable is True and 'highly_variable' not in adata.var.keys():
         raise ValueError('Did not find adata.var[\'highly_variable\']. '
                          'Either your data already only consists of highly-variable genes '
@@ -486,6 +477,15 @@ def pca(
         logg.info('    on highly variable genes')
     adata_comp = adata[:, adata.var['highly_variable']] if use_highly_variable else adata
 
+    if n_comps is None:
+        min_dim = min(adata_comp.n_vars, adata_comp.n_obs)
+        if N_PCS >= min_dim:
+            n_comps = min_dim - 1
+        else:
+            n_comps = N_PCS
+
+    start = logg.info(f'computing PCA with n_comps = {n_comps}')
+
     if chunked:
         if not zero_center or random_state or svd_solver != 'arpack':
             logg.debug('Ignoring zero_center, random_state, svd_solver')


Test patch:
diff --git a/scanpy/tests/test_pca.py b/scanpy/tests/test_pca.py
--- a/scanpy/tests/test_pca.py
+++ b/scanpy/tests/test_pca.py
@@ -51,3 +51,18 @@ def test_pca_transform(typ):
 
     sc.pp.pca(adata, n_comps=4, zero_center=False, dtype='float64', random_state=14)
     assert np.linalg.norm(A_svd_abs[:, :4]-np.abs(adata.obsm['X_pca'])) < 2e-05
+
+
+def test_pca_shapes():
+    """Tests that n_comps behaves correctly"""
+    # https://github.com/theislab/scanpy/issues/1051
+    adata = AnnData(np.random.randn(30, 20))
+    sc.pp.pca(adata)
+    assert adata.obsm["X_pca"].shape == (30, 19)
+
+    adata = AnnData(np.random.randn(20, 30))
+    sc.pp.pca(adata)
+    assert adata.obsm["X_pca"].shape == (20, 19)
+
+    with pytest.raises(ValueError):
+        sc.pp.pca(adata, n_comps=100)