Repository: scipy/scipy

Issue:
LOBPCG inefficinet when computing > 20% of eigenvalues
**Is your feature request related to a problem? Please describe.**
Even if the original input matrices were given as arrays, each of them is first turned into a LinearOperator: 
```
    A = _makeOperator(A, (n, n))
    B = _makeOperator(B, (n, n))
```

When lobpcg is requested to compute > 20% of eigenvalues, it simply switches to `eigh`, but before that needs to recover the matrices back:
```
        A_dense = A(np.eye(n, dtype=A.dtype))
        B_dense = None if B is None else B(np.eye(n, dtype=B.dtype))

        vals, vecs = eigh(A_dense, B_dense, eigvals=eigvals,
                          check_finite=False)
```
which is a dramatic waste. 

**Describe the solution you'd like**
Unless original input matrices were given as LinearOperator's, avoid 
```
        A_dense = A(np.eye(n, dtype=A.dtype))
        B_dense = None if B is None else B(np.eye(n, dtype=B.dtype))
```
when  computing > 20% of eigenvalues and call `eigh` directly on the original matrices.  



Discussion:

Resolving patch:
diff --git a/scipy/sparse/linalg/_eigen/lobpcg/lobpcg.py b/scipy/sparse/linalg/_eigen/lobpcg/lobpcg.py
--- a/scipy/sparse/linalg/_eigen/lobpcg/lobpcg.py
+++ b/scipy/sparse/linalg/_eigen/lobpcg/lobpcg.py
@@ -21,7 +21,8 @@
 import numpy as np
 from scipy.linalg import (inv, eigh, cho_factor, cho_solve,
                           cholesky, LinAlgError)
-from scipy.sparse.linalg import aslinearoperator
+from scipy.sparse.linalg import LinearOperator, aslinearoperator
+from scipy.sparse import isspmatrix
 from numpy import block as bmat
 
 __all__ = ["lobpcg"]
@@ -192,33 +193,36 @@ def lobpcg(
     the return tuple has the following format
     ``(lambda, V, lambda history, residual norms history)``.
 
-    In the following ``n`` denotes the matrix size and ``m`` the number
+    In the following ``n`` denotes the matrix size and ``k`` the number
     of required eigenvalues (smallest or largest).
 
-    The LOBPCG code internally solves eigenproblems of the size ``3m`` on every
-    iteration by calling the "standard" dense eigensolver, so if ``m`` is not
+    The LOBPCG code internally solves eigenproblems of the size ``3k`` on every
+    iteration by calling the "standard" dense eigensolver, so if ``k`` is not
     small enough compared to ``n``, it does not make sense to call the LOBPCG
     code, but rather one should use the "standard" eigensolver, e.g. numpy or
     scipy function in this case.
-    If one calls the LOBPCG algorithm for ``5m > n``, it will most likely break
+    If one calls the LOBPCG algorithm for ``5k > n``, it will most likely break
     internally, so the code tries to call the standard function instead.
 
     It is not that ``n`` should be large for the LOBPCG to work, but rather the
-    ratio ``n / m`` should be large. It you call LOBPCG with ``m=1``
+    ratio ``n / k`` should be large. It you call LOBPCG with ``k=1``
     and ``n=10``, it works though ``n`` is small. The method is intended
-    for extremely large ``n / m``.
+    for extremely large ``n / k``.
 
     The convergence speed depends basically on two factors:
 
-    1. How well relatively separated the seeking eigenvalues are from the rest
-       of the eigenvalues. One can try to vary ``m`` to make this better.
-
-    2. How well conditioned the problem is. This can be changed by using proper
-       preconditioning. For example, a rod vibration test problem (under tests
+    1. Relative separation of the seeking eigenvalues from the rest
+       of the eigenvalues. One can vary ``k`` to improve the absolute
+       separation and use proper preconditioning to shrink the spectral spread.
+       For example, a rod vibration test problem (under tests
        directory) is ill-conditioned for large ``n``, so convergence will be
        slow, unless efficient preconditioning is used. For this specific
        problem, a good simple preconditioner function would be a linear solve
-       for `A`, which is easy to code since A is tridiagonal.
+       for `A`, which is easy to code since `A` is tridiagonal.
+
+    2. Quality of the initial approximations `X` to the seeking eigenvectors.
+       Randomly distributed around the origin vectors work well if no better
+       choice is known.
 
     References
     ----------
@@ -343,10 +347,6 @@ def lobpcg(
                 aux += "%d constraint\n\n" % sizeY
         print(aux)
 
-    A = _makeOperator(A, (n, n))
-    B = _makeOperator(B, (n, n))
-    M = _makeOperator(M, (n, n))
-
     if (n - sizeY) < (5 * sizeX):
         warnings.warn(
             f"The problem size {n} minus the constraints size {sizeY} "
@@ -368,11 +368,23 @@ def lobpcg(
         else:
             eigvals = (0, sizeX - 1)
 
-        A_dense = A(np.eye(n, dtype=A.dtype))
-        B_dense = None if B is None else B(np.eye(n, dtype=B.dtype))
+        if isinstance(A, LinearOperator):
+            A = A(np.eye(n, dtype=A.dtype))
+        elif isspmatrix(A):
+            A = A.toarray()
+        else:
+            A = np.asarray(A)
+
+        if B is not None:
+            if isinstance(B, LinearOperator):
+                B = B(np.eye(n, dtype=B.dtype))
+            elif isspmatrix(B):
+                B = B.toarray()
+            else:
+                B = np.asarray(B)
 
-        vals, vecs = eigh(A_dense,
-                          B_dense,
+        vals, vecs = eigh(A,
+                          B,
                           eigvals=eigvals,
                           check_finite=False)
         if largest:
@@ -385,6 +397,10 @@ def lobpcg(
     if (residualTolerance is None) or (residualTolerance <= 0.0):
         residualTolerance = np.sqrt(1e-15) * n
 
+    A = _makeOperator(A, (n, n))
+    B = _makeOperator(B, (n, n))
+    M = _makeOperator(M, (n, n))
+
     # Apply constraints to X.
     if blockVectorY is not None:
 


Test patch:
diff --git a/scipy/sparse/linalg/_eigen/lobpcg/tests/test_lobpcg.py b/scipy/sparse/linalg/_eigen/lobpcg/tests/test_lobpcg.py
--- a/scipy/sparse/linalg/_eigen/lobpcg/tests/test_lobpcg.py
+++ b/scipy/sparse/linalg/_eigen/lobpcg/tests/test_lobpcg.py
@@ -1,5 +1,6 @@
 """ Test functions for the sparse.linalg._eigen.lobpcg module
 """
+
 import itertools
 import platform
 import sys
@@ -13,7 +14,7 @@
 
 from numpy import ones, r_, diag
 from scipy.linalg import eig, eigh, toeplitz, orth
-from scipy.sparse import spdiags, diags, eye
+from scipy.sparse import spdiags, diags, eye, csr_matrix
 from scipy.sparse.linalg import eigs, LinearOperator
 from scipy.sparse.linalg._eigen.lobpcg import lobpcg
 
@@ -115,36 +116,72 @@ def test_regression():
     assert_allclose(w, [1])
 
 
-def test_diagonal():
-    """Check for diagonal matrices.
+@pytest.mark.filterwarnings("ignore:The problem size")
+@pytest.mark.parametrize('n, m, m_excluded', [(100, 4, 3), (4, 2, 0)])
+def test_diagonal(n, m, m_excluded):
+    """Test ``m - m_excluded`` eigenvalues and eigenvectors of
+    diagonal matrices of the size ``n`` varying matrix formats:
+    dense array, spare matrix, and ``LinearOperator`` for both
+    matrixes in the generalized eigenvalue problem ``Av = cBv``
+    and for the preconditioner.
     """
     rnd = np.random.RandomState(0)
-    n = 100
-    m = 4
 
     # Define the generalized eigenvalue problem Av = cBv
     # where (c, v) is a generalized eigenpair,
     # and where we choose A to be the diagonal matrix whose entries are 1..n
     # and where B is chosen to be the identity matrix.
     vals = np.arange(1, n+1, dtype=float)
-    A = diags([vals], [0], (n, n))
-    B = eye(n)
+    A_s = diags([vals], [0], (n, n))
+    A_a = A_s.toarray()
+
+    def A_f(x):
+        return A_s @ x
+
+    A_lo = LinearOperator(matvec=A_f,
+                          matmat=A_f,
+                          shape=(n, n), dtype=float)
+
+    B_a = eye(n)
+    B_s = csr_matrix(B_a)
+
+    def B_f(x):
+        return B_a @ x
+
+    B_lo = LinearOperator(matvec=B_f,
+                          matmat=B_f,
+                          shape=(n, n), dtype=float)
 
     # Let the preconditioner M be the inverse of A.
-    M = diags([1./vals], [0], (n, n))
+    M_s = diags([1./vals], [0], (n, n))
+    M_a = M_s.toarray()
+
+    def M_f(x):
+        return M_s @ x
+
+    M_lo = LinearOperator(matvec=M_f,
+                          matmat=M_f,
+                          shape=(n, n), dtype=float)
 
     # Pick random initial vectors.
-    X = rnd.random((n, m))
+    X = rnd.normal(size=(n, m))
 
     # Require that the returned eigenvectors be in the orthogonal complement
     # of the first few standard basis vectors.
-    m_excluded = 3
-    Y = np.eye(n, m_excluded)
+    if m_excluded > 0:
+        Y = np.eye(n, m_excluded)
+    else:
+        Y = None
 
-    eigvals, vecs = lobpcg(A, X, B, M=M, Y=Y, tol=1e-4, maxiter=40, largest=False)
+    for A in [A_a, A_s, A_lo]:
+        for B in [B_a, B_s, B_lo]:
+            for M in [M_a, M_s, M_lo]:
+                eigvals, vecs = lobpcg(A, X, B, M=M, Y=Y,
+                                       maxiter=40, largest=False)
 
-    assert_allclose(eigvals, np.arange(1+m_excluded, 1+m_excluded+m))
-    _check_eigen(A, eigvals, vecs, rtol=1e-3, atol=1e-3)
+                assert_allclose(eigvals, np.arange(1+m_excluded,
+                                                   1+m_excluded+m))
+                _check_eigen(A, eigvals, vecs, rtol=1e-3, atol=1e-3)
 
 
 def _check_eigen(M, w, V, rtol=1e-8, atol=1e-14):
@@ -222,7 +259,7 @@ def test_fiedler_large_12():
 def test_failure_to_run_iterations():
     """Check that the code exists gracefully without breaking. Issue #10974.
     """
-    rnd = np.random.RandomState(4120349)
+    rnd = np.random.RandomState(0)
     X = rnd.standard_normal((100, 10))
     A = X @ X.T
     Q = rnd.standard_normal((X.shape[0], 4))
@@ -386,29 +423,33 @@ def test_diagonal_data_types():
         def Ms64precond(x):
             return Ms64 @ x
         Ms64precondLO = LinearOperator(matvec=Ms64precond,
-                                    matmat=Ms64precond,
-                                    shape=(n, n), dtype=float)
+                                       matmat=Ms64precond,
+                                       shape=(n, n),
+                                       dtype=float)
         Mf64 = Ms64.toarray()
 
         def Mf64precond(x):
             return Mf64 @ x
         Mf64precondLO = LinearOperator(matvec=Mf64precond,
-                                    matmat=Mf64precond,
-                                    shape=(n, n), dtype=float)
+                                       matmat=Mf64precond,
+                                       shape=(n, n),
+                                       dtype=float)
         Ms32 = Ms64.astype(np.float32)
 
         def Ms32precond(x):
             return Ms32 @ x
         Ms32precondLO = LinearOperator(matvec=Ms32precond,
-                                    matmat=Ms32precond,
-                                    shape=(n, n), dtype=np.float32)
+                                       matmat=Ms32precond,
+                                       shape=(n, n),
+                                       dtype=np.float32)
         Mf32 = Ms32.toarray()
 
         def Mf32precond(x):
             return Mf32 @ x
         Mf32precondLO = LinearOperator(matvec=Mf32precond,
-                                    matmat=Mf32precond,
-                                    shape=(n, n), dtype=np.float32)
+                                       matmat=Mf32precond,
+                                       shape=(n, n),
+                                       dtype=np.float32)
         listM = [None, Ms64precondLO, Mf64precondLO,
                  Ms32precondLO, Mf32precondLO]