Repository: scipy/scipy

Issue:
Non explicit error message in lobpcg
<!-- 
Thank you for taking the time to file a bug report. 
Please fill in the fields below, deleting the sections that 
don't apply to your issue. You can view the final output
by clicking the preview button above.
Note: This is a comment, and won't appear in the output.
-->

In scikit-learn, we are benchmarking the integration of `lobpcg` to compute some `svd`. On an ill-posed problem, I got a weird error about array being `None` instead of a `LinAlgError`.

The issue is coming from the following line:

https://github.com/scipy/scipy/blob/91f43942cfcfb63c4b2ca58a55562295a7b00cde/scipy/sparse/linalg/eigen/lobpcg/lobpcg.py#L105-L108

The `LinAlgError` is catched. Then the arrays are set to `None` for some reason but this state will not allow to resolve any issue later and just failed with a more cryptic error (see traceback below).
I was wondering if it was really intended to not fail early?

#### Reproducing code example:
<!-- 
If you place your code between the triple backticks below, 
it will be rendered as a code block. 
-->

Here, the matrix `X` should be transposed to be used in `lobpcg` and it is the reason for having an ill-posed problem which will raise an error.

```python
import numpy as np
from scipy import sparse
from scipy.sparse.linalg import lobpcg, aslinearoperator, LinearOperator

rng = np.random.RandomState(42)
X = sparse.random(32561, 123, density=0.4)

MLO = aslinearoperator(X)

def _matvec(V):
    return MLO(MLO.H(V))

A = LinearOperator(
    dtype=X.dtype, shape=(X.shape[0], X.shape[0]), matvec=_matvec,
    matmat=_matvec
)

n_components = 60
Q = rng.normal(size=(X.shape[0], 60))

lobpcg(A, Q, maxiter=10, verbosityLevel=0)
```

#### Error message:
<!-- If any, paste the *full* error message inside a code block
as above (starting from line Traceback)
-->

```pytb
Traceback (most recent call last):
  File "tmp.py", line 25, in <module>
    lobpcg(A, Q, maxiter=10, verbosityLevel=0)
  File "/home/lemaitre/miniconda3/envs/scipy-dev/lib/python3.7/site-packages/scipy/sparse/linalg/eigen/lobpcg/lobpcg.py", line 489, in lobpcg
    activeBlockVectorAR = A(activeBlockVectorR)
  File "/home/lemaitre/miniconda3/envs/scipy-dev/lib/python3.7/site-packages/scipy/sparse/linalg/interface.py", line 387, in __call__
    return self*x
  File "/home/lemaitre/miniconda3/envs/scipy-dev/lib/python3.7/site-packages/scipy/sparse/linalg/interface.py", line 390, in __mul__
    return self.dot(x)
  File "/home/lemaitre/miniconda3/envs/scipy-dev/lib/python3.7/site-packages/scipy/sparse/linalg/interface.py", line 420, in dot
    % x)
ValueError: expected 1-d or 2-d array or matrix, got array(None, dtype=object)
```

I was expected something like:

```pytb
LinAlgError: 49-th leading minor of the array is not positive definite
```

#### Scipy/Numpy/Python version information:
<!-- You can simply run the following and paste the result in a code block
```
import sys, scipy, numpy; print(scipy.__version__, numpy.__version__, sys.version_info)
```
-->

```python
>>> import sys, scipy, numpy; print(scipy.__version__, numpy.__version__, sys.version_info)
1.4.0.dev0+f3d6121 1.17.3 sys.version_info(major=3, minor=7, micro=4, releaselevel='final', serial=0)
```


Discussion:
@glemaitre Yes, it was really intended in scipy/scipy/sparse/linalg/eigen/lobpcg/lobpcg.py Lines 105 to 108 in 91f4394 not not fail early. 

It appears that the error is coming from LinearOperator, evaluating
`A(activeBlockVectorR)`  possibly because  `activeBlockVectorR` is `None`. I'll try to find time to double-check the code logic - why it gets to line 489 in lobpcg if `activeBlockVectorR` is indeed `None`. - Thanks for reporting!

Having said that, let me comment that your script is not using lobpcg well for svd: 
size(X) is 32561x123 so to do SVD(X) the easy way one should use the normal matrix 123x123, which is MLO.H(MLO(V)) where V.shape[0]=123 , not your MLO(MLO.H(V)) which is size 32561x32561, and has all (but 123) zero eigenvalues. 
> Having said that, let me comment that your script is not using lobpcg well for svd:
size(X) is 32561x123 so to do SVD(X) the easy way one should use the normal matrix 123x123, which is MLO.H(MLO(V)) where V.shape[0]=123 , not your MLO(MLO.H(V)) which is size 32561x32561, and has all (but 123) zero eigenvalues.

Yes that's why I am expecting an error to be raised. 
> > Having said that, let me comment that your script is not using lobpcg well for svd:
> > size(X) is 32561x123 so to do SVD(X) the easy way one should use the normal matrix 123x123, which is MLO.H(MLO(V)) where V.shape[0]=123 , not your MLO(MLO.H(V)) which is size 32561x32561, and has all (but 123) zero eigenvalues.
> 
> Yes that's why I am expecting an error to be raised.

Well, it does give an error :-)

However, lobpcg should still be expected to run on this example without errors, in my opinion, just given a symmetric matrix even with mostly zero eigenvalues. 
OK, I get it and I have found a simple example with the same issue: 
```
import numpy as np
from scipy import sparse
from scipy.sparse.linalg import lobpcg, aslinearoperator, LinearOperator

X = np.random.randn(100, 10)

def _matvec(V):
    return X @ (X.T @ V)

A = LinearOperator(
    dtype=X.dtype, shape=(X.shape[0], X.shape[0]), matvec=_matvec,
    matmat=_matvec)

Q = np.random.randn(X.shape[0], 4)
lobpcg(A, Q, maxiter=20, verbosityLevel=1)
```
always fails, while with `Q = np.random.randn(X.shape[0], 3)` it always works. The operator `A` of the eigenvalue problem is 100x100 and has only 10 nonzero eigenvalues. LOBPCG runs in the Krylov space generated by `A` that cannot thus have more than 10 dimensions. LOBPCG uses 3-times-block_size subspace to operate  and runs out of space when block_size = 4, since 3-times-4 = 12 > 10, on Iteration 2. The iterative vectors become linearly dependent, so the orthogonalization fails, activating Lines 105 to 108 in scipy/scipy/sparse/linalg/eigen/lobpcg/lobpcg.py :
```
 except LinAlgError: 
     #raise ValueError('Cholesky has failed') 
     blockVectorV = None 
     blockVectorBV = None 
```
Let me think how to handle it best... @glemaitre - thanks for finding and reporting such a case!
I am using scikit-learn spectral clustering for my clustering problem. I use the following configuration for the spectral clustering

`clustering = sklearn.cluster.SpectralClustering(n_clusters = number_clusters , affinity="cosine",assign_labels="clusterQR",eigen_solver='lobpcg',n_jobs=psutil.cpu_count()).fit(embedding_matrix)`

but I get the error 

```
File "/data/fatemeh/mem2Vec/kym_meme/scikit-learn-clusterQR/sklearn/cluster/_spectral.py", line 559, in fit
    assign_labels=self.assign_labels)
  File "/data/fatemeh/mem2Vec/kym_meme/scikit-learn-clusterQR/sklearn/cluster/_spectral.py", line 301, in spectral_clustering
    eigen_tol=eigen_tol, drop_first=False)
  File "/data/fatemeh/mem2Vec/kym_meme/scikit-learn-clusterQR/sklearn/manifold/_spectral_embedding.py", line 339, in spectral_embedding
    largest=False, maxiter=2000)
  File "/home/ftahmas/venv/lib/python3.6/site-packages/scipy/sparse/linalg/eigen/lobpcg/lobpcg.py", line 489, in lobpcg
    activeBlockVectorAR = A(activeBlockVectorR)
  File "/home/ftahmas/venv/lib/python3.6/site-packages/scipy/sparse/linalg/interface.py", line 387, in __call__
    return self*x
  File "/home/ftahmas/venv/lib/python3.6/site-packages/scipy/sparse/linalg/interface.py", line 390, in __mul__
    return self.dot(x)
  File "/home/ftahmas/venv/lib/python3.6/site-packages/scipy/sparse/linalg/interface.py", line 420, in dot
    % x)
ValueError: expected 1-d or 2-d array or matrix, got array(None, `dtype=object)`
```
when I use `affinity="rbf"` it works  without error! 

any idea why?
@FTB-B  I could investigate if you provide a reproducible example, please.

The issue should not be related to clusterqr, so please run with a different already available in scikit-learn function for labeling and submit a formal bug report with a ping to me.
I ran into the same problem as @glemaitre.

I have a (504, 504) `np.float64` matrix A, I wish to compute a few singular values and singular vectors. `scipy.linalg.svd` and `scipy.linalg.svds(solver='arpack')` work well, however `scipy.linalg.svds(solver='lobpcg')` crashes with a very confusing error message. I can share the matrix if requested, github does not let me upload a `npy` file.

First, I think the line 105 `raise ValueError('Cholesky has failed')` should be uncommented to make the error message explicit (the error also comes from there).

Concerning the fix, my matrix has 78 non-zero singular values, but it crashes for `k=5`. Surprisingly, `lobpcg` crashes only for _some_ values of `k`, worse when it does not crash the result is wrong. Comp
```python
import numpy as np
import scipy.linalg as lg
import scipy.sparse.linalg as slg

A = np.load("data_crash.npy")
u0, s0, v0 = lg.svd(A)
x = lg.norm(A)
for k in range(1, 78):
    try:    
         u1, s1, v1 = slg.svds(A, k=k, solver="lobpcg")
         u2, s2, v2 = slg.svds(A, k=k, solver="arpack")
         print(k, "success")
         print("dense  truncation", lg.norm(u0[:,:k] * s0[:k] @ v0[:k] - A) / x) 
         print("arpack truncation", lg.norm(u2 * s2 @ v2 - A) / x) 
         print("lobpcg truncation", lg.norm(u1 * s1 @ v1 - A) / x) 
    except ValueError as err: 
         print(k, err)
```
gives me

```
1 success
dense  truncation 0.37410744042017613
arpack truncation 0.37410744042017613
lobpcg truncation 0.37410744042017613
2 success
dense  truncation 0.002749949291553053
arpack truncation 0.0027499492915530523
lobpcg truncation 1.2256989944723657
3 success
dense  truncation 0.0020247195761314294
arpack truncation 0.002024719576131428
lobpcg truncation 5.526570759513649
4 success
dense  truncation 0.0012373784439977066
arpack truncation 0.0012373784439977053
lobpcg truncation 0.06735836840347356
5 expected 1-d or 2-d array or matrix, got array(None, dtype=object)
6 success
dense  truncation 0.0008082565817324968
arpack truncation 0.0008082565817324936
lobpcg truncation 0.0009858440032944039
7 expected 1-d or 2-d array or matrix, got array(None, dtype=object)
8 success
dense  truncation 0.00044906990420738645
arpack truncation 0.0004490699042073811
lobpcg truncation 3.7862403491418792
9 expected 1-d or 2-d array or matrix, got array(None, dtype=object)
10 expected 1-d or 2-d array or matrix, got array(None, dtype=object)
11 expected 1-d or 2-d array or matrix, got array(None, dtype=object)
12 expected 1-d or 2-d array or matrix, got array(None, dtype=object)
13 expected 1-d or 2-d array or matrix, got array(None, dtype=object)
14 expected 1-d or 2-d array or matrix, got array(None, dtype=object)
15 expected 1-d or 2-d array or matrix, got array(None, dtype=object)
16 expected 1-d or 2-d array or matrix, got array(None, dtype=object)
17 expected 1-d or 2-d array or matrix, got array(None, dtype=object)
18 expected 1-d or 2-d array or matrix, got array(None, dtype=object)
19 expected 1-d or 2-d array or matrix, got array(None, dtype=object)
20 expected 1-d or 2-d array or matrix, got array(None, dtype=object)
21 expected 1-d or 2-d array or matrix, got array(None, dtype=object)
22 expected 1-d or 2-d array or matrix, got array(None, dtype=object)
23 expected 1-d or 2-d array or matrix, got array(None, dtype=object)
24 expected 1-d or 2-d array or matrix, got array(None, dtype=object)
25 expected 1-d or 2-d array or matrix, got array(None, dtype=object)
26 expected 1-d or 2-d array or matrix, got array(None, dtype=object)
27 expected 1-d or 2-d array or matrix, got array(None, dtype=object)
28 expected 1-d or 2-d array or matrix, got array(None, dtype=object)
29 expected 1-d or 2-d array or matrix, got array(None, dtype=object)
30 expected 1-d or 2-d array or matrix, got array(None, dtype=object)
31 expected 1-d or 2-d array or matrix, got array(None, dtype=object)
32 expected 1-d or 2-d array or matrix, got array(None, dtype=object)
33 expected 1-d or 2-d array or matrix, got array(None, dtype=object)
34 expected 1-d or 2-d array or matrix, got array(None, dtype=object)
35 expected 1-d or 2-d array or matrix, got array(None, dtype=object)
36 expected 1-d or 2-d array or matrix, got array(None, dtype=object)
37 expected 1-d or 2-d array or matrix, got array(None, dtype=object)
38 expected 1-d or 2-d array or matrix, got array(None, dtype=object)
39 expected 1-d or 2-d array or matrix, got array(None, dtype=object)
40 expected 1-d or 2-d array or matrix, got array(None, dtype=object)
41 expected 1-d or 2-d array or matrix, got array(None, dtype=object)
42 expected 1-d or 2-d array or matrix, got array(None, dtype=object)
43 expected 1-d or 2-d array or matrix, got array(None, dtype=object)
44 expected 1-d or 2-d array or matrix, got array(None, dtype=object)
45 expected 1-d or 2-d array or matrix, got array(None, dtype=object)
46 expected 1-d or 2-d array or matrix, got array(None, dtype=object)
47 expected 1-d or 2-d array or matrix, got array(None, dtype=object)
48 expected 1-d or 2-d array or matrix, got array(None, dtype=object)
49 expected 1-d or 2-d array or matrix, got array(None, dtype=object)
50 expected 1-d or 2-d array or matrix, got array(None, dtype=object)
51 expected 1-d or 2-d array or matrix, got array(None, dtype=object)
52 expected 1-d or 2-d array or matrix, got array(None, dtype=object)
53 expected 1-d or 2-d array or matrix, got array(None, dtype=object)
54 expected 1-d or 2-d array or matrix, got array(None, dtype=object)
55 expected 1-d or 2-d array or matrix, got array(None, dtype=object)
56 expected 1-d or 2-d array or matrix, got array(None, dtype=object)
57 expected 1-d or 2-d array or matrix, got array(None, dtype=object)
58 expected 1-d or 2-d array or matrix, got array(None, dtype=object)
59 expected 1-d or 2-d array or matrix, got array(None, dtype=object)
60 expected 1-d or 2-d array or matrix, got array(None, dtype=object)
61 expected 1-d or 2-d array or matrix, got array(None, dtype=object)
62 expected 1-d or 2-d array or matrix, got array(None, dtype=object)
63 expected 1-d or 2-d array or matrix, got array(None, dtype=object)
64 expected 1-d or 2-d array or matrix, got array(None, dtype=object)
65 expected 1-d or 2-d array or matrix, got array(None, dtype=object)
66 expected 1-d or 2-d array or matrix, got array(None, dtype=object)
67 expected 1-d or 2-d array or matrix, got array(None, dtype=object)
68 expected 1-d or 2-d array or matrix, got array(None, dtype=object)
69 expected 1-d or 2-d array or matrix, got array(None, dtype=object)
70 expected 1-d or 2-d array or matrix, got array(None, dtype=object)
71 expected 1-d or 2-d array or matrix, got array(None, dtype=object)
72 expected 1-d or 2-d array or matrix, got array(None, dtype=object)
73 expected 1-d or 2-d array or matrix, got array(None, dtype=object)
74 expected 1-d or 2-d array or matrix, got array(None, dtype=object)
75 expected 1-d or 2-d array or matrix, got array(None, dtype=object)
76 success
dense  truncation 8.826353786938982e-12
arpack truncation 2.7656568280904688e-06
lobpcg truncation 5.069589637576937
77 expected 1-d or 2-d array or matrix, got array(None, dtype=object)
```
@ogauthe this is a tricky matrix for lobpcg due to so many 0 eigenvalues; see https://github.com/scipy/scipy/issues/10974#issuecomment-605666851

One simple fix could be just adding a small-magnitude random matrix to A to turn exact zeros into small nonzero values.  
Thank you for your answer.

Beyond the error message when the function crashes, I think there is a real problem here. For `k=2`, the result is completely wrong, but this is silent. The matrix `A.T @ A` is symmetric positive by construction and quite small. Neither `svds` nor `lobpcg` documentation mention that the result may be wrong if too many eigenvalues are numerical zeros.

I can open a separate issue for this, a wrong result is much worse than a crash for me.
I have not paid attention to the values, sorry. Wrong values for small k is surely very odd, and unrelated to many zeros. 

Please submit a separate bug report with a ping to me and upload the matrix to reproduce the problem. 

Resolving patch:
diff --git a/scipy/sparse/linalg/eigen/lobpcg/lobpcg.py b/scipy/sparse/linalg/eigen/lobpcg/lobpcg.py
--- a/scipy/sparse/linalg/eigen/lobpcg/lobpcg.py
+++ b/scipy/sparse/linalg/eigen/lobpcg/lobpcg.py
@@ -17,13 +17,14 @@
        https://github.com/lobpcg/blopex
 """
 
+import warnings
 import numpy as np
-from scipy.linalg import (inv, eigh, cho_factor, cho_solve, cholesky,
-                          LinAlgError)
+from scipy.linalg import (inv, eigh, cho_factor, cho_solve,
+                          cholesky, LinAlgError)
 from scipy.sparse.linalg import aslinearoperator
 from numpy import block as bmat
 
-__all__ = ['lobpcg']
+__all__ = ["lobpcg"]
 
 
 def _report_nonhermitian(M, name):
@@ -38,9 +39,10 @@ def _report_nonhermitian(M, name):
     tol = 10 * np.finfo(M.dtype).eps
     tol = max(tol, tol * norm(M, 1))
     if nmd > tol:
-        print('matrix %s of the type %s is not sufficiently Hermitian:'
-              % (name, M.dtype))
-        print('condition: %.e < %e' % (nmd, tol))
+        print(
+            "matrix %s of the type %s is not Hermitian:" % (name, M.dtype)
+        )
+        print("condition: %.e < %e" % (nmd, tol))
 
 
 def _as2d(ar):
@@ -66,7 +68,7 @@ def _makeOperator(operatorInput, expectedShape):
         operator = aslinearoperator(operatorInput)
 
     if operator.shape != expectedShape:
-        raise ValueError('operator has invalid shape')
+        raise ValueError("operator has invalid shape")
 
     return operator
 
@@ -80,7 +82,7 @@ def _applyConstraints(blockVectorV, factYBY, blockVectorBY, blockVectorY):
 
 def _b_orthonormalize(B, blockVectorV, blockVectorBV=None, retInvR=False):
     """B-orthonormalize the given block vector using Cholesky."""
-    normalization = blockVectorV.max(axis=0)+np.finfo(blockVectorV.dtype).eps
+    normalization = blockVectorV.max(axis=0) + np.finfo(blockVectorV.dtype).eps
     blockVectorV = blockVectorV / normalization
     if blockVectorBV is None:
         if B is not None:
@@ -89,20 +91,20 @@ def _b_orthonormalize(B, blockVectorV, blockVectorBV=None, retInvR=False):
             blockVectorBV = blockVectorV  # Shared data!!!
     else:
         blockVectorBV = blockVectorBV / normalization
-    VBV = np.matmul(blockVectorV.T.conj(), blockVectorBV)
+    VBV = blockVectorV.T.conj() @ blockVectorBV
     try:
         # VBV is a Cholesky factor from now on...
         VBV = cholesky(VBV, overwrite_a=True)
         VBV = inv(VBV, overwrite_a=True)
-        blockVectorV = np.matmul(blockVectorV, VBV)
+        blockVectorV = blockVectorV @ VBV
         # blockVectorV = (cho_solve((VBV.T, True), blockVectorV.T)).T
         if B is not None:
-            blockVectorBV = np.matmul(blockVectorBV, VBV)
+            blockVectorBV = blockVectorBV @ VBV
             # blockVectorBV = (cho_solve((VBV.T, True), blockVectorBV.T)).T
         else:
             blockVectorBV = None
     except LinAlgError:
-        #raise ValueError('Cholesky has failed')
+        # raise ValueError('Cholesky has failed')
         blockVectorV = None
         blockVectorBV = None
         VBV = None
@@ -117,18 +119,26 @@ def _get_indx(_lambda, num, largest):
     """Get `num` indices into `_lambda` depending on `largest` option."""
     ii = np.argsort(_lambda)
     if largest:
-        ii = ii[:-num-1:-1]
+        ii = ii[:-num - 1:-1]
     else:
         ii = ii[:num]
 
     return ii
 
 
-def lobpcg(A, X,
-           B=None, M=None, Y=None,
-           tol=None, maxiter=None,
-           largest=True, verbosityLevel=0,
-           retLambdaHistory=False, retResidualNormsHistory=False):
+def lobpcg(
+    A,
+    X,
+    B=None,
+    M=None,
+    Y=None,
+    tol=None,
+    maxiter=None,
+    largest=True,
+    verbosityLevel=0,
+    retLambdaHistory=False,
+    retResidualNormsHistory=False,
+):
     """Locally Optimal Block Preconditioned Conjugate Gradient Method (LOBPCG)
 
     LOBPCG is a preconditioned eigensolver for large symmetric positive
@@ -302,7 +312,7 @@ def lobpcg(A, X,
 
     # Block size.
     if len(blockVectorX.shape) != 2:
-        raise ValueError('expected rank-2 array for argument X')
+        raise ValueError("expected rank-2 array for argument X")
 
     n, sizeX = blockVectorX.shape
 
@@ -338,19 +348,22 @@ def lobpcg(A, X,
         sizeX = min(sizeX, n)
 
         if blockVectorY is not None:
-            raise NotImplementedError('The dense eigensolver '
-                                      'does not support constraints.')
+            raise NotImplementedError(
+                "The dense eigensolver " "does not support constraints."
+            )
 
         # Define the closed range of indices of eigenvalues to return.
         if largest:
-            eigvals = (n - sizeX, n-1)
+            eigvals = (n - sizeX, n - 1)
         else:
-            eigvals = (0, sizeX-1)
+            eigvals = (0, sizeX - 1)
 
         A_dense = A(np.eye(n, dtype=A.dtype))
         B_dense = None if B is None else B(np.eye(n, dtype=B.dtype))
 
-        vals, vecs = eigh(A_dense, B_dense, eigvals=eigvals,
+        vals, vecs = eigh(A_dense,
+                          B_dense,
+                          eigvals=eigvals,
                           check_finite=False)
         if largest:
             # Reverse order to be compatible with eigs() in 'LM' mode.
@@ -376,7 +389,7 @@ def lobpcg(A, X,
             # gramYBY is a Cholesky factor from now on...
             gramYBY = cho_factor(gramYBY)
         except LinAlgError as e:
-            raise ValueError('cannot handle linearly dependent constraints') from e
+            raise ValueError("linearly dependent constraints") from e
 
         _applyConstraints(blockVectorX, gramYBY, blockVectorBY, blockVectorY)
 
@@ -423,7 +436,7 @@ def lobpcg(A, X,
     while iterationNumber < maxiter:
         iterationNumber += 1
         if verbosityLevel > 0:
-            print('iteration %d' % iterationNumber)
+            print("iteration %d" % iterationNumber)
 
         if B is not None:
             aux = blockVectorBX * _lambda[np.newaxis, :]
@@ -451,9 +464,9 @@ def lobpcg(A, X,
             break
 
         if verbosityLevel > 0:
-            print('current block size:', currentBlockSize)
-            print('eigenvalue:', _lambda)
-            print('residual norms:', residualNorms)
+            print("current block size:", currentBlockSize)
+            print("eigenvalue:", _lambda)
+            print("residual norms:", residualNorms)
         if verbosityLevel > 10:
             print(eigBlockVector)
 
@@ -473,30 +486,42 @@ def lobpcg(A, X,
         # Apply constraints to the preconditioned residuals.
         if blockVectorY is not None:
             _applyConstraints(activeBlockVectorR,
-                              gramYBY, blockVectorBY, blockVectorY)
+                              gramYBY,
+                              blockVectorBY,
+                              blockVectorY)
 
         ##
         # B-orthogonalize the preconditioned residuals to X.
         if B is not None:
-            activeBlockVectorR = activeBlockVectorR - np.matmul(blockVectorX,
-                                 np.matmul(blockVectorBX.T.conj(),
-                                 activeBlockVectorR))
+            activeBlockVectorR = activeBlockVectorR - (
+                blockVectorX @
+                (blockVectorBX.T.conj() @ activeBlockVectorR)
+            )
         else:
-            activeBlockVectorR = activeBlockVectorR - np.matmul(blockVectorX,
-                                 np.matmul(blockVectorX.T.conj(),
-                                 activeBlockVectorR))
+            activeBlockVectorR = activeBlockVectorR - (
+                blockVectorX @
+                (blockVectorX.T.conj() @ activeBlockVectorR)
+            )
 
         ##
         # B-orthonormalize the preconditioned residuals.
         aux = _b_orthonormalize(B, activeBlockVectorR)
         activeBlockVectorR, activeBlockVectorBR = aux
 
+        if activeBlockVectorR is None:
+            warnings.warn(
+                f"Iteration {iterationNumber} failed at tolerance "
+                f"{np.max(residualNorms)} not reaching {residualTolerance}.",
+                UserWarning, stacklevel=3
+            )
+            break
         activeBlockVectorAR = A(activeBlockVectorR)
 
         if iterationNumber > 0:
             if B is not None:
-                aux = _b_orthonormalize(B, activeBlockVectorP,
-                                        activeBlockVectorBP, retInvR=True)
+                aux = _b_orthonormalize(
+                    B, activeBlockVectorP, activeBlockVectorBP, retInvR=True
+                )
                 activeBlockVectorP, activeBlockVectorBP, invR, normal = aux
             else:
                 aux = _b_orthonormalize(B, activeBlockVectorP, retInvR=True)
@@ -513,9 +538,9 @@ def lobpcg(A, X,
         # Perform the Rayleigh Ritz Procedure:
         # Compute symmetric Gram matrices:
 
-        if activeBlockVectorAR.dtype == 'float32':
+        if activeBlockVectorAR.dtype == "float32":
             myeps = 1
-        elif activeBlockVectorR.dtype == 'float32':
+        elif activeBlockVectorR.dtype == "float32":
             myeps = 1e-4
         else:
             myeps = 1e-8
@@ -538,9 +563,9 @@ def lobpcg(A, X,
         gramRAR = np.dot(activeBlockVectorR.T.conj(), activeBlockVectorAR)
 
         if explicitGramFlag:
-            gramRAR = (gramRAR + gramRAR.T.conj())/2
+            gramRAR = (gramRAR + gramRAR.T.conj()) / 2
             gramXAX = np.dot(blockVectorX.T.conj(), blockVectorAX)
-            gramXAX = (gramXAX + gramXAX.T.conj())/2
+            gramXAX = (gramXAX + gramXAX.T.conj()) / 2
             gramXBX = np.dot(blockVectorX.T.conj(), blockVectorBX)
             gramRBR = np.dot(activeBlockVectorR.T.conj(), activeBlockVectorBR)
             gramXBR = np.dot(blockVectorX.T.conj(), activeBlockVectorBR)
@@ -552,12 +577,12 @@ def lobpcg(A, X,
 
         def _handle_gramA_gramB_verbosity(gramA, gramB):
             if verbosityLevel > 0:
-                _report_nonhermitian(gramA, 'gramA')
-                _report_nonhermitian(gramB, 'gramB')
+                _report_nonhermitian(gramA, "gramA")
+                _report_nonhermitian(gramB, "gramB")
             if verbosityLevel > 10:
                 # Note: not documented, but leave it in here for now
-                np.savetxt('gramA.txt', gramA)
-                np.savetxt('gramB.txt', gramB)
+                np.savetxt("gramA.txt", gramA)
+                np.savetxt("gramB.txt", gramB)
 
         if not restart:
             gramXAP = np.dot(blockVectorX.T.conj(), activeBlockVectorAP)
@@ -566,41 +591,49 @@ def _handle_gramA_gramB_verbosity(gramA, gramB):
             gramXBP = np.dot(blockVectorX.T.conj(), activeBlockVectorBP)
             gramRBP = np.dot(activeBlockVectorR.T.conj(), activeBlockVectorBP)
             if explicitGramFlag:
-                gramPAP = (gramPAP + gramPAP.T.conj())/2
+                gramPAP = (gramPAP + gramPAP.T.conj()) / 2
                 gramPBP = np.dot(activeBlockVectorP.T.conj(),
                                  activeBlockVectorBP)
             else:
                 gramPBP = ident
 
-            gramA = bmat([[gramXAX, gramXAR, gramXAP],
-                          [gramXAR.T.conj(), gramRAR, gramRAP],
-                          [gramXAP.T.conj(), gramRAP.T.conj(), gramPAP]])
-            gramB = bmat([[gramXBX, gramXBR, gramXBP],
-                          [gramXBR.T.conj(), gramRBR, gramRBP],
-                          [gramXBP.T.conj(), gramRBP.T.conj(), gramPBP]])
+            gramA = bmat(
+                [
+                    [gramXAX, gramXAR, gramXAP],
+                    [gramXAR.T.conj(), gramRAR, gramRAP],
+                    [gramXAP.T.conj(), gramRAP.T.conj(), gramPAP],
+                ]
+            )
+            gramB = bmat(
+                [
+                    [gramXBX, gramXBR, gramXBP],
+                    [gramXBR.T.conj(), gramRBR, gramRBP],
+                    [gramXBP.T.conj(), gramRBP.T.conj(), gramPBP],
+                ]
+            )
 
             _handle_gramA_gramB_verbosity(gramA, gramB)
 
             try:
-                _lambda, eigBlockVector = eigh(gramA, gramB,
+                _lambda, eigBlockVector = eigh(gramA,
+                                               gramB,
                                                check_finite=False)
             except LinAlgError:
                 # try again after dropping the direction vectors P from RR
                 restart = True
 
         if restart:
-            gramA = bmat([[gramXAX, gramXAR],
-                          [gramXAR.T.conj(), gramRAR]])
-            gramB = bmat([[gramXBX, gramXBR],
-                          [gramXBR.T.conj(), gramRBR]])
+            gramA = bmat([[gramXAX, gramXAR], [gramXAR.T.conj(), gramRAR]])
+            gramB = bmat([[gramXBX, gramXBR], [gramXBR.T.conj(), gramRBR]])
 
             _handle_gramA_gramB_verbosity(gramA, gramB)
 
             try:
-                _lambda, eigBlockVector = eigh(gramA, gramB,
+                _lambda, eigBlockVector = eigh(gramA,
+                                               gramB,
                                                check_finite=False)
             except LinAlgError as e:
-                raise ValueError('eigh has failed in lobpcg iterations') from e
+                raise ValueError("eigh has failed in lobpcg iterations") from e
 
         ii = _get_indx(_lambda, sizeX, largest)
         if verbosityLevel > 10:
@@ -613,12 +646,12 @@ def _handle_gramA_gramB_verbosity(gramA, gramB):
         lambdaHistory.append(_lambda)
 
         if verbosityLevel > 10:
-            print('lambda:', _lambda)
-#         # Normalize eigenvectors!
-#         aux = np.sum( eigBlockVector.conj() * eigBlockVector, 0 )
-#         eigVecNorms = np.sqrt( aux )
-#         eigBlockVector = eigBlockVector / eigVecNorms[np.newaxis, :]
-#         eigBlockVector, aux = _b_orthonormalize( B, eigBlockVector )
+            print("lambda:", _lambda)
+        #         # Normalize eigenvectors!
+        #         aux = np.sum( eigBlockVector.conj() * eigBlockVector, 0 )
+        #         eigVecNorms = np.sqrt( aux )
+        #         eigBlockVector = eigBlockVector / eigVecNorms[np.newaxis, :]
+        #         eigBlockVector, aux = _b_orthonormalize( B, eigBlockVector )
 
         if verbosityLevel > 10:
             print(eigBlockVector)
@@ -627,8 +660,9 @@ def _handle_gramA_gramB_verbosity(gramA, gramB):
         if B is not None:
             if not restart:
                 eigBlockVectorX = eigBlockVector[:sizeX]
-                eigBlockVectorR = eigBlockVector[sizeX:sizeX+currentBlockSize]
-                eigBlockVectorP = eigBlockVector[sizeX+currentBlockSize:]
+                eigBlockVectorR = eigBlockVector[sizeX:
+                                                 sizeX + currentBlockSize]
+                eigBlockVectorP = eigBlockVector[sizeX + currentBlockSize:]
 
                 pp = np.dot(activeBlockVectorR, eigBlockVectorR)
                 pp += np.dot(activeBlockVectorP, eigBlockVectorP)
@@ -660,8 +694,9 @@ def _handle_gramA_gramB_verbosity(gramA, gramB):
         else:
             if not restart:
                 eigBlockVectorX = eigBlockVector[:sizeX]
-                eigBlockVectorR = eigBlockVector[sizeX:sizeX+currentBlockSize]
-                eigBlockVectorP = eigBlockVector[sizeX+currentBlockSize:]
+                eigBlockVectorR = eigBlockVector[sizeX:
+                                                 sizeX + currentBlockSize]
+                eigBlockVectorP = eigBlockVector[sizeX + currentBlockSize:]
 
                 pp = np.dot(activeBlockVectorR, eigBlockVectorR)
                 pp += np.dot(activeBlockVectorP, eigBlockVectorP)
@@ -701,8 +736,8 @@ def _handle_gramA_gramB_verbosity(gramA, gramB):
     # Computing the actual true residuals
 
     if verbosityLevel > 0:
-        print('final eigenvalue:', _lambda)
-        print('final residual norms:', residualNorms)
+        print("Final eigenvalue(s):", _lambda)
+        print("Final residual norm(s):", residualNorms)
 
     if retLambdaHistory:
         if retResidualNormsHistory:


Test patch:
diff --git a/scipy/sparse/linalg/eigen/lobpcg/tests/test_lobpcg.py b/scipy/sparse/linalg/eigen/lobpcg/tests/test_lobpcg.py
--- a/scipy/sparse/linalg/eigen/lobpcg/tests/test_lobpcg.py
+++ b/scipy/sparse/linalg/eigen/lobpcg/tests/test_lobpcg.py
@@ -5,7 +5,8 @@
 
 import numpy as np
 from numpy.testing import (assert_almost_equal, assert_equal,
-                           assert_allclose, assert_array_less)
+                           assert_allclose, assert_array_less,
+                           suppress_warnings)
 
 import pytest
 
@@ -201,6 +202,18 @@ def test_fiedler_large_12():
     _check_fiedler(12, 2)
 
 
+def test_failure_to_run_iterations():
+    """Check that the code exists gracefully without breaking. Issue #10974.
+    """
+    X = np.random.randn(100, 10)
+    A = X @ X.T
+    Q = np.random.randn(X.shape[0], 4)
+    with suppress_warnings() as sup:
+        sup.filter(UserWarning, ".*not reaching.*")
+        eigenvalues, _ = lobpcg(A, Q, maxiter=20)
+        assert(np.max(eigenvalues) > 0)
+
+
 def test_hermitian():
     """Check complex-value Hermitian cases.
     """