Error with using torch.lobpcg

username: yzxhd
cooked: <p>I have trouble with using <code>torch.lobpcg</code>  with pytorch 1.9.1 .<br>
If I use the following script:</p>
<pre><code class="lang-auto">torch.manual_seed(0)
a = torch.randn((100, 100), requires_grad=True)
a = a + a.T
torch.lobpcg(a, largest=False)
</code></pre>
<p>It will cause an error:</p>
<pre><code class="lang-auto">---------------------------------------------------------------------------

      1 a = torch.randn((100, 100), requires_grad=True)
      2 a = a + a.T
----&gt; 3 torch.lobpcg(a, largest=False)
      4

~/Software/miniconda3/envs/python38/lib/python3.8/site-packages/torch/_lobpcg.py in lobpcg(A, k, B, X, n, iK, niter, tol, largest, method, tracker, ortho_iparams, ortho_fparams, ortho_bparams)
    518             B_sym = (B + B.transpose(-2, -1)) / 2 if (B is not None) else None
    519
--&gt; 520             return LOBPCGAutogradFunction.apply(
    521                 A_sym, k, B_sym, X, n, iK, niter, tol, largest,
    522                 method, tracker, ortho_iparams, ortho_fparams, ortho_bparams

TypeError: save_for_backward can only save variables, but argument 4 is of type bool
</code></pre>
<p>It seems that I can not use <code>bool</code> for <code>largest</code> (The doc here: <a href="https://pytorch.org/docs/1.9.1/generated/torch.lobpcg.html?highlight=lobpcg" class="inline-onebox" rel="noopener nofollow ugc">torch.lobpcg — PyTorch 1.9.1 documentation</a> seems wrong)</p>
<p>After reading the source code of <code>torch.lobpcg</code>, I changed the code to the following:</p>
<pre><code class="lang-auto">e1, v1 = torch.lobpcg(a, largest=torch.tensor(0))
</code></pre>
<p>Then it works. But I am not sure whether this is correct or not because the results are different from that of <code>torch.eig</code>:</p>
<pre><code class="lang-auto">torch.manual_seed(0)
a = torch.randn((100, 100), requires_grad=True)
a = a + a.T
e1, v1 = torch.lobpcg(a, largest=torch.tensor(0), k=10)
e2, v2 = torch.lobpcg(a, largest=torch.tensor(1), k=10)
e3, v3 = torch.eig(a)
print('largest:')
print('lobpcg: ', e2)
print('eig:    ', torch.sort(e3[:, 0], descending=True)[0][:10])
print('smallest:')
print('lobpcg: ', e1)
print('eig:    ', torch.sort(e3[:, 0])[0][:10])
</code></pre>
<p>The largest eigenvalues are the same but the smallest are not. The results are:</p>
<pre><code class="lang-auto">largest:
lobpcg:  tensor([26.9840, 25.8924, 25.0941, 24.4001, 23.1643, 21.8327, 20.7435, 20.5844,
        19.2607, 18.5788], grad_fn=&lt;LOBPCGAutogradFunctionBackward&gt;)
eig:     tensor([26.9839, 25.8924, 25.0942, 24.4000, 23.1643, 21.8327, 20.7435, 20.5845,
        19.2606, 18.5788], grad_fn=&lt;SliceBackward&gt;)
smallest:
lobpcg:  tensor([-20.1911, -19.0881, -18.0355, -15.9095, -14.1135, -18.3233, -17.1983,
        -15.4659, -17.6157, -15.5931],
       grad_fn=&lt;LOBPCGAutogradFunctionBackward&gt;)
eig:     tensor([-27.2953, -25.4611, -25.2526, -24.5712, -23.7227, -22.9677, -22.6331,
        -22.0488, -20.8266, -20.4495], grad_fn=&lt;SliceBackward&gt;)
</code></pre>
<p>Any advice?</p>

username: tom
cooked: <p>The first is clearly a bug and it seems that the autograd test fogets to pass “largest” (<a href="https://github.com/pytorch/pytorch/blob/99c7a9f09d2a89506c661defb61610ddff8859a1/test/test_autograd.py#L2711" class="inline-onebox" rel="noopener nofollow ugc">pytorch/test_autograd.py at 99c7a9f09d2a89506c661defb61610ddff8859a1 · pytorch/pytorch · GitHub</a>). Your workaround seems legitimate, with <code>torch.tensor(True)</code>, you’d get additional beauty points. It would be nice to file a bug for this.</p>
<p>Edit after the first <img src="https://discuss.pytorch.org/images/emoji/apple/tea.png?v=10" title=":tea:" class="emoji" alt=":tea:"> : With the second observation: The matrix is not spd if it has negative eigenvalues, but the algorithm is only for spd matrices.</p>
<p>Best regards</p>
<p>Thomas</p>