Torch svd grad show all zero, when only use Vt[-1]

username: yangtaodummt
cooked: <p>i use want to get homography use svd, as formula, i use the last vector of V, but torch gets grad all zero;<br>
import torch</p>
<p>A_rows = [torch.randn(8, 9)] # 示例数据<br>
device = torch.device(‘cuda’ if torch.cuda.is_available() else ‘cpu’)<br>
A = torch.cat(A_rows).view(-1, 9).to(device)<br>
A.requires_grad_(True)</p>
<p>U, S, Vt = torch.linalg.svd(A)</p>
<p>H = Vt[-1] / Vt[-1, -1]<br>
H = H.view(3, 3)</p>
<p>A.retain_grad()<br>
H.retain_grad()<br>
Vt.retain_grad()</p>
<p>loss = H.norm()<br>
print(“Loss:”, loss)</p>
<p>loss.backward()</p>
<p>print(“A.grad:”, A.grad)<br>
print(“V.grad:”, Vt.grad)<br>
print(“H.grad:”, H.grad)</p>
<p>then, i print H’s Vt’s A’s grad, show:<br>
estimated_homo_matrix grad:<br>
A.grad: tensor([[0., 0., 0., 0., 0., 0., 0., 0., 0.],<br>
[0., 0., 0., 0., 0., 0., 0., 0., 0.],<br>
[0., 0., 0., 0., 0., 0., 0., 0., 0.],<br>
[0., 0., 0., 0., 0., 0., 0., 0., 0.],<br>
[0., 0., 0., 0., 0., 0., 0., 0., 0.],<br>
[0., 0., 0., 0., 0., 0., 0., 0., 0.],<br>
[0., 0., 0., 0., 0., 0., 0., 0., 0.],<br>
[0., 0., 0., 0., 0., 0., 0., 0., 0.]], device=‘cuda:0’)<br>
V.grad: tensor([[ 0.0000,  0.0000,  0.0000,  0.0000,  0.0000,  0.0000,  0.0000,  0.0000,<br>
0.0000],<br>
[ 0.0000,  0.0000,  0.0000,  0.0000,  0.0000,  0.0000,  0.0000,  0.0000,<br>
0.0000],<br>
[ 0.0000,  0.0000,  0.0000,  0.0000,  0.0000,  0.0000,  0.0000,  0.0000,<br>
0.0000],<br>
[ 0.0000,  0.0000,  0.0000,  0.0000,  0.0000,  0.0000,  0.0000,  0.0000,<br>
0.0000],<br>
[ 0.0000,  0.0000,  0.0000,  0.0000,  0.0000,  0.0000,  0.0000,  0.0000,<br>
0.0000],<br>
[ 0.0000,  0.0000,  0.0000,  0.0000,  0.0000,  0.0000,  0.0000,  0.0000,<br>
0.0000],<br>
[ 0.0000,  0.0000,  0.0000,  0.0000,  0.0000,  0.0000,  0.0000,  0.0000,<br>
0.0000],<br>
[ 0.0000,  0.0000,  0.0000,  0.0000,  0.0000,  0.0000,  0.0000,  0.0000,<br>
0.0000],<br>
[ 1.7672, -1.3057,  2.8489,  1.8927,  0.2041,  2.3516,  0.9954, -1.9848,<br>
27.0288]], device=‘cuda:0’)<br>
H.grad: tensor([[-0.3338,  0.2466, -0.5381],<br>
[-0.3575, -0.0386, -0.4442],<br>
[-0.1880,  0.3749,  0.1889]], device=‘cuda:0’)<br>
why, all zeros?</p>

username: KFrank
cooked: <p>Hi Tao!</p>
<aside class="quote no-group quote-modified" data-username="yangtaodummt" data-post="1" data-topic="215307" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/yangtaodummt/48/75071_2.png" class="avatar"> yangtaodummt:</div>
<blockquote>
<p>A.grad: tensor([[0., 0., 0., 0., 0., 0., 0., 0., 0.],<br>
…<br>
why, all zeros?</p>
</blockquote>
</aside>
<p>The short story is that the singular-value decomposition is not necessarily unique,<br>
so pytorch declines to attempt to compute certain gradients.  (There is some nuance<br>
here, so if you have specific questions, please ask.)</p>
<p>In your case, where <code>A</code> has fewer rows than columns, not all of <code>Vt</code> is unique (again,<br>
some nuance), so pytorch just gives you zero for the <code>.grad</code>.</p>
<p>Quoting from the <a href="https://pytorch.org/docs/2.5/generated/torch.linalg.svd.html" rel="noopener nofollow ugc">documentation for torch.linalg.svd:</a></p>
<blockquote>
<p>Note</p>
<p>When full_matrices= True, the gradients with respect to U[…, :, min(m, n):] and Vh[…, min(m, n):, :] will be ignored, as those vectors can be arbitrary bases of the corresponding subspaces.</p>
</blockquote>
<p>Here is a simplified version of your code that calls <code>svd()</code> with <code>full_matrices = False</code>:</p>
<pre data-code-wrap="python"><code class="lang-python">import torch
print (torch.__version__)

_ = torch.manual_seed (2025)

A = torch.randn (2, 3, requires_grad = True)
print ('A = ...')
print (A)

U, S, Vt = torch.linalg.svd (A, full_matrices = False)   # compute "reduced" svd
print ('U.shape: ', U.shape, ', S.shape: ', S.shape, 'Vt.shape: ', Vt.shape)

print ('Vt = ...')
print (Vt)

H = Vt[-1] / Vt[-1, -1]

A.retain_grad()
H.retain_grad()
Vt.retain_grad()

loss = H.norm()
print('loss: ', loss)

loss.backward()

print('A.grad = ...')
print(A.grad)
print('Vt.grad = ...')
print(Vt.grad)
print('H.grad = ...')
print(H.grad)
</code></pre>
<p>And here is its output showing that in this case, gradients do flow back to <code>A</code>:</p>
<pre><code class="lang-plaintext">A = ...
tensor([[-0.8716,  0.1114,  1.2044],
        [-0.1803,  1.0021,  0.7914]], requires_grad=True)
U.shape:  torch.Size([2, 2]) , S.shape:  torch.Size([2]) Vt.shape:  torch.Size([2, 3])
Vt = ...
tensor([[-0.4441,  0.3981,  0.8027],
        [ 0.4845,  0.8603, -0.1586]], grad_fn=&lt;LinalgSvdBackward0&gt;)
loss:  tensor(6.3066, grad_fn=&lt;LinalgVectorNormBackward0&gt;)
A.grad = ...
tensor([[-24.4890,  -7.2269,  -1.7565],
        [ 13.0301, -22.3574,   7.9963]])
Vt.grad = ...
tensor([[ 0.0000,  0.0000,  0.0000],
        [ 3.0556,  5.4255, 38.7727]])
H.grad = ...
tensor([-0.4845, -0.8603,  0.1586])
</code></pre>
<p>Best.</p>
<p>K. Frank</p>