Getting incorrect negative signs when doing SVD, why?

Manaswi_Mancha: <p>I listed my code below. I am trying to prove that left-singular vectors are eigenvectors of AA* and right-singular vectors are eigenvectors of A*A, but I am getting weird negative signs that are throwing off my answer even though the absolute values are the same.</p>
<pre><code class="lang-python">import torch

P = torch.tensor([[25, 2, -5.],
                  [3, -2, 1],
                  [5, 7, 4]])

# Verifying SVD process, works!
print(f"Verifying SVD process, works!\nOriginal matrix:\n{P}")
U, D, VT = torch.linalg.svd(P)
print(f"U:\n{U}\nD:\n{D}\nV:\n{VT.T}")
print(f"Getting orginal matrix from U, D, V values:\n{U@torch.diag(D)@VT}")
print("-"*25)

# Representing U, D, and V as eigenvectors/eigenvalues of P@P.T and P.T@P and verifying SVD process, doesn't work!
print(f"Representing U, D, and V as eigenvectors/eigenvalues of P@P.T and P.T@P and verifying SVD process, doesn't work!\nOriginal matrix:\n{P}")
e_val_1, e_vec_1 = torch.linalg.eig(P@P.T)
e_val_2, e_vec_2 = torch.linalg.eig(P.T@P)
e_vec_2 = e_vec_2[torch.tensor([0,1,2])][:,torch.tensor([0,2,1])].to(torch.float)
e_vec_1 = e_vec_1.to(torch.float)
srt, ind = torch.sort(torch.sqrt(e_val_2).to(torch.float),descending=True)

# U, D, V values are not the same as above, has same absolute values but different signs, but why?
print(f"(These values don't match original) U:\n{e_vec_1}\nD:\n{srt}\nV:\n{e_vec_2}")

print(f"Getting orginal matrix from U, D, V values:\n{e_vec_1.to(torch.float)@torch.diag(srt)@e_vec_2.to(torch.float).T}")

# Need to make following modifications to make code work, but why?
e_vec_2 = -e_vec_2
e_vec_1[:,1] = -e_vec_1[:,1]

print(f"After making modifications (listed in code):\n{e_vec_1.to(torch.float)@torch.diag(srt)@e_vec_2.to(torch.float).T}")
</code></pre>

KFrank: <p>Hi Manaswi!</p>
<aside class="quote no-group" data-username="Manaswi_Mancha" data-post="1" data-topic="184458" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/manaswi_mancha/48/62340_2.png" class="avatar"> Manaswi_Mancha:</div>
<blockquote>
<p>I am getting weird negative signs that are throwing off my answer even though the absolute values are the same.</p>
</blockquote>
</aside>
<p>The short story is that your negative signs represent a legitimate phase<br>
ambiguity, rather than being incorrect.</p>
<p>I haven’t looked at your code in any detail, but the basic issue is that<br>
neither the singular vectors in the singular-value decomposition nor the<br>
eigenvectors in the eigendecomposition are uniquely defined.  Instead<br>
they can carry an arbitrary phase, which in the real case, is an arbitrary<br>
sign.</p>
<p>Quoting, for example, from pytorch’s <a href="https://pytorch.org/docs/2.0/generated/torch.linalg.svd.html" rel="noopener nofollow ugc">svd() documentation:</a></p>
<blockquote>
<p>Warning</p>
<p>The returned tensors U and V are not unique, nor are they continuous with respect to A. Due to this lack of uniqueness, different hardware and software may compute different singular vectors.</p>
<p>This non-uniqueness is caused by the fact that multiplying any pair of singular vectors uk,vk​ by -1 in the real case or by eiϕ,ϕ∈R in the complex case produces another two valid singular vectors of the matrix. For this reason, the loss function shall not depend on this eiϕ quantity, as it is not well-defined. This is checked for complex inputs when computing the gradients of this function. As such, when inputs are complex and are on a CUDA device, the computation of the gradients of this function synchronizes that device with the CPU.</p>
</blockquote>
<p>Best.</p>
<p>K. Frank</p>