About the uniqueness of eigen decomposition with torch.eigh

wasabi_linguist: <p>Hi, I would like to train a network where the loss function uses the eigenvectors of two hermitian matrices to calculate a loss on them. However, as PyTorch’s documentation for <code>eigh</code> says, the computed eigenvalues/eigenvectors are not unique because of their phases. Is there any way to “force” the <code>eigh</code> function to give eigenvalues/eigenvectors only in one phase?<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/2/1/21f5e2cb3119b8de74bce7c3b6be1a76554ed46d.png" data-download-href="https://discuss.pytorch.org/uploads/default/21f5e2cb3119b8de74bce7c3b6be1a76554ed46d" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/2/1/21f5e2cb3119b8de74bce7c3b6be1a76554ed46d.png" alt="image" data-base62-sha1="4Qqz0zaULAZeqJCVJ8Nyl1OrNOB" width="690" height="229" data-dominant-color="E6DBDC"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">859×286 36.3 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

KFrank: <p>Hi Wasabi!</p>
<aside class="quote no-group quote-modified" data-username="wasabi_linguist" data-post="1" data-topic="212468" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/wasabi_linguist/48/73592_2.png" class="avatar"> wasabi_linguist:</div>
<blockquote>
<p>the loss function uses the eigenvectors of two hermitian matrices<br>
…<br>
Is there any way to “force” the <code>eigh</code> function to give eigenvalues/eigenvectors only in one phase?</p>
</blockquote>
</aside>
<p>As noted in the documentation you posted a screen-shot of, what matters<br>
is whether your loss function depends on the (non-unique) phase of your<br>
eigenvectors.  If it doesn’t, things will be fine and autograd will backpropagate<br>
correctly trough <code>eigh()</code>.</p>
<p>If your loss function <em>does</em> depend on the phase of your eigenvectors, then<br>
you are trying to do something that doesn’t make sense.  Even if you were<br>
somehow able to coerce your eigenvectors to have well-defined phases, you<br>
would have merely swept the issue under the rug.  (You could likely expect<br>
your loss function to have some discontinuous jump at some sensible set of<br>
values for your parameters or other pathology.)</p>
<p>As an aside, if you have degenerate eigenvalues, then the whole eigenvector<br>
(not just its phase) is not unique.  I <em>think</em> in such a case that autograd would<br>
raise an exception were you to attempt to backpropagate.  Regardless, if<br>
your loss function were to depend <em>only</em> on the (multidimensional) subspaces<br>
spanned by the sets of degenerate eigenvectors, rather than on any specific<br>
eigenvector, backpropagation of your loss function would be mathematically<br>
well-defined, even if autograd and <code>eigh()</code> couldn’t handle it.</p>
<p>Best.</p>
<p>K. Frank</p>