Autograd with complex value

username: GoldenalCheese
cooked: <p>I am testing autograd with a simple complex scalar, but it doesn’t perform as what I am expecting. Specifically, it seems the imaginary part is completely discarded. Is there anything I am missing?</p>
<pre><code class="lang-auto">import torch

a = torch.tensor([1.0], requires_grad=True)
b = torch.exp(torch.tensor(1.j * 0.25 * torch.pi)) * a
grad = torch.autograd.grad(b, a)[0]

# print tensor([0.7071]), but should be tensor([0.7071+j0.7071]) ?
print(grad) 
</code></pre>

username: ptrblck
cooked: <p>Your code fails with:<br>
``python<br>
RuntimeError: grad can be implicitly created only for real scalar outputs but got torch.complex64</p>
<pre><code class="lang-auto">in my setup using `torch==2.4.0.dev20240610+cu124`.
After using `grad_outputs=torch.ones_like(b)` I get the same result, which seems to be expected since `a` is a real scalar value. 
Could you describe what a complex gradient would represent for a real tensor?</code></pre>

username: GoldenalCheese
cooked: <p>I am using torch 1.12.0. It successfully runs in my local env. <img src="https://discuss.pytorch.org/images/emoji/apple/thinking.png?v=12" title=":thinking:" class="emoji" alt=":thinking:" loading="lazy" width="20" height="20"></p>
<p>I am trying to take advantage of the autograd function to calculate the derivative of a complex scalar with respect to a real scalar (in a simplified scenario).</p>

username: KFrank
cooked: <p>Hi Cheese!</p>
<aside class="quote no-group" data-username="GoldenalCheese" data-post="3" data-topic="205726" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/goldenalcheese/48/69159_2.png" class="avatar"> GoldenalCheese:</div>
<blockquote>
<p>I am using torch 1.12.0. It successfully runs in my local env.</p>
</blockquote>
</aside>
<p>Pytorch’s support for complex tensors and complex autograd had been<br>
a work in progress in the recent past.  I don’t remember which versions<br>
had which changes and fixes, but it would not surprise me if version 1.12.0<br>
were incomplete and incorrect in significant ways.</p>
<p>For complex differentiation, please work with the current stable release<br>
(2.3.1), as described by the current stable documentation.</p>
<aside class="quote no-group">
<blockquote>
<p>I am trying to take advantage of the autograd function to calculate the derivative of a complex scalar with respect to a real scalar</p>
</blockquote>
</aside>
<p>As indicated by the error <a class="mention" href="/u/ptrblck">@ptrblck</a> got running your code (on a 2.4 nightly<br>
build):</p>
<pre><code class="lang-plaintext">RuntimeError: grad can be implicitly created only for real scalar outputs but got torch.complex64
</code></pre>
<p>complex autograd works, in essence, under the assumption that you are<br>
computing the gradient of a real-value loss function (with respect to model<br>
weights that you intend to optimize using gradient descent).</p>
<p>See, for example, pytorch’s <a href="https://pytorch.org/docs/2.3/notes/autograd.html#complex-autograd-doc" rel="noopener nofollow ugc">complex-autograd documentation.</a></p>
<p>If this doesn’t fit your use case, you could do things “the old-fashioned”<br>
way by using real tensors to compute the real and imaginary parts (both<br>
expressed as real tensors) of your complex scalar, and use autograd to<br>
compute the real tensors that correspond to the real and imaginary<br>
components of the specific version of the complex gradient you want.</p>
<p>But refer to the documentation I linked to above for an overview of the<br>
different types of complex derivatives and make sure that the choice<br>
you make is appropriate for your use case.</p>
<p>Best.</p>
<p>K. Frank</p>