Question about implementation of complex Layers

Timwolf01: <p>I have a question about the implementation of complex Layers, maybe its a duplicate, but havent found a similar.</p>
<p>In linear Layer the forward Implementation is x @ w.T + b, made with torch.addmm.<br>
With complex data, weights and biases it could be formed to:<br>
(x.real @ w.real.T + b.real) - (x.imag @ w.imag.T + b.imag) + 1j * (x.real @ w.imag.T + b.imag + x.imag @ w.real.T + b.real)</p>
<p>But with nn.Linear(input_shape, output_shape, dtype=torch.complex64) it is:<br>
(x.real @ w.real.T + b.real) - (x.imag @ w.imag.T + 0) + 1j * (x.real @ w.imag.T + b.imag + x.imag @ w.real.T + 0)</p>
<p>Why is the bias treated as 0 in the two subterms?<br>
Is this intendet and whats the purpose or benefit of it?</p>
<p>In most self-made complex pytorch code found online the upper version is used.</p>
<p>In other layers like nn.Conv2d it is the same thing</p>

KFrank: <p>Hi Tim!</p>
<aside class="quote no-group" data-username="Timwolf01" data-post="1" data-topic="209467" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/timwolf01/48/72700_2.png" class="avatar"> Timwolf01:</div>
<blockquote>
<p>With complex data, weights and biases it could be formed to:<br>
(x.real @ w.real.T + b.real) - (x.imag @ w.imag.T + b.imag) + 1j * (x.real @ w.imag.T + b.imag + x.imag @ w.real.T + b.real)</p>
</blockquote>
</aside>
<p>This just isn’t correct.  Consider the case where everything, in<br>
particular the bias, <code>b</code>, is purely real.  Your result, as written, will<br>
contain an incorrect imaginary term, <code>1j * b.real</code>.</p>
<aside class="quote no-group">
<blockquote>
<p>But with nn.Linear(input_shape, output_shape, dtype=torch.complex64) it is:<br>
(x.real @ w.real.T + b.real) - (x.imag @ w.imag.T + 0) + 1j * (x.real @ w.imag.T + b.imag + x.imag @ w.real.T + 0)</p>
<p>Why is the bias treated as 0 in the two subterms?</p>
</blockquote>
</aside>
<p>The bias, <code>b</code>, is a purely additive term, so the real component<br>
of the result contains just <code>b.real</code> and the imaginary component<br>
of the result contains just <code>b.imag</code>.  I don’t understand why you<br>
would expect <code>b.real</code> and <code>b.imag</code> to both show up in both the<br>
real and imaginary components of the result.</p>
<aside class="quote no-group">
<blockquote>
<p>In most self-made complex pytorch code found online the upper version is used.</p>
</blockquote>
</aside>
<p>I find this hard to believe.  But if it is true, then most self-made<br>
code found online is wrong.</p>
<p>Best.</p>
<p>K. Frank</p>

Timwolf01: <p>Hi K. Frank,</p>
<p>thanks for your clarification. I see my error now. I must have found the exact only 2 or 3 githubs where its wrong and made an error in my thoughts. Of course the bias is just added after the mutliplication.</p>
<p>For interested (and my rehabilitation <img src="https://discuss.pytorch.org/images/emoji/apple/sweat_smile.png?v=12" title=":sweat_smile:" class="emoji" alt=":sweat_smile:" loading="lazy" width="20" height="20">) the top google results for complex pytorch githubs make it like this:</p>
<ul>
<li><a href="https://github.com/soumickmj/pytorch-complex/blob/f5d0c3511a3f3c7cb7f138ee7f42a699d3f8209a/torchcomplex/nn/functional.py#L73" rel="noopener nofollow ugc">repo 1</a></li>
<li><a href="https://github.com/wavefrontshaping/complexPyTorch/blob/636e1cc545be662f41e10808ff86f6daa2d18ce6/complexPyTorch/complexLayers.py#L34" rel="noopener nofollow ugc">repo 2</a></li>
</ul>
<p>I can only post two links, but these are examples of the wrong made ones. I also found repos where its made correctly.</p>
<p>Bests<br>
Tim</p>