Autograd Only Calculating 1 of 2 Derivatives and Returning Incorrect Vales

AlexB: <p>I’m trying to use autograd to find the gradient of a function I’ve written as:</p>
<pre><code class="lang-auto">import numpy as np
import torch 
import matplotlib.pyplot as plt

from AFutilsPhaseMag import getParams

# get shared sim parameters (these don't change w/in a run in this test)
[f0, rho, tht, x, y, fi] = getParams()

# def varaibles
# want to find radients w/r.t. mangitude (a) and phase (alpha)
a = torch.tensor([1., 1.], requires_grad = True)
alpha = torch.tensor([0., 0.], requires_grad = True)
# everything else is kept constant and does not need a gradient found
x = torch.tensor(x)
y = torch.tensor(y)
rho = torch.tensor(rho)
tht = torch.tensor(tht)
fi = torch.tensor(fi)

# num Tx, num Rx
Na = x.size()[0]
Ns = rho.size()[0]

# gains and array factor storage
k = torch.tensor(np.ones((Na,1)))
AF = torch.tensor(np.zeros((Ns,1)), dtype = torch.complex64)

# build array factor at each Rx from the contributions of each Tx
for rec in range(Ns):
    for agent in range(Na):
        AF[rec] = AF[rec] + torch.exp(1j*alpha[agent] + k[agent]*x[agent]*torch.cos(tht[rec]) + k[agent]*y[agent]*torch.sin(tht[rec]))
        
# loss is difference of expected and achieved
loss = (torch.abs(AF) - fi.reshape([-1, 1])).sum()

# backprop AD
a.retain_grad()
alpha.retain_grad()
loss.backward(...)
             
# print grads 
print(a.grad)
print(alpha.grad)
</code></pre>
<p>I’m running into two problems that I’m not entirely sure how to fix:<br>
First, running this without providing any inputs to the loss.backward() call will return:</p>
<pre><code class="lang-auto">None
tensor([0., 0.])
</code></pre>
<p>Which I don’t understand because there’s no reason I can see why autograd can differentiate w/r.t. one variable but not the other. Additionally, I have an analytic solution for this gradient and I know that the gradient w/r.t. the 2nd variable isn’t zero.<br>
I looked for answers in the docs and online and kept finding answers saying that the problem is a nondifferentiable function, except I know this function has a derivative.</p>
<p>Later, I tried running the code but passing arguments into loss.backward(). The two variables I’m interested in finding the gradient w/r.t. are both 2x1 vectors, and when I call backprop as <code> loss.backward(torch.tensor([1,1]),torch.tensor([1,1]))</code> I get the error:</p>
<pre><code class="lang-auto">RuntimeError: Mismatch in shape: grad_output[0] has a shape of torch.Size([2]) and output[0] has a shape of torch.Size([]).
</code></pre>
<p>I know this has to do with the outputs being a scalar of size (,) and there being two inputs to backward(), but how do I fix this?</p>
<p>Sorry if these are simple questions, I’m a bit out of practice in Python and have gotten way too used to how MATLAB handles matrix dimensionality. Can anyone see what I did wrong there? I’m kind of at a loss for the time being.</p>

ptrblck: <aside class="quote no-group" data-username="AlexB" data-post="1" data-topic="180572">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/alexb/48/59275_2.png" class="avatar"> AlexB:</div>
<blockquote>
<p>Which I don’t understand because there’s no reason I can see why autograd can differentiate w/r.t. one variable but not the other.</p>
</blockquote>
</aside>
<p>I don’t see any usage of <code>a</code> in any calculation unless I’m missing it, so it’s expected to see <code>None</code> gradients for it as the <code>loss</code> doesn’t depend on it.</p>

AlexB: <p>I genuinely can’t believe I did that, thank you for pointing that out <img src="https://discuss.pytorch.org/images/emoji/apple/sweat_smile.png?v=12" title=":sweat_smile:" class="emoji" alt=":sweat_smile:" loading="lazy" width="20" height="20"></p>
<p>That fixed the gradient w/r.t. a, then I was able to find a gradient w/r.t. alpha by reshaping the loss and inputs to the .backward() call</p>
<p>Thank you!</p>