Why is weights.grad None after trying to adjust weights?

TenYa: <p>Hello</p>
<p>I’m in my first week of learning PyTorch and currently trying gradient descent. I’ve run into a problem where when I try to adjust the weights of my model, I get an error on my first attempt and the gradient of my weights becomes None. As far as I could see, it’s the calculation itself that causes the problem, as the gradient is still a tensor until right before the calculation.</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/4/a/4a4c75a29f6502b57399e87297d079699a7cddcd.png" data-download-href="https://discuss.pytorch.org/uploads/default/4a4c75a29f6502b57399e87297d079699a7cddcd" title="weights error"><img src="https://discuss.pytorch.org/uploads/default/original/3X/4/a/4a4c75a29f6502b57399e87297d079699a7cddcd.png" alt="weights error" data-base62-sha1="aBh95odvaaSsLhbcshkr5jU7CLH" width="690" height="285" data-dominant-color="F2EAE9"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">weights error</span><span class="informations">1089×450 29.2 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>I don’t know if it’s necessary, but here are my model and my mean-square-error function (can’t put more than one media in a post)</p>
<p>weights = torch.randn(2,3, requires_grad=True)<br>
biases = torch.randn(2, requires_grad=True)</p>
<p>def model(x):<br>
return x @ weights.T + biases</p>
<p>def mse(pred, target):<br>
diff = pred - target<br>
diff_square = diff**2<br>
loss = torch.mean(diff_square)<br>
return loss</p>
<p>Any help is appreciated!</p>

AlphaBetaGamma96: <p>I’m not 100% sure but it seems you’re not populating your <code>.grad</code> attribute when backpropagating your loss. It might happen due to you calling <code>.backward()</code> directly on your loss function and not the loss itself. Regardless, the following minimal reproducible example will population the <code>.grad</code> attributes and return gradients,</p>
<pre><code class="lang-auto">import torch

weights = torch.randn(2,3, requires_grad=True)
biases = torch.randn(2, requires_grad=True)

def model(x):
  return x @ weights.t() + biases

def mse(pred, target):
  diff = pred - target
  diff_square = diff**2
  loss = torch.mean(diff_square)
  return loss
  
inputs=torch.randn(10,3) #random input/output data
targets=torch.randn(10,2)

loss = mse(model(inputs), targets) #define scalar loss
loss.backward() #backprop here

print(weights.grad)
print(biases.grad)
"""
returns 
tensor([[ 0.0731,  0.0566,  0.0354],
        [-1.4290,  0.9217,  2.7733]])
tensor([-0.3319,  1.3362])
"""
</code></pre>

TenYa: <p>I’ve checked the gradient on every step right until the adjustment step and it is still there. But as soon as I execute the subtraction “weights = weights - (1e-5 * weights.grad)” it becomes None, while weights gets calculated correctly. Not calling backward() on my function is definitely a better idea though and better to read.</p>

AlphaBetaGamma96: <p>What’s probably happening then is that when you re-define <code>weights</code> as the difference of two tensors it most likely removes the <code>.grad</code> attribute which is why it gets set to <code>None</code>.</p>
<p>When updating tensors via the <code>.grad</code> attribute you’ll want to it within a <code>torch.no_grad()</code> context manager (which removes the <code>UserWarning</code>) but still has the <code>.grad = None</code> error. The <code>.grad</code> attribute is populated when you call <code>.backward()</code> so  if you want the <code>.grad</code> again you’ll need to recompute the gradient.</p>
<pre><code class="lang-auto">with torch.no_grad():
  weights.retain_grad
  print("grad (before): ",weights.grad)
  weights = weights - weights.grad
  print(weights)
  print("grad (after):  ",weights.grad)

"""
returns 
grad (before):  tensor([[-0.1729, -0.9709,  1.2458],
        [ 0.2753, -0.6027,  0.1230]])
tensor([[ 0.2790,  0.4543, -0.0610],
        [ 1.4604,  0.2984, -0.2316]])
grad (after):   None
"""
</code></pre>

TenYa: <p>I’m not sure I understood you correctly, but I tried calling backward before resetting the gradients and it still gives the same error.</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/f/d/fdbec20c69ba1158e3a80cf8a804c55513fcbb3f.png" data-download-href="https://discuss.pytorch.org/uploads/default/fdbec20c69ba1158e3a80cf8a804c55513fcbb3f" title="Screenshot 2022-06-06 174609"><img src="https://discuss.pytorch.org/uploads/default/original/3X/f/d/fdbec20c69ba1158e3a80cf8a804c55513fcbb3f.png" alt="Screenshot 2022-06-06 174609" data-base62-sha1="AcJuP4CMxNnmD14vzl48SQRQu8D" width="690" height="252" data-dominant-color="F3F2F1"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screenshot 2022-06-06 174609</span><span class="informations">714×261 12.9 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

AlphaBetaGamma96: <p>You need to call <code>loss.backward()</code> before uses <code>.grad</code> as it defaults to <code>None</code>. Then when you update the <code>weights</code> the <code>.grad</code> attribute gets removed so there’s no need to call <code>.grad.zero_()</code> as it being <code>None</code> is the same as zeroing that Tensor.</p>
<p>If you want to do gradient descent, you can use the <code>torch.optim.SGD</code> class which will perform gradient descent too.</p>