Zero grad on single parameter

username: Intel_Novel
cooked: <p>Hi,<br>
I found this this code to zero the gradients on single parameter:<br>
<code>a.grad.zero_()</code></p>
<p>But it is not working: <code>AttributeError: 'NoneType' object has no attribute 'zero_'</code></p>
<p>I previously declared:</p>
<pre><code class="lang-auto">a = torch.tensor(-1., requires_grad=True)
a = nn.Parameter(a)
</code></pre>

username: sparseinference
cooked: <p>The gradient will be computed when you call:</p>
<pre><code class="lang-auto">a.backward()
</code></pre>

username: MariosOreo
cooked: <p>Hi,</p>
<p>Before you call <code>.backward()</code>, the gradient of each tensor which requires_grad=True are all <code>None</code>.<br>
Like the case you posted, you could calculate a.grad firstly and then <code>zero_()</code> its grad.<br>
In <code>opt.zero_grad()</code> it declare explicity:</p>
<pre><code class="lang-auto">    def zero_grad(self):
        r"""Clears the gradients of all optimized :class:`torch.Tensor` s."""
        for group in self.param_groups:
            for p in group['params']:
                if p.grad is not None:
                    p.grad.detach_()
                    p.grad.zero_()
</code></pre>
<p>May this help you.</p>

username: Intel_Novel
cooked: <p>Thanks for the feedback, it was None for sure and this caused the error.<br>
Here is the full code of the problem.</p>
<pre><code class="lang-auto">import torch
import torch.nn as nn
import matplotlib.pyplot as plt

n = 70 # num of points

# x is a tensor
x = torch.linspace(0, 10, steps=n)
k = torch.tensor(2.5)

# y is a tensor
y = k*x + 5*torch.rand(n)

# loss function
def mse(y_hat, y): return ((y_hat-y)**2).mean()

a = torch.tensor(-1., requires_grad=True)
a = nn.Parameter(a)

y_hat = a*x


plt.scatter(x, y);
plt.scatter(x,y_hat.detach());

lr = 0.1

def update_a():    
    global a
    loss = mse(y_hat, y) 
    print(loss) 
    loss.backward(a)
    print(a.grad)
    a.grad.zero_()
    a = a - lr * a.grad
   
    

for t in range(10): 
    a = update_a()

</code></pre>
<p>Plan is to learn the curve direction that was originally <code>-1</code> and it should be <code>2.5</code>.<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/2X/e/ebd78b8ac5cbba8d81a453ab711dd754dff946f2.png" data-download-href="https://discuss.pytorch.org/uploads/default/ebd78b8ac5cbba8d81a453ab711dd754dff946f2" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/2X/e/ebd78b8ac5cbba8d81a453ab711dd754dff946f2_2_690x447.png" alt="image" data-base62-sha1="xEm0xIQWdiUuzJy7RexuP2x2oxk" width="690" height="447" srcset="https://discuss.pytorch.org/uploads/default/optimized/2X/e/ebd78b8ac5cbba8d81a453ab711dd754dff946f2_2_690x447.png, https://discuss.pytorch.org/uploads/default/original/2X/e/ebd78b8ac5cbba8d81a453ab711dd754dff946f2.png 1.5x, https://discuss.pytorch.org/uploads/default/original/2X/e/ebd78b8ac5cbba8d81a453ab711dd754dff946f2.png 2x" data-dominant-color="E5E8EF"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">806×523 57.2 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
Green should be more like blue dots.</p>
<p>I use the mse loss function, and I expect to learn the parameter <code>a</code>.</p>

username: MariosOreo
cooked: <p>Hi,</p>
<p>I just run your code snipper, but there is something wrong, it raised RuntimeError about call <code>.backward</code> on the computation graph which had been freed. And <code>a.grad</code> is still None except the first iteration, so it learned nothing. Additionally, your learning_rate is too high.<br>
I write a snippet based on yours, and it works now.</p>
<pre><code class="lang-auto"> n = 70  # num of points
 # x is a tensor
 x = torch.linspace(0, 10, steps=n)
 k = torch.tensor(2.5)
 # y is a tensor
 y = k * x + 5 * torch.rand(n)

 # loss function
 def mse(y_hat, y):
     return ((y_hat - y) ** 2).mean()

 a = torch.tensor(-1., requires_grad=True)
 a = nn.Parameter(a)
 lr = 0.005

 for t in range(10):
     print(t, ':', a.grad, a.requires_grad)
     if a.grad is not None:
         a.grad.zero_()
     y_hat = a * x
     loss = mse(y_hat, y)
     loss.backward()
     #print(t, 'after backward:', a.grad, a.requires_grad, a.is_leaf)
     a = (a.data - lr * a.grad).requires_grad_(True)
     #print(t, 'after update:', a.requires_grad, a.is_leaf)

 plt.scatter(x, y)
 plt.scatter(x, y_hat.detach())
 plt.show()
</code></pre>
<ul>
<li>You should calculate y_hat in the loop, otherwise, there will raise a RuntimeError mentioned above.</li>
<li>If you assign <code>a</code> directly in each iteration, <code>a</code> will only have grad in the first iteration since <code>a</code> will be a non_leaf  Variable and its grad will be None.</li>
</ul>
<p><img src="https://discuss.pytorch.org/uploads/default/original/2X/e/e7e3e952314e93da008d8288ba4d92431e63e493.png" alt="forums_1" data-base62-sha1="x5oANRUP4FgqmdpbUPBz3pHKn3t" width="320" height="236"></p>
<p>EDIT: oh, I forgot to zero <code>a.grad</code> and I have correct it in the snippet.</p>

username: Intel_Novel
cooked: <p>Phenomenal update. You anticipated the problems I had and helped me.<br>
I learned much from your code.<br>
I noticed that this also works:</p>
<pre><code class="lang-auto">    with torch.no_grad():
          a.sub_(lr * a.grad)
</code></pre>
<p>once replaced this code:<br>
<code>a = (a.data - lr * a.grad).requires_grad_(True)</code></p>

username: sparseinference
cooked: <p>Well, <a class="mention" href="/u/mariosoreo">@MariosOreo</a> beat me to it, but here’s my rewrite of your code anyway:</p>
<pre><code class="lang-auto">import torch
import torch.nn as nn
import matplotlib.pyplot as plt

n = 70
x = torch.linspace(0, 10, steps=n)
k = 2.5

y = k*x + 5*torch.rand(n)
a = torch.tensor([-1.0, 0.0], requires_grad=True)

def y_hat():
    return a[0] * x + a[1]

lr = 0.01
for t in range(10): 
    loss = (y_hat() - y).pow(2.0).mean()
    loss.backward()
    a.data -= lr * a.grad.data
    print(f"{loss.item():7.3f} {a.data} {a.grad.data}") 
    a.grad.zero_()

plt.scatter(x, y)
plt.scatter(x, y_hat().detach().numpy())
plt.show()
</code></pre>
<p>You were zeroing the gradient for <code>a</code> before updating it.</p>
<p>The circularity error in calling <code>backward()</code> was due to the gradient of <code>a</code> being involved in the update. Using <code>a.data</code> instead avoids that.</p>
<p>There is no need to use the <code>nn.Parameter</code> wrapper. That allows the registration of parameter Tensors with a <code>Module</code>, which you are not using here.</p>
<p>As <a class="mention" href="/u/mariosoreo">@MariosOreo</a> mentioned, the gradients of Tensors don’t exist until <code>backward()</code> is called.<br>
This happens to everyone I think when they first try PyTorch because most of the examples show the gradient being zeroed at the beginning of the optimization loop. That’s why I zero them at the end of the loop. It makes the code a bit cleaner to not test for the existence of <code>a.grad</code> on every iteration.</p>
<p>Your constant <code>k</code> doesn’t need to be explicitly a Tensor. PyTorch will broadcast it when you use it.</p>
<p>I took the liberty of adding an extra coefficient to <code>a</code> for another example.</p>

username: MariosOreo
cooked: <p>I think using <code>with torch.no_grad()</code> is a much better way than using <code>.data</code> which ptblck mentioned in <a href="https://discuss.pytorch.org/t/initializing-parameters-with-weight-or-weight-data/39881/4?u=mariosoreo">this thread.</a></p>

username: sparseinference
cooked: <p>Yes, I agree. PyTorch is still evolving.</p>

username: ankurji
cooked: <p>a.grad.data.zero_()  can work…</p>