Backprop in PyTorch?

Response777: <p>Here is a snippet</p>
<pre><code class="lang-auto">import torch
import torch.nn as nn
from torch.autograd import Variable

dtype = torch.FloatTensor
x = Variable(torch.randn(1, 25).type(dtype), requires_grad = True)
t = Variable(torch.randn(1, 25).type(dtype), requires_grad = False)

criterion = nn.MSELoss()
loss = criterion(x, t)
optimizer = torch.optim.Adam([x])

for i in range(5):
    optimizer.zero_grad()
    loss.backward()
    optimizer.step()
</code></pre>
<p>This seems easy that we can even calculate such by hand. However, such code doesn’t work, and would throw an error:</p>
<pre><code class="lang-auto">---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
&lt;ipython-input-1-cace42d6c54e&gt; in &lt;module&gt;()
     13 for i in range(5):
     14     optimizer.zero_grad()
---&gt; 15     loss.backward()
     16     optimizer.step()

RuntimeError: Trying to backward through the graph second time, but the buffers have already been freed. Please specify retain_variables=True when calling backward for the first time.
</code></pre>
<p>Follow such suggestion, I modify the code above as</p>
<pre><code class="lang-auto">import torch
import torch.nn as nn
from torch.autograd import Variable

dtype = torch.FloatTensor
x = Variable(torch.randn(1, 25).type(dtype), requires_grad = True)
t = Variable(torch.randn(1, 25).type(dtype), requires_grad = False)

criterion = nn.MSELoss()
loss = criterion(x, t)
optimizer = torch.optim.Adam([x])

for i in range(5):
    optimizer.zero_grad()
    loss.backward(retain_variables=(i==0))
    optimizer.step()
</code></pre>
<p>The same error is thrown out again.</p>
<p>However, if we write a numpy snippet</p>
<pre><code class="lang-auto">import numpy as np

N = 5
x = np.random.randn(N)
y = np.random.randn(N)

learning_rate = 1e-2
for t in range(500):
    loss = np.square(x - y).sum()
    print(t, loss)
    # Back-propagate
    grad_y = 2.0 * (y - x)
    # Update
    y -= learning_rate * grad_y
</code></pre>
<p>We can see that it would converge in a few iteration.</p>
<p>Is this a <code>bug</code>? Or is there anything wrong in my code?</p>
<p>Thanks.</p>

xwgeng: <p>In pytorch, if you apply same variable <code>backward</code> twice or more, it’s necessary to set the <code>retrain_varaibles=True</code>. Regarding your problem, I think when update the parameters with <code>optimizer.step()</code>, you should re-compute the <code>loss</code></p>
<blockquote>
<p>import torch<br>
import torch.nn as nn<br>
from torch.autograd import Variable</p>
</blockquote>
<blockquote>
<p>dtype = torch.FloatTensor<br>
x = Variable(torch.randn(1, 25).type(dtype), requires_grad = True)<br>
t = Variable(torch.randn(1, 25).type(dtype), requires_grad = False)</p>
</blockquote>
<blockquote>
<p>criterion = nn.MSELoss()<br>
optimizer = torch.optim.Adam(<span class="chcklst-box checked fa fa-check-square-o fa-fw"></span>)</p>
</blockquote>
<blockquote>
<p>for i in range(5):<br>
optimizer.zero_grad()<br>
loss = criterion(x, t)<br>
loss.backward()<br>
optimizer.step()</p>
</blockquote>

Response777: <p>Thanks, it seems that I forget to paste the whole codes here <img src="//discuss.pytorch.org/images/emoji/apple/sweat_smile.png?v=5" title=":sweat_smile:" class="emoji" alt=":sweat_smile:"></p>
<p>Here is another case</p>
<pre><code class="lang-auto">import torch
import torch.nn as nn
from torch.autograd import Variable

dtype = torch.FloatTensor
class SimpleNet(nn.Module):
    def __init__(self):
        super(SimpleNet, self).__init__()
        
        self.x_dim = 50
        self.hidden_size = 25
        self.w = Variable(torch.randn(self.x_dim, self.hidden_size).type(dtype), requires_grad=True)
        
        self.freeze_weights()
        
    def forward(self, x):
        return torch.mm(x, self.w)
    
    def freeze_weights(self):
        for p in self.modules():
            p.requires_grad = False

net = SimpleNet()
x = Variable(torch.randn(1, 25).type(dtype), requires_grad = True)
t = net(Variable(torch.randn(1, 50).type(dtype), requires_grad = False))

criterion = nn.MSELoss()
optimizer = torch.optim.Adam([x])

for i in range(5):
    loss = criterion(x, t)
    
    optimizer.zero_grad()
    loss.backward()
    optimizer.step()
</code></pre>
<p>These code is similar to the one above, except that the target comes from <code>SimpleNet</code>.</p>
<p>But it would throw an error:</p>
<pre><code class="lang-auto">AssertionError: nn criterions don't compute the gradient w.r.t. targets - please mark these variables as volatile or not requiring gradients
</code></pre>
<p>And we still have the numpy codes here:</p>
<pre><code class="lang-auto">import numpy as np

M = 10
N = 5
x = np.random.randn(M)
y = np.random.randn(N)

w = np.random.randn(M, N)

learning_rate = 1e-2
for t in range(500):
    loss = np.square(x.dot(w) - y).sum()
    print(t, loss)
    # Back-propagate
    grad_y = 2.0 * (y - x.dot(w))
    # Update
    y -= learning_rate * grad_y
</code></pre>
<p>Is there anything wrong in my code?<br>
Thanks.</p>

albanD: <p>Hi,</p>
<p>So pytorch is very different from tensorflow in the sense that your don’t create the graph once, and then just backprop through it.<br>
The whole framework is built such that the graph is super cheat to build and so it is built on the fly.<br>
That means that for every input for which you want to compute backprop, you should execute your computation with them, and then call backward on the output.<br>
In your case, you should have:</p>
<pre><code class="lang-auto">import torch
import torch.nn as nn
from torch.autograd import Variable

dtype = torch.FloatTensor
class SimpleNet(nn.Module):
    def __init__(self):
        super(SimpleNet, self).__init__()
        
        self.x_dim = 50
        self.hidden_size = 25
        # Parameters in modules have type Parameter
        self.w = nn.Parameter(torch.randn(self.x_dim, self.hidden_size).type(dtype))
        
        self.freeze_weights()
        
    def forward(self, x):
        return torch.mm(x, self.w)
    
    def freeze_weights(self):
        # There are no submodules to iterate through
        self.w.requires_grad = False

net = SimpleNet()
x = Variable(torch.randn(1, 25).type(dtype), requires_grad = True)

criterion = nn.MSELoss()
# The optimizer is given only x, so it will update only x, not the weights in net
# Set the learning rate to 1 so that the loss evolves quickly
optimizer = torch.optim.Adam([x], lr=1)

input = torch.randn(1, 50).type(dtype)
for i in range(50):
    # You should only package your inputs inside the loop
    input_var = Variable(input)
    # Compute the forward pass here
    t = net(input_var)
    loss = criterion(x, t)
    print("Loss after {}\t steps is {}".format(i, loss.data[0]))
    
    optimizer.zero_grad()
    loss.backward()
    optimizer.step()
</code></pre>

jekbradbury: <p>That code won’t give an error, but it’s still wrong. <a class="mention" href="/u/alband">@albanD</a> has explained why: you need to run your model again for every iteration.</p>

alwynmathew: <p>Do we need to <code>zero_grad</code> the network too? As <code>net</code> isn’t wrapped by the <code>optimizer</code>.</p>
<pre><code class="lang-auto">optimizer = torch.optim.Adam([x], lr=1)
net.eval()
.
.
optimizer.zero_grad()
net.zero_grad()
loss.backward()
optimizer.step()
</code></pre>
<p>according to below link:</p>
<aside class="quote" data-post="7" data-topic="68722">
  <div class="title">
    <div class="quote-controls"></div>
    <img alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/juanfmontesinos/48/76115_2.png" class="avatar">
    <a href="https://discuss.pytorch.org/t/add-one-tensor-from-dataloader-to-optimiser/68722/7">Add one tensor from dataloader to optimiser</a> 
  </div>
  <blockquote>
    In fact, you should zero grad both, model and tensor. Realize that you are doing: 
     while(loss &gt; threshold):
          output = model(input)
          loss = loss_fn(output)
          optimizer.zero_grad()
          loss.backward()
          optimizer.step()

That way gradients are being accumulated on model. As model isn’t wrapped by the optimizer, its gradients are never zeroed. (In pytorch if you don’t make gradients zero they accumulate to the previous ones)
  </blockquote>
</aside>


albanD: <p>Note that this answer is really old. Variables don’t exist anymore.<br>
Also if you don’t are about the gradients in the network, you don’t need to zero them out indeed.</p>