Fail to re-create computational graph in a for-loop

username: ecolss
cooked: <p>I have a variable <code>a</code> and a bunch of functions <code>f_k(a)</code>, so I create tensor to hold the results of all these functions, each time when a function is computed, I also need to compute the gradient for this function, so here is what I did,</p>
<pre><code class="lang-auto">import torch as th

k = 2
a = th.tensor(2.0, dtype=th.float32, requires_grad=True)  # variable 
b = th.zeros(k, dtype=th.float32)  # tensor to hold results of the k functions
c = th.tensor(0.0, dtype=th.float32)

### this for-loop is just a dummy demo when I don't use a tensor to hold results
for i in range(k):
    c = a*i
    g = th.autograd.grad(c, a)   # works fine
    print(g)


### this is what I wanted to do
for i in range(k):
    b[i] = a*i
    g = th.autograd.grad(b[i], a)  # FAIL when i=1, won't create graph again for i=1?
    print(g)
</code></pre>
<p>My code doesn’t work, and I expected that each loop will created a new graph for <code>b[i]</code>, and that’s valid for me to do a <code>grad</code> operation, but it turns out to be false, why??</p>

username: albanD
cooked: <p>Hi,</p>
<p>You should not make <code>b</code> a Tensor. Because that means that when you are at <code>i=1</code>, <code>b</code> still refer to the computations done during <code>i=0</code> and so calling backward will try to backward through the graph of <code>i=0</code> again.<br>
I would just use a list as <code>b</code> and you won’t have any problem.<br>
If you need to ever compute gradients through each f_k, you need to pass <code>retain_graph=True</code> to the grad function.<br>
If you want to use b as a single Tensor after the loop, just call <code>torch.stack</code> or <code>torch.cat</code> on the list <code>b</code> that will contain all your results !</p>

username: ecolss
cooked: <p>Yes, no problem at all if <code>b</code> is a list, I just want to avoid <code>list</code>, since calling <code>torch.cat</code> or <code>stack</code> might also bring in a little overhead (may marginal) of creating a tensor from a list, right?</p>

username: albanD
cooked: <p>Well you create the Tensor here yourself anyway <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=6" title=":wink:" class="emoji" alt=":wink:"><br>
And the caching allocator is good enough that if you repeat the operation at each forward, the allocation is for free.</p>

username: 335621
cooked: <p>Hi~<br>
I have a same question:</p><aside class="onebox stackexchange" data-onebox-src="https://stackoverflow.com/questions/68507559/how-can-we-use-pytorch-autograd-for-sequence-optimization-in-a-for-loop">
  <header class="source">

      <a href="https://stackoverflow.com/questions/68507559/how-can-we-use-pytorch-autograd-for-sequence-optimization-in-a-for-loop" target="_blank" rel="noopener nofollow ugc">stackoverflow.com</a>
  </header>

  <article class="onebox-body">
      <a href="https://stackoverflow.com/users/13482179/frank" target="_blank" rel="noopener nofollow ugc">
    <img alt="Frank" src="https://www.gravatar.com/avatar/9f91dd26c535bb5c28ae9f48d1da95ab?s=128&amp;d=identicon&amp;r=PG&amp;f=1" class="thumbnail onebox-avatar" width="128" height="128">
  </a>

<h4>
  <a href="https://stackoverflow.com/questions/68507559/how-can-we-use-pytorch-autograd-for-sequence-optimization-in-a-for-loop" target="_blank" rel="noopener nofollow ugc">How can we use Pytorch Autograd for sequence optimization (in a for loop)?</a>
</h4>

<div class="tags">
  <strong>python, optimization, deep-learning, pytorch</strong>
</div>

<div class="date">
  asked by
  
  <a href="https://stackoverflow.com/users/13482179/frank" target="_blank" rel="noopener nofollow ugc">
    Frank
  </a>
  on <a href="https://stackoverflow.com/questions/68507559/how-can-we-use-pytorch-autograd-for-sequence-optimization-in-a-for-loop" target="_blank" rel="noopener nofollow ugc">06:26AM - 24 Jul 21 UTC</a>
</div>

  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>

<p>Could you help with this?</p>