Back propagation trough slicing with list

AdilZouitine: <p>I have a similar problem to this <a href="https://github.com/pytorch/pytorch/issues/23653" rel="noopener nofollow ugc">issue</a> on github however no solution is proposed.</p>
<p>The proposed example code :</p>
<pre><code class="lang-python">slize = [1, 2, 3, 4]
x = torch.randn(10, requires_grad=True)
y = x[slize]
y.sum().backward()
# breaks second time calling backward
y.sum().backward() # rasises RuntimeErroe

</code></pre>
<p>In my case large tensor and I want to iterate gradient descent steps on mini batches.</p>
<p>example:</p>
<pre><code class="lang-auto">old_log_probs = ... # size [8000, 1]
for epoch in range(epoch):
...
   for batch_idx in rollout.shuffle_index(batch_size=8):
         loss = ppo_loss(      
                        new_log_probs, # size [8, 1]
                        old_log_probs[batch_index],  # size [8, 1]
                        advantages[batch_index],  # size [8, 1]
                        clip=0.2
                    )
</code></pre>
<p>Because when we get the subtensor with a list it is a copy and not a view.<br>
Is it possible to get a subtensor with random indexes as a view?</p>

albanD: <p>Hi,</p>
<p>There is a solution: set <code>retain_graph=True</code> in the first call to backward if you want to backrprop through it again later <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"><br>
This behavior is expected.</p>

AdilZouitine: <p>Hi, I tried the solution you proposed but it doesn’t work.</p>
<p>Do you have other ideas ? here is my error log when <code>retain_graph = True</code></p>
<pre><code class="lang-auto">RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.FloatTensor [128, 2]], which is output 0 of TBackward, is at version 2; expected version 1 instead. Hint: the backtrace further above shows the operation that failed to compute its gradient. The variable in question was changed in there or anywhere later. Good luck!

</code></pre>
<p>When loss.backward() withour retain graph :</p>
<pre><code class="lang-auto">RuntimeError: Trying to backward through the graph a second time, but the saved intermediate results have already been freed. Specify retain_graph=True when calling backward the first time.

</code></pre>

albanD: <aside class="quote no-group" data-username="AdilZouitine" data-post="3" data-topic="101416">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/adilzouitine/48/8345_2.png" class="avatar"> AdilZouitine:</div>
<blockquote>
<p>Hi, I tried the solution you proposed but it doesn’t work.</p>
</blockquote>
</aside>
<p>It does work <img src="https://discuss.pytorch.org/images/emoji/apple/smiley.png?v=12" title=":smiley:" class="emoji" alt=":smiley:" loading="lazy" width="20" height="20"> It is just that the code has another unrelated problem.<br>
Running the updated sample works fine:</p>
<pre data-code-wrap="python"><code class="lang-python">import torch
slize = [1, 2, 3, 4]
x = torch.randn(10, requires_grad=True)
y = x[slize]
y.sum().backward(retain_graph=True)
y.sum().backward() # runs fine
</code></pre>
<p>As the error mentions, you modify inplace a Tensor whose value is required for the backward computation. You can follow the instructions in the error message to get a pointer to which op is faulty and replace the inplace op there by an out of place one.</p>