Detach, no_grad and requires_grad

michaelzet: <p>Hello<br>
It’s general question but currently I’m looking at tutorial:<br>
<a href="http://pytorch.org/tutorials/intermediate/reinforcement_q_learning.html" class="onebox" target="_blank" rel="nofollow noopener">http://pytorch.org/tutorials/intermediate/reinforcement_q_learning.html</a></p>
<p>why in optimize_model function - we have<br>
<strong><em>detach()</em></strong> method<br>
and in select_action we use<br>
<strong><em>with torch.no_grad():</em></strong></p>
<p>on the other hand doc:<br>
<a href="http://pytorch.org/docs/stable/notes/autograd.html" class="onebox" target="_blank" rel="nofollow noopener">http://pytorch.org/docs/stable/notes/autograd.html</a><br>
mentions only <strong><em>requires_grad</em></strong></p>
<p>of course I understand we don’t want to compute gradients here - but i don’t fully understand the difference between all those 3 methods…<br>
also If i’m not mistaken in previous versions of pytorch we used volatile=true which was considered more memory efficient (please correct me if i’m wrong) which is now replaced by <strong><em>with torch.no_grad():</em></strong></p>
<p>so if we used <strong><em>with torch.no_grad():</em></strong> in optimize_mode, would it be also ok?</p>
<p>could anyone please explain it to me?</p>
<p>thank you</p>
<p>best,<br>
Michael</p>

albanD: <p>Hi,</p>
<p>Does <a href="https://discuss.pytorch.org/t/clone-and-detach-in-v0-4-0/16861/2">this</a> post answer your question?</p>

michaelzet: <p>Hello albanD</p>
<p>thank you for your response.</p>
<p>this answer you mentioned helped me understand general idea but still i don’t fully understand the difference between torch.no_grad and detach<br>
if i can use it exchangeably - or are there any limitations?</p>
<p>thx<br>
Michael</p>

albanD: <p><code>detach()</code> detaches the output from  the computationnal graph. So no gradient will be backproped along this variable.<br>
<code>torch.no_grad</code> says that no operation should build the graph.</p>
<p>The difference is that one refers to only a given variable on which it’s called. The other affects all operations taking place within the <code>with</code> statement.</p>

Irfan_Bulu: <p>good explanation. Can you give some use cases for both? I am guessing you’d use torch.no_grad during eval phase? But what would you use detach() for? Any specific use cases would help understand them better</p>

albanD: <p><code>torch.no_grad</code> yes you can use in eval phase in general.</p>
<p><code>detach()</code> on the other hand should not be used if you’re doing classic cnn like architectures. It is usually used for more tricky operations.<br>
<code>detach()</code> is useful when you want to compute something that you can’t / don’t want to differentiate. Like for example if you’re computing some indices from the output of the network and then want to use that to index a tensor. The indexing operation is not differentiable wrt the indices. So you should <code>detach()</code> the indices before providing them.</p>

michaelzet: <p>Thank you albanD</p>
<p>definitely it helps!</p>
<p>so if i understand properly this part</p>
<aside class="quote no-group" data-username="albanD" data-post="4" data-topic="16915">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/alband/48/215_2.png" class="avatar"> albanD:</div>
<blockquote>
<p>The difference is that one refers to only a given variable on which it’s called. The other affects all operations taking place within the with statement.</p>
</blockquote>
</aside>
<p>(please anyone correct me if i’m wrong) we can use both method exchangeably and it should have same result</p>
<p>so</p>
<pre><code class="lang-auto">        with torch.no_grad():
            return policy_net(state).max(1)[1].view(1, 1)
</code></pre>
<p>should have similar effect as</p>
<pre><code class="lang-auto">       return policy_net(state).detach().max(1)[1].view(1, 1)
</code></pre>
<p>thanx again for your help!</p>

albanD: <p>The returned result will be the same, but the version with <code>torch.no_grad</code> will use less memory because it knows from the beginning that no gradients are needed so it doesn’t need to keep intermediary results.</p>

michaelzet: <p>You’re right. And this is huge difference.</p>
<p>Thank you</p>
<p>all clear now</p>

whoab: <p><a class="mention" href="/u/alband">@albanD</a>  Hi, does detach erase the existing weights?</p>
<p>So if we do:</p>
<p>policy_loss = …<br>
self.policy_optimizer.zero_grad()<br>
policy_loss.backward()<br>
self.policy_optimizer.step(0<br>
policy_loss.detach()</p>
<p>For some reason when I was doing this in a loop the weights of the policy.parameters() [connected to the policy optimizer] stopped changing. I don’t understand the mechanism for why.</p>

albanD: <p>The only thing detach does is to return a new Tensor (it does not change the current one) that does not share the history of the original Tensor.<br>
If the only thing you added to your code is <code>policy_loss.detach()</code> then it does not do anything to your code as you don’t use the result and detach does not change anything else.</p>

whoab: <p><a class="mention" href="/u/alband">@albanD</a> I see, any pointers for why the weights aren’t changing? I notice first the biases stop changing, then more and more hidden layers stop changing incrementally.</p>
<p>The gradients were on the order of E-2 so not super small, but perhaps combined with the 3E-4 learning rate the update becomes numerically ~0 and ignored?</p>

albanD: <p>That looks like this can happen, that would give you an update around 1e-6 which is where float number start to become meaningless.</p>

Xonobo_Xonobo: <p>Assume I have a network in two parts like:</p>
<pre><code class="lang-auto">def forward(self, x):
  x = self.part_1(x)
  return self.part_2(x)
</code></pre>
<p>If I want to update only the second part in training phase, will this modification do the desired thing?</p>
<pre><code class="lang-auto">def forward(self, x):
  with torch.no_grad():
    x = self.part_1(x)
  return self.part_2(x)
</code></pre>

ptrblck: <p>Yes, your code will work.<br>
Alternatively, you could call <code>.detach()</code> on <code>x</code> before passing it to <code>part2</code>:</p>
<pre><code class="lang-python">def forward(self, x):
    x = self.part_1(x)
    return self.part_2(x.detach())
</code></pre>

StrausMG: <p>In torch &gt;= 1.0.0, what is the correct way to copy the data of a tensor so that the copy “is just a copy” (i.e. a standalone tensor that doesn’t know anything about graphs, gradients etc.)?</p>
<pre><code class="lang-auto">x = my_tensor.detach().clone()
x = my_tensor.clone().detach()

# &gt;&gt;&gt; Now, the two lines above, but augmented with .data in all possible places:
x = my_tensor.data.detach().clone()
x = my_tensor.detach().data.clone()
# ...
x = my_tensor.clone().detach().data
# &lt;&lt;&lt;
</code></pre>
<p>It looks like all of these options solve the problem, don’t they? If it’s true, than the options with <code>.data</code> are all redundant and the question is <code>.detach().clone()</code> vs <code>.clone().detach()</code>.</p>

Xonobo_Xonobo: <p>One more case:</p>
<pre><code class="lang-auto">def forward(self,x):
  x = self.root(x)
  out1 = self.branch_1(x)
  out2 = self.branch_2(x.detach())
  return out1, out2

loss = F.l2_loss(out1, target1) + F.l2_loss(out2, target2)
loss.backward()
</code></pre>
<p>I want the gradients for the branch1 to update the parameters of the root and branch1. But the gradients of the second branch should only update the branch_2 parameters. I guess the detach usage will make it happen, but I want to be sure?</p>

albanD: <p>Hi,</p>
<p>Yes this will do what you want !</p>

GorkemP: <p>For example in Faster R-CNN gradients do not backprop through proposals; therefore, once we have computed the proposals in RPN, we can detach them and use the detached tensors (ROIs) for the rest of the networks (<a href="https://github.com/pytorch/vision/blob/8dfcff745a5bd2d4886716bf0deff8dcc8e75fed/torchvision/models/detection/rpn.py#L491" rel="nofollow noopener">https://github.com/pytorch/vision/blob/8dfcff745a5bd2d4886716bf0deff8dcc8e75fed/torchvision/models/detection/rpn.py#L491</a>).</p>
<p>Another usecase is that if there are auxilary losses in the network and if we do not detach the network in proper points (just after these auxiary points), gradients of some parameters’ will be calculated multiple times, which we may not want.</p>

Tejan_Mehndiratta: <p>So, If I do <code>.detach()</code> on a variable, then that variable will never be in the computation graph. Right?</p>