Pytorch 0.3.0 equivalent of "with torch.no_grad()"

KFrank: <p>Hello Forum!</p>
<p>I am, for whatever reason, attempting to convert a simple<br>
pytorch-1.0.1 training script to pytorch version 0.3.0.</p>
<p>As far as I can tell, <code>torch.no_grad()</code> doesn’t exist in 0.3.0.</p>
<p>My use case is that (in the training loop) I run the model<br>
(predict, based on my input data), and then do some sanity<br>
checks and collect some statistics.  I wrap these latter two<br>
in a <code>with torch.no_grad():</code> block.  I don’t really understand<br>
what I am doing (but it works …), but I suppose I avoid<br>
cluttering up my graph with cruft (efficiency) and/or changing<br>
the results of calling <code>loss.backward()</code> (correctness).</p>
<p>What should I do in 0.3.0?  Can I just leave out the<br>
<code>with torch.no_grad()</code> (because my additional calculations<br>
come after calculating the loss, so the gradients won’t be<br>
affected)?  Should I clone / detach any tensors I use in my<br>
sanity / statistics calculations?  Is there something in 0.3.0<br>
like <code>no_grad()</code> that I should be using (but with a different<br>
name or semantics)?</p>
<p>Thanks for any help.</p>
<p>K. Frank</p>

ptrblck: <p>In <code>0.3</code> you would use <code>Variable(torch.tensor(...), volatile=True)</code> to avoid creating the computation graph.<br>
Let me know, if that works for you.</p>

KFrank: <p>Hi Peter!</p>
<aside class="quote no-group" data-username="ptrblck" data-post="2" data-topic="56071" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar"> ptrblck:</div>
<blockquote>
<p>In <code>0.3</code> you would use <code>Variable(torch.tensor(...), volatile=True)</code> to avoid creating the computation graph.<br>
Let me know, if that works for you.</p>
</blockquote>
</aside>
<p>Thank you for your reply.</p>
<p>To clarify in my concrete case:</p>
<p>The output of running the forward pass, for training, of my model<br>
is a <code>torch.autograd.variable.Variable</code>.  Should I retrieve its<br>
<code>FloatTensor</code> by using its <code>data</code> property, wrap that tensor in a<br>
volatile <code>Variable</code>, a la:</p>
<pre data-code-wrap="python"><code class="lang-python">torch.autograd.variable.Variable (preds.data, volatile = True)
</code></pre>
<p>and then use this volatile <code>Variable</code> for any additional calculations?</p>
<p>Thanks.</p>
<p>K. Frank</p>

ptrblck: <p>Hi Frank,</p>
<p>this work flow would detach the output from the model and all following operation put in a "<code>no_grad</code> block".<br>
<code>preds</code> would still hold to the computation graph. The <code>Variable</code> wrapping <code>preds.data</code> is detached and Autograd does not track any operations on it.<br>
Is this the use case?</p>

KFrank: <p>Hello Peter!</p>
<aside class="quote no-group quote-modified" data-username="ptrblck" data-post="4" data-topic="56071" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar"> ptrblck:</div>
<blockquote>
<p>Hi Frank,</p>
<p>this work flow would detach the output from the model and all following operation put in a “<code>no_grad</code> block”.<br>
<code>preds</code> would still hold to the computation graph. The <code>Variable</code> wrapping <code>preds.data</code> is detached and Autograd does not track any operations on it.</p>
</blockquote>
</aside>
<p>Thank you for your clarifying reply.</p>
<aside class="quote no-group">
<blockquote>
<p>Is this the use case?</p>
</blockquote>
</aside>
<p>Yes, I believe that it is.</p>
<p>I will follow up when I’ve I tried this out (and if and when I think I<br>
understand it …).</p>
<p>Best.</p>
<p>K. Frank</p>

KFrank: <p>Hi Peter!</p>
<aside class="quote no-group" data-username="ptrblck" data-post="2" data-topic="56071" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar"> ptrblck:</div>
<blockquote>
<p>In <code>0.3</code> you would use <code>Variable(torch.tensor(...), volatile=True)</code> to avoid creating the computation graph.<br>
Let me know, if that works for you.</p>
</blockquote>
</aside>
<p>Thank you again.  Yes, this works.  With this (plus a couple of<br>
other 0.3.0 tweaks) my 0.3.0 training run appears to be statistically<br>
identical to the 1.0.1 version, so it looks like I’m not corrupting<br>
the gradients, or anything.</p>
<p>Best.</p>
<p>K. Frank</p>