Output from the Loss-function

username: Johannes_L
cooked: <p>Hey,</p>
<p>I am trying to understand a example regarding training of a simple, fully connected net.<br>
The model is trained with the input “data”.</p>
<pre><code class="lang-auto">output = self.model(data)
loss = F.nll_loss(output, target)
if loss.cpu().data.numpy()[0] &gt; 10000:
</code></pre>
<p>What is the loss.data doing? Somehow relating the loss to the input data?</p>
<p>Thanks!</p>

username: KFrank
cooked: <p>Hi Johannes!</p>
<aside class="quote no-group" data-username="Johannes_L" data-post="1" data-topic="100655" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/johannes_l/48/30296_2.png" class="avatar"> Johannes_L:</div>
<blockquote>
<pre><code class="lang-auto">if loss.cpu().data.numpy()[0] &gt; 10000:
</code></pre>
<p>What is the loss.data doing? Somehow relating the loss to the input data?</p>
</blockquote>
</aside>
<p>I’m not sure specifically what you’re asking, but here are some<br>
comments:</p>
<p>In <code>loss.cpu().data</code>, <code>data</code> is a deprecated property that was used<br>
to unwrap a <code>Tensor</code> from a <code>Variable</code>.  (I believe this was prior to<br>
pytorch version 0.4.0.)</p>
<p>It’s just a coincidence that the character string “data” in<br>
<code>loss.cpu().data</code> happens to match that in <code>self.model(data)</code>.<br>
The two are used in different contexts, so the same character string<br>
is allowed to refer to two different completely unrelated things.</p>
<p>In general, the line:</p>
<pre><code class="lang-auto">if loss.cpu().data.numpy()[0] &gt; 10000:
</code></pre>
<p>seems to be garbled overkill.  There is no need to retrieve the loss<br>
value from the gpu or convert it to numpy to perform the <code>&gt;</code> test.</p>
<p>In current versions of pytorch:</p>
<pre data-code-wrap="python"><code class="lang-python">if  loss &gt; 10000:
</code></pre>
<p>suffices.</p>
<p>In old (pre-0.4.0?) versions you have to unwrap the <code>Tensor</code> from the<br>
<code>Variable</code>, and extract the python non-<code>Tensor</code> value from the <code>Tensor</code>.</p>
<p>Either:</p>
<pre data-code-wrap="python"><code class="lang-python">if  loss.data[0] &gt; 10000:
</code></pre>
<p>or:</p>
<pre data-code-wrap="python"><code class="lang-python">if  (loss &gt; 10000).data[0]:
</code></pre>
<p>works.</p>
<p>Best.</p>
<p>K. Frank</p>

username: Johannes_L
cooked: <p>Thanks for this thorough answer! It’s exactly what I needed to understand.<br>
As this comes from the Variable: in newer versions of PyTorch, is it better to just do your training and testing with tensors? Or is there any benefit from using Variable?</p>

username: KFrank
cooked: <p>Hi Johannes!</p>
<aside class="quote no-group" data-username="Johannes_L" data-post="3" data-topic="100655" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/johannes_l/48/30296_2.png" class="avatar"> Johannes_L:</div>
<blockquote>
<p>Or is there any benefit from using Variable?</p>
</blockquote>
</aside>
<p>With newer version of pytorch, don’t use <code>Variable</code>s.  They are<br>
deprecated and don’t actually do anything.  I think that<br>
<code>torch.autograd.Variable</code> is just a stub that is there for backward<br>
compatibility.</p>
<p>If you happen to be using an old version of pytorch, but don’t use<br>
<code>Variable</code>s, your code won’t silently fail – it will immediately complain<br>
if you try to do any kind of autograd stuff on a raw tensor.  So you<br>
don’t need to use <code>Variable</code>s “just to be safe.”</p>
<p>Best.</p>
<p>K. Frank</p>