What is loss.item()

hunar: <p>what does</p>
<blockquote>
<p>running_loss<br>
in this code ? i know it calculated the loss , and we need to get the probability .<br>
please take a look at the comment sections<br>
for e in range(epochs):<br>
running_loss = 0<br>
for images, labels in trainloader:<br>
# this loop through 938 images and labels (length of trainloader<br>
images = images.view(images.shape[0], -1)</p>
</blockquote>
<pre><code>    optimizer.zero_grad()
    
    output = model.forward(images)
    loss = criterion(output, labels)
    loss.backward()
    optimizer.step()
    
    running_loss += loss.item() #what this line does
else:
    print(f"Training loss: {running_loss/len(trainloader)}")  
</code></pre>
<p>thanks</p>

crowsonkb: <p>The <code>item()</code> method extracts the loss’s value as a Python float.</p>

hunar: <p>thank you , but why we need (running_loss += loss.item()) the trainloader has different numbers !? why we mixed the loss values of different numbers ?<br>
maybe a stupid question , sorry</p>

ptrblck: <p>You are calculating the running loss with this line of code so that you could later e.g. calculate the mean loss of that epoch.<br>
Let me know, if I misunderstood the question.</p>

hunar: <p>we iterate through the entire images and labels , per epoch</p>
<p>for images, labels in trainloader:</p>
<p>why we mixed the losses (losses of different images )! ?</p>
<p>{running_loss/len(trainloader)}</p>
<p>and calculated all together ?!<br>
sorry , i’m new , and i want to make my final project with pytorch</p>

ptrblck: <p>The average of the batch losses will give you an estimate of the “epoch loss” during training.<br>
Since you are calculating the loss anyway, you could just sum it and calculate the mean after the epoch finishes.<br>
This training loss is used to see, how well your model performs on the training dataset.</p>
<p>Alternatively you could also plot the batch loss values, but this is usually not necessary and will give you a lot of outputs.</p>

hunar: <p>thanks alot , one more question , do you think google colab GPU is fit to a project of facial recognition in real time</p>

ptrblck: <p>I’ve just used Colab in the past a few times for debugging purposes. Not sure if the runtime changed, but I don’t thing Colab is a good fit for “real time” deployment, e.g. since the notebook runtime is limited.</p>

hunar: <p>so what is your suggestion ?</p>

ptrblck: <p>You could use some cloud deployment service like Microsoft Azure, GCP, AWS or others.</p>

hunar: <p>thank you for your help    <img src="https://discuss.pytorch.org/images/emoji/apple/blush.png?v=9" title=":blush:" class="emoji" alt=":blush:"></p>

Sharafath_Mohammed: <p>Hi, on the topic of deployment, can you guide me to any tutorial, documentation, etc on using Azure DevOps (not Azure MLOps) for Pytorch deployment?</p>

ptrblck: <p>I’m unfortunately not experienced in deployment with Azure DevOps. <img src="https://discuss.pytorch.org/images/emoji/apple/confused.png?v=9" title=":confused:" class="emoji" alt=":confused:"></p>

Brando_Miranda: <p>How is that different from the <code>.data</code> field?</p>

ptrblck: <p>The <code>.data</code> attribute shouldn’t be used, as it might yield unwanted side effects.<br>
The right way to get a Python scalar is via <code>.item()</code>.</p>

Brando_Miranda: <p>Do you have a link discussing why (and what) there might be “unwanted side effects” more precisely?</p>
<p>Thanks!</p>

ptrblck: <p><a class="mention" href="/u/alband">@albanD</a> lists some aspects in <a href="https://discuss.pytorch.org/t/using-tensor-data/79640/4">this post</a>.</p>

saurabh-2905: <p>What if we use <code>.detach()</code> when working with Pytorch lightning. It would give us the data without any computational graph. Will it be correct to use <code>.detach()</code> instead of <code>.item()</code> ?</p>

ptrblck: <p><code>.detach()</code> will return a tensor, which is detached from the computation graph, while <code>.item()</code> will return the Python scalar. I don’t know how and where this is needed in PyTorch Lightning depending on the use case <code>detach()</code> might also work.</p>

saurabh-2905: <p>Thanks for such a quick reply. Yes, I understand this concept now. So because in Pytorch docs it is mentioned that tensors can be passed for logging, I am sure I can do the following:</p>
<pre data-code-wrap="python"><code class="lang-python">self.log('val_loss', loss.detach(), prog_bar=False, on_step=False, on_epoch=True, logger=True)
</code></pre>
<p>However, I am little unsure if the following implementation is correct in which I replaced   <code>.item()</code> with <code>.detach()</code> before the loss value is returned by the model. I am not getting any syntax error though but I am little worried if it would interfere with grad_calculation and affect the performance.</p>
<pre data-code-wrap="python"><code class="lang-python"># Loss : Mask outputs to ignore non-existing objects (except with conf. loss)
            loss_x = self.mse_loss(x[obj_mask], tx[obj_mask])
            loss_y = self.mse_loss(y[obj_mask], ty[obj_mask]) ....

            self.metrics = {
                "loss": to_cpu(total_loss).detach(),
                "x": to_cpu(loss_x).detach(),
                "y": to_cpu(loss_y).detach(), ..... }

            return output, total_loss

</code></pre>
<p>NOTE -  The reason I am trying to replace <code>.item()</code> is because I am training the model on multiple gpus and it was taking very long to train just one epoch. So while going through the pytorch lightning docs I came across this</p>
<blockquote>
<p>Don’t call .item() anywhere in your code. Use .detach() instead to remove the connected graph calls. Lightning takes a great deal of care to be optimized for this. <a href="https://pytorch-lightning.readthedocs.io/en/stable/performance.html" rel="noopener nofollow ugc">https://pytorch-lightning.readthedocs.io/en/stable/performance.html</a></p>
</blockquote>