How to calculate loss per epoch

username: Jam
cooked: <p>Could someone please tell me which of the two methods is the correct way to calculate the loss?<br>
Below you can find my training loop.</p>
<pre><code class="lang-auto">    train_losses = 0
    valid_losses = 0
    avg_train_losses = []
    avg_valid_losses = [] 
    
    iteration = 0

    for epoch in range(num_epochs):
        
        model.train() #setting model in train mode
        train_data = reader.train_data(shuffle=True)
        y_pred = []
        y_true  = []
  
        for i in range(int(len(reader.train)/batch_size)):
            images, labels = get_batch(train_data, batch_size)
            
            # transform numpy arrays to torch tensors
            images = torch.from_numpy(images)
            labels = torch.from_numpy(labels)
    
            # move to GPU
            images, labels = images.to(device, dtype=torch.float), labels.to(device, dtype=torch.float)

            # Clear gradients w.r.t. parameters
            optimizer.zero_grad()

            # Reshape images
            images = images.view(-1, 256,256, 3)
            labels = labels.view(-1,1)            
            images = images.view(-1, 3, 256, 256)


            # Forward pass only to get logits/output
            outputs = model(images)

            # Calculate Loss: sigmoid BCELoss
            loss = criterion(outputs, labels)
            
            # Getting gradients w.r.t. parameters
            loss.backward()
            
            # train AUC
            predicted_train = outputs.view(-1)

            # Updating parameters
            optimizer.step()
            
            # detach from GPU
            loss = loss.detach().cpu().item()   
            # add to train_losses
            train_losses += loss
            
            #move to cpu
            images, labels, predicted_train = images.to("cpu", dtype=torch.float), labels.to("cpu", dtype=torch.float), predicted_train.to("cpu", dtype=torch.float)

            labels = labels.detach().numpy()
            predicted_train = predicted_train.detach().numpy()
            
            # add predictions and labels to list to later calculate AUC
            for i in range(labels.shape[0]):
                y_true.append(labels[i])
                y_pred.append(predicted_train[i])
    
        # calculate AUC of ROC
        roc_auc_train = roc_auc_score(y_true, y_pred)
            
        # print training statistics 
        train_loss = train_losses/len(reader.train)
        avg_valid_losses.append(train_loss)
</code></pre>
<p>Or do I have to calculate the variable train_losses like this:</p>
<pre><code class="lang-auto"> train_losses += loss * images.size(0) 
</code></pre>

username: Kapil_Rana
cooked: <p>You can sum all training or test loss. something like this<br>
test_loss = test_loss + loss.item()</p>
<p>After that, you can simply divide the total loss by the total number of batches per epoch.</p>
<p>alpha = (len(test_loader.dataset))/ batchsize<br>
print(test_loss)<br>
test_loss /= alpha</p>

username: Jam
cooked: <p>Hi!<br>
Thanks for your reply! So this is basically the same as doing the following right:</p>
<pre><code class="lang-auto">train_losses += loss.item() * images.size(0)
train_loss = train_losses/len(reader.train)
</code></pre>
<p>With  images.size(0) corresponding to the batch_size.</p>

username: Kapil_Rana
cooked: <p>Yes, it looks similar.</p>