Finetune example doubt at plot curve

username: rajasekhar
cooked: <p><a class="mention" href="/u/ptrblck">@ptrblck</a>: hi!</p>
<p>I’m following the pytorch/finetuning tutoral. Below is some script for plot curve for validation loss</p>
<pre><code class="lang-auto">ohist = []
shist = []

ohist = [h.cpu().numpy() for h in hist]
shist = [h.cpu().numpy() for h in scratch_hist]

plt.title("Validation Accuracy vs. Number of Training Epochs")
plt.xlabel("Training Epochs")
plt.ylabel("Validation Accuracy")
plt.plot(range(1,num_epochs+1),ohist,label="Pretrained")
plt.plot(range(1,num_epochs+1),shist,label="Scratch")
plt.ylim((0,1.))
plt.xticks(np.arange(1, num_epochs+1, 1.0))
plt.legend()
plt.show()

</code></pre>
<p>could help me to understand this, as i didn’t get what’s going here: <code>plt.plot(range(1,num_epochs+1),ohist,label="Pretrained")</code>,</p>

username: ptrblck
cooked: <p>The code creates a plot for the validation accuracy of the fine-tuned model (<code>ohist</code>) and the model trained from scratch (<code>shist</code>).<br>
Both accuracies will be visualized in a plot where the epochs are given in the x-axis and the accuracies in the y-axis.</p>
<p>PS: I’m not a fan of tagging certain people, as this might discourage others to post an answer. <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=9" title=":wink:" class="emoji" alt=":wink:"></p>

username: rajasekhar
cooked: <p>Sure. Won’t tag again. So, here’s where <code>hist</code> came from:</p>
<pre><code class="lang-auto">model_ft, hist = train_model(model_ft, dataloaders_dict, criterion, optimizer_ft, num_epochs=num_epochs, is_inception=(model_name=="inception"))
</code></pre>
<p>and the <code>train_model</code> function:</p>
<pre><code class="lang-auto">def train_model(model, dataloaders, criterion, optimizer, num_epochs, is_inception=False):
    since = time.time()

    val_acc_history = []
    
    best_model_wts = copy.deepcopy(model.state_dict())
    best_acc = 0.0

    for epoch in range(num_epochs):
        print('Epoch {}/{}'.format(epoch, num_epochs - 1))
        print('-' * 10)

        # Each epoch has a training and validation phase
        for phase in ['train', 'val']:
            if phase == 'train':
                model.train()  # Set model to training mode
            else:
                model.eval()   # Set model to evaluate mode

            running_loss = 0.0
            running_corrects = 0

            # Iterate over data.
            for inputs, labels in dataloaders[phase]:
                inputs = inputs.to(device)
                labels = labels.to(device)

                # zero the parameter gradients
                optimizer.zero_grad()

                # forward
                # track history if only in train
                with torch.set_grad_enabled(phase == 'train'):
                    # Get model outputs and calculate loss
                    # Special case for inception because in training it has an auxiliary output. In train
                    #   mode we calculate the loss by summing the final output and the auxiliary output
                    #   but in testing we only consider the final output.
                    if is_inception and phase == 'train':
                        # From https://discuss.pytorch.org/t/how-to-optimize-inception-model-with-auxiliary-classifiers/7958
                       outputs, aux_outputs = model(inputs)
                       loss1 = criterion(outputs, labels)
                       loss2 = criterion(aux_outputs, labels)
                       loss = loss1 + 0.4*loss2
                    else:
                        outputs = model(inputs)
                        loss = criterion(outputs, labels)

                    _, preds = torch.max(outputs, 1)

                    # backward + optimize only if in training phase
                    if phase == 'train':
                        loss.backward()
                        optimizer.step()

                # statistics
                running_loss += loss.item() * inputs.size(0)
                running_corrects += torch.sum(preds == labels.data)

            epoch_loss = running_loss / len(dataloaders[phase].dataset)
            epoch_acc = running_corrects.double() / len(dataloaders[phase].dataset)

            print('{} Loss: {:.4f} Acc: {:.4f}'.format(phase, epoch_loss, epoch_acc))

            # deep copy the model
            if phase == 'val' and epoch_acc &gt; best_acc:
                best_acc = epoch_acc
                best_model_wts = copy.deepcopy(model.state_dict())
            if phase == 'val':
                val_acc_history.append(epoch_acc)

        print()

    time_elapsed = time.time() - since
    print('Training complete in {:.0f}m {:.0f}s'.format(time_elapsed // 60, time_elapsed % 60))
    print('Best val Acc: {:4f}'.format(best_acc))

    # load best model weights
    model.load_state_dict(best_model_wts)
    return model, val_acc_history
</code></pre>
<p>That plots only the validation accuracy. But I want plots such that it has to plot</p>
<ol>
<li>Training Acc/Loss Vs. Epochs</li>
<li>Validation Acc/Loss Vs.Epochs</li>
</ol>
<p>any suggestions?</p>

username: ptrblck
cooked: <p>You would have to store <code>epoch_acc</code> in another list for the training phase:</p>
<pre><code class="lang-python">def train_model(...):
    train_acc_history = []
    val_acc_history = []
    ...

    for epoch in range(num_epochs):
        for phase in ['train', 'val']:
            ...

            if phase == 'val':
                val_acc_history.append(epoch_acc)
            elif phase == 'train':
                train_acc_history.append(epoch_acc)
            ...

    return model, val_acc_history, train_acc_history
</code></pre>

username: rajasekhar
cooked: <p>Thanks. I tried by adding these lines:</p>
<pre><code class="lang-auto">            if phase == 'val' and epoch_acc &gt; best_acc:
                best_acc = epoch_acc
                best_model_wts = copy.deepcopy(model.state_dict())
            if phase == 'val':
                val_acc_history.append(epoch_acc)
                val_acc_history.append(epoch_loss)
            elif phase == 'train':
                train_acc_history.append(epoch_acc)
                train_acc_history.append(epoch_loss)

....
....
model_ft, hist = train_model(model_ft, dataloaders_dict, criterion, optimizer_ft, num_epochs=num_epochs, is_inception=(model_name=="inception"))
torch.save(model_ft.state_dict(), 'googlenet/standard_googlenet.pth')

ohist = []
#shist = []
ohist = [h.cpu.numpy() for h in hist] 
#shist = [h.cpu().numpy() for h in train_model.epoch_loss]

plt.title("Loss&amp;Acc Vs. Number of Training Epochs")
plt.xlabel(" Epochs")
plt.ylabel("Accuracy &amp; Loss")
plt.plot(range(1,num_epochs+1),ohist,label="Training Accuracy")
plt.plot(range(1,num_epochs+1),ohist,label="Training Loss")
plt.plot(range(1,num_epochs+1),ohist,label="Validation Accuracy")
plt.plot(range(1,num_epochs+1),ohist,label="Validation Loss")
plt.ylim((0,1.))
plt.xticks(np.arange(1, num_epochs+1, 1.0))
plt.legend()
plt.savefig('plots.png')
plt.show()

</code></pre>
<p>but it throws an error:</p>
<pre><code class="lang-auto">Traceback (most recent call last):
  File "ftune.py", line 284, in &lt;module&gt;
    model_ft, hist = train_model(model_ft, dataloaders_dict, criterion, optimizer_ft, num_epochs=num_epochs, is_inception=(model_name=="inception"))
ValueError: too many values to unpack (expected 2)
</code></pre>
<p>I tried also TensorboardX, like these snippets:</p>
<pre><code class="lang-auto">from tensorboardX import SummaryWriter
writer = SummaryWriter('runs')
...
...
def train_model(train_loader, model, criterion, optimizer, epoch):
        if i % args.print_freq == 0:
            print('Epoch: [{0}][{1}/{2}]\t'
                  'Time {batch_time.val:.3f} ({batch_time.avg:.3f})\t'
                  'Data {data_time.val:.3f} ({data_time.avg:.3f})\t'
                  'Loss {loss.val:.4f} ({loss.avg:.4f})\t'
                  'Prec@1 {top1.val:.3f} ({top1.avg:.3f})\t'
                  'Prec@5 {top5.val:.3f} ({top5.avg:.3f})'.format(
                   epoch, i, len(train_loader), batch_time=batch_time,
                   data_time=data_time, loss=losses, top1=top1, top5=top5))
            niter = (epoch * len(train_loader))+i
            writer.add_scalar('Train/Loss', losses.val, niter)
            writer.add_scalar('Train/Prec@1', top1.val, niter)
            writer.add_scalar('Train/Prec@5', top5.val, niter)

def validate(val_loader, model, criterion):

        if i % args.print_freq == 0:
            print('Test: [{0}/{1}]\t'
                  'Time {batch_time.val:.3f} ({batch_time.avg:.3f})\t'
                  'Loss {loss.val:.4f} ({loss.avg:.4f})\t'
                  'Prec@1 {top1.val:.3f} ({top1.avg:.3f})\t'
                  'Prec@5 {top5.val:.3f} ({top5.avg:.3f})'.format(
                   i, len(val_loader), batch_time=batch_time, loss=losses,
                   top1=top1, top5=top5))
            niter = epoch*len(train_loader)+i  
            writer.add_scalar('Test/Loss', losses.val, niter)
            writer.add_scalar('Test/Prec@1', top1.val, niter)
            writer.add_scalar('Test/Prec@5', top5.val, niter)

</code></pre>
<p>but in <code>validate</code> function it throws an error like epoch is not defined etc.</p>
<p>could you share some snippets for TensorboardX?</p>

username: ptrblck
cooked: <p>The first error is thrown, since you are now returning three objects/values:</p>
<pre><code class="lang-python">return model, val_acc_history, train_acc_history
</code></pre>
<p>The second is thrown, because <code>epoch</code> is indeed not defined in <code>validate</code>, so you could pass it as an argument to the function.</p>

username: rajasekhar
cooked: <p>Can I export validation plot data to a a csv file?</p>
<p>Any example?</p>

username: ptrblck
cooked: <p>This should be possible using <code>pandas</code>.<br>
You would have to create a <code>pd.DataFrame</code> containing your validation values and save it using <code>df.to_csv</code>.</p>

username: Niharika_Bhattacharj
cooked: <p>Hi, I am running into the same problem as I am a beginner. I was wondering how do you change the code so it expects all three objects. Even though you include all three objects at the return statement why does it still say it expects only 2.</p>
<aside class="quote no-group" data-username="ptrblck" data-post="6" data-topic="43245">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar"> ptrblck:</div>
<blockquote>
<p>return model, val_acc_history, train_acc_history</p>
</blockquote>
</aside>

username: ptrblck
cooked: <p>I’m not sure I understand the issue correctly.<br>
Could you post the error message or explain your use case a bit? <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"><br>
Which function expects two arguments?</p>

username: Niharika_Bhattacharj
cooked: <p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/c/5/c56bab084b4b541a1e52b4c0ecb910b4c9cb40f3.png" data-download-href="https://discuss.pytorch.org/uploads/default/c56bab084b4b541a1e52b4c0ecb910b4c9cb40f3" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/c/5/c56bab084b4b541a1e52b4c0ecb910b4c9cb40f3_2_690x146.png" alt="image" data-base62-sha1="sasJQFEiRlUHDunsCjQpJbfZqSf" width="690" height="146" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/c/5/c56bab084b4b541a1e52b4c0ecb910b4c9cb40f3_2_690x146.png, https://discuss.pytorch.org/uploads/default/optimized/3X/c/5/c56bab084b4b541a1e52b4c0ecb910b4c9cb40f3_2_1035x219.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/c/5/c56bab084b4b541a1e52b4c0ecb910b4c9cb40f3.png 2x" data-dominant-color="F6F2F2"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1216×259 26.5 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>I am just having trouble creating a graph for my loss values. The tutorial (<a href="https://pytorch.org/tutorials/beginner/finetuning_torchvision_models_tutorial.html" class="inline-onebox" rel="noopener nofollow ugc">Finetuning Torchvision Models — PyTorch Tutorials 2.2.0+cu121 documentation</a>) only shows how to create the graph for validation accuracy, however I wanted to plot the validation loss as well. The error I am getting occurs after I try to make a list for a variable called val_loss_history, similar to how they wrote val_acc_history.</p>
<p>Apologies in advance if this is confusing.</p>

username: Niharika_Bhattacharj
cooked: <p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/e/e/ee67738246c7f45160778f03f6f9c1042ec1dbf3.png" data-download-href="https://discuss.pytorch.org/uploads/default/ee67738246c7f45160778f03f6f9c1042ec1dbf3" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/e/e/ee67738246c7f45160778f03f6f9c1042ec1dbf3.png" alt="image" data-base62-sha1="y11hqL4gFA3eLEdNf6v0aduUjHZ" width="567" height="500" data-dominant-color="F0F0F0"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">879×775 35.2 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

username: ptrblck
cooked: <p>Since you are returning 3 values in <code>train_model</code>, you would also have to assign them:</p>
<pre><code class="lang-python">model_ft, hist, losses = train_model(model_ft, dataloaders_dict, criterion, optimizer_ft, num_epochs=num_epochs, is_inception=(model_name=="inception"))
</code></pre>
<p>PS: you can add code snippets by wrapping them in three backticks ``` <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=9" title=":wink:" class="emoji" alt=":wink:"></p>

username: Niharika_Bhattacharj
cooked: <p>Thank you, I was also wondering how I could save the trained model after every epoch. I am currently saving it like so:  <div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/1/8/18b0ffc4694f306d7b42f59d56277d1b5da51dd5.png" data-download-href="https://discuss.pytorch.org/uploads/default/18b0ffc4694f306d7b42f59d56277d1b5da51dd5" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/1/8/18b0ffc4694f306d7b42f59d56277d1b5da51dd5.png" alt="image" data-base62-sha1="3wqG2AJnMc9GyXAINse96QpGBLv" width="690" height="360" data-dominant-color="EFEFEF"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1063×556 36.1 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div> , but that saves the entire model.</p>

username: ptrblck
cooked: <p>That’s the right approach of saving the model’s <code>state_dict</code>.<br>
Why is it unexpected to save the entire model?</p>

username: Niharika_Bhattacharj
cooked: <p>I want to save it using checkpoints as my best val accuracy is at epoch 10, and I trained to epoch 14. I was looking at this discussion on how to do it <a href="https://discuss.pytorch.org/t/how-resume-the-saved-trained-model-at-specific-epoch/35823/6" class="inline-onebox">How resume the saved trained model at specific epoch</a>, however I don’t know how to change my line (listed below) such that it saves the entire model using those checkpoints at every epoch.</p>
<p>torch.save(model_ft.state_dict(), ‘C:\Users\Niharika\Desktop\CavitiesCNN\results6\model.pth’)<br>
torch.save(optimizer.state_dict(), ‘C:\Users\Niharika\Desktop\CavitiesCNN\results6\optimizer.pth’).</p>

username: ptrblck
cooked: <p>If you want to store a checkpoint for every epoch, you could add the epoch counter to the name:</p>
<pre><code class="lang-python">torch.save(model_ft.state_dict(), 'C:\Users\Niharika\Desktop\CavitiesCNN\results6\model_epoch{}.pth'.format(epoch))
</code></pre>