How to plot training and testing graphs for this pytorch model here?

krishna511: <p>Hi there I am training a model for the function train and test given here, finally called the main function. I need to see the training and testing graphs as per the epochs for observing the model performance. Can someone extend the code here?</p>
<pre><code class="lang-auto">import torch
from torch.utils.data import DataLoader as DL
from torch import nn, optim

import numpy as np
import matplotlib.pyplot as plt
from sklearn.metrics import accuracy_score
from torch.utils.data import DataLoader
from SpeechDataGenerator import SpeechDataGenerator
from models.Emo_Raw_TDNN_StatPool import Emo_Raw_TDNN
import fit_predict as fp
from utils import utils_wav

from sklearn.metrics import confusion_matrix
from utils.utils_wav import speech_collate



torch.manual_seed(0)
torch.cuda.manual_seed(0)


def breaker():
    print("\n" + 50*"-" + "\n")


tr_info_file = "meta/training.txt"
ts_info_file = "meta/testing.txt"

tr_batch_size = 16
ts_batch_size = 16
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

epochs = 100

if __name__ == "__main__":

    tr_audio_links = [line.rstrip('\n').split(' ')[0] for line in open(tr_info_file)]
    ts_audio_links = [line.rstrip('\n').split(' ')[0] for line in open(ts_info_file)]

   

    tr_raw = []
    for i in range(len(tr_audio_links)):
        tr_raw.append(utils_wav.load_data_wav(tr_audio_links[i]))
    tr_raw = np.array(tr_raw)

    ts_raw = []
    for i in range(len(ts_audio_links)):
        ts_raw.append(utils_wav.load_data_wav(ts_audio_links[i]))
    ts_raw = np.array(ts_raw)





    dataset_train = SpeechDataGenerator(manifest='meta/training.txt',mode='train')
    dataloader_train = DataLoader(dataset_train, batch_size=tr_batch_size ,shuffle=True,collate_fn=speech_collate) 

    dataset_test = SpeechDataGenerator(manifest='meta/testing.txt',mode='test')
    dataloader_test = DataLoader(dataset_test, batch_size=ts_batch_size ,shuffle=False,collate_fn=speech_collate) 
   

    torch.manual_seed(0)
  
    model = TDNN(1, 4).to(device)
    optimizer = optim.Adam(model.parameters(), lr=0.0001, weight_decay=0.0, betas=(0.9, 0.98), eps=1e-9)
    loss_fun = nn.CrossEntropyLoss()
    from time import time
    import os

    start_time=time()
    def train(dataloader_train,epoch):
        train_loss_list=[]
        full_preds=[]
        full_gts=[]
        model.train()
        est=time()
        for i_batch, sample_batched in enumerate(dataloader_train):
            
            features = torch.from_numpy(np.asarray([torch_tensor.numpy() for torch_tensor in sample_batched[0]])).float()
            labels = torch.from_numpy(np.asarray([torch_tensor[0].numpy() for torch_tensor in sample_batched[1]])).long()
            features, labels = features.to(device),labels.to(device)
            features.requires_grad = True
            optimizer.zero_grad()
            pred_logits = model(features)
            #### CE loss
            loss = loss_fun(pred_logits,labels)
            loss.backward()
            optimizer.step()
            train_loss_list.append(loss.item())
            #train_acc_list.append(accuracy)
            if i_batch%100==0:
                print('Loss {} after {} iteration'.format(np.mean(np.asarray(train_loss_list)),i_batch))
            
            predictions = np.argmax(pred_logits.detach().cpu().numpy(),axis=1)
            for pred in predictions:
                full_preds.append(pred)
            for lab in labels.detach().cpu().numpy():
                full_gts.append(lab)
                
        mean_acc = accuracy_score(full_gts,full_preds)
        mean_loss = np.mean(np.asarray(train_loss_list))
       
    
        print('Total training loss {} and training Accuracy {} after {} epochs'.format(mean_loss,mean_acc,epoch))
        print('Time Taken for Epoch: {:.2f} minutes'.format((time() - est)/60))
    
    
    def test(dataloader_test,epoch):
        model.eval()
        with torch.no_grad():
            val_loss_list=[]
            full_preds=[]
            full_gts=[]
            for i_batch, sample_batched in enumerate(dataloader_test):
                features = torch.from_numpy(np.asarray([torch_tensor.numpy() for torch_tensor in sample_batched[0]])).float()
                labels = torch.from_numpy(np.asarray([torch_tensor[0].numpy() for torch_tensor in sample_batched[1]])).long()
                features, labels = features.to(device),labels.to(device)
                pred_logits = model(features)
                #### CE loss
                loss = loss_fun(pred_logits,labels)
                val_loss_list.append(loss.item())
                #train_acc_list.append(accuracy)
                predictions = np.argmax(pred_logits.detach().cpu().numpy(),axis=1)
                for pred in predictions:
                    full_preds.append(pred)
                for lab in labels.detach().cpu().numpy():
                    full_gts.append(lab)
                    
            mean_acc = accuracy_score(full_gts,full_preds)
            mean_loss = np.mean(np.asarray(val_loss_list))
           
    
            print('Total Test loss {} and Test accuracy {} after {} epochs'.format(mean_loss,mean_acc,epoch))
            
            model_save_path = os.path.join('save_model', 'best_check_point_'+str(epoch)+'_'+str(mean_acc))
            state_dict = {'model': model.state_dict(),'optimizer': optimizer.state_dict(),'epoch': epoch}
            torch.save(state_dict, model_save_path)
            
if __name__ == '__main__':
    for epoch in range(epochs):
        train(dataloader_train,epoch)
        test(dataloader_test,epoch)
   '''

please guide 
I know the first need is to save the model accuracies and loss, for this model need to save But how to plot this using skelearn or any other way that I need to know here.
Thanks</code></pre>

ZimoNitrome: <p>Look into <a href="https://pytorch.org/tutorials/recipes/recipes/tensorboard_with_pytorch.html" rel="noopener nofollow ugc">tensorboard</a></p>

krishna511: <p><a class="mention" href="/u/zimonitrome">@ZimoNitrome</a> Thanks<br>
I saw this link, did all code change in my training and testing for plotting training and testing loss and ACC by instantiating to SummaryWriter(), But after successfully installing tensorboard by command<br>
<code> !pip install tensorboard</code></p>
<p>When I run<br>
<code> tensorboard --logdir runs</code><br>
It throw error as</p>
<p><code>invalid syntax</code><br>
Also tried<br>
<code>tensorboard --logdir=runs</code><br>
Then the error is</p>
<p><code>cannot assign to operator</code><br>
Please guide what to do.</p>

krishna511: <p><a class="mention" href="/u/ptrblck">@ptrblck</a> Hi sir, sorry for this tag <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=10" title=":slight_smile:" class="emoji" alt=":slight_smile:"><br>
But I think you are the life saver here. I am using tensorboard for graph plots to see my training and testing<code> acc</code> and <code>loss</code> progress. Due to limited GPU support on my laptop I am now using colab for same. Sir I can see the <code>runs</code> folder there. But  when I call</p>
<p><code>!tensorboard --logdir=runs</code><br>
and then opening this <a href="http://localhost:6006/" rel="noopener nofollow ugc">link</a> says</p>
<p><code># No dashboards are active for current data set</code></p>
<p>sir, please help. I have to use these plots in my writing.</p>

ptrblck: <p>You could check if the folder contains some Tensorboard-related files and if so maybe refresh Tensorboard itself.<br>
Personally, I haven’t used Tensorboard much as I was always running into similar trouble of displaying the data “live” (I must admit that I’ve tried it a few years ago so things might have changed/improved by now).<br>
Back then I’ve stuck to <a href="https://github.com/fossasia/visdom"><code>visdom</code></a>, which didn’t support all the features of Tensorboard but was a better fit for my use cases (more flexibility to display plots and they were shown immediately).<br>
I’m not sure if you need to use Tensorboard or if displaying the curves via <code>matplotlib</code> in your notebook would also work.</p>

hema_rathore: <p>yes, sir <a class="mention" href="/u/ptrblck">@ptrblck</a> I just wanted to see the training plots for loss and accuracy of my model.<br>
Are there any used examples for visdom?</p>
<p>It is straight and simple in tensorboard but I don’t know why is it not working in colab, though I can see the <code>runs</code> folder made using the writer(). I cant find any forum or resource to discuss the issue either.</p>

ptrblck: <p><code>visdom</code> provides some <a href="https://github.com/fossasia/visdom/tree/master/example">examples</a> but I don’t know what the current status of its development is and haven’t used it in a while.</p>
<p>An easy way to plot your loss curves would be to just use <code>matplotlib</code> in the notebook as mentioned before.</p>

hema_rathore: <p>Sir <a class="mention" href="/u/ptrblck">@ptrblck</a> I am using the code mentioned above. I don’t know know how to save these scaler values ```<br>
<code>mean_acc</code> and <code>mean_loss</code> in training and testing. and finally, plot them in two plots one for loss and the other for accuracy.</p>

krishna511: <p>Hi <a class="mention" href="/u/hema_rathore">@hema_rathore</a> have you done the plotting also? I mean did you get these plots there?</p>

krishna511: <p><a class="mention" href="/u/ptrblck">@ptrblck</a> yes sir, I tried <code>matplotlib</code></p>
<pre><code class="lang-auto">plt.plot(epochs, mean_loss, 'g', label='Training loss')
plt.plot(epochs, mean_acc, 'b', label='Training  acc')
plt.title('Training  loss and acc')
plt.xlabel('Epochs')
plt.ylabel('Loss and acc')
plt.legend()
plt.show()
</code></pre>
<p>but in my code (in initial question dont know how to use it) as above code in<code> train</code> and <code>test</code> function can plot loss and acc for training and testing. But I want to plot loss for train and test in a single plot and similarly acc for train and test.<br>
Regards</p>

my3bikaht: <p>You can try wandb instead of tensorboard</p>

ptrblck: <p>You could return the tensors or numpy arrays containing the loss values from the <code>train</code> and <code>test</code> functions and plot it in a single <code>plt.plot</code> in the main script.</p>

krishna511: <p>Sir I did that. Using this code out of <code>main</code> unction.</p>
<pre><code class="lang-auto">plt.plot(Loss_train, 'g', label='Training loss')
plt.plot(Loss_test, 'b', label='Testing loss')
plt.title('Training  &amp; Testing loss')
plt.xlabel('Epochs')
plt.ylabel('Loss')
plt.legend()
plt.show()

plt.plot(Acc_train, 'g', label='Training  acc')
plt.plot(Acc_test, 'b', label='Testing  acc')
plt.title('Training  &amp; Testing acc')
plt.xlabel('Epochs')
plt.ylabel('Acc')
plt.legend()
plt.show()
</code></pre>
<p>I have saved the tensors <code>Acc_train, Acc_test, Loss_train</code> and <code>Loss_test</code>. The only confusion is, here the epochs are plotted automatically on x -axis?  As I am not giving epoch here. Result for 5 epochs is plotted as below</p>
<p><img src="https://discuss.pytorch.org/uploads/default/original/3X/e/3/e3b7da269d8f83d2f826a80f67cf0feba2f3d4ce.png" alt="image" data-base62-sha1="wuuhO2gFLMc7hThAH29snQOigEe" width="398" height="278"></p>

ptrblck: <p>Yes, if you don’t pass an <code>x</code> array but only the <code>y</code> array to <code>plot</code> the x-axis will be set as <code>np.arange(len(y))</code>. Refer to the <a href="https://matplotlib.org/stable/api/_as_gen/matplotlib.pyplot.plot.html"><code>matplotlib</code> docs</a> for more information.</p>

ivangrov: <p>Hey Krishna! I work for Weights &amp; Biases and the problem you’re trying to solve with experiment tracking seems like the perfect use-case for the product. Basically, you pass one line of code <code>wandb.watch(model, log_freq=100)</code> (wandb is the name of the Python client) and all your training metrics/test metrics, as well, as CPU/GPU usage all get pulled into a single dashboard where you can compare them side-by-side with interactive charts. Beyond that, W&amp;B also tracks your different models’ hyperparameters, so you don’t have to worry about keeping track of them also. Basically, it’s just a lot more convenient than using matplotlib to track training metrics.</p>
<p>Here are the <a href="https://docs.wandb.ai/guides/integrations/pytorch" rel="noopener nofollow ugc">W&amp;B PyTorch docs</a> you can use to get started and do let me know if I you have any questions.</p>