Visualize live graph of lose and accuracy

Santhoshnumberone: <p>Is there a simple way to plot the loss and accuracy live during training in pytorch?</p>

ptrblck: <p>You could use Visdom for this. Have a look at <a href="https://github.com/facebookresearch/visdom/blob/master/example/demo.py">this example</a> to plot your stats live.<br>
Let me know if you encounter any issues.</p>

Santhoshnumberone: <p>New to pytorch.<br>
Too much of code, looks complicated.<br>
Unable to get what the actual code is doing.</p>

Santhoshnumberone: <p>Used this</p>
<pre><code class="lang-auto">import visdom

vis = visdom.Visdom()

#after calculating error per epoch
vis.line(Y=epoch_loss, opts=dict(showlegend=True))
</code></pre>
<p>I’m getting this error</p>
<blockquote>
<h2>Epoch 0/32</h2>
<p>train Loss: 2.4588 Acc: 0.5385<br>
Traceback (most recent call last):<br>
File “simpletest.py”, line 207, in <br>
model_ft = train_model(net,criterion,optimizer_ft,exp_lr_scheduler,num_epochs=33)<br>
File “simpletest.py”, line 175, in train_model<br>
vis.line(Y=epoch_loss, opts=dict(showlegend=True))<br>
File “/home/ffffff/.virtualenvs/LearnPytorch/lib/python3.6/site-packages/visdom/<strong>init</strong>.py”, line 288, in wrapped_f<br>
return f(*args, **kwargs)<br>
File “/home/ffffff/.virtualenvs/LearnPytorch/lib/python3.6/site-packages/visdom/<strong>init</strong>.py”, line 1171, in line<br>
assert Y.ndim == 1 or Y.ndim == 2, ‘Y should have 1 or 2 dim’<br>
AttributeError: ‘float’ object has no attribute ‘ndim’</p>
</blockquote>
<p>Shouldn’t the plot be error per epoch?</p>
<p>what would i use then matplotlib?</p>

ptrblck: <p>Try to pass the values as numpy arrays, e.g.</p>
<pre><code class="lang-python">viz = visdom.Visdom()
epoch_loss = np.array([1.2])
viz.line(epoch_loss)
</code></pre>

Santhoshnumberone: <p>This did work, found it here <a href="https://www.programcreek.com/python/example/92356/visdom.Visdom" rel="nofollow noopener">https://www.programcreek.com/python/example/92356/visdom.Visdom</a></p>
<pre><code class="lang-auto">import visdom

vis = visdom.Visdom()

loss_window = vis.line(
    Y=torch.zeros((1)).cpu(),
    X=torch.zeros((1)).cpu(),
    opts=dict(xlabel='epoch',ylabel='Loss',title='training loss',legend=['Loss']))

vis.line(X=torch.ones((1,1)).cpu()*epoch,Y=torch.Tensor([epoch_loss]).unsqueeze(0).cpu(),win=loss_window,update='append')
</code></pre>

Santhoshnumberone: <p>How can I plot two lines in the same window?</p>

ptrblck: <p>You have to concatenate the values as column vectors:</p>
<pre><code class="lang-python">Y = np.linspace(-5, 5, 100)
viz.line(
        Y=np.column_stack((Y * Y, np.sqrt(Y + 5))),
        X=np.column_stack((Y, Y)),
        opts=dict(markers=False),
)
</code></pre>

Neda: <p>I am trying to plot the train loss and validation loss in the same window in Visdom but it’s giving me one line instead of two and also wrong values.</p>
<p>this is the snippet I am using.</p>
<pre><code class="lang-auto">criterion = nn.NLLLoss()
optimizer = optim.Adam(model.parameters(), lr=0.001)  # 1e-3
#optimizer = optim.SGD(model.parameters(), lr=0.00001, momentum=0.9)  # 1e-3

# Decay LR by a factor of 0.1 every 4 epochs.
#step size: Period of learning rate decay.
#gamma = Multiplicative factor of learning rate decay. Default: 0.1, should float
scheduler = lr_scheduler.StepLR(optimizer, step_size=2, gamma=0.1)


def train_model(model, criterion, optimizer, scheduler, num_epochs=4):
    
    vis = visdom.Visdom()
    loss_window = vis.line(X=numpy.zeros((1 ,)),
                           Y=numpy.zeros((1)),
                           opts=dict(xlabel='epoch',
                                     ylabel='Loss',
                                     title='epoch Loss and accuracy',
                                     ))                    
    since = time.time()
        
    best_model_wts = copy.deepcopy(model.state_dict())
    best_acc = 0.0
    
    for epoch in range(num_epochs):  # loop over the dataset multiple times
        print('Epoch {}/{}'.format(epoch, num_epochs - 1))
        print('-' * 30)
        
        # Each epoch has a training and validation phase
        for phase in ['train', 'valid']:
            if phase == 'train':
                scheduler.step()
                
                model.train()  # Set model to training mode
            else:
                model.eval()   # Set model to evaluate mode
        
            train_loss = 0.0
            total_train = 0
            correct_train = 0
        
            #iterate over data
            for t_image, mask, image_paths, target_paths in dataLoaders[phase]:
                
                # get the inputs
                t_image = t_image.to(device)
                mask = mask.to(device)
                                
                 # zeroes the gradient buffers of all parameters
                optimizer.zero_grad()
                
                # forward
                # track history if only in train
                with torch.set_grad_enabled(phase == 'train'):
                    outputs = model(t_image) 
                    _, predicted = torch.max(outputs.data, 1)
                    loss = criterion(outputs, mask) # calculate the loss
            
                    # backward + optimize only if in training phase
                    if phase == 'train':
                        loss.backward() # back propagation
                        optimizer.step() # update gradients                        
            
                # accuracy
                train_loss += loss.item()
                total_train += mask.nelement()  # number of pixel in the batch
                correct_train += predicted.eq(mask.data).sum().item() # sum all precited pixel values            
                        
            epoch_loss = train_loss / len(dataLoaders[phase].dataset)
            epoch_acc = (correct_train / total_train)

            print('{} Loss: {:.4f} Acc: {:.4f}'.format(
                phase, epoch_loss, epoch_acc))

            X1 = numpy.ones((1, 1))*epoch
            Y1 = numpy.array([epoch_loss ])
            Y2 = numpy.array([epoch_loss ])
            vis.line(
                    X=numpy.column_stack((X1, X1)),
                    Y=numpy.column_stack((Y1, Y2)),
                    win=loss_window,
                    update='append')

             #deep copy the model
            if phase == 'val' and epoch_acc &gt; best_acc:
                best_acc = epoch_acc
                best_model_wts = copy.deepcopy(model.state_dict())
            
        print()
    time_elapsed = time.time() - since
    print('Training complete in {:.0f}m {:.0f}s'.format(
        time_elapsed // 60, time_elapsed % 60))
    print('Best val Acc: {:4f}'.format(best_acc))

    torch.save(model.state_dict(), 'train_valid_exp1.pth')   
    # load best model weights
    model.load_state_dict(best_model_wts)
    return model
           
if __name__=='__main__':
    train_model()
</code></pre>

ptrblck: <p>Currently you are plotting the same values twice (<code>Y1</code> and <code>Y2</code> have the same value).<br>
I would rather store the train and val losses in an array and plot them together after both were calculated.<br>
Let me know, if that helps.</p>

Neda: <p><a class="mention" href="/u/ptrblck">@ptrblck</a> Could you please give me further explanation or example? Do you mean should I store epoch_loss one time for train and val separately as a numpy array and  then plot them?</p>

ptrblck: <p>Yes, since you are currently overwriting it.<br>
Since you are using <code>phase</code> to switch between <code>train</code> and <code>valid</code>, you could store the losses in a dict using the same keys:</p>
<pre><code class="lang-python">losses = {}
for phase in ['train', 'valid']:
    ...
    losses[phase] = epoch_loss
</code></pre>

chenglu: <p>You could try <a href="https://github.com/louis-she/minetorch" rel="noopener nofollow ugc">Minetorch</a> , it’s a wrapper of PyTorch which support both <code>Tensorboard</code> and <code>Matplotlib</code> to visualize the loss and accuracy out of box. There’s a mnist sample you could try.</p>
<p>Some visualization</p>
<p>mnist example visualized with matplotlib<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/2X/b/b5d0710f8cd24c64c51bc331efed55b118ffae2a.png" data-download-href="https://discuss.pytorch.org/uploads/default/b5d0710f8cd24c64c51bc331efed55b118ffae2a" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/2X/b/b5d0710f8cd24c64c51bc331efed55b118ffae2a_2_690x292.png" alt="image" data-base62-sha1="pWp0ogxE5WUuSMlwshPSBmNAfai" width="690" height="292" srcset="https://discuss.pytorch.org/uploads/default/optimized/2X/b/b5d0710f8cd24c64c51bc331efed55b118ffae2a_2_690x292.png, https://discuss.pytorch.org/uploads/default/optimized/2X/b/b5d0710f8cd24c64c51bc331efed55b118ffae2a_2_1035x438.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/2X/b/b5d0710f8cd24c64c51bc331efed55b118ffae2a_2_1380x584.png 2x" data-dominant-color="FAFAFB"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1788×758 87.6 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>mnist example visualized with Tensorboard<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/2X/5/50f40a7d7ab0d9199dd7928ba4efb5f96da5d518.png" data-download-href="https://discuss.pytorch.org/uploads/default/50f40a7d7ab0d9199dd7928ba4efb5f96da5d518" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/2X/5/50f40a7d7ab0d9199dd7928ba4efb5f96da5d518_2_690x269.png" alt="image" data-base62-sha1="by93hNHp2xP039G8wfsgxUQKcVa" width="690" height="269" srcset="https://discuss.pytorch.org/uploads/default/optimized/2X/5/50f40a7d7ab0d9199dd7928ba4efb5f96da5d518_2_690x269.png, https://discuss.pytorch.org/uploads/default/optimized/2X/5/50f40a7d7ab0d9199dd7928ba4efb5f96da5d518_2_1035x403.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/2X/5/50f40a7d7ab0d9199dd7928ba4efb5f96da5d518_2_1380x538.png 2x" data-dominant-color="F8F8F8"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1394×544 92.5 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

ranklord: <p>Have a look at <a href="https://www.comet.ml" rel="nofollow noopener">https://www.comet.ml</a> and <a href="https://hyperdash.io" rel="nofollow noopener">https://hyperdash.io</a>. Both platforms provide real time monitoring of training parameters and are easy to use.</p>

Manu: <p>You can also try Pierogi (Free and Open Source) :<br>
<aside class="onebox whitelistedgeneric">
  <header class="source">
      <img src="https://github.githubassets.com/favicon.ico" class="site-icon" width="16" height="16">
      <a href="https://github.com/nalepae/pierogi/" target="_blank" rel="nofollow noopener">GitHub</a>
  </header>
  <article class="onebox-body">
    <img src="https://avatars1.githubusercontent.com/u/4943830?s=400&amp;amp;v=4" class="thumbnail onebox-avatar" width="60" height="60">

<h3><a href="https://github.com/nalepae/pierogi/" target="_blank" rel="nofollow noopener">nalepae/pierogi</a></h3>

<p>Plot your learning curves in real time in your web browser. - nalepae/pierogi</p>


  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>
</p>

AS23: <p><a href="https://github.com/stared/livelossplot" rel="nofollow noopener">https://github.com/stared/livelossplot</a> worked like a charm for me! Very easy to use compared to some of the alternatives listed here!</p>

shanaka_jayatilake: <p><a class="mention" href="/u/santhoshnumberone">@Santhoshnumberone</a> if you are new to pytorch i would suggest, <a href="https://medium.com/datadriveninvestor/visualizing-training-and-validation-loss-in-real-time-using-pytorch-and-bokeh-5522401bc9dd" rel="nofollow noopener">https://medium.com/datadriveninvestor/visualizing-training-and-validation-loss-in-real-time-using-pytorch-and-bokeh-5522401bc9dd</a></p>
<p>Very less functionalities… After starting from this, what I use is the tensorboard-pytorch module to get it to tensorboard-- <a href="https://pytorch.org/docs/stable/tensorboard.html" rel="nofollow noopener">https://pytorch.org/docs/stable/tensorboard.html</a></p>

saba: <p>Hi Ptrblck</p>
<p>Sorry to take your time. I face a big problem. Indeed, I want to show the graph of True positive rate (y axis) to false positive rates (x axis) . I define my threshold in the case that sensitivity is consistent an the std is for x axis means false positive rates. I need to show the graph (ROC) of mean and std and the shade between them. the problem is that all the defined rules are as :</p>
<pre><code class="lang-auto">plt.fill_between(x, y1, y2,
                 facecolor="orange", # The fill color
                 color='blue',       # The outline color
                 alpha=0.2) 
``` which give me the vise versa graph, indeed my y is consistent and the x has variation. If I change the x and y content the graph will be useless.  Is there any option in the pytorch to fill the shade between graphs regardless of which of them has variation, indeed my variation or std is in horizontal way not vertical way.</code></pre>

saba: <p>Hi Ptrblck</p>
<p>Do you know how I can save my figure in python to use and open them in Matlab. I need do some formatting in Matlab.</p>

ptrblck: <p>I think the easiest way would be to transform your tensors to numpy arrays via <code>arr = tensor.numpy()</code> and then to store these arrays using <a href="https://docs.scipy.org/doc/scipy/reference/generated/scipy.io.savemat.html"><code>scipy.io.savemat</code></a> to <code>.mat</code> files.</p>
<p>I’m not sure about your first question.<br>
Could you post some dummy tensors for <code>x</code>, <code>y1</code>, and <code>y2</code>?</p>