Graph Visualization

alan_ayu: <p>Does PyTorch have any tool,something like TensorBoard in TensorFlow,to do graph visualization to help users understand and debug network?</p>

mattmacy: <p>Not that I am aware of. However, due to its dynamic nature, it is much easier to debug a network in pytorch than tensorflow.<br>
As one commenter on <a href="https://www.reddit.com/r/MachineLearning/comments/5w3q74/d_so_pytorch_vs_tensorflow_whats_the_verdict_on/?st=j0yoypcr&amp;sh=a5a4df61" rel="nofollow noopener">Reddit</a> opines:<br>
“Debugging is easier because a specific line in your specific code (not something deep under your sess.run() that worked with a large/generated Graph object) fails. Your stack traces don’t fill up three screens and make you play the “spot the actual error!” scrolling game.”</p>
<p>I’ve found it fairly simple to just instrument the code as needed when things don’t go as planned.</p>

xwgeng: <p>look at this post <a href="https://discuss.pytorch.org/t/print-autograd-graph/692/2?u=xwgeng">Graph visualization</a></p>

alan_ayu: <p>got it.thank you:+1:</p>

zsp1197: <p>Try VisualDL: <a href="https://github.com/PaddlePaddle/VisualDL" rel="nofollow noopener">https://github.com/PaddlePaddle/VisualDL</a></p>
<p>VisualDL could produce graph with the ONNX file.</p>

nullgeppetto: <p>Hi <a class="mention" href="/u/zsp1197">@zsp1197</a>, have you managed to plot the graph in VidualDL? I’m trying to do that for a very simple network, following <a href="https://github.com/PaddlePaddle/VisualDL/blob/develop/demo/pytorch/TUTORIAL_EN.md" rel="nofollow noopener">this</a> example.</p>
<p>What I understand is that I need to export my model in a onnx file, as follows:</p>
<pre><code class="lang-auto">import torch.onnx
dummy_input = Variable(torch.randn(4, 3, 32, 32))
torch.onnx.export(net, dummy_input, "model.onnx")
</code></pre>
<p>Then, VisualDL’s documentation suggests <a href="https://github.com/PaddlePaddle/VisualDL#launch-visual-dl" rel="nofollow noopener">this</a>. However, in such a case I don’t what logdir to use.</p>

lutzroeder: <p><a href="https://github.com/lutzroeder/netron" rel="nofollow noopener">Netron</a> can open ONNX files and supports reading basic sequential <code>pth</code> files.</p>
<p>Also check out this thread: <a href="https://discuss.pytorch.org/t/print-autograd-graph">Print Autograd Graph</a>.</p>

spt: <p>I have used Netron to visualize ONNX graphs. If you can export your model to ONNX successfully I would recommend giving it a try.</p>

waleeka: <p>Check out <a href="https://github.com/waleedka/hiddenlayer" rel="nofollow noopener">HiddenLayer</a>. I wrote this tool to visualize network graphs, and more specifically to visualize them in a way that is easier to understand. It merges related nodes together (e.g. Conv/Relu/MaxPool) and folds repeating blocks into one box and adds a x3 to imply that the block repeats 3 times rather than drawing it three times. This helps when you try to draw big networks, such as ResNet and Inception.</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://raw.githubusercontent.com/waleedka/hiddenlayer/master/assets/readme_graph.png" title="readme_graph.png" rel="nofollow noopener"><img src="https://raw.githubusercontent.com/waleedka/hiddenlayer/master/assets/readme_graph.png" alt="https://github.com/waleedka/hiddenlayer" width="511" height="500"><div class="meta">
<svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use xlink:href="#far-image"></use></svg><span class="filename">readme_graph.png</span><span class="informations">591×578</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use xlink:href="#discourse-expand"></use></svg>
</div></a></div></p>

lorinczszabolcs: <p>Great visualization tool! I just tried to visualize my slightly more complex models, and it works seamlessly.</p>

tariqdaouda: <p>I’ve started a new project for experiment documentation (<a href="https://github.com/tariqdaouda/Spyrkle" rel="nofollow noopener">Spyrkle</a>).</p>
<p>It should be able to generate an interactive graph in an HTML page that you can style using CSS.<br>
<aside class="onebox githubblob">
  <header class="source">
      <a href="https://github.com/tariqdaouda/Spyrkle/blob/master/demos/demo_pyTorch_graph_only.py" target="_blank" rel="nofollow noopener">github.com</a>
  </header>
  <article class="onebox-body">
    <h4><a href="https://github.com/tariqdaouda/Spyrkle/blob/master/demos/demo_pyTorch_graph_only.py" target="_blank" rel="nofollow noopener">tariqdaouda/Spyrkle/blob/master/demos/demo_pyTorch_graph_only.py</a></h4>
<pre><code class="lang-py">import Spyrkle.notebook as notebook
import Spyrkle.pages.core_pages as pages
import Spyrkle.pages.graph_pages as graphs
import Spyrkle.pages.graphs_more as gmore

import numpy
import torch
import torch.nn as nn
import torch.nn.functional as F
import torch.optim as optim


# Define model
class TheModelClass(nn.Module):
    def __init__(self):
        super(TheModelClass, self).__init__()
        self.conv1 = nn.Conv2d(3, 6, 5)
        self.pool = nn.MaxPool2d(2, 2)
        self.conv2 = nn.Conv2d(6, 16, 5)
        self.fc1 = nn.Linear(16 * 5 * 5, 120)
</code></pre>

  This file has been truncated. <a href="https://github.com/tariqdaouda/Spyrkle/blob/master/demos/demo_pyTorch_graph_only.py" target="_blank" rel="nofollow noopener">show original</a>

  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>
</p>

abhilash_pinky: <p><a class="mention" href="/u/tariqdaouda">@tariqdaouda</a> great work i have few queries like can we use this for custom pytorch object detection architecture ??</p>

Lindosun: <p>Maybe you can try TensorBoardX.<br>
It is TensorBoard for PyTorch.</p>
<pre><code class="lang-auto">from tensorboardX import SummaryWriter
writer = SummaryWriter(path)
writer.add_scalars(path,  {'Data1' :value1, 'Data2' :value2}, index)
writer.close()
</code></pre>

tariqdaouda: <p><a class="mention" href="/u/abhilash_pinky">@abhilash_pinky</a>  Should be possible, writing custom crawlers inheriting from pyTorchCrawlergives you a lot a possibilities.</p>

mathematics: <p>It would be great if PyTorch have built in function for graph visualization.</p>

nagapavan525: <aside class="quote no-group" data-username="nullgeppetto" data-post="6" data-topic="1558">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/nullgeppetto/48/40331_2.png" class="avatar"> nullgeppetto:</div>
<blockquote>
<pre><code class="lang-auto">import torch.onnx
dummy_input = Variable(torch.randn(4, 3, 32, 32))
torch.onnx.export(net, dummy_input, "model.onnx")
</code></pre>
</blockquote>
</aside>
<p>Thank u so much…it helped me to visualize my ABCDs of resnet to understand it better.</p>

Purushothaman_Srikan: <p><a class="mention" href="/u/waleeka">@waleeka</a> Will this tool support Autoencoders and Deep AutoEncoders?</p>