How to print the computational graph of a Variable?

acgtyrant: <p><a href="http://pytorch.org/tutorials/beginner/blitz/neural_networks_tutorial.html" class="inline-onebox" rel="noopener nofollow ugc">Neural Networks — PyTorch Tutorials 2.1.1+cu121 documentation</a> says:</p>
<blockquote>
<p>Now, if you follow loss in the backward direction, using it’s .creator attribute, you will see a graph of computations that looks like this:</p>
</blockquote>
<pre><code>input -&gt; conv2d -&gt; relu -&gt; maxpool2d -&gt; conv2d -&gt; relu -&gt; maxpool2d
      -&gt; view -&gt; linear -&gt; relu -&gt; linear -&gt; relu -&gt; linear
      -&gt; MSELoss
      -&gt; loss
</code></pre>
<p>But its <code>creator</code> attribute only prints <code>&lt;torch.nn._functions.thnn.auto.MSELoss object at 0x7f784059d5c0&gt;</code>, I would like to know is there any convenient approach that can print its all computation graph such as <code>print(net)</code>?</p>

albanD: <p>Hi,</p>
<p>You can use this script to create a graph <a href="https://github.com/szagoruyko/functional-zoo/blob/master/visualize.py">https://github.com/szagoruyko/functional-zoo/blob/master/visualize.py</a></p>
<p>To open it with a regular pdf viewer you can do <code>make_dot(your_var).view()</code>.</p>

Ynjxsjmh: <p>I want to generate a following computation graph mentioned in <a href="https://pytorch.org/blog/computational-graphs-constructed-in-pytorch/" rel="noopener nofollow ugc">How Computational Graphs are Constructed in PyTorch | PyTorch</a></p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/0/3/0357bad8bd423d8a12b5e528dc68dca3773c4b54.png" data-download-href="https://discuss.pytorch.org/uploads/default/0357bad8bd423d8a12b5e528dc68dca3773c4b54" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/0/3/0357bad8bd423d8a12b5e528dc68dca3773c4b54_2_584x500.png" alt="image" data-base62-sha1="tzolSV17yGubDuFRYohpkvIH7C" width="584" height="500" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/0/3/0357bad8bd423d8a12b5e528dc68dca3773c4b54_2_584x500.png, https://discuss.pytorch.org/uploads/default/optimized/3X/0/3/0357bad8bd423d8a12b5e528dc68dca3773c4b54_2_876x750.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/0/3/0357bad8bd423d8a12b5e528dc68dca3773c4b54_2_1168x1000.png 2x" data-dominant-color="659780"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1262×1079 106 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>I try with <code>make_dot</code> with following code</p>
<pre data-code-wrap="python"><code class="lang-python">from graphviz import Digraph
import torch
from torch.autograd import Variable

# make_dot was moved to https://github.com/szagoruyko/pytorchviz
from torchviz import make_dot

x1 = Variable(torch.Tensor([3]), requires_grad=True)
x2 = Variable(torch.Tensor([5]), requires_grad=True)

a = torch.mul(x1, x2)
y1 = torch.log(a)
y2 = torch.sin(x2)
w = torch.mul(y1, y2)

make_dot(w)
</code></pre>
<p>but it doesn’t work</p>
<pre><code class="lang-auto">Traceback (most recent call last):
  File "graph.py", line 21, in &lt;module&gt;
    make_dot(w)
  File "~/.pyenv/versions/pymarl/lib/python3.6/site-packages/torchviz/dot.py", line 163, in make_dot
    add_base_tensor(var)
  File "~/.pyenv/versions/pymarl/lib/python3.6/site-packages/torchviz/dot.py", line 153, in add_base_tensor
    if var._is_view():
AttributeError: 'Tensor' object has no attribute '_is_view'
</code></pre>

ptrblck: <p>Which PyTorch version are you using? The <code>_is_view</code> method is quite old, if I’m not mistaken.</p>

Ynjxsjmh: <p>Thank you, the original pytorch version is 0.4.1 and upgrading it to 1.9.0a it outputs the following pic</p>
<p><img src="https://discuss.pytorch.org/uploads/default/original/3X/6/6/66ffa5f2a739172b6ad0e19272bd5e2caa0baec7.png" alt="image" data-base62-sha1="eHaqoMeNm2gLYcIwZynipu760NF" width="304" height="435"></p>
<p>I notice that it only plots <code>grad_fn</code> of tensor in square node but no variable name in circle node. I tried to use named tensor like</p>
<pre data-code-wrap="python"><code class="lang-python">x1 = Variable(torch.Tensor([3]), requires_grad=True)
x2 = Variable(torch.Tensor([5]), requires_grad=True)
x1.names=('x1',)
x2.names=('x2',)

a = torch.mul(x1, x2)
a.names=('a',)

y1 = torch.log(a)
y1.names=('y1',)

y2 = torch.sin(x2)
y2.names=('y2',)

w = torch.mul(y1, y2)
w.names=('w',)
</code></pre>
<p>This time the compiler outputs</p>
<pre><code class="lang-auto">UserWarning: Named tensors and all their associated APIs are an experimental feature and subject to change. Please do not use them for anything important until they are released as stable. (Triggered internally at  /pytorch/c10/core/TensorImpl.h:1156.)
  x1.names=('x1',)
Traceback (most recent call last):
  File "graph.py", line 13, in &lt;module&gt;
    a = torch.mul(x1, x2)
RuntimeError: Error when attempting to broadcast dims ['x1'] and dims ['x2']: dim 'x1' and dim 'x2' are at the same position from the right but do not match.
</code></pre>
<p>Is it also possible to plot the variable name in circle node?</p>

ptrblck: <p><code>Variable</code>s are deprecated since PyTorch <code>0.4</code>, too, so you should remove their usage.<br>
The <code>mul</code> operation seems to fail after adding the <code>names</code>. I’m not familiar with the current support of Named tensors, but unless this usage is wrong, it looks like a valid bug. Would you mind creating an issue on GitHub for it, please?</p>

AlphaBetaGamma96: <aside class="quote no-group" data-username="Ynjxsjmh" data-post="5" data-topic="3325">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/y/7c8e57/48.png" class="avatar"> Ynjxsjmh:</div>
<blockquote>
<pre><code class="lang-auto">x1 = Variable(torch.Tensor([3]), requires_grad=True)
x2 = Variable(torch.Tensor([5]), requires_grad=True)
</code></pre>
</blockquote>
</aside>
<p>Doesn’t it failed because you’re trying to <code>mul</code> 1 tensor of size 3 with another of size 5? they need to be the same shape in order for <code>mul</code> to work?</p>

ptrblck: <p>The <code>torch.Tensor</code> usage is deprecated, so I’ve used this as a check (which creates tensors with a single value):</p>
<pre><code class="lang-python">x1 = torch.tensor([3.], requires_grad=True)
x2 = torch.tensor([5.], requires_grad=True)
a = torch.mul(x1, x2)
print(a) # works

x1 = torch.tensor([3.], requires_grad=True)
x2 = torch.tensor([5.], requires_grad=True)
x1.names=('x1',)
x2.names=('x2',)
a = torch.mul(x1, x2)
&gt; RuntimeError: Error when attempting to broadcast dims ['x1'] and dims ['x2']: dim 'x1' and dim 'x2' are at the same position from the right but do not match.
</code></pre>

Ynjxsjmh: <aside class="quote no-group" data-username="ptrblck" data-post="6" data-topic="3325">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar"> ptrblck:</div>
<blockquote>
<p>Would you mind creating an issue on GitHub for it, please?</p>
</blockquote>
</aside>
<p>Issue is created at <a href="https://github.com/pytorch/pytorch/issues/67168" class="inline-onebox" rel="noopener nofollow ugc">Multiply two named tensor causes RuntimeError · Issue #67168 · pytorch/pytorch · GitHub</a>.</p>
<aside class="quote no-group" data-username="AlphaBetaGamma96" data-post="7" data-topic="3325">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/a/3da27b/48.png" class="avatar"> AlphaBetaGamma96:</div>
<blockquote>
<p>tensor of size 3 with another of size 5</p>
</blockquote>
</aside>
<p>No, their size are both <code>torch.Size([1])</code>.</p>

AlphaBetaGamma96: <p>What should we replaced with <code>torch.Tensor</code> if it’s deprecated?</p>
<p>and, <a class="mention" href="/u/ynjxsjmh">@Ynjxsjmh</a> you’re correct ignore my mistake!</p>

ptrblck: <p>You should either replace it with <code>torch.tensor</code> (lowercase t) if you are passing values to it directly or use the factory methods such as <code>torch.randn</code>, <code>torch.zeros</code>, <code>torch.empty</code>.</p>