How to visualize/draw a model

ravijoshi: <p>Hi,</p>
<p>I have a model from torchvision say Mask R-CNN. I wish to visualize/draw this model. For example, please see a sample below:</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/3/d/3d9dd1602cf88bc896b8ef0bc879c8f4ddac1347.png" data-download-href="https://discuss.pytorch.org/uploads/default/3d9dd1602cf88bc896b8ef0bc879c8f4ddac1347" title="sample"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/3/d/3d9dd1602cf88bc896b8ef0bc879c8f4ddac1347_2_385x500.png" alt="sample" data-base62-sha1="8N5fqTAWe6TTA3sumQGAehPKFHp" width="385" height="500" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/3/d/3d9dd1602cf88bc896b8ef0bc879c8f4ddac1347_2_385x500.png, https://discuss.pytorch.org/uploads/default/original/3X/3/d/3d9dd1602cf88bc896b8ef0bc879c8f4ddac1347.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/3/d/3d9dd1602cf88bc896b8ef0bc879c8f4ddac1347.png 2x" data-dominant-color="ECEEED"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">sample</span><span class="informations">486×630 22.6 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
Image Source: <a href="https://github.com/szagoruyko/pytorchviz" rel="noopener nofollow ugc">szagoruyko/pytorchviz</a></p>
<p>My model is initialized as shown below:</p>
<pre><code class="lang-auto">import torchvision
model = torchvision.models.detection.maskrcnn_resnet50_fpn(pretrained=True)
</code></pre>
<p>One way is to simply use <code>print(model)</code> to see the details. However, the skip connections, branch etc are lost with <code>print</code> statement.</p>
<p>So, how to visualize/draw a model?</p>

soulitzer: <p>Tensorboard has a functionality to display pytorch models <a href="https://pytorch.org/tutorials/intermediate/tensorboard_tutorial.html" class="inline-onebox" rel="noopener nofollow ugc">Visualizing Models, Data, and Training with TensorBoard — PyTorch Tutorials 2.0.0+cu117 documentation</a></p>

ConvolutionalAtom: <p>If you need python-only solution with convenient customization, see torchview, which I am the author of</p>
<p>here: <a href="https://github.com/mert-kurttutan/torchview" class="inline-onebox" rel="noopener nofollow ugc">GitHub - mert-kurttutan/torchview: torchview: visualize pytorch models</a></p>
<p>It is as simple as</p>
<pre><code class="lang-auto">import torchvision

model_graph = draw_graph(resnet18(), input_size=(1,3,32,32), expand_nested=True)
model_graph.visual_graph
</code></pre>
<p>The result:<br>
                    <a href="https://user-images.githubusercontent.com/88637659/206036653-293f8ce7-04dd-4ac6-9de8-0061de505bba.png" target="_blank" rel="noopener nofollow ugc" class="onebox">
            <img src="https://user-images.githubusercontent.com/88637659/206036653-293f8ce7-04dd-4ac6-9de8-0061de505bba.png" width="77" height="500">
          </a>

</p>

ravijoshi: <p><a class="mention" href="/u/soulitzer">@soulitzer</a></p>
<p>Thank you very much for sharing the reference. I followed it but couldn’t make it. It seems that Mask R-CNN does not return a single tensor but a dictionary. Therefore <code>add_graph()</code> is showing errors. I found a potential workaround <a href="https://discuss.pytorch.org/t/a-tensorboard-problem-about-use-add-graph-method-for-deeplab-v3-in-torchvision/95808/2">here</a>, but getting the following error:</p>
<pre><code class="lang-auto">Error occurs, No graph saved
Traceback (most recent call last):
  File "maskrcnn_vis.py", line 28, in &lt;module&gt;
    writer.add_graph(model_wrapper, dummy)
  File "/home/ravi/anaconda/envs/torchenv/lib/python3.7/site-packages/torch/utils/tensorboard/writer.py", line 727, in add_graph
    self._get_file_writer().add_graph(graph(model, input_to_model, verbose))
  File "/home/ravi/anaconda/envs/torchenv/lib/python3.7/site-packages/torch/utils/tensorboard/_pytorch_graph.py", line 292, in graph
    raise e
  File "/home/ravi/anaconda/envs/torchenv/lib/python3.7/site-packages/torch/utils/tensorboard/_pytorch_graph.py", line 286, in graph
    trace = torch.jit.trace(model, args)
...
...
...
RuntimeError: Encountering a dict at the output of the tracer might cause the trace to be incorrect, this is only valid if the container structure does not change based on the module's inputs. Consider using a constant container instead (e.g. for `list`, use a `tuple` instead. for `dict`, use a `NamedTuple` instead). If you absolutely need this and know the side effects, pass strict=False to trace() to allow this behavior.
</code></pre>
<p>Below is the version information:</p>
<pre><code class="lang-auto">Python 3.7.13 (default, Mar 29 2022, 02:18:16) 

In [1]: import torch
In [2]: import torchvision
In [3]: import tensorboard

In [4]: torch.__version__
Out[4]: '1.9.0+cu111'

In [5]: torchvision.__version__
Out[5]: '0.10.0+cu111'

In [6]: tensorboard.__version__
Out[6]: '2.11.2'

In [7]: torch.version.cuda
Out[7]: '11.1'

In [8]: torch.cuda.is_available()
Out[8]: True
</code></pre>
<p>The sample code is below:</p>
<pre><code class="lang-auto">import torch
import torchvision
from torch.utils.tensorboard import SummaryWriter
from collections import namedtuple
from typing import Any


class ModelWrapper(torch.nn.Module):
    def __init__(self, model: torch.nn.Module) -&gt; None:
        super().__init__()
        self.model = model

    def forward(self, input_x: torch.Tensor) -&gt; Any:
        data = self.model(input_x)
        if isinstance(data, dict):
            data_named_tuple = namedtuple("ModelEndpoints", sorted(data.keys()))
            data = data_named_tuple(**data)
        elif isinstance(data, list):
            data = tuple(data)
        return data


model = torchvision.models.detection.maskrcnn_resnet50_fpn(pretrained=True)
model_wrapper = ModelWrapper(model)
dummy = torch.rand((4, 3, 64, 64))

writer = SummaryWriter("logs/maskrcnn_resnet50_fpn")
writer.add_graph(model_wrapper, dummy)
writer.close()
</code></pre>

ravijoshi: <p><a class="mention" href="/u/convolutionalatom">@ConvolutionalAtom</a></p>
<p>Thank you so much for introducing <a href="https://github.com/mert-kurttutan/torchview" rel="noopener nofollow ugc">torchview</a>. It looks straightforward. Unfortunatly, I am not getting a fancy graph as yours. Please see below a screenshot:</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/e/b/eb19aac18b3bcb856d9e4e7bea751a4dbe2d864e.png" data-download-href="https://discuss.pytorch.org/uploads/default/eb19aac18b3bcb856d9e4e7bea751a4dbe2d864e" title="torchview"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/e/b/eb19aac18b3bcb856d9e4e7bea751a4dbe2d864e_2_486x500.png" alt="torchview" data-base62-sha1="xxNccOLuBNMqjb7jDIIP19Es9wG" width="486" height="500" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/e/b/eb19aac18b3bcb856d9e4e7bea751a4dbe2d864e_2_486x500.png, https://discuss.pytorch.org/uploads/default/optimized/3X/e/b/eb19aac18b3bcb856d9e4e7bea751a4dbe2d864e_2_729x750.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/e/b/eb19aac18b3bcb856d9e4e7bea751a4dbe2d864e.png 2x" data-dominant-color="EEF0EC"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">torchview</span><span class="informations">780×801 65.8 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

ConvolutionalAtom: <p>Yeah, this is actually in one of the issues.</p>
<p>The thing is that this detection model uses <code>Imagelist</code> object when transferring tensors among some of its submodules. But, torchview supports, <code>tensor</code>, any <code>mappable</code> object, any <code>iterable</code> object. Since <code>Imagelist</code> object is neither <code>mappable</code>, nor <code>iterable</code>, its tensor content is not record by torchview.</p>
<p>Currently working on this.</p>

ravijoshi: <p><a class="mention" href="/u/convolutionalatom">@ConvolutionalAtom</a></p>
<p>Thank you very much. I understand. I am looking forward to the fix.</p>
<p>Meanwhile, is there any workaround for now?</p>

Mohamed_Nabih: <p>You can also use Netron software to visualize your model and tensor.  “<a href="https://github.com/lutzroeder/netron" class="inline-onebox" rel="noopener nofollow ugc">GitHub - lutzroeder/netron: Visualizer for neural network, deep learning, and machine learning models</a>”</p>

ravijoshi: <p><a class="mention" href="/u/mohamed_nabih">@Mohamed_Nabih</a></p>
<p>Thank you very much. I tired it way before posting my question.</p>
<p>This tool is asking for weight file (state dictionary) and I supplied it. It rendered some graphic but without any node connecting to each other. All I could see some nodes.</p>
<p>It seems that visualizing Mask R-CNN is not supported by this tool.</p>
<p>I am looking forward hearing your opinion on possible workaround.</p>
<p>Thanks again.</p>

AbdulsalamBande: <p>Try tensorboard:</p>
<blockquote>
<p>pip install tensorboard</p>
</blockquote>
<pre><code class="lang-auto">import torch
import torchvision
from torch.utils.tensorboard import SummaryWriter

model = torchvision.models.detection.maskrcnn_resnet50_fpn(pretrained=True)
dummy_input = torch.rand(1, 3, 800, 800)

writer = SummaryWriter("runs/maskrcnn")
writer.add_graph(model, dummy_input)
writer.close()

</code></pre>
<blockquote>
<p>Now, run TensorBoard in your terminal:</p>
</blockquote>
<pre><code class="lang-auto">tensorboard --logdir runs

</code></pre>
<blockquote>
<p>Open the provided URL in your browser to view the model in TensorBoard.</p>
</blockquote>

ravijoshi: <p><a class="mention" href="/u/abdulsalambande">@AbdulsalamBande</a></p>
<p>Thank you very much. I tried it before, as mentioned <a href="https://discuss.pytorch.org/t/how-to-visualize-draw-a-model/176263/4">here</a>. However, upon your suggestion, I attempted it again and found the following error:</p>
<pre><code class="lang-auto">  File "/home/ravi/tools/anaconda/envs/visenv/lib/python3.7/site-packages/torch/jit/_trace.py", line 959, in trace_module
    argument_names,
RuntimeError: Only tensors, lists, tuples of tensors, or dictionary of tensors can be output from traced functions
</code></pre>
<p>Thank you very much</p>

ConvolutionalAtom: <p>This is probably due to the same error I mentioned about the usage of <code>ImageList</code> object.</p>

ravijoshi: <p><a class="mention" href="/u/convolutionalatom">@ConvolutionalAtom</a></p>
<p>Yes, you are right. The same cause is making errors on everywhere.</p>
<p>Please let me know if there is any workaround or if your fix is implemented.</p>
<p>Thanks again</p>

johnmarktaylor: <p>I recently released a package, <a href="https://github.com/johnmarktaylor91/torchlens" rel="noopener nofollow ugc">TorchLens</a>, for visualizing arbitrary PyTorch models. I verified that it works for maskrcnn_resnet50_fpn–the full visual is quite daunting (and too big to attach here) since TorchLens captures every operation in the model’s forward pass, but note that TorchLens also has options to visualize a model at different levels of nesting.</p>

xiangyu: <p>Btw, I’ve got around this problem by doing this:</p>
<p>First print the model:</p>
<pre><code class="lang-auto">&gt;&gt; print(mode)
FasterRCNN(
  (transform): GeneralizedRCNNTransform(
      Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
      Resize(min_size=(800,), max_size=1333, mode='bilinear')
  )
  (backbone): ....
</code></pre>
<p>Then visualize the backbone or other internal components instead:</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/5/d/5df4d16c7468d8cc03898abfd36358235b94bb7a.png" data-download-href="https://discuss.pytorch.org/uploads/default/5df4d16c7468d8cc03898abfd36358235b94bb7a" title="Screenshot 2024-05-18 at 5.07.12 PM"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/5/d/5df4d16c7468d8cc03898abfd36358235b94bb7a_2_690x389.png" alt="Screenshot 2024-05-18 at 5.07.12 PM" data-base62-sha1="dpaVlOeHE3EtYiAf3i0tzAiTx7s" width="690" height="389" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/5/d/5df4d16c7468d8cc03898abfd36358235b94bb7a_2_690x389.png, https://discuss.pytorch.org/uploads/default/optimized/3X/5/d/5df4d16c7468d8cc03898abfd36358235b94bb7a_2_1035x583.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/5/d/5df4d16c7468d8cc03898abfd36358235b94bb7a_2_1380x778.png 2x" data-dominant-color="F5F8F4"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screenshot 2024-05-18 at 5.07.12 PM</span><span class="informations">2222×1254 231 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>Not the prettiest, but it gets the job done.</p>