Maximum recursion bug?

username: Martin_K
cooked: <p>I was trying to build a tree like neural net, where there are nested modules in ModuleList/ModuleDict objects. However I encountered a maximum recursion bug in the top node, the ‘root’ of the tree. To make it simple, I created a minimal example to reproduce this error (Pythorch 1.2):</p>
<pre><code class="lang-auto">
class TreeNode_Test(nn.Module):
    def __init__(self):
        super(TreeNode_Test, self).__init__()
        self.nodesInLevels = nn.ModuleList([self]) 

myModel = TreeNode_Test()
myModel # when calling this or myModel.nodesInLevels I ll get max recursion error:


  File "C:\Users\mk23\AppData\Local\Continuum\anaconda3\lib\site-packages\torch\nn\modules\module.py", line 1042, in __repr__
    mod_str = repr(module)

  File "C:\Users\mk23\AppData\Local\Continuum\anaconda3\lib\site-packages\torch\nn\modules\module.py", line 1042, in __repr__
    mod_str = repr(module)

  File "C:\Users\mk23\AppData\Local\Continuum\anaconda3\lib\site-packages\torch\nn\modules\module.py", line 1036, in __repr__
    extra_repr = self.extra_repr()

RecursionError: maximum recursion depth exceeded
</code></pre>
<ul>
<li>Any ideas?</li>
</ul>

username: albanD
cooked: <p>Hi,</p>
<p>You model references itself when you use <code>self</code> in your ModuleList. So the print function that tries to print all the module that are contained in your model will run infinitely.</p>

username: Martin_K
cooked: <p>That is an explanation but not a solution <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>
<p>I don’t think this behaviour is correct, if I change the parent class to object instead of nn.Module, or the nn.ModuleList to python list(), then it will work as expected - but then it won’t work with DataParallel. As then it won’t replicate the model properly across multiple GPUs and I will end up with the dreaded tensors/parameters on different GPUs error…</p>
<p>It isn’t just print, (which I could avoid by not calling it), pretty much everything ends up in an infinite loop, eg  module.apply(fn)  which I can’t avoid using.</p>
<ul>
<li>
</li>
</ul>

username: ptrblck
cooked: <p>What would your use case be?<br>
If you use <code>self</code> as a module inside <code>nn.Module</code>, even the <code>__call__</code> function will try to recursively call itself.</p>

username: WilliamOnVoyage
cooked: <p>Hi, I have encountered the same issue when following this parallelism tutorial: <a href="https://pytorch.org/tutorials/beginner/former_torchies/parallelism_tutorial.html" class="inline-onebox" rel="noopener nofollow ugc">Multi-GPU Examples — PyTorch Tutorials 1.7.1 documentation</a> → <strong>Attributes of the wrapped module</strong></p>
<p>Simple code snippet to reproduce:</p>
<pre><code class="lang-auto">&gt;&gt;&gt; import torch
&gt;&gt;&gt; class TorchDataParallel(torch.nn.DataParallel):
...     def __getattr__(self, name):
...         return getattr(self.module, name)
... 
&gt;&gt;&gt; block = torch.nn.Module()
&gt;&gt;&gt; parallel_block = TorchDataParallel(block)
&gt;&gt;&gt; parallel_block.stream_names
Traceback (most recent call last):
  File "&lt;stdin&gt;", line 1, in &lt;module&gt;
  File "&lt;stdin&gt;", line 3, in __getattr__
  File "&lt;stdin&gt;", line 3, in __getattr__
  File "&lt;stdin&gt;", line 3, in __getattr__
  [Previous line repeated 995 more times]
RecursionError: maximum recursion depth exceeded
</code></pre>
<p>Wondering what would be the right way to access custom attributes?</p>

username: ptrblck
cooked: <p>Which PyTorch version are you using?<br>
I’ve just tried it out on a ~1 week old source build and don’t get an error.</p>

username: radugrosu
cooked: <p>I get the same on ‘1.7.1+cu101’.<br>
Can get it to work doing:</p>
<pre><code class="lang-auto">class DataParallel(torch.nn.parallel.DataParallel):
    def __getattr__(self, name):
        module = object.__getattribute__(self, "_modules")["module"]
        if name == "module":
            return module
        return getattr(module, name)
</code></pre>