Module.children() vs Module.modules()

rishabh: <p>I was trying to remove the last layer (fc) of Resnet18 to create something like <a href="https://discuss.pytorch.org/t/how-to-perform-finetuning-in-pytorch/419/12?u=rishabh">this</a> by using the following</p>
<pre><code class="lang-auto">pretrained_model = models.resnet18(pretrained=True)
for param in pretrained_model.parameters():
    param.requires_grad = False
my_model = nn.Sequential(*list(pretrained_model.modules())[:-1])
model = MyModel(my_model)
</code></pre>
<p>As it turns out this did not work (the layer is still there in the new model). I then found <a href="https://discuss.pytorch.org/t/how-to-extract-features-of-an-image-from-a-trained-model/119/3?u=rishabh">this</a> post and used the below line which works.</p>
<pre><code class="lang-auto">my_model = nn.Sequential(*list(pretrained_model.children())[:-1])
</code></pre>
<p>What is the difference between the two and why does one work but not the other?</p>

smth: <p><code>.modules()</code> also returns <code>pretrained_model</code> as one of the elements.</p>
<p>For example see:</p>
<pre><code class="lang-auto">list(nn.Sequential(nn.Linear(10, 20), nn.ReLU()).modules())
Out[9]:
[Sequential (
   (0): Linear (10 -&gt; 20)
   (1): ReLU ()
 ), Linear (10 -&gt; 20), ReLU ()]

In [10]: list(nn.Sequential(nn.Linear(10, 20), nn.ReLU()).children())
Out[10]: [Linear (10 -&gt; 20), ReLU ()]
</code></pre>

rishabh: <p>Ah, Makes sense.<br>
Sorry for such a basic question but what is the intended use for <code>.modules()</code>?</p>

fmassa: <p>If you want to recursively iterate over modules, then you want to use <code>.modules()</code><br>
For example, in the following network</p>
<pre><code class="lang-python">m = nn.Sequential(nn.Linear(2,2), 
                  nn.ReLU(),
                  nn.Sequential(nn.Sigmoid(), nn.ReLU()))
</code></pre>
<p>calling <code>m.children()</code> will return</p>
<pre><code class="lang-auto">[Linear (2 -&gt; 2), ReLU (), Sequential (
   (0): Sigmoid ()
   (1): ReLU ()
 )]
</code></pre>
<p>which means that it does not go inside the second <code>Sequential</code>, and thus does not print individually <code>Sigmoid</code>. On the other hand, <code>m.modules()</code> recursively walks into all modules in the network, and yields</p>
<pre><code class="lang-auto">[Sequential (
   (0): Linear (2 -&gt; 2)
   (1): ReLU ()
   (2): Sequential (
     (0): Sigmoid ()
     (1): ReLU ()
   )
 ), Linear (2 -&gt; 2), ReLU (), Sequential (
   (0): Sigmoid ()
   (1): ReLU ()
 ), Sigmoid (), ReLU ()]
</code></pre>

harryhan618: <p>So, when i iterate, is there a way to get these indiviual single components and get rid of those Sequential components?</p>

mnazaal: <p>@ post above me<br>
You could iterate through the whole network. Think of it as a tree.</p>
<pre><code class="lang-auto">all_layers = []
def remove_sequential(network):
    for layer in network.children():
        if type(layer) == nn.Sequential: # if sequential layer, apply recursively to layers in sequential layer
            remove_sequential(layer)
        if list(layer.children()) == []: # if leaf node, add it to list
            all_layers.append(layer)
</code></pre>

nimning: <p>Hi fmassa</p>
<p>In this operation, the names of all the nodes are different from the original name. For example, the original name of a convolution layer may be ‘conv_3_3’. However, the name becomes some numeric number, e.g., ‘10’, under this operation. Is there way to remain each layer’s original name after this kind of operation?</p>

VasLem: <p>I bumped to that issue while trying to find memory leakage in a custom architecture. To get the number of the children that are not parents to any other module, thus the real number of modules inside the provided one, I am using this recursive function:</p>
<pre><code class="lang-auto">def dim(module):
    total_num = 0
    f = False
    for child in module.children():
        f = True
        total_num += dim(child)
    if not f:
        return 1
    return total_num
</code></pre>
<p>I hope it helps</p>

AntoineC: <p>or</p>
<pre><code class="lang-python">isinstance(layer, nn.Sequential)
</code></pre>
<p>instead of</p>
<pre><code class="lang-python">type(layer) == nn.Sequential
</code></pre>
<p>as it is considered more pythonic?</p>

Brando_Miranda: <p>my modules are mapped to a name. I want to get the module and the name (but not the parameters). Is there a way to do that?</p>
<p>e.g.</p>
<pre><code class="lang-auto">mport torch
import torch.nn as nn

from collections import OrderedDict

params = OrderedDict([
    ('fc0', nn.Linear(in_features=4,out_features=4)),
    ('ReLU0', nn.ReLU()),
    ('fc1:final', nn.Linear(in_features=4,out_features=1))
])
mdl = nn.Sequential(params)
</code></pre>

Brando_Miranda: <p>answering my own question: <a href="https://discuss.pytorch.org/t/how-to-get-the-module-names-of-nn-sequential/39682" class="inline-onebox">How to get the module names of nn.Sequential</a></p>
<p>You could print all names and sub-modules using:</p>
<pre><code class="lang-auto">for name, module in model.named_modules():
    print(name)
</code></pre>
<p>If you want to directly access these modules, you can just use:</p>
<pre><code class="lang-auto">print(model.conv0)
</code></pre>

Brando_Miranda: <p>this might useful for you if you are trying to modify a pre-trained net (like I was): <a href="https://discuss.pytorch.org/t/how-to-modify-a-pretrained-model/60509/10" class="inline-onebox">How to modify a pretrained model</a></p>

mohit117: <p>Try this simple thing</p>
<pre><code class="lang-auto">for k, m in model._modules.items():
   print(m) # m is the indiviual single components
</code></pre>