Access first layer of non-Sequential CNN

snowe: <p>Hi everyone <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>
<p>I was wondering whether there is an easy option to access the first layer in a custom non-Sequential CNN. When the network is constructed the ‘sequential way’, you can just use: <code>network[0]</code>. Is there anything similar to that?</p>
<p>Any help is very much appreciated!</p>
<p>All the best<br>
snowe</p>

ptrblck: <p>Yes, you can directly access the layer by its name as seen here:</p>
<pre><code class="lang-python">class MyModel(nn.Module):
    def __init__(self):
        super(MyModel, self).__init__()
        self.conv = nn.Conv2d(3, 3, 3, 1, 1)
        self.fc = nn.Linear(10, 10)
        
    def forward(self, x):
        x = F.relu(self.conv(x))
        x = self.fc(x)
        return x

model = MyModel()
print(model.conv)
&gt; Conv2d(3, 3, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
</code></pre>

snowe: <p>Thank you <a class="mention" href="/u/ptrblck">@ptrblck</a>, but I can only do that when I know the name of the first layer. How can I access it if I don’t know the name?</p>
<p>For example: I want to create a function that takes a model as parameter and returns the first layer of this model…</p>

ptrblck: <p>In custom modules the “first layer” annotation might be misleading as seen in this example:</p>
<pre><code class="lang-python">class MyModel(nn.Module):
    def __init__(self):
        super(MyModel, self).__init__()
        self.conv2 = nn.Conv2d(3, 3, 3, 1, 1)
        self.conv1 = nn.Conv2d(3, 3, 3, 1, 1)
        
    def forward(self, x):
        x = F.relu(self.conv1(x))
        x = self.conv2(x)
        return x

model = MyModel()
print(model)
print(next(model.named_children()))
</code></pre>
<p><code>model.named_children()</code> as well as the <code>print(model)</code> statement will return <code>conv2</code> as the “first” layer, but in the end the usage of the modules in the <code>forward</code> defines the order of execution.<br>
Anyway, <code>next(model.named_children())</code> might work for you.<br>
Note that finding the “first layer” also depends what you mean by layer, as <code>nn.Modules</code> might themselves contain more modules as seen here:</p>
<pre><code class="lang-python">class MyModel(nn.Module):
    def __init__(self):
        super(MyModel, self).__init__()
        self.conv2 = models.resnet18()
        self.conv1 = nn.Conv2d(3, 3, 3, 1, 1)
        
    def forward(self, x):
        x = F.relu(self.conv1(x))
        x = self.conv2(x)
        return x

model = MyModel()
print(model)
print(next(model.named_children()))
</code></pre>

snowe: <p>Oh I see, I did not take this into consideration. In my case, the term first means the first convolutional layer. And since most of the times (at least with the models I am working with) the first layer is a convolutional layer I mismatched the terms.</p>
<p>Is there a similar way to access the order of execution in the forward method then?</p>

ptrblck: <aside class="quote no-group" data-username="snowe" data-post="5" data-topic="101375">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/s/e68b1a/48.png" class="avatar"> snowe:</div>
<blockquote>
<p>Is there a similar way to access the order of execution in the forward method then?</p>
</blockquote>
</aside>
<p>Not a beautiful approach, but you could register forward hooks for each layer and print its name when it’s called.<br>
Based on the output you would see which layer was called first in this execution.<br>
Note that this is also not a bulletproof approach, as your <code>forward</code> might have conditions, loops, etc. which can change during the runtime.</p>
<p>I guess for the majority of models you might be find to iterate <code>model.named_modules()</code> or <code>.named_children()</code>, filter using <code>if isinstance(module, nn.Conv2d)</code> and print the first occurrence.</p>

snowe: <p>Thank you <a class="mention" href="/u/ptrblck">@ptrblck</a> !</p>
<p>This will work for now <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>