Can I get the ouput of a layer that sits inside nn.Sequential?

username: iArunava
cooked: <p>Say,</p>
<pre><code class="lang-auto">class Model(nn.Module):
  def __init__(self):
    super().__init__()
    self.net = nn.Sequential(nn.Conv2d(3, 3, 1, 1, 0), 
                             nn.Conv2d(3, 3, 1, 1, 0))

  def forward(self, x):
    for name, layer in self.net.named_modules():
      x = layer(x)
</code></pre>
<p>Now the thing is <code>.named_modlues()</code> will give me<br>
first the whole thing<br>
then the <code>Sequential</code> (so now my code has already propagated the input through the entire network)<br>
And then the two convs.</p>
<p>I just want the layers to iterate over one by one.</p>
<p>One hardcoded solution is to skip over the first 2. which is not a good solution.<br>
Another is to, check if the layer is present in a <code>list</code> of acceptable layers and then pass the <code>x</code> through it.</p>
<p>Any other solutions?</p>

username: tymokvo
cooked: <pre><code class="lang-auto">&gt;&gt;&gt; import torch
&gt;&gt;&gt; import torch.nn as nn
&gt;&gt;&gt; s = nn.Sequential()
&gt;&gt;&gt; s.add_module("conv", nn.Conv2d(1, 1, 4))
&gt;&gt;&gt; s.conv
Conv2d(1, 1, kernel_size=(4, 4), stride=(1, 1))
&gt;&gt;&gt;
</code></pre>
<p>You can do this. Also, you should not be looping inside your custom <code>forward()</code> method. The <code>Sequential</code> class will take care of passing outputs to inputs of the subsequent layers. Just return the result of <code>self.net(x)</code></p>

username: iArunava
cooked: <p>I am sorry but I think you misunderstood my question.<br>
I want the output of the first <code>conv</code> not the output after it has passed through all the layers. That’s why its necessary to loop.<br>
In general, I want the output of the intermediate layers in a <code>nn.Sequential</code> as the question says.</p>

username: tymokvo
cooked: <p>Yep! I understand.</p>
<p>If you change your code to be like this:</p>
<pre><code class="lang-auto">import torch
import torch.nn as nn

class Model(nn.Sequential):
  def __init__(self):
    super().__init__()
    [self.add_module("conv_{}".format(n), nn.Conv2d(3, 3, 1, 1, 0)) for n in range(2)]

m = Model()

print(m)
print(m.conv_0)
</code></pre>
<p>You will have a <code>Sequential</code> model that implicitly chains the output of <code>conv_0</code> to the input of <code>conv_1</code> so there is no need for the loop in the forward method.</p>
<p>If you want to get a layer, it will be available as a property that matches the string passed for the module name in <code>add_module</code>. So in this case <code>m.conv_0</code> is how you would get at the first layer without looping and matching the name.</p>
<p>You can call it directly on some input like <code>m.conv_0(torch.randn(1, 3, 4, 4))</code>. Hope that helps!</p>
<p>If you want to keep something closer to what you have, you could also do:</p>
<pre><code class="lang-auto">from collections import OrderedDict as odict
import torch.nn as nn

class Model(nn.Module):
    def __init__(self):
        super().__init__()

        layers = odict(
            [
                ("conv_0", nn.Conv2d(3, 3, 1, 1, 0)),
                ("conv_1", nn.Conv2d(3, 3, 1, 1, 0))
            ]
        )

        self.net = nn.Sequential(layers)

    def forward(self, x):
        return self.net(x)

m = Model()

print(m.net.conv_0)
</code></pre>
<p>But it seems weird to use a <code>Module</code> to wrap a single <code>Sequential</code> container. Why not inherit from <code>nn.Sequential</code> directly?</p>

username: iArunava
cooked: <p>Yeah, you are right. Thanks I will update my code.</p>