Why is the difference between both methods?

username: brijraj
cooked: <p>Why does out varies in both of these executions:</p>
<p>Execution 1:</p>
<p><code>input=torch.rand(2,3,32,32)</code><br>
<code>resnet18 = models.resnet18(pretrained=True)</code><br>
<code>output1=resnet18(input)</code></p>
<p>Execution 2:</p>
<p><code>model_1=resnet18</code><br>
<code>c=model_1.children</code><br>
<code>children=[]</code><br>
<code>for item in c():</code><br>
<code>    children.append(item)</code></p>
<p><code>class CHILD(torch.nn.Module):</code></p>
<pre><code>def __init__(self):
    super(CHILD, self).__init__()
    self.models = nn.ModuleList(children).eval()     
    
def forward(self, x):
    for ii,model in enumerate(self.models):
        print(np.shape(x))
        print(model)
        if ii==9:
            x = torch.flatten(x,0)
        x=model(x)
    return x
</code></pre>
<p><code>MOD=CHILD()</code><br>
<code>output2=MOD(d1)</code><br>
I observed that <code>output1</code> is not same as <code>output2</code><br>
I guess it is because of the residual connections, which have been omitted in second case, If my point is correct then plz suggest me the way to know the position of residual connections (if from the<br>
model’s definition its not clear to me), so that I can match the second case with the actual case.</p>

username: ptrblck
cooked: <p>You will get the same output, of both approaches use <code>model.train()</code> or <code>model.eval()</code>.<br>
Currently you are updating the running statistics of all batchnorm layers in your first approach, while the second uses these (updated) stats by calling <code>eval()</code>.</p>

username: brijraj
cooked: <p>Thank you, yes this worked. Batchnorm was creating the problem where as I explicitly removed Dropout in the code.<br>
Problem Solved.</p>

username: brijraj
cooked: <p>On a similar case I am not realizing that how are these different:<br>
Method (1):</p>
<p><code>class new_m(torch.nn.Module):</code></p>
<pre><code>def __init__(self):
    super(new_m, self).__init__()
    Model = models.mobilenet_v2(args).cuda()
    self.network1=Model.features
    self.network2=Model.classifier
def forward(self, x):
    for ii,model in enumerate(self.network1):
        x = model(x)
    x=self.network4(x)
</code></pre>
<p>`<br>
Gives correct result,<br>
whereas<br>
Method (2):</p>
<p><code>class new_m(torch.nn.Module): </code><br>
<code>def __init__(self):</code><br>
<code> super(new_m, self).__init__()</code><br>
<code> Model = models.mobilenet_v2(args).cuda()</code><br>
<code>self.network1=list(Model.features)</code><br>
<code> self.network2=Model.classifier</code></p>
<pre><code>def forward(self, x):
    for ii,model in enumerate(self.network1):
       ` x = model(x)`

    x=self.network4(x)
</code></pre>
<p>Doesn’t give the correct result. Which has only one change at <em>Model.features</em></p>
<p>Why is listing the layers create the problem?</p>

username: ptrblck
cooked: <p>Modules or parameters in Python <code>list</code> and <code>dict</code> won’t be registered properly as such (and will thus be missing in e.g. <code>model.parameters()</code>).<br>
Use <code>nn.ModuleList</code> or <code>nn.ModuleDict</code> instead.</p>

username: brijraj
cooked: <p>Yes, It worked and improved my understanding.<br>
Thank you so much, that’s a great help.</p>