Adding a fully connected layer in Resnet

username: Niki
cooked: <p>If I want to add a fully connected layer after pooling in the Resnet, how can use <code>setattr and getattr</code> instead of this:</p>
<pre><code class="lang-auto">self.layer1 = nn.Linear(512, 512)
self.layer2 = nn.Linear(512, num_classes)

self.layer3 = nn.Linear(512, 512)
self.layer4 = nn.Linear(512, num_classes)


out = out.view(out.size(0), -1)
out1 = self.layer1(out)
out1 = F.relu(out1)
out1 = self.layer2(out1)

out2 = self.layer3(out)
out2 = F.relu(out2)
out2 = self.layer4(out2)
</code></pre>
<p>this one shows error,</p>
<pre><code class="lang-auto">self.num_ftrs = num_filters   
       for i in range(self.num_ftrs):
           setattr(self, "fc%d" % i, nn.Linear(512, 512))
           setattr(self, "fc%d" % i, nn.Linear(512, num_classes))


out = out.view(out.size(0), -1)
outputs = []      
for i in range(self.num_ftrs):
       out = getattr(self, "fc%d" % i)(out)
       out = F.relu(out)
       outputs.append(getattr(self, "fc%d" % i)(out))
</code></pre>

username: ptrblck
cooked: <p>What kind of error are you seeing?<br>
Based on the provided code I would guess you might be running into a shape mismatch, as  you are currently redefining <code>fc%d</code> in the first loop, which would create a stack of layers as:</p>
<pre><code class="lang-python">self.fc1 = nn.Linear(512, num_classes)
self.fc2 = nn.Linear(512, num_classes)
self.fc3 = nn.Linear(512, num_classes)
</code></pre>

username: Niki
cooked: <p>Thank you for your reply, <a class="mention" href="/u/ptrblck">@ptrblck</a>.<br>
Yes, the error is shape mismatch. How can I solve it? I want to have a fully connected layer with Relu before each fully connected layer.</p>

username: ptrblck
cooked: <p>You could iterate all linear layers you wish to manipulate and replace them with your additional linear + relu and the original linear layer as an <code>nn.Sequential</code> container.</p>

username: Niki
cooked: <aside class="quote no-group" data-username="ptrblck" data-post="4" data-topic="71343">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar"> ptrblck:</div>
<blockquote>
<p>nn.Sequential</p>
</blockquote>
</aside>
<p>Thank you for your reply, <a class="mention" href="/u/ptrblck">@ptrblck</a>.</p>
<p>I tried</p>
<pre><code class="lang-auto">self.layer5 = nn.Sequential(nn.Linear(512, 512),
                          nn.ReLU(),
                          nn.Linear(512, num_classes))
        
self.num_ftrs = 2   
for i in range(self.num_ftrs):
       setattr(self, "fc%d" % i, self.layer5)
</code></pre>
<p>and in the forward</p>
<pre><code class="lang-auto">clf_outputs = []      
for i in range(self.num_ftrs):
      clf_outputs.append(getattr(self, "fc%d" % i)(out))
</code></pre>
<p>but the output <code>clf_outputs[0]</code> and <code>clf_outputs[1]</code> were the same. The output is different if I change it to</p>
<pre><code class="lang-auto">self.layer5 = nn.Sequential(nn.Linear(512, 512),
                    nn.ReLU(),
                    nn.Linear(512, num_classes))

self.layer6 = nn.Sequential(nn.Linear(512, 512),
                    nn.ReLU(),
                    nn.Linear(512, num_classes))
</code></pre>
<p>and in forward</p>
<pre><code class="lang-auto">clf_outputs1 = self.layer5(out)
clf_outputs2 = self.layer6(out)
clf_outputs = [clf_outputs1, clf_outputs2]
</code></pre>
<p>But I want the first approach getting work, Is there any ways that I can keep using <code>setattr, getattr</code> and getting different outputs?</p>