Good advice to double the input and output channel from the pre-trained model?

dhkim: <p>Hi,</p>
<p>I have loaded the pre-trained ResNet-50 model, and now I want to change the input and output channels of all layers. Not only the first or last layers. But maintaining the same model structure.</p>
<p>is there any way to do this?</p>
<p>Here is a simple example of what I want to do.</p>
<pre><code class="lang-auto">class CustomCNN(nn.Module):
    def __init__(self):
        super(MiniCustomCNN, self).__init__()
        self.features = nn.Sequential(
            nn.Conv2d(3, 6, (5, 5)),
            nn.ReLU(),
            nn.MaxPool2d((2, 2)),
            nn.Conv2d(6, 8, (5, 5)),
            nn.ReLU(),
            nn.MaxPool2d((2, 2))
        )
        self.fc = nn.Sequential(
            nn.Linear(200, 40),
            nn.Linear(40, 10)
        )

    def forward(self, x):
        x = self.features(x)
        x = torch.flatten(x, 1)
        x = self.fc(x)
        return x

model = CustomCNN()
# To do something...
</code></pre>
<p>The model summary results I am aiming for…</p>
<blockquote>
<p>CustomCNN(<br>
(features): Sequential(<br>
(0): Conv2d(6, 12, kernel_size=(5, 5), stride=(1, 1))<br>
(1): ReLU()<br>
(2): MaxPool2d(kernel_size=(2, 2), stride=(2, 2), padding=0, dilation=1, ceil_mode=False)<br>
(3): Conv2d(12, 16, kernel_size=(5, 5), stride=(1, 1))<br>
(4): ReLU()<br>
(5): MaxPool2d(kernel_size=(2, 2), stride=(2, 2), padding=0, dilation=1, ceil_mode=False)<br>
)<br>
(fc): Sequential(<br>
(0): Linear(in_features=400, out_features=80, bias=True)<br>
(1): Linear(in_features=80, out_features=10, bias=True)<br>
)<br>
)</p>
</blockquote>
<p>Simply, I want to double(or customize) the input and output channels from the pre-build model.</p>
<p>Any help will be appreciated.</p>
<p>Thanks.</p>

jimmiemunyi: <p>Not a professional here but here is how I’d approach the situation.</p>
<p>Here is what I understood from your question, you have downloaded a pre-trained model, and now you want to change the internal channels of the layers, but keep the layers themselves, right?</p>
<p>Would you mind explaining further why you would need to do this? Maybe it would shed some light on how best to solve the issue.</p>
<p>I’m not sure if you can change the channels without hurting the weights because you’d have to reinitialize the layers to get the channels you want.</p>
<p>A solution might be to use the pre-trained model as an encoder, pass your data through the encoder, then create a custom network and pass it through. e.g</p>
<pre><code class="lang-auto">class CustomCNN(nn.Module):
    def __init__(self, encoder):
        super(CustomCNN, self).__init__()
        # pass the pretrained model here
        self.encoder = encoder
        # define your custom architecture
        self.features = nn.Sequential(
            nn.Conv2d(3, 6, (5, 5)),
            nn.ReLU(),
            nn.MaxPool2d((2, 2)),
            nn.Conv2d(6, 8, (5, 5)),
            nn.ReLU(),
            nn.MaxPool2d((2, 2))
        )
        self.fc = nn.Sequential(
            nn.Linear(200, 40),
            nn.Linear(40, 10)
        )

    def forward(self, x):
       # pass through encoder first
        x = self.encoder(x)
        x = self.features(x)
        x = torch.flatten(x, 1)
        x = self.fc(x)
        return x

model = CustomCNN()
</code></pre>
<p>This way, your model works on whatever channels you have.</p>
<aside class="quote no-group" data-username="dhkim" data-post="1" data-topic="134625">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/d/87869e/48.png" class="avatar"> dhkim:</div>
<blockquote>
<p>and now I want to change the input and output channels of all layers. Not only the first or last layers</p>
</blockquote>
</aside>
<p>I’m not sure this is possible because you have to redefine</p>
<p>The tricky part would be how to match the output shape of the encoder to your first Conv2D.</p>

dhkim: <p>Thanks for your reply.</p>
<p>I am implementing channel-growing models for my business purpose.</p>
<p>Training from the small size of data and minimal model, and then upsize the model and data size after certain epochs.</p>
<p>Previous many upsizing model methods have been implemented by adding layers, but for me, I am trying to upsize the model by channels. (Migrate the small size of model’s weights into some part of the big model and retrain with new data)</p>
<p>Therefore, It is ok with hurting the weights of the pre-trained model.</p>
<p>Maybe I can make a big-size model by hard coding, but I want to make a kind of automated growing model.</p>
<p>By the way, your suggestion also makes sense to me <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=10" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>
<p>I will try if it works as I intended.</p>
<p>Thanks.</p>