Train cnn simultaneously

username: SHASHANK_KUMAR_MISHR
cooked: <p>How to train theses CNN network simultaneously for the given input and then merge their loss function and then carry out backpropagation on combined loss function in pytorch?<br>
I have attached the network architecture<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/2X/0/0102bcff6398e510cd66fc95a06b57130c532140.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/0102bcff6398e510cd66fc95a06b57130c532140" title="Capture"><img src="https://discuss.pytorch.org/uploads/default/optimized/2X/0/0102bcff6398e510cd66fc95a06b57130c532140_2_690x386.jpeg" alt="Capture" data-base62-sha1="8Wll4lOSSD9AIWsQi08wmcRrz2" width="690" height="386" srcset="https://discuss.pytorch.org/uploads/default/optimized/2X/0/0102bcff6398e510cd66fc95a06b57130c532140_2_690x386.jpeg, https://discuss.pytorch.org/uploads/default/optimized/2X/0/0102bcff6398e510cd66fc95a06b57130c532140_2_1035x579.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/original/2X/0/0102bcff6398e510cd66fc95a06b57130c532140.jpeg 2x" data-dominant-color="F3F5F3"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Capture</span><span class="informations">1118×626 42.7 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

username: ptrblck
cooked: <p>You can basically define the model exactly as shown in the image.<br>
You would just have to pass the output of the base conv module to both linear layers, and accumulate the losses at the end.<br>
Here is a small example:</p>
<pre><code class="lang-python">class MyModel(nn.Module):
    def __init__(self, nb_vowels=5, nb_consonants=24):
        super(MyModel, self).__init__()
        self.base = nn.Sequential(
            nn.Conv2d(3, 6, 3, 1, 1),
            nn.ReLU(),
            nn.MaxPool2d(2),
            nn.Conv2d(6, 12, 3, 1, 1),
            nn.ReLU(),
            nn.MaxPool2d(2)
        )
        self.fc1 = nn.Linear(12 * 6 * 6, 64)
        
        self.fc_vowels = nn.Linear(64, nb_vowels)
        self.fc_cons = nn.Linear(64, nb_consonants)
        
    def forward(self, x):
        x = self.base(x)
        x = x.view(x.size(0), -1)
        x = F.relu(self.fc1(x))
        
        x_vowels = self.fc_vowels(x)
        x_cons = self.fc_cons(x)
        return x_vowels, x_cons


model = MyModel()

# Define your different loss functions here
criterion_vowels = nn.CrossEntropyLoss()
criterion_cons = nn.CrossEntropyLoss()

x = torch.randn(2, 3, 24, 24)
target_vowels = torch.randint(0, 5, (2,))
target_cons = torch.randint(0, 24, (2,))

output_vowels, output_cons = model(x)
loss_vowels = criterion_vowels(output_vowels, target_vowels)
loss_conv = criterion_cons(output_cons, target_cons)

loss = loss_vowels + loss_conv
loss.backward()
</code></pre>

username: SHASHANK_KUMAR_MISHR
cooked: <p>Thanks  @ <a href="https://discuss.pytorch.org/u/ptrblck">ptrblck</a></p>

username: SHASHANK_KUMAR_MISHR
cooked: <p>How to make such layers of vowels and consonats  incase of Resnet.Can u share the code for it.@ <a href="https://discuss.pytorch.org/u/ptrblck">ptrblck</a></p>

username: ptrblck
cooked: <p>If you would like to use a ResNet as the base model, just assign it as such and change the number of output features:</p>
<pre><code class="lang-python">self.base = models.resnet50(pretrained=True)
self.base.fc = nn.Linear(self.base.fc.in_features, 64)
self.fc_vowels = nn.Linear(64, ...)
...
</code></pre>