Concatenate two layer with pytorch

randinoo: <p>I want to concatenate two layers of convolution</p>
<pre><code class="lang-auto">class Net(nn.Module):
    def __init__(self):
        super(Net,self).__init__()
        
        self.cnn1 = nn.Conv2d(in_channels=1, out_channels=32, kernel_size=3,stride=1, padding=1)
        self.batchnorm1 = nn.BatchNorm2d(32)       
      
        self.cnn2 = nn.Conv2d(in_channels=32, out_channels=16, kernel_size=5, stride=1, padding=2)
        self.batchnorm2 = nn.BatchNorm2d(16)
        
        
        self.maxpool2 = nn.MaxPool2d(kernel_size=2) 
        
        self.cnn3 = nn.Conv3d(in_channels=16, out_channels=64, kernel_size=3, stride=1, padding=2)
        self.relu = nn.ReLU() 
        self.maxpool3 = nn.MaxPool2d(kernel_size=2) 
        self.fc1 = nn.Linear(in_features=6451, out_features=800)   
        #self.droput = nn.Dropout(p=0.1)                   
        self.fc2 = nn.Linear(in_features=800, out_features=9)
        #self.droput = nn.Dropout(p=0.1)
        self.fc3 = nn.Linear(in_features=9, out_features=2)
       
        
    def forward(self,x):
        out = self.cnn1(x)
        out = self.batchnorm1(out)
        out1 = self.cnn2(out)
        out1 = self.batchnorm2(out1)
        combined = torch.cat((out,out1),1)
         out = out.view(combined.size(0),-1)   
         out = self.fc1(out)
        out = self.relu(out)
        out = self.fc2(out)
        out = self.relu(out)
        
        out = self.fc3(out)
        out = self.relu(out)
       
        return out
       
 
</code></pre>
<blockquote>
<p>RuntimeError: mat1 and mat2 shapes cannot be multiplied (6451x800 and 6451x800)</p>
</blockquote>
<p>I want to do conv1(3.3) → conv2(1,1)-&gt; concat two layers-&gt;pooling-&gt;conv-&gt;pooling-&gt;FC-&gt;softmax</p>
<p>help me pleaase</p>

InnovArul: <p>Can you specify the shape of the input passed to the network?<br>
It is difficult to debug the error without the input shape numbers.</p>

randinoo: <p>good morning InnovArul ,OKEY</p>
<pre><code class="lang-auto">b = sio.loadmat(pjoin('./sample/', 'image.mat'))
    
    patches_gt = image.extract_patches_2d(b, ((5,5)))
</code></pre>

randinoo: <p>I tested this code</p>
<pre><code class="lang-auto">class Net(nn.Module):
    def __init__(self):
        super(Net,self).__init__()
        

        self.features1 = nn.Sequential(
            nn.Conv2d(1, 6, 3, 1, 1),
            nn.ReLU(),
            nn.MaxPool2d(2),
            nn.Conv2d(6,6, 3, 1, 1),
            nn.ReLU(),
            nn.MaxPool2d(2)
        )
        
        self.features2 = nn.Sequential(
            nn.Conv2d(1, 6, 3, 1, 1),
            nn.ReLU(),
            nn.MaxPool2d(2),
            nn.Conv2d(6, 12, 3, 1, 1),
            nn.ReLU(),
            nn.MaxPool2d(2)
        )
        
        self.fc1 = nn.Linear(12*6*6, 64)
        self.fc2 = nn.Linear(12*6*6, 64)
        
        self.fc_out = nn.Linear(128, 10)
        
    def forward(self, x1, x2):
        x1 = self.features1(x1)
        x1 = x1.view(x1.size(0), -1)
        x1 = F.relu(self.fc1(x1))
        
        x2 = self.features2(x2)
        x2 = x2.view(x2.size(0), -1)
        x2 = F.relu(self.fc2(x2))

        # Concatenate in dim1 (feature dimension)
        x = torch.cat((x1, x2), 1)
        x = self.fc_out(x)
        return x
    
</code></pre>
<p>But, this problem is displayed</p>
<pre><code class="lang-auto">RuntimeError: Given groups=1, weight of size [6, 1, 3, 3], expected input[2, 3, 24, 24] to have 1 channels, but got 3 channels instead
</code></pre>
<p>help me please</p>

randinoo: <p>the problem in layer conv??!!!</p>

InnovArul: <aside class="quote no-group" data-username="randinoo" data-post="4" data-topic="107094">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/r/c5a1d2/48.png" class="avatar"> randinoo:</div>
<blockquote>
<pre><code class="lang-auto">        self.features1 = nn.Sequential(
            nn.Conv2d(1, 6, 3, 1, 1),
            nn.ReLU(),
            nn.MaxPool2d(2),
            nn.Conv2d(6,6, 3, 1, 1),
            nn.ReLU(),
            nn.MaxPool2d(2)
        )
</code></pre>
</blockquote>
</aside>
<p>It seems the first conv expects 1 channel, but your input has 3 channels.</p>

randinoo: <p>thanks  InnovArul <img src="https://discuss.pytorch.org/images/emoji/apple/slightly_smiling_face.png?v=9" title=":slightly_smiling_face:" class="emoji" alt=":slightly_smiling_face:"></p>

ptrblck: <p>The <code>in_fearures</code> of <code>self.fc1</code> do not match the feature dimension of the activation, which is <code>800</code> instead of <code>6451</code>, so you would have to change this value.</p>

randinoo: <p>Okay thanks ptrblck <img src="https://discuss.pytorch.org/images/emoji/apple/slightly_smiling_face.png?v=9" title=":slightly_smiling_face:" class="emoji" alt=":slightly_smiling_face:"></p>

randinoo: <p>ptrblck how I improve this code in order to find a good result at 50 iterations?</p>
<pre><code class="lang-auto">class Net(nn.Module):
    def __init__(self):
        super(Net,self).__init__()
        

        self.cnn1 = nn.Conv2d(in_channels=1, out_channels=8, kernel_size=3,stride=1, padding=1)
        self.batchnorm1 = nn.BatchNorm2d(8)        #Batch normalization
        self.relu = nn.ReLU()                 #RELU Activation
        self.maxpool1 = nn.MaxPool2d(kernel_size=2)   
      
        self.cnn2 = nn.Conv2d(in_channels=8, out_channels=10, kernel_size=5, stride=1, padding=2)
        self.batchnorm2 = nn.BatchNorm2d(10)
        self.relu = nn.ReLU() 
        self.maxpool2 = nn.MaxPool2d(kernel_size=2) 
        
        self.cnn3 = nn.Conv2d(in_channels=10, out_channels=32, kernel_size=5, stride=1, padding=2)
        self.batchnorm3 = nn.BatchNorm2d(32)
        self.relu = nn.ReLU() 
        #self.maxpool3 = nn.MaxPool2d(kernel_size=2) 
        
       
         
        
        self.fc1 = nn.Linear(in_features=32, out_features=20)   
       
        self.fc2 = nn.Linear(in_features=20, out_features=10)
       
        self.fc3 = nn.Linear(in_features=10, out_features=2)
       
      
       
        
    def forward(self,x):
        out = self.cnn1(x)
        out = self.batchnorm1(out)
        out = self.relu(out)
        out = self.maxpool1(out)
        out = self.cnn2(out)
        out = self.batchnorm2(out)
        out = self.relu(out)
        out = self.maxpool2(out)
        out = self.cnn3(out)
        out = self.batchnorm3(out)
        out = self.relu(out)
        #out = self.maxpool3(out)
        #out = self.maxpool3(out)
        #Flattening is done here with .view() -&gt; (batch_size, 32*16*16) = (100, 8192)
        out = out.view(out.size(0),-1)  
       
        out = self.fc1(out)
        out = self.relu(out)
        out = self.fc2(out)
        out = self.relu(out)
        out = self.fc3(out)
        out = self.relu(out)
       
        return out
</code></pre>
<p>plz help me thanks</p>

ptrblck: <p>I would remove the last relu activation, as this could potentially make the training harder, e.g. for a classification use case.<br>
Besides that, you would have to play around with the architecture as well as other hyperparameters in order to find a suitable workflow for your use case.</p>

randinoo: <p>Hi ptrblck ,  parameters such as for example what (pooling?)</p>

ptrblck: <p>Pooling would be one example of changing the architecture or other general hyperparameters such as the learning rate, weight decay etc.</p>

randinoo: <p>how I change the weight?</p>