RuntimeError: mat1 and mat2 shapes cannot be multiplied (1x524288 and 32768x2048)

123456: <p>Please see the code below.</p>
<p>class residualBlock(nn.Module):<br>
def <strong>init</strong>(self, in_channels=64, k=3, n=64, s=1):<br>
super(residualBlock, self).<strong>init</strong>()<br>
m = OrderedDict()<br>
m[‘conv1’] = nn.Conv2d(in_channels, n, k, stride=s, padding=1)<br>
m[‘bn1’] = nn.BatchNorm2d(n)<br>
m[‘ReLU1’] = nn.ReLU(inplace=True)<br>
m[‘conv2’] = nn.Conv2d(n, n, k, stride=s, padding=1)<br>
m[‘bn2’] = nn.BatchNorm2d(n)<br>
self.group1 = nn.Sequential(m)<br>
self.relu = nn.Sequential(nn.ReLU(inplace=True))</p>
<pre><code>def forward(self, x):
    out = self.group1(x) + x
    out = self.relu(out)
    return out
</code></pre>
<p>class Generator(nn.Module):<br>
def <strong>init</strong>(self, n_residual_blocks):<br>
super(Generator, self).<strong>init</strong>()<br>
self.n_residual_blocks = n_residual_blocks<br>
<span class="hashtag-raw">#self</span>.upsample_factor = upsample_factor</p>
<pre><code>    self.conv1 = nn.Conv2d(6, 64, 9, stride=1, padding=4)  #9 4
    self.bn1 = nn.BatchNorm2d(64)
    self.relu1 = nn.ReLU(inplace=True)
    self.conv2 = nn.Conv2d(64, 64, 3, stride=1, padding=1)
    self.bn2 = nn.BatchNorm2d(64)
    self.relu2 = nn.ReLU(inplace=True)
    self.conv3 = nn.Conv2d(64, 64, 3, stride=1, padding=1)
    self.bn3 = nn.BatchNorm2d(64)
    self.relu3 = nn.ReLU(inplace=True)

    for i in range(self.n_residual_blocks):
        self.add_module('residual_block' + str(i + 1), residualBlock())

    self.conv4 = nn.Conv2d(64, 32, 3, stride=1, padding=1)
    self.bn4 = nn.BatchNorm2d(32)
    self.relu4 = nn.ReLU(inplace=True)
    self.conv5 = nn.Conv2d(32, 32, 3, stride=1, padding=1)
    self.bn5 = nn.BatchNorm2d(32)
    self.relu5 = nn.ReLU(inplace=True)
    self.conv6 = nn.Conv2d(32, 32, 3, stride=1, padding=1) #64,2,3 for pair
    self.bn6 = nn.BatchNorm2d(32)
    self.relu6 = nn.ReLU(inplace=True)

    self.fc = nn.Linear(32*32*32,2*32*32)   # 32768,2048
    self.sigmoid = nn.Sigmoid()

def forward(self, x):
    x = swish(self.relu1(self.bn1(self.conv1(x))))
    print("1= ",x.size())
    x = swish(self.relu2(self.bn2(self.conv2(x))))
    print("2= ",x.size())
    x = swish(self.relu3(self.bn3(self.conv3(x))))
    print("3= ",x.size())
    y = x.clone()
    for i in range(self.n_residual_blocks):
        y = self.__getattr__('residual_block' + str(i + 1))(y)
    x = swish(self.relu4(self.bn4(self.conv4(y)))) # + x
    print("4= ",x.size())
    x = swish(self.relu5(self.bn5(self.conv5(x))))
    print("5= ",x.size())
    output_map = swish(self.relu6(self.bn6(self.conv6(x))))
    print("output_map= ",output_map.size())
    flattened_output = output_map.view(output_map.size(0), -1)  # Flatten the output

    fc_output = self.fc(flattened_output)
    print("fc_output= ",fc_output.size())
    fc_output = self.sigmoid(fc_output)  # Apply sigmoid activation
    fc_output = fc_output.view(x.size(0), 2,128*128)  # Reshape output to (batch_size, 2, 128, 128)
    return fc_output
</code></pre>
<p>I am getting error as<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/8/f/8f70215370739d3fa8c6da29820a6d6577c376ab.png" data-download-href="https://discuss.pytorch.org/uploads/default/8f70215370739d3fa8c6da29820a6d6577c376ab" title="error"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/8/f/8f70215370739d3fa8c6da29820a6d6577c376ab_2_690x207.png" alt="error" data-base62-sha1="ksUvZNhH2odmdkvr6zmZRD6ZZur" width="690" height="207" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/8/f/8f70215370739d3fa8c6da29820a6d6577c376ab_2_690x207.png, https://discuss.pytorch.org/uploads/default/optimized/3X/8/f/8f70215370739d3fa8c6da29820a6d6577c376ab_2_1035x310.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/8/f/8f70215370739d3fa8c6da29820a6d6577c376ab_2_1380x414.png 2x" data-dominant-color="39152E"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">error</span><span class="informations">1450×436 87.8 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>My input image size is 128*128. there are two input images, and correspondingly two output images of the same size.</p>
<p>Please tell where I am wrong</p>

ptrblck: <p>The shape mismatch is caused by <code>self.fc</code> which expects <code>32*32*32=32768</code> input features while your input activation has <code>524288</code> features. Set the <code>in_features</code> value of <code>self.fc</code> to <code>524288</code> and it should work.</p>

123456: <p>But after setting this value, I got another error<br>
fc_output = fc_output.view(x.size(0), 2,128*128)  # Reshape output to (batch_size, 2, 128, 128)<br>
RuntimeError: shape ‘[1, 2, 16384]’ is invalid for input of size 2048</p>
<p>printed values are</p>
<p>1=  torch.Size([1, 64, 128, 128])<br>
2=  torch.Size([1, 64, 128, 128])<br>
3=  torch.Size([1, 64, 128, 128])<br>
4=  torch.Size([1, 32, 128, 128])<br>
5=  torch.Size([1, 32, 128, 128])<br>
output_map=  torch.Size([1, 32, 128, 128])<br>
fc_output=  torch.Size([1, 2048])<br>
outputs shape=  torch.Size([1, 2048])</p>

ptrblck: <p>The <code>self.fc</code> layer outputs <code>2*32*32=2048</code> features while you are trying to reshape the feature dimension if the output to <code>[batch_size, 2, 128, 128]</code> which is invalid as it expects to have <code>32768</code> features.</p>

123456: <p>I have changed the shape now to 2 * 128 * 128.<br>
Now it does not give error related to any mismatch of shape but error related to memory.</p>
<p>RuntimeError: [enforce fail at alloc_cpu.cpp:73] . DefaultCPUAllocator: can’t allocate memory: you tried to allocate 68719476736 bytes. Error code 12 (Cannot allocate memory)</p>
<p>I will try to solve this at my end. If there will be any problem, I will get back to you.</p>
<p>If you know any idea for this, you can share.<br>
Thank you so much for your help.</p>

ptrblck: <p>I don’t know which layer causes the issue, but note that fixing both shape mismatch errors via increasing the feature dimension will create a weight matrix of <code>524288 * 32768</code> which uses 16GB, so you might need to check if you want to reduce the features instead.</p>

123456: <p>Actually my GPU is of 8GB only. I have already reduce it from 64 to 32 in the 4th conv. layer, but if I reduce it more, than it will impact the performance of my network.<br>
In self.fc=nn.Linear(524288*32768),<br>
the first argument is the no. of feature maps from previous layer * spatial dimensions of image (height, width).<br>
The second argument is the total no. of output feature maps * spatial dimensions of image</p>
<p>i.e we got 32 * 128 * 128 = 524288 and 2 * 128 * 128 = 32768</p>
<p>Am I right?</p>

ptrblck: <p>The second argument is just the number of features in the output. You are then reinterpreting it as the number of feature maps * spatial dimensions of these features, but the linear layer doesn’t have any knowledge of this representation.<br>
Also, in my previous post I forgot to multiply the size of the number of bytes per element (4 for <code>float32</code>), which then yields the needed 64GB raised in your error message.</p>

123456: <p>Before multiplying the output feature maps with spatial dimension, I have written only  2 in the second argument, but at that time I was getting error like below</p>
<p>RuntimeError: The size of tensor a (2) must match the size of tensor b (128) at non-singleton dimension 3</p>

ptrblck: <p>If you change the output shape of your model you would most likely need to change more parts of the code, e.g. the target tensor shape etc. Check which line of code raises the error and try to understand what is causing the shape mismatch now.</p>

123456: <p>“The size of tensor a (2) must match the size of tensor b (128) at non-singleton dimension 3”</p>
<p>This Error is raised in training file where loss is calculate between the output feature map and ground truth labels.<br>
criterion(outputs,labels)</p>
<p>here input tensor is of shape 128 * 128 * 6. (6 channels because two images are concatenated in a tensor).<br>
output (binary map) is two images of size 128 * 128 corresponding to two input images.</p>
<p>output_shape =torch.Size({1,2])<br>
target labels have size =torch.Size([1, 32, 128, 128])</p>
<p>I am very much confused in all this shapes and errors.</p>