Fixing RuntimeError: size mismatch, m1: [2 x 21504], m2: [25088 x 512]

username: archy2211
cooked: <p>HI Im not sure what happen but when i try to run my code this error keep pop up:RuntimeError: size mismatch, m1: [2 x 21504], m2: [25088 x 512] at C:\Users\builder\AppData\Local\Temp\pip-req-build-r5jc5joa\aten\src\TH/generic/THTensorMath.cpp:41</p>
<p>Here is my code can anyone check what went wrong<br>
class Bottleneck(nn.Module):<br>
def <strong>init</strong>(self, inp, oup, stride, expansion):<br>
super(Bottleneck, self).<strong>init</strong>()<br>
self.connect = stride == 1 and inp == oup<br>
#<br>
self.conv = nn.Sequential(<br>
# pw<br>
nn.Conv2d(inp, inp * expansion, 1, 1, 0, bias=False),<br>
nn.BatchNorm2d(inp * expansion),<br>
nn.PReLU(inp * expansion),<br>
# nn.ReLU(inplace=True),</p>
<pre><code>        # dw
        nn.Conv2d(inp * expansion, inp * expansion, 3, stride, 1, groups=inp * expansion, bias=False),
        nn.BatchNorm2d(inp * expansion),
        nn.PReLU(inp * expansion),
        # nn.ReLU(inplace=True),

        # pw-linear
        nn.Conv2d(inp * expansion, oup, 1, 1, 0, bias=False),
        nn.BatchNorm2d(oup),
    )

def forward(self, x):
    if self.connect:
        return x + self.conv(x)
    else:
        return self.conv(x)
</code></pre>
<p>class ConvBlock(nn.Module):<br>
def <strong>init</strong>(self, inp, oup, k, s, p, dw=False, linear=False):<br>
super(ConvBlock, self).<strong>init</strong>()<br>
self.linear = linear<br>
if dw:<br>
self.conv = nn.Conv2d(inp, oup, k, s, p, groups=inp, bias=False)<br>
else:<br>
self.conv = nn.Conv2d(inp, oup, k, s, p, bias=False)<br>
self.bn = nn.BatchNorm2d(oup)<br>
if not linear:<br>
self.prelu = nn.PReLU(oup)</p>
<pre><code>def forward(self, x):
    x = self.conv(x)
    x = self.bn(x)
    if self.linear:
        return x
    else:
        return self.prelu(x)
</code></pre>
<p>MobiFace_bottleneck_setting = [<br>
# t, c , n, s<br>
[2, 64, 1, 2],<br>
[2, 64, 2, 1],<br>
[4, 128, 1, 2],<br>
[2, 128, 3, 1],<br>
[4, 256, 1, 2],<br>
[2, 256, 6, 1]<br>
]<br>
class MobiFace(nn.Module):<br>
def <strong>init</strong>(self, bottleneck_setting=MobiFace_bottleneck_setting, final_linear=False):<br>
super(MobiFace, self).<strong>init</strong>()<br>
self.final_linear = final_linear</p>
<pre><code>    self.conv1 = ConvBlock(3, 64, 3, 2, 1)

    self.dw_conv1 = ConvBlock(64, 64, 3, 1, 1, dw=True)

    self.inplanes = 64
    block = Bottleneck
    self.blocks = self._make_layer(block, bottleneck_setting)

    self.conv2 = ConvBlock(256, 512, 1, 1, 0, linear=True)

    self.linear1 = nn.Linear(7*7*512, 512)

    self.prelu1 = nn.PReLU()

    for m in self.modules():
        if isinstance(m, nn.Conv2d):
            n = m.kernel_size[0] * m.kernel_size[1] * m.out_channels
            m.weight.data.normal_(0, math.sqrt(2. / n))
        elif isinstance(m, nn.BatchNorm2d):
            m.weight.data.fill_(1)
            m.bias.data.zero_()

def _make_layer(self, block, setting):
    layers = []
    for t, c, n, s in setting:
        for i in range(n):
            if i == 0:
                layers.append(block(self.inplanes, c, s, t))
            else:
                layers.append(block(self.inplanes, c, 1, t))
            self.inplanes = c

    return nn.Sequential(*layers)

def forward(self, x):
    x = self.conv1(x)
    x = self.dw_conv1(x)
    x = self.blocks(x)
    x = self.conv2(x)
    x = x.view(x.size(0), -1)
    x = self.linear1(x)
    if self.final_linear is False:
        x = self.prelu1(x)

    return x
</code></pre>
<p>class ArcMarginProduct(nn.Module):<br>
def <strong>init</strong>(self, in_features=128, out_features=200, s=32.0, m=0.50, easy_margin=False):<br>
super(ArcMarginProduct, self).<strong>init</strong>()<br>
self.in_features = in_features<br>
self.out_features = out_features<br>
self.s = s<br>
self.m = m<br>
self.weight = Parameter(torch.Tensor(out_features, in_features))<br>
nn.init.xavier_uniform_(self.weight)<br>
# init.kaiming_uniform_()<br>
# self.weight.data.normal_(std=0.001)</p>
<pre><code>    self.easy_margin = easy_margin
    self.cos_m = math.cos(m)
    self.sin_m = math.sin(m)
    # make the function cos(theta+m) monotonic decreasing while theta in [0°,180°]
    self.th = math.cos(math.pi - m)
    self.mm = math.sin(math.pi - m) * m

def forward(self, x, label):
    cosine = F.linear(F.normalize(x), F.normalize(self.weight))
    sine = torch.sqrt(1.0 - torch.pow(cosine, 2))
    phi = cosine * self.cos_m - sine * self.sin_m
    if self.easy_margin:
        phi = torch.where(cosine &gt; 0, phi, cosine)
    else:
        phi = torch.where((cosine - self.th) &gt; 0, phi, cosine - self.mm)

    one_hot = torch.zeros(cosine.size(), device='cuda')
    one_hot.scatter_(1, label.view(-1, 1).long(), 1)
    output = (one_hot * phi) + ((1.0 - one_hot) * cosine)
    output *= self.s
    return output
</code></pre>
<p>if <strong>name</strong> == “<strong>main</strong>”:<br>
input = Variable(torch.FloatTensor(2, 3, 112, 96))<br>
net = MobiFace()<br>
print(net)<br>
x = net(input)<br>
print(x.shape)</p>

username: ptrblck
cooked: <p>I assume the error might be raised in this line of code:</p>
<pre><code class="lang-python">x = self.linear1(x)
</code></pre>
<p>If that’s the case, the number of features in the flattened activation doesn’t match the specified <code>in_features</code> in the <code>self.linear1</code> layer and you would have to adapt the layer.</p>
<p>PS: you can post code snippets by wrapping them into three backticks ```, which makes debugging easier <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=9" title=":wink:" class="emoji" alt=":wink:"></p>

username: archy2211
cooked: <p>can you explain what you mean byy specified.</p>

username: ptrblck
cooked: <p>While creating the <code>nn.Linear</code> layer, you would have to set specific numbers for the <code>in_features</code> and <code>out_features</code>. I guess, that the number for <code>in_features</code> is wrong and has to be changed to the number of expected features from the incoming activation.</p>
<p>To get this number, you could add a <code>print</code> statement inside the <code>forward</code> method and print the shape of this activation (e.g. via <code>print(x.shape)</code>) before passing it to the layer, which raises this error.</p>