RuntimeError: mat1 and mat2 shapes cannot be multiplied (4x48 and 192x256)

MLHafizur: <p>I am trying to construct a <code>Convolutional Autoencoder</code> model for time series data, Where <code>embed_size = 32</code> and <code>time_step = 400</code>, the model constructor is like this:</p>
<pre><code class="lang-auto">class CNNEncoder(nn.Module):
    def __init__(self, embedding_size, timestep) -&gt; None:
        super().__init__()
        self.conv = nn.Sequential(
            nn.Conv1d(in_channels=timestep, out_channels=128, kernel_size=3, padding=1),
            nn.AvgPool1d(3),
            nn.ReLU(True),
            nn.Conv1d(in_channels=128, out_channels=64, kernel_size=3, padding=1),
            nn.AvgPool1d(3),
            nn.ReLU(True),
            nn.Conv1d(in_channels=64, out_channels=32, kernel_size=3, padding=1),  
            nn.ReLU(True),
            nn.Conv1d(in_channels=32, out_channels=16, kernel_size=3, padding=1), 
            #nn.ReLU(True),
        )

        self.fc = nn.Sequential(
            nn.Linear(192, 256),
            nn.BatchNorm1d(256),
            nn.LeakyReLU(0.2),
            nn.Linear(256, embedding_size),
        )

    def forward(self, x):
        conv = self.conv(x)
        conv = torch.flatten(conv, start_dim=1)
        return self.fc(conv)


class CNNDecoder(nn.Module):
    def __init__(self, embedding_size, timestep) -&gt; None:
        super().__init__()
        self.fc = nn.Sequential(
            nn.Linear(embedding_size, 256),
            nn.BatchNorm1d(256),
            nn.ReLU(True),
            nn.Linear(256, 576),
            nn.ReLU(True),
            nn.Linear(576, 1024), 
            #nn.ReLU(True),
        )

        self.deconv = nn.Sequential(
            nn.ConvTranspose1d(1024, 576, 3, dilation=2),
            nn.BatchNorm1d(576),
            nn.ReLU(True),
            nn.ConvTranspose1d(576, 384, 4, padding=3, stride=3),
            nn.BatchNorm1d(384),
            nn.ReLU(True),
            nn.ConvTranspose1d(384, timestep, 3, 3, dilation=2),
            nn.Sigmoid(),
        )

    def forward(self, x):
        fc = self.fc(x)
        fc = fc.view(-1, 1024, 1) 
        return self.deconv(fc)


class AutoEncoder(nn.Module):
    def __init__(self, embed_size, time_step) -&gt; None:
        super().__init__()

        self.encoder = CNNEncoder(embed_size, time_step)
        self.decoder = CNNDecoder(embed_size, time_step)

    def encode(self, x):
        return self.encoder(x)

    def decode(self, x):
        return self.decoder(x)

    def forward(self, x):
        z = self.encode(x)
        return self.decode(z)
</code></pre>
<p>But I’m getting the error:</p>
<p><code>RuntimeError: mat1 and mat2 shapes cannot be multiplied (4x48 and 192x256)</code></p>
<p>I understand that there is a mismatch between the dimensions of the matrices I’m trying to multiply in one of the linear layers. And most probably the error occurs in <code>CNNEncoder</code> class, in the first linear layer of the <code>self.fc</code> sequential block. But I’m not sure regarding this. What would be the solution?</p>

akt42: <aside class="quote no-group" data-username="MLHafizur" data-post="1" data-topic="189627">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/m/bb73d2/48.png" class="avatar"> MLHafizur:</div>
<blockquote>
<p>And most probably the error occurs in <code>CNNEncoder</code> class, in the first linear layer of the <code>self.fc</code> sequential block. But I’m not sure regarding this.</p>
</blockquote>
</aside>
<p>I didn’t look at your code, but if you check the complete error trace, you can identify <em>exactly</em> where the error happens in your code.</p>

MLHafizur: <p>I am struggling to construct the model, need help !!!</p>

ptrblck: <p>The error message points to a shape mismatch in <code>self.fc</code> in <code>CNNEncoder</code>. Set the <code>in_features</code> of the first linear layer to <code>48</code> and it should work.</p>

MLHafizur: <aside class="quote no-group" data-username="ptrblck" data-post="4" data-topic="189627">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar"> ptrblck:</div>
<blockquote>
<p>48</p>
</blockquote>
</aside>
<p><a class="mention" href="/u/ptrblck">@ptrblck</a> Thanks a lot, can you please explain why <code>in_features=48</code> works?</p>

ptrblck: <p>The error message shows the activation shape first and the weight parameter shape afterwards:</p>
<aside class="quote no-group" data-username="MLHafizur" data-post="1" data-topic="189627">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/m/bb73d2/48.png" class="avatar"> MLHafizur:</div>
<blockquote>
<p>RuntimeError: mat1 and mat2 shapes cannot be multiplied (4x48 and 192x256)</p>
</blockquote>
</aside>
<p>I then checked if your flattening looks alright and it does:</p>
<pre data-code-wrap="python"><code class="lang-python">        conv = torch.flatten(conv, start_dim=1)
        return self.fc(conv)
</code></pre>
<p>so the previously used <code>in_features=192</code> is wrong for the current input shape and should be changed to the features in the incoming activation. Note that <code>conv</code> has a shape of <code>[4, 48]</code> based on the error message which is representing <code>[batch_size, features]</code> after the flattening.</p>