Can't design neural network to map mfcc to images

username: ai007
cooked: <p>I am trying to map mfcc to images but I can’t find out what the correct parameters should be for my nn.Linear and Conv2d layers.</p>
<p>This is the error I get:<br>
“RuntimeError: mat1 and mat2 shapes cannot be multiplied (64x196608 and 25565904x26)”</p>
<p>The images and mfcc features which are in npy format are pre loaded in the facedataset. (Get item from the dataset class returns both image and audio tuple.<br>
This is the code: (Can’t upload more than one image<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/d/c/dcc949a918581263443849826fd33f52e99da89b.png" data-download-href="https://discuss.pytorch.org/uploads/default/dcc949a918581263443849826fd33f52e99da89b" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/d/c/dcc949a918581263443849826fd33f52e99da89b.png" alt="image" data-base62-sha1="vvaiD41QnJnr5lrN5v1QPxIzk03" width="666" height="500" data-dominant-color="202222"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1188×891 46.5 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

username: ptrblck
cooked: <aside class="quote no-group" data-username="ai007" data-post="1" data-topic="146973">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/a/a88e4f/48.png" class="avatar"> ai007:</div>
<blockquote>
<p>“RuntimeError: mat1 and mat2 shapes cannot be multiplied (64x196608 and 25565904x26)”</p>
</blockquote>
</aside>
<p>This error is most likely raised in an <code>nn.Linear</code> layer, which expects the input to have <code>25565904</code> features while your current input has <code>196608</code> features.<br>
Based on the screenshot it seems th be <code>lin1</code> so you might want to remove the multiplication with <code>13</code>.</p>
<p>PS: you can post code snippets by wrapping them into three backticks ```, which makes debugging easier <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=12" title=":wink:" class="emoji" alt=":wink:" loading="lazy" width="20" height="20"></p>

username: ptrblck
cooked: <p><code>self.conv1</code> expects a 4-dimensional tensor in the shape <code>[batch_size, channels, height, width]</code> while your <code>nn.Sequential</code> container passes the 2-dmensional output of the previous linear layer to it.<br>
Either create a custom <code>Unflatten/Unsqueeze</code> module and add it between these layers or split the modules into two parts and <code>unsqueeze</code> the activation manually in the <code>forward</code> method.</p>

username: ptrblck
cooked: <p>I’m not sure which dimensions you are <code>unsqueezing</code> now but based on the initial code snippet it seems you are adding the batch dimension, which sounds wrong.<br>
Keep the batch size equal in the <code>forward</code> method to avoid these shape mismatches.</p>

username: ptrblck
cooked: <p>You can write a custom <code>View</code> module, e.g. as:</p>
<pre><code class="lang-python">class View(nn.Module):
    def __init__(self, size):
        super().__init__()
        self.size = size
        
    def forward(self, x):
        x = x.view(self.size)
        return x
    

model = nn.Sequential(
    nn.Linear(1, 2048),
    View(size=(-1, 128, 4, 4)),
    nn.ConvTranspose2d(128, 1, 1)
)

x = torch.randn(1, 1)
out = model(x)
</code></pre>

username: ptrblck
cooked: <aside class="quote no-group quote-post-not-found" data-username="ai007" data-post="11" data-topic="146973">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/a/a88e4f/48.png" class="avatar"> ai007:</div>
<blockquote>
<p><code>TypeError: 'View' object is not iterable</code></p>
</blockquote>
</aside>
<p>This error is raised as you are using <code>OrderedDict</code> wrong and the <code>View</code> module misses it’s key.</p>

username: ptrblck
cooked: <p>You have forgotten to copy the <code>forward</code> method from my code snippet.</p>

username: ptrblck
cooked: <p>I cannot reproduce the error and your code works fine for me:</p>
<pre><code class="lang-python">model = NeuralNetwork()
x = torch.randn(1, 13)
out = model(x)
print(out.shape)
# torch.Size([1, 3, 766, 766])
</code></pre>
<p>Make sure the posted code can be executed with e.g. random data and in fact is able to reproduce the claimed error, please.</p>
<p>You are only seeing two print statements since you are overwriting the <code>PrintSize</code> modules with the same names as <code>print1</code>. Change the <code>key</code> and it would print the others, too.</p>

username: ptrblck
cooked: <p>It seems the <code>import</code> is failing so make sure <code>model</code> is actually a script which can be imported.<br>
If you get stuck, just copy/paste the model definition into your current script.</p>

username: ptrblck
cooked: <aside class="quote no-group quote-post-not-found" data-username="ai007" data-post="20" data-topic="146973">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/a/a88e4f/48.png" class="avatar"> ai007:</div>
<blockquote>
<p>For a reason when I am printing the prediction I get tensors but not in size of (255x255x3)which is the output that my neural network is supposed to produce.</p>
</blockquote>
</aside>
<p>I don’t understand this statement so could you explain a bit more where you are printing the shape of <code>pred</code>, what it shows, and where it seems to change?</p>
<p>Based on the error message it seems you are passing an unsupported input to <code>cv2.imwrite</code>, so you would need to at least call <code>pred.numpy()</code> once the shape issue is resolved.</p>

username: ptrblck
cooked: <aside class="quote no-group quote-post-not-found" data-username="ai007" data-post="22" data-topic="146973">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/a/a88e4f/48.png" class="avatar"> ai007:</div>
<blockquote>
<p>When I type pred.numpy() I get this error:</p>
</blockquote>
</aside>
<p>Did you try to use the suggested code, i.e. detaching the tensor and pushing it to the CPU?</p>
<aside class="quote no-group quote-post-not-found" data-username="ai007" data-post="22" data-topic="146973">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/a/a88e4f/48.png" class="avatar"> ai007:</div>
<blockquote>
<p>Those tensors should be (255x255x3) after the prediction of the network so I think I am not doing the prediction properly since I have tested the network before and I know it produces (255x255x3) output.</p>
</blockquote>
</aside>
<p>By default PyTorch expects an input of <code>[batch_size, channels, height, width]</code> for <code>*2d</code> layers. As I don’t see any <code>permute</code> call in your code I would thus expect that the output is still in this shape.</p>