Error "mat1 and mat2 shapes cannot be multiplied"

alex234321: <p>Hello, new to pytorch I am trying to create a model using the script found under <a href="https://codeshare.io/1Y4k8m" class="inline-onebox" rel="noopener nofollow ugc">Dec 01 9:08 AM - Codeshare</a></p>
<p>but when I run this code I get the error</p>
<p><code>RuntimeError: mat1 and mat2 shapes cannot be multiplied (64x38850 and 259x512)</code></p>
<p>I am not sure where the 38850 comes from (which is 150 * 259). Any ideas?</p>

ptrblck: <p>Based on your code the shape mismatch is created in the first linear layer.<br>
Your flattened input tensor has a shape of <code>[batch_size=64, features=38850]</code> while the linear layer expects 259 features. I guess your input tensors might contain an additional dimension (maybe a temporal dimension as a sequence length) of 512 samples?</p>

alex234321: <p>As far as I can see the input data has the shape “torch.Size([150, 259])”. The code is here:</p>
<pre><code class="lang-auto">class CustomMusicDataset(Dataset):
    def __init__(self, maps, toread = None):
           
        self.data = {}
        self.len = {}
    
        # Read all the maps
        for music_map in maps:
            # prepare the datafile
            self.data[music_map] = []
            self.len[music_map] = 0

            # Read map and select some random elements
            mapname = music_map + ".map"
            with open(mapname) as filein:
                for line in filein.readlines():                                        
                    location, myhash, title = line.split(";")
                    print(f"Analyzing {location}")

                    # Read the actual spetral data
                    for idx in range(10):
                        specfilename = f"{location}/{myhash}_{idx}.pt"
                        if Path(specfilename).exists():
                            spec = torch.load(specfilename)
                            if spec.shape[1] &gt; 259:
                                spec = spec[:,:259]

                            #print(f"appending spectra with shape {spec.shape}")
                            self.data[music_map].append(spec)
                            self.len[music_map] += 1

                    if toread is not None and self.len[music_map]&gt;toread:
                        break

        for music_map, n in self.len.items():
            print(f"{music_map} has {n} entries.")
</code></pre>

ptrblck: <p>That’s not necessarily the case, since you are only checking and slicing <code>dim1</code> in:</p>
<pre><code class="lang-python">if spec.shape[1] &gt; 259:
    spec = spec[:,:259]
</code></pre>
<p>The tensor can have multiple dimensions as seen here:</p>
<pre><code class="lang-python">spec = torch.randn(64, 500, 150)

if spec.shape[1] &gt; 259:
    spec = spec[:,:259]

print(spec.shape)
# torch.Size([64, 259, 150])

print(spec.view(spec.size(0), -1).shape)
# torch.Size([64, 38850])
</code></pre>

alex234321: <p>But when I check all the entries I see every single entry has shape <code>[150, 259]</code>!</p>
<p>There must be an error in the code itself that I shared yesterday.</p>

ptrblck: <p>This is exactly the issue I’m trying to point out.<br>
As you’ve confirmed, each sample has already two dimensions in the shape <code>[150, 259]</code>. A batch of these samples would thus have the shape <code>[batch_size, 150, 259]</code> and flattening the “feature dimensions” creates a tensor in the shape <code>[batch_size, 38850]</code> (as also seen in my code) which then causes the error:</p>
<pre><code class="lang-python">RuntimeError: mat1 and mat2 shapes cannot be multiplied (64x38850 and 259x512)
</code></pre>
<p>since <code>295</code> features are expected, while you are passing <code>38850</code> features.</p>

alex234321: <p>But how do I fix this problem?</p>
<p>I did follow a tutorial for PyTorch to train a network, and this is the code I created.</p>
<p>So somewhere I seem to have made a mistake. But where, and how to fix it?</p>

ptrblck: <p>Check what the dimensions in your samples represent and what your actual use case is.<br>
Currently you are using a linear layer, which expects <code>259</code> features, while your input samples contains <code>150*259</code> features, so what does the shape <code>150</code> represent?<br>
Is it a temporal dimension? If so, would you like to use <em>all</em> time steps, only a the last one, or any other?<br>
There isn’t a single way to solve the issue as it depends on your actual use case of course.<br>
If you want to use all features and cannot figure out what these dimensions represent, set <code>in_features</code> of the first linear layer to <code>38850</code>.</p>

alex234321: <p>What do you mean by <code>in_feature</code>? I do not see that parameter in my code.</p>
<p>Could you please show exactly where and how I can change the code? In fact, the <code>150*259</code> is a time spectrogram of the input data which I all want to use.</p>
<p>Ah, when I set <code>size_data = 259*150</code> that steps seems to work now!</p>
<p>But then when I want to use the network I get the error</p>
<p><code>size mismatch for linear_relu_stack.0.weight: copying a param with shape torch.Size([512, 38850]) from checkpoint, the shape in current model is torch.Size([512, 259]). </code></p>

alex234321: <p>Here is the code to use the network:</p>
<pre><code class="lang-auto">test_data = dataset_music.CustomMusicDataset(["Instrumental", "Pop"], 100)

size_data = 259

# Define model
class NeuralNetwork(nn.Module):
    def __init__(self):
        super(NeuralNetwork, self).__init__()
        self.flatten = nn.Flatten()
        self.linear_relu_stack = nn.Sequential(
            nn.Linear(size_data, 512),
            nn.ReLU(),
            nn.Linear(512, 512),
            nn.ReLU(),
            nn.Linear(512, 10),
            nn.ReLU()
        )

    def forward(self, x):
        x = self.flatten(x)
        logits = self.linear_relu_stack(x)
        return logits

model = NeuralNetwork()
model.load_state_dict(torch.load("model.pth"))
model.eval()

x = test_data.data["Instrumental"][0]
pred = model(x)
print(pred)
</code></pre>

alex234321: <p>When I change the <code>size_data</code> to</p>
<p><code>size_data = 259 * 150</code></p>
<p>I get the matrix multiplication error again:</p>
<p><code>RuntimeError: mat1 and mat2 shapes cannot be multiplied (150x259 and 38850x512)</code></p>
<p>So maybe I need to flatten something, or redo the model, or there is a bug, or PyTorch has a bug…?</p>

alex234321: <p>I tried to flatten the test data and changed the code to</p>
<p><code>pred = model(torch.flatten(x))</code></p>
<p>but this gives another error:</p>
<p><code>IndexError: Dimension out of range (expected to be in range of [-1, 0], but got 1)</code></p>

ptrblck: <p>Your test data is missing the batch dimension and you are now trying to use the dimension with the size <code>150</code> as the batch dim, which causes the error.<br>
Either use a <code>DataLoader</code>, which will add the missing batch dimension for you or add it manually via <code>unsqueeze(0)</code>.</p>

alex234321: <p>What is a batch dimension?</p>
<p>Anyway, I will create a github code example with the complete reproducible code, and I hope someone is able to help find the error…</p>

ptrblck: <p>The batch dimension is a dimension in tensors representing the samples in the tensor.<br>
E.g. a standard <code>nn.Linear</code> layer expects an input in the shape <code>[batch_size, in_features]</code>.<br>
I.e. an input in the shape <code>[64, 38850]</code> represents a tensor with 64 samples (data points) each containing 38850 features.<br>
Your new error is raised, since your input tensor does not contain the batch dimension any more and you are trying to use the tensor in the shape <code>[150, 259]</code> directly while <code>259*150</code> should be the flattened feature dimension.<br>
Use <code>x = x.unsqueeze(0)</code> before passing the input to the model, make sure it has the shape <code>[batch_size=1, 150, 259]</code> and it should work as before.</p>

alex234321: <p>Ok thanks,</p>
<p>that seems to “work” now, I do not get any error! Maybe the tutorial got that bit wrong?</p>
<p>But as a result I now get a “tensor”, especially something like</p>
<pre><code class="lang-auto">tensor([[ 0.0000, 11.3125,  0.0000,  0.0000,  0.0000,  0.0000,  0.0000,  0.0000,
          0.0000,  0.0000]], grad_fn=&lt;ReluBackward0&gt;)

Is there some tutorial/page on how to interpret that? Or I am doing still something wrong? I thought the result was be a single number?
</code></pre>

alex234321: <p>Ah wait, it gives me 10 values as this is the size of my last layer, correct?</p>

ptrblck: <p>Yes, the output represents clipped logits in the shape <code>[batch_size, 10]</code> where each entry in <code>dim1</code> represents the logit for the corresponding class index (out of 10 classes).<br>
I would recommend to remove the last <code>nn.ReLU</code> from your model to allow for negative and positive logits.</p>

alex234321: <p>I guess the best course of action would be to redo a tutorial.</p>
<p>Can you suggest a SIMPLE , REALLY SIMPLE tutorial to learn how to TRAIN a network and to USE a network, as SIMPLE as possible? I have two datasets: small snippets of music WITH and WITHOUT singing and I want to train a network to distinguish between music with and without singing?</p>
<p>Is there a tutorial which stick to the VERY essential of creating and using a network, skipping the billion options there are?</p>

ptrblck: <p>I would recommend to check <a href="https://pytorch.org/tutorials/recipes/recipes/defining_a_neural_network.html">this tutorial</a> which explains how a neural network can be defined for the <code>MNIST</code> dataset and you could also play around with this dataset first to get familiar with the dimensions of the input tensors etc. as I believe the tensor shapes and dimensions are what’s causing the trouble right now.<br>
Also, <a href="https://pytorch.org/tutorials/beginner/blitz/neural_networks_tutorial.html">this tutorial</a> might give you more insight into the model definition and training.<br>
Generally, add print statements showing the shape of tensors (inside the <code>model.forward</code> method as well as “outside”) to get a better idea what the batch dimension is, how the <code>DataLoader</code> is creating it etc.</p>