Trying to matmul a (1,4,2) and a (2,4,2) tensor - why is the 2nd expected to have dimension (2,2,2))

make1234: <p>Hello,</p>
<p>I want to execute this to understand how matmul works - i know it should throw an error.</p>
<pre><code class="lang-auto">import torch.nn
import numpy as np
data = [[[1, 2],[3, 4],[5,6],[7,8]]]
data1 = [[[1, 2],[3, 4],[5,6],[7,8]],[[1, 2],[3, 4],[5,6],[7,8]]]
t0 = torch.tensor(data)
t1 = torch.tensor(data1)
print(f"{t0.shape}  {t1.shape}")
print(f"{t0 @ t1}")

</code></pre>
<p>what I do not understand is why the error message says</p>
<pre><code class="lang-auto">RuntimeError: Expected size for first two dimensions of batch2 tensor to be: [2, 2] but got: [2, 4].
</code></pre>
<p>specifically why is dimension (2,2) in the first two entries expected?</p>

KFrank: <p>Hi Make!</p>
<aside class="quote no-group" data-username="make1234" data-post="1" data-topic="216710">
<div class="title">
<div class="quote-controls"></div>
<img alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/make1234/48/74826_2.png" class="avatar"> make1234:</div>
<blockquote>
<p>I want to execute this to understand how matmul works - i know it should throw an error.</p>
</blockquote>
</aside>
<p>A quick overview of <code>matmul()</code>:</p>
<p>When the tensors being multiplied have more than two dimensions, the leading<br>
dimensions (that is, all but the last two) are treated as batch dimensions and need<br>
to be broadcastable with one another.  In your case the leading dimensions are<br>
<code>[1]</code> and <code>[2]</code>, which are broadcastable.</p>
<p>The final two dimensions are the matrix dimensions.  In your case, these are <code>[4, 2]</code><br>
and <code>[4, 2]</code>.  However, for matrix multiplication, the second dimension of the first<br>
matrix much match the first dimension of the second matrix.  That is, a matrix <code>A</code> of<br>
shape <code>[m, o]</code> and a matrix <code>B</code> of shape <code>[o, n]</code> can be multiplied together to produce<br>
the matrix <code>A @ B</code> of shape <code>[m, n]</code>.  This is because <code>A</code> and <code>B</code> share the dimension<br>
<code>o</code> in the appropriate locations.</p>
<p>The matrix dimensions (the last two dimensions) of your <code>t0</code> and <code>t1</code> don’t match up<br>
correctly for matrix multiplication.  But if we swap the last two dimensions of <code>t1</code> they<br>
will.</p>
<p>Consider:</p>
<pre><code class="lang-plaintext">&gt;&gt;&gt; import torch
&gt;&gt;&gt; print (torch.__version__)
2.6.0+cu126
&gt;&gt;&gt; 
&gt;&gt;&gt; data = [[[1, 2],[3, 4],[5,6],[7,8]]]
&gt;&gt;&gt; data1 = [[[1, 2],[3, 4],[5,6],[7,8]],[[1, 2],[3, 4],[5,6],[7,8]]]
&gt;&gt;&gt; t0 = torch.tensor(data)
&gt;&gt;&gt; t1 = torch.tensor(data1)
&gt;&gt;&gt; t1p = t1.permute (0, 2, 1)
&gt;&gt;&gt; 
&gt;&gt;&gt; print(f"{t0.shape}  {t1.shape}  {t1p.shape}")
torch.Size([1, 4, 2])  torch.Size([2, 4, 2])  torch.Size([2, 2, 4])
&gt;&gt;&gt; 
&gt;&gt;&gt; print(f"{(t0 @ t1p).shape}")
torch.Size([2, 4, 4])
</code></pre>
<aside class="quote no-group">
<blockquote>
<p>what I do not understand is why the error message says</p>
<pre><code class="lang-auto">RuntimeError: Expected size for first two dimensions of batch2 tensor to be: [2, 2] but got: [2, 4].
</code></pre>
<p>specifically why is dimension (2,2) in the first two entries expected?</p>
</blockquote>
</aside>
<p>Assuming that <code>t0</code> is as given and that the first dimension of <code>t1</code> happens to be <code>2</code>,<br>
then for matrix multiplication to work out, the second dimension of <code>t1</code> (its first matrix<br>
dimension) must be <code>2</code> to match the last dimension of <code>t0</code>, so indeed the first two<br>
dimensions of <code>t1</code> would be expected to be <code>[2. 2]</code>.</p>
<p>I will say that the error message is not as clear as it could be.  For example, <code>t1</code> could<br>
have shape <code>[7, 2, 13]</code>, and the broadcasted, batched matrix multiplication would work<br>
fine.  So the required condition is that the last dimension of <code>t0</code> equals the next-to-last<br>
dimension of <code>t1</code> (and that the batch dimensions be broadcastable).</p>
<p>Best.</p>
<p>K. Frank</p>

make1234: <p>thanks a lot for your efforts and thorough reply!</p>