How does the @ sign work in this instance?

Prateek_Malhotra: <p>I don’t understand why this code snippet gives the output it gives. Can you explain the math behind this or post a link to someplace explaining higher dimensional matrix multiplication? Thanks!</p>
<pre><code>A = torch.randn(1, 64, 1152, 1, 8)
B = torch.randn(10, 1, 1152, 8, 16)

C = A @ B
print(C.size())
</code></pre>
<p>output: torch.Size([10, 64, 1152, 1, 16])</p>

tom: <p>The matrix multiplication(s) are done between the last two dimensions (1×8 @ 8×16 --&gt; 1×16).  The remaining first three dimensions are broadcast and are ‘batch’, so you get 10×64×1152 matrix multiplications.</p>
<p>Best regards</p>
<p>Thomas</p>

Prateek_Malhotra: <p>Thank you that’s very helpful!</p>

Yuliang_Zhu: <pre><code class="lang-auto">torch.bmm()
</code></pre>
<p><a href="https://pytorch.org/docs/stable/generated/torch.bmm.html?highlight=bmm#torch.bmm" class="onebox" target="_blank" rel="noopener nofollow ugc">https://pytorch.org/docs/stable/generated/torch.bmm.html?highlight=bmm#torch.bmm</a></p>

TriKri: <p><a class="mention" href="/u/tom">@tom</a> Is the <code>@</code> operator documented somewhere? What if <code>A</code> and <code>B</code> have different number of dimensions? What if either <code>A</code> or <code>B</code> only has one dimension? I’m trying to understand how this operator can be used to calculate the weighted average according to a formula given in <a href="https://discuss.pytorch.org/t/compute-the-weighted-average-in-pytorch/133070/2">this answer</a>.</p>

tom: <p>It’s a bit tricky and the link is not obvious, but in fact the A @ B operator internally maps to  <a href="https://pytorch.org/docs/stable/generated/torch.matmul.html" rel="noopener nofollow ugc">torch.matmul(A, B)</a>, so you can use the linked documentation for the latter.</p>
<p>Best regards</p>
<p>Thomas</p>

JonathanSum: <p>Is there a class or tutorial that talks about this? I guess this is math? I didn’t see this on my linear algebra class.<br>
Thank you.</p>

tom: <p>To my mind, the trouble of maths lectures is that of all the explanations of a given thing, the subset of those that resonate with the student is very individual and whether the explanation presented in a class is one of resonating ones for you is a bit of a chance thing.</p>
<p>That said, I like to identify n x m matrices with linear functions mapping R^m to R^n by multiplication to the left. Then when you spell out the composition of these functions and believe that it is represented by the product matrix, this tells you how the dimensions have to be in matrix multiplication.</p>
<p>Best regards</p>
<p>Thomas</p>

CallMeMSL: <p>Small correction for people that stumble upon this Thread: <code>A @ B</code> calls <code>torch.matmul(B,A)</code></p>

ptrblck: <p>I don’t think so:</p>
<pre data-code-wrap="python"><code class="lang-python">A = torch.randn(1, 2)
B = torch.randn(2, 3)

# works
C = torch.matmul(A, B)
print(C.shape)
# torch.Size([1, 3])

# works
C = A @ B
print(C.shape)
# torch.Size([1, 3])

# breaks
torch.matmul(B, A)
# RuntimeError: mat1 and mat2 shapes cannot be multiplied (2x3 and 1x2)

# breaks
B @ A
# RuntimeError: mat1 and mat2 shapes cannot be multiplied (2x3 and 1x2)
</code></pre>

Roshan_Santhosh: <p>Confirming <a class="mention" href="/u/ptrblck">@ptrblck</a>’s comment. A@B is in fact equal to torch.matmul(A,B) and not matmul(B,A)</p>