Multiply and accumulate two tensors across batch size

psakamoori: <p>I have two tensors with below size</p>
<p>x = torch.Size([10, 3, 128])   → [batch_size, no_of_IDs, no_events_per_ID]<br>
attention_value = torch.Size([10, 3, 1])   → [batch_size, no_of_IDs, att_value]</p>
<p>How to mac (multiply and accumulate)  these 3D tensors across batch_size i.e. “x with attention_value”</p>
<p>When tried: torch.matmul(x, att_value)  giving below error</p>
<p>RuntimeError: Expected batch2_sizes[0] == bs &amp;&amp; batch2_sizes[1] == contraction_size to be true, but got false.  (Could this error message be improved?  If so, please report an enhancement request to PyTorch.)</p>
<p>Output size should be [10, 128]</p>

KFrank: <p>Hi Pradeep!</p>
<aside class="quote no-group quote-modified" data-username="psakamoori" data-post="1" data-topic="145128" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/psakamoori/48/46995_2.png" class="avatar"> psakamoori:</div>
<blockquote>
<p>x = torch.Size([10, 3, 128])   → [batch_size, no_of_IDs, no_events_per_ID]<br>
attention_value = torch.Size([10, 3, 1])   → [batch_size, no_of_IDs, att_value]<br>
…<br>
Output size should be [10, 128]</p>
</blockquote>
</aside>
<p>Based on the tensor shapes you give, I assume that you want to multiply<br>
each matrix in the batch <code>x</code> by the corresponding matrix in the batch<br>
<code>attention_value</code> (matrix-multiply, not element wise).</p>
<p>I assume that you are <em>not</em> somehow summing across elements of the<br>
batches, so I’m not sure what you mean by “accumulate.”</p>
<p>You can use <code>bmm()</code> (“batch matrix multiply”) after using <code>transpose()</code><br>
to line the dimensions up correctly (and then use <code>squeeze()</code> to get rid<br>
of the singleton dimension):</p>
<pre><code class="lang-plaintext">&gt;&gt;&gt; import torch
&gt;&gt;&gt; torch.__version__
'1.10.2'
&gt;&gt;&gt; x = torch.randn (10, 3, 5)   # using 5 instead of 128
&gt;&gt;&gt; attention_value = torch.randn (10, 3, 1)
&gt;&gt;&gt; torch.bmm (x.transpose (1, 2), attention_value).squeeze()
tensor([[ 2.3298,  0.5740,  2.4056,  1.2855, -0.7073],
        [-0.0287, -4.0949, -0.5672,  1.7701,  1.9519],
        [ 0.6117,  0.1664, -1.3646, -1.6738, -1.7130],
        [ 0.6985, -0.3951, -0.5439, -0.5091, -0.2386],
        [ 2.7237, -0.8359, -0.4583,  1.1985,  3.0083],
        [ 1.1887, -0.4801,  1.3998, -1.3990,  2.6126],
        [ 1.2119,  0.2461,  1.8558, -0.8090,  2.0118],
        [-0.2383,  1.4226, -0.2152,  0.9591,  1.5240],
        [-0.4384,  0.0184, -0.9334,  0.0796, -0.6989],
        [ 0.7386, -0.6474,  0.9869, -0.5547, -0.0543]])
</code></pre>
<p>Best.</p>
<p>K. Frank</p>

psakamoori: <p>Thank you Frank. Was able to solve the logic with below steps…</p>
<p>Perfoming permute on x[10, 3, 128]</p>
<p>x = torch.permute(x, (0, 2, 1))  =&gt; x[10, 128, 3] and att_value[10, 3, 1]</p>
<p>And perform<br>
context_vector = torch.matmual(x, att_value) → [10, 128, 1] followed by<br>
torch.squeeze(context_veactor, 2) → [10, 128]</p>
<p>Will try your method as well.<br>
Thank you for your time and details.</p>