How can I do element-wise batch matrix multiplication?

username: wasiahmad
cooked: <p>I have two tensors of shape (16, 300) and (16, 300) where 16 is the batch size and 300 is some representation vector. I want to compute the element-wise batch matrix multiplication to produce a matrix (2d tensor) whose dimension will be (16, 300). So, in short I want to do 16 element-wise multiplication of two 1d-tensors.</p>
<p>I can do this using a for loop but is there any way, I can do it using torch API?</p>

username: jekbradbury
cooked: <p>If you want elementwise multiplication, use the multiplication operator (<code>*</code>); if you want batched matrix multiplication use <code>torch.bmm</code>.</p>

username: wasiahmad
cooked: <p><code>torch.bmm</code> does matrix multiplication, not element-wise multiplication, so it can’t fulfill my purpose. (*) operator with a for loop is working for me. Btw, I also tried <code>torch.cmul()</code> but it is not working. Can you tell me why? Its giving me the error - <code>AttributeError: module 'torch' has no attribute 'cmul'</code>. What I am missing?</p>

username: smth
cooked: <p>What is batch element-wise multiplication? Can you show me your for loop?</p>
<p>there is no <code>torch.cmul</code> in pytorch, not sure why/where you picked it up or why you expect it to be there.</p>

username: ZeweiChu
cooked: <p>If you have tensor a and b both of shape (16, 300) and you want to get a tensor of shape (16, 300) by element-wise multiplication, I assume you can do a*b?</p>

username: chenzhekl
cooked: <p>That is basically what <code>torch.mul</code> do. You may safely use it.</p>

username: wasiahmad
cooked: <p>Thank you everyone. <code>torch.mul</code> worked for me. I am getting (16, 300) tensor from the two tensors of shape (16, 300) using <code>torch.mul</code>.</p>

username: apaszke
cooked: <p>You should be able to get the same result just doing <code>a * b</code> (this expression expands to the same thing as <code>torch.mul(a, b)</code> or <code>a.mul(b)</code>).</p>

username: wangg12
cooked: <p>How can I do element-wise mul between a matrix and a vector, say <code>3x5</code> mat and <code>3x1</code> vec, as in numpy? Pytorch’s <code>torch.mul</code> told me that the sizes are inconsistent.</p>
<p>(solved by <code>mat*vec.expand_as(mat)</code>)</p>

username: jekbradbury
cooked: <p><code>mat @ vec</code> will also work in Py3.5+.</p>

username: wangg12
cooked: <p><code>mat @ vec</code> is not what I am looking for. I want element-wise multipication with auto broadcasting. <a class="mention" href="/u/jekbradbury">@jekbradbury</a></p>

username: smth
cooked: <p>we dont have auto-broadcasting yet.</p>

username: dpappas
cooked: <p>Could we use just</p>
<p>A * B<br>
if both of them are matrices</p>
<p>and<br>
A * v.expand_as(A)</p>
<p>if A is matrix and v is a vector ?</p>
<p>It worked for me but i am not sure if the BP works fine.</p>

username: wandering007
cooked: <p>I find that <code>torch.mul</code> is an overloaded function since <code>b</code> can be a tensor or value. Is <code>a = a.mul(b)</code> an in-place operation when <code>b</code> is a value?</p>

username: e.pignatelli
cooked: <p>I solved like this, but maybe I am late to the party:</p>
<pre><code class="lang-auto">def bmul(vec, mat, axis=0):
    mat = mat.transpose(axis, -1)
    return (mat * vec.expand_as(mat)).transpose(axis, -1)
</code></pre>