Einsum, summation over ellipsis

username: Eva
cooked: <p>I’m trying to use einsum in my implementation of Losses (eg. Dice Loss) for network training. So far I trained on 2D images, and it worked nicely, but now I tried to extend the code to work also on 3D data.<br>
As I want the same code to work on both, 2 and 3D, I went for example from using<br>
torch.einsum(‘bkwh,bkwh-&gt;bk’)<br>
to<br>
torch.einsum('bk…,‘bk…-&gt;bk’),<br>
basically using ellipsis for the unknown number of dimensions. The desired output is still the same though - elementwise multiplication and then summation over all but first two dimensions.</p>
<p>But so, with this syntax, I want to do the summation over the ellipsis, which according to the documentation should work ( <a href="https://pytorch.org/docs/stable/generated/torch.einsum.html" class="inline-onebox" rel="noopener nofollow ugc">torch.einsum — PyTorch 1.8.1 documentation</a> :<br>
See note: " <code>torch.einsum</code> handles ellipsis (‘…’) differently from NumPy in that it allows dimensions covered by the ellipsis to be summed over, that is, ellipsis are not required to be part of the output. " ).</p>
<p>But for some reason it doesn’t work for me; when I use this particular operation on two tensors, both of size [32, 7, 24, 24, 24] for example, it throws an error: " RuntimeError: shape ‘[32, 7]’ is invalid for input of size 3096576 ". So it would seem like it doesn’t know how to sum over ellipsis after all?<br>
Or am I somehow using it wrong?</p>

username: I-Love-U
cooked: <p>I get the following results, and it seems to be working fine:</p>
<pre><code class="lang-python">torch.einsum('bk..., bk... -&gt; bk', torch.randn(32, 7, 24, 24, 24), torch.randn(32, 7, 24, 24, 24)).shape
Out[26]: torch.Size([32, 7])
</code></pre>
<p>Maybe you need to ensure your tensor’s shape.</p>

username: KFrank
cooked: <p>Hi Eva (and Lart)!</p>
<aside class="quote no-group quote-modified" data-username="Eva" data-post="1" data-topic="122296" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/e/dbc845/48.png" class="avatar"> Eva:</div>
<blockquote>
<p>torch.einsum('bk…,‘bk…-&gt;bk’),<br>
…<br>
it throws an error: " RuntimeError: shape ‘[32, 7]’ is invalid for input of size 3096576 ".</p>
</blockquote>
</aside>
<p>Please check your pytorch version – this looks like an <code>einsum()</code><br>
bug that got fixed somewhere along the line.  (I’m not sure exactly<br>
when.)</p>
<p>Lart’s example works for me with a 1.8 nightly build, version<br>
‘1.8.0.dev20210117’, but fails – giving your error – with an older<br>
1.7 stable build, version ‘1.7.1’.</p>
<p>If you’re not using the current stable build (1.8.1), you might try<br>
upgrading to that to see if things work.  (I haven’t tried it on the<br>
1.8.1 stable build, but the 1.8.0 nightly version suggests that the<br>
stable build should work.)</p>
<p>Best.</p>
<p>K. Frank</p>

username: I-Love-U
cooked: <p>Yes, you are right! <img src="https://discuss.pytorch.org/images/emoji/apple/grinning_face_with_smiling_eyes.png?v=9" title=":grinning_face_with_smiling_eyes:" class="emoji" alt=":grinning_face_with_smiling_eyes:"></p>