Dice score changes for the same input tensors after reshaping

stark: <p>I’m calculating the Dice score to evaluate my model for a binary image segmentation problem.</p>
<p>The function I wrote in PyTorch is:</p>
<pre><code class="lang-auto">def dice_score_reduced_over_batch(x, y, smooth=1):
    assert x.ndim == y.ndim
    # reduction over all axes except 0 i.e. batch
    axes = tuple([i for i in range(1, x.ndim)])

    intersection = torch.abs((x * y).sum(dim=axes))
    union = torch.abs(x.sum(dim=axes)) + torch.abs(y.sum(dim=axes))
    dice = torch.mean(2. * (intersection + smooth) / (union + smooth), dim=0)
    return dice
</code></pre>
<p>The input tensors <code>x</code> and <code>y</code> have the shape <code>[batch_size, nChannel, height, width]</code> where <code>nChannel=1</code> since ground truth is a 2d binary mask. The standard way to calculate the dice score is to compute it along the <code>batch</code> axis and taking the mean value at the end (Right?). I found that the score is affected by the way inputs are flattened.</p>
<pre><code class="lang-auto">╔═══════════════════╦══════════════════╦════════╗
║ input tensor      ║ flattened tensor ║ dice   ║
╠═══════════════════╬══════════════════╬════════╣
║ [64, 1, 128, 128] ║ -                ║ 0.2754 ║
╠═══════════════════╬══════════════════╬════════╣
║ [64, 1, 128, 128] ║ [64, 16384]      ║ 0.2754 ║
╠═══════════════════╬══════════════════╬════════╣
║ [64, 1, 128, 128] ║ [1, 1048576]     ║ 0.3121 ║
╚═══════════════════╩══════════════════╩════════╝
</code></pre>
<p>My best guess was this difference is due to the way values are being averaged but it’s not the case. <strong>The code must return the exact same answer irrespective of the arrangement/shape of the input data.</strong> How this behavior can be explained? What’s the best way to avoid it?</p>

KFrank: <p>Hi Stark!</p>
<aside class="quote no-group quote-modified" data-username="stark" data-post="1" data-topic="104962" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/stark/48/29013_2.png" class="avatar"> stark:</div>
<blockquote>
<pre><code class="lang-auto">    # reduction over all axes except 0 i.e. batch
    axes = tuple([i for i in range(1, x.ndim)])
</code></pre>
<p>The standard way to calculate the dice score is to compute it along the <code>batch</code> axis and taking the mean value at the end (Right?).</p>
<pre><code class="lang-auto">║ [64, 1, 128, 128] ║ [1, 1048576]     ║ 0.3121 ║
</code></pre>
<p><strong>The code must return the exact same answer irrespective of the arrangement/shape of the input data.</strong></p>
</blockquote>
</aside>
<p>Flattening <code>[64, 1, 128, 128]</code> to <code>[1, 1048576]</code> mixes your batch<br>
dimension in with your image dimensions.</p>
<p>Based on your comments and the comment in your code (rather than<br>
on my absent knowledge of the Dice score), it seems that the Dice<br>
score should be independent of how the <em>image</em> dimensions are<br>
rearranged, but that the batch dimension plays a different role.</p>
<p>Note for example, that</p>
<pre data-code-wrap="python"><code class="lang-python"> intersection = torch.abs((x * y).sum(dim=axes))
</code></pre>
<p>performs the image sum before calling <code>abs()</code>.  If you also sum over<br>
the batch before calling <code>abs()</code> (and negative values are present), you’ll<br>
get a different value.</p>
<p>Similarly,</p>
<pre data-code-wrap="python"><code class="lang-python">dice = torch.mean(2. * (intersection + smooth) / (union + smooth), dim=0)
</code></pre>
<p>takes the batch mean after performing the division.  If you take the<br>
batch mean (by flattening <code>nBatch = 64</code> away) before you perform<br>
the division, you will get a different answer.</p>
<p>Best.</p>
<p>K. Frank</p>