How to enforce symmetry in segmentation task loss

davidsriker: <p>I have a segmentation-like task, where the output is highly unbalanced. In order to address the fact it’s unbalanced I use <a href="https://arxiv.org/abs/1708.02002" rel="noopener nofollow ugc">Focal Loss</a>. I have another piece of information that I wish to use in the training loss and it’s the fact that the segmentation maps are always symmetric.</p>
<p>Is there a way to use this information in the loss?<br>
I couldn’t find a way.</p>

Eta_C: <p>Try <a href="https://github.com/JunMa11/SegLoss/blob/master/losses_pytorch/dice_loss.py" rel="noopener nofollow ugc">DiceLoss</a> ?</p>

davidsriker: <p>And why dice loss will guarantee symmetry?</p>

Eta_C: <p>DiceLoss is often used when the data is unbalanced.<br>
As for symmetry, I think it might be possible to train dataset using a half of gt.</p>

KFrank: <p>Hi David!</p>
<aside class="quote no-group" data-username="davidsriker" data-post="1" data-topic="109996" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/d/b5a626/48.png" class="avatar"> davidsriker:</div>
<blockquote>
<p>I have another piece of information that I wish to use in the training loss and it’s the fact that the segmentation maps are always symmetric.</p>
</blockquote>
</aside>
<p>As a general rule, if your problem has structure that (nearly) is always true,<br>
it is best to see if you can build the structure into you your network so that<br>
you get the desired result “automatically,” without the network having to<br>
“learn” the structure through training.</p>
<p>Let me use simple reflection symmetry as an example.  (You haven’t told<br>
us what specific symmetry your problem exhibits.)</p>
<p>Let’s say that the input to your network are 1024x1024 single-channel<br>
grayscale images that are <em>not</em> symmetric, and your ground-truth<br>
segmentation masks are 1024x1024 binary masks that always have a<br>
symmetry when reflected across the x-axis, that is, they are unchanged<br>
when flipped left to right.  Therefore all of the information in the masks is<br>
contained in only their right halves  – say, the 512x1024 unflipped right half.</p>
<p>So you could build a network that takes as input 512x1024x2-channel<br>
images, where the two channels are the right half of your original image,<br>
unchanged, and the left half, flipped left to right.  Have the output of your<br>
network (the predicted segmentation masks) be 512x1024 binary masks<br>
(which will be natural, given the 512x104 input).</p>
<p>This network has the reflection symmetry “built in,” makes full use of all<br>
of the information in the original (not-necessarily-symmetric) 1024x1024<br>
input images, and predicts what can be understood as fully-symmetric<br>
segmentation masks (which you could “unfold” back out to be full 1024x1024<br>
symmetric masks if you wanted to for some reason).</p>
<p>If you can’t go this route, and want to use the loss function to encourage<br>
symmetry of your predictions, you could add an “asymmetry penalty” to<br>
your loss.  Continuing with the reflection-symmetry example, take your<br>
predicted asymmetrical, 1024x1024 mask, and calculate, say, the<br>
intersection-over-union loss of the flipped 512x1024 left half versus the<br>
unflipped 512x1024 right half.  Add this penalty loss (with an adjustable<br>
multiplicative weight) to whatever regular training loss you are using,<br>
and train with this combined loss.  As you make your penalty weight<br>
increasing large, the training will be pushed to predict masks that are<br>
closer and closer to being symmetric.</p>
<p>Best.</p>
<p>K. Frank</p>

davidsriker: <aside class="quote no-group" data-username="KFrank" data-post="5" data-topic="109996">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/k/ecb155/48.png" class="avatar"> KFrank:</div>
<blockquote>
<p>This network has the reflection symmetry “built in,” makes full use of all<br>
of the information in the original (not-necessarily-symmetric) 1024x1024<br>
input images, and predicts what can be understood as fully-symmetric<br>
segmentation masks (which you could “unfold” back out to be full 1024x1024<br>
symmetric masks if you wanted to for some reason).</p>
<p>If you can’t go this route, and want to use the loss function to encourage<br>
symmetry of your predictions, you could add an “asymmetry penalty” to<br>
your loss. Continuing with the reflection-symmetry example, take your<br>
predicted asymmetrical, 1024x1024 mask, and calculate, say, the<br>
intersection-over-union loss of the flipped 512x1024 left half versus the<br>
unflipped 512x1024 right half. Add this penalty loss (with an adjustable<br>
multiplicative weight) to whatever regular training loss you are using,<br>
and train with this combined loss. As you make your penalty weight</p>
</blockquote>
</aside>
<p>So I can’t use the first approach because I have different spatial input sizes to the network.<br>
Regarding the second approach, I have symmetry along the diagonal. But the example you gave was spot on.</p>
<p>Thanks</p>

KFrank: <p>Hi David!</p>
<aside class="quote no-group" data-username="davidsriker" data-post="6" data-topic="109996" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/d/b5a626/48.png" class="avatar"> davidsriker:</div>
<blockquote>
<p>So I can’t use the first approach because I have different spatial input sizes to the network.</p>
</blockquote>
</aside>
<p>I think you still could build symmetry into the network itself.</p>
<p>Pad your smaller images to be the size of the largest, and pad them to be<br>
square, with the line of symmetry being, say, the lower-left-to-upper-right<br>
main diagonal.  Form the two-channel image out of the flipped and unflipped<br>
halves.  Set the upper-left (triangular) halves of both channels to zero, and<br>
predict the mask (half of the symmetrical mask) that lies in the lower-right<br>
triangular half of your square output “image.”</p>
<p>Best.</p>
<p>K. Frank</p>