Unet obviously loosing accuracy with increasing epoch

username: Caipi
cooked: <p>Hello there,</p>
<p>I build myself a Unet accordingly to the original Unet paper with input 564x564 RGB and smaller output segmentation map: <a href="https://github.com/CaipiDE/unet_mirror" rel="noopener nofollow ugc">https://github.com/CaipiDE/unet_mirror</a></p>
<p>I also implemented the mirroring strategy from the paper. So given the fact that I wanted to have a segmentation mask as big as my input, my mirror_extrapolate algorithm mirrors each side with 98px. As a result I now insert a 756x756 input (image with mirroring sides) which results in a 564x564 segmentation map, perfect <img src="https://discuss.pytorch.org/images/emoji/apple/smiley.png?v=12" title=":smiley:" class="emoji" alt=":smiley:" loading="lazy" width="20" height="20"></p>
<p>I testet it on the inria satellite imagary labeling dataset for <strong>building detection</strong> and was surprised how good it looked after just the first epoch!</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/5/d/5dda831f529d94b298c03723bc7ad1f90dab7e03.png" data-download-href="https://discuss.pytorch.org/uploads/default/5dda831f529d94b298c03723bc7ad1f90dab7e03" title="vienna5_png"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/5/d/5dda831f529d94b298c03723bc7ad1f90dab7e03_2_249x249.png" alt="vienna5_png" data-base62-sha1="dogz1L8r4XjjH9czS8qVa86RsR5" width="249" height="249" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/5/d/5dda831f529d94b298c03723bc7ad1f90dab7e03_2_249x249.png, https://discuss.pytorch.org/uploads/default/optimized/3X/5/d/5dda831f529d94b298c03723bc7ad1f90dab7e03_2_373x373.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/5/d/5dda831f529d94b298c03723bc7ad1f90dab7e03_2_498x498.png 2x" data-dominant-color="888679"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">vienna5_png</span><span class="informations">564×564 672 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/d/7/d720e497355c38ecbf2c92a288f2e0797f16b247.png" data-download-href="https://discuss.pytorch.org/uploads/default/d720e497355c38ecbf2c92a288f2e0797f16b247" title="epoch_0_mask"><img src="https://discuss.pytorch.org/uploads/default/original/3X/d/7/d720e497355c38ecbf2c92a288f2e0797f16b247.png" alt="epoch_0_mask" data-base62-sha1="uH77ZBmCqUGR3cikmv0U4eb7Tnx" width="249" height="249" data-dominant-color="4A4A4A"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">epoch_0_mask</span><span class="informations">564×564 9.15 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>But with increasing epochs, the map just literally disappears? <img src="https://discuss.pytorch.org/images/emoji/apple/thinking.png?v=12" title=":thinking:" class="emoji" alt=":thinking:" loading="lazy" width="20" height="20"> The segmented objects have no sharp edges and are just some wobbely floating random points (not complete random but you get the point)…</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/9/7/97eab457060aa9215933b051fb19ad66aafe0555.png" data-download-href="https://discuss.pytorch.org/uploads/default/97eab457060aa9215933b051fb19ad66aafe0555" title="Capture"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/9/7/97eab457060aa9215933b051fb19ad66aafe0555_2_517x218.png" alt="Capture" data-base62-sha1="lFUWTPJDt48AA7k8H2f9GErH7IV" width="517" height="218" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/9/7/97eab457060aa9215933b051fb19ad66aafe0555_2_517x218.png, https://discuss.pytorch.org/uploads/default/optimized/3X/9/7/97eab457060aa9215933b051fb19ad66aafe0555_2_775x327.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/9/7/97eab457060aa9215933b051fb19ad66aafe0555_2_1034x436.png 2x" data-dominant-color="9A9A9A"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Capture</span><span class="informations">1419×600 346 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>Did anyone else maybe faced of this behaviour when they did unet or maybe its typicall for a mistake I made?</p>
<p>PS: here is again the Unet structure and my mirroring stuff how that works:<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/1/4/14d3f71618bb903d4e996c6e1cda571d049e36d6.png" data-download-href="https://discuss.pytorch.org/uploads/default/14d3f71618bb903d4e996c6e1cda571d049e36d6" title="u-net-architecture"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/1/4/14d3f71618bb903d4e996c6e1cda571d049e36d6_2_345x229.png" alt="u-net-architecture" data-base62-sha1="2YfGkm19Lj0IVwLZJVlmUK90WcS" width="345" height="229" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/1/4/14d3f71618bb903d4e996c6e1cda571d049e36d6_2_345x229.png, https://discuss.pytorch.org/uploads/default/optimized/3X/1/4/14d3f71618bb903d4e996c6e1cda571d049e36d6_2_517x343.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/1/4/14d3f71618bb903d4e996c6e1cda571d049e36d6_2_690x458.png 2x" data-dominant-color="F4F6F8"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">u-net-architecture</span><span class="informations">1555×1036 74.5 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/8/5/853be6f77b6b6aebdf75458cf786f25f52eea478.png" alt="Capture" data-base62-sha1="j0DQiiIBypNVvr46XRZ4QY9Nrao" width="151" height="151"></p>

username: user_123454321
cooked: <p>Hi, does the dice score keep increasing?<br>
Also, I noticed the masks have thin edges rather than ‘blobs’<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/c/b/cbed0b9d89595fa7c125b664349d8567793f5910.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/cbed0b9d89595fa7c125b664349d8567793f5910" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/c/b/cbed0b9d89595fa7c125b664349d8567793f5910_2_690x227.jpeg" alt="image" data-base62-sha1="t60NdDmhmtoeXj1JL0OaAVyBe00" width="690" height="227" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/c/b/cbed0b9d89595fa7c125b664349d8567793f5910_2_690x227.jpeg, https://discuss.pytorch.org/uploads/default/optimized/3X/c/b/cbed0b9d89595fa7c125b664349d8567793f5910_2_1035x340.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/c/b/cbed0b9d89595fa7c125b664349d8567793f5910_2_1380x454.jpeg 2x" data-dominant-color="272B29"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1682×554 197 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
The center image is the mask while the right one is the prediction (I used your model). So, I believe it is drawing now only the edges (and thus looks like the prediction is disappeared ?).</p>

username: Caipi
cooked: <p>Ground Truth are actually filled in my dataset:<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/e/6/e6af5717e2650a889f0d54f8ffc603f47ed9da98.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/e6af5717e2650a889f0d54f8ffc603f47ed9da98" title="tyrol-w26"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/e/6/e6af5717e2650a889f0d54f8ffc603f47ed9da98_2_517x172.jpeg" alt="tyrol-w26" data-base62-sha1="wUJtW8G1frBVY1QWQlWjhwF2MaI" width="517" height="172" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/e/6/e6af5717e2650a889f0d54f8ffc603f47ed9da98_2_517x172.jpeg, https://discuss.pytorch.org/uploads/default/optimized/3X/e/6/e6af5717e2650a889f0d54f8ffc603f47ed9da98_2_775x258.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/e/6/e6af5717e2650a889f0d54f8ffc603f47ed9da98_2_1034x344.jpeg 2x" data-dominant-color="272B29"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">tyrol-w26</span><span class="informations">1692×564 221 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>Also the Dicescore is doing what ever the flip he wants… Further here is my loss despite being negative the curve does look too bad I guess:<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/5/7/57dd973e12543460b55ee754d62a5cbfd20f5aab.png" alt="myplot" data-base62-sha1="cxiilRekrxHQfDzAbSv2XWbBgqn" width="320" height="240"></p>
<p>Another Unet implementation was done by Aladdin Persson, might have heard of him on YT, he implemented Unet like this: <a href="https://github.com/aladdinpersson/Machine-Learning-Collection/tree/master/ML/Pytorch/image_segmentation/semantic_segmentation_unet" class="inline-onebox" rel="noopener nofollow ugc">Machine-Learning-Collection/ML/Pytorch/image_segmentation/semantic_segmentation_unet at master · aladdinpersson/Machine-Learning-Collection · GitHub</a><br>
I firstly think that he cropped the tensors for the skip connection the wrong way (cropped those coming from the upconv not the skip ones) and he has not mirrowing… How does his net outperform mine this heavily?<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/8/5/85b7140cf030f4ea4fca12dd1ffd4da62f9e7c3c.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/85b7140cf030f4ea4fca12dd1ffd4da62f9e7c3c" title="vienna5"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/8/5/85b7140cf030f4ea4fca12dd1ffd4da62f9e7c3c_2_345x172.jpeg" alt="vienna5" data-base62-sha1="j4TKjIRHpewClDKEkxnbOTvVH9q" width="345" height="172" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/8/5/85b7140cf030f4ea4fca12dd1ffd4da62f9e7c3c_2_345x172.jpeg, https://discuss.pytorch.org/uploads/default/optimized/3X/8/5/85b7140cf030f4ea4fca12dd1ffd4da62f9e7c3c_2_517x258.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/8/5/85b7140cf030f4ea4fca12dd1ffd4da62f9e7c3c_2_690x344.jpeg 2x" data-dominant-color="54534D"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">vienna5</span><span class="informations">1000×500 214 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

username: user_123454321
cooked: <p>Dice loss should not go below 0 usually. Need to verify the loss I think.</p>

username: Caipi
cooked: <p>Loss is fixed now, I guess your not supposed to mix torchvision transforms with albumentation transforms, anyway, loss is not slightly positive but never decreases and I think following is the problem now:<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/6/e/6e2b78895308e8e9c79777b90a693ff69e09b061.png" data-download-href="https://discuss.pytorch.org/uploads/default/6e2b78895308e8e9c79777b90a693ff69e09b061" title="Capture"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/6/e/6e2b78895308e8e9c79777b90a693ff69e09b061_2_690x365.png" alt="Capture" data-base62-sha1="fIBELPXcodt8VP8NCoH07GwSNZ7" width="690" height="365" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/6/e/6e2b78895308e8e9c79777b90a693ff69e09b061_2_690x365.png, https://discuss.pytorch.org/uploads/default/original/3X/6/e/6e2b78895308e8e9c79777b90a693ff69e09b061.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/6/e/6e2b78895308e8e9c79777b90a693ff69e09b061.png 2x" data-dominant-color="A0A19F"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Capture</span><span class="informations">1000×530 793 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>Left side the input data 756x756, right side the prediction in first iteration 564x564, first image. The prediction is still mirrowed at all sides (top is the most visible)… But how is this possible thus my model is never resizing but always cropping and uses no padding? Of course the ground truth is not mirrowed but has the same size of the prediction…</p>
<p><strong>EDIT</strong> Wait I am talking crap sorry, I am resizing in the model, that must be the solution for that… I will crop now and update you guys.</p>
<p>Fixed it, my mistake. Learn: Do not mix albumentations with torchvision transforms for whatever reason…</p>