Fully understand FCN

username: MC95
cooked: <p>Hello everyone,<br>
I’m trying to figure out how an fully convolutional image for segmentation works.<br>
I found the following image</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/3/2/32008b38be5d436b1c0193c8aaa655d13d5ecda7.png" data-download-href="https://discuss.pytorch.org/uploads/default/32008b38be5d436b1c0193c8aaa655d13d5ecda7" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/3/2/32008b38be5d436b1c0193c8aaa655d13d5ecda7_2_690x285.png" alt="image" data-base62-sha1="78l2oHamaJkaJui9O9UZilZWNz9" width="690" height="285" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/3/2/32008b38be5d436b1c0193c8aaa655d13d5ecda7_2_690x285.png, https://discuss.pytorch.org/uploads/default/original/3X/3/2/32008b38be5d436b1c0193c8aaa655d13d5ecda7.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/3/2/32008b38be5d436b1c0193c8aaa655d13d5ecda7.png 2x" data-dominant-color="ECEEEC"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">708×293 52.9 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
(<a href="https://www.researchgate.net/figure/Fully-convolutional-neural-network-architecture-FCN-8_fig1_327521314" rel="noopener nofollow ugc">https://www.researchgate.net/figure/Fully-convolutional-neural-network-architecture-FCN-8_fig1_327521314</a>)</p>
<p>I understand the structure till the last blue rectangle (7x7x4096), because this is just a normal cnn structure.<br>
After that we are doing a 1x1 convolution to reduce the number of feature maps right? To be precise, according to the image the number of filters are reduced to the number of classes?<br>
After that the process of umsampling reaches the same resolution as the input image.</p>
<p>Two questions now:</p>
<ol>
<li>
<p>I don’t understand it why we are using K filters, if K is the number of classes. How is the output of 224x244xK interpreted? How do we get the colored/segmented output image like in this picture?<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/b/b/bb7c6ac63f61dd82f65e3dea2e13d90348dca810.png" alt="image" data-base62-sha1="qKzQEUZ4l70wkHOPzcjRfBvAn9m" width="482" height="245"></p>
</li>
<li>
<p>What does the train data look like? What is the ground truth and what is the loss function? How do we calculate the loss between the segmented image and ground truth?</p>
</li>
</ol>
<p>Thanks for helping.</p>

username: bluesky314
cooked: <p>This video explains segmentation in NNs intutively: <a href="https://www.youtube.com/watch?v=NzY5IJodjek&amp;t=1357s" rel="nofollow noopener">https://www.youtube.com/watch?v=NzY5IJodjek&amp;t=1357s</a> . Your first question is answered at the end. Loss is found via flattening out the output array and doing a sum of classification loss for each point in the array or doing dice loss which is a ratio of areas.</p>

username: MC95
cooked: <p>thanks for your answer.<br>
I watched the video, just that I get it right: According to my image above at the end I have a 240x240xK Image, were for every pixel there are K probabilities. The highest probability shows the class of the pixel right?</p>
<p>With regard to your second answer, that would mean that e.g. just use some kind of cross-entropy like: the truth should be [ 0 0 1 0 ] (class 3) but the fcn predicts [ 0.2, 0.8, 0.4, 0.1] right?</p>