IOU Metric for semantic Segmentation

Haider_Ali_Shuvo: <p>I have a batch of output mask of semantic segmentation &lt;N,H,W&gt; and my predicted mask of &lt;N,H,W&gt; , There are 22 categories. . How can i calculate mean IOU metric / Dice Coeff effeciently ?</p>

ptrblck: <p>For IOU take a look at <a href="https://discuss.pytorch.org/t/iou-pytorch-implementation/21473/3">this post</a> for dice at <a href="https://github.com/pytorch/pytorch/issues/1249#issuecomment-339904369">this one</a>.</p>

Haider_Ali_Shuvo: <p><a class="mention" href="/u/ptrblck">@ptrblck</a> Thanks for the links,That seems to work for me.However i have one more question.<br>
I tried to load from torchvision the Fully Convolutional network (FCN ResNet50).However when i am viewing the model i am not seeing any transpose convolution or upsampling layer , How does it keep spatial dimention same yet ? The model architecture is shown below.</p>
<p>FCN(<br>
(backbone): IntermediateLayerGetter(<br>
(conv1): Conv2d(3, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)<br>
(bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu): ReLU(inplace=True)<br>
(maxpool): MaxPool2d(kernel_size=3, stride=2, padding=1, dilation=1, ceil_mode=False)<br>
(layer1): Sequential(<br>
(0): Bottleneck(<br>
(conv1): Conv2d(64, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
(bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv3): Conv2d(64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(bn3): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu): ReLU(inplace=True)<br>
(downsample): Sequential(<br>
(0): Conv2d(64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
)<br>
)<br>
(1): Bottleneck(<br>
(conv1): Conv2d(256, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
(bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv3): Conv2d(64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(bn3): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu): ReLU(inplace=True)<br>
)<br>
(2): Bottleneck(<br>
(conv1): Conv2d(256, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
(bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv3): Conv2d(64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(bn3): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu): ReLU(inplace=True)<br>
)<br>
)<br>
(layer2): Sequential(<br>
(0): Bottleneck(<br>
(conv1): Conv2d(256, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)<br>
(bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv3): Conv2d(128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(bn3): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu): ReLU(inplace=True)<br>
(downsample): Sequential(<br>
(0): Conv2d(256, 512, kernel_size=(1, 1), stride=(2, 2), bias=False)<br>
(1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
)<br>
)<br>
(1): Bottleneck(<br>
(conv1): Conv2d(512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
(bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv3): Conv2d(128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(bn3): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu): ReLU(inplace=True)<br>
)<br>
(2): Bottleneck(<br>
(conv1): Conv2d(512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
(bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv3): Conv2d(128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(bn3): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu): ReLU(inplace=True)<br>
)<br>
(3): Bottleneck(<br>
(conv1): Conv2d(512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
(bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv3): Conv2d(128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(bn3): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu): ReLU(inplace=True)<br>
)<br>
)<br>
(layer3): Sequential(<br>
(0): Bottleneck(<br>
(conv1): Conv2d(512, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
(bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv3): Conv2d(256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(bn3): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu): ReLU(inplace=True)<br>
(downsample): Sequential(<br>
(0): Conv2d(512, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(1): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
)<br>
)<br>
(1): Bottleneck(<br>
(conv1): Conv2d(1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(2, 2), dilation=(2, 2), bias=False)<br>
(bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv3): Conv2d(256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(bn3): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu): ReLU(inplace=True)<br>
)<br>
(2): Bottleneck(<br>
(conv1): Conv2d(1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(2, 2), dilation=(2, 2), bias=False)<br>
(bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv3): Conv2d(256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(bn3): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu): ReLU(inplace=True)<br>
)<br>
(3): Bottleneck(<br>
(conv1): Conv2d(1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(2, 2), dilation=(2, 2), bias=False)<br>
(bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv3): Conv2d(256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(bn3): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu): ReLU(inplace=True)<br>
)<br>
(4): Bottleneck(<br>
(conv1): Conv2d(1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(2, 2), dilation=(2, 2), bias=False)<br>
(bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv3): Conv2d(256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(bn3): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu): ReLU(inplace=True)<br>
)<br>
(5): Bottleneck(<br>
(conv1): Conv2d(1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(2, 2), dilation=(2, 2), bias=False)<br>
(bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv3): Conv2d(256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(bn3): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu): ReLU(inplace=True)<br>
)<br>
)<br>
(layer4): Sequential(<br>
(0): Bottleneck(<br>
(conv1): Conv2d(1024, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(2, 2), dilation=(2, 2), bias=False)<br>
(bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv3): Conv2d(512, 2048, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(bn3): BatchNorm2d(2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu): ReLU(inplace=True)<br>
(downsample): Sequential(<br>
(0): Conv2d(1024, 2048, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(1): BatchNorm2d(2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
)<br>
)<br>
(1): Bottleneck(<br>
(conv1): Conv2d(2048, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(4, 4), dilation=(4, 4), bias=False)<br>
(bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv3): Conv2d(512, 2048, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(bn3): BatchNorm2d(2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu): ReLU(inplace=True)<br>
)<br>
(2): Bottleneck(<br>
(conv1): Conv2d(2048, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(4, 4), dilation=(4, 4), bias=False)<br>
(bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(conv3): Conv2d(512, 2048, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(bn3): BatchNorm2d(2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu): ReLU(inplace=True)<br>
)<br>
)<br>
)<br>
(classifier): FCNHead(<br>
(0): Conv2d(2048, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
(1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(2): ReLU()<br>
(3): Dropout(p=0.1, inplace=False)<br>
(4): Conv2d(512, 21, kernel_size=(1, 1), stride=(1, 1))<br>
)<br>
)</p>

ptrblck: <aside class="quote no-group" data-username="Haider_Ali_Shuvo" data-post="3" data-topic="87050">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/haider_ali_shuvo/48/24642_2.png" class="avatar"> Haider_Ali_Shuvo:</div>
<blockquote>
<p>How does it keep spatial dimention same yet ?</p>
</blockquote>
</aside>
<p>A FCN model doesn’t necessarily keep the spatial dimensions the same, but uses conv layers instead of linear layers to be able to accept variable input shapes.<br>
This can also be done using adaptive pooling layers.</p>
<p>Usually the last linear layers are replaced with a <code>1x1</code> conv layer (and potentially a global pooling layer to get the number of classes as the output).</p>