# 2D Semantic Segmentation: Recent Developments and Future Directions
## 5 Common Data Sets and Evaluation Indicators
\begin{table}
\begin{tabular}{c c c c c} \hline \hline \multirow{2}{*}{**Method**} & \multirow{2}{*}{**Time**} & \multirow{2}{*}{**Backbone**} & **VOC 2012** & **Cityscapes** & **CamVid** \\  & & & **(MIoU/\%)** & **(MIoU/\%)** & **(MIoU/\%)** \\ \hline FCN & 2015 & VGG16 & 62.2 & 65.3 & — \\ U-Net & 2015 & VGG16 & — & — & — \\ DeepLab v1 & 2016 & ResNet & 71.6 & — & — \\ DeepLab v2 & 2017 & ResNet & 79.7 & 70.4 & — \\ DeepLab v3 & 2017 & ResNet & 86.9 & 81.3 & — \\ DeepLab v3+ & 2018 & ResNet & 89 & 82.1 & \\ SegNet & 2017 & VGG16 & — & — & 60.1 \\ PSPNet & 2017 & ResNet & 85.4 & 80.2 & — \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Performances of classical semantic segmentation methods on different data sets. \begin{table}
\begin{tabular}{c c c c c} \hline \hline \multirow{2}{*}{**Method**} & \multirow{2}{*}{**Time**} & \multirow{2}{*}{**Backbone**} & **Cityscapes** & **CamVid** \\  & & & **(MIoU/\%)** & **(MIoU/\%)** \\ \hline HFEN [110] & 2021 & — & 69.5 & 66.1 \\ FaPN [111] & 2021 & ResNet-18 & 75.0 & — \\ DDRNet [112] & 2021 & DDRNet-39 & 80.4 & — \\ MIFNet [113] & 2021 & MobileNetV2 & 72.2 & — \\ ViT-Adaptor-L [114] & 2022 & UPerNet & 85.2 & — \\ DRANet [115] & 2020 & ResNet-101 & 82.9 & — \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Performance of recent semantic segmentation methods on different data sets. where \(p_{ii}\) represents the number of correctly classified pixels, \(p_{ij}\) is the number of pixels that should belong to the \(i\)-th category but are classified into the \(j\)-th category, and n is the number of categories. PA describes the segmentation accuracy of all categories in an image and is commonly used to estimate the overall segmentation performance. The closer PA is to 1, the better the model performs. However, PA contains limited information and may mask poor segmentation results in certain categories, thereby failing to reflect the segmentation accuracy of individual categories. To address this issue, class pixel accuracy (CPA) can be used to evaluate the segmentation accuracy of each category separately. For the segmentation result of the \(i\)-th category, the CPA calculation formula is as follows (2):  
\[CPA=\frac{p_{ii}}{\sum_{j=0}^{k}p_{ij}} \tag{2}\]  
MPA represents the average pixel accuracy of all object categories in the image. The main advantages of MPA are its relatively simple computation and its ability to provide an overall assessment of the model's performance across all categories. However, it also has some limitations. In semantic segmentation tasks, the number of pixels per category in the image is often imbalanced, with some categories having considerably more pixels than others. MPA assigns equal weights to all categories during calculation. As a result, categories containing more pixels than others exert a disproportionately large effect on MPA, and the performance of categories with few pixels is disregarded. MPA's calculation considers only the model's correct classification of pixels for each category and does not consider cases in which the model incorrectly classifies pixels as other categories. This situation may lead to an overestimation of the model's performance, as shown in Formula (3):  
\[MPA=\frac{1}{n+1}\sum_{i=0}^{n}\frac{p_{ii}}{\sum\limits_{j=0}^{n}p_{ij}} \tag{3}\]  
MIoU is the ratio of the intersection of the true value of the segmentation result and its union, which is calculated by class then averaged. MIoU considers the model's correct classification of pixels for each category (true positive; TP), cases in which the model misclassifies pixels as other categories (false positive; FP), and cases in which the model incorrectly classifies category i's pixels as other categories (false negative (FN)). Thus, MIoU provides a comprehensive performance evaluation. Compared with MPA, MIoU is more tolerant of class imbalances in data sets because MIoU accounts for the correct, incorrect, and unclassified pixels of each category during computation, thus better reflecting the model's performance across all categories.