# An Improved Image Semantic Segmentation Method Based on Superpixels and Conditional Random Fields
## 4 Experimental Evaluation
### Experimental Setup  
We use the PASCAL VOC 2012 segmentation benchmark [34], as it has become the standard dataset to comprehensively evaluate any new semantic segmentation methods. It involves 20 foreground classes and one background class. For our experiments on VOC 2012, we adopt the extended training set of 10,581 images [35] and a reduced validation set of 346 images [20]. We further evaluate the improved method on the Cityscapes dataset [36], which focuses on semantic understanding of urban street scenes. It consists of around 5000 fine annotated images of street scenes and 20,000 coarse annotated ones, in which all annotations are from 19 semantic classes.  
From Figure 7, it can be seen that as the number of superpixels increases; a single superpixel will get closer to the edge of the object. Most of images in VOC dataset have a resolution of around \(500\times 500\), so we set the number of superpixels to 1000. For Cityscapes dataset, we set the number of superpixels to 6000 due to the high-resolution of images. In our experiments, 10 mean field iterations  
Figure 6: Accurate boundary recovery and partial enlarged details.  
are employed for CRF. Meanwhile, we use default values of \(\omega_{1}=\sigma_{r}=3\) and set \(\omega_{1}=5\), \(\sigma_{\alpha}=49\) and \(\sigma_{\beta}=3\) by the same strategy in [22].  
The standard Jaccard Index (Figure 8), also known as the PASCAL VOC intersection-over-union (IoU) metric [34], is introduced for the performance assessment in this paper:  
According to Jaccard Index, many evaluation criteria have been proposed to evaluate the segmentation accuracy. Among them, PA, IoU, and mIoU are often used, and their definitions can be found in the previous work [37]. We assume a total of \(k+1\) classes (including a background class), and \(p_{ij}\) is the amount of pixels of class \(i\) inferred to class \(j\). \(p_{ii}\) represents the number of true positives, while \(p_{ij}\) and \(p_{ji}\) are usually interpreted as false positives and false negatives, respectively. The formulas of PA, IoU, and mIoU are shown below:  
\(\bullet\) Pixel Accuracy (PA):  
\[PA=\frac{\sum_{i=0}^{k}p_{ii}}{\sum_{i=0}^{k}\sum_{j=0}^{k}p_{ij}}, \tag{4}\]  
which computes a ratio between the number of properly classified pixels and the total number of them.  
\(\bullet\) Intersection Over Union (IoU):  
\[IoU=\frac{p_{ii}}{\sum_{j=0}^{k}p_{ij}+\sum_{j=0}^{k}p_{ji}-p_{ii}}, \tag{5}\]  
which is used to measure whether the target in the image is detected.  
Figure 8: The standard Jaccard Index, in which TP, FP, and FN are the numbers of true positive, false positive, and false negative pixels.  
Figure 7: Results with different superpixel number. The first row: SLIC segmentation results with 100, 500, and 1000 superpixels on an example image in VOC dataset. The second row: SLIC segmentation results with 1000, 3000, and 6000 superpixels on an example image in Cityscapes dataset.  
Mean Intersection Over Union (mIoU):  
\[mIoU=\frac{1}{k+1}\sum_{i=0}^{k}IoU_{i}, \tag{6}\]  
which is the standard metric for segmentation purposes and computed by averaging IoU. mIoU computes a ratio between the ground truth and our predicted segmentation.