# _Applied sciences_
## 4 Experimental Evaluation
### Comparison Experiment
By U-Net segmentation, the number of earth red pixels is 48,277, and the number of ultramarine blue pixels is 51,118. By the proposed UR-Net segmentation, the number of earth red pixels is 48,058, and the number of ultramarine blue pixels is 48,981. The true number of earth red pixels is 46,546, and that of ultramarine blue pixels is 48,433. It is obvious that the segmentation results of the proposed method are more similar to the true results. However, the number of pixels recognized by the network model cannot be fully used as an evaluation criterion because the trained model may overfit during segmentation, which may cause it to overidentify color pixels and background pixels, resulting in inaccurate pixel numbers. In other words, if the model segmentation accuracy is higher, the number of recognized pixels must be more accurate. In contrast, if the model accuracy is low, the number of recognized pixels  
Figure 15: Comparison of different segmentation results with the true results. (**a**) K-means clustering algorithm. (**b**) ResNet. (**c**) U-Net. (**d**) The proposed UR-Net. (**e**) True results. may not differ much from the true value. Therefore, there is a need to use a more objective method to evaluate segmentation accuracy. Finally, the confusion matrix is calculated to quantify the difference between the true results and the segmentation results for a more objective evaluation. As shown in Figure 17, the blank area represents TP, the black area represents TN, the magenta area represents FN, and the green area represents FP. The magenta area indicates that the label pixels are recognized as background pixels, and the green area indicates that the background pixels are recognized as label pixels, both of which show the difference between the true results and the segmentation results. The value of TP + TN denotes the total number of correctly recognized pixels, and the value of FN + FP denotes the total number of unrecognized pixels. If the value of TP + TN is larger and the value of FN + FP is smaller, the magenta and green areas are smaller. Comparing the four segmentation methods in Figure 17, the proposed UR-Net model resulted in the smallest magenta and green areas. The TP + TN and FN + FP values of the earth red and ultramarine blue pixels can be calculated from the TP, TN, FN, and FP values of the confusion matrix after the test sample segmentation is completed, as shown in Table 5. The results showed that the proposed UR-Net model has a TP + TN value of 336,102 when segmenting the earth red pixels, which is larger than those of the other methods, and an FN + FP value of 3198, which is smaller than those of the other methods. The TP + TN value of 335,393, when segmenting the ultramarine blue pixels, is larger than those of the other methods, and the FN + FP value of 3907 is smaller than those of the other methods. It is demonstrated that the proposed UR-Net model outperforms the other methods regarding the number of pixels correctly recognized in segmentation. Figure 16: Numbers of pixels in different segmentation results compared to the true results. After determining the FN, FP, TP, and TN values in the confusion matrix, data such as PA, F1-score, and IoU can be calculated to evaluate the accuracy of segmentation, as shown in Table 4. The results are as follows: (1) The PA is 0.9906, the F1-score is 0.9662, and the IoU is 0.9346 when using the proposed UR-Net model to segment the earth red pixels. (2) The PA is 0.9890, the F1-score is 0.9615, and the IoU is 0.9259 when using the proposed UR-Net model to segment the ultramarine blue pixels. All the values are higher than those of the other segmentation methods, and the IoU values are all above 0.92, which gives a very high segmentation accuracy. Although the color paintings segmented by the U-Net model can also achieve high accuracy, there is salt-and-pepper noise, so this method cannot meet the requirements and cannot be applied to the segmentation of color paintings, as shown in Figure 17c.