# Semantic Segmentation of Packaged and Unpackaged Fresh-Cut Apples Using Deep Learning
## 4 Conclusions  
Computer vision systems are increasingly being used in industrial quality control to offer fast, objective, non-destructive, and contactless evaluation of fruit. This paper addresses the problem of semantically segmenting images of packaged and unpackaged fresh-cut apples as a preliminary step to develop a computer vision system that will analyze the quality and internal properties of this product. The identification and selection of the pulp is necessary to restrict the analysis to this part as it is more significantly affected by progressive browning caused by senescence. Since apples are widely consumed fruits and an essential component of a healthy diet and are increasingly being offered as fresh-cut pieces in proper packaging, images of packaged and unpackaged products belonging to four different varieties have been acquired and used in the experiments: Granny smith (greenish peel), Golden (yellowish peel), Fuji, and Pink Lady (reddish peel). The visual similarity between the pulp and peel has made unsatisfactory in several cases the application of segmentation approaches relying on traditional colour analysis. A convolutional neural network model (CNN), based on the DeepLabV3+ architecture, has been designed, implemented, tested, and validated for achieving a robust semantic segmentation in spite of colour similarity between the peel and pulp. The achieved accuracy was greater than 99% on all four considered varieties despite the orientation of the apples, even in the presence of packaging. The testing was done by running the network to segment the test images semantically, which are not used during training. A qualitative visual verification of the performance of trained networks was conducted by overlaying the segmentation results on the colour images. A quantitative evaluation of accuracy was performed by semantically segmenting the entire testing data set (composed of images \begin{table}
\begin{tabular}{c c c c c} \hline  & **Pulp** & **Peel** & **Background** & **Glitter** \\ \hline
**Pulp** & 160,014 & 989 & 440 & 1664 \\
**Peel** & 4661 & 108,229 & 137 & 742 \\
**Background** & 33 & 157 & 2,869,687 & 76 \\
**Glitter** & 1104 & 699 & 2546 & 22,425 \\ \hline \end{tabular}
\end{table} 
Table 10: Confusion matrix maxima over the test dataset. The first column is especially relevant because it shows the number of pixels misclassified as pulp while belonging to peel, background, or glitter. \begin{table}
\begin{tabular}{c c c c c} \hline  & **Pulp** & **Peel** & **Background** & **Glitter** \\ \hline
**Pulp** & 27,413 & 0 & 0 & 0 \\
**Peel** & 0 & 0 & 0 & 0 \\
**Background** & 0 & 0 & 2,646,664 & 0 \\
**Glitter** & 0 & 0 & 0 & 0 \\ \hline \end{tabular}
\end{table} 
Table 9: Confusion matrix minima over the test dataset. manually labelled but not used during training) and by evaluating metrics such as global accuracy, mean accuracy, mean IoU, weighted IoU, and mean BF score. On randomly disposed unpackaged apples, the network achieved mean accuracy, global accuracy, and mean IoU greater than 99%. In the case of randomly disposed packaged apples, the network attained mean accuracy and global accuracy higher than 99%, and mean IoU greater than 98%. Moreover, the confusion matrix was calculated for each image in the testing data set. Four different global confusion matrices (confusion matrix minima, confusion matrix maxima, confusion matrix mean, and confusion matrix standard deviation) were evaluated. They possessed the minimum, maximum, mean, and standard deviation of each position over the complete testing data set. The analysis of the confusion matrices, especially of the confusion matrix maxima that catch the performance on the worst cases, was conducted to quantify the type of error that is more relevant for our application. In fact, the number of misclassified pixels erroneously classified as pulp can reduce the performance of the following colour analysis by introducing information which are related to parts of the fruit that are not significant for the tasks at hand.