## 3 Application of the Proposed Method to the Fruit Variety Classification
### Research Method
#### 3.2.3 Proposed Fruit Classification Method Using the Certainty Factor  
This study proposes a fruit classification method for a retail sales system. The method uses machine vision system together with machine learning methods (shown in Figure 1). The first stage of the method involves creating an image with all fruit objects. The image includes one or many fruits (intended for one variety and species) with the background, and it is called original image. In order to be sure of the obtained fruit classification result, the proposed method has two separate pathways for fruit classification. Figure 1: Proposed fruit classification method on an example of apple variety classification. The first pathway was to identify the fruit variety based on the entire original image. For this purpose, the previously described nine-layer CNN was used, which was trained based on the original images. The result of classification \(\overline{A}\) was determined with a certainty factor \(CF\) with the following value:  
\[CF=0.5+d, \tag{2}\]  
where \(d\) is a small positive value (here, \(d=10^{-4}\)). In order not to introduce errors in the interpretation of the results, it is recommended that the \(d\) value be less than 0.5/(\(o+1\)), where \(o\) is the maximum number of objects (apples) on the one image in the dataset. Studies have shown slightly higher accuracy for CNN trained with original images than with ROI objects. Therefore, the small positive value \(d\) for the certainty factor gave slightly more importance to the result obtained in the first pathway of the fruit classification algorithm compared to the second pathway described below. The second pathway of fruit classification consisted of recognizing the fruit variety based on single fruits from the original image. The first step was to use the object detection method to identify single apple objects in the amount of \(N\) (\(N=0,\ 1,\ 2,\ldots\)). The recorded objects of single apples were images with ROIs. The You Only Look Once (YOLO) method was used to extract images of individual objects from the original images. The fruit variety classification was done based on each \(n\)th ROI image (\(n=1,\ldots,N\)). For this purpose, the previously described nine-layer CNN was used. It was trained with the ROI images. Each result of the classification \(\overline{A}_{n}\) (each CNN inference) was provided with the appropriate value of certainty factor \(CF_{n}\):  
\[CF_{n}=\frac{1-0.5-d}{N},\ n=1,\ldots,N, \tag{3}\]  
where \(N\) is the number of objects (apples) detected in the original image and \(d\) is a small positive value (in research \(d=10^{-4}\)). The results obtained from both pathways were grouped, and factor \(CF_{k}\) for each \(k\)th variety was calculated as follows:  
\[CF_{k}=sum\Big{(}\big{\{}CF\big{|}\overline{A}=A_{k}\big{\}},\big{\{}CF_{1}, \ldots CF_{n},\ldots CF_{N}\big{|}\overline{A}_{n}=A_{k}\big{\}}\Big{)},\ k=1, \ldots K, \tag{4}\]  
where \(CF_{k}\) is a certainty factor for the \(k\)th variety of fruit \(A_{k}\) (in the researched variety of apples). The indirect result of the classification can be determined in the form of the following model:  
\[ \begin{array}{l}A_{1}\ \text{with}\ CF_{1}\ \ \ldots\\ A_{k}\ \text{with}\ CF_{k}\ \ldots\\ A_{K}\ \text{with}\ CF_{K},\end{array}  \tag{5}\]  
where \(K\) is the number of fruit varieties that were detected. Based on:  
\[CF_{max}=\max(CF_{1},\ldots,CF_{K}) \tag{6}\]  
the final result of the classification was provided. If the value of \(CF_{max}\) was higher than the limit value of certainty factor (\(CF_{limit}\)) and is not equal \(CF=0.5+d\), then an unambiguous classification was obtained:  
Fruits (for example, apples) are variety of \(A_{k}\) with \(CF_{k}\) where \(CF_{k}=CF_{max}\). If the value of \(CF_{max}\) did not exceed the limit value of the certainty factor (\(CF_{limit}\)) and was not equal to \(CF=0.5+d\), then an ambiguous classification was obtained.