# Fruit Classification Model Based on Residual Filtering Network for Smart Community Robot

Authors: Yulin Chen; Hailing Sun; Guofu Zhou; Bao Peng
Journal: Wireless Communications and Mobile Computing
 (2021)

---

## Abstract

With the rapid development of computer vision and robot technology, smart community robots based on artificial intelligence technology have been widely used in smart cities. Considering the process of feature extraction in fruit classification is very complicated. And manual feature extraction has low reliability and high randomness. Therefore, a method of residual filtering network (RFN) and support vector machine (SVM) for fruit classification is proposed in this paper. The classification of fruits includes two stages. In the first stage, RFN is used to extract features. The network consists of Gabor filter and residual block. In the second stage, SVM is used to classify fruit features extracted by RFN. In addition, a performance estimate for the training process carried out by theK-fold cross-validation method. The performance of this method is assessed with the accuracy, recall, F1 score, and precision. The accuracy of this method on the Fruits-360 dataset is 99.955%. The experimental results and comparative analyses with similar methods testify the efficacy of the proposed method over existing systems on fruit classification.


---

## Body

## 1. Introduction

China has achieved rich results in the field of smart cities through years of construction. These achievements have played an important role in alleviating the “big city disease,” improving the quality of urbanization, realizing refined and dynamic management, enhancing the effectiveness of urban management, and improving the residents’ quality of life. Community is the “smallest cell” of social management. It is an important aspect of the modernization of urban management system and management capacity to enhance the community management capabilities [1, 2]. With the increasing scale of community, there are many large communities in China. These communities need a lot of fruits and other living materials every day. A lot of labors are required to tasks such as fruit sorting. Labor force has dropped sharply during the epidemic. Therefore, demand for community robot with automatic sorting and store management is particularly urgent. It can not only make up for the shortage of labor but also save costs and improve efficiency [3].Fruit sorting is the core function of intelligent community robot. It has been highly concerned by researchers and achieved a lot of research results in recent years. A fruit classification method was proposed by Arivazhagan et al. [4]. It extracted different features of fruits and used BP classification to classify. The accuracy of this method was reported to be 86%. A fuzzy logic and k-means clustering method for sorting fruit and vegetable was proposed by George [5]. The method achieved an accuracy of 86%. Kuang et al. [6] proposed a fruit detection method based on multiple-color channels. The method fused the histogram of oriented gradient, local binary pattern (LBP), and Gabor wavelet-based LBP features. The features were fused and used to train SVM on previously divided blocks of image to produce optimal results.In addition, deep learning models based on fruit classification had been proposed by some researchers. Yu et al. [7] proposed a 13-layer convolutional neural network (CNN) for fruit recognition classification. The method used three types of data enhancement methods: image rotation, gamma correction, and noise injection. It also compared maximum pooling and average pooling. The gradient descent algorithm was used to optimize the CNN network. And the method achieved an accuracy of 94.94%. The method was 5% higher than the accuracy of the common CNN recognition method. An improved CNN to detect the appearance of cherries was proposed by Momeny et al. [8]. The method combining maximum pooling and average pooling was used to classify cherries after getting preprocessed images. Gao et al. [9] proposed a multicategory apple detection method based on Fast Region-based Convolutional Network (Fast R-CNN) method. The method achieved an average accuracy of 0.909, 0.899, 0.858, and 0.848 under five different shading conditions, respectively.Traditional methods of fruit classification have achieved good classification results. But there had been many problems. Machine learning of fruit classification is a complex process. It needs to extract features of fruit [10]. However, the method of deep learning for fruit classification has problems of deep network layer and slow speed. Therefore, a fruit classification method based on the above problems is proposed in this paper. It is based on RFN and SVM for classification. RFN and SVM are used as the methods to extract features and classification, respectively.The organization of the paper is as below. The principle of the method introduced in this paper is discussed in Section2. The experimental process and result analysis are introduced in Section 3. The paper is concluded in Section 4.


## 2. Materials and Methods

Traditional methods of fruit classification need to extract features of fruits by manual work. Therefore, a method for fruit classification based on RFN and SVM is proposed in this paper. This section introduces the process and principle of fruit classification.


### 2.1. Feature Extraction

Feature extraction is an important process in fruit classification. RFN is proposed in this paper for feature extraction. It consists of residual filtering block. Residual filtering block is composed of Gabor filter and residual block. Gabor filter is used in residual filtering block to replace the convolution kernel. It can extract features of fruits in different directions and scales [11]. And residual block can solve problems of feature loss and gradient disappearance. The structure of the residual filtering block is shown in Figure 1. X and Y are input and output of the residual filtering block, respectively. The G1 layer is a convolutional layer using a 3×3 Gabor filter. The residual block is constituted by C1, C2, and C3.Figure 1
Residual filtering block structure. The G1 layer is a convolutional layer using a3×3 Gabor filter. Residual block is constituted by C1, C2, and C3.The Gabor filter can be convolved with image to obtain the local information in the spatial domain [12, 13]. The Gabor filter can be generated according to
(1)Gθx′,y′,σ,γ,λ,ψ=e−a2x′2+γ′2+y′2cosλx′+ψ,where x and y mean the location of the pixels in the spatial domain [14]. The positions x′ and y′ after pixel rotation are calculated by Equations (2) and (3). Gθx′,y′,σ,γ,λ,φ denotes the response of a Gabor filter with standard deviation σ, aspect ratio γ, wavelength λ, and phase offset ψ to an image at point x′,y′ on the image plane [15].
(2)x′=xcosθ−ysinθ,(3)y′=xsinθ+ycosθ.The convolution process of the Gabor filter is shown in Figure2. The parameters of the Gabor filter can be adjusted during BP. It is similar to the common convolution kernel. The Gabor filter can only convolve a single channel of image. It is different from common convolution kernel. The feature extracted by the Gabor filter is more robust than that extracted by common convolution kernel.Figure 2
Gabor filter operations. (a) Filters are generated by rotating a sampled grid over four orientationsθθ1,θ2,θ3,θ4. (b) Then, Gabor filters K1 and K2 are constructed. Finally, the Gabor filter operations (c) and (d) separately convolve every filter in Gabor kernel with every channel from the input.Feature loss and gradient disappearance occur in network transmission process [16]. These problems are solved by adding residual network in this paper. The output of the residual network is calculated by Equations (4) and (5). The equation of hxl+Fxl+Wl can be realized through feedforward neural networks with “shortcut connections.” xl and Xl+1 are input and output of the convolutional layers l, respectively. hxl is used when matching dimensions [17]. The dimension of input features can be increased or decreased by a convolutional layer when the number of input and output features of the residual network is different. The equation Fx,lWl represents the residual mapping to be learned [18]. Wl and Wl′ are different square matrices, respectively. The forward neural network uses two convolutional layers. And ReLU activation function is used to increase nonlinearity of the network.
(4)Xl+1=hxl+Fxl,Wl,(5)hxl=Wl′x.Gradient of the residual block corresponding to chain rule is calculated by Equation (6), where x1 denotes input of the entire neural network. xn denotes the output parameter closest to the loss function. The chain rule is used to convert the derivation of x into the derivation and multiplication in Equation (6). As a result, the gradient is difficult to disappear [19].
(6)∂lossx1=∂lossxn⋅∂xnx1=∂lossxn⋅1+∂∂xn∑i=1n−1Fxi,wi.The feature extraction network in this paper is shown in Figure3. It consists of residual filtering block, pooling layer, convolutional layer, fully connected (FC) layer, and softmax layer. The feature is flattened before entering the FC layer. RFN includes two processes during training: forward propagation (FP) and BP. The current layer is a convolutional layer during FP. Output Cl of the layer is calculated by Equation (7), where X denotes feature vectors of the input. Wl denotes the weight parameter of the layer l. bl denotes the value of bias.
(7)Cl=ReLUXWl+bl.Figure 3
Residual filtering network structure. The G1 layer is a convolutional layer using a3∗3 Gabor filter. Convolutional layers C1, C2, and C3 constitute a residual block. P1 and C4 layers are the pooling layer and convolutional layer, respectively. FC is a fully connected layer. And softmax is the output layer.Finally, data is inputted into the RFN after it is trained. And feature vectors are outputted at the FC layer of the network. The feature vectors outputted in the FC layer are more robust than the output after flattening. Because the dimension of data after flattening is too high. It is difficult to calculate. Then, many neurons after flattening are inactive due to the ReLU function used in the network. So the data obtained is sparse. Therefore, feature vectors are outputted by the FC layer in this method. The feature vectors of the FC layer are spliced using the feature vectors. And they are calculated by Equation (8), where ⊕ is concatenation and CmL denotes the feature vectors of the mth channel of the L layer.
(8)C=C1L⊕C2L⊕.⋯⊕CmL.The extracted feature maps are shown in Figure4. The features are extracted through the Gabor filter and multiple convolutional layers. Then, they are concatenated through the FC layer. The features extracted by the RFN are more scientific and robust compared with precision of manual feature extraction.Figure 4
Feature map visualization. (a) The feature maps are obtained by convolution of the input image and the Gabor filter. (b) Feature maps output by the convolutional layer. (c) Feature maps output by the FC layer.


### 2.2. Classification

Common CNN uses softmax for classification. The output of softmax is the predicted probability of each category. The sum of the probability category is one [20]. Therefore, this method is susceptible to impact of global data classification for each sample at the same time. SVM is used to replace softmax of the CNN for classification in this paper.
(9)Minimize12w2+c∑εj.The decision boundary of SVM is to solve the maximum margin hyperplane for sample. The parameters can be obtained by Equation (9) in order to obtain the best hyperplane w⋅x+b=0 in the second classification, where c is the penalty coefficient and ε is the error [21].Only the inner product (xaTxb) between samples needs to be obtained in the dual problem of linear SVM. And there is no need to specify a nonlinear transformation [22]. Therefore, the inner product of samples needs to be replaced by kernel function in a nonlinear classification problem. There is a mapping from the input space to the feature space as ϕx [23]. The kernel Equation (10) is satisfied for xa,xb in any input space. Therefore, it is used to replace the mapped sample inner product [24]. SVM classification can be obtained by Equation (11), where N is number of features. Labels of the ith feature vectors ai∗ and b∗ are related parameters of the ith feature vectors. The mapped sample inner product is replaced by the kernel function, which can effectively solve the linear inseparable sample classification problem [25].
(10)Kxa,xb=ϕxaTϕxb.The features extracted from the previous section are used to train the SVM. The 10-fold cross-validation method is added during training. This method divides the data into 10 equally. One of the data is selected as the test set during each training. And the rest are used for the training set. The above operation is repeated ten times. And the selected test set is not repeated each time. TheK-fold cross-validation method is used to reduce the occasionality and improve the generalization of the model.
(11)fx=sign∑i=1Nai∗yiKx,xi+b∗.


## 2.1. Feature Extraction

Feature extraction is an important process in fruit classification. RFN is proposed in this paper for feature extraction. It consists of residual filtering block. Residual filtering block is composed of Gabor filter and residual block. Gabor filter is used in residual filtering block to replace the convolution kernel. It can extract features of fruits in different directions and scales [11]. And residual block can solve problems of feature loss and gradient disappearance. The structure of the residual filtering block is shown in Figure 1. X and Y are input and output of the residual filtering block, respectively. The G1 layer is a convolutional layer using a 3×3 Gabor filter. The residual block is constituted by C1, C2, and C3.Figure 1
Residual filtering block structure. The G1 layer is a convolutional layer using a3×3 Gabor filter. Residual block is constituted by C1, C2, and C3.The Gabor filter can be convolved with image to obtain the local information in the spatial domain [12, 13]. The Gabor filter can be generated according to
(1)Gθx′,y′,σ,γ,λ,ψ=e−a2x′2+γ′2+y′2cosλx′+ψ,where x and y mean the location of the pixels in the spatial domain [14]. The positions x′ and y′ after pixel rotation are calculated by Equations (2) and (3). Gθx′,y′,σ,γ,λ,φ denotes the response of a Gabor filter with standard deviation σ, aspect ratio γ, wavelength λ, and phase offset ψ to an image at point x′,y′ on the image plane [15].
(2)x′=xcosθ−ysinθ,(3)y′=xsinθ+ycosθ.The convolution process of the Gabor filter is shown in Figure2. The parameters of the Gabor filter can be adjusted during BP. It is similar to the common convolution kernel. The Gabor filter can only convolve a single channel of image. It is different from common convolution kernel. The feature extracted by the Gabor filter is more robust than that extracted by common convolution kernel.Figure 2
Gabor filter operations. (a) Filters are generated by rotating a sampled grid over four orientationsθθ1,θ2,θ3,θ4. (b) Then, Gabor filters K1 and K2 are constructed. Finally, the Gabor filter operations (c) and (d) separately convolve every filter in Gabor kernel with every channel from the input.Feature loss and gradient disappearance occur in network transmission process [16]. These problems are solved by adding residual network in this paper. The output of the residual network is calculated by Equations (4) and (5). The equation of hxl+Fxl+Wl can be realized through feedforward neural networks with “shortcut connections.” xl and Xl+1 are input and output of the convolutional layers l, respectively. hxl is used when matching dimensions [17]. The dimension of input features can be increased or decreased by a convolutional layer when the number of input and output features of the residual network is different. The equation Fx,lWl represents the residual mapping to be learned [18]. Wl and Wl′ are different square matrices, respectively. The forward neural network uses two convolutional layers. And ReLU activation function is used to increase nonlinearity of the network.
(4)Xl+1=hxl+Fxl,Wl,(5)hxl=Wl′x.Gradient of the residual block corresponding to chain rule is calculated by Equation (6), where x1 denotes input of the entire neural network. xn denotes the output parameter closest to the loss function. The chain rule is used to convert the derivation of x into the derivation and multiplication in Equation (6). As a result, the gradient is difficult to disappear [19].
(6)∂lossx1=∂lossxn⋅∂xnx1=∂lossxn⋅1+∂∂xn∑i=1n−1Fxi,wi.The feature extraction network in this paper is shown in Figure3. It consists of residual filtering block, pooling layer, convolutional layer, fully connected (FC) layer, and softmax layer. The feature is flattened before entering the FC layer. RFN includes two processes during training: forward propagation (FP) and BP. The current layer is a convolutional layer during FP. Output Cl of the layer is calculated by Equation (7), where X denotes feature vectors of the input. Wl denotes the weight parameter of the layer l. bl denotes the value of bias.
(7)Cl=ReLUXWl+bl.Figure 3
Residual filtering network structure. The G1 layer is a convolutional layer using a3∗3 Gabor filter. Convolutional layers C1, C2, and C3 constitute a residual block. P1 and C4 layers are the pooling layer and convolutional layer, respectively. FC is a fully connected layer. And softmax is the output layer.Finally, data is inputted into the RFN after it is trained. And feature vectors are outputted at the FC layer of the network. The feature vectors outputted in the FC layer are more robust than the output after flattening. Because the dimension of data after flattening is too high. It is difficult to calculate. Then, many neurons after flattening are inactive due to the ReLU function used in the network. So the data obtained is sparse. Therefore, feature vectors are outputted by the FC layer in this method. The feature vectors of the FC layer are spliced using the feature vectors. And they are calculated by Equation (8), where ⊕ is concatenation and CmL denotes the feature vectors of the mth channel of the L layer.
(8)C=C1L⊕C2L⊕.⋯⊕CmL.The extracted feature maps are shown in Figure4. The features are extracted through the Gabor filter and multiple convolutional layers. Then, they are concatenated through the FC layer. The features extracted by the RFN are more scientific and robust compared with precision of manual feature extraction.Figure 4
Feature map visualization. (a) The feature maps are obtained by convolution of the input image and the Gabor filter. (b) Feature maps output by the convolutional layer. (c) Feature maps output by the FC layer.


## 2.2. Classification

Common CNN uses softmax for classification. The output of softmax is the predicted probability of each category. The sum of the probability category is one [20]. Therefore, this method is susceptible to impact of global data classification for each sample at the same time. SVM is used to replace softmax of the CNN for classification in this paper.
(9)Minimize12w2+c∑εj.The decision boundary of SVM is to solve the maximum margin hyperplane for sample. The parameters can be obtained by Equation (9) in order to obtain the best hyperplane w⋅x+b=0 in the second classification, where c is the penalty coefficient and ε is the error [21].Only the inner product (xaTxb) between samples needs to be obtained in the dual problem of linear SVM. And there is no need to specify a nonlinear transformation [22]. Therefore, the inner product of samples needs to be replaced by kernel function in a nonlinear classification problem. There is a mapping from the input space to the feature space as ϕx [23]. The kernel Equation (10) is satisfied for xa,xb in any input space. Therefore, it is used to replace the mapped sample inner product [24]. SVM classification can be obtained by Equation (11), where N is number of features. Labels of the ith feature vectors ai∗ and b∗ are related parameters of the ith feature vectors. The mapped sample inner product is replaced by the kernel function, which can effectively solve the linear inseparable sample classification problem [25].
(10)Kxa,xb=ϕxaTϕxb.The features extracted from the previous section are used to train the SVM. The 10-fold cross-validation method is added during training. This method divides the data into 10 equally. One of the data is selected as the test set during each training. And the rest are used for the training set. The above operation is repeated ten times. And the selected test set is not repeated each time. TheK-fold cross-validation method is used to reduce the occasionality and improve the generalization of the model.
(11)fx=sign∑i=1Nai∗yiKx,xi+b∗.


## 3. Experiment and Analysis

The operating system of this experiment is Windows 10. The GPU device is NVIDIA GEFORCE GTX 960M. The framework is PyTorch 1.6.


### 3.1. Data Processing

The data in this paper comes from the public dataset Fruits-360. It has 22688 pictures of 131 classes. Training set and testing set are divided in a ratio of 7 : 3. They are 15882 and 6806 pictures, respectively. The size of the image is scaled to50×50. And one of the batch data is shown in Figure 5.
(12)Accuracy=TP+TNTP+TN+FP+FN,(13)F1score=2×recall×precisionrecall+precision,(14)Precision=TPTP+FP,(15)Recall=TPTP+FN.Figure 5
Dataset sample.


### 3.2. Feature Extraction

Feature extraction is an important step in the fruit classification process. Features from input Fruits-360 images are extracted by using RFN. Firstly, RFN proposed in this paper is trained. The dimension of the input image is50×50×3. And SGD is used as an optimizer. In addition, learning rate is set to 0.001. The training result of the network is shown in Figure 6. The horizontal axis and the vertical axis are epoch and accuracy, respectively. The accuracy of RFN is 97%. Therefore, the features extracted by the model have certain reliability. Then, the data is inputted to the RFN. And feature vectors are outputted in the FC layer. The feature vectors are used for SVM classification.Figure 6
The relationship between accuracy of the training residual filtering model and epoch.


### 3.3. Results and Discussion

The feature vectors are extracted from the RFN to train the SVM. And the indicators of evaluation are accuracy, precision, recall, and F1 score. They are calculated by Equations (12), (13), (14), and (15), respectively. True positive (TP) denotes the positive sample predicted by the model to be positive. True negative (TN) denotes the negative sample predicted to be negative by the model. False positive (FP) denotes the negative sample predicted to be positive by the model. False negative (FN) denotes the positive sample predicted to be negative by the model.Accuracy reflects the ability of the model to judge the entire sample. Precision reflects proportion of the TP samples in the positive samples judged by the classifier. Recall reflects proportion of the positive samples correctly judged by the classifier in the total positive samples proportion. F1score is the harmonic average of precision and recall.The feature vectors extracted by the RFN are used to train four ML models. Figure7 shows the relationship between the model scores of the ML models under different parameters and value of parameter K. Three different kernel functions rbf, poly, and linear are used by SVM for comparison. SVM with linear kernel function is the highest score of one. Comparing Decision Trees (DT) under different maximum depths (d), DT stops splitting when its depth reaches the specified maximum depth threshold. The highest score for a DT with d=10 is 0.9411. Comparison of Random Forest (RF) under different number of DT (n). The highest score for a RF with n=6 is 0.9941. Compared to KNN under different K values, the value of K means that the number of samples closed to the training sample from the prediction sample is K. The highest score for a KNN with K=1 is one. It can be concluded that the features extracted by the RFN have better robustness.Figure 7
The relationship between the scores of different models and value of parameterK: (a) SVM; (b) DT; (c) RF; (d) KNN.
(a)
(b)
(c)
(d)Then, the classifiers with the highest scores for each model in the above were compared. The results are shown in Table1. The SVM with linear kernel has the best effect. Accuracy is 99.955%. Recall is 99.958%. F1score is 99.962%, and precision is 99.997%.Table 1
Comparison of different classifies.
Accuracy (Avg)
Recall (Avg)
F1 score (Avg)
Precision (Avg)
SVM (linear)
99.955%
99.958%
99.962%
99.967%
DT (d=10)
94.652%
94.633%
94.362%
94.415%
Forest (n=6)
98.898%
98.899%
98.852%
98.848%
KNN (k=1)
99.926%
99.931%
99.927%
99.923%TheK-fold cross-validation method is used for the training of four ML models. Figure 8 shows the comparison between four ML models using 10-fold cross-validation and without K-fold cross-validation. The horizontal axis denotes the different parameters of the classifier. And the vertical axis denotes the score of model performance. The red and black lines are the methods of adding and not adding K-fold cross-validation during training, respectively. It can be concluded that the K-fold cross-validation method is used to reduce the occasionality.Figure 8
Comparison of whether to useK-fold cross-validation method during training: (a) SVM; (b) DT; (c) RF; (d) KNN.
(a)
(b)
(c)
(d)In the experiments, three features are compared. They are the features extracted by the RFN, the features extracted by common CNN, and the RGB features. The above three features are used to train four ML models. The results are shown in Tables2–5.Table 2
Comparison with different features using SVM.
Accuracy (Avg)
Recall (Avg)
F1 score (Avg)
Precision (Avg)
RFN-SVM
99.955%
99.958%
99.962%
99.967%
RGB-SVM
92.415%
92.296%
92.188%
92.772%
CNN-SVM
97.166%
96.825%
96.944%
97.472%Table 3
Comparison with different features using DT.
Accuracy (Avg)
Recall (Avg)
F1 score (Avg)
Precision (Avg)
RFN-DT
94.652%
94.633%
94.362%
94.415%
RGB-DT
87.555%
87.276%
87.136%
87.391%
CNN-DT
93.636%
93.454%
93.375%
93.476%Table 4
Comparison with different features using RF.
Accuracy (Avg)
Recall (Avg)
F1 score (Avg)
Precision (Avg)
RFN-RF
98.898%
98.899%
98.852%
98.848%
RGB-RF
90.015%
89.902%
89.744%
89.893%
CNN-RF
97.948%
97.914%
97.865%
97.945%Table 5
Comparison with different features using KNN.
Accuracy (Avg)
Recall (Avg)
F1 score (Avg)
Precision (Avg)
RFN-KNN
99.926%
99.931%
99.927%
99.923%
RGB-KNN
92.553%
92.464%
92.368%
92.613%
CNN-KNN
98.677%
98.565%
98.531%
98.569%The classification effect of the features extracted by the RFN is better than that by the other two methods. The results show that the classification method using the RFN combined with SVM has the best effect.Accuracy is 99.955%. Recall is 99.958%. F1score is 99.962%. And precision is 99.967%.In addition, the proposed method is also evaluated to classify similar classes of category Apple, Cherry, Grape, Pear, Potato and Tomato as shown in Table6. The results are shown in the confusion matrix in Figure 9. The confusion matrix shows the relationship between the predicted value of the classifier and true value. So the confusion matrix is used to evaluate the performance of the method. The best SVM classification effect is to use the features extracted by RFN. Accuracy is 99.966%. The method of RFN and SVM proposed in this paper for fruit classification has a good classification effect.Table 6
Classes and subclasses in the dataset.
Classes
Subclasses
Apple
Apple Braeburn, Apple Crimson Snow, Apple Golden 1, Apple Golden 2, Apple Golden 3, Apple Granny Smith, Apple Pink Lady, Apple Red 1, Apple Red 2, Apple Red 3, Apple Red Delicious, Apple Red Yellow 1, Apple Red Yellow 2
Cherry
Cherry 1, Cherry 2, Cherry Rainier, Cherry Wax Black, Cherry Wax Red, Cherry Wax Yellow
Grape
Grape Blue, Grape Pink, Grape White, Grape White 2, Grape White 3, Grape White 4, Grapefruit Pink, Grapefruit White
Pear
Pear, Pear 2, Pear Abate, Pear Forelle, Pear Kaiser, Pear Monster, Pear Red, Pear Stone, Pear Williams
Potato
Potato Red, Potato Red Washed, Potato Sweet, Potato White
Tomato
Tomato 1, Tomato 2, Tomato 3, Tomato 4, Tomato Cherry Red, Tomato Heart, Tomato Maroon, Tomato not Ripened, Tomato YellowFigure 9
Confusion matrix is used to evaluate different methods: (a) RFN with SVM; (b) CNN with SVM; (c) RGB with SVM.
(a)
(b)
(c)


## 3.1. Data Processing

The data in this paper comes from the public dataset Fruits-360. It has 22688 pictures of 131 classes. Training set and testing set are divided in a ratio of 7 : 3. They are 15882 and 6806 pictures, respectively. The size of the image is scaled to50×50. And one of the batch data is shown in Figure 5.
(12)Accuracy=TP+TNTP+TN+FP+FN,(13)F1score=2×recall×precisionrecall+precision,(14)Precision=TPTP+FP,(15)Recall=TPTP+FN.Figure 5
Dataset sample.


## 3.2. Feature Extraction

Feature extraction is an important step in the fruit classification process. Features from input Fruits-360 images are extracted by using RFN. Firstly, RFN proposed in this paper is trained. The dimension of the input image is50×50×3. And SGD is used as an optimizer. In addition, learning rate is set to 0.001. The training result of the network is shown in Figure 6. The horizontal axis and the vertical axis are epoch and accuracy, respectively. The accuracy of RFN is 97%. Therefore, the features extracted by the model have certain reliability. Then, the data is inputted to the RFN. And feature vectors are outputted in the FC layer. The feature vectors are used for SVM classification.Figure 6
The relationship between accuracy of the training residual filtering model and epoch.


## 3.3. Results and Discussion

The feature vectors are extracted from the RFN to train the SVM. And the indicators of evaluation are accuracy, precision, recall, and F1 score. They are calculated by Equations (12), (13), (14), and (15), respectively. True positive (TP) denotes the positive sample predicted by the model to be positive. True negative (TN) denotes the negative sample predicted to be negative by the model. False positive (FP) denotes the negative sample predicted to be positive by the model. False negative (FN) denotes the positive sample predicted to be negative by the model.Accuracy reflects the ability of the model to judge the entire sample. Precision reflects proportion of the TP samples in the positive samples judged by the classifier. Recall reflects proportion of the positive samples correctly judged by the classifier in the total positive samples proportion. F1score is the harmonic average of precision and recall.The feature vectors extracted by the RFN are used to train four ML models. Figure7 shows the relationship between the model scores of the ML models under different parameters and value of parameter K. Three different kernel functions rbf, poly, and linear are used by SVM for comparison. SVM with linear kernel function is the highest score of one. Comparing Decision Trees (DT) under different maximum depths (d), DT stops splitting when its depth reaches the specified maximum depth threshold. The highest score for a DT with d=10 is 0.9411. Comparison of Random Forest (RF) under different number of DT (n). The highest score for a RF with n=6 is 0.9941. Compared to KNN under different K values, the value of K means that the number of samples closed to the training sample from the prediction sample is K. The highest score for a KNN with K=1 is one. It can be concluded that the features extracted by the RFN have better robustness.Figure 7
The relationship between the scores of different models and value of parameterK: (a) SVM; (b) DT; (c) RF; (d) KNN.
(a)
(b)
(c)
(d)Then, the classifiers with the highest scores for each model in the above were compared. The results are shown in Table1. The SVM with linear kernel has the best effect. Accuracy is 99.955%. Recall is 99.958%. F1score is 99.962%, and precision is 99.997%.Table 1
Comparison of different classifies.
Accuracy (Avg)
Recall (Avg)
F1 score (Avg)
Precision (Avg)
SVM (linear)
99.955%
99.958%
99.962%
99.967%
DT (d=10)
94.652%
94.633%
94.362%
94.415%
Forest (n=6)
98.898%
98.899%
98.852%
98.848%
KNN (k=1)
99.926%
99.931%
99.927%
99.923%TheK-fold cross-validation method is used for the training of four ML models. Figure 8 shows the comparison between four ML models using 10-fold cross-validation and without K-fold cross-validation. The horizontal axis denotes the different parameters of the classifier. And the vertical axis denotes the score of model performance. The red and black lines are the methods of adding and not adding K-fold cross-validation during training, respectively. It can be concluded that the K-fold cross-validation method is used to reduce the occasionality.Figure 8
Comparison of whether to useK-fold cross-validation method during training: (a) SVM; (b) DT; (c) RF; (d) KNN.
(a)
(b)
(c)
(d)In the experiments, three features are compared. They are the features extracted by the RFN, the features extracted by common CNN, and the RGB features. The above three features are used to train four ML models. The results are shown in Tables2–5.Table 2
Comparison with different features using SVM.
Accuracy (Avg)
Recall (Avg)
F1 score (Avg)
Precision (Avg)
RFN-SVM
99.955%
99.958%
99.962%
99.967%
RGB-SVM
92.415%
92.296%
92.188%
92.772%
CNN-SVM
97.166%
96.825%
96.944%
97.472%Table 3
Comparison with different features using DT.
Accuracy (Avg)
Recall (Avg)
F1 score (Avg)
Precision (Avg)
RFN-DT
94.652%
94.633%
94.362%
94.415%
RGB-DT
87.555%
87.276%
87.136%
87.391%
CNN-DT
93.636%
93.454%
93.375%
93.476%Table 4
Comparison with different features using RF.
Accuracy (Avg)
Recall (Avg)
F1 score (Avg)
Precision (Avg)
RFN-RF
98.898%
98.899%
98.852%
98.848%
RGB-RF
90.015%
89.902%
89.744%
89.893%
CNN-RF
97.948%
97.914%
97.865%
97.945%Table 5
Comparison with different features using KNN.
Accuracy (Avg)
Recall (Avg)
F1 score (Avg)
Precision (Avg)
RFN-KNN
99.926%
99.931%
99.927%
99.923%
RGB-KNN
92.553%
92.464%
92.368%
92.613%
CNN-KNN
98.677%
98.565%
98.531%
98.569%The classification effect of the features extracted by the RFN is better than that by the other two methods. The results show that the classification method using the RFN combined with SVM has the best effect.Accuracy is 99.955%. Recall is 99.958%. F1score is 99.962%. And precision is 99.967%.In addition, the proposed method is also evaluated to classify similar classes of category Apple, Cherry, Grape, Pear, Potato and Tomato as shown in Table6. The results are shown in the confusion matrix in Figure 9. The confusion matrix shows the relationship between the predicted value of the classifier and true value. So the confusion matrix is used to evaluate the performance of the method. The best SVM classification effect is to use the features extracted by RFN. Accuracy is 99.966%. The method of RFN and SVM proposed in this paper for fruit classification has a good classification effect.Table 6
Classes and subclasses in the dataset.
Classes
Subclasses
Apple
Apple Braeburn, Apple Crimson Snow, Apple Golden 1, Apple Golden 2, Apple Golden 3, Apple Granny Smith, Apple Pink Lady, Apple Red 1, Apple Red 2, Apple Red 3, Apple Red Delicious, Apple Red Yellow 1, Apple Red Yellow 2
Cherry
Cherry 1, Cherry 2, Cherry Rainier, Cherry Wax Black, Cherry Wax Red, Cherry Wax Yellow
Grape
Grape Blue, Grape Pink, Grape White, Grape White 2, Grape White 3, Grape White 4, Grapefruit Pink, Grapefruit White
Pear
Pear, Pear 2, Pear Abate, Pear Forelle, Pear Kaiser, Pear Monster, Pear Red, Pear Stone, Pear Williams
Potato
Potato Red, Potato Red Washed, Potato Sweet, Potato White
Tomato
Tomato 1, Tomato 2, Tomato 3, Tomato 4, Tomato Cherry Red, Tomato Heart, Tomato Maroon, Tomato not Ripened, Tomato YellowFigure 9
Confusion matrix is used to evaluate different methods: (a) RFN with SVM; (b) CNN with SVM; (c) RGB with SVM.
(a)
(b)
(c)


## 4. Conclusion

A method of RFN for smart community robot fruit classification is proposed in this paper. The features are more robust that are extracted by the RFN which combines the Gabor filter and the residual block. Four ML models (SVM, RF, DT, KNN) are used to test the features extracted by the RFN. And SVM is used to replace the softmax classification of the common CNN. It improves the classification effect of the model. And theK-fold cross-validation method is used in training. It can improve the accuracy of the model and reduce the occasionality by random classification. The method proposed in this paper is compared with other methods. The results show that the accuracy of the proposed method on the Fruits-360 dataset is 99.955%. It is an improvement compared with the original result of author. In addition, the accuracy among the 6 categories of fruits that are difficult to distinguish is 99.966%. The method for smart community robot fruit classification proposed in this paper can achieve good results. This method can replace the traditional method of artificial feature extraction for classification. And it can also be further extended to other fields.



---

*Source: 5541665-2021-03-28.xml*