## 3 Experiments and Analysis
### Comparison with Different Models
Hence, the neural network pays more attention to the differences in texture and ignores some difficult-to-identify shape defects. Even so, we can later extract the gear foreground by Blob analysis and screen out the breakages by the size of the foreground area. Other network models perform less well than the proposed model with GAN in terms of identifying other defect categories. For example, in Figure 9b, the proposed model without GAN predicted 13% of normal images as circles; in Figure 9c Vgg11 was prone to confusion on the two defect categories of stains and scratches, with 4% of stains recognized as scratches and 3% of scratches recognized as stains; and in Figure 9d, Vgg16 identified 13% of scratches as stains. Figure 8: Comparison of results for each network model validation sets: (**a**) Course of training accuracy; (**b**) course of training loss; (**c**) course of validation accuracy; (**d**) course of validation loss. From the above analysis, it can be concluded that through the optimization of the CNN network structure, the proposed model is better than the original Vgg network model, which proves that the improvement proposed in this paper is effective, because the deep structure, the number of small-sized strides and filters enable the network capture combined features, add invariance and class discrimination, and improve the network defect classification performance [22; 23], while the use of the dropout layers effectively alleviates the overfitting of the network by randomly disabling neurons, providing higher efficiency and better generalization ability for the network [24]. For binary classification, we used the \(F1\) score as the performance metric which depends upon precision and recall. Precision (\(P\)) represents the ratio of the number of true positives to the number of positive predicted samples (the sum of true positives and false positives). Precision indicates the correctness of the test results, as shown in Equation (3). Recall (\(R\)) represents the ratio of true positives to the number of true positive samples on the ground (sum of true positives and false negatives). The recall indicates the comprehensiveness of the test results, as shown in Equation (4). The \(F1\) score is the  
Figure 9: Normalized confusion matrix of the gear end classification for different networks: (**a**) Proposed model with GAN; (**b**) proposed model without GAN; (**c**) Vgg11; (**d**) Vgg16. harmonic mean of precision and recall, which takes into account both correctness and missed detection, as shown in Equation (5). \[P=\frac{TP}{TP+FP} \tag{3}\]  
\[R=\frac{TP}{TP+FN} \tag{4}\]  
\[F1=\frac{2\times P\times R}{P+R}=\frac{2 TP}{2 TP+FN+FP} \tag{5}\]  
The \(F1\) score also has its variant in multi-classification problems; academics often use micro-\(F1\) and macro-\(F1\) to evaluate the performance of multi-classification models, the two are very different in the data imbalance of multi-classification problems [25], but for the balanced dataset used in our experiments, the difference is not much. Here, the macro-\(F1\) score was used as the model evaluation metric. As shown in Equations (6)-(8), macro-precision (\(P_{macro}\)), macro-recall (\(R_{macro}\)), and macro-F1 score (\(F1_{macro}\)) are the arithmetic averages of the precision, recall, and \(F1\) score for each classification, respectively. A higher value for macro-\(F1\) score means that the results of multi-classification are more accurate and comprehensive. \[P_{macro}=\frac{1}{|G|}\sum_{i=1}^{|G|}\frac{TP_{i}}{TP_{i}+FP_{i}}=\frac{\sum _{i=1}^{|G|}P_{i}}{|G|} \tag{6}\]  
\[R_{macro}=\frac{1}{|G|}\sum_{i=1}^{|G|}\frac{TP_{i}}{TP_{i}+FN_{i}}=\frac{\sum _{i=1}^{|G|}R_{i}}{|G|} \tag{7}\]  
\[F1_{macro}=\frac{1}{|G|}\sum_{i=1}^{|G|}\frac{2\times P_{i}\times R_{i}}{P_{i} +R_{i}}=\frac{\sum_{i=1}^{|G|}F1_{i}}{|G|} \tag{8}\]  
where \(i\) is the category, \(G\) is the total number of categories, and \(P_{i}\), \(R_{i}\), and \(F1_{i}\) are the precision, recall, and \(F1\) score for each category, respectively.