## 5 Experiments
### Performance Evaluation  
At present, the most commonly-used model evaluation methods in the field of deep learning are the error rate and accuracy rate evaluation, F1 evaluation, mAP, etc. Each evaluation method has its own advantages and disadvantages. (1) Error rate and accuracy rate  
Error rate and accuracy rate are the most commonly used evaluation methods in the classification field. The applicability is very strong. The error rate is the ratio of the number of samples with incorrect classification to the total number of samples. The accuracy rate is the ratio of the number of samples with the correct classification to the total number of samples. Regarding the classification, it is assumed that for the data set \(\{(x_{1},y_{1}),(x_{2},y_{2}),\ldots,(x_{n},y_{n})\}\) containing \(n\) samples, there is a corresponding prediction result \(\{Y_{1},Y_{2},\ldots Y_{n}\}\), where \(x\) is the defect picture sample, \(y\) is the defect real label, and \(Y\) is the prediction result by the object detection model, and the error and accuracy rate formula is as shown in Equations(9) and (10):  
\[error\_rate=\frac{\sum\limits_{i=1}^{n}\{1\big{|}y_{i}\
eq Y_{i}\}}{n} \tag{9}\]  
\[accurate\_rate=\frac{\sum\limits_{i=1}^{n}\{1\big{|}y_{i}=Y_{i}\}}{n} \tag{10}\]  
The accuracy rate assessment in the project can meet the project requirements. Figure 15: Faster R-CNN+ ResNet101 three-object recognition effect. (2) Recall rate, precision rate and F1 assessment  
The recall ratio is the proportion of the number of predicted samples in all samples. The precision ratio is the ratio of the number of examples of accurate predictions to the total number of prediction samples in all prediction examples in which the prediction result is a certain category. For the classification problem, the combination of the real category and the prediction category can be divided into real cases, false positive cases, true counterexamples, and false counterexamples, and the numbers are represented by \(TP\), \(FP\), \(TN\), and \(FN\), respectively. Then, the calculation formula of the recall ratio is as shown in Equation (11):  
\[P=\frac{TP}{TP+FP} \tag{11}\]  
The formula of the precision ratio is as shown in Equation (12):  
\[R=\frac{TP}{TP+FN} \tag{12}\]  
The recall rate and the precision rate are a pair of contradictory measures. The higher the recall rate, the lower the precision; the lower the recall rate, the higher the precision. In the general classification problem, to balance the recall rate and the precision rate, it is necessary to find a balance point between the recall rate and the precision rate. The \(F1\) assessment is based on the harmonic mean of the precision and recall. By calculating \(F1\), the equilibrium can be established. The formula for \(F1\) is shown in Equation (13):  
\[F1=\frac{2\times P\times R}{P+R} \tag{13}\]  
(3) mAP assessment [44]  
Mean average accuracy (mAP) is the most commonly used evaluation criterion in object detection. In common classification problems, recall and precision are the most commonly-used statistics. However, in object detection, we also need to confirm the position of the object in the image, and so this is different from the normal classification in calculating the recall ratio and precision ratio. The mAP calculation is based on the IoU between the predicted bounding box and the real bounding box. We calculate the IoU of each detection bounding box and compare it with the calculated IoU value and the threshold (usually set to 0.5), then we obtain the number of correct detections in each image. The object position in this project is not required to be accurate. Because it is a small object in a big image, the approximate position has been determined by the screenshot during the screening stage (not covered in this paper). Therefore, mAP is not used as an evaluation criterion.