# Developing the Smart Sorting Screw System Based on Deep Learning Approaches
## 3 Experiment and Discussion
### Performance of the CAE and AAE Models
#### 3.3.4 Evaluation Criteria of CAE and AAE Networks  
It is necessary to evaluate the predictive performance after the model training process. In this study, three evaluation criteria, the intersection over union (IoU), dice coefficient (DC), and frames per second (FPS) [24] were used to quantify the experiment results of the two anomaly networks. IoU is known to be a popular metric to calculate the overlap percentage of the common pixels between the predicted region and ground truth. The range of the IoU is from 0 to 1, where an IoU of 0 represents no overlap between the predicted region and ground truth. An IoU of 1 indicates that the predicted region and ground truth perfectly overlap. Moreover, the dice coefficient is another widely used indicator, which is similar to the IoU. The dice coefficient is used to evaluate the similarity of the predicted region and ground truth. Both of these indicators were compared with the ground truth (GT), which were provided in Equations (10) and (11).  
\[\text{IoU}=\frac{\textit{Predict}\cap GT}{\textit{Predict}\cup GT} \tag{10}\]  
\[\text{Dice Coefficient}=2*\frac{|\textit{Predict}\cap GT|}{|\textit{Predict}| +|GT|} \tag{11}\]  
Seven screw datasets were employed to test the two networks under the same circumstances of the model parameters, such as the epoch and learning rate. The experiment results of AAE and CAE are shown in Table 3. The quantitative detection of the two networks is given in Table 4. According to the results, the average IoU of CAE and AAE are 0.31 and 0.34, respectively and the average DC of CAE and AAE are 0.45 and 0.51, respectively. It can be found that a predictive result of the AAE is similar to the ground truth. The AAE network exhibits a higher performance than the CAE network in detecting  
Figure 11: The meaning of the normal patch for two networks (unit: 0.03 mm/pixel). (**a**) Original patch image; (**b**) reconstruct patch image; (**c**) residual patch image; (**d**) superimposed patch image.  
Figure 12: The meaning of the abnormal patch for two networks (unit: 0.03 mm/pixel). (**a**) Original patch image; (**b**) reconstruct patch image; (**c**) residual patch image; (**d**) superimposed patch image.  
the defective region of the screw image. Moreover, the frame per second (FPS) number is another indicator to figure out the real-time detection on the AAE and CAE networks. The different structures of the networks have different depths, which affect the recognition rate of the network. The results show that with more complexity layers of the CAE network, the detection speed is slower.  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline \multirow{2}{*}{**Index**} & \multirow{2}{*}{**Original**} & \multicolumn{2}{c}{**CAE**} & \multicolumn{2}{c}{**AAE**} \\ \cline{3-6}  & & **Reconstruct** & **Residual** & **Superimposed** & **Reconstruct** & **Residual** & **Superimposed** \\ \hline \multirow{2}{*}{\(1\)} & \multirow{2}{*}{\(2\)} & \multirow{2}{*}{\(1\)} & \multirow{2}{*}{\(2\)} & \multirow{2}{*}{\(2\)} & \multirow{2}{*}{\(2\)} & \multirow{2}{*}{\(2\)} \\  & & & & & \\ \end{tabular}
\end{table}
Table 2: The screw patches of dataset 1 tested for CAE and AAE networks (unit: 0.03 mm/pixel).