# Automatic Metallic Surface Defect Detection and Recognition with Convolutional Neural Networks
## 5 Experiments
### Performance of Classification Module  
To evaluate the classification performance of the compact CNN quantitatively, we compared it with traditional machine learning methods with three features whose codes are publicly available. (1) GLCM [7]: this feature is the classical texture feature, which includes four typical descriptions: energy, contrast, entropy, and correlation. (2) HOG [25]: This is a directional histogram feature that is usually obtained from the following steps. Firstly, cell units are obtained by dividing the image into small, connected areas. A gradient or edge direction histogram of each pixel in the cell unit is then acquired. Finally, the complete feature descriptors are constituted by combining these histograms. (3) HOG + SOBEL: We calculate the gradient amplitude based on the SOBEL operation as a feature and combine the above HOG feature to form a new feature.  
Depending on the above-mentioned features, three defect classification experiments are performed using the multi-layer perceptron (MLP). The MLP consists of 15 units in a hidden layer and an output layer with three output variables. The number of input layers is determined by the dimensions of the above features. The maximum number of iterations of the optimization algorithm in MLP is 1000. The GLCM feature consists of six gray levels to be distinguished and with a 90\({}^{\circ}\) direction to be calculated in the co-occurrence matrix. The quantization of the gray values in HOG is eight. The size of the filter mask in SOBEL is 3 \(\times\) 3. Table 4 shows the experiments results. It can be shown that the shallow feature methods based on machine learning can only achieve an accuracy of about 70%, while CNN surpasses these methods by more than 15% accuracy. The combined shallow features have a slight improvement over the single features.  
Figure 7 shows the detailed classification results of the four methods. Conventional machine learning methods usually need to design features to train the model. However, CNN has achieved end-to-end training, from the feature learning to the direct output of the classification results. As shown in Figure 7, the traditional method is difficult to distinguish between the two types of dust and damage spots. This may be because both their texture and gradient information are so close that it is sometimes difficult to distinguish them from each other. However, our method can better distinguish between damage spots and dust, and the classification accuracy of the damage spots can reach more than 84%. This is explained by the existence of defects being very complex in the industrial scenarios. It is difficult to fully represent the features of actual defects only by texture and gradient features.  
\begin{table}
\begin{tabular}{c c} \hline \hline
**Method** & **Accuracy** \\ \hline GLCM + MLP [7] & 72.86\% \\ HOG + MLP [25] & 68.99\% \\ HOG + SOBEL + MLP & 69.76\% \\ Compact CNN & **86.82\%** \\ \hline \hline \end{tabular}
\end{table}
Table 4: The performance of classification results with different methods.  
\begin{table}
\begin{tabular}{c c} \hline \hline
**Method** & **IoU** \\ \hline FCN [36] & 81.58\% \\ Single AE without atrous convolution & 83.40\% \\ Single AE & 84.68\% \\ CASAE without atrous convolution & 87.30\% \\ CASAE & **89.60\%** \\ \hline \hline \end{tabular}
\end{table}
Table 3: The quantitative performance of segmentation results with different methods.