# A Generic Automated Surface Defect Detection Based on a Bilinear Model
###### Abstract  
Aiming at the problems of complex texture, variable interference factors and large sample acquisition in surface defect detection, a generic method of automated surface defect detection based on a bilinear model was proposed. To realize the automatic classification and localization of surface defects, a new Double-Visual Geometry Group16 (D-VGG16) is firstly designed as feature functions of the bilinear model. The global and local features fully extracted from the bilinear model by D-VGG16 are output to the soft-max function to realize the automatic classification of surface defects. Then the heat map of the original image is obtained by applying Gradient-weighted Class Activation Mapping (Grad-CAM) to the output features of D-VGG16. Finally, the defects in the original input image can be located automatically after processing the heat map with a threshold segmentation method. The training process of the proposed method is characterized by a small sample, end-to-end, and is weakly-supervised. Furthermore, experiments are performed on two public and two industrial datasets, which have different defective features in texture, shape and color. The results show that the proposed method can simultaneously realize the classification and localization of defects with different defective features. The average precision of the proposed method is above 99% on the four datasets, and is higher than the known latest algorithms. automated surface inspection; D-VGG16; bilinear model; Grad-CAM; classification; localization +
Footnote †: journal: Applied  
1
Footnote 1: School of Information Engineering, Southwest University of Science and Technology, Mianyang 621010, China  
[PERSON] et al. [2] utilized histogram analysis to threshold the texture image and then map it into a special data structure of the skeleton representation, achieving the extraction of texture image defects. [PERSON] et al. [3] used the image edge intensity and the distribution of the gray values of pixels in the edge domain to model the surface defects. [PERSON] et al. [4] realized the defect of the metal surface by wavelet analysis. Although the detection and segmentation of defects can be realized by analyzing the structural features of the surface of the object, the parameters of the algorithm need to be set manually for most of these methods, making them easily affected by interference factors, such as illumination in the environment, thereby affecting the detection effect. The methods of combining statistical features with machine learning are mainly to extract statistical features from the defect surface, and then use machine learning algorithms to learn these features in order to realize surface defect detection. [PERSON] et al. [5] used a combination of discrete wavelet transforms and a Support Vector Machine (SVM) to detect surface defects in steel. [PERSON] et al. [6] realized the detection of the surface defects of steel strips by constructing a series of SVMs with a random subspace of the features, and an evolutionary separator with a Bayesian kernel to train the results from the sub-SVM to form an integrated classifier. The combination of statistical features and machine learning can obtain higher accuracy and robustness than traditional structure-based methods. However, in image feature modeling, the accuracy of detection may be altered due to the different selections of feature types, and is closely linked to the extracted features, so it is necessary to find a suitable feature descriptor for a specific detection object. Recently, because of the rapid development of deep learning, especially in terms of its strong feature extraction ability, it has been widely used in image-related tasks, such as graphic analysis [7], semantic segmentation [8] and target tracking [9]. Many researchers have also applied deep learning to surface defect detection. [PERSON] et al.