# A Generic Automated Surface Defect Detection Based on a Bilinear Model
###### Abstract
[10] proposed a CNN-based LEDNet network for light-emitting diode (LED) defect detection, and used Class Activation Mapping (CAM) [11] to achieve an automatic location of defects. [PERSON] et al. [12] used a novel cascade auto-encoder to segment and locate metal surface defects automatically. [PERSON] et al. [13] used a combination of the Convolutional Auto Encoder (CAE) and Semi-supervised Generative Adversarial Networks (SGAN) to detect surface defects in steel, where CAE was used to extract the fine-grained features of the steel surface, and SGAN was used to further improve the generalization ability of the network. The authors tested the steel defect dataset to verify the effectiveness of the proposed method. Compared with the traditional methods based on the image structure and statistical features, combined with machine learning, the advantage of using CNN-based deep learning for surface defect detection is that CNN can simultaneously realize the automatic extraction and recognition of features in a network, and get rid of the trouble of manually extracting features. Defect localization can make the observer find and understand the location of surface defects more intuitively. In essence, defect localization belongs to the category of object detection. Therefore, some researchers regarded surface defect detection as the problem of defect detection. [PERSON] et al. [14] used a Faster-Region Convolutional Neural Network (Faster-RCNN) [15] and a Single Shot MultiBox Detector (SSD) [16] object detection algorithm to detect steel surface defects, and achieved a higher accuracy and recall rate. [PERSON] et al. [17] proposed a defect detection method based upon Faster-RCNN, and verified the effectiveness of the proposed defect detection method on concrete cracks, steel corrosion, bolt corrosion and steel delamination. The advantage of using an object detection algorithm to detect and locate surface defects, is that it can directly draw lessons from the successful and excellent algorithms in object detection tasks, but these algorithms require a large number of pixel-level labeled training samples, which is difficult to achieve in actual industrial production. Aiming at the problem of sample labeling difficulty for defect detection in actual industrial production, [PERSON] et al. [10] and [PERSON] et al. [18] used Class Activation Mapping (CAM), which is a class-discriminative localization technique that generates visual explanations from the CNN-based network to automatically locate surface defects. The CAM replaced the last full connection layer of the CNN network with Global Average Pooling (GAP) [19] to calculate the spatial average of each feature mapping in the last convolution layer, serving as input features to the fully-connected layer. In this way, the importance of the image region can be recognized by projecting the weights of the output layer back to the convolutional feature map. However, the network with CAM needs to change the original design structure of the network, resulting in the need to retrain the network, therefore its usage scenarios are limited. To overcome the shortcomings of CAM, [PERSON] et al. [20] proposed Gradient-based Class Activation Mapping (Grad-CAM), but calculated the weights by using the global average of the gradient, which is the generalization of CAM, and is suitable for any CNN-based network without modifying any architecture of the network or re-training. Therefore, to solve the problems above, a generic method of automated surface defect detection based upon a bilinear model is presented in this paper. Firstly, the Double-VGG16 (D-VGG16) that consists of two completely symmetric sub-networks based on VGG16 [21] is proposed as the feature extraction network of the bilinear model [22]. The output of the bilinear model uses the soft-max function to predict the corresponding type of the input image, which is realized as the automatic detection of surface defects. Then the heat map of the original image is obtained by applying Grad-CAM to one of the output features of D-VGG16.