# Automated Surface Defect Inspection Based on Autoencoders and Fully Convolutional Neural Networks
## 2 The Proposed Algorithm
### The Training of AE and FCN  
Figure 6 shows the procedure for the training of the AE and FCN. As shown in Figure 6, there are two training stages. The first stage is the training for the AE. After the training process in the first stage is completed, we use the resulting AE network model to generate the training images for the FCN in the second stage of the training process. Figure 4: An example of the AE for the proposed algorithm. The AE in this example contains 5 convolution layers, which are denoted by Conv \(i\), \(i=1,...,5\). Figure 5: An example of the FCN for the proposed algorithm. The FCN in this example contains 11 convolution layers, which are denoted by Conv \(i\), \(i=1,...,11\). Let \(X_{k}\), \(k=1,...,K\), be the \(k\)-th images for the training of the AE, where \(K\) is the number of training images. All the training images are defective images. Furthermore, let \(Y_{k}\), \(k=1,...,K\), be the image at the output of the AE when its input is \(X_{k}\). Given a training image \(X_{k}\), let \(S_{k}\) be the ground truth of \(X_{k}\). That is, \(S_{k}\) is the defect free version of \(X_{k}\). \(S_{k}\) can be regarded as images from a normal sample. The loss function, denoted by \(J\), for the training of the AE is given by  
\[J=\frac{1}{K}\sum_{k=1}^{K}D(S_{k},Y_{k}). \tag{7}\]  
Note that \(Y_{k}\) and \(S_{k}\), \(k=1,...,K\), are the reconstructed images and their ground truth, respectively. Therefore, the goal of the training is to guide the AE to effectively remove defective parts of input samples \(X_{k}\) so that the discrepancy between \(S_{k}\) and \(Y_{k}\) can be minimized. In this way, the AE is only able to reproduce normal patterns of the input images. The images produced by the AE can then be viewed as the templates of the corresponding input images for defect detection. For applications where only normal samples are available, it is only possible to acquire \(S_{k},k=1,...,K\), from the normal samples for training. In these cases, it may be necessary to obtain \(X_{k}\) from \(S_{k}\) by data augmentation. One simple approach for the augmentation is by adding a zero mean Gaussian noise to \(S_{k}\). We then view the corresponding image after noise corruption as \(X_{k}\). That is,  
\[X_{k}=S_{k}+\eta_{k}, \tag{8}\]  
where \(X_{k}\), \(S_{k}\) and \(\eta_{k}\) have the same dimension, denoted by \(M\). Each element \(\varepsilon\) of \(\eta_{k}\) is drawn from a zero-mean Gaussian distribution with variance \(\sigma^{2}\). That is, the density function for each element \(\varepsilon\) of \(\eta_{k}\) is given by \(\frac{1}{\sqrt{2\pi\
u}}\exp{(-\frac{\varepsilon}{2\sigma^{2}})}\). In this way, the template generation process can be regarded as the denoising process [24], where the defective pixels are those corrupted by noises. The input image \(X_{k}\) and output image \(Y_{k}\) to the AE are the corrupted and restored versions of \(S_{k}\), respectively. The AE in the proposed algorithm is then equivalent to a denoiser, where the ground truth \(S_{k}\) is available for each corrupted observation \(X_{k}\) for the training. An advantage of the proposed AE training approach is that a single image from normal sample can be used to generate multiple defective images. That is, different training images \(X_{k}\) may have the same ground truth \(S_{k}\). Therefore, even for the cases where only a small number of normal samples are available, a large number of training images can still be produced. This could be beneficial for the avoidance of overfitting for the training of the AE. The training of FCN is based on \(Y_{k},k=1,...,K\), which are the reconstructed images produced by the AE. Let \(y_{k}^{j}\) be the \(j\)-th pixel of the image \(Y_{k}\). Let \(P_{k}^{j}(n)\) be the estimated  
Figure 6: Procedure for the training of AE and FCN. probability that the pixel \(y_{k}^{j}\) belongs to the region \(n\). Therefore, for fixed \(k\) and \(j\), it follows that  
\[\sum_{n=1}^{N}P_{k}^{j}(n)=1, \tag{9}\]  
where \(0\leq P_{k}^{j}(n)\leq 1\).