## 3 Autoencoder Model for Repairing Defect Areas
### Deep Embedded Clustering Algorithm  
The purpose of feature clustering training using the DEC algorithm [23] is to concentrate the probability distribution of latent vectors from scattered to concentrated probability distribution. The DEC algorithm uses the distance between the latent vector and the cluster center to calculate the probability that the latent vector belongs to each cluster, and it is called the source probability distribution. The source probability that the \(i\)-th latent vector belongs to the \(j\)-th cluster is defined as Equation (4).  
\[S_{ij}=\frac{\left(1+\left|\left|c_{i}-\mu_{j}\right|\right|^{2}/\alpha\right) ^{-\frac{\alpha+1}{2}}}{\sum_{j}\left(1+\left|\left|c_{i}-\mu_{j}\right|\right| ^{2}/\alpha\right)^{-\frac{\alpha+1}{2}}} \tag{4}\]  
where \(c_{i}\) is the \(i\)-th latent vector and \(\mu_{j}\) is the \(j\)-th cluster center. The value of \(\alpha\) is used to adjust the clustering degree of the source probability distribution \(S_{ij}\). The target probability distribution defines the target that the source probability distribution approaches, as in Equation (5).  
\[T_{ij}=\frac{S_{ij}^{2}/f_{j}}{\sum_{j^{\prime}}S_{ij^{\prime}}^{2}/f_{j^{ \prime}}}\text{, }f_{j}=\sum_{i}S_{ij} \tag{5}\]  
Finally, the KL-Divergence function defined in Equation (6) is used to describe the difference between \(S_{ij}\) and \(T_{ij}\) as the loss function.  
\[L_{DEC}=\text{KL}(\text{T}|\left|\text{S}\right)=\sum_{i}\sum_{j}T_{ij}log \frac{T_{ij}}{S_{ij}} \tag{6}\]  
Figure 12 illustrates the clustering effect for the latent vectors generated from the encoder trained with the KL-Divergence function.  
Figure 11: The Autoencoder model with defect area repair capability.  
In order to make the Autoencoder model have both normal area reconstruction capability and defect area repair capability, this paper proposes a two-stage training method. The first stage uses the SSIM loss function to train the encoder and decoder modules to make them have normal area reconstruction capabilities. In the second stage of model training, different combinations of loss functions are used to optimize each module to make it have the capability to repair defective areas. The procedures of the model training method are described in the following steps:  
Step 1 Train Encoder Module + Decoder Module with \(L_{\text{SSIM}}\)  
Step 2.1 Train Encoder Module with \(\alpha_{\text{SSIM}}\)\(L_{\text{SSIM}}\) + \(L_{\text{DEC}}\)  
Step 2.2 Train Decoder Module with \(L_{\text{SSIM}}\)  
Step 2.3 Train Clustering Module with \(L_{\text{DEC}}\)  
Since the encoder module needs to output the feature map to the decoder module and the clustering module, it is necessary to use \(L_{\text{SSIM}}\) and \(L_{\text{DEC}}\) functions at the same time, and use \(\alpha_{\text{SSIM}}\) to adjust the ratio between the two functions to optimize the training results. The decoder module is only responsible for reconstructing the latent map back to the image, so it only needs \(L_{\text{SSIM}}\) function to train the decoder. Finally, the clustering module is trained with \(L_{\text{DEC}}\) function to update the cluster centers and KL-Divergence function.