# Dual Auto-Encoder GAN-Based Anomaly Detection for Industrial Control System
## 3 Preliminaries on GAN for Anomaly Detection
### GANomaly-Based Anomaly Detection  
A novel GAN-based anomaly detection model, called GANomaly, was proposed in 2018. This model ingeniously combines auto-encoder and GAN, to construct an \"encoder-decoder-encoder\" architecture. This new architecture does not rely on any abnormal samples; only normal samples are used for training. In addition, this architecture can provide accurate anomaly detection from the feature space. The pipeline of the GANomaly model is shown in Figure 1.  
The GANomaly model consists of three sub-networks. The first sub-network is a generator \(G(x)\) consisting of an encoder \(G_{En}\) and a decoder \(G_{Dn}\). It first encodes the input data \(x\) to obtain a low-dimensional vector \(z\), and then uses the decoder to reconstruct the data \(x^{\prime}\) based on the \(z\). The second sub-network is an encoding network, Encoder, which re-encodes the generated data \(x^{\prime}\) to obtain low-dimensional latent features \(z^{\prime}\). The third sub-network is the discriminator network \(D\), which is responsible for the binary classification of the generated data \(x^{\prime}\) and the original data \(x\).  
Based on the above three sub-networks, the loss function of the GANomaly model is as follows:  
**(1) Adversarial Loss:** If high-dimensional and variable data are directly measured, it is easy to cause mode collapse and gradient disappearance. Therefore, a feature matching loss is used instead of cross-entropy, and is defined as follows:  
\[L_{adv}=\underset{x\sim P_{X}}{E}\|f(x)-\underset{x\sim P_{X}}{E}f(G(x))\|_{2} \tag{2}\]  
where the \(f(^{*})\) function is the output of an intermediate layer in \(D\). The goal of \(G\) is to minimize the loss, and the goal of \(D\) is to maximize the loss.  
**(2) Contextual Loss:**\(G\) can be optimized to learn the underlying distribution of the real data by minimizing the difference between the reconstructed data \(x^{\prime}\) and the original data \(x\). It is defined as follows:  
\[L_{con}=\underset{x\sim P_{X}}{E}\|x-G(x)\|_{1} \tag{3}\]  
Figure 1: The pipeline of GANomaly model.  
**(3) Encoder Loss:** This function minimizes the difference between the low-dimensional features of \(x^{\prime}\) and the low-dimensional features of \(x\), ensuring that anomaly detection can be performed in the feature space. It is defined as follows:  
\[L_{enc}=\underset{x\sim p_{X}}{E}\|G_{E}(x)-E(G(x))\|_{2} \tag{4}\]  
Therefore, the full loss function of the GANomaly model is computed as follows:  
\[L=w_{adv}L_{adv}+w_{con}L_{con}+w_{enc}L_{enc} \tag{5}\]  
where \(w_{adv}\), \(w_{out}\), and \(w_{enc}\) are the weights corresponding to the three loss functions.  
After the GANomaly model is trained, the encoder \(G_{Ent}\), the decoder \(G_{Dtr}\), and the reconstructed encoder, Encoder, can only identify the latent distribution of normal samples. Then, when an abnormal sample is input into the model, the encoder \(G_{En}\) and the decoder \(G_{Dn}\) will try to convert the abnormal sample into a normal sample \(G(x)\). This results in a significant difference between the low-dimensional feature \(z\) of the input \(x\) and the low-dimensional feature \(z^{\prime}\) of the generated \(x^{\prime}\). Therefore, the GANomaly model uses an anomaly score to measure this difference, defined as follows:  
\[\mathrm{A}(x)=\left\|G_{E}(x)-E(G(x))\right\|_{1} \tag{6}\]  
When the score of a sample is greater than a preset threshold \(\mu\), the sample is regarded as an outlier.