# Dual Auto-Encoder GAN-Based Anomaly Detection for Industrial Control System
## 4 Dual Auto-Encoder GAN-Based Anomaly Detection for Industrial Control System (DAGAN)
### Model Training
#### 4.2.1 Training Objectives  
**(1) The first GAN sub-network**  
**Generator \(G_{1}\). ** The \(G_{1}\) consists of an encoder \(G_{En}\) and a decoder \(G_{Dn1}\), which aims to learn the normal distribution of the training dataset. Moreover, the \(G_{1}\) is also directly concatenated with the feature extractor to form an encoder-decoder-encoder structure (\(G_{En^{*}}\)-\(G_{Dn1}\)-\(En_{1}\)). Therefore, the loss function of the generator \(G_{1}\) consists of three parts:  
* **Adversarial Loss:** This function is responsible for making the generated sample as real as possible, so that the discriminator \(D_{1}\) cannot correctly identify whether the samples are real or generated. The loss function is defined as follows: \[L_{adv}^{G_{1}}=\frac{1}{N}\sum_{i=1}^{N}[\log(1-D_{1}(G_{1}(x)))]\] (7) By minimizing the above adversarial loss, the generator \(G_{1}\) can learn the normal distribution of the training dataset. * **Reconstruction Loss:** The purpose is to ensure that \(G_{Dn1}\) can accurately learn the normal distribution of the training dataset by making the sample \(x_{1}^{\prime}\) reconstructed by \(G_{Dn1}\) as similar as possible to the original sample \(x\). The loss function is defined as follows: \[L_{rec}^{G_{1}}=\frac{1}{N}\sum_{i=1}^{N}\left\|x-G_{1}(x)\right\|_{1}\] (8)
* **Encoder Loss:** This function is responsible for ensuring that the low-dimensional feature \(z_{1}^{\prime}\) of \(x_{1}^{\prime}\) obtained by the encoder \(En_{1}\) is as similar as possible to the low-dimensional feature \(z\) of the original sample \(x\). The loss function is defined as follows: \[L_{enc}^{G_{1}}=\frac{1}{N}\sum_{i=1}^{N}\left\|G_{En}(x)-En_{1}(x^{\prime}_{ 1})\right\|_{2}\] (9)  
In summary, the training objective of the generator \(G_{1}\) (\(G_{En^{*}}\)-\(G_{Dn1}\)-\(En_{1}\)) of the first GAN is as follows:  
\[L_{G1}=w_{adv}^{G_{1}}L_{adv}^{G_{1}}+w_{rec}^{G_{1}}L_{rec}^{G_{1}}+w_{enc}^{ G_{1}}L_{enc}^{G_{1}} \tag{10}\]  
where \(w_{adv}^{G_{1}}\), \(w_{rec}^{G_{1}}\), and \(w_{enc}^{G_{1}}\) are the weights corresponding to the three loss functions. The above loss function will help the first GAN to learn the normal distribution of the training dataset, so that it cannot only reconstruct the real samples, but also accurately get the low-dimensional features of the real samples. **Discriminator \(D_{1}\)**: The discriminator \(D_{1}\) of the first GAN is a binary classifier with a multi-layer perceptron structure. It takes real samples and generated samples of \(G_{1}\) as input, and the goal is to accurately identify whether the input sample is a generator sample or a real sample. The loss function is defined as follows:  
\[L_{D1}=\frac{1}{N}\sum_{i=1}^{N}[-\log(D_{1}(x))-\log(1-D_{1}(G_{1}(x)))] \tag{11}\]  
**(2) The second GAN sub-network**  
**Generator \(G_{2}\). ** The \(G_{2}\) consists of an encoder \(G_{En}\) and a decoder \(G_{Dn2}\). Moreover, the \(G_{2}\) is also directly concatenated with the \(En_{2}\) to form an encoder-decoder-encoder structure (\(G_{En^{*}}\)-\(G_{Dn2}\)-\(En_{2}\)). In addition, \(G_{Dn2}\) and \(En_{2}\) share the same neural network structure asand \(En_{1}\). The goal of Generator \(G_{2}\) in the second GAN is to learn the marginal distribution of the training dataset. Therefore, the loss function of the generator \(G_{2}\) includes four parts:  
* **Adversarial Loss**: The purpose is to help the generator \(G_{2}\) generate data with the discriminator score \(\mu\), so as to learn the marginal distribution of the real training set. The loss function is defined as follows: \[L_{adv}^{G_{2}}=\frac{1}{N}\sum\limits_{i=1}^{N}[\log(\mu-D_{2}(G_{2}(x)))]\] (12) where \(\mu\) is the average of the scores of all marginal samples, and the detailed calculation process is described in Section 4.1.3. * **Dispersion Loss**: The purpose is to supervise the data generated by \(G_{2}\) to be as evenly dispersed as possible in the marginal distribution, rather than fixed in some small regions.