# Dual Auto-Encoder GAN-Based Anomaly Detection for Industrial Control System
## 4 Dual Auto-Encoder GAN-Based Anomaly Detection for Industrial Control System (DAGAN)
### DAGAN Model
#### 4.1.3 Parameter-Free Marginal Distribution Learning  
For the DAGAN model to work efficiently, a key problem needs to be solved: **how do we learn the marginal distribution directly from the training samples? **  
To solve this problem, a discriminator score-based strategy inspired by the FenceGAN model is used in this paper to learn the marginal distribution of the training dataset. The basic idea is that when a large number of normal samples (non-tail data) and a small number of marginal samples (tail data) are used to train the discriminator in GAN, the discriminator will try to accurately identify the normal samples with a score of 1. At the same time, a small number of marginal samples reach a score less than 1, such as 0.8. That is to say, the discriminator score can be used as an indicator to measure whether a sample is a marginal sample, as shown in Figure 3. In the figure, a long-tail data distribution is presented, where the non-tail region represents the normal distribution with a very large area, and the tail regions on both sides represent the marginal distribution with a very small area. If a discriminator in GAN is used to identify the data of this distribution, the red point P1 has a score of 1, and the green point P2 has a score of 0.68. Moreover, as the data points are closer to the center of the data distribution, the discriminator score (D-Score) gets larger and closer to 1. In other words, the smaller the D-Score of a training sample, the more likely that training sample is in the marginal distribution. Based on the above idea, the FenceGAN model introduces a preset threshold \(\mu\) to treat all samples with a discrimination score less than u as marginal samples. Through the adversarial training between the generator and the discriminator in the GAN, the generator can generate marginal samples with a D-Score close to \(\mu\), so as to learn the marginal distribution of the training dataset. However, different datasets in ICS have different characteristics, and their preset thresholds \(\mu\) must be different. In addition, it is difficult to know which marginal samples are collected from the real training samples. These limitations  
Figure 3: Correlation of discriminator score with marginal distribution. application of the FenGAN model. To enhance robustness and applicability, this paper tries to design a parameter-free marginal distribution learning strategy based on discriminator score, which can quickly and efficiently learn the marginal distribution of any industrial control system. To achieve this goal, three difficulties need to be solved:  
(1) The first difficulty is how to automatically calculate the threshold \(\mu\) without any supervision information. The easiest way is to take the minimum discriminant score (D-Score) in the dataset as \(\mu\). However, this strategy is prone to giving high scores to many outliers close to the marginal distribution, resulting in these outliers being misjudged, as shown in Figure 3(a). In the figure, all black points are marginal points and the region where they are located is marginal distribution, and all blue points are outliers close to marginal distribution. When the discriminator uses the minimum score (blue oval circle) as the learning criterion, the discriminator will assign a higher score to the point closer to the blue oval. For example, P1 has a score of 1.0, P3 has a score of 0.86, and P2 has a score of only 0.8. This results in the blue outlier P3 being misjudged as a normal sample. To solve this difficulty, the average value of the scores of multiple marginal points is used as the value of the parameter \(\mu\) in this paper, as shown in Figure 3(b). In this figure, the red dotted ellipse is the average of the scores of multiple marginal points, which is used to help the discriminator learn the marginal distribution. Based on the new threshold \(\mu\), the blue anomaly point P3 will have a small score of 0.46, and the green point P2 will have a larger score of 0.86.