# Dual Auto-Encoder GAN-Based Anomaly Detection for Industrial Control System
## 5 Experimental Evaluation
### Performance Analysis of Optimization Strategy for the Marginal Distribution Learning  
The second experimental objective is to verify the effectiveness of optimization strategy for the marginal distribution learning. In this paper, an optimization strategy is proposed to achieve parameter-free marginal distribution learning, aiming to improve the robustness of our model. Furthermore, in the optimization strategy, the average of the discriminator scores of multiple marginal nodes is used instead of the minimum value for better learning the marginal distribution. To verify the effectiveness of our strategy, the minimum value-based strategy is used as another competitor. In this experiment, we try to analyze the anomaly detection performance based on two strategies for verifying the effectiveness of our strategy, as shown in Figures 7 and 8. The ACC, Rec, FPR and F1 are used as evaluation metrics, and DS2 OS and SWaT are chosen as the experimental datasets. For two datasets, multiple samples of each attack are combined with multiple normal samples to establish a  
Figure 6: Sensitivity analysis of hyper-parameter \(\gamma\).  
In Figure 7, the first row plots the ACC performance of two strategies on 6 test sets, the second row plots the Rec performance, the third row plots the FPR performance, and the four row plots the F1 performance, respectively. On each test set, the first bars represent the anomaly detection performance of the minimum value-based marginal distribution learning strategy, the second bars represent the performance of our optimization strategy, and the red numbers are the values of performance optimization. From the figure, the following observations are easily obtained. (1) For the ACC, our strategy shows good optimization performance on 6 test sets, with an average optimization rate of 9%. Moreover, the optimization performance of our strategy is stronger for the test set with lower detection accuracy. For example, when the detection accuracy on the DoS test set reaches 97%, and the optimized value of our strategy is only 4%. When the detection accuracy on the WS test set is only 80%, the optimized value of our strategy reaches 13%. (2) For the Rec, our strategy also shows good performance, with an average optimization value of 8%. (3) For the FPR, the optimization performance of our strategy is better than that of the ACC, Rec, and F1, with an average optimization value of 15%. For example, the optimized value is 20% on the MC test set, the value is 17% on the MO test set, and the value is 16% on the SC test set. (4) For the F1, the average optimized value of our strategy is 9%. In addition, the lower the detection accuracy, the stronger the optimization ability of our strategy.  
Figure 8 further plots the performance analysis of two optimization strategies on the SWaT dataset. Same as Figure 7, the first, second, third and fourth rows in this figure show the ACC, Rec, FPR and F1 performance of the two strategies on the 4 test sets, respectively. From Figure 8, it is easy to get some patterns. (1) Our strategy can effectively improve the performance of anomaly detection. For example, the average optimization value on the ACC is 8%, the value on the Rec is 7%, the value on the FPR is 11%, and the value on the F1 is 6%. Moreover, the anomaly detection performance of ACC, Rec, and F1 exceeds 95% on the SSSP test set, exceeds 90% on the SSMP test set. (2) The lower the performance of ACC, Rec, FPR, and F1 on a test set, the stronger the optimization ability of our strategy. Such as, on the SSSP test set, the accuracy of anomaly detection (ACC, Rec, and F1) exceeds 95%, the optimized value of our strategy is 6% on the ACC, 7% on the Rec, 7% on the F1. However, on the MSMP test set, accuracy of anomaly detection (ACC, Rec, and F1) is only around 83%, the optimized value of our strategy is 10% on the ACC, 12% on the FRP, and 8% on the F1.