## 5 Results
### Adversarial Example Attacks
#### 5.2.5 Adversarial Examples in Cyber-Security Domains
The authors noted that even when the adversarial generation algorithm and the victim RNN were implemented with different models and trained on different training sets, the majority of the adversarial examples successfully attacked the victim RNN through the _transferability_ property of adversarial examples. In MLP, they reported a TPR of 94.89% that fell to 0.00% under adversarial perturbations. [PERSON] et al. [98] preserved the functionality of malware while evading static windows malware detectors. Their attacks exploit the structure of the portable executable (PE) file format. Their framework has three categories of functionality-preserving manipulations: structural, behavioural, and padding. Some of their attacks work by injecting unexecuted (benign) content in new sections in the PE file, or at the end of the malware file. The attacks are a constrained minimization problem optimizing the trade-off between the probability of evading detection and the size of injected content. Their experiments successfully evaded two Windows malware detectors with few queries and a small payload size. Furthermore, they discovered that their attacks transfered to other Windows malware products. We note that the creation of new sections provides a larger attack surface that may be populated with adversarial content. They reported that their section-injection attack was able to drastically decrease the detection rate (e.g., from an original detection rate of 93.5% to 30.5%, also significantly outperforming their random attack at 85.5%). [PERSON] et al. [53] presented a gradient-based method to generate valid executable files that preserve their intended malicious functionality. They noted that malware evasion is a current area of adversarial learning research. Evading the classifier is often the foremost objective; however, the perturbations must also be carefully crafted to preserve the functionality of malware. They noted that removing objects from a PE file often leads to corrupt files. Therefore, they only implement additive or modifying perturbations. Their gradient-based attack relies on _complete knowledge_ of the system with the advantage that the likelihood of evasion can be calculated and maximised. Furthermore, they stated that their system only generates valid executable malware files. **Intrusion Detection:** [PERSON] et al. [94] used a generative adversarial network (GAN) to generate functionality-preserving adversarial examples. They noted that adversarial examples aiming to evade IDS should not invalidate network traffic features. A typical GAN composed of two neural networks, a generator \(G\) and discriminator \(D\), was used to construct adversarial examples that masquerade as benign but functionally probe the network. Their attack was able to evade an IDS while preserving the intended behaviour. They suggested that adversarial training using GAN-generated adversarial examples improved the robustness of their model. They reported F1-scores of 89.03 (original), 40.86 (After attack), 78.49 (after adversarial training), and an improved 83.56 after GAN-based adversarial training. [PERSON] et al. [117] noted that relatively few researchers are addressing adversarial examples against IDS. They proposed an ensemble defense for network intrusion detection that integrates GANS and adversarial retraining. Their training framework improved robustness while maintaining accuracy of unperturbed samples. Unfortunately, they evaluated their defences against traditional attack algorithms: FGSM, basic iterative method (BIM), Deepfool, and JSMA. However, they did not specifically consider functionality-preserving adversarial examples. They further recognised the importance of using recent datasets for intrusion detection. They reported F1-scores for three classifiers and a range of adversarial example algorithms. For example, the F1-score for an ensemble classifier tested on clean data was 0.998 compared to 0.746 for JSMA. Among all classifiers, the ensemble classifier achieved superior F1-scores under all conditions. [PERSON] et al.