## 5 Results
### Adversarial Example Attacks
#### 5.2.5 Adversarial Examples in Cyber-Security Domains  
Adversarial examples (AE) have been shown to exist in many domains. Indeed, no domain identified (so far) is immune to adversarial examples [56]. Researchers are beginning to consider cyber-security domains (Figure 5) where features are often a mixture of categorical, continuous, and discrete. Some research focuses on adversarial example attacks against IDS, although few studies specifically consider functionality-preserving attacks. In the visual domain, we briefly discussed the consensus that adversarial examples are undetectable to humans. However, it is unclear how this idea should be translated to other domains. [PERSON] [64] held that, strictly speaking, adversarial examples must be similar to the original input. However, [PERSON] et al. [56] noted that research in non-visual domains provides domain-specific definitions: perturbed malware must preserve its malware functionality [56], perturbations in audio must be _nearly_ inaudible [56], and perturbed text must preserve its meaning. [PERSON] et al. further offered a definition for adversarial examples in intrusion detection: perturbed network flows must maintain their attack behaviour. We consider that human perception may not be the best criterion for defining adversarial examples in cyber-security domains. Indeed, human perception in some domains might be immaterial. For example, only very skilled engineers could _perceive_ network packets in any meaningful way even with the use of network analysis tools. Furthermore, users likely cannot perceive a difference between the execution of benign or malicious software. After malware is executed, the effects are clear; however, during malware execution users often suspect nothing wrong. We therefore consider that while fooling human perception remains a valid ambition, it is critical that adversarial perturbations in cyber-security domains preserve functionality and behaviour. In the cyber-security domain, traditional gradient descent algorithms may be insufficient. Algorithms that preserve functionality are required. Moreover, some models used in the cyber-security domain are distinct from those used for purely visual problems. For example, RNNs are useful for time sequences of network traffic analysis. We now consider recent functionality-preserving attacks in the cybersecurity domains of malware, intrusion detection, and CPS. We further examine Functionality-preserving attacks in Table 7.  
Figure 5: Common machine learning tasks in cyber security. **Malware:** [PERSON] and [PERSON] [84] proposed a novel algorithm to generate adversarial sequences to attack an RNN-based malware detection system. They claimed that algorithms adapted for RNNs are limited because they are not truly sequential. They considered a system to detect malicious API sequences. Generating adversarial examples effective against such systems is non-trivial because API sequences are discrete values. There is a discrete set of API calls; changing any single letter in an API call will create an invalid API call and cause that API call to fail. This will result in a program crash. Therefore, any perturbation of an API call must result in a set of valid API calls. They proposed an algorithm based around a generative RNN and a substitute RNN. The generative RNN takes an API sequence as input and generates an adversarial API sequence. The substitute RNN is trained on benign