## 5 Discussion & Research Directions  
Adversarial Machine Learning has been applied to evaluate the robustness of ML-based NIDS by proposing context-related methods for attack, defense, or both. Despite the noticeable progress in exploiting AML within NIDS applications, the surveyed works exhibit a considerable scarcity of recent comprehensive surveys relevant to AML applications within IoT frameworks, specifically. In the case of attack methods, several adversarial crafting techniques have been proposed. One can notice that FGSM [26] was the most used one followed by JSMA [38] and C&W [80, 81] attacks. However, FGSM [26] might be an impractical option for the NIDS field since it works on perturbing each possible feature for crafting adversarial examples. Those features exhibit high dependency, strong correlation, and hard constraints. This makes the process impractical and even harder to deploy. Additionally, the process of perturbation is challenging since the adversary needs complete control to perform such an operation. In terms of attack strategies, most of the works assume a black box setting that is suitable for simulating real-world attacks where the attacker can have access to the system's outputs with limited knowledge about the inputs. Additionally, the validity and properness of adversarial examples generated by generic crafting techniques are not assessed sufficiently. This yields potential inconsistency in the generated adversarial traffic. The features of network traffic are structured using specific data types and value ranges. Namely, they can have binary, categorical, or continuous values compared to other domains such as image recognition where such constraints are relaxed. Moreover, many ML algorithms are designed to manage numerical values which require the conversion of categorical values into multiple binary features. This conversion results in serious flaws in applying the perturbations by the famous AML crafting methods such as FGSM [103]. It is worth noting the criteria for selecting the proper crafting techniques by considering several important aspects such as computational power, features space, and perturbations magnitude. Consequently, the validity of adversarial examples can be measured by ensuring limited perturbations, defined values range, retained features semantic relations, and preserved network traffic information [104]. Adversarial crafting methods have been performed against a wide range of ML-based classifiers including both shallow and deep learning methods such as NB, SVM, DT, RF, MLP, CNN, and DNN. Notably, the reported performance of these classifiers degraded drastically. In particular, CNN, DT, and NB yielded the worst results. Additionally, one can notice the lack of publicly available datasets for AML applications in the IoT domain. In fact, most of the state-of-the-art works used Kitsune [92] and CICIDS (2017/2018) [87, 97] datasets. As such, there is an urgent need to have benchmarking datasets that reflect the current IoT network structures and features in order to evaluate the performance of NIDSs under adversarial machine learning attacks. Even though different attack strategies have been studied, only a few research consider defense mechanisms for enhancing the robustness of ML-based classifiers. Moreover, no customized mechanisms have been proposed to defend IoT against AML attacks compared to image recognition applications. Moreover, some studies did not incorporate any defense solutions while others used a limited number of generic techniques. Actually, adversarial training is the most used defense technique compared to feature reduction, ensemble learning, and GAN-based defense. NIDS requires a real-time operation to allow proper processing of the network's traffic. However, several challenges such as process overhead, computational power, and memory requirements are faced by defense mechanisms in the context of NIDS in IoT environments.