## 6 Threats to Validity  
The results of this work heavily rely on the experiments conducted in both physical and simulated SDN environments. In this section, we discuss various factors that may affect the validity of the results. The discussion not only clarifies the strengths and assumptions of this work, but also facilitates future research on the detection of saturation attacks in SDN (e.g., replicating the experiments and comparing a new approach to this work). **Threats to external validity** are concerned about generalizing our findings. The results of our experiments show the differences between the physical and the simulated environments in terms of the detection performance of the K-NN, SVM, and NB classifiers and the proper time window to detect known and unknown saturation attacks. For instance, the proper time window for the NB classifier to detect the known and unknown saturation attacks in the physical environment was 3 min. In the simulation environment, the proper time window was 1 min of OpenFlow traffic analysis. This raises a concern related to the relationship between the setup of the SDN environment and the task of allocating the proper time window for the detection of saturation attacks. In our study, the physical environment is a relatively small SDN environment, which consists of an OpenFlow vSwitch and five hosts. We were able to replicate real-world internet traffic using different traffic generation tools. In contrast with the simulated environment, we were able to configure medium-to-large scale SDN environments with some limitations of mimicking real-world network traffics. Thus, we believe that the scale of the SDN network and the network traffic are important parameters that influence identifying the proper time window for each machine-learning classifier. Therefore, we cannot guarantee that the proper time windows of OpenFlow traffic analysis that were obtained in this research, by their corresponding machine learning classifiers, are applicable to all SDN environments. Specifically, different SDN environments might have different proper time windows of OpenFlow traffic analysis for the same task (detection of saturation attacks). However, our analysis provides the basis for further research to obtain the proper time window for different SDN environments. **Threats to construct validity** concern the relationship between the construct and observation. One issue is related to the dataset mislabeling noise and the proper time window for detecting saturation attacks. Our experiments resulted in a small number of normal samples that were mislabeled as attack samples in the generated datasets. Also relevant is the relationship between the dataset mislabeling noise and the identification of the proper time window. The mislabeling noise of training datasets may influence the detection performance of the classifiers. The choice of the proper time window of OpenFlow traffic analysis for the classifiers may change with different datasets. However, in this research, we used a large number of correctly sampled normal traffic in order to minimize the impact of measurement noise coming from a few mislabeled training samples. In addition, we evaluated the robustness of the variational autoencoder classifier against the noise of training datasets, but not all the supervised machine learning classifiers were evaluated against the training dataset noisiness. **Threats to internal validity** concern the internal parameters that could influence the variables and the relations being studied. In our experiments, we used the suggested configurations for SVM and NB classifiers, for instance, the SVM kernel type. However, some parameters could be configured that could affect the obtained proper time window of the OpenFlow traffic analysis. **Threats to conclusion validity** concern the treatment and the findings. The generated datasets have attack samples that are extracted from the malicious OpenFlow traffic. It comprises the saturation attacks that are mostly used by hackers to compromise the SDN networks.