dataesr/scientific-paragraphs-categorization huggingface dataset:
The adoption of ML and DL approaches increased thanks to the computer's evolution, which has provided us with processing capabilities needed for training/testing their models, the availability of large datasets, and the cost-effectiveness of storage capabilities. In addition, the availability of open-source libraries and frameworks causes rapid diffusion within the research community and in our daily life. ML/DL algorithms can be implemented in various platforms and languages such as Weka (does not include DL algorithms), R, and Python. Python is one of the most popular environments for ML/DL as it provides various libraries to developers. Additionally, its simplicity helps developers to save more time, as they only require concentrating on solving the ML/DL problems rather than focusing on the technicality of language.
Therefore, the software agent makes observations and executes actions within an environment, and receives rewards in return. The agent's job is to maximize cumulative reward. The most well-known reinforcement learning technique is Q-learning  [30] , and it is widely used for routing  [31, 32]. Moreover, deep learning has been used to improve the performance of RL algorithms (i. e. , allows reinforcement learning to be applied to larger problems). Therefore, the combination of DL and RL gives the so-called DRL. DRL began in 2013 with Google Deep Mind  [33]. A good survey that presents RL and DRL approaches is available in  [34]  for more details. In Section 2. 1, a synthesis of DL architecture and models is presented, followed by an overview of conventional ML models as well as feature reduction techniques.
Deep Learning Deep learning (DL), also known as deep neural networks (DNN) represents one of the most active areas of AI research  [13]. It is a branch of ML that evolved from neural network (NNs) which enables an algorithm to make predictions or classifications based on large datasets without being explicitly programmed. In many domains, DL algorithms are able to exceed human accuracy. Using conventional ML-based models requires some feature engineering tasks like feature selection  [35] , which is not the case with DL models, as these can hierarchically extract knowledge automatically from raw data by stacked layers  [23]. The major benefits of DL over conventional ML models are its superior performance for large datasets  [36]  and the integration of feature learning and model training in one architecture, as illustrated in Figure  2.
Efficient against over-fitting Requires a large training dataset, impractical for real-time applications Support vector machine (SVM) SVM is a powerful classification algorithm which can be used for both regression and classification problems. However, it is mostly used as classification technique. It was initially developed for binary classification, but it could be efficiently extended to multiclass problems. It can be a linear and non-linear classifier by creating a splitting hyperplane in original input space to separate the data points.
Scalable, handles complex data Computationally expensive, there is no theorem to select the right kernel function K-nearest neighbour (KNN) KNN is a supervised model reason with the underlying principal "Tell me who are your friends, I will tell you who are you"  [48]. It classifies new instances using the information provided by the K nearest neighbors, so that the assigned class will be the most common among them (majority vote). Additionally, as it does not build a model and no work is done until an unlabeled data pattern arrives, it is thus considered as a lazy approach. Easy to implement, has good performance with simple problems, non-expert users can use it efficiently Requires large storage space, determining the optimal value of K is time consuming, K values varies depending on the dataset, testing is slow, and when the training dataset is large, it is not suitable for real-time classification K-means K-means is a well-known unsupervised model.
Feature Reduction The success of conventional ML algorithms generally depends on the features to which they are applied. In other words, their performance depends not only on the parameters but also on the features, which are used to describe each object to the ML models  [50]. In this section, we provide an overview of basic concepts as well as the motivation of the dimensionality reduction technique. In general, the dataset can contain irrelevant (a feature that provides no useful information) and redundant (a feature with predictive ability covered by another) features, which cause an extra computational cost for both storage and processing in an SDN environment and decreases the model's performance  [51, 52]. For example, the time complexity of many ML algorithms is polynomial in the number of dimensions; for example, the time complexity of logistic regression is O(mn 2 + n 3 ), where n represents the number of features and m represents the number of instances  [53].
These challenges are referred to as the curse of dimensionality, which is one of the most important challenges in ML. The curse of dimensionality was introduced by Bellman in 1961  [18]. In addition, each model is only as good as the given features. Additionally, the model accuracy depends not only on the quality of input data but also on the amount features. The model may perform poorly and can overfit if the number of features is larger than the number of observations  [54]. Moreover, the low-latency applications (e. g. , anomaly detection) require fast analysis, especially in the context of the fifth-generation (5G) networks, and feature reduction can help to leverage heavy analysis at network levels. All these make feature reduction difficult tasks not only because of the higher size of the initial dataset but because it needs to meet two challenges, which are to maximize the learning capacity and to reduce the computational cost and delay by reducing the number of features.
In fact, there are many benefits of feature reduction techniques, such as (i) improving the performance of learning algorithms either in terms of learning speed or generalization capacity and (ii) reducing the storage requirement. It reduces the original data and retains as much information as possible that is valuable for the learning process. Therefore, in a dataset with a high number of features, the data reduction process is a must to produce accurate ML models in a reasonable time. It can be divided into feature selection (i. e feature elimination) and feature extraction, as shown in Figure  3. The main difference is that feature selection selects a subset of original features, while feature extraction creates new features from the original ones. Table  7  presents the advantages and disadvantages of each approach.
Each approach can be used in isolation or in a combination way with the aim of improving performance, such as the accuracy, visualization, and comprehensibility of learned knowledge  [55]. Rangarajan et al. [56]  proposed bi-level dimensionality reduction methods that combine feature selection methods and feature extraction methods to improve classification performance. Then, they compared the performance of three-dimensionality reduction techniques with three datasets. Based on their results, these combinations improve the precision, recall, and F-measure of the classifier. Additionally, we can find feature construction, which is different from the feature reduction technique. It is one of the preprocessing tasks that tries to increase the expressive power of initial features when the initial features set do not help us classify our data well enough.
It helps us to identify the relevant features (contribute to the identification of the output) and discard the irrelevant ones from the dataset to perform a more focused and faster analysis. The feature selection process consists of four basic steps, which are: 1. Subset generation is a search procedure that generates candidate feature subsets for evaluation based on a search strategy (i. e. , start with no feature or with all features); 2. Evaluation of subset tries to measure the discriminating ability of a feature or a subset to distinguish the target variables; 3. Stopping criteria determines when the feature selection process should stop (i. e. , addition or deletion of any feature does not produce a better subset); 4. Result validation tries to test the validity of the selected features.
As DL helps to eliminate the manual feature engineering task, Hu et al. [83]  proposed a CNN-based deep learning method to address the SDN-based application awareness called CDSA. As shown in Figure  6 , CDSA consists of three components, which are traffic collection, data pre-processing, and application awareness. To evaluate the performance of this framework, the open Moore dataset  [84]  has been used. This dataset was released by the University of Cambridge. As new types of traffic emerge every day (and they are generally not labeled), this opens a new challenge to be handled. In fact, labeling data is often difficult and timeconsuming  [28]. Therefore, researchers started to reformulate traffic classification into semisupervised learning, where both supervised learning (using labeled data) and unsupervised learning (no label data) were combined.
However, the authors did not present any detail about the complexity of the proposed framework. Similar work is done by Yu et al. [86] , where the authors proposed a novel SDN flow classification framework using DPI and semi-supervised ensemble learning. The significant difference is the ML algorithm used for the classification. Firstly, they used DPI to generate a partially labeled dataset with the QoS type of flow according to the application type. To train the classifier, they used the heteroid tri-training algorithm. This algorithm is a modified version of tri-training, which is a classic semi-supervised learning mechanism that uses three identical classifiers to enable iterative training. However, they used three heterogeneous classifiers (SVM, Bayes classifier, K-NN) instead of three identical classifiers to increase the difference among classifiers.
To verify the performance of their framework, they used a real dataset and trained the classifier with different ratios of unlabeled application flows. However, the authors did not mention information about the feature selection method and the dataset used during the experiments. In addition, Zhang et al. [63]  proposed a DL-based model in the SDN-based environment to classify the traffic to one of several classes (Bulk, Database, Interactive, Mail, Services, WWW, P2P, Attack, Games, Multimedia). It consists of the stacked autoencoder (SAE) and softmax classifier. SAE was used for feature extraction and softmax was used as a supervised classifier. The experimental results show that the proposed model outperforms the SVM. However, the authors proposed only the framework without testing its performance in the SDN environment.
Finally, to avoid data labeling, K-means has been proposed by Kuranage et al. [87]. In this paper, several supervised learning methods were trained and evaluated individually, including SVM, DT, RF, and KNN. The results demonstrate that SVM has the highest accuracy with 96. 37%. For this paper, the "IP Network Traffic Flows, Labeled with 75 Apps" dataset was used  [88]. However, the features used for the models' training are selected manually; •.
Fine-grained traffic classification Fine-grained classification aims to classify the network traffic by application (the exact application generating the traffic) and facilitates the operators to understand users' profiles and hence provide better QoS. Qazi et al. [89]  presented a mobile application detection framework called Atlas. This framework enables fine-grained application classification. The netstat logs from the employee devices are collected by the agents to collect ground truth data and are sent to the control plane, which runs the decision tree algorithm (C5. 0) to identify the application. Their framework is capable of detecting a mobile application with 94% accuracy for the top 40 Android applications. However, the authors did not present the features used in this framework.
The accuracy of each application is used as an evaluation metric. However, the study lacks the exploration of the other performance metrics (e. g. , f-measure). Additionally, there is no information on the distribution of applications in the dataset. As each application can have several types of flows, like voice and chat, a fine-grained traffic classiication is needed. To solve this issue, Uddin and Nadeem  [92]  introduced a traffic classification framework called TrafficVision that identifies the applications and their corresponding flow-type in real-time. TrafficVision is deployed on the controller, and one of the kernel modules of TrafficVision is named TV engine, which has three major tasks: (i) collecting, storing, and extracting flow statistics and ground-truth training data from end devices; (ii) building the classifiers from the training data; and (iii) applying these classifiers to identify the application and flow-types in real-time and providing this information to the upper layer application.
As a proof of concept, the authors developed two prototypes of "network management" services using the TrafficVision framework. The classification task of the TV engine has two modules, which are application detection using decision tree (C5. 0) and flow-type detection using KNN with K = 3. Both of these classifiers show more than 90% accuracy. The shortcoming in this paper is the classification of the 40 popular applications and the ignorance of other applications; this can cause problems in an enterprise or university network. Amaral et al. [93]  proposed an application-aware SDN architecture, and the experimental results demonstrate that the semi-supervised classifier outperforms the supervised classifier (random forest classifier) with the same amount of labeled data. This is attributed to the fact that the incorporation of unlabeled data in the semi-supervised training process boosts the performance of the classifiers.
Nakao and Du  [94]  used deep NN to identify mobile applications. Mobile network traffic was captured from the mobile virtual network operator (MVNO). Five flow features (i. e. , a destination address, destination port, protocol type, TTL, and packet size) were selected to train an 8-layer deep NN model. DL achieves 93. 5% accuracy for the identification of 200 mobile applications. The features used for classification are selected filter methods in order to train and improve the performance of the DL model. Wang et al. [95]  developed an encrypted data classification framework called DataNet which is embedded in the SDN home gateway. This classification was achieved through the use of several DL techniques, including multilayer perceptron (MLP), stacked autoencoder (SAE), and convolutional neural networks (CNN).
They used the "ISCX VPN-nonVPN" encrypted traffic dataset  [96] , which consists of 15 encrypted applications (i. e. , Facebook, Skype, Hangout, etc. ) and more than 200,000 data packets. However, they used only 73,392 flows after adopting the under-sampling method to balance the classes' distribution. This framework helps them to classify the traffic without compromising the security/privacy of services providers or users. Chang et al. [97]  proposed an application for offline and online traffic classification. More specifically, the authors used three DL-based models, including CNN, MLP, and SAE models. Using the Tcpreplay tool, the traffic observations are re-produced and analyzed in an SDN testbed to emulate the online traffic service. The experimental results show that the offline training results have achieved more than 93% accuracy, whereas the online testing prediction achieved 87% accuracy for application-based detection.
• VPN-nonVPN dataset  [96]   Dataset-Unicauca  [88]  is released by the Universidad Del Cauca, Popayán, Colombia. It was collected through packet captures at different hours during the morning and afternoon over 6 days in 2017. This dataset consists of 87 features, 3,577,296 observations, and 78 classes (Twitter, Google, Amazon, Dropbox, etc. ).
They implemented their framework and deployed it on SDN, then trained it on a real-world dataset using different model configurations. The experimental results show that their model converges quickly and can outperform linear forecasting models and feed-forward deep neural networks. Alvizu et al. [110]  applied a neural network model to predict network traffic load in a mobile network operator. The prediction results are used to make online routing decisions. The authors in this work train a real dataset for mobile network measured during November and December 2013 in Milano, Italy. Similarly, based on ANN, Chen-Xiao and Ya-Bin  [111]  proposed a solution to keep network load balanced using SDN. This solution aims to select the least-loaded path for newcomer data flow. They take advantage of the global view of SDN architecture to collect four features of each path, which are bandwidth utilization ratio, packet loss rate, transmission latency, and transmission hop, and use ANN to calculate the load condition of each path.
However, for the semi-centralized SDN system, each switch uses just a deep belief network to predict the traffic generated by connected sensing devices; then, centralized SDN uses Deep CNN to make the final prediction. Based on the obtained results, the accuracy with a centralized SDN system is always better than the two other systems (more than 90%). Due to the volatile nature of network traffic in smaller time scales, Lazaris et al. [113]  focused on network traffic prediction over short time scales using several variations of the LSTMs model. To evaluate the performance of their solution, the authors used real-life traffic (CAIDA Anonymized Internet Traces 2016 Dataset)  [114]. The experimental results demonstrate that the LSTM models perform much better than the ARIMA models in all scenarios (e.
Discussion In the above subsection, several ML-based solutions for traffic prediction in SDN have been presented, as summarized in Table  9. Mobile communication faces many challenges when the number of smart devices increases, and hence, the load in the network can lead to network congestion. In this context, ML models can be used to predict the network situation and help the operator to anticipate exceptional resource demand or reduce resources when no longer needed. Traffic prediction is considered a regression problem (i. e. , dataset outputs have continuous values). Therefore, supervised learning can be used for traffic prediction. Based on the literature, DL algorithms, especially LSTM, are widely adopted and can be considered a promising solution for network traffic prediction. However, building an LSTM model does not mean successful results, since several factors can affect the performance of the model, such as the model hyperparameters and the size of the dataset used during the training process.
The 10% of the KDD Cup 1999 dataset was used to evaluate the proposed system. The results demonstrate that, using the selected features, the classifier can better differentiate the attack traffic and produce a low overhead system. Similarly, Li et al. [125]  proposed a two-stage intrusion detection and attack classification system in an SD-IoT network. In the first stage, they selected the relevant features, which were used in the second stage for attack anomaly detection. The experiments on the KDD Cup 1999 dataset show that random forest achieved a higher accuracy with an acceptable complexity time compared to the existing approaches. However, as conventional ML models have difficulties detecting attacks in largescale network environments  [126] , DL models have been used. They can give better performance for intrusion detection, as with network traffic classification and prediction (Sections 3.
1 and 3. 2) without the need to extract the features manually. In this context, Tang et al. [45]  proposed a DNN model for anomaly detection using only six features that can be easily obtained in an SDN environment. The experimental results on the NSL-KDD dataset show that the deep learning approach outperforms the conventional ML models. Then, Tang et al. [127, 128]  proposed, for the first time, the GRU-RNN model for anomaly detection in SDN, which is an extension of their previous work  [45]. This model has the ability to learn the relationship between current and previous events. Using the NSL-KDD dataset, the GRU-RNN model achieved 89% accuracy using the 6 features. Additionally, the experimental results demonstrate that the GRU-RNN model outperforms VanilaRNN, SVM, and DNN. To also improve the detection rate of the attack or the intrusion, some researchers have started to combine DL-based models and conventional or classical ML models.
More specifically, they take advantage of the benefits of the deep features extracted through deep learning, which can be used with conventional models for classification or attack detection. In this context, Elsayed et al. [129]  combined CNN architecture with conventional ML-based models, including SVM, KNN, and RF. Specifically, CNN extracts the deeper representations from the initial features, while the classification task is performed through SVM, KNN, and RF. To evaluate the performance of their approach, the authors used a novel dataset, called InSDN, which is specific to an SDN network  [130] , along with UNSW-NB15, and CIC-IDS2018 datasets. The results demonstrate the potential of CNN for anomaly detection even with a few amount of features (9 features). Also, the combination of CNN and SVM, KNN, and especially the RF algorithm provide higher performance compared to a single CNN.
Unlike the centralized architecture, distributed SDN can deploy multiple IDS for the active detection of attacks. In contrast to the other IDS approaches, Shu et al. [131]  proposed a collaborative IDS based on distributed SDN in VANETs, called CIDS. In other words, CIDS enables all the distributed SDN controllers to collaboratively train a stronger detection model based on the whole network flow. To do so, generative adversarial networks (GAN) have been used, wherein a single discriminator is trained on the Cloud server and several generators are trained on the SDN controllers. Using the KDD99 and NSL-KDD datasets, the evaluation results show that the proposed method achieves better performance as compared to centralized detection methods. Although distributed SDNs are trained on richer data, they are limited in terms of collaboration.
Consequently, the FL started to attract researchers since there is a need for collaboration among the domain. In this context, Thapa et al. [74]  proposed an FL model for the detection and mitigation of ransomware attacks in the healthcare system called FedDICE. Specifically, FedDICE integrates FL in an SDN environment to enable collaborative learning without data exchange. The performance of the FedDICE framework is similar to the performance achieved by centralized learning. Similarly, Qin et al. [132]  used FL for intrusion detection in the SDN environment. They combined FL and binarized neural networks for intrusion detection using programmable network switches (e. g. , the P4 language). Using programmable switches and FL enables the incoming packet to be classified directly in the gateway instead of forwarded to the edge controller or Cloud server.