dataesr/scientific-paragraphs-categorization huggingface dataset:
Datasets Several datasets have been proposed to evaluate the performances of intrusion detection tools. DARPA98  [8]  was one of the first datasets provided to the academic community. Several datasets like KDD99 or NSLKDD  [15]  were then derived from this first dataset. Even though they are still widely used by the research community, these datasets have been heavily criticized  [16, 5]  and are generally considered obsolete. In 2015, the UNSW-NB15 dataset  [10]  was proposed to offer modern traffic to evaluate NIDSes. This dataset is not well fitted for anomalybased intrusion detection as the experiment duration is only about 30 hours, and there is no period of time free of attacks. In 2018, the Canadian Institue for Cybersecurity (CIC) provided the CICIDS2017 dataset  [14]  and, together with the Communications Security Establishment, the CSE-CIC-IDS2018  [3].
CICIDS2017 uses a network architecture with machines using several common Operating Systems (OS), namely GNU/Linux, macOS and Windows, along with a firewall, switches and routers. The traffic is emulated through a testbed architecture. This architecture is divided into a victim network with four machines and an attacker network with fifteen machines. The traffic was collected on work hours during five days, from Monday to Friday. Only the first day of the week is free from attack. During the four remaining days, a large variety of attacks was conducted. The attacks in the datasets are brute force attacks (FTP and SSH), Web attacks like XSS and SQL injection, Deny of Service (DoS) attacks and its distributed version (DDoS), port scan, botnet communications and infiltration. The CSE-CIC-IDS2018 includes the same attacks, but the network architecture is much larger and more complex.
Machine learning use on CICDS2017 Most papers that use these datasets rely on machine learning models to learn and detect attacks. In that case, the datasets are generally split in two: one part is used for learning the model (called the "train set"), and the other part is used for evaluation (called the "test set"). The popular models  [7] ,  [11] ,  [9]  for these datasets include decision trees, k nearest neighbors, naive Bayes classifier, Random Forest  [1] , SVM  [2] , and multilayer perceptron  [6]. These methods are used in a supervised learning setting, where the train set is labelled and contains both benign and malicious traffic. For example, Maseet et al. [9]  obtain very high performances on CICIDS2017: out of the seven experimented supervised methods, five of them have an F1-Score, a recall and a precision higher than 0.
Previous criticism on CICIDS2017 In 2021, Engelen et al. [4]  revealed several issues they found in the CICIDS2017 intrusion detection dataset. They found several flaws in the CICFlowMeter tool and that some attacks in the dataset were not well executed and thus ineffective. About the first issue, CICFlowMeter wrongly splits TCP connections because of a wrong implementation of the TCP connection termination. This phenomenon has two consequences. The first one is to create a lot of erroneous network flows since it splits a unique network connection into multiple ones. The second consequence is that the direction of the network flow description can be inverted. In that case, all the forward and backward data are swapped, including the IP addresses, which is damageable when it comes to labelling the dataset since the network flows are labelled based on their source and destination IP addresses.
flow descriptions. 3 Errors in the CICIDS2017 dataset and the CICFlowMeter tool, and their fixes Pursuing the work of Engelen et al. , we found four different issues in the CI-CIDS2017 dataset: a case where CICFlowMeter failed to properly create correct flow descriptions, incoherent timestamps, some duplication in the network captures, and an attack that is omitted from the labels. The first two issues have consequences on the network flow descriptions and lead to an inversion of the source and the destination of the network flow descriptions that may impact labels. The third issue has only an impact on the network flow descriptions. The last one has an impact on the labels directly. To explain why the first two issues may impact the labels, we must explain the labelling process we used. It must be noted that we do not have insights into how the authors of CICIDS2017 labelled those network flow descriptions after they generated the network flow descriptions from the network capture with the CICFlowMeter tool.
However, Engelen et al. provide an automated script to label the network flows as attacks using their source IP address, destination IP address and timestamp. Indeed, the documentation of the CICIDS2017 dataset provides the time periods and the IP addresses concerned by the attacks. We used the same process to label the network flows. As this process takes into account the source and destination, an inversion of those IP addresses may lead to errors in the labels of the network flows. Those four issues are presented in the next subsections. The fixed version is available on the repository https://gitlab. inria. fr/mlanvin/crisis2022.
CICFlowMeter issue with misordered packets CICFlowMeter is a tool that extracts network flow descriptions from pcap files that contain network captures. The network flow descriptions generated by CI-CFlowMeter are bidirectional and distinguish a source (the machine that initiates the communication) and a destination. As a reminder, the initial TCP handshake consists in exchanging three messages (SYN from source to destination, SYN-ACK from destination to source, and ACK from source to destination) to establish a connection. We identified a remaining flaw in CICFlowMeter that occurs when pcap files are not sorted by timestamps, as it happens in the original dataset of CICIDS2017. In that case, the tool reads the network packets in the order of the network capture files, but the SYN-ACK packet can sometimes be stored before the SYN packet in the pcap file, even though, according to the timestamp, the SYN packet did occur before the SYN-ACK one.
Dealing with data duplication Observing the network captures, we found many duplicated packets in the data. Listing 3 contains an example of such duplicated packets. We can see the repetition of the SYN and RST packets: the time interval between two identical packets is only a few microseconds. Besides, their content is the same. We cannot be sure of the cause of that phenomenon as we do not have enough detailed information on the network capture. As the time interval between two identical packets is very small and as UDP and ICMP packets are duplicated, we can rule out the hypothesis that this behaviour is normal due to the TCP retransmission mechanism. For now, our main hypothesis is that the port mirroring on the main switch of the CICIDS2017 testbed was not configured correctly. We did not analyze the network capture entirely, but we only saw duplicated packets between the testbed's internal hosts.
That could be explained by the fact that all the ports of the switch connected to internal hosts are configured to mirror incoming and outcoming packets to the mirror port. That hypothesis is reinforced by the fact that broadcast packets to the internal subnetwork are duplicated 13 times. We corrected this issue with the tool editcap 8 that can find and remove duplicated packets within a given time window. Using this tool with a time window of 500µs, we measured how many packets were duplicated during the whole week of the CICIDS2017 dataset. The Table  5  reports the number of duplicated packets per day. On average, more than 497000 packets are duplicated per day, representing 4. 5% of the packets per day. The duplication modifies the network flow description that is extracted by the tool CICFlowMeter.
So, for the dataset without duplication, we use the patterns "1 forward -1 backward", "1 forward -0 backward" and "2 forward -2 backward" and "2 forward -0 backward". When taking duplicated packets into account (left part of the table), the expected pattern "1 forward -1 backward" was only the third most frequent pattern. Indeed, the first two patterns, "2 forward -0 backward" and "2 forward -2 backward" are the duplicated equivalent of the expected pattern ("1 forward -1 backward" and "1 forward -0 backward") but seen twice, i. e. , with two emitted and/or received packets instead of just one. This is the consequence of the duplication problem described in Subsection 3. 3. In addition to the duplication phenomenon, sometimes timestamps are shifted between the two observations of the same network flow.
When these two effects combine, it can produce some unexpected behaviours where a simple SYN and RST connection is duplicated into SYN -RST -SYN -RST packets, and then transformed into SYN -RST -RST -SYN packets due to timestamp errors. In this case, there are one emitted packet (a SYN packet) and two received packets (two RST packets), so a "1 forward -2 backward" pattern. The last SYN packet triggers the creation of a new flow having the pattern "1 forward -0 backward". So, for the dataset with duplication, we use the same patterns as for the dataset with duplication (that are still valid because not all packets are duplicated), and we add the "2 forward -2 backward", "2 forward -0 backward", "1 forward -2 backward" patterns. Besides, we manually exclude the network flows associated with legitimate protocols (DNS, LDAP, NTP, and NETBIOS-NS) that are present in the network traffic.
Assessment of the consequences on intrusion detection models performances Finding and fixing these issues in CICIDS2017 is a great opportunity to examine the current experimental evaluation performed on this dataset and assess its consequences on the performances. More specifically, we would like to answer the following research questions: -Q1: How does the pcap ordering fix affect the performances of these models? -Q2: How does the correct labelling of the port scan affect the performances of these models? -Q3: How does the flow deduplication affect the performances of these models? -Q4: How could previous articles obtain very high performances with machine learning algorithms in the presence of these issues? In this section, we first describe our experimental evaluation protocol, evaluate several supervised classification models, namely decision trees, random forest, naive Bayes and support-vector machine classifiers, and finally answer the four research questions we defined.
Experimental evaluation protocol To answer the research questions raised previously, we build several train sets and test sets based on the network captures files with different corrections to assess their effect. The corrections include the reordering of the pcap file before applying CICFlowMeter (denoted as R for the rest of the article). If the pcap file isn't reordered, then the labels provided by  [4]  are used. Our second correction is the addition of the new port scan labels denoted as P. We will also assess the impact of the duplicated network packets, and we denote the presence of duplicated packets by D. As we described in section 3. 1, the labelling script proposed by  [4]  sets three kinds of labels: "benign", "malicious" and "attempted". To simplify the experiments and the analysis of their results, we decided to label these attempts as benign.
We also ran our experiments by labelling them as attacks: the differences were slight and did not impact our conclusions. The train and test set configurations are detailed in Table  7. We distinguish three sets of experiments. The goal of the first set (RD and RPD) is to observe the effect of the reordering of the pcap file by comparing with D and PD and answer Q1. The second set (RD and RPD) is built to assess the impact of the port scan addition on the labels provided by Engelen et al. and answer Q2. The last set of experiments (R and RP) consists in assessing the removal of the duplicated packets in the network capture and answering question Q3. It must be noted that we think the correct version of the dataset is the one noted RP, i. e. with the port scan correctly labelled, the reordering of the packets and without duplication.
We evaluated four supervised models: a naive Bayes classifier, a supportvector machine (SVM), a decision tree and a random forest. We chose these supervised models since they obtained the best performances for intrusion detection given the survey  [11]. A benchmark on CICIDS2017  [9]  also highlighted the good performances of the decision tree and naive Bayes models on the dataset. We used the scikit-learn library implementation for all these models. We used standard configuration for the different models except for tree-based models, for which we limited their maximum depth to 15 to prevent overfitting. We adopted a particular strategy for splitting the dataset into train and test sets to conduct a fair study. Indeed, we did not want to put mislabelled flows in   7. Experimental configurations depending on capture and labelling corrections the train set.
Otherwise, if the train set were mislabelled, the supervised methods would certainly have poor performances on the test set. To train the models on consistent data, the train set only includes correctly labelled flows but not any flow that Engelen et al. classified as "attempts". More precisely, we first collected about 65% of each attack, except for the newly discovered port scan attack. This means that the attack types (except the new port scan attack) are present in the training data with about the same proportion as in the dataset. Then, we added as many benign flows as to have a balanced dataset with as many benign examples as malicious examples. This dataset depends on the applied correction, so there are several train sets. However, each one of the three sets of experiments shares a common train set.
1%) that we decided not to include them on the charts for the sake of brevity. Q2: How does the correct labelling of the port scan affect the performances of these models? This experiment relies on train and test sets RD (no port scan label) and RPD (with port scan label). We can consider two cases, depending on the labels of the considered dataset used as the ground truth for computing the metrics. With RD labels, the port scan attack flows that are detected are counted as false positives because the port scan is not labelled (even if the port scan is really present in the network data). With RPD labels, the port scan attack flows that are detected are counted as true positives because the port scan is labelled. Once the port scan attack is correctly labelled, we observe in the Figure  1  that the tested supervised models lose, on average, about 20% of recall.
Therefore the backward number of packets per second is doubled as well as the number of packets per second. For the two other features, the Inter-Arrival Time (IAT) is modified since the duplication makes duplicated packets very close in time. This reduces a lot the minimum and the mean IAT values. So, the duplication of the packets disturbs significantly the flows extracted by CICFlowMeter. So, with duplication, the models do not detect the port scan attack correctly because it mostly consists of flows with duplicated packets that do not match the behaviours learned on the other correctly labelled port scan, which do not have duplicated packets. Without duplication, the newly labelled scan port matches the learned attack behaviour. Q4: How could previous articles obtain very high performances with machine learning algorithms in presence of these issues? Previous experiments use the same scenario as our dataset RD.
As we can see, we can achieve very high recalls with classic models without fine-tuning. Besides, except for the Naive Bayes model, all of them have a relatively small FPR. As we can see on Figure  2 , part of these false positives are related to the newly labelled scan attack. For this reason, results such as those obtained by  [9]  look like overfitting. Overfitting is also one of the conclusions of the survey  [7]  on intrusion detection models used on CSE-CIC-IDS2018. For the recall, we observe that the results on the correct dataset RP are as good as on the original dataset RD, while the recall drops when we label correctly the missing port scan (dataset RPD). We also observe that the FPR raises for the dataset R compared to RD. It seems that two issues we found (the missing port scan and the duplication of packets) offset each other and allow the ML models to obtain good results on the original dataset RD.
Conclusion CICIDS2017 is often used to evaluate the performances of NIDS. However, it has several flaws in both its traffic captures and its labelling. First, the tool CICFlowMeter misbehaves when packets are misordered in the traffic capture, which leads to source/destination inversion and wrong feature values in the network flows. Second, the traffic capture contains incoherent timestamps. Third, about 5% of the packets are duplicated, modifying the network characteristics of the flows used for detection. Finally, the infiltration step contains a port scan attack that was not labelled as such either in the original authors' CSV files or in the revision of the labels provided by  [4]. Once the dataset was fixed and relabelled according to these modifications, we measured the intrusion detection performances of several supervised models that are often used.
The newly labelled port scan attack is not entirely detected by the models, leading to a loss of recall of about 20%. However, we get back to good performances on the new port scan attack by removing duplicated packets. We allow us to conclude that previous work could not obtain a precision close to 100% without overfitting. Finally, the security community could benefit from merging our corrections with the features calculations and protocol handling of  [13]. Through this article, we highlighted the importance of the quality of the network intrusion detection datasets to evaluate the NIDS accurately. A good quality comes from a clean labelling process, and an accurate network captures management to prevent packet duplication, for instance. Without these prerequisites, the evaluation is distorted, and the models learnt on the dataset may be unfit for realistic network characteristics.
Subsequently, the key components of the resulting actuation are extracted and filtered in the frequency domain. These are eventually used as input to a Central Pattern Generator (CPG)-based model that takes over the generation of rhythmic patterns at the motor level. The proposed methodology is evaluated on a twomodule soft arm. Results show that the first algorithmic component (ProMP) provides an immediate estimation of the requested actuation by avoiding timeconsuming training, while the latter (CPG) further simplifies the execution by allowing its control through a low-dimensional parameterization. Altogether, these results open new avenues for the rapid acquisition of periodic movements in soft robots, and their compression into CPG parameters for long-term storage and execution.
Focusing on data-driven methods, in Gijsberts and Metta (2013) a novel approach is presented, called Incremental Sparse Spectrum Gaussian Process Regression (I-SSGPR), lying within the general category of model learning algorithms. The authors of this work capitalised on the exhaustively studied Gaussian Process Regression aiming at designing a method that cope with unstructured and non-stationary environments where adaptability to changing conditions is required. At the same time, low computational complexity is achieved, while automated hyperparameter optimization is provided. Another interesting feature of this approach is the capability to perform both offline training using an existing dataset, as well as online updates when new data are available. In the frames of this work, the I-SSGPR method provides an approximation of the inverse kinematics of the robot.
Particularly, two separate modules based on I-SSGPR have been implemented. The first one provides a feasible target for the robot by computing the z coordinate when a {x, y}pair is received, as explained in Section 3. 2. The second module is used for the mapping of only the conditioning points (derived from the path segmentation algorithm -Section 3. 4) from the task to the actuation-space. Hence, it receives the desired position (x, y, z) of the robot's tip, whose z component has been computed by the former I-SSGPR module, and outputs the (six) motors' angular positions. Both modules are initially trained offline using the dataset derived during the demonstration generation process described in Section 3. 1. Later on, online updates are performed during trajectory execution by the robot, as depicted in Figure  3.
Even though four primitives are used for approximating the position and velocity at each conditioning point, simplex method might result in zero velocity for some of them, that means no contribution. In Figure  5 , an example of ProMP composition at each conditioning point of a demonstrated trajectory is depicted. Subsequently, the skill transfer process is performed and all parameters derived for the composition of primitives in the taskspace, such as d (i) n , are transferred unchanged to the actuation space. At the same time, the MP library -where the mapping at the primitive level between the two spaces is stored (Section 3. 1)provides the corresponding primitives in the actuation-space, while the I-SSGPR module (Section 3. 3) interprets all conditioning points into motor commands.
Computation of CPG-parameters This module follows the ProMP-based part of the methodology, and its goal is to detect the intrinsic periodicity of the actuation signals along with their features, and subsequently to decompose it into discrete single-frequency oscillations. Here, the first step is the isolation of the oscillatory part from the resulting signal which contains also the actuation performing the transition from the border of the primitive's grid to the edge of the requested trajectory (see Section 3. 2 and Figure  4B ). The periodic signal can be easily obtained since the timestamps of the conditioning points are already known. Subsequently, the simplest way to compute a periodic signal's components is by applying Fast Fourier Transformation (FFT)  (Cooley and Tukey, 1965). Through this process the original actuation is filtered since only the dominant frequency is kept, and eventually the computed features -offset, amplitude and phase bias -for each motor are promoted to the next block, where a CPG-based reconstruction of the actuation takes place.
Learning the ProMP library and the I-SSGPR modules Initially, directed demonstrations are generated by the robot while a magnetic tracker attached on the robot's tip was capturing not only the generated paths in the task-space, but also the sequence of actions. Then, the demonstrations recorded in both task and actuation spaces are grouped into classes and trained to form the MP library following the process described in Section 3. 1. Additionally, the data collected during the aforementioned process -the robot tip's positions along with the corresponding actuations -are exploited to form the dataset D pod , which is used to offline train the two I-SSGPR modules as described in Section 3. 3. In the frames of the present work, ten trigonometric basis functions are used to construct each I-SSGPR module.