dataesr/scientific-paragraphs-categorization huggingface dataset:
The ISCX-2017 dataset consists of 51G network traffic metadata that is labeled including 80 features and the full packet payload. The network traffic is provided on protocols, such as HTTP, HTTPS, FTP, SSH, and email. The dataset includes the most common attacks based on the 2016 McAfee report, such as Web-based, Brute force, Denial of Service (DoS), Distributed Denial of Service (DDoS), Infiltration, Heart-bleed, Bot and Scan covered in this dataset  (Monshizadeh et al. , 2021). In this dataset, some packets cannot be correlated to any of the labels provided by the dataset owners. Therefore, they are considered as unknown traffic. The ISCX-2012 dataset was captured in 2012 over one week in an emulated environment. Dataset includes normal and malicious traffic  (Sharafaldin et al. , 2018; Ring et al.
, 2019; Shiravi et al. , 2012). For current paper experiments, the attacks are grouped into three categories: Local to Remote (L2R), Remote to Local (R2L), and Local to Local (L2L). In this dataset, some packets are considered unknown as they cannot be correlated to any of the labels provided by the dataset owners. While the network traffic payload may have different characteristics for every dataset, this study only analyzes the header of the network traffic datasets that consist of similar attributes and protocols. Therefore, mixing datasets has not been an issue as the data points were also close in the feature space during the experiments. The distribution of packets in the selected subsets of the datasets is shown in Table  1. The packets are considered in normal (-2), unknown (0) and attacks (1,.
Data preprocessing Data cleaning, converting the columns to the right types, handling missing values, splitting IP addresses into four fields, vectorizing categorical variables, normalizing the dataset, and changing the labels of attack classes in order to differentiate different attack classes are the processes carried out in this phase. For the normalization, statistical and scaling normalization are used  (Wang et al. , 2009). In order to improve the performance of the algorithms, nominal attributes are transformed into numeric attributes. In addition, the IP address and hexadecimal Medium Access Control (MAC) address of the applied datasets are transformed into separate numeric attributes. Each numeric attribute is normalized using batch mean and standard deviation unless there is an already defined range (e.
Experimental results All the experiments are carried out on a server with Intel ® Xeon ® 16  E5-2623 CPU @3. 0 GHz (4 cores in each processor), 128 GB RAM and 1. 6 TB HDD. The scripts were developed in Python in a Linux environment (Ubuntu 20. 04. 1 LTS) and utilized Scikit-learn and Tensorflow2 library  (Scikit-learn, 2021). In order to prove the proposed architecture superiority to prior art, this section evaluates and compares performance of ADBC against prior art algorithms. For this purpose, the following prior art methods are implemented by authors in the same testing environment and by employing similar datasets that are applied to ADBC. • DBSCAN and SVMonline (with known and unknown traffic) • DBSCAN and lambda VAE (with known and unknown traffic) • DBSCAN and vanilla VAE (with known and unknown traffic) • ADBC (with known and unknown traffic) The known traffic is used in all experiments to verify the model is able to cluster traffic properly.
DBSCAN and SVM 𝑜 In this experiment SVM  feature selection is used with DBSCAN in order to cluster unknown traffic. The known traffic is included in the experiments, but the labels are only used during the evaluation as the applied method is unsupervised. The best selected features for each dataset and ordered by importance are as follows: • MAWILab-2018: IP source 4, IP fragment offset, IP source 3, TCP source port, UDP destination port, and IP source 2. Figs. 4-6 shows the performance evaluation of DBSCAN-SVM  for each dataset. Though, this method is examined for two values of MinPts hyperparameter where the minimum number of packets assigned in each cluster are not less than 5 and 10, only the results for MinPts=10 is presented in this section. From Figs. 4-6, it is obvious that maximum silhouette score achieved by this method depends on the dataset (0.
475 for MAWILab-2018, 0. 81 for ISCX-2017 and 0. 84 for ISCX-2012 dataset). Furthermore, the epsilon hyperparameter varies for each dataset in order to achieve the highest silhouette score (epsilon = 0. 3 for MAWILab-2018, 0. 1 for ISCX-2017 and 0. 20 for ISCX-2012). Hence DBSCAN needs to be tuned based on each dataset. These results reject the method robustness. Furthermore, Figs. 7-12, represent the data points using the six best features selected by SVM. At each time, the data points are projected into a 3D feature space for visualization purposes. As it is shown in these figures, even some clusters are more visible, still due to the high degree of overlapping (due to low silhouette score), distinguishing each cluster is a challenging task. This indicates the data space created by the selected features in this method are not suitable for clustering.
DBSCAN and vanilla VAE This experiment applies vanilla VAE feature extraction with DB-SCAN to cluster unknown traffic. Notice that the known traffic is used without its labels. The unknown traffic (label 0) should not be considered as a class because it cannot be a single cluster. In fact, it is composed of many classes. However, it is represented in the figures using the same color to distinguish it from the rest of the traffic. The        packets assigned in each cluster are not less than 5 and 10, only the results for MinPts=10 is presented in this section. From Figs. 13-15, it is obvious that maximum silhouette score achieved by this method depends on the dataset (0. 67 for MAWILab-2018, 0. 43 for ISCX-2017 and 0. 47 for ISCX-2012 dataset). Furthermore, the epsilon hyperparameter varies for each dataset in order to achieve the highest silhouette score (epsilon = 0.
2 for MAWILab-2018, 0. 3 for ISCX-2017 and 0. 20 for ISCX-2012). Hence DBSCAN needs to be tuned based on each dataset. Similar to previous method, the silhouette score and epsilon value depends on the dataset meaning the model is not robust. Furthermore, as illustrated in Figs. 16-21, though the Vanilla VAE makes the classes look more like clusters, yet the clusters are still close to each other and overlap in this new feature space which will make clustering a hard task.
DBSCAN and Lambda VAE This experiment applies Lambda VAE feature extraction with DB-SCAN to cluster unknown traffic. Notice that known traffic is used in these experiments without its labels. The Lambda VAE was used here in order to encourage the clustering of the data in the latent space because the original VAE does not enforce any clustering in the latent space. In fact, the VAE loss function encourages the latent embeddings to be around the origin which is not very useful since it creates overlapping clusters. Figs. 22-24 show the performance evaluation of DBSCAN-LambdaVAE for each dataset. Though, this method is examined for two values of MinPts hyperparameter where the minimum number of   packets assigned in each cluster are not less than 5 and 10, only the results for MinPts = 10 is presented in this section.
From Figs. 22-24, it is obvious that maximum silhouette score achieved by this method depends on the dataset (0. 38 for MAWILab-2018, 0. 47 for ISCX-2017 and 0. 49 for ISCX-2012 dataset). Furthermore, the epsilon hyperparameter varies for each dataset in order to achieve the highest silhouette score (epsilon = 0. 2 for MAWILab-2018, 0. 3 for ISCX-2017 and 0. 3 for ISCX-2012). Hence DBSCAN needs to be tuned based on each dataset. Similar to previous method, the silhouette score and epsilon value depends on the dataset meaning the model is not robust. In this experiment, the Lambda VAE is supposed to encourage pushing each class far from others in the latent space in order solve     the class overlapping problem. While the lambda VAE slightly reduces the overlapping, it does not solve the problem completely as shown in Figs.
ADBC This method aims to change the data space by creating a more robust distance metric that replaces the Euclidean distance. Therefore, the distance between the data points in the new space will be computed according to a clustering-based distance instead of the usual Euclidean distance. In order to create this distance, the first step is to apply different clustering algorithms to the input data. For instance, this paper applies the same clustering algorithm k-means, with different numbers of clusters. In other words, it apples k-means with several values for hyperparameters to create different clusterings. In the second step, the previous clusterings are used in order to create the Co-Association Matrix as explained in Section 3. The Co-Association Matrix allows to directly deduce the distance matrix which will be used as an input for DBSCAN.
The following explains the steps of ADBC architecture. In general, the ADBC architecture has four main phases: 1. Creating a Co-Association Matrix with applying N different clustering methods to a subset of the input data in order to produce N clustering sets. This step will help us avoid the drawbacks of a single clustering technique applied to an intrusion detection dataset with clusters having very different densities. In fact, the application of a single clustering algorithm is never sufficient nor stable because the result varies a lot with minor changes in the hyperparameters or the input data. In this step, the different clustering algorithms or even the same algorithm with different hyperparameters or initializations are applied. 2. Applying DBSCAN on the distance matrix (deduced from the Co-Association Matrix) representing the new distance between data points in order to detect the attack clusters.
4. The created clusters will be generalized to the whole dataset and analyzed by a security investigator (human or an algorithm) in order to identify malicious clusters. Figs. 31-33 show the distance matrices which are issued from the co-association matrices for the three datasets. These distance matrices are grouped by clusters and reveal that the proposed architecture can detect the known attack classes without using the labels. The similar distance matrices can be obtained with variations in the DBSCAN parameters, which shows the robustness of the final clustering. This is due to the voting mechanism that was used to create the distances it means the more often two data points appear in the same clusters, the smaller the distance between them. In these figures, data points are grouped into attack classes.
Performance evaluation In order to evaluate ADBC performance, a comprehensive examination has been done based on the architecture robustness and for different implementation factors such as hyperparameter setting and diverse datasets. Figs. 34-39 depict the performance of the obtained clustering architecture for each dataset. Though most studies in this field consider only silhouette score for performance evaluation, this paper presents the homogeneity score in addition to silhouette score to evaluate the ADBC performance. If the homogeneity score is high, each cluster will be mainly composed of the same attack class. However, the overall number of clusters will be high which is, in general, a drawback for the security investigator as this means that more investigation time will be needed.
Moreover, this study considers the homogeneity score which ensures that the clusters contain mainly the same data points at each time (this score does not include the unknown traffic as it is a supervised score). As it is illustrated in Figs. 34-39 for the MinPts=10 for the same hyperparameter epsilon(eps = 0. 05), the architecture provides the highest silhouette score 0. 99 for MAWILab-2018, 0. 99 for ISCX-2017 and 0. 98 for ISCX-2012. Similarly, for MinPts=5 for epsilon = 0. 05 the highest silhouette score is achieved for three datasets. In addition, the highest homogeneity score is achieved for all datasets and with same hyperparameter settings. These results conclude that the architecture does not need to be tuned often. Therefore, it can be claimed that one of the main achievements of this architecture is the robustness against different datasets.
• Class 0: Though the output of the model still has an unknown cluster, the packets that are categorized as unknown are much less than original ones. In addition, the approach gives possibility to security investigator to analyze one unknown packet and generalize the analysis to other packets in the cluster. • Class 3, 4 and 5: These clusters represent multipoint attack categories and they are labeled based on the services rather than attack types. Therefore, there is overlapping between some of the clusters for the mentioned attacks. An example of cluster composition for a subset of MAWILab-2018 and hyperparameters: eps = 0. 1 and MinPts = 5 is depicted in Table  3  after applying ADBC on the mentioned dataset. This table depicts the number of packets in each cluster distributed according to various attack categories.
Discussion and future direction As it was described in Section 2, several studies have applied unsupervised techniques to analyze datasets and cluster the datapoints based on their similarities. As it is shown in Table  4 , these studies apply optimized version of DBSCAN or a combination of DBSCAN with various clustering techniques in order to categorize different type of datasets. Though, some of these methods provide good silhouette score, yet the achieved performance considerably depends on the datasets. In addition, hyperparameter settings are changing based on the different datasets. It must be noted that some of the prior art methods are applied for different application than IDS, their employed datasets are dissimilar to network traffic dataset, their testing conditions are distinct and even the presented results lack a thorough analysis of clusters quality.
Therefore, in order to have a valid comparison between ADBC and other methods, similar dataset and testing environment are required, therefore authors implemented some of the prior art methods such as DBSCAN, DBSCAN-SVM  , DBSCAN-VAEs in similar testing environment and on the same datasets. The achieved results proved the ADBC outperforms prior art methods in term of clustering performance and robustness. In Fig. 40 , the first plot shows the classes in the input space, where there are no natural clusters. Therefore, a clustering algorithm like DBSCAN cannot be applied directly to this input space. Hence, the DBSCAN is applied on different feature spaces as described in the previous sections. In the second and third experiments, DBSCAN was applied on a latent space of a VAE and a lambda VAE where the classes look more like clusters.
On the other hand, the point to multipoint attacks provided in the applied dataset are labeled based on services rather than attack categories. Therefore, the clustering results provided in Table  3  demonstrate overlapping between some of the clusters for the mentioned attack categories. In order to avoid such overlapping (in the evaluation phase) another dataset can be applied with distinct distribution of attacks per protocol. Moreover, the complexity and scale of learning algorithms are still questionable and out of scope of this paper and will be investigated by authors in the future work. In addition, the authors aim to automate the cluster analysis part and reduce the human interaction with ML algorithms. • Datasets consist of randomly selected points and do not have similar pattern as network traffic dataset.
• This method is limited to hard clustering and fails if there is recovery between the different clusters. • The silhouette score greatly depends on the dataset where the achieved values are between -0. 593 and 0. 910. • Hyperparameters setting are changing based on datasets. • DBSCAN • Kmeans • Agglomerative  (Turner and Joseph, 2017)  • Snort rule 2. 9. 9. 0 • 0. 81 • 0. 84 • 0. 84 • Datasets are snort rules for UDP, TCP and ICMP. • Experiments are on few samples and concentrate on snort rules. • Results are provided for one dataset without any analysis on hyperparameter settings. • Optimized DBSCAN  (Sun et al. , 2020)  • set1-upc • set2-camus • 0. 77 (eps = 4. 2) • 0. 67 (eps = 4. 8) • Datasets are internet messages. • Datasets are small size (36MB and 518KB). • Hyperparameter epsilon changes for datasets to achieve highest silhouette score.
• IIDBG (Xue-yong et al. , 2010) • KDD Cup 1999 Data • NA • Model is applied for IDS application with network traffic dataset. • The study provides supervised metrics such as detection rate and false negative ratio for performance evaluation. • The study applied clustering technique and discussed about hyperparameter such as epsilon and MinPts setting. • The study did not provide any analysis on clusters and unsupervised metric such as silhouette score. • DB-Kmeans  (Dong et al. , 2019)  • NSL-KDD • NA • Model is applied for IDS application and employed network traffic dataset. • The study provides supervised metrics such as detection rates and false negative ratio for performance evaluation. • The study applied clustering technique but did not provide analysis on clusters and unsupervised metric such as silhouette score.
• Combination of DBSCAN, K-Means++ and SMO algorithms  (Shakya and Makwana, 2017)  • KDD • NA • Study is for IDS application with network traffic dataset. • The study provides supervised metric such as accuracy for performance evaluation. • DBSCAN is used for noise reduction and not for clustering. • DBSCAN  (Sabottke et al. , 2019)  • Web server logs • 0. 99 (eps = 0. 1) • Robustness against different dataset is not evaluated. • The obtained clusters are composed of different types of attacks (mixed clusters) that makes analysis difficult. • The result of packet per cluster analysis cannot be generalized to the entire cluster. • HDBSCAN  (Dockhorn et al. , 2015)  • • V-measures are provided for performance evaluation. • Model is proper for convex shaped clusters rather than density-based clusters.
• Hyperparameter epsilon changes for different datasets to achieve high silhouette score. • Evidence Accumulation Clustering  (Fred and Jain, 2005)  • Random data in a five-dimensional hypercube • Half-rings • Three concentric rings • Cigar • Iris • NA • Method is applied for image analysis. • Metrics such as number of clusters in clustering ensemble is used for the performance evaluation. a ku is the number of clusters used for the test.
Conclusion In the field of machine learning, non-labeled data analysis is one of the well-known challenges. In real-life scenarios, considerable amount of incoming data does not belong to any known category; and for unknown traffic, dividing data into the classes without having information on the nature of the traffic is challenging. Hence, clustering methods are introduced to gain some insight into the structure of the data. However, clustering techniques also have some drawbacks such as overlapping as clusters can appear with different shapes, sizes, and data density; in addition, it is difficult to generalize an algorithm for different dataset with the same hyperparameters tuning. Therefore, to solve the mentioned challenges, a novel and combined unsupervised approach is proposed in this paper.
The architecture utilizes several clustering algorithms in conjunction with co-association matrix and a DBSCAN in order to categorize unknown traffic into various clusters. The model Introduces a new metric that helps to achieve high silhouette score by changing the feature space. The new similarity metric is different from Euclidean distance, where the distance between data points reflects the similarity between packets. The illustrated results in this paper prove that architecture provides very high silhouette score (almost 0. 99) that means clusters (attack classes) are distinct and with minimum overlapping. Therefore, this method helped to resolve the problem of varying densities of clusters and the overlapping problem. Similar analysis has been done for other datasets with different size and divers attacks in order to verify architecture robustness and scalability.
The reference-point coordinates measured by the UAV are still taken as the input of CNN, the false displacement of the target points caused by the UAV will be outputted by the trained CNN according to the characteristic relationship between the reference points and the target points, and the real displacements of the target points measured by the UAV are obtained by subtracting the output value from the uncorrected displacement of the target point. The natural frequency of the model is obtained by OMA from the time-history curves. The specific process is shown in Figure  1 , and the process of correcting displacement measured by a UAV through the CNN is shown in Figure  10. Finally, the tracking effects of DIC and KLT optical-flow method are compared under the fixed-camera and UAV measurements to validate the feasibility of KLT optical-flow method replacing DIC, and the results of fixed camera and UAV are compared under the KLT optical-flow method to validate the reliability of UAV data combined with CNN and KLT optical-flow method.
While this behavior had a negligible effect on bearing displacement demands, it had a significant effect on the floor response spectra. In this respect, Y. -N. Huang et al. (2016)  clearly showed and quantified the correlation between vertical and horizontal components of strong ground motions on the basis of records coming from a database compiled for large-magnitude shallow crustal earthquakes. Along the same vein, that is, the protection of NPPs components against the vertical component of the earthquake,  Fukasawa et al. (2019)  proposed a new 3D complex isolator consisting of rubber bearings, vertical oil dampers, and disc spring units. In addition,  Najafijozani et al. (2020) , proposed other six systems based on adaptive control and consisted of in-parallel configurations of linear and nonlinear springs and dampers.
The total height of the SMR is 40 m; however, to minimize the effects of impact-type loads such as airplane crashes or explosions, it is buried underground for 25 m. The total mass of the SMR amount to 117,700 tons when at the operational level, the pools are full. The main views and finer details can be found in a few documents; see, among others, IAEA (2014). To evaluate the modal properties of the SMR, experimental data were not available  (Amezquita-Sanchez et al. , 2017) ; thus, for analysis purposes, the RC components of the SMR were modeled by means of the FE software SAP2000 with quadrilateral shell elements (CSI, 2022). Since the soil was characterized by small deformations, the base slab was directly connected to the ground by means of fixed restraints. The masses associated with the reactor modules were also included using point masses with rigid links.
Digital Image correlation (DIC) technique A two-dimension DIC technique was used during compressive loading to investigate the onset of macroscopic buckling bands  [13]. A DIC pattern is painted on the specimen surface with a non-periodic texture and without preferred orientation. The use of DIC technique at high temperature requires the use of a high intensity source combined with a fiber optic light guide that is used to illuminate the patterns on the surface of the specimen placed into the thermal chamber. A Grass-hopper® high-speed monochrome camera allows digital images to be recorded at full resolution (1920 × 1200 pixels), and at a rate of one frame-per-second during thermomechanical loading. Ultimately, the Green-Lagrange strain field is derived from the 2D displacement field by means of the VIC-2D correlation software (provided by Correlated Solutions Company).