dataesr/scientific-paragraphs-categorization huggingface dataset:
Thus, this work did not target the problem of data unavailability and drift but only focused on data complexity. In  [10] , the authors suggested a full graph dynamic autoencoder, which consists of modules for graph attention and full connection autoencoders. Damage severity was not explored in this situation; rather, it was solely considered in terms of multi-condition fault identification (i. e. , diagnosis). The method was also used in many examples, including the study's primary dataset. Dealing with automated feature extraction, denoising, imbalanced classification, and data drift were all part of this. However, in this case, only a small number of samples (i. e. , hundreds) were used, and the problem complexity was lowered by dividing it into just two operating modes (i. e. , healthy and unhealthy), even though the total number of data was massive (i.
As a result, this study had several limitations and did not examine the complete diagnosis dataset. Additionally, no work was performed on prognosis issues. Likewise, the problem of data drift and system's dynamics received no attention. Interesting work was performed in  [14]  to provide solutions to both data unavailability and data drift using domain adaptation, generative modeling, and adaptive learning. Complexity was seen as a problem of automatic feature extraction via deep learning, but no additional data complexity and outlier analysis were considered, with the exception of sliding-window overlap sampling, which increases the number of samples while normalization is performed, without further details regarding dataset splitting and the evaluation methodology.
Research Gaps Analysis using these precise criteria uncovered some significant research gaps in the literature on data-driven diagnosis and prognosis of aeronautical bearings, the most significant of which are given below: 1. Most of the discussed related works (i) limited their study to a specific set of subsets and working conditions (e. g. ,  [13] ), (ii) reduced the problem complexity by turning it into the healthy-non-healthy problem only without in-depth multi-class classification of the seven operating modes (e. g. ,  [10] ), or (iii) sub-sampled the dataset by increasing the sliding-window length (e. g. ,  [12] ). This considerably reduces the complexity of the problem in general and also reduces the effectiveness of the model in terms of generalization when it comes to real-world application; 2.
Outline To make these contributions clearly illustrated in this work, this article is divided into four sections. Besides the introduction in Section 1, in Section 2, the description of the dataset, as well as its processing methodology, will be introduced. The description will be oriented towards the necessary information required for the understanding and reproduction of the experiments in the progress of this work. Likewise, some important illustrative examples will be used to understand the main purposes of the processing scheme and the advantages brought to data representation. Section 3 will introduce the proposed CSIDTL approach, its main concepts, and learning rules. In addition, this section is also devoted to describing the application procedures and the main results with enough illustrations and discussions.
Dataset Description and Processing The test rig depicted in Figure  1  was utilized to produce the data used in this study. It consists of a high-speed spindle that rotates a shaft supported by three roller bearings, two accelerometers, and a load cell. The load cell is used to measure the axial force applied to the shaft, while the accelerometers are positioned at two separate points along the shaft, A1 and A2. Additionally, the test rig has a lubrication system distributing oil to the bearings throughout the test. The test rig was developed to monitor system vibration while different measurements are taken under varying operating conditions with variously damaged bearings. Specially designed for high-speed aerospace applications, bearings designated B1, B2, and B3 have varying characteristics in terms of pitch diameter, roller diameter, contact angle, and number of rolling components.
In fact, the introductory paper provides a comprehensive analysis of the behavior of different bearing types and sizes under various working conditions. However, it is worth mentioning that the authors did not study each bearing separately. Instead, these bearings were studied as a single system coupled to a single shaft. This is the reason why the authors carried out all their experiments on the B1 bearing (see first paragraph of Section 3. 1 in  [7] ). The Dynamic Identification and Research Group (DIRG) of Politecnico di Torino provided, from this test bed, a huge dataset with two main subsets (i. e. , 46,592,000 samples × 6 channels and 54,067,200 samples × 6 channels, respectively)  [7]. The first subset, "Variable speed and load", includes tests under different working conditions up to 30,000 rpm and tests with different types and levels of damage and is mainly dedicated to diagnosis studies.
The second one is the "endurance" set, comprising experiments verifying the speed of damage severity under standard operating conditions of load and speed and is dedicated to prognosis studies. Accordingly, six channels with duration T and sampling frequency f s were collected. Each time-domain acceleration record consists of T × f s s samples, while the values of T and f s are given differently for the two investigations performed on the bearings. The data were collected using accelerometric acquisitions at variable rotational speed, radial load, and level of damage, as well as temperature measurements. Specific accelerometer sensitivity values were set in the OR38 signal analyzer, so the files contain acceleration time histories in m/s 2. The accuracy of the OR38 input channels is as follows: phase of ±0.
02 • , amplitude of ±0. 02 dB, frequency of ±0. 005%. The introductory paper  [7]  does not explicitly mention any potential sources of bias or limitations in the dataset. However, it is important to note that the data were collected under specific working conditions and may not be representative of all possible scenarios. Additionally, the analysis techniques used in the paper may have limitations and may not be applicable to all types of data. This work uses both subsets to test the capability of the proposed data engineering scheme associated with the SCIDTL methodology. Accordingly, this section is dedicated to describing the most important features of the dataset and its processing steps while passing through some important illustrative examples.
Variable Speed and Load The variable-speed and -load experiment aimed to study the behavior of bearings with damage of different types and sizes under different operating conditions of rotational speed and load. The test was mainly carried out on the bearing in position B1 (see Figure  1a-c ), which was designed to be easily removed from its support to allow for the artificial crafting of different types of defects, mounting, and the monitoring (i. e. , taking photos) of bearings with damage of different types and sizes during testing. The test involved running the bearings at different speeds and under different loads to collect data on bearing behavior and performance using vibration sensors. Table  1  discusses different experimental conditions in this case. Table  1  demonstrates that there are 17 operational conditions in this scenario; however, only 13 are available in the public dataset files.
Unfortunately, there is no direct and explicit explanation of this matter in the dataset's original introductory publication  [7]. However, according to paragraph 2 on page 265 in  [7] , the differences between the 17 conditions in Table  1  and the 13 conditions publicly available in the dataset may be attributed to the fact that only a restricted number of samples for each condition were used in the study. The remaining samples might have been removed for a variety of reasons, including data quality concerns or a desire to reduce the computational cost of the study. It is assumed that the load voltage was actually measured during the experiment. Its different values are provided in Appendix A, Table  A1  in  [7]. As a result, a list of several operating modes was investigated, including one healthy mode (0A) and six unhealthy/fault ones (1A-6A).
• Synthetic minority oversampling technique (SMOTE) is used to balance The samples number per dataset classes. Finally, it should be noted that the tools described in the following section might be well known in the literature. However, the main contribution of this paper in terms of data processing lies in the use of these tools in the form of different layers (which will be explained in the next sub-sections) in specific order. In addition, the combination of Finally, it should be noted that the tools described in the following section might be well known in the literature. However, the main contribution of this paper in terms of data processing lies in the use of these tools in the form of different layers (which will be explained in the next sub-sections) in specific order. In addition, the combination of these different tools is specifically designed to tackle problems presented by data.
Outlier Removal Being the most important layer in the proposed flowchart of data processing, the outlier removal layer is very effective in improving data quality, especially when aircraft engines are considered, as they are highly dynamic systems susceptible to outliers due to harsh operating conditions. In this case, an outlier detection list that is based on different types of distances and statistical tests (i. e. , Grubbs test; Mahalanobis, Euclidean, and Minkowski distances) is used. Overall, these distances and statistical tests play a vital role in outlier detection by quantifying the dissimilarity or anomaly of data points compared with the majority of the dataset. They provide valuable insights into the presence of outliers, helping to identify potential errors, anomalies, or interesting patterns in the data  [24].
Data Balancing Layer This work follows a minority oversampling technique (SMOTE)  [25]  with k-nearest neighbors, which is a powerful method used in machine learning and data analysis to solve class imbalance problems. The technique offers several advantages. First, it helps solve the problem of data scarcity or lack of representative samples for the minority class by generating synthetic samples that closely resemble real instances. This improves the model's ability to learn patterns and make accurate predictions for the minority class. Second, SMOTE reduces the bias towards the majority class, ensuring a fair representation of both classes in the dataset. This leads to better model performance and avoids the risk of misclassification or underestimation of the minority class. Overall, by effectively improving the training data, SMOTE significantly improves the performance and reliability of machine learning models.
Variable-Speed and -Load Set Regarding the variable-speed and -load test, since data scatters are very numerous, we decided to take some interesting examples of load and speed to be able to observe their effect on data complexity. Accordingly, four conditions, namely, 1, 4, 8, 13, with nominal speed (Hz) and voltage load (mV) of {(100, 0), (100, 900), (300, 500), (500, 500)} corresponding to the qualitative variables of {(minimum speed, minimum load), (minimum speed, maximum load), (average speed, maximum load), (maximum speed, maximum load)} are selected. The reason for selecting only these conditions is that the dataset is massive and contains a wide range of subsets and classes, which cannot be illustrated at once. Therefore, these examples are considered very adequate to obtain a general conclusion on the whole dataset, since they contain information on almost all possible cases of variations in load and working conditions.
In our example, we provide rounded values for load voltage with sensitivity of 0. 499 mV/N (see first paragraph, page 6 in  [7] ). In this context, t-Distributed Stochastic Neighbor Embedding (t-SNE) is used to map high-dimensional data to a low-dimensional space while preserving similarities among data points, with emphasis on distances relative to each other. As a result, the data scatters in Figure  6  are obtained to visualize and explore our dataset. Figure  6a  refers to healthy operating modes, low distortions, and less harsh working conditions. There is minimum speed and null load, and the bearings are completely healthy. In this case, it can be observed that samples of different data classes have some pattern agglomeration of different operating modes (e. g. , class 4 and class 2) that can be easily distinguished even with a simple linear model.
Endurance Set Concerning the endurance test, the experiments show regression in the long-term damage of bearing 4A. This means that the data are highly correlated and the propagation of damage is regressive and slow. Therefore, as the experiment is related to So far, presentation and argumentation are based on comparing dataset scatters of raw data and prepared data. However, to obtain a clear conclusion about data characteristics beneficial for the next stages of adaptive deep learning, another important note in this case can be mentioned. We can observe, in Figure  6e-h , that the harsher the conditions are (i. e. , greater speed and load), the more they lead to numerous distortions. This further complicates the process of understanding and distinguishing different kinds of patterns, increasing the level of cardinality, which definitely increases the complexity of the deep learning process.
2) The selection criteria in this case is a user-predefined classification accuracy defining the best learners. Selected parameters of (n) models are stored incrementally and all used at all rounds. (3) The selected parameters follow a collaborative average aggregation for transfer learning in a sort of parameter initialization for fine-tuning the upcoming models in the next rounds. Step 1: A set of homogeneous learning models, namely, LSTM networks (refer to  [26]  for details on the mathematical background of LSTM networks), are trained in a loop on the different subsets (13 subsets related to operating conditions) of the variable-speed and -load dataset. This means that each LSTM network is trained for the classification process of 7 different operating modes. In the first round, k = 0, the LSTM networks' learning parameters are randomly initialized from specific probability distribution P with mean µ and standard deviation δ as in  (1).
Application and Result Discussion In this work, we launched two experiments on two subsets of the dataset using a laptop with a four-core i7 microprocessor, 16 GB RAM, and 12 MB cache memory. In the first experiment of "variable speed and load", the CSIDTL algorithm was run for five particular rounds in an amount of time of 5. 9170 h. Since the process is very slow, there is a small chance of involving random search algorithms and the grid search mechanism for hyperparameter tuning. Therefore, the LSTM network parameters were manually tuned based on an error-and-trial basis instead. Three-fold cross-validation was involved in studying the performance of the proposed approach, while well-known metrics, such as Accuracy, Recall, Precision, and F1 score, were used to evaluate the performance of the training process.
Variable-Speed and -Load Experiment Accordingly, Figure  9  is introduced to showcase the obtained global results of the CSIDTL approach compared with the LSTM network, while Table  3  is used to indicate in which rounds the best learners were obtained under each condition. In this case, the classification performance metrics were improved by about 12. 15% compared with traditional LSTM. This means that the accuracy was about 93. 63% for the entire variable-speed and -load dataset. This is considered an excellent improvement in the field. More specifically, the LSTM network is considered a robust generalizer only when data are processed under less harsh conditions, as it is seen for conditions 3, 6, and 8 in Figure  9a. Contrariwise, the CSIDTL showed its performance on the entire subset. This means that the adopted approach of selecting and aggregating the best learners and transferring them across conditions better helps to overcome the lack of representative data and not only benefits the data drift mechanism of LSTM.
Instead, the averaged results are showcased in Figure  10. Accordingly, after the learning process, which took about 0. 1754 h, the learning performance was improved by about 10. 87%, reaching about 95. 65% accuracy, which is considered a great achievement for a very complex dataset with a higher level of complexity and drift. experiment). Similarly, in this case, there were 3 × 3 confusion matrices, which are difficult to be illustrated in this case. Instead, the averaged results are showcased in Figure  10. Accordingly, after the learning process, which took about 0. 1754 h, the learning performance was improved by about 10. 87%, reaching about 95. 65% accuracy, which is considered a great achievement for a very complex dataset with a higher level of complexity and drift.
General Discussion In general terms, the problem addressed in the case of the two experiments shows the following: 1. The need to transfer learning across conditions to improve classification performance in terms of the generalization of the learning model (Figure  9  and Table  3 ); 2. CSDTL better expresses the relationships between the number of rounds required for training and data complexity to overcome data drift problems. This is highlighted by the more rounds needed for further complex conditions, as shown in Table  3 ; 3. The training process showcases the effectiveness of LSTM by retaining adaptive learning, especially for less complex datasets. This is why it needs to be boosted by transfer learning and collaborative aggregation; 4. The best results on the endurance set were obtained quickly compared with the variable-speed and -load experiment.
Comparison with Previous State-of-the-Art Works As mentioned earlier in the section on research gaps, previous state-of-the-art works transformed datasets into less complex feature spaces by performing sub-sampling, selecting a few sets to illustrate the performance of their model, or merging different sub-classes of unhealthy modes to reduce the number of classes, thereby reducing the complexity of the problem to be solved. While doing so, most of the experiments discussed a simple random splitting of the dataset, while the prognosis case was not considered the main goal of their studies. As a result, these works clearly achieved great accuracy in this case. However, as far as generalization is concerned, their models lack generalization at some point compared with this study. In fact, this study is conducted on a way-too-massive feature space, with multi-class classification problems (i.
Acknowledgments: The authors would like to thank the  Dynamic Research and Identification Group (DIRG) of Politecnico di Torino  and the authors of the introductory article ( Daga ,  A. P. ;  Fasana, A. ;  Marchesiello ,  S. ;  Garibaldi , L. ) for making their dataset publicly available, which proved very important and useful for conducting the proposed CSIDTL experiments.
Introduction The accurate estimation of the spatial distributions of animal populations is important for understanding patterns of resource use and demographic change, as well as for informing biodiversity conservation and management  (Hays et al. , 2019). The at-sea distributions of many species of marine megafauna have in recent decades been revealed using data from animal-borne tracking devices  (Bernard et al. , 2021; Hussey et al. , 2015). To ensure population-level inferences are robust and spatial management is properly targeted, it is vital to consider how representative is a tracking dataset of the movements of the entire population across both space and time  (Shimada et al. , 2020). In the marine realm, tracking data have been used widely to inform conservation and management  (Davies et al.
, 2020; Evans et al. , 2021). Here, we analyzed movement tracks from 23 seabird species to investigate the importance of annual variability in space use for areabased conservation. We analyzed the distributions of each species during chick-rearing, as this is the time of year when seabirds are most spatially restricted and therefore when area-based management measures can be particularly effective  (Oppel et al. , 2018). Indeed, tracking data collected during chick-rearing are often used to identify priority areas for conservation  (Dias et al. , 2018; Handley et al. , 2021; Heerah et al. , 2019). To estimate the average degree of consistency across years for each species, we quantified the spatial similarity between distributions from different years, and explored whether taxonomy, type of foraging habitat, and latitude could explain inter-specific variation therein.
Study species and data assembly We compiled Global Positioning System (GPS) tracking data using 1 Present address. several selection criteria to ensure comparability across species and sufficient sample sizes to allow for rigorous testing of sampling effects (Table  1 ). We solicited datasets meeting the following criteria: at least four years of GPS data with a minimum of 10 birds tracked in each year, where all birds were tracked from the same breeding colony and during the chick-rearing stage (alias 'chick-provisioning'). In total, we collated tracking data for 23 species, representing seven of 14 families, and four of seven orders of seabirds. Collated tracking data came from 3 to 6 different breeding years for each colony, and sampled years were 1-17 years apart (median difference 3 years; Table  1 ).
For two species, Australasian gannet (Morus serrator) and little penguin (Eudyptula minor), we had sufficient data from two colonies; for all other species, we analyzed data from a single colony (Table  1 ). When producing population distributions (see Population distribution estimation section), we treated each species-colony dataset separately, and then took the mean across colonies when comparing species-level metrics (e. g. , annual overlap). Although all datasets were from breeding adults during chickrearing, the relative timing in terms of the age of the brood (i. e. , whether adults were brooding chicks or not) differed in some cases between species (see Table  S2  for breeding-stage coverage). Nevertheless, for 104 out of 106 annual datasets (98 %), tracking was initiated during the first half of the chick-rearing period.
Tracking data standardization We cleaned and filtered tracking datasets for each species and study colony to improve comparability. First, we applied speed filters, with thresholds set to include only biologically realistic travel speeds (Table  S1 ;  Adams and Flora, 2010; Baylis et al. , 2019). Then, using the R package track2KBA  (Beal et al. , 2021) , we split tracks from individual birds into discrete foraging trips, which we defined as periods of a minimum duration spent outside a spatial buffer around the breeding.