dataesr/scientific-paragraphs-categorization huggingface dataset:
Transcription-factor binding sites (TFBS) enrichment analysis We used the HOMER software suite version 4. 11  [75]  to identify enriched TFBS in sets of regulatory regions (findMotifsGenome. pl script with default parameters, with "-h" to conduct hypergeometric tests and "-size given" to only search for motifs within regulatory regions). We used the Damaraland mole-rat genome as background to search for enriched TFBS in the regulatory elements with an activity shift on the ancestral branch. We retain the TOP 10 enriched TF for each element set (all BH-corrected p-values <0. 05). We conducted hypergeometric tests to identify significant TFBS-GO term associations amongst the top 10 TFs and top 100 GO enriched in each element set. For each GO-TFBS pair, we counted (i) the number of regulatory elements both containing the TFBS and associated with the GO, and (ii) computed the enrichment ratio compared to the expected number of overlaps.
De novo repeat annotation in mole-rat species We constructed de novo repeat libraries for naked mole-rat and Damaraland mole-rat using RepeatModeler version 2. 0. 2a [76]  with default parameters. For guinea pig and mouse, we downloaded pre-computed de novo repeat libraries from the Dfam database  [77]. We next ran RepeatMasker version 4. 1. 2-p1 to annotate the location of repeats in mole-rat, guinea pig and mouse genomes, in sensitive mode (-s) and skipping bacterial insertion check (-no_is), with rmblastn version 2. 10. 0+. The average 2-parameters Kimura distance of TE sequences to their consensus were computed using the calcDivergenceFromAlign. pl script from RepeatMasker, which corrects for elevated mutation rates in CpG loci. Finally, landscape plots were drawn using the createRepeatLandscape.
Overlaps between regulatory elements and repeats We used bedtools version v2. 29. 2  [78] , across "orthologous", "non-alignable" and "genome" elements sets. Specifically, we used bedtools intersect to compute overlaps between elements and repeats; and bedtools jaccard to obtain the corresponding jaccard index (ratio of the intersection to the union of the datasets, in number of bases). "Genome" sets were constructed from n=10,000 random permutations of the corresponding "orthologous" or "nonalignable" set, on the whole genome. Following the approach from the jaccard test of the GenomTriCorr R package  [79] , we tested for significant differences in overlaps with repeats across sets. Here, for comparisons with "genome" elements, the n=10,000 permutation-based jaccard indexes form the null distribution to which we compare the jaccard obtained on the corresponding "orthologous" (or "non-alignable") set.
Enrichment of TFBS in repeats We identified enriched TFBS in the set of non-alignable enhancers of each mole-rat using the HOMER software (see "Transcription-factor binding sites (TFBS) enrichment analysis"). We again used a permutation of intervals approach to identify TFBS significantly found fully included in a repeat more often than expected by chance. The null distribution was obtained from n=100 random permutations of TFBS within non-alignable enhancers. In mole-rats, we tested for significant association between enriched repeats and the TOP 10 enriched TFBS. In the outgroups, we tested only for significant association between HNF4 and RAR TFBS and enriched repeats. Finally, we used GREAT (see "Gene ontologies, pathways and genes enrichment tests" for details) to test for functional enrichment for enhancers with the co-occurring RAR TFBS and Sine/Alu repeats and HNF4 TFBS and Sine/ID repeats.
ACKNOWLEDGEMENTS We thank the  QMUL  and  CRUK-CI Genomics cores ,  David Gaynor  and  Katy Goddard  ( Kalahari Research Trust ), and  Nigel Bennet  ( University of Pretoria ) for technical assistance;  Pierre Vincens  ( IBENS, Paris ),  Bronwen L Aken  ( European Bioinformatics Institute ) and  Queen Mary's Apocrita HPC facility , supported by  QMUL Research-IT , for the coordination of computing resources; and  Alexander de Mendoza  (QMUL) and  Masa Roller  ( European Bioinformatics Institute ) for critical comments on the manuscript.
FUNDING This project has been funded by the  British Heart Foundation  (Fellowship FS/18/39/33684 to D. V. ), the  Royal Society  ( RGS\R2\202330  to D. V. ), the  European Research Council (ERC)  under the European Union ! s  Horizon 2020 research and innovation programme  (grant agreement No  851360  to C. B. ),  Cancer Research UK  (travel grants  C46232/A15517  and  22761  to D. V. ), and the  European Molecular Biology Organisation  (Short-term Fellowship to D. V. ). E. P. is supported by a  Newton International Fellowship  from the  Royal Society  ( NIF\R1\222125 ).
Code and data availability The ChIP-sequencing data reported here has been submitted to Array Express (E-MTAB-15922). All original code and datasets generated in this study have been deposited in Zenodo (https://doi. org/10. 5281/zenodo. 7442105). Briefly, these include all scripts, inputs and environments used to: (i) predict regulatory elements from histone mark peaks, (ii) identify orthologous elements across the 4 rodents, (iii) compute normalized H3k27ac read densities for orthologous elements (iv) identify shifted elements in mole-rats with phylogenetic modeling and (v) conduct gene ontology enrichment tests for mole-rats regulatory elements. All included supplemental datasets are listed in the Supplemental Material.
Instead of relying on address-centered division, the increased focus is on the dynamic slice allocation with the aid of Machine Learning tools, where resources for the slices are allocated depending upon the specific context or network conditions in general. Such tools are trained with large datasets representing the network conditions and correct sets of decisions and are later used for making predictions on the incoming sets of requests. For instance, based on the Packet Delivery Ratio (PDR) requirements, a suitable slice is allocated to the device based on the ML predictive analysis framework. Although Machine Learning serves as the right tool for making predictive analyses on slice types, however, bias control remains an important consideration. In networks, objects demanding a singular slice category likely render overload in a certain area of the network while leaving poor utilization of other more optimized resources available.
Across the various uses of this novel deep learning unsupervised scheme including the autocorrelation for anomaly detection or time series predictive analysis. Although the spectrum is highly diverse including the stock exchange predictive analysis or indoor localization  [25] , etc. this method can directly be employed for the current problem under review. Article  [26]  proposes a deep learning network slicing based on the dataset using the data generated for a week where 7-8 different devices send communication requests and are allocated network slices based on the predictive settings using ANN. The study further makes use of a master slice to avoid network disbalance and congestion of slices. However, the consideration of network optimization has been ignored. The slices having lower connection requests can be configured for the devices biased towards a single type, thus the overall network conditions can be optimized.
A. Dataset The dataset employed in this study has been taken from  [27]  and offers a comprehensive insight into the QoS within cellular networks, encompassing LTE and 5G technologies, across a wide array of use cases. The dataset comprises several crucial input features and an output feature that collectively shed light on network performance, reliability, and latency characteristics across diverse scenarios. These input features encompass the type of use case or application, LTE/5G User Equipment (UE) categories, supported cellular technologies, days of the week, times of data collection, and the differentiation between Guaranteed Bit Rate (GBR) and non-GBR services. Moreover, the dataset captures essential metrics for assessing network reliability, such as Packet Loss Rate (PLR), as well as latency metrics, notably the Packet Delay Budget (PDB), specifying acceptable latency thresholds in milliseconds.
B. Classification Framework The classification of network frames is based on the service requirement requests that involve the use of the QoS class, Packet delay budget, and PLR. The models take the data as the input at a given time frame and render two parallel operations: future predictive analysis to predict upcoming network conditions and predictive load on slices, and the classification of the incoming request into the desired slice class. The sampling rate of the rata is recorded at approximately 338 requests per second, and the LSTM model has been trained to predict the upcoming 10,000 requests which constitute around 30 seconds of data. The slice classification has been based on the similar norms presented in this dataset and reference article  [26] , however, the use of the master slice has been discarded and all routing is based on the network conditions and load preferences.
The system level and data parameters adopted in this study are provided in Table  2. The architecture of the LSTM model employed in this study is designed for predictive purposes, specifically to forecast the future requirements of users for a particular network slice (URLLC, mMTC, eMBB) based on historical time series data. The model begins by preparing the training and testing datasets, to specify the number of past and future time steps considered in the prediction. The input layer implicitly takes sequences of historical data, with each sequence consisting of a certain number of past time steps and a single input feature, representing the type of network slice transformed into integer values. The model then comprises two LSTM layers, with the first having 64 units and the second having 32 units, both employing the 'relu' activation function.
A. Slice Classification using ANN This article has been accepted for publication in IEEE Access. This is the author's version which has not been fully edited and content may change prior to final publication. Citation information: DOI 10. 1109/ACCESS. 2023. 3332225 Slice classification using ANN has been performed by training a neural network having 3 dense layers to intrigue into the feature set of the network slicing data. The train and test datasets have been generated to identify the accuracy of the model which is found to be highly accurate as depicted in Table  3. The model fits perfectly with the data while bringing in a significant reduction in both the validation loss (10 percent of the training data) and training loss. 04 different performance metrics have been selected namely Precision, Recall, Accuracy, and F1 Score.
Additionally, a confusion matrix has been generated to identify the probability of true and false predictive measures. The training accuracy and loss model for both Validation and Training sets has been depicted in Figures  3,  and 4 , respectively. In Figure  5 , the predictive results based on the LSTM model have been presented. The two lines reveal the original and predicted network slices against a chuck of time samples taken across the training dataset. A small bias of 0. 1 has been added in the predicted class for ease of visualization. The model is found to accurately predict the future data network topology that has mainly been employed in this study to enhance the latency and reliability figures across the entire network.
Acknowledge support from the  CONEX-Plus program  funded by  Universidad Carlos III de Madrid  and the  European Union 's  Horizon 2020 research and innovation program  under the  Marie Sklodowska-Curie  grant agreement No. 801538. The authors also sincerely appreciate funding from  Researchers Supporting  Project number ( RSP2023R58 ),  King Saud University, Riyadh, Saudi Arabia. Additionally, this work was supported in part by the  National Key Research and Development Program of China  under Grant  2020YFB1807900 , in part by the  National Natural Science Foundation of China  under Grant  62101306.
Starting from an initial physics-based model of a wind turbine drivetrain, which is trained with real data, the proposed methodology has two major innovative outcomes. The first innovation aspect is the application of generative stochastic models coupled with a hybrid-model-based digital twin (DT) for the creation of synthetic failure data based on real anomalies observed in SCADA data. The second innovation aspect is the classification of failures based on machine learning techniques, that allows anomaly conditions to be identified in the operation of the wind turbine. Firstly, technique and methodology were contrasted and validated with operation data of a real wind farm owned by Engie, including labelled failure conditions. Although the selected use case technology is based on a double-fed induction generator (DFIG) and its corresponding partial-scale power converter, the methodology could be applied to other wind conversion technologies.
8% for power converters of offshore wind farms  [12, 13]. These components are equipped with sensors (temperature, vibrations, electric parameters and others) and connected to the wind turbine supervisory control and data acquisition (SCADA) and condition-based monitoring (CBM) systems. Thus, a long historical real operation dataset exists for each turbine of a wind farm. Sometimes, this dataset includes recorded anomalies or failure in the operation of the turbine. Data-driven models extract knowledge from real measurements that apply AI (artificial intelligence) techniques, which analyse large amounts of data to identify meaningful patterns in them. In the field of wind energy generation, there are several approaches for this type of model. For instance, the spectral analysis of current signals has been used for health monitoring of stator and rotor windings, as well as the main bearing of wind turbines  [14].
In  [15] , a data-driven model is directly constructed with the objective of detecting and isolating sensor and actuator failures in wind turbines, while the study of  [16]  develops a hierarchical bank of negative selection algorithms (NSAs) to detect and isolate common failures in wind turbines. The study of  [17]  uses a data-driven failure diagnosis and isolation (FDI) method for wind turbines. It consists of the implementation of long short-term memory (LSTM) networks for residual generators. The decision-making process is made by applying a random forest algorithm. These FDI methods are designed using experimental and historical data generated both under normal and failure conditions; therefore, the availability of well-developed databases that include labelled anomaly/failure data is mandatory.
The accuracy of data-driven methods is generally poor for cases not included in a training dataset. In addition, black box models (e. g. , deep learning models) show a low explainability, making it difficult for domain experts to interpret results and gain the required trust to make decisions based on the output of the models. As a solution to this main drawback of data-driven models, DTs that use physicsbased models are developed to make the DT self-explanatory. The term "digital twin" can be defined as "a virtual representation of a real-life system or asset with the same behaviour". It allows system states to be calculated using integrated models and data, aiding the decision-making process over its life cycle from design to decommissioning. The concept of DT was first described in David Gelernter's 1991 book Mirror Worlds  [18] , and the term "digital twin" was first mentioned in a roadmap report developed by John Vickers (NASA) in 2010.
In addition, the DT preserves the constrains, significance and explainability of a physical model, overcoming some of the main limitations of a purely statistical generative model (i. e. , generative adversarial networks). The physics-based model for the drivetrain of a wind turbine is developed using MATLAB Simulink R2020b. The paper is organized as follows: Section 1 describes the developed technical approaches and the literature review related to such technical approaches, as well the problems of using data-driven approaches in comparison with hybrid models. Section 2 explains the proposed methodology for developing a hybrid-model-based digital twin and the advantages of combining both physics-based and data-driven models. Moreover, this section describes the principles of synthetic data generation and how such principles can be applied to failure data generation.
Normality Hybrid Model The normality hybrid model of the DT is composed of a physics-based model trained with real operation SCADA data in normality conditions. The paper considers the drivetrain of a wind turbine with DFIG technology as a reference use case in which the proposed DT development methodology is illustrated and applied. Figure  2  shows how the physics-based model is divided in two modules that could be used either coupled together or separately, depending on the available operational data. The first module represents the conversion from kinetic energy from the wind to mechanical power, taking the real values of the wind speed measured at the turbine and the pitch angle of the blades as inputs. The second module represents the electro-mechanical conversion. It takes the mechanical torque in the shaft of the DFIG as the input and the generated electric power and its related signals, such as phase currents and voltages or electromagnetic torque, are the outputs.
Failure Hybrid Model Once the normality hybrid model is constructed, it can be extended or adapted to include anomaly or failure situations. This new model is called a failure model. Following the same process used in the normality hybrid model, this model is trained using the operation real SCADA data. Similarly, calibration consists of optimizing the values of certain independent design parameters that represent failure, whose exact values are estimated between given realistic intervals. This resulting new model is also trained with historical and actual operational data of both normal and failure operation. This is achieved using real failure operation data inputs, which are fed to the failure models. In other words, when the normality hybrid model is adapted to represent a failure and trained with failure data (data representing failure operation), the normality hybrid model becomes a failure hybrid model.
Failure Synthetic Data Generation The methodology analysed in the article has a fundamental contribution in the generation of synthetic data. The generation of synthetic data is a key point because it allows immediate availability of operation data (either normality or failure data), that are difficult to obtain from simple observation of the reality. In addition, the training of classification models for failure prognosis is much enriching using a broad and balanced dataset that represents a variability of behaviour. Ref. [22]  proposes GANs for the generation of synthetic data for wind turbine failure diagnosis research. This article proposes a method to generate synthetic data using the hybrid model and a statistical process. The statistical process characterizes the probability distributions of the occurrence of normal and failure operating scenarios.
However, this might not be the case for the generation of failure conditions. Although the failure hybrid model has been calibrated to simulate the instances belonging to this type of conditions present in the training SCADA data, this does not guarantee that these instances are a representation of the entire anomalous feature space. In fact, the frequency of anomalous conditions and failures is relatively low in SCADA data, and often these instances are not annotated (labelled). Hence, relaying merely on a deterministic model to generate synthetic failure scenarios would provide a narrow data sample constrain to patterns already seen before. To resolve this limitation, the DT can incorporate stochastic failure models for the generation of failure scenarios. Each of these models can generate an unlimited number of synthetic failure scenarios for a particular failure type based on real observations in SCADA data.
The process for the synthetic failure data generation of Figure  1    The PDFs of the parameters are learnt from the observed instances in the SCADA data. These instances might be exclusively sourced from a single turbine or, in case of an insufficient number, they can be sourced from different turbines that share some design and operations characteristics. The decision to include instances from more than one turbine should be made on the basis of turbine similarity and the variability of failure parameters, which depends on operation and design characteristics. The distribution of most parameters might be approximated by a normal PDF with the required precision. However, other distributions might need to be considered for certain parameters. In the case of having access to SCADA data with several instances of a given failure for more than one turbine, a hierarchical parameter modelling might provide a better balance between accuracy and generalization.
Potential Application of the Hybrid Models Conforming the Digital Twin The development of data-driven algorithms for diagnosing normality or failure conditions is a complex task that involves: (i) defining the condition indicators (CIs), (ii) labelling normality and failure operation data, (iii) conceptualization of the classification model, (iv) validation of the model (e. g. , number of false positives and negatives), and (v) evaluation of the generalization capacity of the model analysing whether it is representative for a set of machines. The DT can add value to this endeavour by providing additional synthetic data to strengthen the dataset. Figure  7  shows a proposed schema of a supervised classifier training process for failure diagnosis where the explained models in the previous sections are leveraged.
The classifier is trained with a labelled dataset composed of real SCADA data, augmented with synthetic data generated via the process described in the previous section. In addition, the normality hybrid model is used as a baseline to create new CIs that may improve the accuracy of the classifier. These CIs are calculated by comparing real operation SCADA data with respect to synthetic failure data and/or normality data generated by the normality hybrid model. Finally, Figure  8  shows the execution phase, where CIs are created by comparing real SCADA data with the data simulated by the normality hybrid model. When the values of these CIs meet certain criteria detected by the classifier, an early alarm is generated.
Results of Application of the Methodology to a Use Case: 1. 5 MW DFIG Wind Turbine The methodology described in previous section was applied and validated with real SCADA data from a wind turbine in operation owned by Engie. The drivetrain of this wind turbine comprises a 1. 5 MW DFIG and its corresponding back-to-back power converter. Three years of real operational data were organized and preprocessed before use. During the data exploration and pre-processing of SCADA data, relationships between physic parameters were analysed, in order to detect possible outliers, which were removed. Once the initial data analysis was carried out, the physical model of the power conversion was developed in Simulink-Matlab R2020b (Figure  9 ). Information on the design parameters of both the generator and power converter was used as a basis for constructing the model.