dataesr/scientific-paragraphs-categorization huggingface dataset:
As such, the dataset was initially split into two partitions, where each partition included exclusively a single category of samples. Each VAE model was trained over 20 epochs, and 30% of the dataset was used for validation. Figures  6  and 7  plot the model loss in the training and validation sets for the positive and negative datasets, respectively. It can be observed that the VAE models both largely converged after 10 epochs. The model was implemented using Keras  [61]  with the TensorFlow backend  [62]. Eventually, the VAE models were used to generate synthetic scanpath images. Around 300 images were generated for each category.
Classification Experiments This part aims to investigate the impact of data augmentation on the performance of classification models. Specifically, we compared the model performance before and after the inclusion of the VAE-generated images as part of the training set. A CNN model was implemented for the classification experiments. The model was composed of four convolutional layers. Each convolutional layer was followed by a maxpooling operation. Eventually, the model included two fully connected layers. A Rectified Linear Unit (ReLU) was used as the activation function in all layers. The dataset was partitioned into training and test sets based on a three-fold cross-validation. The experiments included two scenarios. On the one hand, the model was trained without including the synthetic images.
On the other hand, the model was re-trained after the inclusion of the VAE-generated images in the training set. However, the test set always included samples from the original dataset in both scenarios. The classification accuracy was analyzed based on the receiver operating characteristics (ROC) curve. The ROC curve plots the relationship between the true positive rate and the false positive rate across a full range of possible thresholds. Figure  9  plots the ROC curve in the baseline case (i. e. , without augmentation), while Figure  10  plots the ROC curve in case of applying the VAE-based data augmentation, as previously explained. The figures give the approximate value of the area under the curve and its standard deviation over the three-fold cross-validation. The AUC-ROC values demonstrate that the model performance consistently improved after augmenting the dataset with the synthetic images.
On the other hand, the model was re-trained after the inclusion of the VAE-generated images in the training set. However, the test set always included samples from the original dataset in both scenarios. The classification accuracy was analyzed based on the receiver operating characteristics (ROC) curve. The ROC curve plots the relationship between the true positive rate and the false positive rate across a full range of possible thresholds. Figure  9  plots the ROC curve in the baseline case (i. e. ; without augmentation), while Figure  10  plots the ROC curve in case of applying the VAE-based data augmentation, as previously explained. The figures give the approximate value of the area under the curve and its standard deviation over the three-fold cross-validation. The AUC-ROC values demonstrate that the model performance consistently improved after augmenting the dataset with the synthetic images.
The mainstay of our approach is the visual representation of eye-tracking data, which allowed for an amenable representation for training the VAE model. The empirical results clearly confirmed the positive impact of data augmentation on the model's performance. The classification accuracy could be improved by augmenting the training set with the VAE-generated images. It is proposed that the lack of open access eye-tracking datasets could make our approach attractive for further investigation. For instance, VAE models can serve as an alternative method for data generation in a wide range of eye-tracking applications.
To early detect the disease, SAGIR had to detect, sample, and test as many WB carcasses as possible (roadkill included). As the movement restrictions and the hunting ban reduce the chances for passive surveillance, surveillance reinforcement through active carcass search was proposed. Protocols that assured professional and voluntary observation in good biosecurity conditions were developed for three types of searches: (i) hunter patrols, (ii) systematic combing of forest in at-risk forests, and (iii) dog detection. From mid-February 2019, an active surveillance program was also conducted. Twenty percent of the hunted WBs were sampled and tested by RT-PCR (data not presented). We also developed a procedure for dating carcasses (with the support of forensic police) in case of confirmed infection.
In January 2019, three dog handlers contracted by ONCFS started to train their dogs to detect the scent of dead WB, and fieldwork started 3-4 weeks later. During the study period, five other dogs were trained and used in the field. The protocol included restrictions during very hot or very cold periods. A specific biosecurity procedure was developed for this activity, including washing and disinfecting the dogs' legs after each field session (a field day was made up of several field sessions as dogs cannot search for a long time). For each of those surveillance activities, a specific form was designed, filled out, and compiled. Hunting organisations developed a shared database and were responsible for compiling and entering data for hunter patrols. They also collected forms from foresters and entered data for systematic combing.
Evaluation of Surveillance Efficacy To evaluate surveillance efficacy, we developed an indicator enabling us to roughly compare the surveillance results between zones by dividing the number of carcasses detected by a proxy for the WB population. The proxy used is the forest surface area because it is assumed to be proportional to the WB population  (7, 8). The forest surface area was extracted from the CORINE Land Cover R database (2018, vector data). Three classes of vegetation were considered: broad-leaved forest, coniferous forest, and mixed forest. The zones to be compared included the three zones of our Level III area in France (ZBN, ZNC, and ZBS) and three zones in Belgium at the border with France (ZOR_NO, ZOR_ZI_SO, and ZOR_SUD) (Figure  1 ). Due to their location, the three zones in Belgium present landscape continuity with the French Level III area.
Two of them (ZOR_NO and ZOR_SUD) share a similar epidemiological situation with the French zones: they were not infected but close to an infected area. They also experienced active search activities not described in this study. The zone named ZOR_ZI_SO, initially of similar status to the other two, became infected in January 2019 and regained its previous status in May 2020 (9). Data from Belgium surveillance was compiled from the reports produced and shared by the Public Service of Wallonia.
Surveillance Efficacy Fifty-four carcasses were detected between September 2018 and the end of August 2020 in our Level III area, mainly due to opportunistic surveillance. Nevertheless, 50% of them were roadkill known to be less likely to be infected than other carcasses found in ASF-infected countries  (10). Thus, they were recorded separately to give a more precise picture of the surveillance results. Without precise knowledge of the WB population, the question of our surveillance system's efficacy in detecting most of the carcasses is difficult to answer. Analysing data from the WB depopulation programme may help to do so. The hunting bag for the 2018-2019 season was 936 animals in the Level III area. In January 2019, fences were built, and a depopulation programme started. Thus, the WB population within the Level III area was under strict surveillance with limited opportunity to move outside the area.
Over the 2019-2020 hunting season, the number of hunted WBs within the depopulation programme (by hunters and professionals) was 951 on April 19, 2020 (OFB data). On that date, the remaining population within this Level III area was estimated to be between 150 and 220 animals (based on field observations and data from camera traps-OFB data). Although it is probably imperfect, compiling this data gives a rough idea of the population level within this restricted zone of 300 km 2. Guberti et al. (6)  estimate that a desirable goal for dead WB surveillance is to report 10% of the carcasses. They estimated that natural mortality in WBs is 10% of the population. In our case, if we roughly estimate that 2,500 WBs lived in this area over the study period, around 250 animals may have died naturally.
Our surveillance system succeeded in detecting 17% of this estimated dead WB population. In the context of a depopulation programme, the natural mortality percentage is probably even lower due to the intense hunting pressure, and as the depopulation progressed, it became more difficult to found one carcass. Another way to evaluate our surveillance activities was to compare the French and Belgian surveillance data in similar areas (Belgian ZOR neighbouring the French border). We can hypothesise that the WB population was shared to some extent between the two countries (especially when a forest lays on both sides of the border). Thus, we used a proxy for this population (surface area of the forest) to calculate an indicator of the surveillance efficacy and we observed increased detection of carcasses in proximity to the ASF cases.
ACKNOWLEDGMENTS We would like to acknowledge the work done by all the field actors: hunters, departmental hunters' federations, OFB and ONF officers, and dog handlers. We would like to thank our colleagues from the  Belgian Department of Nature and Forest  for sharing their experience and data throughout the crisis period. We would also like to thank the  Departmental Veterinary Laboratory of Strasbourg  in charge of the ASF testing for the study area, as well as the other members of the national working group on ASF surveillance ( Plateforme ESA ), who contributed to the definition of the risk-level approach for ASF in wildlife:  Florence Baurier  and  Dominique Gautier  (ADILVA),  Marie-Frédérique Le Potier  and  Céline Richomme  ( ANSES ),  Yves Lambert  and  Sebastien Wendling  (DGAL),  Benoit Bourbon  (MTES), and  Patrick Bourguignon  (SNGTV).
Background The raw genomic sequences generated by next generation sequencing (NGS) are an important source of data for studying and understanding organisms and biological mechanisms. However, without a crucial step of extracting biological knowledge from the raw data, a process called 'genome annotation' , the sequences are difficult to exploit and may even be useless. A critical step in the annotation process involves the location of genes (i. e. 'structural annotation'), in particular the protein-coding genes and the characterization of their intron/exon structures. A large number of automatic annotation pipelines have been developed to identify protein-coding genes, such as Braker2 [1], Maker [2] or PASA [3], as well as dedicated resources, such as Ensembl [4] or NCBI [5]. Automatic annotation methods are generally based on a combination of empirical evidence, e.
To our knowledge, there are no SS prediction tools trained on data from a large range of less well studied organisms, such as insects (except fruit fly), fungi or protists. In this context, we have developed Spliceator, a new tool for ab initio prediction of eukaryotic multi-species splice sites. Spliceator is based on the CNN technology and more importantly, is trained on an original high quality dataset  [45]  containing genomic sequences from organisms ranging from human to protists. The training dataset has been rigorously established and validated to reduce the number of errors in the input data and avoid introducing bias in the learning process. This dataset allows us to limit the 'garbage-in, garbage-out' effect  [46] , meaning that poor quality data lead to less reliable results. Based on several benchmark experiments, we show that Spliceator achieves overall high accuracy compared to other state-of-the-art programs, including the neural network-based NNSplice  [47] , MaxEntScan  [48]  that models SS using the maximum entropy distribution, and two CNN-based methods: DSSP  [39]  and SpliceFinder  [41].
Design of training and test datasets for multi-species SS prediction Since we employ a supervised learning approach, the careful construction of the positive and negative datasets used for training the CNN models is essential. We designed eight strategies to build different datasets, where each strategy highlights a parameter that can influence the model performance, such as the input sequence length, the data quality, the type of negative sequences (only false positives (FP) or exon, intron and FP sequences) and the dataset composition, i. e. the effect of balanced or unbalanced datasets with different ratios between the number of positive and negative sequences. Each dataset was then split into separate training and test sets in order to build prediction models for donor and acceptor SS using CNN.
The first dataset, called All Sequences (AS), includes sequences from the 1361 'Confirmed' (error-free) gene sequences available in the G3PO+ dataset (see "Methods" section), as well as the 1380 'Unconfirmed' sequences that contain potential gene prediction errors. The AS dataset is designed to represent real-world problems, in the sense that the data is extracted directly from public databases. For the second dataset, called Gold Standard (GS), we exploited only the 'Confirmed' sequences implying that this set is error-free. The resulting AS and GS positive subsets include donor and acceptor SS from human, as well as from a diverse set of 147 eukaryotic organisms, ranging from primates to protists. In order to construct a robust negative subset of non-SS sequences, we again exploited both 'Confirmed' and 'Unconfirmed' sequences for the AS dataset and only 'Confirmed' sequences for the GS dataset.
To do this, we randomly selected sequences from the exon/intron regions of the G3PO+ genomic sequences, as well as sequences containing GT/AG dinucleotides that are not SS. In order to verify that there is no over-representation of sequences from specific organisms or clades, we calculated the mean pairwise percent identity for the input sequences with a length of 600 nucleotides (nt) (Table  1 ) and showed that the majority (> 90%) of the sequences in each positive subset (AS and GS) for both donor and acceptor SS share between 20 and 30% identity. We also calculated the mean pairwise percent identity for sequences with a length of 20 nt, i. e. specifically the short region around SS. Again, the pairwise percent identity is similar for AS and GS positive datasets, however the majority of these sequences share between 20 and 60%, showing that the context close to the SS is more conserved.
Impact of genomic context To evaluate the impact of the genomic context around the SS on the prediction performance of our CNN method, we constructed subsets of sequences for the AS and GS datasets having different lengths, ranging from 20 to 600 nt. The sequence segments upstream and downstream of the SS dinucleotide contain information allowing the discrimination of SS and non-SS, such as the BPS, polypyrimidine tract (PPT) or regulatory cis-elements including exon/intron splicing enhancers or silencers (ESE/ISE or ESS/ISS)   [49]. Determining a pertinent sequence length is important because too short genomic regions would prevent the model from using important discriminatory sites, while too large genomic regions may introduce noise-inducing features and loss of accuracy  [50]. We then built CNN prediction models for donor and acceptor SS, using these different sequence lengths.
AS_1 (respectively GS_1) has the same balanced ratio of positive to negative examples as AS_0 (respectively GS_0), but the negative examples are more heterogeneous, consisting of exon, intron and FP sequences. Based on this initial analysis, in the following experiments, we used a sequence length of 200 nt for the prediction of donor and acceptor sites with the AS and GS dataset, to consider a genomic context that is neither too small nor too large.
Impact of data quality As described above, the GS dataset contains only true SS from the 'Confirmed' gene sequences, while the AS dataset includes some noise (i. e. false SS) from 'Unconfirmed' sequences. To estimate the impact of this noise on model prediction, we compared the average accuracy of the AS models with the corresponding GS models for each SS (donor and acceptor) and for different dataset compositions, as shown in Fig. 3  (Additional file 1: Table  S1 ). As expected, the GS models achieved the best accuracy with an average of 94. 11%, 95. 34%, 95. 11% and 97. 68% respectively for the GS_0, GS_1, GS_2 and GS_10 donor datasets and 92. 73%, 94. 19%, 94. 59%, 97. 45% respectively for the acceptor datasets. In comparison, the AS models obtained an average accuracy of 93. 09% (-1. 02%), 93.
95% (-1. 39%), 93. 88% (-1. 23%) and 97. 33% (-0. 35%) respectively for the AS_0, AS_1, AS_2 and AS_10 donor datasets, and 91. 99% (-0. 74%), 92. 62% (-1. 57%), 93. 70% (-0. 49%) and 97. 17% (-0. 28%) for the acceptor datasets. The results of unpaired t-tests (Additional file 1: Table  S2 ) show that all differences between AS and GS datasets are statistically significant. Interestingly, the difference between AS_1 and GS_1 is the largest for both donor and acceptor models. Based on these results, we selected only the GS models for the following experiments.
Impact of negative dataset composition While the definition of reliable positive examples is clearly essential, the construction of the negative dataset will also have an impact on the ability of CNN methods to distinguish between positive and negative examples. The prediction of SS is an intrinsically unbalanced problem, since in a protein coding gene the SS represent only a small proportion of the total nucleotide length. Therefore, to investigate the impact of the negative subset on prediction performance, we constructed a number of datasets with different types of negative sequences and different ratios of positive and negative examples. We designed two balanced datasets, both with a ratio 1:1 of positive to negative examples, but with either homogeneous (GS_0) or heterogeneous (GS_1) negative examples, as well as two unbalanced datasets with ratios of 1:2 (GS_2) and 1:10 (GS_10) of positive to negative examples.
The unbalanced datasets both have heterogeneous negative examples. We then computed different metrics to evaluate the performance of each model on the test set, as shown in Fig. 4. The overall best performance for the test set is obtained using the GS_1 dataset, with a balanced number of positive and negative sequences and heterogeneous negative examples (exon, intron and FP sequences). Although the average accuracy is lower than for the unbalanced GS_10 dataset (positive/negative ratio of 1:10), the other metrics including the average F1 score and precision are higher for the GS_1 balanced dataset. To confirm these results, the GS models were also evaluated on a set of 5 independent benchmarks (human, fish, fly, worm, plant) and using different metrics. The results are shown in Additional file 1: Figure  S1A  and B.
Performance of optimized CNN model Based on our initial analyses, we determined the optimal training set for the CNN models to predict donor and acceptor SS, namely the GS_1 dataset: a high quality balanced dataset with an equal number of positive and negative sequences, heterogeneous negative examples containing exon, intron and FP sequences and an input sequence length of 200 nt. For this optimized model, we further characterized the prediction performance of Spliceator averaged over a total of 10 experiments due to the random selection of negative sequences. The results are shown in Fig. 5  (Additional file 1: Table  S3 ). The average accuracy is 95. 34% for the optimized donor model and 94. 19% for the acceptor model. The precision of the models is similar, ranging from 92. 50% (donor) to 91.
73% (acceptor). We observed high sensitivity for both models, with 98. 31% (donor) and 97. 20% (acceptor), although the specificity is slightly lower, with 92. 11% (donor) and 91. 14% (acceptor). Finally, the F1 scores are similar for both donor and acceptor with 95. 32% for donor and 94. 39% for acceptor SS. Thus, the average performance for the donor model is slightly higher than for the acceptor model, which for each dataset composition and for each type of SS (donor or acceptor). GS_0 = positive/negative ratio of 1:1 with only FP sequences in negative subset, GS_1 = positive/negative ratio of 1:1 with exon, intron and FP sequences in negative subset, GS_2 = positive/negative ratio of 1:2 and GS_10 = positive/negative ratio of 1:10 might be explained by the fact that the donor SS consensus motif is more conserved than the acceptor SS motif (Table  1 ).
A high precision indicates that the program predicts few FP. Spliceator obtains the second best average precision of 89. 88% for the donor SS and 86. 23% for the acceptor SS, behind DSSP (96. 17% for donor and 92. 28% for acceptor SS). SpliceFinder, which has generally good accuracy, obtains lower precision (74. 92% for the SS donor and 69. 52% for the SS acceptor). Sensitivity and specificity are two inseparable metrics. They describe the proportion of well predicted elements and their quality, i. e. if elements have been correctly predicted. The F1 score combines the precision and the sensitivity metrics and provides a more global view of the number of correctly predicted elements. Spliceator obtains the best average F1 score for both donor and acceptor SS, with 93. 08% (+ 1. 15%) and 89.
4% (+ 3. 83%) compared to the second best program for donor SS, MaxEntScan with an F1 score of 91. 93% or acceptor SS, DSSP with a F1 score of 85. 57%. Figure  7  shows the accuracy and F1 score for each program on the individual benchmarks containing SS from different organisms. While most of the programs tested achieve high scores on the vertebrate sequences (human and fish), reflecting their training sets focused on human SS, Spliceator performance is generally better on the more distant organisms (worm, plant and PVP). Finally, for each model, we observe that the predictions of the donor SS are slightly better than those of the acceptor SS. This difference is probably due to the divergence of the acceptor SS motif, but also to the genomic context around it which seems to be more complex.
Discussion Thanks to high-throughput technologies, as well as the development of computing power, huge amounts of data can now be exploited by DL algorithms and produce remarkable results  [54] [55] [56] [57]. In particular, CNN are increasingly used in the field of bioinformatics  [58] [59] [60] , for example to detect specific patterns in a genomic sequence  [61]  where the reduced number of parameters allows for better generalization compared to other ML methods. Moreover, maxpooling techniques allows the algorithm to focus on the local features that it considers most important. In this context, we have developed a SS prediction program called Spliceator, based on a three-layer convolutional CNN. Despite the recent use of RNA-seq to accurately identify SS, it is currently impossible to obtain experimental data for the full panel of transcripts for all tissues and all developmental stages.
Since SS and other regulatory motifs may be conserved across similar species  [20] , some work has been done to try to transfer models trained on model organisms to related organisms, for example between different vertebrate genomes  [66]. Others have built cross-species models for specific clades, such as animals or plants using Helixer  [67] , but unfortunately the source code for this program is not yet stable (according to the authors). The aim of our work was to extend the idea of cross-species models to conceive universal SS prediction models (one for each SS), that are applicable to a wider range of organisms. For the training of the Spliceator models, we focused on the construction of a multispecies dataset that is as representative as possible of the eukaryotic domain (from primates to protists).