dataesr/scientific-paragraphs-categorization huggingface dataset:
Inferred relations signs are validated by public data. To validate the signed relations inferred for the networks using FANTOM5 data, we looked at the two major databases containing signed regulatory relations: Trrust  (38)  and Signor  (39). Trrust and Signor both contain signed relations that have been described in the literature. Some of these relations can be unsigned or contradictory signed (within the same resource or between both), as evidence for activation or inhibition depends on the biological context. Trrust contains 9,396 relations (4,325 of which with unknown signs) of those only 2,428 can appear in our data meaning both TF and Genes present in our dataset. Signor contains 2,820 relations and only 851 we can obtain. 1 In Table  5d  we compiled the relations in our networks that are found in either Trrust or Signor.
In average we found 0. 22% of the computed relations of a set in Trrust and 0. 04% in Signor. In Trrust 45% of the relations are found unsigned, 39% signed in the same direction and 16% signed differently as the database. In Signor 76% are signed the same way and 23% in opposite direction. The relations found in common between Trrust and Signor in our networks are signed the same way in both resources in most cases and in 72% signed the same way as in our networks. For the relations generated by Regulus and found at least in one of Signor or Trrust, the sign we predicted is also consistent with the databases in two third of the cases. It is important to note that Trrust and Signor relations are not necessary found in the same tissues as the ones we used, which may explain some of the differences in signs.
Workflows comparison. Figure  7b  presents the main steps of the Regulus pipeline, and compares them to those of Regulatory Circuits (Figure  7a ). Both take as input: activities of the regions and the genes (or approximation of the transcript activity) and regions localization. As shown in Figure  7 , both also share similar pre-processing steps like computing the distance between regions and genes or finding the TF binding sites occurrences in the regulatory regions. The main differences are: • Our method uses the activity of the TF (extracted from its coding gene activity) which was not taken into account by Regulatory Circuits workflow. • Regulatory Circuits uses a composite score in which each component must be strictly positive, and is taken as a maximum when several concordant relations exist.
Networks and outputs comparison. Networks topology. For the FANTOM5 data used in 3. 2, Regulatory Circuits number of potential TF-genes relations (2,060,960) was calculated based on the supplementary data files describing entities (TF, promoter, enhancer, transcript) relations, while ignoring the scores (see  (10)  for details and the tissues networks, while (b) compute a regulatory network and filter it with a consistency rule and sign every interaction. In (c) using Regulatory Circuits and (d) using Regulus, we look at the percentage of genes from the RNA-seq related to the tissues found in the resulting networks, the datasets are the one presented in section 3. 2 and fig 5. Overall Regulus is better at recalling the least expressed genes (from 15% to 35%) and similar on the other points.
data availability at 2 ). All the TF-gene relations found by Regulus are included in the potential Regulatory Circuits relations, meaning that our method does not create irrelevant relations. However, Regulatory Circuits provides very large networks, comprising from 407,056 to 1,796,098 unsigned relations for a single tissue or cell type; whereas Regulus signs and refines these relations to less than 180,000 for sets of four different tissues, adding an interesting filtering power. Inclusion of genes based on their expression level. When using a similar validation of output genes using Gtex RNAseq datasets as in 3. 2, Regulatory Circuits was well able to retrieve highly or moderately expressed genes in its networks. However, as shown in Figure  7c , lowly expressed genes were poorly incorporated, in agreement with their methodology favoring inductive relations and high activities (see above).
A limit of this approach might be the oversampling of the patterns as some patterns were very poorly populated: 33 with less that 10 genes including 18 with less than 5 genes on the B cells data. Those patterns could be grouped with patterns of similar direction or removed from the analysis, since they may bias the coverage and specificity filter. Indeed small patterns are easily covered and may introduce high specificity percentages. Another solution would have been to use co-expression analysis for example using the WGCNA R (40) package. Unfortunately, after testing, it gave poor results with our limited number of datasets. Our work shows the added value of integrating large and heterogeneous data from biological experiments when inferring regulatory networks. Even on closely related cell types, such as the B cells subsets, we are able to identify subtle differences based on our use of patterns and of the consistency step.
ACKNOWLEDGEMENTS We acknowledge the  GenOuest bioinformatics core facility  (https://www. genouest. org) for providing the computing infrastructure. FUNDING ML was financed by the  " Médecine Numérique" joint PhD program  from  INRIA & INSERM. Data acquisition on B cells subsets was funded by an internal grant from the  Hematology Laboratory, Pôle de Biologie, Centre Hospitalier Universitaire de Rennes, Rennes, France.
DATA AVAILABILITY STATEMENT The data underlying this article are available in the article and in its online supplementary material. More precisely, Regulus software and source code are available on gitlab at: https://gitlab. com/gcollet/regulus. ClassFac-torY software and source code are available on gitlab at: https://gitlab. com/EveBarre/ClassFactorY. FANTOM5 and GTex data used for testing and validation are available in the supplementary data of (10) at http: //regulatorycircuits. org. B cells datasets will be shared on reasonable request to the corresponding author.
Composition of the tissues of the 4 sub-sets from Fantom5: 2 composed of biologically-similar tissues and 2 composed of dissimilar tissues. Sub-set Name Tissue 1 Tissue 2 Tissue 3 Tissue 4 Dataset 1 B lymphoblastoid cell line CD4+ T cells CD8+ T cells peripheral blood mononuclear cell Dataset 2 colon adult colon fetal small intestine adult small intestine fetal Dataset 3 CD34+ stem cells adult brain fetal epitheloid cancer cell line pancreas adult Dataset 4 CD4+ T cells brain fetal colon adult epitheloid cancer cell line (a) Sub-set Nb relations Nb relations Nb relations Nb relations Nb relations Ratio relations TF-region-Gene TF-region-Gene TF-Gene TF-Gene TF-Gene TF-Gene Predicted before filtering After filtering After filtering "+" "-" +/- Dataset 1 3,005,934 219,495 164,251 114,624 49,627 2.
3 Dataset 2 3,005,934 237,487 178,514 154,276 24,238 6. 3 Dataset 3 3,005,934 165,804 125,451 79,359 46,092 1. 7 Dataset 4 3,005,934 165,597 126,145 88,609 37,536 2. 3 (b) Number of relations by network after Regulus, on the 4 sub-sets of Fantom5. Genes regulated in Top 10% most expressed Mid 10% 10% least expressed Dataset 1 90. 75 86. 25 41 Dataset 2 90 88. 25 32. 5 Dataset 3 89 88 49. 25 Dataset 4 88. 75 85. 75 55 (c) Percentage of genes from the RNA-seq related to the tissues found in the resulting networks. The RNA-seq genes are separated in three categories: the top 10% most expressed, the middle 10% and the 10%least expressed. Relations Trrust Signor Trrust Signor TF-gene in Total True False Unknown Total True False Unknown Total Different True False Dataset 1 432 164 65 203 76 54 21 0 51 9 33 9 Dataset 2 357 156 37 164 54 45 9 0 39 5 31 3 Dataset 3 293 100 54 139 56 42 14 0 39 4 28 7 Dataset 4 264 113 50 101 58 44 14 0 37 2 26 7.
A three-dimensional ensemble-variational (3DEnVar) data assimilation algorithm has been developed for the high-resolution AROME NWP system. Building on previous work on 3DEnVar for AROME, we describe a formulation of the 3DEnVar that is based on the traditional square-root preconditioning. The localization may be performed either in gridpoint or spectral space, and allows for cross-covariances between surface pressure and the three-dimensional variables. The scheme has capacity for dual resolution, with the ensemble running at a lower 3. 2-km spatial resolution than the deterministic AROME running at 1. 3 km. This formulation is compatible with the variational bias correction scheme used in AROME. Hybrid covariances are implemented with climatological covariances at 1. 3-km resolution being combined with ensemble perturbations that are interpolated to high resolution on the fly in the computation of the gradient.
Introduction Bayes's law aims at updating a prior distribution with recent information brought by observations. If we make the Gaussian approximation, the relevant probability density functions (pdf) are fully described by their means and covariances. However, computing the mean of the posterior pdf is challenging in numerical weather prediction (NWP). This is due to the high computational cost associated with large dimensions of the state and of the observation vectors, respectively, ;10 9 and ;10 5 for the Application of Research to Operations at Mesoscale (AROME) limited-area NWP system. Variational data assimilation schemes estimate the mode (rather than the mean) of the posterior distribution. This is done efficiently by minimizing a cost function. In NWP, the covariance matrices of the background and observations errors are never stored explicitly  (Dee 1995) ; rather they are implemented as matrix-free operations  (Bannister 2008 ).
2007 ) is also active, contrary to M18. Also, the experiments cover an extended summer period of three months that is more representative of the high precipitating events that AROME is targeted at. The choice of a square-root formulation is possibly not final, other algorithms such as the observation-space algorithm from  Gratton and Tshimanga (2009)  being possibly better candidates. In the linear framework, those algorithms result in mathematically equivalent results at every iteration of the minimization  (Gürol et al. 2014). The square-root implementation requires an extension of the control variable  (Lorenc 2003; Buehner 2005). With a square-root implementation, it is straightforward to implement a full-B preconditioning with B 5 B 1/2 B T/2. It may also be easier to check the correctness of the code with an adjoint test.
Note that the two components of the wind are localized rather than vorticity and divergence as stated above, thus this inverse Fourier transform also incorporates the corresponding change of variable. There is no increment of the mean wind over the domain, as for 3DVar  (Fischer et al. 2005). The spectral formulation of spatial localization may then be written as follows: L s 5 F 21 S, (13) whereas the gridpoint formulation of spatial localization may be written as follows: L s 5 GF 21. ( 14 ) M18 indicate that the gridpoint formulation of spatial localization was performing better, a finding that was reproduced here with our set of experiments (not shown). Thus only the formulation (  14 ) is used in the rest of this paper. A spectral formulation of the control variable is kept even when using the gridpoint formulation of spatial localization mostly to preserve the code compatibility between AROME and ARPEGE-IFS formulations, and possibly in the future ease the implementation of multi-incremental schemes  (Trémolet 2007 ).
Acknowledgments. We thank  L. Berre  and  C. Fischer  for their careful reviews of this manuscript. E. Arbogast  and  T. Montmerle  contributed to the implementation by developing the interpolation routines that perform the change of spatial resolution and its adjoint. This research has been supported by the  SuNDAE  project funded by the  ANR, Paris, France , under Grant  ANR-18-CE46-0007. This article is a contribution to the  HyMeX program.
Over the past decade, deep learning has achieved unprecedented successes in a diversity of application domains, given large-scale datasets. However, particular domains, such as healthcare, inherently suffer from data paucity and imbalance. Moreover, datasets could be largely inaccessible due to privacy concerns, or lack of data-sharing incentives. Such challenges have attached significance to the application of generative modeling and data augmentation in that domain. In this context, this study explores a machine learning-based approach for generating synthetic eye-tracking data. We explore a novel application of variational autoencoders (VAEs) in this regard. More specifically, a VAE model is trained to generate an image-based representation of the eye-tracking output, socalled scanpaths.
Examples include marketing  [9] , psychology studies  [10] , product design  [11] , and many other applications. However, the scarce availability or difficulty of acquiring eye-tracking datasets represents a key challenge, while access to image or time series data, for example, has been largely facilitated thanks to large-scale repositories such as ImageNet  [12]  or UCR  [13]. The eye-tracking literature still lacks such data repositories. In this respect, we explore the use of machine learning (ML) for generating synthetic eye-tracking data in this study. An image-based approach is adopted based on transforming the eye-tracking scanpaths into a visual representation. Using unsupervised learning, a variational autoencoder (VAE) is employed for the generative modeling task. Subsequently, empirical experiments robustly demonstrated that the inclusion of VAE-generated images could improve the performance of models in classification tasks.
Eye-tracking has been widely utilized in a multitude of applications for commercial and research purposes. Examples include marketing  [9] , psychology studies  [10] , product design  [11] , and many other applications. However, the scarce availability or difficulty of acquiring eye-tracking datasets represents a key challenge, while access to image or time series data, for example, has been largely facilitated thanks to large-scale repositories such as ImageNet  [12]  or UCR  [13]. The eye-tracking literature still lacks such data repositories. In this respect, we explore the use of machine learning (ML) for generating synthetic eye-tracking data in this study. An image-based approach is adopted based on transforming the eye-tracking scanpaths into a visual representation. Using unsupervised learning, a variational autoencoder (VAE) is employed for the generative modeling task.
Variational Autoencoder (VAE)-Based Methods for Data Augmentation The VAE approach has been intensively applied for synthetic data generation, or representation learning in a broader sense. The literature already includes a diversity of studies that made use of VAE-based implementations as a mechanism for data augmentation. For instance, a study by  [29]  explored the beneficial use of VAEs in the case of imbalanced datasets. To this end, they extracted an imbalanced subset of the popular MNIST dataset. The dataset was augmented with synthetic samples generated by a VAE model. Their empirical results demonstrated that the inclusion of VAE-generated samples had a positive impact on the classification accuracy in general. Similarly, a more recent study analyzed the impact of using different augmentation techniques on the model accuracy in supervised ML problems  [30].
Their experiments focused on smaller datasets, where the number of samples per class were lower than 1000. The experiments were based on a set of 19 benchmark datasets selected from the University of California Irvine (UCI) data repository  [31]. Using VAE and GAN models, their results demonstrated that data augmentation could boost the prediction accuracy by approximately 3%. From a practical standpoint, the literature includes a broad variety of applications using the VAE approach for augmentation. One recent study used a VAE model to generate traffic data pertaining to crash events  [32]. Their work demonstrated how the VAE latent space could be used to generate millions of synthetic crash samples. The use of data augmentation had a significant effect on the model performance since the original dataset was extremely imbalanced.
In another application related to acoustic modeling, a VAEbased framework was developed to perform data augmentation and feature extraction  [33]. The dataset size of speech corpus could be doubled using the latent variables extracted by the VAE model. Similarly, their results demonstrated that augmentation could improve the performance of speech recognition. In the context of electroencephalography (EEG), a study used augmentation techniques including VAE  [34]. They applied a VAE model to generate realistic features of EEG records, which were used to augment the training data. The empirical results reported a significant improvement in the accuracy of the emotion recognition models. More specifically, the models could achieve up to 10% improvement. Similarly, recent efforts  [35]  have explored VAE-based methods to augment EEG datasets.
Furthermore, numerous applications have been experimentally studied in the field of medical imaging. For instance, a convolutional VAE model was developed to generate realistic samples of left ventricular segmentations for data augmentation  [36]. Another study demonstrated the effectiveness of VAEs for generating synthetic images of clinical datasets including ultrasound spine images and Magnetic Resonance Imaging (MRI) brain im-ages  [37]. More complex tasks were approached using VAE-based architectures as well. For example, a VAE-based approach was adopted for the three-dimensional (3D) reconstruction of the fetal skull from two-dimensional (2D) ultrasound planes acquired during the screening process  [38]. They developed a VAE model that could integrate ultrasound planes into conditional variables to generate a consolidated latent space.
A CNN-based architecture was utilized for the reconstruction and generation of eye movement data. Another study proposed a convolutional-recurrent architecture, named "PathGAN"  [48]. On the basis of adversarial learning, the PathGAN framework presented an end-to-end model for predicting the visual scanpath. In another application, a real-time system for gaze animation was developed using RNNs  [49]. Motion and video data were both used to train the RNN model, which could predict the motion of body and eyes. The data were captured by a head-mounted camera. Moreover, long short-term memory (LSTM) architectures have been developed to generate synthetic eye-tracking data, for instance, a sequence-to-sequence LSTM-based architecture was developed to this end  [50]. More recently, another recent study proposed a text-based approach using an LSTM implementation  [51].
Data Description The dataset under consideration was collected as part of our earlier work related to the detection of autism using eye-tracking  [52]. Abnormalities of eye gaze have been largely identified as the hallmark of autism spectrum disorder (ASD)  [53]. As such, eye-tracking methods are widely utilized in this context. The dataset was originally constructed as follows: A group of 59 children participated in a set of eye-tracking experiments. The age of participants ranged from 3 to 12 years old. The participants were grouped into two cohorts as follows: (i) typically developing (TD) and (ii) ASD. The participants engaged in watching a set of photographs and videos, which included social cognition scenarios according to their age, to stimulate the viewer's gaze. The average period of time of each eye-tracking experiment was about 5 min.
The experiments were conducted using an eye-tracker by SensoMotoric Instruments (SMI) (Teltow, Germany) with 60 Hz sampling rate. The eye-tracking device captured three categories of eye movements including fixations, saccades, and blinks. A fixation describes a brief period of gaze focus on an object, which allows the brain to perform the process of perception. The average timespan of fixations is estimated to be around 330 ms  [54]. Saccades include rapid and short eye movements that perform constant scanning and consist of quick ballistic jumps of 2 • or longer, with an average duration of about 30-120 ms each  [55]. The output of a sequence of fixations and saccades is defined as a scanpath. A set of 25 eye-tracking experiments was conducted to produce the output dataset. The dataset was stored in multiple CSV files, which collectively included more than 2M records.
The diameter of fixations indicates the duration, and the lengths of lines represent the continuation of saccades. As we previously mentioned, our approach was based on transforming eye-tracking output (i. e. ; scanpaths) into an image-based format. Our representation of scanpaths follows on the core idea of visualizing fixations and saccades. Moreover, we aimed to visually encode the dynamics of gaze using color gradients. Given the coordinates/time information, we were able to calculate the velocity of gaze movement. Using the grayscale spectrum, the color values were tuned based on the magnitude of velocity with respect to time. The visualizations were produced using Matplotlib library  [58]. A comprehensive presentation of that part is elaborated in our earlier work  [52]. The outcome of the transformation process was an image dataset containing more than 500 images.
Specifically, 328 images related to the TD participants, and another 219 images for the ASD-diagnosed. The default image dimensions were set as 640 × 480. The dataset along with its metadata files have been made publicly available on the Figshare repository  [59]. Figure  4  presents two examples from the dataset. As we previously mentioned, our approach was based on transforming eye-tracking output (i. e. , scanpaths) into an image-based format. Our representation of scanpaths follows on the core idea of visualizing fixations and saccades. Moreover, we aimed to visually encode the dynamics of gaze using color gradients. Given the coordinates/time information, we were able to calculate the velocity of gaze movement. Using the grayscale spectrum, the color values were tuned based on the magnitude of velocity with respect to time.
The visualizations were produced using Matplotlib library  [58]. A comprehensive presentation of that part is elaborated in our earlier work  [52]. The outcome of the transformation process was an image dataset containing more than 500 images. Specifically, 328 images related to the TD participants, and another 219 images for the ASD-diagnosed. The default image dimensions were set as 640 × 480. The dataset along with its metadata files have been made publicly available on the Figshare repository  [59]. Figure  4  presents two examples from the dataset. duce a compact image-based format of eye-tracking scanpaths. This section elaborates on the data transformation procedures. Initially, it is important to clearly define a scanpath, which is the building block of data. A scanpath represents a sequence of consecutive fixations and saccades as a trace through time and space that may overlap itself  [56].
Given the coordinates/time information, we were able to calculate the velocity of gaze movement. Using the grayscale spectrum, the color values were tuned based on the magnitude of velocity with respect to time. The visualizations were produced using Matplotlib library  [58]. A comprehensive presentation of that part is elaborated in our earlier work  [52]. The outcome of the transformation process was an image dataset containing more than 500 images. Specifically, 328 images related to the TD participants, and another 219 images for the ASD-diagnosed. The default image dimensions were set as 640 × 480. The dataset along with its metadata files have been made publicly available on the Figshare repository  [59]. Figure  4  presents two examples from the dataset.