dataesr/scientific-paragraphs-categorization huggingface dataset:
Performance comparison In this section, we compare LRCstats and ELECTOR runtime and memory consumption on several datasets chosen to represent different use cases. Results are presented in Tables 5 and 6. For the experiments presented in Table  5 , both tools were ran on a 20-core cluster node equipped with 250 GB of RAM. For experiments presented in Table  6 , we used a 16-core computer equipped with 64 GB of RAM. In order to compare similar operations, ELECTOR's runtime and memory consumption do not consider the remapping and assembly steps. We present the metrics and resource consumption of this module apart, in Supplementary Table  S4. We first assess, in Table  5 , the performances of both tools on several simulated E. coli datasets with different read lengths, ranging from 1 k bases to 1 M bases.
Another observation from Table  6  is that we can notice large divergence in ELECTOR runtimes on the same dataset corrected by different tools. This behavior can be due to two factors. On the one hand, ELECTOR's runtime optimization is prone to be more or less pronounced according to the read length (segmentation is expected to be easier with larger reads) and quality of the correction (more errors make it more difficult to find common seeds). On the other, ELECTOR's runtime is also related to the number of split corrected reads output by the corrector. Indeed, a larger number of split reads imply a more significant number of triplet multiple alignments, and thus an increased runtime. In particular, in the experiments presented here, the largest runtimes can be observed for the evaluation of LoRDEC and HALC on the C.
Simulations for the validation of ELECTOR's real data mode In order to validate ELECTOR's real data mode, we ran the following experiment. We used a simulated dataset, and we assessed its correction using the two different modes of ELECTOR: simulated and real data. First, we ran it classically, by providing the simulation files as an input so that ELECTOR could retrieve the actual 'reference' reads by parsing the files. Second, we ran it by only providing the FASTA file of simulated reads as an input, so ELECTOR had to retrieve the 'reference' reads by aligning the uncorrected long reads to the reference genome, as if they were not simulated. We ran this experiment on the S. cerevisiae dataset. To further validate ELECTOR's behavior on real data, we assessed the correction of both a hybrid corrector, HALC and a self-corrector, Canu.
Results on a real human dataset To demonstrate ELECTOR's results in a realistic scenario for large genomes, we evaluate the correction of a real human dataset. We report results, as well as runtime of the evaluation, in Table  8. The reads were corrected with MECAT, using default parameters, before running ELEC-TOR. Using 20 threads, we were able to obtain the results for the 650 771 corrected reads of the dataset in <19 h. Results reported by ELECTOR show that MECAT can correct human reads with a 20% error rate with >90% of recall and precision, which is consistent with the published results.
Indeed, these metrics are highly dependent on the result of the alignment of the 'uncorrected' reads to the reference. These metrics can thus be subject to errors, especially when aligning relatively short or highly erroneous/chimeric reads, or reads coming from repeated regions. A future application is the evaluation of correction methods directly targeted at RNA long reads sequencing. As shown in a recent study  (23) , RNA long reads have specific requirements that are not met by current methods, which calls for new correctors in the future. ELECTOR could be coupled with a reference transcriptome or a RNA long read simulator, although, currently, such a simulation software does not exist to our knowledge.
The generation of synthetic data is an essential tool to study complex systems, allowing for example to test models of these in precisely controlled settings, or to parametrize simulation models when data is missing. This paper focuses on the generation of synthetic data with an emphasis on correlation structure. We introduce a new methodology to generate such correlated synthetic data. It is implemented in the field of socio-spatial systems, more precisely by coupling an urban growth model with a transportation network generation model. We also show the genericity of the method with an application on financial time-series. The simulation results show that the generation of correlated synthetic data for such systems is indeed feasible within a broad range of correlations, and suggest applications of such synthetic datasets.
Introduction Developing methods to study complex systems, such as simulation models or data-mining techniques, often requires testbeds and benchmarks to ensure expected properties. The use of synthetic data, in the sense of statistical populations generated randomly under constraints of proximity of patterns to a studied system, is a widely used methodology tackling this issue. This approach is used in several disciplines related to complex systems such as therapeutic evaluation  [1] , territorial science  [2, 3] , machine learning  [4]  or bio-informatics  [5]. Generation of synthetic datasets can consist in data disaggregation by producing a microscopic population with fixed macroscopic properties  [6]. The creation of synthetic populations for microsimulation models is a typical example where empirical statistical distributions are reproduced  [7].
In data extensive contexts, several methods have been developed and improved for a better reproduction of margin distributions  [8]. Synthetic datasets can also be generated at the same scale than the targeted real dataset, with a broad range of realism levels and corresponding constraints on the generated data  [9]. For example,  [10]  show that some datamining techniques such as decision trees can be inverted to produce datasets capturing complex non-linear patterns. The constraints of proximity to reality of synthetic dataset will depend on expected applications. They range for example from a strong statistical fit on given indicators, to weaker assumptions of similarity on aggregated patterns. In the case of systems where emergence plays a central role, a microscopic property does not directly imply given macroscopic patterns, and synthetic datasets may have to capture some of these.
Proposed approach This literature review on different aspects of synthetic data generation unveils at least two gaps: (i) a lack of attention on controlling covariance structures when generating synthetic data; and (ii) an absence of such methods applied to the study of socio-spatial systems at aggregated scales. As spatio-temporal dependencies structures are essential in driving the dynamics of such systems  [29, 30] , the combination of these two aspects appears as an unexplored research problem. We propose in this paper to study the generation of correlated synthetic data, and more particularly in the case of socio-spatial systems. We introduce here a generic methodology taking into account the dependance structure for the generation of synthetic datasets, more precisely by controlling the average of correlation matrices.
Method Formalization The domain-specific methods described above are too broad to be summarized within a same formalism. We therefore introduce here a generic and model-agnostic framework, focused on the control of correlations structures in synthetic data. Let X I a multidimensional stochastic process (which can be indexed e. g. with time in the case of time-series, but also with space, or any other indexation). We assume to have a real dataset X = (X i,j ), which is interpreted as a set of realizations of the stochastic process. We propose to generate a statistical population X = Xi,j such that 1. A given criteria of proximity to data is verified, i. e. given a precision ε and some aggregated indicator f , we have f (X) -f ( X) < ε (1) 2. The level of correlation is controlled, i. e. given a matrix R representing the correlation structure (any symmetric matrix with coefficients in [-1, 1] and a unity diagonal), we have the estimated covariance matrix given by Ĉ ov X = Σ T • R • Σ (2) where the standard deviation diagonal matrix Σ is estimated on the synthetic population.
The second requirement will generally be conditional to parameter values determining generation procedure, either generation models being simple or complex (R itself is a parameter). Formally, we can also understand synthetic processes as parametric families Xi  [ α]. We propose to apply the methodology on very different examples, both typical of complex systems: territorial systems and financial high-frequency time-series. We illustrate the flexibility of the method, and claim to help building interdisciplinary bridges by methodology transposition and reasoning analogy. In the first case, morphological calibration of a population density distribution model allows to respect real data proximity. Correlations of urban form with transportation network measures are empirically obtained by exploration of coupling with a network morphogenesis model.
Generating correlated synthetic data The coupling of generative models is done both in a formal and operational way. We indeed loosely couple independent implementations. The OpenMOLE software  [44]  for model exploration offers a proper framework for this. Its modular workflow language allows to compose model tasks and integrate these into diverse numerical experiments. For the population density generation, we use the scala implementation provided by  [36]. The network generation model is implemented in NetLogo  [45] , which offers a good compromise between performance and interactive model validation and exploration. The two models are coupled with a specific OpenMOLE script. Source code is available at https://github. com/JusteRaimbault/CityNetwork/tree/master/Models/Synthetic.
A more refined study at higher orders (correlation of correlations) would be necessary to precisely understand degrees of freedom in the generation of correlations. However, the covered area remains broad and confirms a rather flexible output space for generated correlations. When comparing to the null model runs (black dots and error bars), we find as expected that null model correlations are around 0 (all confidence intervals covering the origin), and that a part of the generated point cloud falls in the same area. An other important part of points fall outside the range of the null model in a statistically significant way. These are the interesting points for a possible application of the synthetic dataset, and we show thus that the method produces non-trivial and significant correlation patterns.
Theoretical results must be tested on synthetic datasets as they ensure a control of most parameters in order to check that a predicted effect is indeed observable all things being otherwise equal. Empirical confirmation of the improvement of estimators is obtained on a synthetic dataset at a fixed correlation level. We consider a network of assets (X i (t)) 1≤i≤N sampled at high-frequency (typically 1s). We use a multi-scalar framework (used e. g. in wavelet analysis approaches  [52]  or in multi-fractal signal processing  [53] ) to interpret observed signals as the superposition of components at different time scales. We thus write X i = ω X ω i. We denote by T ω i = ω ≤ω X ω i the filtered signal at a given frequency ω. A typical problem in the study of complex systems is the prediction of a trend at a given scale.
We reconstruct then the hybrid synthetic signals by taking Xi = T ω0 i + Xω1 i (9) The method is tested on an example with two assets from foreign exchange market (EUR/USD and EUR/GBP), on a six month period from June 2015 to November 2015. Data was obtained from http://www. histdata. com/. The data cleaning procedure, starting from original series sampled at a frequency around 1s, consists in a first step to the determination of the minimal common temporal range (missing sequences being ignored, by vertical translation of series, i. e. S(t) := S(t) • S(tn) S(tn-1) when t n-1 , t n are extremities of the "hole" and S(t) value of the asset, what is equivalent to keep the constraint to have returns at similar temporal steps between assets). We study then log-prices and log-returns  [47] , defined by X(t) := log S(t) S0 and ∆X(t) = X(t) -X(t -1).
It is however applied in an adaptive way, in the sense that given a time window T W , we estimate for any t the model on [t -T W + 1, t] in order to predict signals at t + 1. Experiments are implemented in the R language, using in particular the MTS  [55]  library for time-series models. Cleaned data and source code are available on an open git repository at https://github. com/JusteRaimbault/ SyntheticAsset. Fig. 4  shows the effective correlations computed on synthetic data. For standard parameter values (for example ω 0 = 24h, ω 1 = 2h and ρ = -0. 5), we find ρ 0 0. 71 et ε i 0. 3 what yields |ρ e -ρ| 0. 05. We observe a good agreement between observed ρ e and values predicted by Equation 11 in the interval ρ ∈ [-0. 5, 0. 5]. On the contrary, for larger absolute values, a deviation increasing with |ρ| and as ω 1 decreases : it confirms the intuition that when frequency decreases and becomes closer to ω 0 , interferences between the two components are not negligible anymore and invalidate independence assumptions for example.
We postulate that the method can be also applied in other fields where similar constraints can be of interest. Indeed, in the context of financial data, considering data proximity indicators based on low-frequency components of signals, we showed how correlation can be controlled and even analytically predicted to a certain extent. Our work recalls thus the interest in generating hybrid data, and is differentiated from most work where only the microscopic level is taken into account. As already mentioned, most of simulation models need an initial state generated artificially as soon as model parametrization is not done completely on real data. An advanced model sensitivity analysis implies a control on parameters for synthetic dataset generation, seen as model meta-parameters  [27]. In the case of a statistical analysis of model outputs it provides a way to operate a second order statistical control.
Future work Regarding the application to geographical data, the calibration of the network generation component at given density, on real data for transportation network, is a potential development. It would be relevant typically on road networks given the shape of generated networks, what should be possible using OpenStreetMap open data which have a reasonable quality for Europe  [56]. This should theoretically allow to unveil parameter sets reproducing accurately existing configurations both for urban morphology and network shape. By attributing a synthetic dataset similar to a given real configuration, we would be able to compute a sort of intrinsic correlation proper to this configuration. We studied in the second example stochastic processes in the sense of random time-series, whereas time did not have a role in the first case.
Conclusion We proposed an abstract method to generate synthetic datasets in which correlation structure is controlled, but the empirical data required can be sparse or targeting macroscopic aggregated criteria. Its implementation in two very different fields shows its flexibility and the broad range of possible applications. More generally, it is crucial to favorise such practices of systematic validation of computational models by statistical analysis, in particular for agent-based models for which the question of validation remains an open issue. Furthermore, our overall approach enters a particular epistemological frame. On the one hand it has a strong multidisciplinary aspect, and on the other hand the importance of empirical component through computational exploration methods make this approach typical of Complex Systems science  [59].
Acknowledgements Results obtained in this paper were computed on the vo. complex-system. euvirtual organization of the  European Grid Infrastructure  ( http://www. egi. eu ). We thank the  European Grid Infrastructure  and its supporting  National Grid Initiatives (France-Grilles in particular)  for providing the technical support and infrastructure. The author thanks the organizers and participants of  Journées de Rochebrune 2016  for which the work was originally conceived. The author also thanks  E. Marandon  ( L2 Technologies ) for the original idea of correlated financial signals.
Availability of data and material All data and code used in this study, including simulation results, are openly available on git repositories at https://github. com/JusteRaimbault/CityNetwork/tree/master/Models/Synthetic and https://github. com/JusteRaimbault/SyntheticAsset. Large dataset are available on the dataverse repository at http://dx. doi. org/10. 7910/DVN/UIHBC7.
Single-cell transcriptomic assays have enabled the de novo reconstruction of lineage differentiation trajectories, along with the characterization of cellular heterogeneity and state transitions. Several methods have been developed for reconstructing developmental trajectories from single-cell transcriptomic data, but efforts on analyzing single-cell epigenomic data and on trajectory visualization remain limited. Here we present STREAM, an interactive pipeline capable of disentangling and visualizing complex branching trajectories from both single-cell transcriptomic and epigenomic data. We have tested STREAM on several synthetic and real datasets generated with different single-cell technologies. We further demonstrate its utility for understanding myoblast differentiation and disentangling known heterogeneity in hematopoiesis for different organisms.
Although many computational methods have been developed for this task, these methods have been designed for analyzing single-cell transcriptomic data only. On the other hand, no endto-end and open-source software solution exists, to our knowledge, to characterize epigenomic data such as single-cell chromatin-accessibility data (scATAC-seq). In addition, efforts on trajectory visualization remain limited. Current methods mainly focus on displaying single cells or clusters (or stable states) along the pseudotime, which makes it difficult to study subpopulation composition and its continuous transition along trajectories, especially for large datasets. Also, no trajectory inference method provides the possibility to map new cells to previously obtained reference trajectories without pooling cells and re-computing trajectories.
In the MLLE embedding, STREAM infers cellular trajectories using an Elastic Principal Graph implementation called ElPiGraph  26. ElPiGraph is a completely redesigned algorithm for the previously introduced elastic principal graph optimization  [27] [28] [29]  based on the use of elastic matrix Laplacian, trimmed mean square error, explicit control of topological complexity and scalability to millions of points on an ordinary laptop. In STREAM, the ElPiGraph was further developed to integrate a new heuristic graph structure seeding to learn principal graphs in high dimensions with several problem-specific topological graph grammar rules optimized for single-cell trajectory inference (Methods, Supplementary Fig. 1 ). To illustrate STREAM, we first reanalyzed a published scRNAseq dataset from Nestorowa et al.
This allows easy re-organization of the tree using a breadth-first search to obtain a subway map plot that better represents pseudotime progression from a selected starting node (Fig. 1c ). Although these visualizations capture trajectories and branching points, they are not informative on the density and composition of cell types along pseudotime, a common challenge when modeling large datasets. In fact, density information, an aspect overlooked by existing methods, is important to track not only how the composition of subpopulations changes along a trajectory but also how they get partitioned around branching events. To solve this problem, we develop a trajectory visualization method called the stream plot. This compact representation summarizes cellular developmental trajectories, user-defined annotations, branching points, cell density, and gene expression patterns (Fig.
This can be accomplished since all the steps are deterministic and the MLLE dimensionality reduction provides an explicit function that maps points from the original space to the target subspace. A reference structure is important when studying genetic or epigenetic perturbation, or when comparing different conditions (for example normal and cancer, response to stimuli, etc. ). In fact, the mapping procedure not only avoids pooling old and new cells and re-computing trajectories from scratch (a computationallyintensive operation), but more importantly does not distort the original structure. Keeping the original structure unperturbed is important to avoid incorrect interpretations of the reference pseudotime. To illustrate the utility of the mapping feature, we applied STREAM to analyze a published scRNA-seq dataset from Olsson et al.
left). A key result of this study is the discovery of metastable mixed-lineage states and the presence of co-expressed genes at single-cell level from competing lineages. The authors suggest that these metastable states are important in cell-fate decisions and that master regulators play a key role in this process. In fact, they uncovered and validated two key transcription factors, i. e. , Gfi1 and Irf8, that are co-expressed in a subpopulation and are shown to be important for the commitment to neutrophils or macrophages. Importantly, this dataset contains, in addition to wild-type data, genetic perturbations of those two key regulators. Using the wild-type data, STREAM unbiasedly and correctly reconstructed the cell lineage hierarchy as shown by inspection of the labels proposed in the original study (either cell surface markers or predicted lineages) (Fig.
2a right ). Starting from hematopoietic stem cell/progenitor (HSCP), cell lineage bifurcates into an erythrocytic branch (which contains megakaryocytic (Meg) and erythrocytic (Eryth) cells) and into a multi-lineage primed (Multi-Lin) branch. Multi-Lin cell lineage further separates into the granulocytic (Gran) branch and monocytic (Mono) branch. The hierarchical progression can be easily visualized by our proposed 2D visualizations: subway map and stream plots (Fig. 2b ). Importantly, STREAM precisely recovers the bifurcation event from Multi-lineage to Mono and Gran as shown in the original study within the wild-type GMP cellular population (Fig. 2b, c ), whereas the proposed Monocle2 analysis of the same dataset  20  incorrectly assigns Multi-Lin cells to a very short erythroid branch. Furthermore, Monocle2 branch lengths are overall very diverse and distorted in their hierarchical representation (F E branch in Fig.
2f, g ). Our predictions are validated by the original study where the authors used GMP cells with inducible expression and GFP reporters for Gfi1 and Irf8. Irf8 loss led to cells that differentiated toward granulocyte. Conversely, Gfi1 loss led the cells to differentiate toward monocytes. Interestingly they showed that cells from the hematopoietic stem cell/progenitor and myeloid compartments are trapped with the double knockouts of Irf8 and Gfi1, and in fact, are rarely differentiating towards monocytes or granulocytes. These results are in full agreement with our unbiased analysis. In addition, compared to the Monocle2 analysis of this dataset, our reference structure can be fixed to recapitulate only the wild-type cells and is not influenced by the fmapping of new cells, whereas the Monocle2 analysis requires to recompute trajectories (compare A,B with C, D in Supplementary Fig.