dataesr/scientific-paragraphs-categorization huggingface dataset:
Scalability and aggregation To evaluate the runtime overhead of our approach as data size increases, we use the TPC-H benchmark (v. 2. 18. 0). We generate datasets using the TPC-H data generator with scale factors of the form 10 /4-2 for  = {1, 2,. , 8}. We wrote a conversion tool to convert TPC-H data to RDF. Each row of a table in a TPC-H dataset corresponds to an -ary relation in RDF, which we reify according to the direct mapping scheme in Table  1. The numbers of triples in the resulting RDF datasets range from 1. 2M to 123M. The TPC-H query generator comes with 22 query templates with variables that are randomly instantiated to produce concrete queries. We omit query templates 4, 13, 15, 17, 18, 20, 21, and 22 from the benchmark because they include features outside the SPARQL fragment studied in this paper.
For each SQL query template we generate an equivalent SPARQL query template (base aggregate) as well as a corresponding version without aggregation (base non-aggregate). For each of these two queries, we then apply our rewriting approach and generate corresponding provenance queries: provenance aggregate and provenance non-aggregate. By considering two groups of base queries (base aggregate and base non-aggregate) we can study the overhead of computing provenance with and without aggregates. We generate 10 queries for each query template and each scale factor. We highlight that runtime grows linearly with data size, and that Virtuoso consistently outperforms Fuseki (less clearly in the case of template 1 though). Even though Virtuoso achieves overall lower runtimes than Fuseki, the overhead of aggregation in the base queries is more pronounced for Virtuoso.
The provenance overhead is larger for the non-monotonic minus queries. This is not surprising since we compute explanations for, on average, 3599 bindings in the support of the minus queries -even though the queries do not have actual results. The general provenance overhead is smaller for Fuseki, because this engine extends the intermediate results with provenance variables more efficiently than Virtuoso. These results show that our approach can compute how-provenance on top of a standard engine on large real-world datasets with a reasonable runtime overhead.
Comparison with other systems We also compare SPARQLprov with TripleProv  [43] , a specialized how-provenance solution for SPARQL, and GProM  [4] , a system to compute how-provenance for SQL queries. 5. 4. 1 Runtime evaluation. Since TripleProv is a pure main-memory storage engine, these experiments were executed using an inmemory file system to guarantee a fair comparison. For GProM, we translated the WatDiv queries into SQL and modeled the RDF data using one table per predicate as this translation yields the best performance for GProM (we use PostgreSQL as backend). We limit our evaluation to monotonic queries, i. e. , queries without optional and diff, since GProM and TripleProv build upon semirings and therefore not produce sound polynomials for such queries  [20]. For SPARQLprov we use Virtuoso with named graphs reification.
CONCLUSIONS We have proposed an approach to compute how-provenance polynomials for SPARQL query results via query rewriting. This makes our approach directly applicable to any engine with a SPARQL interface. Unlike existing solutions, our approach can compute howprovenance explanations that commute with homomorphisms for monotonic and non-monotonic queries. It can also provide lineage annotations for queries with aggregates. As shown by our experiments, our query rewriting layer deployed on standard RDF/SPARQL engines can compute provenance for large real-world datasets such as Wikidata. Furthermore, our evaluation on the TPC-H benchmark suggests that our approach is viable for computing provenance explanations in an OLAP setting where queries with aggregates are very common.
Participants Participants for Study 1 were 30 native speakers of English from Grand Valley State University who received course credit for their participation. Participants for Study 2 were 31 native speakers of German, residents of Berlin that were recruited from a participant database. Each participant was paid 10 Euro for their participation. Study 3 was carried out with 32 French native participants from the Aix-of Provence area. They received 10 Euro for compensation. All participants had a similar educational background and were students at the university.
Apparatus We collected eye movement data using the table-mounted EyeLink 1,000 eye tracking system for Study 1 (American English) and Study 3 (French speakers). The system sampled eye position data once every 2 ms and was facilitated by Experiment Builder software. For Study 2 (German speakers) eye movement data were recorded using the head-mounted EyeLink I eye tracking system, which sampled eye position data once every 4 ms. The calculations that we made for eye gaze measurements were adjusted based on the sampling rate differences. We used Experiment Builder software to facilitate the production task.
Director Materials The 48 individual objects used as stimuli were the same as those used in the Swets et al. (  2014 ) study, which were adopted from a database of color images that can be found at the website https://wiki. cnbc. cmu. edu/Objects (Rossion and Pourtois, 2004). These 48 objects were combined into larger arrays of three objects apiece and used as the stimuli to elicit the Directors' target utterances (e. g. , "The cat moves below the train and the wheel moves above the train"). All objects were real-world objects with high naming accuracy and low reaction times  (Rossion and Pourtois, 2004). The 48 individual objects were combined in various 3-object permutations to create 16 experimental arrays, with each of the three objects aligned horizontally across the screen. Each of the 16 arrays was created with both a control version and an experimental, contrast version, with 2 target and 6 filler arrays presented per matching round.
Region 2 was unrelated to advance planning of the possible contrast between the first and third objects). To obtain this measure, we created a script in Matlab. The script calculated how many eye tracking samples showed that the speaker was looking at Region 3 (the third object of the display) during the Initiation Time window for each trial. Recall that eye tracking samples were collected every 2 ms during English and French trials and every 4 s during German trials. Based on this procedure, we were able to calculate the proportion of eye tracking samples during the initiation time window of each trial that speakers were gazing at Region 3 of the display. This was possible by leveraging the fact that samples were taken by the eye tracking system on a known millisecond time scale. In our analyses of these proportions, the total number of samples was included in the models along with the proportion to make sure that overall initiation time was part of the weighting in the models.
RESULTS First, we present data from the individual differences measures of WM and processing speed from the three language samples. Then, we introduce data from the speech measures. The first speech measure we report is in regards to the way speakers modified the first noun phrase-whether they did so at all, whether they did so pre-nominally, and whether they did so post-nominally. The next two measures we will present are initiation time and pause frequency, which help to establish whether the three languages may have been planned differently in the present task. Then, we examine eye movements to investigate whether the scope of planning in each language differs among each language, and whether different individual cognitive capacities might be used differently in how that scope of planning varies.
(2014)  in English did not replicate with statistical significance in English, nor did the effect emerge in German (although the overall trend looks similar for both languages). It is of course possible that these are true null effects. Another possible explanation for the lack of these effects is that the present study was underpowered to detect them. It is also possible that we did not obtain a significant effect for those languages in the present study due to differences in how we obtained samples. In the  Swets et al. (2014)  study that the present studies are meant to partially replicate and extend, the researchers used a two-phase data collection process. In Phase I, they first collected WM data from a large (N = 92) sample of participants. Then, to ensure a wide range of WM scores in the speech planning task, they selectively recruited 26 participants for Phase II, which was the language production task itself.
Along these lines, a recent paper by  Amici et al. (2019)  offers evidence that the mastery of a particular language "exercises" certain cognitive capacities more than other languages by virtue of having different word order requirements. In that study, the researchers conducted short term memory and WM tests among speakers of consistently left-branching languages (i. e. , languages that require pre-nominal modification) and consistently right-branching languages (i. e. , languages that require post-nominal modification). The results showed that speakers of left-branching languages performed better on items that appeared early in serial order in a WM task than items that appeared later, while speakers of right-branching languages showed the opposite effect. It is possible that a similar direction of influence could be occurring in our data, but we did not set out to test that possibility, and there is little in the data that we collected to suggest it as a possibility.
FUNDING This work was supported by a grant from ( 01UG1411 ) from the  Ministry for Education and Research (Bundesministerium für Bildung und Forschung ) to SF, and by a grant from the  Agence Nationale de la Recherche (ANR)  to CP for the project  Representation and Planning of Prosody  ( ANR-14-CE30-0005-01 ). The research was further supported by grant  ANR-16-CONV-0002  (ILCB), and the  Excellence Initiative of Aix-Marseille University  (A * MIDEX).
Despite many cultural, methodological, and technical improvements, one of the major obstacle to results reproducibility remains the pervasive low statistical power. In response to this problem, a lot of attention has recently been drawn to sequential analyses. This type of procedure has been shown to be more efficient (to require less observations and therefore less resources) than classical fixed-N procedures. However, these procedures are submitted to both intrapersonal and interpersonal biases during data collection and data analysis. In this tutorial, we explain how automation can be used to prevent these biases. We show how to synchronise open and free experiment software programs with the Open Science Framework and how to automate sequential data analyses in R. This tutorial is intended to researchers with beginner experience with R but no previous experience with sequential analyses is required.
Therefore, we propose that all the steps of the sequential procedure (i. e. , data preparation and cleaning, outliers detection, data transformation, model assumptions checks, and data analysis) should be automated and performed in an incremental manner. The entire dataset can be continually reanalysed, including former outliers, so that the process follows the progressive incorporation of new observations. Put more formally, such a procedure should be able to take into account that an extreme observation at time t might not be extreme anymore at time t n , and should therefore be reincluded in subsequent analyses. The fact that this iterative procedure is automated should prevent the data analyst from the hazards that are commonly encountered during data manipulation (see previous section).
To illustrate this idea, we describe below how to perform transparent blind sequential analysis. We propose an example for two independent-groups comparisons (as in  Schönbrodt et al. , 2017). This tutorial covers all experimental steps from preregistration to results reporting. When several options are available for a specific step, we detail one in the manuscript and other possibilities in the supplementary materials. We provide a functional example of the procedure on the OSF (Open Science Framework) based on an emotional Stroop experiment. In order to describe the procedure, we use the idea of  Rouder (2016)  who took the perspective of his dog Kirby to explain Git and Github. Here we take the perspective of Lisa Loud, 7  who loves carrying out experiments but has probably never used automated sequential analysis before.
"Born-open" data The aim of this this tutorial is not to discuss the theoretical work necessary before carrying out an experi-ment. We will directly focus on the preparation of materials needed to collect data. This tutorial mainly deals with computerised experiments. 8 In this situation, users need to program an experiment to collect data. The logic proposed here is compatible with experiments programmed with Psy-choPy  (Peirce, 2007 (Peirce, , 2008;; Peirce et al. , 2019)  or OpenSesame  (Mathôt et al. , 2012). These software programs have the advantage of being free and able to communicate with the OSF. These two qualities are crucial for transparent procedures and easy sharing. We will use the possibility to link the software program with the OSF in order to propose an intuitive "born-open" data procedure  (Rouder, 2016).
Although OSF synchronisation tools are generally easy to install on Windows and Mac OS, it can be slightly more "complicated" on other operating systems (OS) such as Linux OS. Most users probably work on Windows and Mac OS. However, because Linux OSs are free and open we believe that they fit well with the open science philosophy. Consequently, we will provide minimal examples on how to use this tutorial on Ubuntu as a popular and easy-to-use Linux distribution. We propose a procedure using OpenSesame below. Procedures with PsychoPy can be found in the supplementary materials. OpenSesame probably allows the simplest synchronisation with the OSF. However, it is less flexible than PsychoPy for programming experiments because, unlike PsychoPy, it does not allow to access to a coder view. Direct OSF synchronisation is available with PsychoPy 2 but not with PsychoPy 3.
The example available on the OSF is based on PsychoPy 2 but has also been successfully tested with OpenSesame. Table  2  proposes a summary of the main strengths and weaknesses of each method. Programming the experiment for "born-open" data. We present how to program an experiment al- lowing automatic "born-open" data with OpenSesame, currently the easiest way to program automatic "bornopen" data. For more flexibility in experiment programming, PsychoPy options are described in the supplementary materials. OpenSesame is "a graphical experiment builder for the social sciences". It is free, open-source, and cross-platform. It features "a comprehensive and intuitive graphical user interface and supports Python scripting for complex tasks. " (Mathôt et al. , 2012). OSF integration is normally included by default for Windows and Mac OS.
Script preparation and piloting Performing sequential analyses requires strict experiment programming and data analysis preparation. Be-Figure  1. OSF log-in button in the main toolbar in OpenSesame cause data is continuously analysed during data collection, everything must be ready before collecting data. This can be seen as a disadvantage because there is a lot of work to be done before data collection. Indeed, analysing data after data collection allows us to delay several choices thus allowing to launch data collection more quickly. 10 However, this can also be considered as a huge advantage because there are no unexpected surprises after data collection. It is not possible to discover that something has not been recorded or that data is not in the appropriate format, or that data is more difficult to analyse than expected.
Preregistration Let's assume Lisa has successfully achieved script preparation and piloting. When everything is ready, she has everything she needs to preregister her study. Preregistration is very important in sequential testing because it forces Lisa to explicitly state her statistical criteria of interest and therefore announce when data collection will end. This is very useful in order to limit biases due to Lisa's degrees of freedom  (Lakens, 2014; Wicherts et al. , 2016). In addition to generic preregistration, the first important thing to preregister is the sequential data cleaning procedure. In this part, Lisa should describe how data will be handled before inferential statistical modelling. This can include, physiological signal processing, potential observation or participant removal criteria or any other data manipulation happening between data collection and modelling.
Automated blind data analysis Because Lisa has prepared everything needed for her procedure, she is able to automate data analysis and therefore to be blind to the details of the analysis while she is collecting data. Here is how she can proceed and how we proceeded in our example. We describe how Lisa can handle tasks-scheduling on a UNIX system (macOS and Linux) in this paragraph. Lisa can use the cronR package  (Wijffels, 2018a)  to schedule tasks in R. This package will be useful in order to retrieve data from the OSF and to analyse it. Lisa will have to create a task by running the appropriate script (main_script. R in our example) once before collecting the first observation. She will be able to decide how often she wants the script to run automatically. Lisa could also apply the same procedure on a Windows system thanks to the taskscheduleR package  (Wijffels, 2018b).
A short description on how to use this package can be found at: https://cran. r-project. org/web/packages/ taskscheduleR/vignettes/taskscheduleR. html. If Lisa chooses that the script should auto-run each hour, the main script will check data from the OSF each hour. This will be done with the osfr package  (Wolen & Hartgerink, 2019). In our example (Line 106 to 143 in main_script. R), we check if new data is available on the OSF and download it if it is the case. After some data formatting (lines 145 to 197), the script will run the sequential analysis (line 199 to 243). The main script calls the sequential_analyses. R script which contains the sequential analyses function. Lisa will also have to specify some parameters depending on the model she is interested in. At the end of the analysis, Lisa will receive an email telling her whether to stop or to continue data collection.
Reproducible reporting Lisa will stop data collection when she reaches her statistical criterion or her maximum affordable sample size. She will then be able to write a report describing her results. She can do it using RMarkdown  (Allaire et al. , 2019; Xie et al. , 2018)  in order to incorporate her results automatically from her R scripts into her report (e. g. , see  Bauer, 2018). Lisa could for instance use the R package papaja  (Aust & Barth, 2018)  which would allow her to write a reproducible APA manuscript with RMarkdown. Scientific writing with Rmarkdown has important benefits for any type of research design but would be even more valuable for sequential analyses.
Feasibility of the proposed protocol and limitations A graphical summary of the procedure is depicted in Figure  2. 11  All the tools needed to set it up are available for free to everyone. Using OpenSesame or PsychoPy is relatively straightforward and does not necessarily require coding skills. We proposed standard R scripts in order to automate sequential analysis. However, we concede that using this scripts requires minimal knowledge of R. Hence, we also propose a Shiny application available at https://barelysignificant. shinyapps. io/ blind_sequential_analyses/. With this application, Lisa would just have to specify all the important details of her analysis in boxes, from which the application generates the corresponding R script almost ready for sequential analysis. This application is meant to facilitate the creation of R scripts.
It automatically writes around 90% of the code Lisa would have to write to use such a sequential analysis procedure. However, it is almost certain that the produced R code would not work immediately. It would require some minor tweaking from her, such as checking the local path, making sure that the scripts and the data are in the same repository, adapting the data import step to specific properties of the data under consideration, and so on. The procedure proposed here only requires one computer. Hence, the implementation cost is rather low. This procedure is also well suited for multi-lab studies. Indeed, an experiment can be run at different places but data is automatically centralised on one platform and can be analyse automatically and sequentially by one single computer. The experiment we designed with PsychoPy 2 (see the supplementary material for more information) is specifically thought for multi-lab studies by automatically recording information about the computer which runs the experiment.
1, in contrast to the distinct identity by state clusters seen for 17q21. 31 haplotypes (Fig. S21 ). In total, these analyses suggest that the 8p23. 1 inversion arose independently on different genetic backgrounds, in contrast to the 17q2. 31 inverted haplotype which arose once and distinctly from the directly orientated haplotype. To reconstruct the evolutionary history of these balanced inversions, we inferred the underlying genealogical relationship among haplotypes (Methods). Because of the massive size of the 8p23. 1 inversion, we focused our analysis on a 100 kbp region located at the distal portion of this locus, revealing that the inferred trees have both global bootstrap and local, marginal tree support. This suggests that the underlying genealogical relationship among haplotypes is well recapitulated (Fig.
In both cases, we scaled an initial bin size by multiples of 2, 3, 4, 5, 10 and 20. This resulted in a redundant dataset with putative inversions detected per sample (Fig. S45 ) (mean and median: 144 per sample). To construct a nonredundant set of Strand-seq inversions, we merged and filtered all detected strand-state changes (putative inversions) in multiple stages as follows: We start out by cropping out each inversion flanks that overlap with highly identical SDs (≥98% identity) or gaps defined in GRCh38. Second, we iteratively merged inversion ranges with ≥50% reciprocal overlap until no more ranges could be merged. Such collapsed ranges were then subjected to a re-genotyping step using the 'genotypeRegions' function of the primatR package  (Porubsky et al. 2020). Each region in each sample was assigned a genotype: 'HET' -approximately equal mixture of plus and minus reads, 'HOM' -majority of minus reads, 'REF' -majority of plus (reference) reads and 'lowReads' -less than 20 reads in a region.
L1 annotation evaluation via simulations. We generated a simulated dataset, including 9,000 synthetic L1 inserts (Fig. S48 ), evenly distributed among the three possible insertion configurations: full-length (FL), 5' deleted, or inverted L1. All inserts derived from the same consensus L1 sequence were used as reference for L1 annotation. While the complete consensus was included for FL-L1, random breakpoint positions were sampled for the generation of truncated and inverted events. A single breakpoint located between 10 and 6,013 positions was sampled for 5' deletions, ensuring a minimum deletion and insertion size of 10 bp. To simulate 5' inversions, the inversion junction structure was randomly selected among three possible configurations: blunt, deletion, and duplication. Duplication and deletion sizes at the junctions were sampled between 1 and a maximum length of 100 bp and 500 bp, respectively.