dataesr/scientific-paragraphs-categorization huggingface dataset:
This sublexical variable provides information about the frequency of the sublexical spelling patterns that are composing a word. Previous studies have indeed found that low frequency spelling patterns are more likely to be misspelled and replaced by more frequent ones  [27]. • The increase of the log frequency of the less frequent bigram in misspelled strings with respect to that of the target word. This is the log frequency of the less frequent bigram in the misspelled string minus the log frequency of the less frequent bigram in the target word (thus this equals zero for all correct words). This regressor applies only to misspellings and has been shown to affect spelling performances  [27]. The correlations between the six regressors are reported in Table  2. Correlation coefficients between these regressors and the observed frequency of strings are reported in Table  3 , for the website database and the dictation data, and for correct spellings and misspellings separately.
In what concerns correct spellings, the only regressor having a significant effect is the target word log-frequency, which not surprisingly increases the frequency of correct responses for both the web database and the dictation data. However, the correlation between the word logfrequency and the frequency of correct spellings is significantly lower in the database (0. 235) than in the dictation data (0. 4767), according to Williams T 2 test: T 2 (97) = 3. 0951, p<0. 003. Another difference between the website and the dictation databases on correct spelling frequencies was observed in the correlations with string length. These correlations were not significant, however, they were in opposite directions and their difference was significant according to Williams T 2 test: T 2 (97) = 2. 01, p<0.
05. Consistent with previous observations  [27]  in the dictation data, word length tends to have a negative effect on the frequency of correct spellings (r = -0. 1039, i. e. , there were more correct spellings on short than on long words). However, surprisingly, in the website database, this effect tends to be positive (r = 0. 071). Although this effect itself is not significant, the significant difference between the website and the dictation data on this point certainly requires an explanation. In what concerns misspellings, we observed significant positive correlations between the observed string frequency and the increase of the log-frequency of the less frequent bigram, for both the database and the dictation data. In other words, misspellings tend to replace infrequent bigrams with more frequent ones, this being true in the two data sets.
We also found a significant negative correlation for the website database between string length and the observed string frequency, showing that the shortest misspellings tend to be more frequent than the longest ones. No other significant effect was observed for misspellings, and globally, no qualitative difference was detected between the two data sets in what concerns misspellings. To summarize, with the 6 regressors used in this first regression analysis, we found that observed spelling errors led globally to the same regression patterns in the two data sets. In both cases, misspellings tended to replace the less frequent bigrams of the target word with more frequent bigrams, a result that is consistent with those obtained in previous studies  [27]. Thus, on the basis of this first analysis, spelling errors and their distributions look reasonably similar in the website and in the dictation data, except that misspellings are globally less frequent in the website data.
Conversely, spelling performances were significantly more accurate in the website (about 71% correct) than in the dictation database (about 48% correct). Not surprisingly, in both data sets, the correct spelling frequency increases with word frequency. However, this relation is significantly weaker in the website database. Moreover, although the word length effect in correct spelling frequencies was not significant, the correlations between word length and correct spelling frequencies were significantly different between the two data sets. These discrepancies can certainly be explained by considering the situations leading a user to consult a dictionary website. We can indeed identify two main situations in which a user searches for the synonym of a word. In the first case, the user can have an approximate knowledge of a word that she/he planes to write in a document or to replace by a synonym.
If the word is short, then it is probably faster to type it, however, if it is a long word, one will probably prefer to copy and paste it. As a result, the probability of a spelling error tends to decrease (thus the accuracy increases) as word length increases, contrarily to what logically happens in other situations. If this is actually the case, it should be possible to detect a significant positive correlation between word length and spelling accuracy in a large sample of words from the website database. This issue will be considered in the next regression analysis.
Regression analysis 2 While the correspondence between spelling errors obtained from the dictionary website and those obtained in the spelling to dictation experiment seems generally good, systematic differences appeared between the two data sets on the frequencies of correct spellings. In particular, the difference on the word length effect provides interesting clues to understand the observed discrepancies. Studying this effect (i. e. , a positive correlation between word length and spelling accuracy) on a larger sample of words is therefore critical in order to test its reliability and to better characterize the task-dependent differences between the website and the dictation databases. In a previous large-scale study based on the spelling production of children, Leteet al. (  2008 ) have identified various factors influencing spelling accuracy.
Although children are not necessarily representative of the general population, it is of interest to see if some of the observed effects can be reproduced using the data from the dictionary website. As for the dictation experiment, comparing the patterns of correlations between the website database and previously reported results can also help us determining how to use the new big data database adequately in order to inform psychological theories of spelling production. To examine these questions, we selected a sample of 6567 words common to the website database and to the French MANULEX database  [28] , which provides a number of word characteristics that we are going to use as regressors hereafter. We selected a set of 8 regressors that were used in the study by  [27]  or that were available from the MANULEX database: • WF: Logarithm of the word frequency (plus 1) per million in books (from Lexique 3).
The word frequency positively influenced children's spelling accuracy in  [27] , as well as in the present study. • Len: Number of letters of the word. The word length negatively influenced children's spelling accuracy in  [27] , as in the present spelling to dictation experiment (although the correlation was not significant in the dictation experiment). The reverse (i. e. , a positive correlation) was observed for the website database (the correlation being also statistically non-significant at p =. 05). • CGP: Grapheme to phoneme correspondence-minimum consistency (from MANULEX). This is the consistency of the less consistent grapheme to phoneme correspondence in the word. This type of regressor had no clear effect on children's spelling accuracy in  [27]. • CPG: Phoneme to grapheme correspondence-minimum consistency (from MANULEX).
Discussion The goal of the present study was to determine if a large-scale database on spelling productions automatically collected from a dictionary website could be used to inform models and theories of written word production. By comparing the distribution of performances on correct and erroneous responses between the website database and a spelling to dictation database recorded under standard experimental conditions, we found strong similarities between the two databases regarding the distribution of errors but also significant differences regarding the proportion of correct responses, indicating the influence of task-specific factors. Regarding the distribution of spelling errors, it seems that the website database could provide useful empirical data that are both qualitatively and quantitatively consistent with the data collected in a standard spelling to dictation test.
In a regression analysis, we found that the generation of errors is indeed mainly constrained by sublexical factors and notably by the presence of low-frequency bigrams that could be replaced by higher frequency spelling patterns (see  [30] [31] [32] [33] [34]  for recent studies investigating the role of alternative factors accounting for spelling error production). Similarly, both databases produced approximately the same set of error forms (i. e. , out of the 653 recorded misspellings, 593 were observed in both databases). Therefore, for errors, task-dependent factors do not seem to change drastically spelling performances and we can confidently state that the distribution of errors in the large-scale website database can be used to constrain models of written word production. The comparison between the Web database and the lab dictation test also revealed that the two data sets share about 67.
We also found that significant differences appeared between the two data sets on the proportion of correct responses, the number of correct spellings being larger in the website database and the correlation with word frequency being twice larger in the dictation database. Regression analyses revealed a lower correlation between correct responses and word frequency, together with a reversed correlation with word length. These results are consistent with the idea that on many occasions, website users already have access to the target word and they are simply entering it in the search engine by typing (with possible errors) or by copy and paste (without error). This initial access to the target word is likely reducing the word frequency effect that is observed in the dictation database. Similarly, there is a greater probability of using copy and paste for the longest words than for the shortest ones, therefore reversing the length effect observed in the dictation database.
The second regression analysis is consistent with that hypothesis by also revealing a positive correlation between the proportion of correct responses and word length on a larger sample of words from the website database. The effect of the number of heterographic homophones was also found to be positively related to the proportion of correct responses, contrary to what has been reported in a previous study  [27]. As argued above, a possible explanation of this result also comes from the way users are entering their request on the website. Indeed, if an heterographic homophone (instead of the target word) is entered, then no error will be detected by the search engine since the homophone is also a word. This situation artificially increases the spelling accuracy score for words having many heterographic homophones.
The present set of results therefore suggest that length effects and effects of the number of heterographic homophones must be considered with caution, since they are possibly biased in the website database by the procedure related to data collection. Due to these task-dependent factors, effects can therefore be modulated even if previously reported effects that are of particular importance in modelling spelling processes are in fact clearly reproduced. This is the case of the word frequency effect and of the phoneme-to-grapheme correspondence consistency effect, whose coexistence characterizes "dual-route" models of spelling  [2]. So, as for any experimental paradigm, the use of large-scale website databases requires a fine-grained analysis of the task-dependent procedures that may generate qualitative bias in the collected data.
In the present study, we found that errors collected on the Web largely follow the distribution of errors collected in a standard spelling-to-dictation test suggesting that the same word production processes were engaged. These errors could then be used confidently to test the predictions of computational models of single word production for the entire set of existing words from a given language. Conversely, concerning the percentage of correct spellings, we now know from our analyses that word production on the Web can differ from a spelling-to-dictation test due to copy/paste procedures that will bias the resulting distribution of correct spellings. Unless we find a way to correct this bias, psychological theories of word production can therefore only benefit from this large-scale database on the distribution of errors.
Conclusion The use of big data in psychology requires the same task analysis as for any study in experimental psychology in order to adequately use these massive flows of information to inform psychological theories about the structure and dynamics of mental processes. We have shown that comparing the results of these large-scale databases with the ones of standard and controlled experimental paradigms is certainly a good way to identify these task-dependent factors that any theory needs to take into account. In the present situation, while the percentage of correct responses is certainly not adequate for studying written word production processes, spelling error distributions from the large-scale internet database appears not only to be suitable to constrain models of word production at the item level but also, to provide reliable and almost noise-free observations due to the extremely large number of data points.
Funding: AR was supported by the  BLRI Labex  ( ANR-11-LABX-0036 ), the  Institut Convergence ILCB  ( ANR-16-CONV-0002 ), and by the  CHUNKED ANR project  (# ANR-17-CE28-0013-02 ). SP was supported by grants from  UniversiteParis Descartes  and by the  Labex ''Empirical Foundations of Linguistics " ( ANR-10-LABX-0083 ). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
• We present how such queries can be answered efficiently exploiting the previously introduced BTSR-tree index. • To achieve greater savings in execution time by further reducing node accesses, we propose an enhanced variant of BTSR-tree, called SBTSR-tree, which additionally employs temporal segmentation in each node to derive tighter, more fine-grained time series bounds. • We experimentally evaluate our methods using real-world datasets from different application domains, showing that BTSR-tree can efficiently handle hybrid queries under local similarity search, while SBTSR-tree achieves even higher performance due to the additional temporal segmentation. The remainder of the paper is structured as follows. Section 2 reviews related work. Section 3 formally defines the problem. Section 4 presents how query evaluation under local time series similarity can be executed using the BTSR-tree, while Section 5 presents the enhanced SBTSR-tree.
, n} among all time series in set T as follows: B = {max T ∈T T 1 ,. , max T ∈T T n } B = {min T ∈T T 1 ,. , min T ∈T T n } (1) A BTSR-tree index is initialized as an R-tree  [9]  built on the spatial attributes of the given geolocated time series dataset, as depicted in the example of Figure  3. Besides MBRs, each node is enhanced to also store MBTSs, shown as colored strips per node in Figure  3c. This enables efficient pruning of the search space when evaluating hybrid queries combining time series similarity with spatial proximity. For each child, a node stores a pre-specified number of MBTSs. Each MBTS is calculated according to Eq. 1. Construction and maintenance of the BTSR-tree follow the procedures of the R-tree for data insertion, deletion and node splitting. Objects (i. e. , geolocated time series) are inserted into leaf nodes and any resulting changes are propagated upwards.
Datasets We use three real-world datasets (Table  1 ) selected from different application domains, containing diverse types of geolocated time series, as detailed below: UK historical crime data (Crime). Contains time series representing the temporal variation in the number of crime incidents reported across England and Wales over 76 months (December 2010-March 2017). We generated time series over a grid with cell size 200 meters applied on the original data  1. For each month, we counted incidents having their location within each cell. Flickr geotagged photos (Flickr). Contains time series data extracted from geolocated Flickr images between 2006 and 2013 over the entire planet 2. To get meaningful geolocated time series, we partitioned the space by a uniform grid of of 7200 × 3600 cells (each one spanning 0.
05 decimal degrees in each dimension) and counted the number of photos contained in every cell each month. We excluded empty cells (e. g. , in the oceans). Each time series conveys the visits pattern (in terms of number of photos taken per month) of that region over this period. NYC taxi dropoffs (Taxi). Contains time series extracted from yellow taxi rides in New York City during 2015. The original data 3  provide pick-up and drop-off locations, as well as corresponding timestamps for each ride. For each month, we generated time series by applying a uniform spatial grid over the entire city (cell side was 200 meters) and counting all drop-offs therein for each day of the week at the time granularity of one hour. Thus, we obtained the number of drop-offs for 24 × 7 time intervals in every cell, which essentially captures the weekly fluctuation of taxi destinations there.
Index and Query Parameters To evaluate the performance benefits observed in the experiments only based on pruning, we tuned the index parameters to fixed values. The (m) and maximum (M ) number of entries stored in each node are set to 40 and 100, respectively. For both BTSR-tree and SBTSR-tree, the number of MBTS to 10 and for SBTSR-tree, the number of segments s is also set to 10. The query parameters involve the spatial distance and local similarity thresholds, i. e. , ρ, , δ and k. The values of these parameters are set differently for each dataset, based on their characteristics; default values are shown in Table  1. The value of ρ is set relatively, by setting the covered area as a percentage of the total area. Similarly, is set as a percentage of the maximum difference between the observed values.
Evaluation Setting Each experiment is performed using a randomly selected workload of 100 queries for each dataset and we report the average response time. All indices are held in memory, while the leafs contain pointers to files with geolocated time series stored on disk. All methods were developed in Java. Tests were executed on a server with 4 CPUs, each containing 8 cores clocked at 2. 13GHz, and 256 GB RAM running Debian Linux.
Query Performance We compare the average per query execution time for all three queries using sweep line and checkpoint methods on BTSR-tree and the checkpoint method on SBTSR-tree. 6. 2. 1 Q rr (T q , ρ, , δ) Figure  9  illustrates the query performance for varying thresholds ρ and and the first column of Figure  10  for varying δ, on all three datasets. It is apparent that the SBTSR-tree with the checkpoint approach outperforms the rest in all cases. Its superior pruning power is attributed to the segmentation, which yields tighter bounds within the nodes and consequently less disk accesses. The sweep line and checkpoint methods over BTSR-tree perform similarly in all cases. Both methods access the same nodes, but the checkpoint approach needs to examine significantly less values across time in order to determine local similarities.
However, since all local similarity calculations take place inmemory, computation cost does not make a big difference, compared to the lesser node accesses required with the SBTSR-tree. More specifically, for the crime dataset, relaxing ρ (Figure  9b ) has a negative impact on all three methods as more nodes have to be accessed and pruning depends mostly on the value. SBTSR-tree increasingly outperforms the rest as ρ increases, due to its more aggressive pruning on local similarity. For the case of increasing (Figure  9c ), the result is the opposite, as this way the parameter is relaxed and more nodes get accessed. For very large values, pruning is solely based on spatial distance and all approaches perform similarly. Finally, increasing δ (Figure  10b ) also increases the difference in performance among the three approaches, while it also reduces the average query response time.
This is due to large numbers of subsequences qualifying for small δ values, resulting in more node accesses. As δ increases, pruning is more rapidly improved in the case of SBTSR-tree due to its tighter bounds. The results are similar but with larger differences for the Flickr dataset (Figures 9d, 9e and 10f). Intuitively, the less periodicity in a dataset, the more the benefit from segmentation; if the time series in the dataset exhibit periodicity, the bounds that will occur from applying k-means clustering on the whole sequences will be relatively tighter than otherwise. The Flickr dataset, due to its nature, is more random than the crime dataset, which justifies the larger differences. This explanation is also supported by the results for the taxi dataset, illustrated in Figures 9b, 9c and 10b.
Despite a similar behavior in varying all thresholds, the differences in average query response time among the different approaches are smaller than in the crime and Flickr datasets, due to the high daily periodicity of taxi drop-offs. 6. 2. 2 Q kr (T q , k, , δ) Figures 10c, 10g and 10k depict the results for the Q kr (T q , k, , δ) query for the three datasets. As k increases, more nodes have to be traversed in order to fetch the additional results, and the execution time increases for all methods. Nevertheless, SBTSR-tree still clearly outperforms the other two algorithms. Figure  9 : Query Q rr (T q , ρ, , δ) for varying ρ and. 6. 2. 3 Q rk (T q , k, ρ) Finally, Figures 10d, 10h and 10l depict the results for the Q rk (T q , k, ρ) query. In this case, the performance deterioration as k increases is less abrupt, especially for the crime dataset, as usually the top-k results are spatially closely located and are retrieved quickly.
Scalability We performed a scalability evaluation for all three queries using the Flickr-based synthetic datasets, again measuring the average query response time for the same query workload. The results for increasing dataset size (up to four times) are depicted in Figure  10. In all cases, the SBTSR-tree-based approach scales better, especially in the top-k queries (Figures  10i  and 10m ), where the larger difference observed in Figures 10g and 10h is further augmented.
Results A community challenge to assess network module identification methods. We developed a panel of diverse, human molecular networks for the challenge, including custom versions of two protein-protein interaction and a signaling network extracted from the STRING  14  , InWeb  15  and OmniPath  16  databases, a co-expression network inferred from 19,019 tissue samples from the Gene Expression Omnibus (GEO) repository  17  , a network of genetic dependencies derived from loss-of-function screens in 216 cancer cell lines  18, 19  and a homology-based network built from phylogenetic patterns across 138 eukaryotic species  20, 21  (Methods). We included different types of network, which also vary in their size and structural properties, to provide a heterogeneous benchmark resource (Fig. 1a ).