dataesr/scientific-paragraphs-categorization huggingface dataset:
brainmap. org). Importantly, this distribution was solely based on those experiments in BrainMap that report foci from healthy adults (i. e. , studies with pathological populations or children were excluded) and were coded as "normal mapping studies" (i. e. , intervention studies and group comparisons were excluded). Moreover, we restricted the analysis to peaks reflecting task-based activations, and discarded deactivations from the analysis. This resulted in approximately 7,200 eligible experiments at the time of analysis. For each experiment in each of the simulation datasets, we first determined the "number of subjects" by randomly drawing this parameter from the aforementioned 7,200 experiments. Independent of sampling the empirical subject numbers, we drew the "number of reported foci" for each experiment, again based on the distribution of this parameter among the normal mapping experiments in BrainMap  (Laird et al.
, 2011a; Laird et al. , 2009). Finally, for each experiment and independently from the other steps, the respective foci from the empirical distribution thereof in the BrainMap database were drawn. That is, for each experiment the number of subjects, the number of reported foci and the spatial distribution of foci in the simulation all corresponded to the distribution of neuroimaging literature stored in BrainMap, but were randomly reassembled by independent draws. Critically, for those experiments that were supposed to feature a "true" activation of the target-location, we replaced one of the foci by a coordinate that was located at -42 / 32 / 26 (the coordinates of the simulated effect of interest) with a random Gaussian displacement corresponding to the spread as described above. We than conducted state-of-the-art ALE analyses on the ensuing 122,500 datasets and performed statistical inference on significant convergence using voxel-and cluster-level FWE corrected at p<0.
05  (Eickhoff et al. , 2012) , voxel-level FDR corrected at p<0. 05  (Laird et al. , 2005)  and uncorrected p<0. 001 with an additional extent-threshold of 200 mm 3. All simulations were run using in-house MATLAB scripts implementing the ALE algorithm as described in the respective publications  (Eickhoff et al. , 2009 , 2012 , Turkeltaub et al. , 2012).
In order to remedy this unsatisfactory situation and to provide quantitative guidelines on the minimal number of experiments needed for thorough ALE analyses, we quantified the contribution of the most dominant individual experiments to the significant clusters under different thresholding conditions. Importantly, this analysis was based on the "true" location of the effect according to the structure of the BrainMap database  (Langner et al. , 2014). For each cluster that survived statistical thresholding, we computed the fraction of the ALE value that was accounted for by the one and two, respectively, most dominant experiments and, as well as the fraction accounted by the two most dominant experiments (Figure  3 ). This was computed as the ratio of the ALE values with and without the respective experiment.
Susceptibility to spurious convergence In a next step, we assessed the average (across random realizations of the same simulated ALE analysis) number of clusters of significant convergence outside of the target location. We emphasize that these are not equivalent to false positives in a strict sense, which would be controlled through correction for multiple comparisons. Rather, they reflect incidental convergence through the structure of the BrainMap database  (Langner et al. , 2014). Such convergence arises from the fact that the frequency of observed activations in neuroimaging experiments and hence also BrainMap is not homogeneous across the brain. Some brain regions like the anterior insula  (Kurth et al. , 2010; Yarkoni et al. , 2011)  and the posteriormedial frontal cortex  (Muller et al.
In other words, only relatively consistent effects may be detected with sufficient power using the recommended minimum number of experiments. In turn, 80% power to detect an effect size of 0. 2, i. e. , a finding that is present in one out of five experiments, is only present when more than 30 experiments are included in the ALE analysis, given cluster-level FWE thresholding. This evidently raises the question as to which effect sizes may reasonably be expected for ALE analyses. While this depends on the subject under investigation, we provide some first insight into this matter in figure  8  illustrating the "effect sizes" for the real-life ALE analyses. Here we used the same datasets as in Figure  1 , and combined them with the sample-size calculations for cluster-level FWE inference. It may be noted, that strong effects such as 40% or more of a dataset showing a particular effect are rare, while "effect sizes" of 0.
Conceptual considerations The present empirical simulation study based on the BrainMap database provides a quantitative assessment of the statistical properties of ALE analyses. These parameters for the simulations, in particular the assumed spatial spread around a "true" location were based on a large number of stored meta-analyses and moreover converged nicely between handcoded and automatically generated datasets. Consequently, we feel confident, that the simulations reflect real-world situations that researchers will encounter when performing future ALE analyses in everyday research practice. In that context, we need to acknowledge, however, that the true spread around a hypothetical location of activation must remain unknown, as we can only provide estimates based on databased empirical observations.
Moreover, by using a resampled dataset from the BrainMap database to construct the simulations, we ensured that all parameters that otherwise would have to be set manually reflect the distribution thereof in a large sample of published neuroimaging studies. This in particular pertains to the number of subjects and reported number of foci in each experiment as well as the localization of the "noise" foci. We would thus argue, that our results should represent a realistic reflection of situations that are encountered when conducting metaanalyses and should therefore generalize to real-life settings.
• We would strongly discourage the use of voxel-wise false-discovery rate (FDR) thresholding in the context of ALE meta-analyses. As previously pointed out, voxel-wise FDR correction, the type considered here and almost exclusively used in ALE analysis is not appropriate for inference on topological features such as regions of significant convergence of a smooth dataset  (Chumbley and Friston, 2009). The present investigation corroborates this notion by showing that voxel-level FDR correction entails both relatively low sensitivity and a high susceptibility to spurious, false positive findings. Moreover, it also highlights another negative property of FDR thresholding, namely dependence of the latter on the strength of true convergence in other parts of the brain. If there are regions in the image where a strong effect is present, other regions are most likely to be declared significant as well as compared to a situation where the former is not present  (Genovese et al.
How many experiments are needed for an ALE analysis? There is no single right answer to this question. Each topic and hence each dataset will have its idiosyncrasies with respect to many potentially influencing parameters, such as the heterogeneity of the included experiments, the variability of peak locations in these experiments, the average number of subjects and so on. Furthermore, there are two different levels to this question. Namely, how many experiments are needed to avoid results that are largely driven by one experiment and how many experiments are needed to have sufficient statistical power for detecting less consistent effects. The current simulation study addressed both aspects based on a large dataset that should be well representative for situations encountered in current neuroimaging meta-analyses and yielded a rather clear answer to at least the first question.
Based on the obtained results, we would make a strong case for including at least 17 experiments into an ALE meta-analysis to control the influence of any individual experiment. This recommendation is based on the observation that, when controlling for multiple comparisons using cluster-level family wise error correction, from this size on, the average contribution of the most dominant experiment to any above-threshold cluster is less than half and the two most dominant experiments contribute on average less than 80%. While the absolute numbers, in particular the criterion of 50% contribution for the most dominant experiment, may be disputed, this recommendation is well in line with our empirical evidence across many ALE analyses on a large range of topics. In particular, in line with the data summarized in figure  3  we observed that in analyses involving less than 10 experiments, it becomes very likely that results are largely driven by a singly experiment.
This is in particular the case if there is a marked heterogeneity in the number of subjects involved in the different experiments. More specifically, this is quite likely to happen if one experiment is based on a substantially larger N than the remaining ones and hence features a tighter Gaussian leading to higher voxel-wise probabilities around the reported foci  (Eickhoff et al. , 2009b). In this case, even a relatively minor contribution from a second experiment will almost certainly move the ensuing ALE scores above the significance threshold. From the current simulations, however, it became clear that previous recommendation "at least 10-15 experiments should be included in an ALE meta-analyses" was still somewhat optimistic and needs to be adjusted upwardly closer to 20. We acknowledge that a robust approach to ensure that results are not driven by any one experiment may be provided by the use of jackknife analyses, that is, re-computation of the ALE analysis while in turn leaving out each experiment from the dataset.
There is no question that results obtained throughout all realizations of the jackknife procedure must be considered robustly present in the dataset and not dependent on any one experiment  (Amanzio et al. , 2013; Palaniyappan et al. , 2012). There are, however, two downsides to using this statistical procedure as primary inferential tool, which became most apparent when considering the two extreme cases, i. e. , meta-analyses involving very few and very many experiments. In the latter case, repeating the full analysis, including the calculation of the cluster-size null-distribution as many times as there are experiments will be extremely computationally expensive and hence impractical. Moreover, our simulations also indicated, that in these cases jackknife analyses will not really be necessary, given that already in the case of 30 experiments the influence of any one or two experiments becomes quite limited.
Relevance for meta-analytic co-activation modeling In addition to topic-based meta-analyses, i. e. , studies integrating previous neuroimaging findings on a particular type of mental process or task, there is a growing interest in using meta-analytic methods for the assessment of task-based interaction patterns  (Eickhoff et al. , 2010; Fox et al. , 2014; Laird et al. , 2013). The fundamental idea behind these types of location-based meta-analyses is to assess which regions are co-active above chance given activation in a predefined region of interest as a marker of functional connectivity between them  (Robinson et al. , 2010; Rottschy et al. , 2013). Practically, this is usually implemented by filtering a database such as BrainMap or Neurosynth  (Yarkoni et al. , 2011)  to only retrieve experiments that feature at least one focus of activation in the seed region.
Likewise, several different methods for statistical inference, including cluster-and voxellevel FWE correction, false-discovery rate thresholding and also uncorrected inference combined with extent thresholds, have been proposed and frequently used in the context of ALE analyses without informed recommendation on which one is most appropriate. In the present paper, we characterized the behavior of ALE analyses and addressed the two questions raised in the last paragraph by means of >120,000 datasets simulated using realistic parameters and distributions and assessed using the current ALE algorithm. From the analyses of these simulated datasets, which should be well representative of those that may be encountered in future ALE projects, we formulate two main recommendations. Evaluation of the activation spread to be used in the simulations.
We assessed 15 hand-coded datasets for topic-based ALE meta-analyses (left side) as well as 105 datasets defined by combinations of the Behavioral Domain and Paradigm class meta-data from BrainMap (right side). After identifying the principal significant peaks in the ALE meta-analysis map, we computed the standard deviation of foci that contributed to that peak. In spite of the differences between both sets of meta-analyses with respect, they show homogeneity with the contributing experiments with a similar concentration (3-4 mm standard deviation) in each case (upper panels). Importantly, this spread seems independent of the number of experiments that constitute the respective meta-analyses (lower panels). Based on these data, the 500 simulations used spreads of 2 mm (50 times), 3 mm (200 times), 4 mm (200 times) and 5 mm (50 times) around the true location of the simulated effect.
To quantify the empirical observation that significant effects may be largely driven by a single experiment if the total number of experiments is relatively low and hence to provide quantitative guidelines on the minimal number of experiments needed for valid ALE analyses, we quantified the number of experiments contributing to the significant clusters under different thresholding methods. This analysis is not based on the "true" location of the effect but rather on those at which random convergence happened through the structure of the BrainMap database. For each of these additional clusters, surviving statistical thresholding, we computed the fraction of the ALE value that was accounted for by the most dominant (top panel) and two most dominant (lower panel) experiments. The light lines denote the average across the different number of experiments activating the "true" location and illustrate the robustness of these findings.
It becomes apparent that the cluster size increases strongly with the number of experiments activating the "true" location. In turn, clusters become smaller when the total number of experiments increases given the same number of experiments activating the target location. Finally, we note that FDR and in particular voxellevel FWE thresholding yields much smaller clusters than cluster-level FWE thresholding. Average number of additional clusters of significant convergence outside of the "true", i. e. , target location in relationship to the total number of experiments, and the number of experiments activating the target location and the significance thresholding. Given that the distribution of the entire BrainMap database is known, these analyses allow quantifying deviations from the ground truth, similar to false positive findings.
In addition, we note that voxel-level FDR thresholding combines a low sensitivity (cf. Figure  4 ) with a high potential for false positive or spurious findings especially when there is a strong true effect (cf. figure  6 ). Sample size calculations for ALE meta-analyses assuming a desired power of 80% given different "effect sizes" (proportion of the experiments in the underlying population showing an effect at a given location) for each of the four assessed thresholding approaches. Illustration of "effect sizes" (proportion of the experiments in the underlying population showing an effect at a given location) found for real-life ALE analyses. Here we used the same datasets as in Figure  1  and combined them with the sample-size calculations for cluster-level FWE inference. Red lines correspond to peaks from the hand-coded datasets for topic-based ALE meta-analyses and blue lines to the datasets defined by combinations of the Behavioral Domain and Paradigm class meta-data in BrainMap.
Acknowledgements This study was supported by the  Deutsche Forschungsgemeinschaft (DFG ,  EI 816/4-1 ,  LA 3071/3-1 ;  EI 816/6-1 ), the  National Institute of Mental Health  ( R01-MH074457 ), the  Helmholtz Portfolio Theme "Supercomputing and Modeling for the Human Brain " and the  European Union Seventh Framework Programme  ( FP7/2007(FP7/ -2013 ) ) under grant agreement no. 604102 ) Appendix: Behavioral domains and paradigm classes for the BrainMap generated datasets BD: Emotion / PC: Face Monitor/Discrimination -> 90 Experiments BD: Emotion / PC:  Cued Explicit Recognition -> 35 Experiments BD: Emotion / PC: Passive Viewing -> 80 Experiments BD: Emotion / PC: Visuospatial Attention -> 29 Experiments BD: Emotion / PC: Emotion Induction -> 136 Experiments BD: Emotion / PC: Encoding -> 38 Experiments BD: Emotion / PC: Film Viewing -> 46 Experiments BD: Emotion / PC: Task Switching -> 48 Experiments BD: Emotion / PC: Music Comprehension/Production -> 22 Experiments BD: Emotion / PC: Theory of Mind -> 26 Experiments BD: Emotion / PC: Affective Pictures -> 40 Experiments BD: Emotion / PC: Deception -> 46 Experiments BD: Emotion / PC: Delay Discounting -> 39 Experiments BD: Cognition.
Repeats and Low-Complexity Sequences Determination of repeat sequences involved first the calculation of de novo repeat families followed by assessment of known repeat elements in a second step. Families of previously unknown interspersed repeat elements were identified by RepeatScout  (Price et al. 2005). Repeat families were included when they comprised more than ten repeats and when their consensus sequence length was longer than 50 bp. Additionally, low complexity and simple sequence repeats were determined with the tools NSEG  (Wootton and Federhen 1993)  and Tandem-Repeat-Finder  (Benson 1999)  that are part of the RepeatScout procedure and remove them from the interspersed repeat library. The RepBase database  (Jurka et al. 2005 ) was used to detect previously published families of transposable elements, pseudogenes, and retroviruses.
Analyses of Divergence and Synteny The newly sequenced SscI8 strain was aligned with the scaffolds from the 2014001 strain and with the chromosome sequences of the Ssc39 strain using TBA  (Blanchette et al. 2004). The code of the blastZwrapper program of TBA was modified in order to use LastZ instead of the deprecated BLASTZ program. The resulting alignment was further processed using MafFilter  (Dutheil et al. 2014 ) in order to discard ambiguously aligned regions. A window of 10 nt was slid by 1 nt along the genome alignment and windows with more than three gaps were discarded. From the resulting alignment blocks, 883 nonoverlapping windows of 10 kb without synteny breaks were generated and pairwise nucleotide differences were computed using MafFilter. Synteny was assessed at the nucleotide level, using all contig/chromosomal sequences from S.
Annotation of CSEPs SignalP version 4. 1 was used for signal peptide prediction and ProtComp 9. 0 (online version) for localization prediction (http://linux1. softberry. com/berry. phtml). We used two prediction schemes. First, we considered all genes predicted to be secreted by SignalP (that is, genes with a signal peptide) as encoding CSEPs. In addition, genes predicted to be extracellular by ProtComp were also considered as encoding CSEPs in the absence of predicted signal peptide, as these could be nonconventionally secreted proteins. A second, more stringent prediction was derived by 1) excluding genes where ProtComp confidently predicted a nonextracellular location (score higher or equal to 8) and 2) excluding genes predicted to have an extracellular localization but with low confidence (score lower or equal to 8).
Phylogenetic Reconstruction All 44,716 protein sequences from the seven genomes of U. maydis, S. reilianum, U. hordei, S. scitamineum, Ma. globosa, P. flocculosa, and M. pennsylvanicum were gathered to perform an all-against-all BLASTp search, using the sequences both as query and database  (Altschul et al. 1990 ). The query results were given as input to the SiLiX program to build families of homologous proteins  (Miele et al. 2011). In order to perform a phylogenetic reconstruction, only families of orthologous genes can be used. We therefore tried various values of identity and coverage with SiLiX, and found that the maximum number of usable families was obtained when a minimum 35% identity on 40% of the average length was used for gathering two genes into a common family. With such parameters, SiLiX inferred 12,593 families, of which 1,764 had exactly one member in each species.
Gene Ontology Analysis We used the proteome of U. maydis to associate Gene Ontology (GO) terms to each gene family, using blast2GO  (Conesa et al. 2005). Thirteen thousand three hundred thirty-six annotations could be retrieved on 4,113 U. maydis genes. The topGO package  (Alexa et al. 2006 ) from Bioconductor  (Gentleman et al. 2004 ) was used to perform GO terms enrichment analyses. P values for each GO term  (Grossmann et al. 2007)  were computed using Fisher's classic test with the parent-child correction. The set of 1,846 families of one-to-one orthologs was tested against all 4,113 annotated genes and reported categories significant at the 5% level for the three ontologies "Biological Process", "Molecular Function," and "Cellular Component" (supplementary table S1 Supplementary Material online).
Building of CSEP Cluster Families All clustered genes were blasted against each other using BLASTp. Homology relationships were determined using BLASTp E value: two genes were considered to be homologous if they had at least one High-scoring Segment Pair (HSP) with an E value <1e-6. Clusters with at least one homologous gene pair were considered homologous and grouped in the same family. All genes from all clusters in a given family were further used as query against a BLASTp database of all proteincoding genes of the five smut species and the two related species, and additional, unclustered genes were reported. Each family of clusters was represented as a Circos diagram (version 0. 66)  (Krzywinski et al. 2009) , including homologous unclustered genes as well as flanking genes (these were omitted for family 3 as they compromised the readability of the resulting figure).
Synonymous rates were computed as the average over the three pairwise distances between S. scitamineum, S. reilianum, and U. maydis, using the PopGen module of BioPerl  (Stajich et al. 2002). Ks values could be computed for 5,358 families where sequences were not too divergent for the Jukes-Cantor correction to be applied. The comparison of gene categories was conducted using the nonparametric Kruskal-Wallis rank test. Post hoc tests were conducted using the agricolae package for R (de Mendiburu 2014) and significance groups were determined using an FDR of 1%, using the FDR method  (Benjamini and Hochberg 1995).
This latter category was further divided depending on whether the gene encode a CSEP or not. The distribution of distances for each category was compared using a Kruskal-Wallis rank test with Post hoc tests as implemented in the agricolae package for R (de Mendiburu 2014). An FDR of 1% was used for correcting for multiple testing. Only repeat classes represented by at least 100 occurrences in a genome were tested.