dataesr/scientific-paragraphs-categorization huggingface dataset:
Tsimane' language and community The Tsimane' are an indigenous group residing in the forest, riverine, and savanna areas of lowland Bolivia, in the Beni department (for a map, see Fig  1  of  [58] ). While they are experiencing a fast market integration into broader Bolivian society (increasing town visits, schooling, knowledge of Spanish, wage earnings and consumption of market goods;  [59] ), almost all of the food the Tsimane' consume comes from horticulture, fishing, and hunting. There are about 16,000 individuals (about half of them being under 15 years of age) living in about 90 villages  [59]. Brief summaries of various aspects of Tsimane' history and culture can be read in  [60]. Data were collected in two Tsimane' villages. Both villages had a school, and there was school every weekday when it did not rain while we visited.
Methods This paper was written using the rticles package  [66]  and RMarkDown  [67]  in R  [68]  running on Rstudio  [69]. It can be downloaded and reproduced using the data also available from the Open Science Framework, https://osf. io/bhg94. We had created a pre-registration for this study prior to departing for Bolivia (see Supplementary document https://osf. io/b3sfa/, section 1, for more information on the field work setting). However, our methods deviated considerably from the pre-registered ones starting with the procedure: It soon became obvious that eliciting non-word repetitions by playing back sounds from a computer, one person at a time, was not feasible, and would yield absolutely no data from children. This lead us to test people in groups, which resulted in data structured differently from the data originally expected.
Participants were not compensated for participating in the NWR game specifically, and thus there was no gift to promote higher NWR scores. Instead, participants were compensated in lump sum for the battery of protocols they participated in. Participants consented verbally, for themselves and for their children. This verbal consent/assent procedure was approved by the above-mentioned IRB. Groups were typically formed by members of one family (mother and one or more children), and sometimes members of two families, in addition to one or both Tsimane' research assistants and another investigator (CS), and sometimes additional children who followed us on visits or visited our camp. Data were collected from seven groups of between three and nine participants, of which at least one participant was always one of the Tsimane' research assistants.
Although children as young as one year of age were present, they seldom attempted to repeat the items. The youngest children in a given group almost always refused to talk, or if they did repeat some of the shorter items, they eventually stopped as items got longer, and thus most data come from older children and female adult family members. We have data from a total of 16 children (mean age 7. 07, range 1-12 years, 11 female) and 13 adults (mean age 32. 46, range 15-60 years, 11 female). We obtained age estimates by asking participants their date of birth and crossing the information provided with the census information that the Tsimane' Health and Life History Project has been updating for over a decade  [59]. We also asked what the highest grade of schooling completed was. Tsimane' children typically begin first grade at 5 years of age.
For this analysis, we included only participants who provided multiple datapoints on the same non-words, and inspected only scores on these precise non-words (which is therefore a subset of all the non-words, and does not generalize to the whole set of non-words since phonologically complex words may be repeated more than phonologically simple ones). A paired t-test between first and other attempts across participants revealed no significant difference, t(11) = 1. 12, p = 0. 29. Additionally, a logistic mixed model predicting individual attempts' score, declaring as fixed factors age group (child, adult) and whether it was the first or subsequent attempts, with target and participant ID as random factors, did not reveal a significant estimate for attemt (ß = -0. 27, SE = 0. 28). Since there was no significant difference, and it is unclear that previous work relied exclusively on first attempts, results rely on all the data (including repeated attempts).
Discussion This paper reports on exploratory analyses carried out on data collected in the context of a non-word repetition group game. We aimed to contribute to two bodies of literature, one on the relevance of early input for the development of phonological processing, and another on the role of literacy and/or schooling. Some of these analyses involve comparisons between overall NWR scores found here and that found in previous work; others rely on comparisons between subgroups of participants (e. g. , self-reported readers versus non-readers). We had hoped to contribute to the study of individual variation, particularly in terms of differences due to age, schooling, and self-reported literacy. However, we discovered upon analyzing the data that split-half reliability levels were very poor for children, and relatively poor for adults.
Conclusions To conclude, we collected non-word repetition data from Tsimane' children and adults to contribute to two lines of research. First, we found descriptively lower levels of NWR scores in both children and adults compared to previous work, which is consistent with the hypothesis that there are long-term effects on phonological processing of experiencing low levels of directed input in infancy. Second, we found some evidence that literacy and/or schooling increases NWR scores in non-word repetition, although these results should be interpreted with caution given the small sample size. Finally, we did not find strong age effects within childhood or in a comparison of children and adults. Altough we believe these data are important to share because of the rarity with which populations like the Tsimane' are allowed participation in psycholinguistic research, we also pointed out limitations both in terms of the data collected and the comparisons against previous work.
Acknowledgments We are grateful to  Radhia Achheb ,  Michel Dutat ,  Vireack Ul , and  Catherine Urban  for logistical assistance in organizing the field trip; to  Angèle Barbedette  and  Xuan Nga Cao  for setting up analysis pipelines and constructing parsable versions of Tsimane' written materials which were crucial in the design of the items; to the three Tsimane' research assistants who helped us prepare and deliver the stimuli; and to all the Tsimane' individuals who participated in the study.
AC acknowledges financial and institutional support from  Agence Nationale de la Recherche  ( ANR-17-CE28-0007   LangAge ,  ANR-16-DATA-0004   ACLEW ,  ANR-14-CE30-0003   MechELex ,  ANR-17-EURE-0017 ) and the  J. S. McDonnell Foundation   Understanding Human Cognition Scholar Award. JS acknowledges IAST funding from the  French National Research Agency (ANR)  under grant  ANR-17-EURE-0010  ( Investissements d'Avenir program ). url ANR: https://anr. fr/url McDonnell Foundation: https:// www. jsmf. org/The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Multivariate pattern analysis (MVPA) has become vastly popular for analyzing functional neuroimaging data. At the group level, two main strategies are used in the literature. The standard one is hierarchical, combining the outcomes of within-subject decoding results in a second-level analysis. The alternative one, inter-subject pattern analysis, directly works at the group-level by using, e. g. a leave-one-subject-out cross-validation. This study provides a thorough comparison of these two group-level decoding schemes, using both a large number of artificial datasets where the size of the multivariate effect and the amount of inter-individual variability are parametrically controlled, as well as two real fMRI datasets comprising 15 and 39 subjects, respectively. We show that these two strategies uncover distinct significant regions with partial overlap, and that inter-subject pattern analysis is able to detect smaller effects and to facilitate the interpretation.
In this case, the decoding performance is assessed on data from new participants, i. e. participants who did not provide data for the training of the classifier (see e. g. [32, 17, 19, 20, 18, 10] ), ensuring that the nature of the information is consistent across all individuals of the population that was sampled for the experiment. This strategy takes several denominations in the literature such as across-, between-or inter-subject classification or subject-transfer decoding. We hereafter retain the name inter-subject pattern analysis (ISPA). In this paper, we describe a comparison of the results provided by these two classifier-based group-level decoding strategies with both artificial and real fMRI datasets, which, to the best of our knowledge, is the first of its kind. This experimental study was carefully designed to exclusively focus on the differences induced by the within-vs.
inter-subject nature of the decoding, i. e. by making all other steps of the analysis workflow strictly identical. We provide results for both two real fMRI datasets and a large number of artificial datasets where the characteristics of the data are parametrically controlled. This allows us to demonstrate that these strategies offer different detection power, with a clear advantage for the inter-subject scheme, but furthermore that they can provide results of different nature, for which we put forward a potential explanation supported by the results of our simulations on artificial data. The paper is orga-nized as follows. Section 2 describes our methodology, including our multivariate analysis pipeline for the two group-level strategies, as well as a description of the real datasets and the generative model of the artificial datasets.
Artificial data The first type of data we use to compare G-MVPA and ISPA is created artificially. We generate a large number of datasets in order to conduct numerous experiments and obtain robust results. Each dataset is composed of 21 subjects (for ISPA: 20 for training, 1 for testing), with data points in two classes labeled as Y = {+1, -1}, simulating a paradigm with two experimental conditions. For a given dataset, each subject s ∈ {1, 2,. , 21} provides 200 labeled observations, 100 per class. We denote the i-th observation and corresponding class label (x s i , y s i ), where x s i ∈ R 2 and y s i ∈ Y. The pattern x s i is created as x s i = cos θ s -sin θ s sin θ s cos θ s xs i , where • xs i is randomly drawn from a 2D Gaussian distribution, N(C + , Σ) and N(C -, Σ) if y s i = +1 or y s i = -1, respectively, which are defined by their centers C + = (+ d 2 , 0) and C -= (-d 2 , 0) , where d ∈ R + and their covariance matrix Σ, here fixed to 1 0 0 5 (see Supplementary Materials for results with other values of Σ); • θ s defines a rotation around the origin that is applied to all patterns of subject s; the value of θ s is randomly drawn from the Gaussian distribution N(0, Θ), where Θ defines the within-population variance.
Let X s = (x s i ) 200 i=1 and Y s = (y s i ) 200 i=1 be the set of patterns and labels for subject s. A full dataset D is defined by D = s=21 s=1 {X s , Y s }. The characteristics of such a dataset are in fact governed by two parameters: • d, which defines the distance between the point clouds of each of the two classes, i. e. the multivariate effect size; • Θ, which controls the amplitude of the rotation that can be applied to the data, separately for each subject: when Θ is small, all the θ s angles remain small, which means that the data of all subjects are similar; when Θ increases, the differences between subjects become larger; therefore, Θ quantifies the amount of inter-individual variability that exists within the group of 21 subjects for a given dataset. Figures 2a and 2b illustrate the influence of each of these two parameters.
fMRI data We also used two real fMRI datasets that were acquired at the Centre IRM-INT in Marseille, France. For both experiments, participants provided written informed consent in agreement with the local guidelines of the South Mediterranean ethics committee. In the first experiment (hereafter Dataset1), fifteen subjects participated in an investigation of the neural basis of cognitive control in the frontal lobe, largely reproducing the experimental procedure described in  [21]. Participants lying supine in the MRI scanner were presented with audiovisual stimuli that required a button response, with the right or left thumb. Four inter-stimulus intervals were used equally in a fully randomized order (1. 8, 3. 5, 5. 5, 7. 1 seconds), with an average of 4. 5 seconds over a session. Data was collected with a 3T Bruker Medspec 30/80 Avance scanner running ParaVision 3.
0. 2. Eight MRI acquisitions were performed. First, a field map using a double echo Flash sequence recorded distortions in the magnetic field. Six sessions with 60 trials each were recorded, each comprising 133 volumes (EPI sequence, isotropic resolution of 3 × 3 × 3 mm, TE of 30 ms, flip angle of 81. 6 • , field of view of 192 × 192 mm, 36 interleaved ascending axial slices acquired within the TR of 2400 ms) encompassing the whole brain parallel to the AC-PC plane. Finally, we acquired a high-resolution T1-weighted anatomical image of each participant (MPRAGE sequence, isotropic voxels of 1 × 1 × 1 mm, field of view of 256 × 256 × 180 mm, T R = 9. 4 ms, T E = 4. 424 ms). In the second experiment (Dataset2), thirty-nine subjects were scanned using a voice localizer paradigm, adapted from the one analyzed in  [29].
While in the scanner, the participants were asked to close their eyes while passively listening to a set of 144 audio stimuli, half of them being voice sounds, the other half being non-vocal. Most of the stimuli were taken from a database created for a previous study  [6] , while the others were extracted from copyright-free online databases. The paradigm was event-related, with inter-stimulus intervals randomly chosen between 4 and 5 seconds. The images were acquired on a 3T Prisma MRI scanner (Siemens, Eerlangen, Germany) with a 64-channels head coil. A pair of phase-reversed spin echo images was first acquired to estimate a map of the magnetic field. Then, a multi-band gradient echo-planar imaging (EPI) sequence with a factor of 5 was used to cover the whole brain and cerebellum with 60 slices during the TR of 955 ms, with an isotropic resolution of 2 × 2 × 2 mm, a TE of 35.
2 ms, a flip angle of 56 degrees and a field of view of 200 × 200 mm for each slice. A total of 792 volumes were acquired in a single run of 12 minutes and 36 seconds. Then, a high resolution 3D T1 image was acquired for each subject (isotropic voxel size 0. 8 mm 3  , T R = 2400 ms, T E = 2. 28 ms, field of view of 256 × 256 × 204. 8 mm). Dataset2 is part of the InterTVA data set  [1] , which is fully available online 1.
fMRI data analysis The two datasets were processed using the same sets of operations. The pre-processing steps were performed in SPM12 2. They included co-registration of the EPIs with the T1 anatomical image, correction of the image distortions using the field 1 https://openneuro. org/datasets/ds001771 2 https://www. fil. ion. ucl. ac. uk/spm/ maps, motion correction of the EPIs, construction of a populationspecific anatomical template using the DARTEL method, transformation of the DARTEL template into MNI space and warping of the EPIs into this template space. Then, a general linear model was set up with one regressor per trial, as well as other regressors of non interest such as motion parameters, following the least-squares-all approach described in  [26]. The estimation of the parameters of this model yielded a set of beta maps that was each associated with a given experimental trial.
The beta values contained in these maps allowed constructing the vectors that serve as inputs to the decoding algorithms, that therefore operate on single trials. We obtained 360 and 144 beta maps per subject for Dataset1 and Dataset2 respectively. No spatial smoothing was applied on these data for the results presented below (the results obtained with smoothing are provided as Supplementary Materials). For these real fMRI datasets, we performed a searchlight decoding analysis  [23] , which allows to map local multivariate effects by sliding a spherical window throughout the whole brain and performing independent decoding analyses within each sphere. For our experiments, we exploited the searchlight implementation available in nilearn 3 to allow obtaining the singlefold accuracy maps necessary to perform inference.
For Dataset1, the decoding task was to guess whether the participant had used his left vs. right thumb to answer during the trial corresponding to the activation pattern provided to classifier. For G-MVPA, the within-subject cross-validation followed a leave-two-sessionsout scheme. For Dataset2, the binary classification task consisted in deciphering whether the sound presented to the participant was vocal or non-vocal. For G-MVPA, because a single session was available, we used an 8-fold cross-validation. Finally, all experiments were repeated with five different values of the searchlight radius (r ∈ {4 mm, 6 mm, 8 mm, 10 mm, 12 mm}).
Classifiers, Statistical inference and performance evaluation In practice, we employed the logistic regression classification algorithm (with l 2 regularization and a regularization weight of C = 0. 1), as available in the scikit-learn 4  python module, for both artificial and real fMRI data. The logistic regression has been widely used in neuroimaging because it is a linear model that enables neuroscientific interpretation by examining the weights of the model, and because it provides results on par with state of the art methods while offering an appealing computational efficiency  [30]. In order to perform statistical inference at the group level, the common practice is to use a t-test on the decoding accuracies. Such a test assesses whether the null hypothesis of a chance-level average accuracy can be rejected, which would reveal the existence of a multivariate difference between conditions at the group level (note that, as detailed in 2.
This test assesses the significance of the average accuracy at the group-level in a non-parametric manner for all the experiments conducted in the present study, whether conducted with G-MVPA or ISPA. Furthermore, this choice allowed maintaining the computational cost at a reasonable level, a condition that other alternatives, e. g inspired by  [31] , would not have met (see the Supplementary Materials for a longer discussion on this matter). In practice, for the real fMRI experiments, we used the implementation offered in the SnPM toolbox  5  to analyse the within-subject (for G-MVPA) or the single-fold (for the inter-subject cross-validation of ISPA) accuracy maps, with 1000 permutations and a significance threshold (p < 0. 05, FWE corrected). For the simulations that used the artificial datasets, we used an in-house implementation of the equivalent permutation test (also available in our open source code; see 2.
3), with 1000 permutations and a threshold at p < 0. 05. Critically, it should be noted that the same number of samples were available for this statistical procedure when using G-MVPA or ISPA, as shown on Figure  1. In order to compare the group-level decoding results provided by G-MVPA and ISPA, we use the following set of metrics. For the artificial data, we generated 100 datasets at each of the 143 points of the two dimensional parameter space spanned by the two parameters d and Θ. For each of these datasets, we estimate the probability p to reject the null hypothesis of no group-level decoding. We then simply count the number of datasets for which this null hypothesis can be rejected, using the p < 0. 05 threshold, which we denote N G and N I for G-MVPA and ISPA, respectively. For the two real fMRI datasets, we examine the thresholded statistical map obtained for each experiment.
Results In this section, we present the results obtained when comparing G-MVPA and ISPA on both artificial and real fMRI datasets. With the artificial datasets, because we know the ground truth, we can unambiguously quantify the differences between the two strategies. Our focus is therefore on the characterization of the space spanned by the two parameters that control the characteristics of the data to evaluate which of these two strategies provides better detection power. For the real datasets, we do not have access to a ground truth. After having assessed the consistency of the obtained results with previously published work, we therefore focus on describing the differences between the statistical maps produced by G-MVPA and ISPA, examine the influence of the searchlight radius, and try to relate these results to the ones obtained on the artificial datasets.
Results for artificial datasets The results of the application of G-MVPA and ISPA on the 14300 datasets that were artificially created are summarized in Tables  1, 2  and 3. In order to facilitate grasping the results on this very large number of datasets, we proceed in two steps. First, we represent the two-dimensional parameter space spanned by d and Θ as a table where the columns and lines represent a given value for these two parameters, respectively. The values in these tables are the number of datasets N G and N I (out of the 100 datasets available for each cell) for which a significant group level decoding accuracy (p < 0. 05, permutation test) is obtained with G-MVPA (Table  1  ) or ISPA (Table  2 ). In order to analyse these two tables, we performed a multiple linear regression where d and Θ are the independent variables used to explain the number of significant results in each cell.
In Table  1 , the effect of ). This means that, as expected, the effect size d has an effect on the detection power of both G-MVPA and ISPA, i. e the smaller the effect size, the more difficult the detection. But the amount of inter-individual variability, here quantified by Θ, influences the detection capability of ISPA, but not the one of G-MVPA. This produces the rectangle-like area visible in green on Table  1  and the triangle-like area visible in red on Table  2. When the inter-individual variability is low, ISPA can detect significant effects even with very small effect sizes. When the variability increases, the detection power of ISPA decreases -i. e. for a given effect size, the number of datasets for which ISPA yields a significant result decreases, but the one of G-MVPA remains constant.
d is significant (F G-WS PA d = 941. 75, p G-WS PA d < 10 -10 ), but the effect of Θ is not (F G-WS PA Θ = 0. 83, p G-S PA Θ = 0. 36), while in Table 2, Secondly, in order to easily depict the compared behaviors of G-MVPA and ISPA, we overlapped the results of the two strategies into Table  3. In this third table, the blue cells indicate that N G 50 and N I 50 (i. e. that both G-MVPA and ISPA produce significant results in more than half of the 100 datasets), while the green and red regions contain cells where it is the case only for G-MVPA or ISPA respectively (i. e. N G 50 and N I < 50 in green cells; N I 50 and N G < 50 in red cells). Note that for completeness, we also used different values of this arbitrary threshold set at 50, which did not qualitatively change the nature of the results described below (hence these results are not shown).