dataesr/scientific-paragraphs-categorization huggingface dataset:
A multi-step framework capitalized on two independent, large datasets (n = 500 and n = 81) with a total of 36 typical neuroimaging tasks. Each of these two task batteries attempted to cover the diversity of human cognitive processes. In an unsupervised learning step (i. e. , naive to task labels), we first derived a small number of representative brain networks from these extensive neuroimaging resources. This first step yielded explicit models of possible network patterns with minimal statistical assumptions and without recourse to cognitive theory. In a supervised learning step (i. e. , based on task labels), the dimensionality-reduced neuroimaging data were then submitted to an 18-task classification problem. For each task, this second step determined a plausible combination of the modes of variation in fMRI signals from the brain.
Overview The analyses, results, and figures are divided into two different sections (see schematic in Fig  1 ). In the first section (Figs 2-5, entitled "network co-occurrence modeling"), we statistically tested whether neural activity patterns measured with fMRI in humans can be largely explained by changes in cohesive network units. The term "network" henceforth refers to spatiotemporal modes of variation extracted from time series of fMRI activity whose weighted linear combination sum up to whole-brain activity patterns  [6, 47, 48]. Whole-brain activity maps were expressed as a linear combination of 40 spatiotemporally coherent patterns (i. e. , network components). The distributed BOLD signals from voxel space were thus reduced to 40 component loadings in a network space. The network engagement pattern is shown to successfully classify and restore neural activity from the 18 psychological tasks across two large datasets.
In the second section (Figs 6-8, entitled "task-rest correspondence"), this possibility to compress neural activity maps into low-dimensional summaries of brain network involvements was used to revisit the relationship between the functional brain architecture during defined psychological tasks and unconstrained task-free mental activity. While the first section used a same task dataset (either HCP or ARCHI) for both unsupervised network discovery and supervised multitask classification, the applied second section probed diverging ways of deriving the network dictionary for subsequent classification. In particular, networks from i) the same task data dataset, ii) the respective other task dataset, iii) rest data, and iv) Gaussian noise were quantitatively assessed for their commonalities and differences in restoring task activity patterns.
Establishing an approach to quantifying the implication of network sets: Network co-occurrence modeling We first assessed the predictive accuracy of network co-occurrence models across methodological choices and datasets (Figs  2 3 4 5 ). Without using any rest maps, ICA and sparse PCA were applied to 50% of the task maps to then translate the remaining 50% of task activity maps into sets of network loadings. l1-penalized support vector machines (SVM) with C-hyperparameter tuning on ICA and sparse PCA loadings (i. e. , second 50% data) correctly detected 18 tasks 90% to 93% of the time in both HCP and ARCHI datasets (Fig  2 ). Note that l1 penalization in these and below classification model estimations increased interpretability by introducing zeros into the model weights corresponding to network loadings (cf.
The approach has been validated using four different decomposition methods in both HCP  [50]  and ARCHI  [51]  datasets. In sum, using task-derived network dictionaries (from an identical dataset), the extraction of and projection into network sets from ICA and sparse PCA achieved the highest prediction accuracies, precision and recall scores, as well as the most parsimonious model parameters. ICA and sparse PCA hence identified hidden networks underlying task settings that allowed the most efficient quantification of distinctive neural activity aspects. After validating the proportional implication of large-scale networks as a salient property of task activity, we aimed at the interpretability of the network co-occurrence models from task maps (without any recourse to rest maps). The 18-task classification problem was solved by l2-penalized SVM of the selected k most important network loadings for each task (k = 40, 20, 10, 5, and 1) (Fig  3 ).
See single most important ICA or sparse PCA network loading per task yielded classification scores between 83% (ICA, standard deviation [SD] computed across participant-wise data folds in cross-validation scheme: 1. 3) and 46% (Sparse PCA, across-fold SD = 1. 7) for HCP as well as between 72% (ICA, SD = 3. 0) and 38% (sparse PCA, SD = 4. 7) for ARCHI. Increasing the number of k discriminative networks per task rapidly saturated the predictive accuracy. In both datasets, the classification accuracy was virtually identical when knowing all 40 or only the 20 most distinctive network loadings, which in turn was comparable to knowing only 10 and 5 network loadings. In sum, when assuming existence of 40 major networks, experimentally The ensuing network loadings from the second half of the ARCHI task data were submitted to classification of the psychological tasks based on the implication of brain networks (l2-penalized support vector machines, multi-class, one-versus-rest).
e. , k = 5 analogous to analyses in Figs  3  and 4 ). Although the experimental tasks in the HCP and ARCHI repositories, "story versus math" and "sentences versus computation" were the most similar cognitive contrasts in both datasets. For these four experimental conditions the modelderived task maps are highly similar. Consequently, two independent classification problems in two independent datasets with a six-fold difference in sample size resulted in two independent explicit models that, nevertheless, generated comparable task-specific maps. This indicated that network co-occurrence modeling indeed captures genuine aspects of neurobiology rather than arbitrary discriminatory aspects of the data. doi:10. 1371/journal. pcbi. 1004994. g007 compared to fitting the same l1-penalized support vector machine models on all 61,472 graymatter voxels without the initial feature-learning step, that is, without recourse to global spatiotemporal components or local voxel clusters.
How much information on the actual 3D activation patterns is lost by information compression in a network space is measured by the recovery performance. Learning network co-occurrence models from a non-identical task battery (i. e. , networks from HCP task data and classification in ARCHI task data, or vice versa) yielded recovery performances between r = 0. 50 (across-task SD = 0. 17) and r = 0. 43 (SD = 0. 17) across two decompositions and datasets. Importantly, learning network co-occurrences models from taskunrelated resting-state correlations yielded very similar recovery performances between r = 0. 51 (SD = 0. 15) and r = 0. 46 (SD = 0. 11). Assessed by independent t-tests, recovery performance based on network sets from different task data was in no instance significantly better than recovery from networks discovered in task-unrelated rest data.
Discussion We confirm the central hypothesis that fMRI brain activity can be formally assessed in sets of macroscopical network units in a goal-direct task and an idlying rest mindset. The proposed statistical approach effectively exploits the recently increasing neurobiological evidence that functional coupling between canonical brain networks is more context-sensitive than functional coupling within a same network  [15] [16] [17]. Two large datasets allowed for direct comparison of 36 diverse experimental tasks. A large proportion of activity patterns evoked by experimental tasks is shown to reflect combinations of stereotypical network patterns. Despite their diverging statistical assumptions (e. g. , orthogonality, sparsity, independence, and heteroscedasticity), all used decomposition methods allowed for generative network models that can express neural activity maps as combinations of spatiotemporal activity components.
We thus computed neurobiologically interpretable stratifications of large-scale network recruitment specific to cognitive tasks without relying on assumptions from cognitive theory cf. [98]. We validated decomposition and reconstruction of network ensembles in two large task fMRI datasets, each aimed at capturing human cognition comprehensively. Statistically, being able to create synthetic, never observed activity maps for each task is an important property of "generative models"  [55]. Although the task-specific network implications are not directly observable using fMRI techniques, we demonstrate that the underlying organization can be numerically approximated in order to test explicit hypotheses. Functional coupling between brain regions and coupling changes between task and rest has been investigated in several previous studies (e.
g. ,  [6, 103]. Future research might therefore evaluate to what extent the identical statistical approach scales to the quantification of task effects subserved by local specialized brain regions after matrix factorization up to single-region patterns. In the present study, the reconfiguration of large-scale networks was identified as a characteristic property of fMRI activity during experimental tasks. Multivariate decomposition of neural activity into 40 major networks allowed distinguishing 18 typical tasks in up to 90% (HCP dataset) and 93% (ARCHI dataset) of unseen activity maps. Selecting only the loading of the single most distinctive of 40 total networks for each task, classification still scored at up to 83% (HCP dataset) and 72% (ARCHI dataset). In particular, ICA and sparse PCA appeared most successful in searching the space of plausible network co-occurrence architectures.
This study might be the first to carefully quantify the often held view that contrast approaches of experimental fMRI studies do not necessarily elicit idiosyncratic activity patterns, but mainly a recombination of the same underlying networks cf. [44, 64, [65] [66] [67]. This is true in the sense that rest network dictionaries have been shown to reflect sufficiently rich directions of variation to encode and reconstruct whole-brain task images with minimal information loss. It follows that a single major brain network may not be task-specific (e. g. , the "language network", the "empathy network") but the entire activity pattern of the various brain networks may be a distinctive correlate underlying each cognitive process. Indeed, an interplay of limited entities generates an infinity of different observations in numerous natural systems  [68] , formalized in General System Theory  [69].
Materials and Methods Data Dataset: HCP task data. The first of two task batteries was drawn from the Human Connectome Project (HCP; www. humanconnectomeproject. org). HCP is an international longterm project dedicated to network exploration  [77]. 500 HCP participants (2 removed for quality reasons) were without psychiatric or neurological history. Informed consent was obtained from all participants by the Washington University in St. Louis institutional review board. The ethics approval for using the HCP data was obtained from the local ethics committee of the Heinrich-Heine University Düsseldorf, Germany. Network discovery is facilitated by probing experimental task paradigms that are known to tap on well-characterized neural networks. This was achieved by selecting tasks that feature known suitability and reliability across participants  [50].
96 per task across participants  [50]. Note that we statistically analyzed the GLM-derived participant-level z-score maps in MNI space. Dataset: ARCHI task data. The second task battery was drawn from the ARCHI dataset  [51]. Importantly, it was conceived and acquired with the same goal for neural networks shared by a maximum of participants as the HCP task dataset. The diverse experimental tasks (Table  2 ) capture the cerebral basis of auditory and visual perception, motor action, reading, language comprehension and mental calculation. The ethics approval was obtained from the regional ethical committee (Hôpital de Bicêtre, France). 81 right-handed healthy participants (35 males and 46 females, 24 +/-4. 1 years, three not included in present analyses due to incomplete data) without psychiatric or neurological history participated in four fMRI sessions acquired under different experimental paradigms.
Wholebrain EPI data were acquired with the same Siemens Trio with a 32 channel head coil (TR = 2400ms, TE = 30ms, flip angle = 60, in-plane FOV = 19. 2 × 19. 2cm, 40 slices, 3. 0mm isotropic voxels). A posterior-anterior phase encoding scheme was used for all images. Standard preprocessing was performed with Nipype  (Gorgolewski et al. , 2011) , including slice timing, motion correction, alignment, and spatial normalization. Activation maps were spatially smoothed by a Gaussian kernel of 5mm (FWHM). Note that we statistically analyzed the GLM-derived participant-level z-score maps in MNI space, analogous to the HCP task data. Dataset: Rest data. These two task datasets were complemented by HCP resting-state acquisitions of brain activity in the absence of experimental paradigms. In line with the goal of the present study, acquisition of these data was specifically aimed at the study of task-rest correspondence  [77].
Every participant who contributed scans to the present rest dataset was included with one left-right and one right-left phase encoding session. Importantly, PCA was applied to each set of 1,200 rest maps for denoising and dimensionality reduction into 20 main modes of variation  [79]. That is, the 40 reduced rest maps for each of 25 randomly selected participants constituted the present rest dataset of 1000 concatenated, noise-cleaned rest activity maps. In sum, the HCP task data incorporated 8650 first-level activity maps from 18 diverse paradigms administered to 498 participants, while the ARCHI task data incorporated 1404 firstlevel activity maps from 18 diverse paradigms administered to 78 participants. The rest data, in turn, incorporated 1000 activity maps drawn from different scanning sessions in 25 participants.
The maps from all three datasets were downsampled to a common 53 × 63 × 46 space of 3mm isotropic voxels. The resampled activity maps were masked by a gray-matter probability of at least 10% according to tissue maps from the Intertional Consortium of Brain Mapping (ICBM). Each of the task and rest maps was hence represented by 61,472 voxels with z-values in gray matter.
Workflow Data folding and model selection were performed in the following fashion. In the first step, one half of the task datasets (i. e. , HCP and ARCHI) and the entire rest dataset were used for unsupervised (i. e. , label-independent) discovery of latent structure by matrix decomposition and clustering methods. The components of variation identified in the data allowed for feature engineering from biological structures. In the second step, we applied supervised (i. e. , labeldependent) classification algorithms to the other half of the activity maps to predict cognitive tasks. The winning models for classifying 18 cognitive tasks were compared by cross-validation. This inferential statistical framework is the gold standard to obtain an unbiased estimate of how well a trained classifier generalizes beyond the data samples at hand  [80, 81].
The previously unseen half of the task data (i. e. , 4325 maps from HCP and 702 maps from ARCHI] were split into as many data folds as participants (i. e. , 498 for HCP and 78 for ARCHI]. In each fold, all task maps of a given participant were left out as the test set for assessment of out-of-sample performance, while the task maps from the remaining participants served as the training set for model estimation. This leave-one-participant-out cross-validation scheme ensured model fitting of task effects, rather than interindividual differences. In the third and last step, the averaged winning models were back-projected into realistic task activity maps as face validity for the reduced representation of task activity patterns. 1. Unsupervised analysis layer. The first step uncovered hidden structure in large quantities of neural activity maps.
To this end, each dataset (i. e. , first half of task datasets and entirety of rest dataset] was decomposed into the 40 modes of variation. A large set of 40 "network" archetypes should ensure more parsimonious representation of the unknown target function, despite increased problem difficulty due to higher variance. Latent large-scale networks were computed by maximizing the intra-network homogeneity in neural activity and maximizing the between-network heterogeneity. We selected four different decomposition methods that are in frequent general use. Decomposition into brain networks. Four different network decomposition techniques were applied to the neural activity maps in order to test whether the neuroscientific findings generalize across diverging methodological choices. First, independent component analysis (ICA] was used to unmix the multivariate BOLD signals in the activity maps into separate spatial components by minimizing their mutual information  [82].
Third, sparse PCA was used to separate the activity maps into network components with few regions  [85] , which scales well to large datasets. This decomposition method reformulates a PCA-like modeling goal as a regression-type optimization problem constrained by l1-penalty terms. An implementation without orthogonality assumptions yielded spatially less distributed components to explain the variation in the data (1000 maximum iterations, per-iteration tolerance of 1 Ã 10-8, sparsity alpha = 1, ridge-shrinkage at 0. 01, Lasso solution computed with coordinate descent)  [86]. Fourth, factor analysis (FA) generalizes (probabilistic) PCA by a heteroscedastic noise model  [55]. Similar to PCA, FA performs a rotation-invariant low-rank approximation by identifying the latent space of variation (1000 maximum iterations, per-iteration tolerance of 0.
01, 3 iterations in power method, randomized singular vector decomposition). Note that all four decomposition methods implicitly assume large variance to be indicative of impor-tant structure. They reflect different ways to model sets of partially-overlapping major brain networks without access to any task information. All decompositions are soft in that each voxel with its BOLD signals can be part of the support in more than one network component. For most (ICA, PCA, FA) but not all (sparse PCA) decompositions, the ensuing components are linear combinations of all the original voxel signals. Clustering into brain regions. The decompositions into spatially overlapping network components were evaluated against clustering of the same data into non-overlapping regional components. That is, the similarity between voxels in neural activity changes across maps was indicative of coherent regional compartments, rather than distributed spatiotemporal networks.
e. , task and rest activity maps). From a neurobiological perspective, this approach acknowledged the assumption that smaller groups of, but not all, network components should be characteristic for the observed activity pattern of the various tasks. We performed hyperparameter tuning by a grid-search of the SVM hyperparameter "C" (7 steps between 10-3 and 103 on a logarithmic scale) and averaged the fold-wise model parameters (i. e. , bagging; 80). The amount of sparsity (induced by l1-constrained SVMs) was thus adapted to best predicting cognitive tasks. The collapsed SVM classification models thus represented the average hypothesis from that dataset with reduced variance but essentially unchanged bias. Second, the l2-regularized SVM analyses (only used by analyses described in next passage) were used for the tendency to shrink the model parameters i) towards zero and ii) towards each other.
In so doing, we shrunk all directions of variation. Comparing to l1-regularized, no implicit feature selection is performed as the ensuing parameters are typically small but non-zero. That is, both l1 and l2 penalization perform variable shrinkage, yet l2-constrained SVM does not attempt to improve sparsity and domain interpretability by introducing zero coefficients. In one subanalysis, ANOVA was used for explicit univariate component selection of the k most important (i. e. , 1, 5, 10, and 20) component loadings for each given task comparing to other tasks. That is, fitting the multivariate classifier was preceded by a univariate feature-selection procedure on the training data in each cross-validation fold. In particular, the feature space (i. e. , 40 component involvements) was reduced to the k component loadings most relevant for each task.
Software implementation Python was selected as scientific computing engine. Capitalizing on its open-source ecosystem helps enhance replicability, reusability, and provenance tracking. Nipy  [95]  performed basic analyses of functional neuroimaging data (http://nipy. org/). Scikit-learn  [96]  provided efficient, unit-tested implementations of state-of-the-art statistical learning algorithms (http://scikitlearn. org). This general-purpose machine-learning library  [97]  was interfaced with the neuroimaging-specific nilearn library for high-dimensional neuroimaging datasets (http://github. com/nilearn/nilearn). 3D visualization of brain maps was performed using PySurfer (http:// pysurfer. github. io/). All analysis scripts of the present study are readily accessible to the reader online (http://github.
Supporting Information S1 Fig. Sparse  PCA network decomposition of HCP task maps and network-task assignment. 40 network components underlying 18 HCP tasks (left) have been discovered by sparse PCA (Comp1-40). The ensuing network loadings from the second half of the HCP task data were submitted to classification of the psychological tasks based on the implication of brain networks (l2-penalized support vector machines, multi-class, one-versus-rest). l2-contrained support vector machines were employed to choose the most discriminatory network variables by a classical univariate test in a discrete fashion rather than by soft variable selection based on l1 penalization (cf. methods section). This diagnostic analysis (right) revealed the most distinctive k = 1, 5, 10, and 20 network features (red cubes) for each experimental condition of the task battery (cf.
e. , not selected) for a given task at k = 20. See Table  1  of the manuscript body for descriptions of task 1 to 18. 40). The ensuing network loadings from the second half of the ARCHI task data were submitted to classification of the psychological tasks based on the implication of brain networks (l2-penalized support vector machines, multi-class, one-versus-rest). l2-contrained support vector machines were employed to choose the most discriminatory network variables by a classical univariate test in a discrete fashion rather than by soft variable selection based on l1 penalization (cf. methods section). This diagnostic analysis (right) revealed the most distinctive k = 1, 5, 10, and 20 network features (red cubes) for each experimental condition of the task battery (cf. Fig 3). The thus discretely selected network features per task were then fed into supervised multi-task classification as a feature space of activity-map-wise continuous activity values.