dataesr/scientific-paragraphs-categorization huggingface dataset:
Specifically, our model's Hausdorff distance showed a marginal decrease relative to other baselines. Figures  4b, 4c , 4d, 4h, 4i and 4j show a slight difference between each prediction, irrespective of the amount of annotated data used for training. Generally, prediction results are positive and segmentations generated are visually matched for all models. However, as in the endocardium segmentation, our method produces a segmentation that is very close to the ground truth. On an adult database, our model performs similarly to U-Net regarding Dice scores. We can see on table 2 that when the number of annotated data decrease GAN models perform better comparing to U-Net. Finally, the usage of our loss function enable to our GAN model to produces similar results regardless of the amout of data.
Myocardium segmentation Table  3  and figure  5  show the Dice scores, the Hausdorff distance and figure for myocardium segmentation. On the York dataset our model reach a Dice score of 0. 79±0. 10 using 50% of annotated data for training while U-Net and a classic GAN reach respectively 0. 74±0. 04 and 0. 77±0. 08. Our model also seems to be less sensitive to the amount of data compared to other models. Indeed on both database, although the Dice scores are better with the U-Net model (except for the York database), we note that with 10% of annotated data for training, our model DT-GAN obtain the best performance (see figures 5d and 5j) whereas U-Net performances collapse. However regarding the Hausdorff distances GANs obtain the best results on the York dataset. On the ACDC dataset U-Net and DT-GAN have similar results but DT-GAN is better with less annotated data.
Discussion From our experiments, as expected, the results varied depending on the data used for training, which was selected randomly during cross-validation. A total of 5011 images on 10022 images were manually annotated. Among annotated images, a quarter (1/4) had rough manual annotations, where the endocardium was not clearly visible. If these images are included in the training set, the model learns more complex structures, and will allow recognizing more complex anatomical structures in MR images. Conversely, if the training set contains only good quality images with very visible left ventricles, the model will not be able to generalize information. A dataset with a manual image distribution could be created in order to obtain a training set with both good and difficult images to enable an efficient and generalizable training, as illustrated in  Fig  instance, Figure  4b  shows that our model produces a better segmentation of the epicardium as compared to the U-Net model in Figure  4f.
A semi-supervised framework combined with a GAN model constitutes a very promising approach to produce good LV segmentation, with a limited amount of annotated data. Compara-tively (with the same dataset), our model achieves a Dice score of 0. 86 ± 0. 15 using only 10% of annotated data and 0. 89 ± 0. 18 using the whole dataset while Nasr-Esfahani et al. (  [41] ) reach a Dice score of 0. 8724 using the whole dataset, a fully connected network and a post processing step. One limitation of our method is that it less efficient for myocardium segmentation, unlike other methods using GANs or other types of neural network models (  [12] ,  [33]  and  [34] ). Indeed using our method, we did not obtain results as good as we wanted for myocardium segmentation (see Table  3 ). Our approach seems to be similar to the ter myocardium segmentation, we will investigate a weighted cross-entropy loss function based on distance maps for multiclass segmentation in a future study.
Population imaging markedly increased the size of functional-imaging datasets, shedding new light on the neural basis of inter-individual differences. Analyzing these large data entails new scalability challenges, computational and statistical. For this reason, brain images are typically summarized in a few signals, for instance reducing voxel-level measures with brain atlases or functional modes. A good choice of the corresponding brain networks is important, as most data analyses start from these reduced signals. We contribute finely-resolved atlases of functional modes, comprising from 64 to 1024 networks. These dictionaries of functional modes (DiFuMo) are trained on millions of fMRI functional brain volumes of total size 2. 4TB, spanned over 27 studies and many research groups. We demonstrate the benefits of extracting reduced signals on our fine-grain atlases for many classic functional data analysis pipelines: stimuli decoding from 12,334 brain responses, standard GLM analysis of fMRI across sessions and individuals, extraction of resting-state functionalconnectomes biomarkers for 2,500 individuals, data compression and meta-analysis over more than 15,000 statistical maps.
g. provide IDPs, with different dimensionalities  (64, 128, 256, 512, and 1024). These modes are optimized to represent BOLD data well, over a wide range of experimental conditions. They are more finelyresolved than existing brain decompositions with continuous networks. By providing validated fine functional atlases, our goal is to streamline fMRI analysis with reduced representations, to facilitate large-cohort and inter-studies work. Through thorough benchmarking over classic data analysis tasks, we show that these modes gives IDPs that ground better analysis of functional images. Finally, we provide a meaningful label to each mode, summarizing its anatomical location, to facilitate reporting of results.
, 2006). For predictive modeling, e. g. in multi-variate decoding  (Mourão-Miranda et al. , 2005) , the high-dimensionality of the signals is a challenge to learning models that generalize well-a phenomenon known as the curse of dimensionality in machine learning  (Hastie et al. , 2009). Finally, for functional connectomes, working at voxel-level is computationally and statistically intractable 1 https://parietal-inria. github. io/DiFuMo as it entails modeling billions of connections. The standard approach is therefore to average signals on regions or networks  (Varoquaux and Craddock, 2013). Functional neuroimaging is currently largely dependent on neuroanatomy for mapping function to structure  (Destrieux et al. , 2010; Devlin and Poldrack, 2007). Some anatomical structures support well a direct mapping to specific functions  (Brett et al.
It has been extensively used to define resting-state networks  (Kiviniemi et al. , 2003; Beckmann et al. , 2005; Calhoun et al. , 2001)  and implicitly outlines soft parcellations of the brain at high order  (Kiviniemi et al. , 2009; Varoquaux et al. , 2010b). ICA-defined networks are used to extract the official IDPs of UK BioBank, the largest brain-imaging cohort to date; these have been shown to relate to behavior  (Miller et al. , 2016). We rely on another decomposition model, dictionary learning  (Olshausen and Field, 1997)  spatial maps. While less popular than ICA in neuroimaging, sparsity brings the benefit of segmenting well functional regions on a zeroed-out background  (Lee et al. , 2010; Varoquaux et al. , 2011). For our purposes, an important aspect of sparse models is that they have computationallyscalable formulations even with high model order and on large datasets  (Mensch et al.
, 2016a (Mensch et al. , , 2018)). Functional modes defined from sparse dictionary learning have been used to predict Autism Spectrum Disorder  (Abraham et al. , 2017) , or mental processes  (Mensch et al. , 2017). Rest and task fMRI. Most functional brain atlases have been extracted from rest fMRI  (Bellec et al. , 2010; Power et al. , 2011; Craddock et al. , 2012; Yeo et al. , 2011; Miller et al. , 2016; Schaefer et al. , 2017). Brain networks can also be extracted from task fMRI data  (Calhoun et al. , 2008; Lee et al. , 2010) , and segment a similar intrinsic largescale structure  (Smith et al. , 2009). In our work, we build functional modes from datasets with different experimental conditions, including task and rest. Our goal is to be as general as possible and capture information from different protocols.
DiFuMo extraction: model and data We consider BOLD time-series from fMRI volumes, resampled and registered to the MNI template. After temporal concatenation, those form a large matrix X ∈ R p×n , where p is the number of voxels of the images (around 2 • 10 5 ), and n is the number of brain images, of the order of 10 6 in the following. To extract DiFuMos, each brain volume is modeled as the linear combination of k spatial functional networks, assembled in a dictionary matrix D ∈ R p×k. We thus assume that X approximately factorizes as DA, where the matrix A ∈ R k×n holds in every column the loadings α i necessary to reconstruct the brain image x i from the networks D. The dictionary D is to be learned from data. For this, we rely on Stochastic Online Matrix Factorization 2   (Mensch et al.
, 2017)  -Table  A4  lists the corresponding studies while Table  A5  gives their dataacquisition parameters. We use fMRIprep  (Esteban et al. , 2019)  for minimal preprocessing: brain extraction giving as a reference to correct for head-motion  (Jenkinson et al. , 2002) , and coregistration to anatomy  (Greve and Fischl, 2009). All the fMRI images are transformed to MNI template space. We then use MRIQC  (Esteban et al. , 2017)  for quality control. Multi-dimensional DiFuMo atlases. We estimate dictionaries of dimensionality k ∈ {64, 128, 256, 512, 1024}. This is useful as the optimal dimensionality for extracting IDPs often depends on the downstream data analysis task. The obtained functional modes segment well-localized regions, as illustrated in Figure  2.
Mapping brain response: standard task-fMRI analysis Standard analysis in task fMRI relates psychological manipulations to brain activity separately for each voxel or region. It models the BOLD signal as a linear combination of experimental conditions-the General Linear Model (GLM,  Friston et al. , 1995). The BOLD signal forms a matrix Y ∈ R n×p , where p is the number of voxels. With data reduction, we use as input the reduced signal  1 ). The GLM models Y or Y red as Y = Xβ + ε where X ∈ R n×q is the design matrix formed by q temporal regressors of interest or nuisance and ε is noise  (Friston et al. , 1998). In our experiments, we use the Nistats library 3. Y red = Y voxel (D † ) ∈ R n×k (Equation With reduced input Y red , we obtain one signal per region, as β ∈ R q×k. The full β-maps can then be reconstructed by setting β rec = βD ∈ R q×p.
Decoding experimental stimuli from brain responses Decoding predicts psychological conditions from taskrelated z-maps  (Haynes and Rees, 2006). The validity of a decoding model is evaluated on left-out data (following 3 https://nistats. github. io/ Varoquaux et al. , 2017) , e. g. left-out subjects for intersubject decoding  (Poldrack et al. , 2009). We use linear decoding models: ridge regression for continuous target and Support Vector Machine (SVC,  Hastie et al. , 2009)  for classification. For each study, we separate sessions (for intra-subject decoding) or subjects (for inter-subject decoding) into randomly-chosen train and test folds (20 folds with 30% test size), and measure the test accuracy. We compare the performance of predictive models using the voxel-level z-maps or using the data reduced with functional atlases.
, 2010a; Barachant et al. , 2013; Pervaiz et al. , 2019). Those are used to fit an 2 -penalized logistic regression for classification and a ridge regression for continuous targets. We assess predictive performance with 20 folds, random splits of subjects in train and test sets, with 25% test size. Data. We use 7 openly-accessible datasets with diverse phenotypic targets, as summarized in Table  A3. We predict diagnostic status for Alzheimer's disease on ADNI  (Mueller et al. , 2005).
Data. We use NeuroVault  (Gorgolewski et al. , 2015) , the largest public database of statistical maps. To avoid circularity, we exclude maps derived from the studies used to extract the DiFuMo atlases, along with maps that fail semiautomated quality inspection (filtering out thresholded or non-statistical maps), resulting in 15,542 maps. Meta-analysis of contrasts maps. Ideally, the extracted IDPs should allow to compute meta-analytical summaries of brain activity maps. In this setting, a single map, corresponding to a certain cognitive concept, is computed from many z-maps across different studies, associated to conditions that involve this cognitive concept. We compare the summaries obtained at voxel-level, i. e. averaging the maps {x}, with the ones obtained using reconstructed images, i.
Predicting traits from functional connectomes Figure  6  shows the impact of the choice of functional atlas when predicting phenotypes from functional connectomes. It reports the relative prediction accuracy for 7 different prediction problems (each composed of a dataset and a target phenotype); the lines give the median across the prediction problems. Here, we do not report a voxellevel baseline, as it requires to compute covariance matrices of dimensions around 100, 000 × 100, 000 and is therefore computationally and statistically intractable. Unlike with the previous results, high-resolution atlases do not provide the best performance, likely because the complexity of the statistical models increases with the square of the number of nodes. The best prediction overall is achieved using DiFuMo k = 256, followed by Craddock k = 400 and BASC k = 444 atlases.
For instance, they define regions larger in the white matter and in the CSF than in the grey matter (Figure  A1 ). A large dataset is needed to capture such implicit regularities of the signal with high-dimensional spatial decompositions. Indeed, running the same model on less data extracts modes with less spatial regularity (Figure  A2 ). The combination of high dimensionality and large dataset leads to significant computational demands. The extraction of DiFuMos was possible thanks to fast algorithms for huge matrix factorization  (Mensch et al. , 2018) , and gathering data representative of a wide variety of scanning protocols via openfMRI  (Poldrack et al. , 2013). We did not limit the DiFuMo modes to gray matter, as measures outside gray matter can be useful in subsequent analysis, for instance to remove the global signal  (Murphy and Fox, 2017).
Conclusion We provide multidimensional atlases of functional modes that can be used to extract functional signals: parietal-inria. github. io/DiFuMo. They give excellent performance for a wide variety of analytic tasks: GLMbased analysis, mental-process decoding or functionalconnectivity analysis. Their availability reduces computational burdens: practitioners can readily perform analyses on a reduced signal, without a costly ROI-definition step. In addition, working on common functional modes across studies facilitates comparison and interpretations of results. To help communication, we have labeled every functional mode to reflect the neuroanatomical structures that it contains. To date, these are the only highdimensional soft functional modes available. As they have been extracted from a variety of data (more than 2,000 sessions across 27 studies, 2.
4TB in size) and improve many analytic tasks, the rich descriptions of neural activity that they capture is well suited for a broad set of fMRI studies. Pre-encoded maps downloaded from Neurovault. org. We download maps related to emotion and pain  (Chang et al. , 2015)  using Neurovault, querying the collections 503 and 504. We use the "Rating" & "PainLevel" labels as predictive targets. We predict emotion using ridge regression, and pain-level over 3 classes using Linear SVC. The supervised learning pipeline, that includes cross-validation and linear models is implemented with Python based scikitlearn  (Pedregosa et al. , 2011). We use nilearn  (Abraham et al. , 2014)  to download maps from Neurovault. org interface  (Gorgolewski et al. , 2015). The data acquisition parameters, preprocessing details and estimation of statistical maps are described in  Chang et al.
Acknowledgments Statistical maps encoded using the GLM. We compute zmaps from HCP900  (Van Essen et al. , 2012)  and IBC  (Pinho et al. , 2018)    HCP. We download fMRI data from the HCP900 release; those are already preprocessed using HCP pipelines  (Glasser et al. , 2013). We use MNINonLinear-based registered data as input for the GLM, that outputs one zmap per condition per subject. We consider three taskbased studies, namely: for Working Memory, we con- Psychological traits. We first stratify individuals from HCP900 release  (Van Essen et al. , 2013)  into groups of high and low IQ, and perform binary classification on these. The details about the stratification into these groups are described in  Dadi et al. (2019). Age regression. We use Cambridge Center for Ageing and Neuroscience (CamCAN) dataset  (Taylor et al.
, 2017)  to study brain ageing. This study comprises wide range of age groups spanning from 24 -86. We use ridge regression to predict age on this dataset. Appendix C. 2. Data acquisition parameters and preprocessing pipelines The data acquisition details for ADNI, ADNIDOD, COBRE, ABIDE, ACPI and HCP are described in  Dadi et al. (2019) ; those for CamCAN in  Taylor et al. (2017). We pre-process individuals from CamCAN, ADNI, AD-NIDOD and COBRE. All rs-fMRI acquistions are preprocessed with standard steps, described in Dadi et al. (2019). The other considered datasets provide preprocessed data. We report the total number of subjects included in the analysis in Table  A3 , after excluding for severe scanning artifacts, head movements with amplitude larger than 2 mm and individuals who have more than one clinical diagnosis, 0.
Figure6: Impact of the choice of atlas for predictions based on functional connectomes. Each data point gives the prediction accuracy relative to the median for one of the 7 phenotypic prediction targets, i. e. each point a dataset. The thick line shows the median over the datasets. While the results are noisy, the optimal dimensionality seems to lie around 300 nodes, and the best-performing atlas is DiFuMo k = 256, followed by Craddock k = 400 and BASC k = 444. FigureA5report absolute results for each prediction problem.
This project has received funding from the European Union's Horizon 2020 Research and Innovation Programme under Grant Agreement No. 785907 (HBP SGA2) and No 826421 (VirtualBrainCloud). The work of Arthur Mensch has been supported by the European Research Council (ERC Grant Noria). This work acknowledges the support of ANR NeuroRef and ERC-StG NeuroLang. We also thank Pierre Bellec and Vincent Frouin for their helpful discussions on the experimental work, the neuroimaging community for giving access to fMRI datasets, and open-source contributors on the packages we build upon (including nilearn, fMRIprep, and MRIQC ). Data collection and sharing for this project was funded by the Alzheimer's Disease Neuroimaging Initiative (ADNI) (National Institutes of Health Grant U01 AG024904) and DOD ADNI (Department of Defense award number W81XWH-12-2-0012).
Martino et al. (2014)tasets used in the pipeline described on §3. 4 for predicting phenotypic labels from functional connectomes. In CamCAN, age is predicted using ridge regression. The groups from other datasets are predicted using logistic regression. IQ -Fluid intelligence, PTSD -Post Traumatic Stress Disorder, MCI -Mild Cognitive Impairment. Cognitive Impairment (MCI) group on ADNI. We discriminate between post-traumatic stress disorder (PTSD) and healthy individuals on ADNIDOD. We use data from the Center for Biomedical Research Excellence 6 (COBRECalhoun et al. (2012)to predict schizophrenia diagnosis of individuals. We classify autism and healthy individuals on Autism Brain Imaging Data Exchange database (ABIDE, DiMartino et al. (2014), Finally, we consider data from Addiction Connectome Preprocessed Initiative 7 (ACPI), where we discriminate Marijuana consumers versus control subjects.
The EU method has been slightly revised for the 2017 assessment (European Commission, 2017a) to decrease the influence of expert judgement, e. g. by a more precise calculation of substitution or economic importance and establishing the priority of data to be used. Companies also must be able to defend their results, although mostly internally. Therefore, for these, it will be easier to use confidential data. On the other hand,  Kolotzek et al. (2018)  eliminate expert judgment in order to keep indicators more quantitative, which they consider to better serve decision-making. Also, Granta aims with their proprietary software at quantifying multi-faceted parameters such as substitutability  (Granta Design, 2014). As stated in Section 3. 1. 4, data availability has a direct influence on the materials and life cycle stages included in a criticality assessment.
Indicator evaluation Fig. 2  shows that the evaluation of indicators is a key parameter which affects the outcome of a criticality study. Indicator evaluation is based on two components: indicator selection and indicator scoring. As there is no adequate database specifically created for criticality studies, criticality evaluators have to resort to idealized constructs more or less fitting their situation, and relate these to available indicators and data sources  (Buijs et al. , 2012; Graedel et al. , 2012).
e. , peer-reviewed publications and technical reports), UNEP IRP data on recycling rates (UNEP, 2011), UNDP's Human Development Index (HDI), ecoinvent for environmental data, as well as a wide range of industry reports (e. g. , Roskill) and expert opinions. Furthermore, each method uses unique data sources not widely shared among methods. For example, the Yale criticality method developed for corporations uses INSEAD's Global Innovation Index, while the Augsburg method uses the Social Hotspot Database and Granta's method uses material property data from Material Universe as well as company-specific information. Further examples can be deducted directly from Fig. 5. Ideally, the criticality assessor has complete awareness of material flows and added economic value at each in the supply chain of all materials that are used within the system under study.
For example, due to lacking available data for a large range of materials to accurately represent the supply of raw materials to the EU, the revised EU method uses a new indicator that combines global supply and actual supply to the EU (i. e. the mix of domestic production plus import)  (Blengini et al. , 2017a,b; European Commission, 2017c). Proxies can also be useful to estimate future indicator values. The OECD study conducts the same assessment on two different points in time. For their assessment of 2015, the country concentration of production in 2030 is assessed by the estimation of global reserves using data from 2014. Hence, "global reserves" are used as a proxy for "global production in 2030". Habib and Wenzel (2016)  use a similar approach to assess the country concentration of raw materials production in 2050.
These are provided by, among others, the World Bank for the WGI. However, for most indicators, such uncertainty ranges are not provided, implying that the data point is an absolute, true value. This is overcome by Graedel et al. by calculating their own uncertainty ranges  (Graedel et al. , 2012). Besides the inherent uncertainty of data, data should be evaluated regarding their representativeness for the technology, the specific material, the geographical area, and the time frame formulated in the goal and scope of the study. Data often represent current or historic situations, and should be updated regularly. Data from different sources, such as production and trade nomenclature, are usually not harmonized. This leads to inconsistencies that weaken the robustness of the monitoring of the physical material flows and the evaluation of raw material criticality.