dataesr/scientific-paragraphs-categorization huggingface dataset:
Discussion C-Norm significantly outperforms other methods on the two datasets of the 2019 BBnorm task. An advantage of the C-Norm method is also that it is domain-independent, in contrast with methods such as HONOR which obtained good results but used domain-specific rules. The C-Norm ensemble method is able to improve upon the performance of the individual SLFNN and S-CNN components. We performed an error analysis on predictions made on the development set by the best run of our methods. Out of the 610 mentions in this set, 382 mentions (62. 6%) have been correctly normalized by C-Norm. Compared to C-Norm, the SLFNN has only correctly normalized 194 mentions (31. 8%), the S-CNN only 308 mentions (50. 5%) and the union of the correct  predictions of both components results in 342 mentions (56.
4%) out of 610 were incorrectly normalized. We classified errors into several types: • Partially correct normalization 40 mentions (17. 5% of the errors) have only been partially correctly normalized. Indeed, the BB-norm dataset includes mentions that have to be normalized with more than one concept (e. g. the mention "healthy adult" has to be normalized with both <OBT:002712: healthy person> and <OBT:003245: adult human>). Existing methods, including C-Norm, typically only predict one concept per mention. • Overgeneralization 64 mentions (28. 1% of the errors) have been normalized by a concept more general (i. e. higher in the is_a hierarchy graph) than an exact one, and the predicted concepts are at a maximum of 5 concepts above the exact concept (e. g. the "scimudin cheese" mention should have been normalized by the concept <OBT:003522: Scimudin>, and has been normalized by <OBT:001480: cheese>, and <OBT:003522: Scimudin> is_a <OBT:003492: mould ripened cheese>, which is_a <OBT:003459: soft cheese>, which is_a <OBT:003428: ripened cheese>, which is_a <OBT:003381: fermented cheese>, which is_a < BT:001480: cheese>).
We hypothesize that there are three main explanations for this increase: • An ontology well-suited for natural language processing The OntoBiotope ontology has been built from expert analysis of microbiological publications. This implies that the labels are terms which are close to those found in texts. However, this feature is dependent on the task and the ontology, and will not necessarily generalize well to other contexts. For instance, in the Social Media Mining for Health Applications (SMM4H) shared task  [47]  where adverse drug reaction mentions in tweets are normalized with concepts from the MeDRA terminology, the language used in tweets can be quite different from term labels (e. g. "head is killing me" normalized with the concept <MEDDRA:10019211: headache>). C-Norm has been evaluated on two normalization tasks belonging to the same dataset from a specialized domain (i.
Conclusions C-Norm is a new neural method which synergistically integrates several strategies to handle few-shot normalization tasks: weak supervision, ontological knowledge integration and distributional semantics. C-Norm is built in an ensemble averaging way, which learns to dynamically combine predictions from two neural components with complementary results: a shallow CNN and a single layer feedforward neural network. These components are representative of two recent types of approach that have shown good performance in entity normalization in specialized domains. Our results show that the choice of components and the method to combine them is the source of a significant gain in the predictive capacity of algorithms, greatly outperforming other methods on the Bacteria Biotope datasets of BioNLP-OST 2019, while keeping an overall relatively shallow algorithm.
We analysed the genomic architecture of neuroanatomical diversity using magnetic resonance imaging and single nucleotide polymorphism (SNP) data from >26,000 individuals from the UK Biobank project and 5 other projects that had previously participated in the ENIGMA consortium. Our results confirm the polygenic architecture of neuroanatomical diversity, with SNPs capturing from 40% to 54% of regional brain volume variance. Chromosomal length correlated with the amount of phenotypic variance captured, r~0. 64 on average, suggesting that at a global scale causal variants are homogeneously distributed across the genome. At a local scale, SNPs within genes (~51%) captured ~1. 5 times more genetic variance than the rest; and SNPs with low minor allele frequency (MAF) captured less variance than the rest: the 40% of SNPs with MAF<5% captured <1/4th of the genetic variance.
We also observed extensive pleiotropy across regions, with an average genetic correlation of r G ~0. 45. Genetic correlations were similar to phenotypic and environmental correlations, however, genetic correlations were often larger than phenotypic correlations for the left/right volumes of the same region. The heritability of differences in left/right volumes was generally not statistically significant, suggesting an important influence of environmental causes in the variability of brain asymmetry. Our code is available at https://github. com/neuroanatomy/genomic-architecture.
The advent of collaborative efforts such as the ENIGMA consortium, and large scale projects such as UK Biobank, in particular, has allowed researchers to greatly increase the number of subjects used for heritability estimation (see for example,  Elliott et al. 2018 , Zhao et al. 2018 ). Here we replicate and follow up on our original results using a sample ten times larger, N = 26,818, which combines data from the UK Biobank project as well as 5 other projects that had previously participated in the ENIGMA consortium (IMAGEN, SHIP, TREND, Lothian, ADNI). We used the same regional brain volumes estimates than those used in the ENIGMA consortium studies by  Stein et al. (2012)  and  Hibar et al. (2015)  : several subcortical structures, total brain volume and intracranial volume. In addition, we also studied height and intelligence scores.
Data sharing We obtained whole-genome genotyping from N = 26,818 subjects from 6 different projects: UK Biobank, IMAGEN, ADNI, Lothian Birth Cohort 1936, SHIP and TREND. Extensive efforts have been made to homogenise the neuroanatomical measurements across sites, which were described in the ENIGMA 1 and 2 reports  (Stein et al. 2012 , Hibar et al. 2015) , and the UK Biobank neuroimaging analysis group  (Alfaro-Almagro et al. 2018). The UK Biobank project ( https://imaging. ukbiobank. ac. uk ) is a large, long-term biobank study in the United Kingdom aiming at investigating the contributions of genetic predisposition and environmental exposure to the development of disease. The study is following about 500,000 volunteers enrolled at ages from 40 to 69 years old, 54% females. IMAGEN ( https://imagen-europe.
Regional brain volumes The measurements of regional brain volume coming from projects that had previously participated in ENIGMA  (IMAGEN, Lothian Birth Cohort 1936 , TREND, SHIP and ADNI)  were the same that had been used in  Stein et al. (2012)  and  Hibar et al. (2015). For the UK Biobank subjects, the estimation of the volumes was performed using FreeSurfer 6. 0 ( https://surfer. nmr. mgh. harvard. edu ). For comparison, we also included the estimates obtained using FSL FIRST ( https://fsl. fmrib. ox. ac. uk/fsl ) that were made 14,144 subjects were used for heritability analyses (13,086 included), and N = 6,678 subjects were used for validation of genome-wide polygenic scores  (6,184 included). between total brain volume and intracranial volume. For this, we used a kernel density estimator to fit a probability density function to the intracranial versus brain volume data, and tagged as outliers all subjects with a local density inferior to 1% of the maximum density for UK Biobank, and 2% for other datasets.
Genotype filtering All genetic analyses were performed for each project independently. Genotyping data was converted to the hg19 reference wherever required using UCSC LiftOver ( http://genome. ucsc. edu/cgi-bin/hgLiftOver ). We used genotyped autosomal SNPs (single nucleotide polymorphisms) which passed UK Biobank quality control for all batches ( http://www. ukbiobank. ac. uk/wp-content/uploads/2014/04/UKBiobank_geno typing_QC_documentation-web. pdf ). Additionally, we removed SNPs in 24 regions with long range linkage disequilibrium (LD, see  Price et al. 2008 ). SNPs were then filtered to exclude those with minor allele frequency (MAF) <0. 1%, missing rate >1%, or Hardy-Weinberg disequilibrium with a p<10 -6. Individuals were removed when >10% of their SNPs were missing. We finally pruned SNPs which were in linkage disequilibrium with a variance inflation factor >10, which corresponds to a multiple for the R 2 regression over linked SNPs <0.
Accounting for measurement error The estimation of regional brain volumes, height and intelligence scores is submitted to measurement errors. In particular, small, poorly defined brain regions are more difficult to measure accurately than larger ones. Measurement errors enter as environmental variance in the decomposition of phenotypic variance into genetic and environmental variance, and hence decrease the heritability estimates ( /V V G P ). Heritability estimates unbiased for measurement errors can be obtained with repeated measurements  (Ponzi et al. 2018)  , however, the datasets we used to measure brain volumes provide only one MRI per subject. We used datasets from CoRR, the Consortium for Reliability and Reproducibility  (Zuo et al. 2014)  , which includes multiple MRIs per individual to estimate intraclass correlation coefficients (ICC)  (Fisher 1970 ) for each regional volume.
We segmented the volumes as we did in the UK Biobank dataset, using FreeSurfer 6. 0. ICCs were computed on the volumes of 836 subjects with two MRI sessions after covarying for age, sex and scanning site. For intelligence, we estimated ICC from 1,301 subjects in UK Biobank who took the fluid intelligence test 3 times. The ICC for height was considered to be equal to 1 (no measurement error). Adjusted phenotypic variance and covariance estimates were obtained by multiplying the raw estimates by their corresponding ICC (see Supplemental Materials). 12 F o r P e e r R e v i e w.
Genome-wide polygenic scores We used the SNP effects estimated in the association analysis of 13,086 UK Biobank subjects to estimate the phenotypes of 6,184 additional unrelated subjects with MRI data from the latest release of the UK Biobank project. The scores were estimated from the filtered SNPs (not LD pruned). SNPs under various association p-value thresholds were selected and the ones in LD with a more significantly associated SNP were clumped. The p-value threshold that produced the best fit with the target dataset was selected. We used the software PRSice associated with PLINK for the computation of genome-wide polygenic scores  (Euesden, Lewis, and O'Reilly 2015). Each phenotype was regressed on age, sex, scanning centre, and the 10 first principal components of the GRM. The analyses were performed on the residuals of this linear regression.
Genetic factors influencing volume are shared among brain regions We observed extensive pleiotropy across brain regions, with an average genetic correlation of r G ~0. 45 (Fig. 4 ). Genetic correlations, which represent the correlation between genetic effects of two phenotypes, were computed for each pair of brain regions, ICV, Height and IS, using data from the UK Biobank project. Figure  4  shows the genetic correlation matrix together with the phenotypic correlation matrix (see also Table  S5 ). Genetic correlations were in general similar to phenotypic and environmental correlations, however, genetic correlations were often larger than phenotypic correlations for the left/right volumes of the same region. The concordance between phenotypic and genetic correlation was high ( = 0. 80, Fig  5c ), R 2 consistent with the report by  Sodini et al.
The genome-wide polygenic scores were computed for ~6,000 additional participants from the UK Biobank who were not used in the GWAS. The predictions captured an amount of phenotypic variance ranging from 0. 5% (p < 10 -22 ) for the amygdala to 2. 2% (p < 10 -31 ) for brain volume (Fig. S15 , Fig. S16 , and Supplemental Table  S8 ). For height, a genome-wide polygenic score obtained based on GWAS summary statistics from the 13,086 UK Biobank subjects with MRI captured ~3% of the variance. To evaluate the impact of the number of samples used in the GWAS on the amount of variance captured by the genome-wide polygenic scores, we also computed the genome-wide polygenic scores for height using the GWAS summary statistics from ~277k unrelated UK Biobank subjects not included in the validation dataset.
We sought to take into account errors in automatic segmentation of brain regions by analysing data from the CoRR project  (Zuo et al. 2014)  , where the same subjects were scanned several times. For intelligence scores, we used the subset of UK Biobank where subjects passed the fluid intelligence test on multiple occasions. Our results remained for the most part unchanged after adjusting the estimates for phenotypic measurement errors. A limitation of this approach, however, is that we used MRI scans from a different project (although the processing pipeline was the same). Ideally, one would have preferred to have repeated scans for as subset of UK Biobank subjects, because segmentation quality depends on MRI quality which varies between datasets. The availability of repeated measurements in the same datasets from which genetic variance is estimated may allow to more precisely distinguish between the different sources of phenotypic variations.
Another possible source of bias could be the multi-centric nature of our data. The results presented here come from six different projects. Rather than combining our genetic raw data into a single dataset, we chose to estimate the part of phenotypic variance captured by SNPs independently in each dataset and then combine estimates in a meta-analysis. While our chosen solution helps to handle heterogeneity, it trades on statistical power. The reason for this is that the standard error of the GCTA GREML heritability estimate is approximately inversely proportional to the number of subjects  (Visscher et al. 2014 ) (Fig. S14 ). In the present analysis, the UK Biobank project accounted for ~94% of the estimates, largely driving the results. Indeed, the inclusion of the other projects reduced the standard error of the estimates based 27 F o r P e e r R e v i e w on the UK Biobank project only by a factor of ~1.
03. If the raw genotyping data had been combined in a single mega-analysis instead of in a meta-analysis, the decrease in standard error would have been ~1. 36 smaller than what we reported. The meta-analytical approach may however prove interesting in the future when large imaging genetics datasets other than UK Biobank will be available or in cases where raw genotyping data cannot be easily shared. Our analyses are also limited in their ability to provide information at the individual level. With the advent of large GWAS studies, genome-wide polygenic scores (GPS) have become increasingly used to predict phenotypes from whole-genome genotyping data. GPSs are based on effect sizes estimated through GWAS in a large population. These estimates can then be used to predict the phenotype in an independent sample.
We aimed at evaluating to which extent GWAS of the N~13k UK Biobank subjects used for our heritability analyses allowed us to predict the phenotypes of additional N~6k subjects. GPSs captured a very small, although statistically significant, proportion of the variance. For brain volume, for example, GPS captured ~2. 5% of the variance. This is the expected result in the presence of a strongly polygenic phenotype  (Wray et al. 2013). It is also expected that prediction accuracy will improve as the number of subjects used for effect size estimation increases (the UK Biobank project alone should provide data for N~100k in the years to come). We aimed at testing the potential increase in predictive power by computing a GPS for height. When effect sizes were estimated from N~13k subjects, GPS captured ~3% of height variance, however, when effect sizes were estimated from N~277k subjects, the amount of variance explained increased to ~27%.
Comparison of genetic variance estimates with those of previous studies of the UK Biobank and ADNI projects by Elliott et al (2018) and Zhao et al (2018). Several recent studies have used the UK Biobank data to estimate genetic variance of regional brain volumes. The reported estimates vary across studies, with subcortical volumes, in particular the volume of the amygdala, showing the largest differences. We compared our genetic variance estimates with those obtained by two recent studies  (Elliott et al. 2018; Zhao et al. 2018). Zhao et al. (2018)  used GCTA to compute V G /V P estimates of the left and right volumes of various subcortical structures on a smaller sample of 9,031 subjects from the UK Biobank project. Regional brain volumes were quantified using ANTs (Advanced Normalization Tools software (Avants et al.
2014) together with the MindBoggle-101 atlas (Klein and Tourville 2012). Elliott et al. (2018)  reported genetic variance estimates for thousands of phenotypes on 8,411 subjects, using SBAT (Sparse Bayesian Association Test  (Elliott et al. 2018) ). These phenotypes included the volumes of subcortical structures that we report here. Elliott et al. (2018)  provide V G /V P estimates obtained from measurements of subcortical structures performed using FSL's FIRST (FMRIB's Integrated Registration and Segmentation Tool (Patenaude et al. 2011)) and FreeSurfer. In previous study of the IMAGEN data we reported only total volumes (left plus right) computed using FIRST. For the sake of comparison, we computed V G /V P estimates in the UK Biobank data for the left and right volumes separately, using both FIRST and FreeSurfer (Table  S2 ).
In yellow are represented the estimates of V G /V P we computed using GCTA REML unconstrained method, the other V G /V P estimates were published by  Elliott et al. 2018 (green)  and  Zhao et al. 2018 (orange). Age, sex, center, and the top 10 principal components were included as covariates. The error bars show the standard errors of the V G /V P estimates. (b) Fraction of phenotypic variance (V P ) accounted for by genetic variance (V G ), computed from all genotyped SNPs for the volumes of six subcortical structures in the ADNI dataset. We observed that the sample size was too small to get reliable estimations. Estimates published by  Zhao et al. , 2018.
Comparison between meta-analysis and mega-analysis Fig. S14  shows the difference in standard errors obtained when a single group is analysed (labelled as "merging") versus a meta-analytic approach combining several small groups (labelled as "meta-analysis"). The plot represents the change in standard error after including an additional dataset for GREML analysis. For each base dataset , two plots are shown: a scatter plot of the phenotype residuals (after covarying by age, sex, imaging centre and genetic principal components) as a function of the genome-wide polygenic scores (GPS), and boxplots showing the distribution of the phenotype residuals of the individuals located more than minus (low_gps) or plus (high_gps) two standard deviations away from the average GPS. Units are cm 3 for volumes and cm for height.
S4. Meta-Analysis of the genetic partitioning of SNP-heritability across the six datasets. The partitioning of SNP-heritability estimates were averaged across the six datasets using the inverse variance-weighted average method. Table  S4. 1 provides four partitions based on genic status. Table  S4. 2 provides four partitions based on minimum allele frequency (MAF). Table  S4. 3 provides two partitions based on preferential central nervous system (CNS) gene expression  (Raychaudhuri et al. 2010; Lee, DeCandia, et al. 2012  ) using ± 50 kbp as gene boundaries. For each partition i, the following results are provided: f i the average fraction of SNPs belonging to the SNP set i, the average values of V Gi /V P and V Gi /V G /f i and the associated standard errors, the enrichment p-value of the Z-test comparing V Gi /V G /f i to 1.
of this supplementary document). Table  S5. 3 provides the values used to correct the phenotypic correlations for measurement errors (see equation Eq. 5 of this supplementary document). Table  S6 : Distribution of estimated values for V G /V P of simulated phenotypes for different sub-sample sizes in ADNI dataset. For four sample sizes, 1000 sub-sampling were performed without replacement. nb_ind: number of sub-sampled individuals, nb_conv: number of times GREML converged and an estimate was given, th_mean: theoretical mean for estimates (equal to the simulated V G /V P ), obs_mean, obs_mean_se: observed mean of estimates and standard error, p_mean: p value for difference between theoretical and observed mean (z-test), th_sd: theoretical standard deviation of estimates (computed as the mean of standard errors reported by GCTA), obs_sd, obs_sd_se: observed standard deviation of estimates and standard error, p_sd: p value for difference between theoretical and observed standard deviation (z-test), D_ks, p_ks: effect size and p-value of Kolmogorov-Smirnov test of concordance between theoretical normal distribution and observed distribution.
for overt psychopathology later in life. It includes general population 13 to 17 year old adolescents (from Germany, France, Ireland, and the United Kingdom. ADNI, the Alzheimer's Disease Neuroimaging Initiative ( http://adni. loni. usc. edu ), is a longitudinal multicenter study designed to develop clinical, imaging, genetic, and biochemical biomarkers for the early detection and tracking of Alzheimer's disease in the United States of America. The dataset combines data from the initial five-year study (ADNI-1), and the follow-ups ADNI-GO, ADNI-2, and ADNI-3. It includes subjects 54 to 90 years old, 42% female. The Lothian Birth Cohort 1936 ( https://www. lothianbirthcohort. ed. ac. uk ) is a follow-up of the Scottish Mental Surveys of 1947, which tested the intelligence of almost every child born in 1936 and attending school in Scotland in the month of June 1947.
It includes subjects 71 to 73 years old, 47% female. The SHIP and TREND cohorts contain data from the Study of Health in Pomerania (SHIP, http://www2. medizin. uni-greifswald. de/cm/fv/ship. html ), a population-based epidemiological study consisting of two independent cohorts SHIP and SHIP-TREND. These projects investigate common risk factors, subclinical disorders and diseases in a population of northeast Germany. The dataset included data from subjects 21 to 90 years old, 44% female for TREND, 48% female for SHIP. All data sharing was approved by our local ethical board as well as by those of the participating projects wherever required.
this heterogeneity in effect sizes, using more flexible models that do not make strong assumptions on the relationships between effect size, LD, and MAF, such as GREML-LDMS (Yang et al. 2015)  , should improve the accuracy of the heritability estimates. Our study of rare variants is limited because of the use of datasets based on SNP arrays. The availability of whole-genome sequencing data for large cohorts is starting to allow the study of more refined partitions of rare variants and is showing 25.