dataesr/scientific-paragraphs-categorization huggingface dataset:
Multilingual IE  [3]  and -Task 2. Consumer Health Search (CHS)  [16]. The Multilingual IE task builds upon the six previous editions of the task (2015-2020) which already addressed the analysis of biomedical text in English, French, Hungarian, Italian, Spanish, and German  [29, 27, 28, 30, 31, 25]. This year, the task focuses on Named Entity Recognition in Spanish ultrasound reports. Ten different classes of concepts in the radiology domain are distinguished, including Anatomical Entities, and Findings, that describe a pathological or abnormal event, negations, and indicators of probability or future outcomes. As well as complex entities, the task includes the challenge of semantic split of the dataset. That is, training, development, and test sets cover different semantic fields. This allows for a more realistic held-out evaluation.
The Consumer Health Search task is a continuation of the previous CLEF eHealth IR tasks that ran in 2013-2018, and 2020  [7, 9, 33, 42, 34, 17, 8, 14]. It embraces the Text REtrieval Conference (TREC) -style evaluation process, with a shared collection of documents and queries, the contribution of runs from participants and the subsequent formation of relevance assessments and evaluation of the participants submissions. The 2021 task generates a new representative web corpus and collection of layperson medical queries. The task is structured into a number of optional subtasks as follows: (1) ad-hoc search, (2) weakly-supervised IR, and (3) document credibility assessment. The remainder of this paper is structured as follows: in Section 2, we detail the tasks, evaluation, and datasets created; in Section 3, we describe the submission and results for each task; and in Section 4, we provide conclusions.
Text Documents Task 1. The dataset for this task consists of a corpus of Spanish radiology reports, more concretely pediatric ultrasounds from an Argentinian public hospital. These reports are generally written within a hospital information system by direct typing in a computer and are informed in only one section, where the most relevant findings are described. They are written using standard templates that guide physicians on the structure of the report when the findings are normal, but most of the time they are written in free text to be able to describe the findings discovered in abnormal studies. This fact results in great variations in both size and content of the reports, ranging from 8 to 193 words. Also, there are misspellings and inconsistencies in the usage of abbreviations, punctuation, and line breaks, as can be seen in Figure  1.
Reports were manually annotated by clinical experts  [4]  and then revised by linguists. Annotation guidelines and training were provided for both rounds of annotation. More information about the dataset can be found in  [4]. Nevertheless, for the challenge the annotation criteria has been reviewed and some annotations have been modified. The task, called SpRadIE (for Spanish Radiology Information Extraction), was inspired by previous research on this subject  [5, 2]. Task 2. The CHS task, Task 2, used a new set of 55 queries in English for realistic search scenarios. The queries were constructed either by hand, based on research interests and expertise of the organizers on multiple sclerosis and diabetes, or by using searches issued by the public to social media search services. Namely, the queries were manually authored and tailored by experts from established search scenarios and manually selected from a list of Google trends related queries to best fit each automatically extracted search scenario from social media (e.
Results The number of teams who registered their interest in CLEF eHealth 2021 Tasks 1 and 2 was 58 and 43 (and a total of 67 unique teams). In total, 7 and 4 teams submitted to the two shared tasks, respectively. Task 1 Overall seven different teams participated in our shared task. Most prominent were participants from Spain, but also from Italy, UK and Colombia. Most participating teams were experimenting with different variations of neural networks, particularly transformer-based approaches  [37, 23, 41, 36] , but also bi-LSTMs  [6]. Besides the challenge also includes submissions of a CRF  [24] , and a pattern based approach  [32]. Overall, overlapping and discontinuous entities of the given dataset were the biggest challenge of the dataset, which made preand post-processing steps necessary.
The first task was on multilingual IE and the second task was on CHS. These tasks built on the IE and IR tasks offered by the CLEF eHealth lab series since its inception in 2013. Test collections generated by these shared tasks offered a specific task definition, implemented in a dataset distributed together with an implementation of relevant evaluation metrics to allow for direct comparability of the results reported by the systems evaluated on the collections. These established CLEF IE and IR tasks used a traditional shared task model for evaluation in which a community-wide evaluation is executed in a controlled setting: independent training and test datasets were used and all participants gained access to the test data at the same time, following which no further updates to systems were allowed.
Shortly after releasing the test data (without labels or other solutions), the participating teams submitted their outputs from the frozen systems to the task organizers, who evaluated these results and reported the resulting benchmarks to the community. The annual CLEF eHealth workhops and evaluation labs have matured and established their presence in 2012-2021. In total, 67 unique teams registered their interest and 11 teams took part in the 2021 tasks (7 in Task 1 on IE and 4 in Task 2 on IR). Given the significance of the tasks, all problem specifications, test collections, and text analytics resources associated with the lab have been made available to the wider research community through our CLEF eHealth website 17.
Acknowledgements The CLEF eHealth 2021 evaluation lab has been supported in part by the  CLEF Initiative. It has also been supported in part by the  Our Health in Our Hands (OHIOH) initiative of the Australian National University (ANU) , as well as the  ANU School of Computing, ANU Research School of Population Health , and Data61 /Commonwealth Scientific and Industrial Research Organisation. OHIOH is a strategic initiative of the  ANU  which aims to transform health care by developing new personalised health technologies and solutions in collaboration with patients, clinicians, and health care providers. Moreover, the lab has been supported in part by the  bi-lateral Kodicare (Knowledge Delta  based improvement and continuous evaluation of retrieval engines) project funded by the  French ANR  ( ANR-19-CE23-0029 ) and  Austrian FWF.
European and Chinese cultivated apricots share similar specific crop features, such as fruit shape and size, as well as tree phenology, suggesting convergent adaptation during parallel domestication. However, the impacts of gene flow and selection during these domestication events have not been studied using high-quality apricot genome assemblies or taking into account heterozygosity and previous genetic maps with identified quantitative trait loci (https://www. rosaceae. org/search/qtl). In this work, we produce four high-quality and chromosomescale assemblies of P. armeniaca, P. sibirica and P. mandshurica species. We also sequence the genomes of 578 Armeniaca individuals (Supplementary Note 1, Supplementary Data 1). We reveal that the Chinese and European cultivated apricots result from independent domestication events from distinct wild populations.
Baseline genome sequencing, RNA sequencing, analyses and metadata for the four de novo assembled genomes are summarized in Table  1 , Supplementary Notes 3 and 4, and Supplementary Data 2-4. We found high synteny between our assemblies and the two available apricot genome assemblies of similar high quality  35, 36  , with, however, rearrangements around centromeres (Supplementary Note 4; Supplementary Data 5, Supplementary Figs. 2 3 4 5 ). Molecular markers (SSR and SNP) have been used to check some structural variations and to align physical and genetic maps (Supplementary Data 6). Characteristics of the Armeniaca genomes and patterns of structural variation. The heterozygosity rate estimated from the corrected reads (Supplementary Note 4) indicated that the least heterozygous assembled genome was the apricot Marouch #14 accession (0.
Based on thirteen P. armeniaca RNAseq datasets (Supplementary Data 2), we annotated between 40,067 and 46,196 proteins depending on the assembled genome (Table  1 ; Supplementary Note 5; Supplementary Fig. 6 ). The number and class of transposable elements (TEs), as well as their relative abundance, showed considerable variation among the four genome assemblies (Supplementary Note 5; Supplementary Data 7; Supplementary Figs. 7  and 8 ). The most common class of TEs found in Armeniaca genomes corresponded to LTR (long terminal repeat) retrotransposons (Supplementary Fig. 8 ). We found a higher synteny between P. armeniaca Marouch #14 and cv. Stella (Supplementary Fig. 9 ) and to a lesser extent between Marouch #14 and Siberian CH320_5, while the P. mandshurica CH264_4 accession showed more re-arrangements when compared to the other apricot genomes (Supplementary Fig.
9 ). We observed few large structural variations between Marouch #14 and Stella or between Marouch #14 and CH320_5 or CH264_4 (Fig. 1d ) (Supplementary Note 6; Supplementary Fig. 10  and 11 ; Supplementary Data 8-9). The structural variants were mostly insertions/deletions and ranged in size from 501 bp to 4. 1 Mb, with a majority of variants smaller than 10 kb (Supplementary Fig. 12 ; Fig. 1d ). In particular, an inversion of ca. 600 Kb was detected in the P. armeniaca Marouch #14 genome when compared to the three other genomes assembled in this study, P. armeniaca cv. Stella, Siberian CH320_5 and the P. mandshurica CH264_4 (Supplementary Data 8; Supplementary Fig. 10 ; Fig. 1d ). This large inversion, validated by PCR (Supplementary Fig. 10 ), is located at the edge of chromosome 4 (approximately position 3.
65 Mbp) and likely corresponds to a recent structural rearrangement as it is present only in the Marouch #14 genome. From a breeder's perspective, such information will be important when Marouch #14 is used as a reference genome for read mapping and when the Marouch #14 individual is used as a parent in crosses. Reconstruction of Armeniaca phylogeny. A genome-wide analysis of fourfold degenerated (neutral) polymorphism of diploid Rosaceae species, together with three more distantly related species with known divergence times (i. e. , between Populus trichocarpa and Arabidopsis thaliana or Fragaria vesca and Rosa chinensis  40, 41  ), estimated the divergence between Armeniaca and Amygdalus lineages >7 Mya (million years ago) (Supplementary Note 7, Supplementary Data 10, Fig. 2a  and Supplementary Fig.
Phylogenetic analysis of the Armeniaca chloroplast genomes. Short-read sequencing data of 578 Armeniaca accessions (this   43  , were used for reference-based reconstruction of chloroplast genomes (cpDNA, Supplementary Note 8). For phylogenetic inferences, we selected 2-4 chloroplast genomes per species, representing the cpDNA diversity of wild and cultivated P. armeniaca, P. sibirica, P. mume and P. brigantina populations. The cpDNA assembly of Prunus padus L. (KP760072) was included as an outgroup. The haplotype network of chloroplast genomes closely mirrored the pattern observed on the maximum likelihood tree (Supplementary Note 8; Fig. 3  and Supplementary Fig. 15 ). Three closely related cpDNA haplotypes were found in most P. armeniaca individuals (A1, A2, A3, in both wild and cultivated groups; Fig.
armeniaca trees from Central Asian natural populations (Fig. 1b ), 43 wild P. sibirica trees from eight Chinese natural populations, one P. mandshurica (Fig. 1a ), and 264 cultivated P. armeniaca apricot accessions, comprising 27 Chinese, 166 European-Irano Caucasian and 71 Central Asian apricot cultivars (Supplementary Data 1 and Fig. 1 ; Supplementary Notes 1 and 9, Supplementary Fig. 16 ). We also used previously published genomes of P. mume (N = 348)  31. Fourteen accessions of P. brigantina were used as outgroups  27. SNPs were called using GATK best practices for this collection of 926 individuals (Supplementary Note 9). For population genetic structure inferences, genetic diversity and differentiation analyses, only Armeniaca species were retained (Supplementary Data 1). After SNP calling (Supplementary Note 9), a set of 15,111,266 SNPs was used in the following population-based genomic analyses.
Excluding the North Eastern P. sibirica from a third PCA, the Central Asian individuals fell between the two well differentiated clusters of P. armeniaca, the European cultivated apricots (in grey, Fig. 4c ) and the wild Central Asian P. armeniaca populations (red and yellow, Fig. 4c ) while the North-Western P. sibirica individuals still grouped with Chinese apricots. This result indicates that the North Western P. sibirica individuals had been mis-assigned to the P. sibirica species while they belonged to P. armeniaca. We also ran fastSTRUCTURE on the entire Armeniaca dataset (n = 917) from K = 2 to K = 12, revealing population subdivision Evidence for post-domestication selection specific to Chinese and European apricot populations. We looked for signatures of positive selection in the genomes of the two cultivated populations, the European cultivars originating from Northern Central Asian wild apricots, and the Chinese cultivars originating from Southern Central Asian populations.
42% and 0. 22% of the genome affected, respectively; Supplementary Data 21). The selective sweep regions did not overlap at all between the European and Chinese cultivated populations, suggesting the lack of parallel selection on the same loci despite convergent phenotypic traits (Supplementary Fig. 32 ). When taking as threshold the top 0. 5% of CLR scores for European apricots, more than half of the selective sweeps detected (54 in total) were located in the middle of chromosome 4 (from 7 Mbp to 18 Mbp), indicating a potential hotspot of human selection targets (Fig. 6a-b ) (Supplementary Note 14). In Chinese apricots, one third of the selective sweeps mapped on chromosome 1 and no particular enrichment was observed for chromosome 4 (Fig. 6cd ). We examined overlaps between known QTLs (quantitative trait loci) identified by GWAS (genome-wide association studies) or linkage mapping and the genomic regions with footprints of selection identified by the above tests.
Acknowledgements Most of the computational resources and infrastructure used in present publication were provided by the  Bordeaux Bioinformatics Center (CBiB). Additional computer time for this study was provided by  MCIA (Mesocentre de Calcul Intensif Aquitain) of the Universities of Bordeaux and of Pau and des Pays de l'Adour. We are also grateful to the  Genotoul bioinformatics platform, Toulouse , for providing help, computing and/or storage resources (http://bioinfo. genotoul. fr/index. php)and the URGI platform (https://urgi. versailles. inra. fr/Tools/REPET),for help in running the REPET package v2. 5. We acknowledge valuable contribution of  Dr Peter Civan  ( INRAE GDEC, Clermont-Ferrand ) for chloroplastic phylogenomics and admixture analysis and  Dr Ricardo Rodriguez de la Vega  ( Université Paris-Saclay, ESE, Orsay ) for help in 'managing' the BEAST.
0) and BLASTp hits against NCBI NR database restricted to Viridiplantae proteins as input datasets for Blast2GO annotation service to produce functional descriptions and gene ontology terms. Repetitive elements were predicted in the four Armeniaca genomes assembled in this study using REPET package v2. 5 (https://urgi. versailles. inra. fr/Tools/REPET)  91 (Supplementary Note 5). Comparison with previously assembled P. armeniaca genomes. We downloaded the three existing assemblies from the Rosaceae genome database (cv. Chuanzhihong  35  ) and from NCBI (cv. Rojo Pasion  36  ). Contigs were obtained by splitting the scaffolds at each gap (of at least one N), and gene completion was calculated using BUSCO (v4. 0. 2 with default parameters)  92  and the eudicotyledon odb10 database (N = 2,121 genes).
Whole genome alignments were performed using minimap2 (version 2. 15 with default parameters  93  ) and dotplots were generated from alignments larger than 5Kb using dotPlotly (https://github. com/ tpoorten/dotPlotly). Whole genome alignment and variant calling. The assembled genomes of cv. Stella, CH320_5 and CH264_4 were aligned to the reference Marouch #14 reported in this work using the runCharacterize script provided by Bionano Genomics, with the default settings. The genome alignments were imported into Bionano Access software for visualization (Supplementary Note 6). The assembly alignments obtained above were used to call structural variants using the runSV script provided by Bionano Genomics, with default settings. The smap file resulting from this analysis was filtered out to extract the insertions, deletions, inversions, duplications and translocations.
The structural variations can be visualized into Bionano Access software. The R package OmicCircos was used to edit the circos plot figure from the filtered smap file. Phylogeny and reconstruction of ancestral chromosomal arrangements of Armeniaca species. We identified only 298 single-copy orthologous genes shared among the 12 following species: Arabidopsis thaliana, Populus trichocarpa, Vitis vinifera, Rosa chinensis, Fragaria vesca, Prunus persica cv. Lovell, P. dulcis cv. Texas, P. mume, P. mandshurica, P. sibirica, P. armeniaca Marouch #14 and P. armeniaca cv. Stella (Supplementary Data 10). Fourfold degenerate sites (4DTv) from these 298 single-copy orthologous genes were extracted and concatenated into a "supergene" format for each species. The 12 aligned fourfold degenerate site supergenes were used to construct a phylogenetic tree using the BEAST software  94  (Supplementary Note 7).
The Armeniaca chloroplast phylogeny was inferred as detailed in Supplementary Note 8 and the evolutionary scenario of genome chromosomal arrangement was inferred based on synteny relationships identified between the Armeniaca genomes and other Rosaceae genomes  44  (Supplementary Note 7; Supplementary Data 10). Sequence alignment and variation calling. ILLUMINA sequence reads for each accession were mapped to the Marouch #14 genome (Supplementary Note 9). Reads were filtered for low mapping quality (MQ < 20) and by removal of PCR duplicates (Supplementary Data 1). Both paired-end and single-end mapped reads were used for SNP detection throughout the entire Armeniaca accessions in the GATK toolkit (version 3. 8)  95  (Supplementary Note 9). A subset of 15,111,266 SNPs was selected after filtering for bi-allelic SNPs, SNP quality (>30) and missing data (< 15 %).
90  96  (Supplementary Note 11). The fas-tSTRUCTURE software (version 1. 0) was used to infer the Armeniaca population structure  100. We ran fastSTRUCTURE on four datasets: (1) the whole Armeniaca dataset made of 917 individuals (after removal of the other Prunus species, outside of the Armeniaca section, and of interspecific hybrids), (2) P. mume (N = 348), (3) the rest of the Armeniaca species, without P. mume, that were sequenced in the current study (later referred as the set of individuals of other Armeniaca species, N = 555) and (  4 ) the set of 202 unique and non-admixed accessions (Supplementary Notes 11 and 12). fastSTRUCTURE was run on a subset of 95,686 MAF > 0. 05 filtered and LD pruned SNPs for the first three datasets while 9,613 SnpEfffiltered 101 , synonymous SNPs were used for the last dataset of 202 unique, nonadmixed accessions.
This dataset was also used for demographic inferences and the average number of nucleotide differences between genotypes (π) was drawn from each population 102 using pixy  103  and Stacks  104. Other summary statistics (H E , H O , F IS , number of private alleles) were computed with Stacks 104. Principal component analysis was performed using the smartPCA program of the EIGENSOFT package (version 6. 1. 4) in R software environment  105. Inferences of demographic and divergence histories. We used random forest approximate Bayesian computation  106  to unravel the evolutionary history of the cultivated and wild apricots. From the inferred population structure for K = 7 including the 202 Armeniaca unique accessions (Supplementary Note 12), we filtered out admixed individuals (i. e. , individuals with a membership coefficient < 0.
CBX2. 2 has been suggested to cause 46,XY DSD  [35]. To support this, the authors identified two variants, a missense (p. Cys132Arg) variant and a complete loss-of-function (LOF) variant (p. Cys154fs). However, population genetics data indicate that the latter variant is carried by 1/125 African Americans (https://gnomad. broadinstitute. org/variant/17-77755771-CT-C?dataset=gnomad_r2_1), including several XY males who are homozygous for the variant. This variant is carried by 12% of North African individuals (unpublished data). Consequently, this variant cannot be pathologic. A further analysis of population genetics datasets indicates that the CBX2. 2 isoform has currently 23 loss-of-function (LOF) variants (https://gnomad. broadinstitute. org/gene/ENSG00000173894?dataset=gnomad_r2_1). Several of these LOFs variants are present in the general populations at frequencies that are both inconsistent with CBX2.
Mice lacking Mamld1 show reduced expression of Leydig cell transcripts but otherwise have normal genitalia and are fertile  [85]. Therefore, there is both an inability to model missense variants using the mouse model and, in the absence of a precisely known biological function, a functional relevant biological assay is not available. The other aspect to consider is the population genetics. Some of the missense variants proposed to cause XY DSD are actually common polymorphisms in the general population. For example the reported variants p. H322Q (p. H347Q) and p. V480A (p. V505A) carried by 6. 5% and 17. 8% of African/African Americans are polymorphisms  (83). They are both considered benign by ClinVar (ClinVar accession IDs 712305, 804096). The recently published p. P334S variant, which is reported as likely pathogenic (  86 ) is carried by 12% of Europeans and it is therefore unlikely to be pathogenic.
broadinstitute. org/gene/ENSG00000013619?dataset=gnomad_r2_1) with a small number (n=14) of LOF alleles reported in >68,000 alleles sequenced. A recent study and review of the literature by Li et al. , 2020, indicated that approximately one third of MAMLD1 variants that cause XY DSD are LOF  (86). This enrichment of MAMLD1 LOF variants in DSD cohorts compared with the general population is consistent with the hypothesis that variants which severely disrupt or abolish biological function are indeed responsible for DSD. However, in the absence of relevant and robust functional assays the contribution of rare or novel missense variants to DSD remains to be determined and these are likely to be continued to be classified as VUS in genomic studies.
Patients carrying MAP3K1 variants show no other apparent phenotypic anomalies other than 46,XY GD  (88). In our experience about 10% of 46,XY GD cases harbour rare or novel variants in the MAP3K1 gene that could potentially contribute to the phenotype. However, the MAP3K1 transcript is large (>7 kb) spanning 20 exons and there are over 600 rare (MAF<0. 001) or novel LOF or missense variants reported in the general population (https://gnomad. broadinstitute. org/gene/ENSG00000095015?dataset=gnomad_r2_1). This makes the pathogenic interpretation of a missense variant carried by an individual with DSD difficult, although there is some evidence to suggest that pathogenic variants cluster in specific functional domains of the protein  (90). However, in the absence of a robust and simple functional assay, the clinical interpretation of MAP3K1 variants associated with 46,XY DSD will remain a challenge and most will continue to be classified as VUS.