dataesr/scientific-paragraphs-categorization huggingface dataset:
Acknowledgments This work was supported by grants from the  Ministry of Education, Culture, Sports, Science, and Technology, Japan  ( 16K15254  and  19K08534  to YK,  19H01048  and  22H04991  to HK); Platform for Dynamic Approaches to Living System from the  Ministry of Education, Culture, Sports, Science and Technology, Japan ;  Core Research for Evolutional Science and Technology (CREST)  of the  Japan Science and Technology Agency (JST), Japan  ( JPM-JCR13W2  to HK);  Platform Project for Supporting Drug Discovery and Life Science Research (Basis for Supporting Innovative.
Genomic studies Whole-genome sequencing (WGS) of the 13 tumors and 10 paired normal DNA samples was performed using the Tru-Seq DNA PCR Free or TruSeq DNA nano library preparation. Raw reads were mapped to the human reference genome (GRCh37) using the BWA-MEM algorithm. 10  Immunoglobulin gene rearrangements were extracted using IgCaller (version 1. 1)  11  and annotated using IMGT/V-QUEST. 12  Genomic alterations were identified using a multi-caller bioinformatics approach (Online Supplementary Appendix). 13  Driver mutations were studied considering a list of 247 recurrently mutated genes in B-cell neoplasms (Online Supplementary Appendix; Online Supplementary Table  S2 ). Total RNA sequencing (RNA-seq) was performed in seven tumors with BCL3-R and nine CLL without BCL3-R. Raw data were analyzed as previously described (Online Supplementary Appendix)  13  using the DESeq2 package.
14  mRNA-seq data from our International Cancer Genome Consortium CLL cohort were used for comparison. 15  DNA methylation profiles of ten BCL3-R tumors were generated using EPIC methylation arrays. Similar data from 85 CLL were obtained for comparison from two previous publications: cohort 1 (C1) included 12 CLL from our institution,  13  and cohort 2 (C2) 73 CLL from University Hospital Heidelberg. 16  Data analyses were performed using minfi and limma packages. 17, 18  The AME tool from the MEME suite  19  was used for enrichment analysis of known motifs (2022 JASPAR database; Online Supplementary Appendix). 20  WGS, RNA-seq, and DNA methylation data are deposited in the European Genome-phenome Archive.
Acknowledgments The authors thank the Hematopathology Collection registered at the  Biobank of Hospital Cli ́nic -Institut d'Investigacions Biomèdiques August Pi i Sunyer (IDIBAPS) , the  Biobank HUB-ICO-IDIBELL  ( PT17/0015/0024 ), integrated in the  Spanish Biobank Network  and funded by  Instituto de Salud Carlos III  ( PT17/0015/0024 ), and  Xarxa de Bancs de Tumors de Catalunya  sponsored by  Pla Director d'Oncologia de Catalunya (XBTC) , and the  Molecular Cytogenetics   Platform of IMIM, Hospital del Mar (Barcelona)  for providing BAC clones. This work was partially developed at the  Center Esther Koplowitz (CEK, Barcelona, Spain ).
Improved visualization of cortical lesions has been achieved with the development of advanced MRI techniques, such as double inversion-recovery (DIR) and phase-sensitive inversion-recovery (PSIR) sequences  (4, 5, 7, 13, 14). However, these sequences are largely absent in routine diagnostic and clinical trial evaluation protocols due to their substantial acquisition times (ie, 10-15 minutes). Pilot studies using artificial intelligence (AI) have recently enabled the generation of artificial DIR images from routine clinical MRI protocols (ie, combinations of T1-and proton density-/T2-weighted sequences)  (15, 16). Histopathologic validation of AI-generated DIR images vis-à-vis MRI-acquired images showed equal sensitivity and specificity values  (17). To leverage the potential of AI-generated DIR and PSIR images to serve as an alternative to MRI-acquired DIR and PSIR images, multicenter data evaluation is crucial for understanding their value in a clinical or research setting with hardware and sequence parametrization variability.
Image Preprocessing and Generation of Artificial Images An extensive description of the full image processing pipeline is provided elsewhere  (15). In brief, all available contrasts of each patient were coregistered with each other with use of rigid body transformation using FLIRT (liner image registration tool, part of the Functional MRI of the Brain Software Library, or FSL, version 5. 0. 4; http://fsl. fmrib. ox. ac. uk). The resulting transformation matrixes were used to rigidly transform the data of all individuals into 1-mm Montreal Neurological Institute system standard space (spline interpolation). The skull was removed from the data by using bet-premask (FSL). Variance scaling, but not any other intensity or bias field correction, was applied to each individual contrast in each patient.
Next, two separate generative adversarial networks (U-Netlike convolutional networks) were trained to generate either artificial DIR or PSIR images from the available clinical MRI sequences (ie, T1-weighted and proton density-/T2-weighted or fluid-attenuated inversion-recovery sequences). Images from the different centers were equally distributed into training and test sets (2:1 ratio), after which the generative adversarial networks were applied to the training sets. The code that was used to generate the artificial images is deposited at https://github. com/ MrtnStnwk/DeepContrast. Lesion Identification AI-generated DIR and PSIR images, along with their MRI-acquired counterparts, were randomly distributed for lesion assessment among seven readers, all from different centers. Each reader received 30 data sets: eight AI-generated DIR images, seven AIgenerated PSIR images, and the corresponding MRI-acquired images.
One randomly selected DIR and one randomly selected PSIR data set were provided to all readers for reliability calculations. The other data sets were reviewed by only one reader. Images were provided in random order and randomly left-right flipped to reduce recognition probability. Lesion detection was performed by seven authors (B. M. , C. L. , D. T. C. , E. S. B. , M. M. , P. E. , and P. P. , all with >10 years of experience in cortical lesion detection) in accordance with consensus recommendations by the Magnetic Resonance Imaging in MS, or MAGNIMS, group for MRI-acquired as well as AI-generated DIR images and sequence-specific guidelines for MRI-acquired and AI-generated PSIR images  (14, 20). Readers were also asked to indicate lesion type as juxtacortical (touching but not entering cortical gray matter), leukocortical (situated in part in the cortex and in part in the adjacent white matter), intracortical (fully situated in the cortex), or infratentorial (below the cerebellar tentorium).
Statistical Analysis The number of lesions detected on the AI-generated images was compared with that on the corresponding MRI-acquired images, using the latter as reference. Distribution of the data was assessed for normality with use of Kolmogorov-Smirnov tests and log-transformed if not normally distributed. Statistical analyses were performed by three authors (P. M. B. , L. E. J. , and J. W. R. T. , with 4, 8, and >15 years of experience, respectively). Differences in the number of detected lesions per patient were assessed using pairwise t tests, and differences in the number of detected lesions at the lesion subtype level were assessed using pairwise Wilcoxon signed-rank tests. Betweenreader agreement (reliability; identical patient) was assessed by calculating the intraclass correlation coefficient (ICC) (two-way mixed model with absolute agreement).
Calculations were based on the number of lesions identified per 20 sections (ie, 0-20; 21-40; 41-60; 60-80). Using a similar model, reliability between AI-generated and MRI-acquired DIR and PSIR images across all centers (nonidentical patients) was determined. Post hoc precision of the AI-generated images was estimated using data from five randomly selected patients per contrast. Lesions detected on AI-generated and MRI-acquired images were matched to obtain true-and false-positive results (taking MRI-acquired images as the reference). This allowed the calculation of precision (precision = true-positive results divided by total positive results; performed by P. M. B. ). Additionally, image contrast ratios were calculated for a random subsample of 14 patients to explore the origin of differences.
Results Patient Characteristics DIR images (n = 160) and PSIR images (n = 125) from a total of 202 patients with multiple sclerosis (mean age, 46 years ± 11 [SD]; 127 women) were received. Data from four patients were excluded due to missing or incorrect sequences. The training and test sets consisted of 106 and 50 patients, respectively, for DIR and 82 and 43 patients, respectively, for PSIR sequences. A flowchart of the procedure is displayed in Figure  1. The demographic characteristics of the patients for the DIR and PSIR training and test sets are displayed in Table  1.
Lesion Detection on AI-generated Images An overview of the number of lesions that were detected on AI-generated versus MRI-acquired DIR and PSIR images is provided in Figure  2. On AI-generated DIR images, a total of 1154 lesions were detected, compared with 855 on MRI-acquired DIR images. Some examples of cortical lesions that were detected on AI-generated and MRI-acquired DIR images are depicted in Figure  3. Averaged over patients, the readers detected 35% more lesions on AI-generated DIR images than on MRI-acquired DIR images, with differences ranging from 15% to 79% more lesions (t = 2. 39, P =. 02). The median number of detected lesions was 15 (IQR, 8-23) for AI-generated DIR and 13 (IQR, 6-20) for MRI-acquired DIR images. Reliability analysis between centers showed good agreement in the number of detected lesions on AI-generated and MRI-acquired DIR images across all assessed data sets (ICC, 0.
Reliability analysis between centers for AI-generated PSIR images versus MRI-acquired PSIR images showed good agreement for the number of detected lesions across all assessed data sets (ICC, 0. 75 [95% CI: 0. 59, 0. 85]). The precision of the AI-generated PSIR images was 69. 5% ± 13. 1. Assessment of between-reader variability (same data set for all readers) showed good agreement for AI-generated DIR images (ICC, 0. 76 [95% CI: 0. 43, 0. 98]), MRI-acquired DIR images (ICC, 0. 85 [95% CI: 0. 58, 0. 99]), AI-generated PSIR images (ICC, 0. 85 [95% CI: 0. 59, 0. 99]), and MRIacquired PSIR images (ICC, 0. 85 [95% CI: 0. 59, 0. 99]). The median number of lesions detected in the patient whose images were assessed by all readers was 15 (IQR,  [11] [12] [13] [14] [15] [16] [17] [18] [19] [20] [21] [22]  for AI-generated DIR, 12 (IQR,  [11] [12] [13] [14] [15]  for MRI-acquired DIR, eight (IQR, five to eight) for AI-generated PSIR, and seven (IQR, five to nine) for MRI-acquired PSIR.
Discussion Our study investigated whether otherwise inconspicuous cortical or juxtacortical multiple sclerosis lesions can be detected on artificial intelligence (AI)-generated double inversion-recovery (DIR) and phase-sensitive inversion-recovery (PSIR) images in a multicenter data set. An earlier study in a singlecenter setting using 1. 5-T MRI showed that AI could potentially mitigate the issue of oftenabsent DIR sequences in clinical care and clinical trials  (15). Our results demonstrate that more lesions can be detected on AI-generated DIR images than on their MRI-acquired counterparts (t = 2. 39, P =. 02) and an equal number of lesions on the AI-generated PSIR images as on their MRI-acquired counterparts (t = -0. 16, P =. 87), with good between-center (intraclass correlation coefficient [ICC], 0.
Named entity recognition (NER) is a widely used text-mining and natural language processing (NLP) sub-task. In recent years, deep learning methods have superseded traditional dictionary, and rule-based NER approaches. A high-quality dataset is essential to take full advantage of the recent deep learning advancements. While several gold standard corpora for biomedical entities in abstracts exist, only a few are based on full-text research articles. The Europe PMC literature database routinely annotates Gene/Proteins, Diseases and Organisms entities; to transition this pipeline from a dictionary-based to a machine learning-based approach, we have developed a human-annotated full-text corpus for these entities comprising 300 full-text open access research articles. Over 72,000 mentions of biomedical concepts have been identified within approximately 114,000 sentences.
Background & Summary Europe PubMed Central (Europe PMC)  1  is a repository of life science research articles, including peer-reviewed full-text research articles and abstracts, and preprints -all freely available for use via the website (https://europepmc. org). Europe PMC contains over 33. 3 million abstracts and 8. 7 million full-text articles, and since 2020 adds over 1. 7 million new articles yearly. The rapid growth in the number of publications within the biological research space makes tracking research trends and assimilating knowledge a challenging and time-consuming effort. With the digitisation of large portions of biological literature and advancements in natural language processing (NLP)/machine learning (ML), it is now possible to build sophisticated tools and the required infrastructure to process research articles in order to extract biological entities, concepts and relationships in a scalable manner.
This dataset consists of full-text scientific articles about COVID-19 and related coronaviruses. Additionally, BioC  14  provides a subset of those full-text articles in a simple BioC format, which can reduce the efforts of text processing. Biomedical datasets, such as those from BioASQ  15  and BioNLP  16  shared tasks, enable the development and testing of novel ideas, including deep learning methodologies. With the development of such biomedical datasets, great improvements in biomedical text mining systems have been made. From the results of recent BioASQ challenges (2013 to 2019), the performance of cutting edge systems keep advancing for tasks such as large-scale semantic indexing and question answering (QA)  17. while corpora without annotations are good for learning semantics, text-mining tools trained on human-annotated corpora outperform those trained on non-annotated ones.
Therefore, open-source gold-standard datasets are crucial for improving biomedical text mining systems. In particular, transformer based deep learning models, such as BERT  18  and GPT  19  , have show that pre-training language models with large text corpora improves performance on downstream applications. However, compared to the text corpora, gold-standard biomedical datasets with human annotations are expensive to obtain, because they require domain experts to spend significant amounts of time creating accurate annotations. Therefore, generating human-annotated biomedical datasets is valuable for biomedical text mining, because once they are available, machine learning algorithms have an accurate starting point to learn from. The Colorado Richl Annotated Full-Text Corpus (CRAFT)  20  is one such human-annotated biomedical dataset, that has been widely applied by researchers to develop and evaluate novel text mining algorithms.
It contains 97 full-text open access biomedical journal articles that have both semantic and syntactic annotations, including coreference annotations and 10 biomedical concepts. Besides the CRAFT corpus, other corpora such as BioCreative V CDR corpus (BC5CDR)  21  , BC2GM  22  , Linnaeus  23  , S800  24  , GAD  25  , EUADR  26  , and BioASQ  15  have also been used to extensively test new algorithms on the concept recognition task. Comprehensive and high-quality annotations make the CRAFT corpus one of the most important goldstandard datasets in the biomedical domain. Recent publications  27, 28  have shown that sophisticated systems can be developed by using annotated biomedical datasets. One example is OGER  27  , a system trained with neural networks that outperforms the previous dictionary-based systems in the concept recognition task.
In particular, when evaluated on the CRAFT corpus  20  , more pronounced improvements have been achieved in identifying Chemical, Organism, Protein, and Sequences Concepts  27. Notably, as pre-trained models such as BERT  18  have gained traction in the biomedical domain, many systems have been developed by training models with multiple biomedical datasets. For example, the BioBert model  29  has been trained and evaluated on multiple datasets for downstream tasks including Named Entity Recognition, Relation Extraction and Question Answering. This study presents the Europe PMC Annotated Full-text Corpus (EPMCA), a collection of 300 research articles from the Europe PMC Open Access subset. The selected articles have been human annotated to indicate mentions of three biomedical concepts; Gene/Protein, Disease, and Organism.
Since all annotations are created based on guidelines, this helped the human annotators select the correct text span and type of annotation. Three additional articles that were used in a pilot study are also published with this study. The size of the EPMCA is among the largest human-annotated biomedical corpora. We believe that the high-quality gold standard annotations of the EPMCA corpus will be an important addition to other existing datasets and provide significant benefits for biomedical text mining.
Entity frequency distribution The pool of 503, 950 "standard-sized" articles were further stratified based on the term frequency of the three entities of interest, namely; Gene/Proteins, Diseases, and Organisms. Using the current Europe PMC dictionary-based annotation pipeline to annotate the articles, we established the range of entity frequencies in the articles (Figure  3 ) and created high (H), medium (M), and low (L) frequency tertiles by splitting them at the 33 and the 66 percentiles (Table  1 ). This resulted in 27 bins of articles from these tertiles of three entities (33) (Figure  4 ). All the articles in the Low-Low-Low bin contain a small number or no mentions of any of the entities but represent the largest number of articles (42,261 articles, more than 8% of total articles). Because these would add little value to the training dataset, this bin was excluded from the article selection process.
Gene/Protein The Gene/Proteins dictionary is periodically generated from the SwissProt 32 knowledgebase from the 2014 release. SwissProt is a manually reviewed resource of proteins and genes, and the knowledgebase is released in multiple formats. The entries in the Uniprot knowledgebase are structured to make it both human and machine-readable (for more details please follow https: //www. uniprot. org/docs/userman. htm#convent). For tagging Gene/Proteins in the Europe PMC annotations workflow, the DAT file of the knowledgebase release is parsed, generating a Gene/Proteins dictionary from the gene name lines and their aliases (the gene name lines are denoted by starting the line with GN tag according to the knowledgebase data structure). The UniProt knowledgebase release, dated 2014, was used to generate the Gene/Proteins dictionary.
"exact" is the annotation itself while "prefix" and "suffix" are characters before and after the annotation, respectively. By combining "prefix", "exact", and "suffix", the snippet can locate the annotation using regular expressions. Raw annotations from all three human annotators are available on GitLab, which are helpful for studies of agreement between annotators. Annotations in JavaScript Object Notation (JSON) and IOB formats are provided in addition to raw annotations. Both JSON and IOB format annotations are preprocessed so that only annotations agreed on by at least two annotators are included. The IOB format provides sentences with IOB tags and follows the CoNLL NER corpus standards  36. While the IOB format is widely used in named entity recognition (NER), researchers may prefer other tagging formats so the JSON format provides sentences and annotations for researchers that are interested in transforming annotations into other tagging formats.
Discussion and Conclusion Gold standard datasets are essential for capitalise on the revolutionary deep learning methods for NLP and text-mining. Although a number of gold standard datasets are available for biomedical entities, all except the CRAFT corpus and NLM-Chem are based on abstracts. As demonstrated here, high numbers of the entities appear in the body of the articles. Moreover, the abstract is often a summary of the rest of the article and has very different language constructs. These differences make the gold standard abstract datasets unsuitable for full-text article annotation tasks. The EPMCA corpus is larger than the CRAFT corpus and annotates Gene/Protein, Disease, and Organism entity types and comprises ≈ 1000 sentences with Gene/Protein and Disease association annotations, making it a unique addition to this domain.
The approach to construction is also slightly different. We used a triple-anonymous approach rather than the lead annotator approach used for the CRAFT corpus and we achieved a very high inter-annotator agreement across the corpus and all entity types. The use of the pre-annotated articles for curation is another important feature of this dataset, as it allowed us to start a large set of initial annotations to validate at the same time as the annotators identified new annotations. We hope that the two approaches complement each other to support the future development of novel deep learning approaches. We are currently using this corpus to train deep learning methods to address the challenges mentioned above for named entity recognition (NER). Moreover, we have achieved a better sentence co-occurrence based target-disease association identification through the improved NER system.
Number of Articles Disease-Gene/Protein-Organism     3. Overall annotation statistics comparing the existing Europe PMC dictionary-based text mining approach to the human curation for the selected 300 gold standard articles. Overall we have gained around 11k term annotations, with the highest gain existing for the Gene/Protein category. We report unique term count based on the string match and how many normalise to a database identifier of the databases mentioned above rather than unique database identifier counts. are confirmed as correct by the human annotators, although they also added/annotated a significant number of previously unidentified/unannotated terms. The Europe PMC pipeline misses a proportion of these terms due to outdated dictionaries. The removed terms are often common English words or short acronyms.
Acknowledgements We thank the curators at Molecular Connections (https://www. molecularconnections. com/) for their biocuration efforts. We would also extend our thanks to  Heidi J. Imker  from  University of Illinois at Urbana-Champaign  and  Melissa Harrison Matt Jeffryes , and  Mariia Levchenko  from team  Europe PMC  for their valuable suggestions. Last but not least, we would like to thank  Europe PMC  and  Open Targets  for strategic guidance on the development of the Europe PMC corpus. This work was supported by: the  European Molecular Biology Laboratory-European Bioinformatics Institute  (X. Y, A. V);  Europe PMC  grant, provided by 37 funders of life science research (https://europepmc. org/Funders/) under  Wellcome Trust  Grant  221523  (A. V, V. V) and the  OpenTargets grant   2056  (S.
have embedded NLP processes in their data workflows to serve their user community better. Developement of such NLP tools require the availability of open data (full-text corpora). Thanks to the biomedical text mining community, which has endorsed open data; resources such as PubMed, PubMed Central and Europe PMC provide open access abstracts and full-text for researchers to download. The COVID-19 Open Research Dataset Challenge (CORD-19 dataset).