DOI: 10.5281/zenodo.3837305
title: CodiEsp corpus: gold standard Spanish clinical cases coded in ICD10 (CIE10) - eHealth CLEF2020

description: <strong>Introduction</strong> These are the train, development and test sets of the CodiEsp corpus. Train, development and test have gold standard annotations. In addition, the unannotated background set is also distributed. All documents are released in the context of the CodiEsp track for CLEF ehealth 2020 (http://temu.bsc.es/codiesp/). The CodiEsp corpus contains manually coded clinical cases. All documents are in Spanish language and CIE10 is the coding terminology (it is the Spanish version of ICD10-CM and ICD10-PCS). The CodiEsp corpus has been randomly sampled into three subsets: the train, the development, and the test set. The train set contains 500 clinical cases, and the development and test set 250 clinical cases each. CodiEsp participants must submit predictions for the test and background set, but they will only be evaluated on the test set. <strong>Please cite if you use this dataset:</strong> Antonio Miranda-Escalada, Aitor Gonzalez-Agirre, Jordi Armengol-Estapé and Martin Krallinger. Overview of automatic clinical coding: annotations, guidelines, and solutions for non-English clinical cases at CodiEsp track of CLEF eHealth 2020. In CLEF (Working Notes). 2020 <pre><code>@inproceedings{miranda2020overview, title={Overview of automatic clinical coding: annotations, guidelines, and solutions for non-english clinical cases at codiesp track of CLEF eHealth 2020}, author={Miranda-Escalada, Antonio and Gonzalez-Agirre, Aitor and Armengol-Estap{\'e}, Jordi and Krallinger, Martin}, booktitle={Working Notes of Conference and Labs of the Evaluation (CLEF) Forum. CEUR Workshop Proceedings}, year={2020} }</code></pre> <strong>Annotation quality</strong> Inter-annotator agreement: 88.6% for diagnosis coding, 88.9% for procedure coding and 80.5% for the textual reference annotation. For more information, see the paper. <br> <strong>Zip structure</strong><br> Four folders: train, dev, test and background. Each one of them contains the files for the train, development, test and background corpora, respectively. <strong>train, dev and test</strong> folders have: 3 tab-separated files with the annotation information relevant for each of the 3 sub-tracks of CodiEsp. A subfolder named <em>text_files</em> with the plain text files of the clinical cases. A subfolder named <em>text_files_en</em> with the plain text files machine-translated to English. Due to the translation process, the text files are sentence-splitted. The <strong>background</strong> folder has only <em>text_files</em> and <em>text_files_en</em> subfolders with the plain text files. <br> <strong>Format</strong><br> The CodiEsp corpus is distributed in plain text in UTF8 encoding, where each clinical case is stored as a single file whose name is the clinical case identifier. Annotations are released in a tab-separated file. Since the CodiEsp track has 3 sub-tracks, every set of documents (train and test) has 3 tab-separated files associated with it. For the sub-tracks CodiEsp-D and CodiEsp-P, the file has the following fields: <pre>articleID ICD10-code </pre> Tab-separated files for the sub-track CodiEsp-X contain extra fields that provide the text-reference and its position: <pre>articleID label ICD10-code text-reference reference-position</pre> <br> <strong>Corpus summary statistics</strong><br> The final collection of 1000 clinical cases that make up the corpus had a total of 16504 sentences, with an average of 16.5 sentences per clinical case. It contains a total of 396,988 words, with an average of 396.2 words per clinical case. <strong>Resources:</strong> <strong>Web</strong> <strong>Citation: </strong>Antonio Miranda-Escalada, Aitor Gonzalez-Agirre, Jordi Armengol-Estapé and Martin Krallinger. Overview of automatic clinical coding: annotations, guidelines, and solutions for non-English clinical cases at CodiEsp track of CLEF eHealth 2020. In CLEF (Working Notes). 2020 <strong>Silver Standard corpus</strong> <strong>Annotation guidelines</strong> <strong>YouTube presentations</strong> <strong>Participant codes</strong> For more information, visit the track webpage: http://temu.bsc.es/codiesp/ or email us at encargo-pln-life@bsc.es Copyright (c) 2019 Secretaría de Estado para el Avance Digital
description: Funded by the Plan de Impulso de las Tecnologías del Lenguaje (Plan TL).
description: {"references": ["Villegas M, de la Pe\u00f1a S, Intxaurrondo A, Santamaria J, Krallinger M. Esfuerzos para fomentar la miner\u00eda de textos en biomedicina m\u00e1s all\u00e1 del ingl\u00e9s: el plan estrat\u00e9gico nacional espa\u00f1ol para las tecnolog\u00edas del lenguaje. Procesamiento del Lenguaje Natural. 2017(59):141-4."]}