GGPONC : A Corpus of G erman Medical Text with Rich Metadata Based on Clinical Practice Guidelines (2020)

Abstract: The lack of publicly accessible text corpora is a major obstacle for progress in natural language processing. For medical applications, unfortunately, all language communities other than English are low-resourced. In this work, we present GGPONC (German Guideline Program in Oncology NLP Corpus), a freely dis tributable German language corpus based on clinical practice guidelines for oncology. This corpus is one of the largest ever built from German medical documents. Unlike clinical documents, clinical guidelines do not contain any patient-related information and can therefore be used without data protection restrictions. Moreover, GGPONC is the first corpus for the German language covering diverse conditions in a large medical subfield and provides a variety of metadata, such as literature references and evidence levels. By applying and evaluating existing medical information extraction pipelines for German text, we are able to draw comparisons for the use of medical language to other corpora, medical and non-medical ones.

## introduction the synthesis of validated experience in the form of clinical practice guidelines (cpgs) serves as a basis for evidence-based decision making in clinical practice. to leverage the knowledge in cpgs for clinical decision support systems, e.g., for integration with electronic health records or automated evaluation of adherence to these guidelines, machine-readable versions of cpgs are necessary. however, cpgs today are disseminated mostly as free-text documents, with few formal elements. thus, natural language processing (nlp) * authors marked by * equally share first authorship. might be helpful to automatically extract information from these unstructured texts and transform them into a structured, or even machine-executable, format. as cpgs are also specific to their country of origin, they are usually formulated in the respective native language, so nlp technology has to be adapted properly. a major reason for the progress in nlp research in the past years is the public availability of large text corpora (and attached metadata). yet, for documents originating from a clinical context, the protection of personal information is a major requirement imposed by legal privacy regulations. some research initiatives, e.g., i2b2 @xcite , mimic-iii @xcite , the shared task of social media for health @xcite , or clef ehealth @xcite ) make de-identified clinical text document collections available under the conditions of data use agreements (dua). besides, databases of biomedical research articles like pubmed provide an abundant amount of examples for medical language. however, with only few exceptions, such easily accesssible text corpora are hardly available for the german @xcite and other non-english languages. today, there is no viable solution for sharing even de-identified clinical texts in germany. in effect, not only are large-scaled research datasets missing but also pretrained language models for german medical language, such as an equivalent of biobert @xcite . to address (1) the lack of available german medical text resources for nlp research, and (2) the need for machine-readable cpgs, we constructed a corpus based on a set of german cpgs for oncology. the german guideline program in oncology (ggpo) @xcite , operated by the association of the scientific medical societies in germany, the german cancer society and the german cancer aid, is in a unique position to enable this research, as their guidelines are also provided via a mobile app @xcite . hence, this data set is already available in a semi-structured format with rich, formatted metadata, resulting in a much higher data quality than data extracted a posteriori from pdf versions of the guidelines. an excerpt from the xml version of ggponc is depicted in listing 1. other than clinical text relating to individual patients, we deal with scientific medical text that does not contain any privacysensitive data requiring de-identification. this way, we can provide access to ggponc for other researchers via a dua. ## related work due to legal data protection measures, germanlanguage clinical text corpora are extremely rare and existing ones almost impossible to (re)use. typically, accessibility is restricted to research staff only within the lifetime of a project and blocked for the outside world. in cpgs as a target for automated text analytics have been much less utilized compared to other scientific publications and clinical documents. most of that work took place in the context of formalizing cpgs as computer-interpretable guidelines @xcite . @xcite describe an approach to fill in a semi-structured guideline elements model template by segmenting unstructured guidelines using linguistic patterns. an evaluation was run on 18 french guidelines. @xcite describe the extraction and instantiation of linguistic templates for guideline formalization, evaluated on a dutch guideline for breast cancer treatment. german cpgs were the focus of @xcite who adapted apache ctakes to detect german umls concepts and evaluated their approach on a single german breast cancer guideline. zadrozny et al. (2017) outline a system which identifies contradictions and disagreements in english cpgs. some authors have focused on extracting more task-specific information, such as activities @xcite , process structures @xcite @xcite or negation triggers @xcite . @xcite al. (2013) lstms and fuzzy rules to extract "action takers", "symptoms", "actions" and "purposes" from cpgs, recognize recommendations and predict the grade ## discussion & limitations of this study while the initial results of the information extraction pipelines we employed are promising, there is much room for improvement. the extraction of genes suffers from a large number of false positives, as there are many common german words (e.g., "gilt", "dar") and three-letter-acronyms (e.g., "cll", "hcc") with strings identical with gene names in our large dictionary (around 562k entries). thus, supplying an improved gene tagger which balances german lexical noise with advanced capabilities of gene taggers for english texts will be a desideratum of future research. the german umls has a number of issues, which severely affect our dictionary-based entity extraction pipelines. first and foremost, its vocabulary size is extremely limited. the english umls contains over 6.5m entries and the spanish one around 750k, whereas there are only around 234k entries in the german version (3.6% coverage of the english version). recently introduced drugs are missing in the umls chemistry category, so a more up-to-date dictionary of drug names is also needed for future work. moreover, the surface representation of german umlauts is notoriously inconsistent in umls, e.g., "ä" is sometimes transcribed as "ae" or even simplified as "a", as in "eingeschraenkte nierenfunktion", which results in an increasing false negative rate. all of these factors contribute to rather low recall values, as shown in the accuracy of dictionary matches further decreases due to inconsistent handling of compounds throughout the corpus. for instance, "pankreaskarzinompatienten" (patients with pancreas carcinoma) would not be detected as an entity, whereas hyphen-connected "pankreaskarzinom-patienten" would, yielding two entities (disorders and living beings), respectively. in this case, we would choose to annotate the whole compound as living beings to avoid annotation on a subword level, which could be addressed using a more finely adapted tokenization algorithm. while precision and recall of the rule-based tnm extraction approach are high on ggponc, one has to be careful as certain tnm expressions can cause context-dependent semantic ambiguities. for instance, "v1" and "v2" are valid tnm components referring to venous invasion, but are also detected in the wikiwarsde corpus referring in this context to german missiles from world war ii. ## conclusion we presented ggponc, one of the currently largest corpora composed of german medical texts, assembled from the cpgs in oncology and equipped with rich structure information and metadata. we applied information extraction pipelines to extract a variety of named entity classes. despite the limitations we discussed, the information extracted so far can be of immediate use to enable semantic search functionalities in the guideline app @xcite , precision medicine search engines @xcite or in clinical decision support systems @xcite . our results indicate that ggponc shares many characteristics with existing clinical text corpora. this can facilitate the development of machine learning-based nlp algorithms for german clinical text. @xcite suggest that combining corpora covering different parts of medical terminology can improve the utility of trained word embeddings. in addition to the german documents discussed in this work, some of the ggpo guidelines have an additional english version, which could be used to construct parallel corpora for research in multilingual medical nlp. extending our work to clinical guidelines from other medical specialities besides oncology will be a straightforward way to extend the volume of the corpus, provided that the document structures can be harmonized across medical societies. however, as most cpgs are distributed as pdf documents, extraction of the plain text content from these can result in quality issues not encountered in this work. the structured metadata of the corpus provide ample opportunities for future research. for instance, the corpus can be used as a resource for evidence-based medicine summarization, as it contains mappings from literature references to recommendation statements and evidence levels. as we plan to create future versions of the corpus based on updated guideline versions, the extracted concepts can also be used to track changes in cpgs, like the emergence of new treatments and other changes in recommended clinical practice. we envision to combine information extracted from scientific articles, such as study reports, or clinical trial registers with information from cpgs to automatically detect whether these cpgs might be outdated given changes in the underlying evidence base. in addition to the existing annotations for a wide selection of umls semantic types, we can easi-