Multilingual Negation Scope Resolution for Clinical Text (2021)

Abstract: Negation scope resolution is key to high-quality information extraction from clinical texts, but so far, efforts to make encoders used for information extraction negation-aware have been limited to English. We present a universal approach to multilingual negation scope resolution, that overcomes the lack of training data by relying on disparate resources in different languages and domains. We evaluate two approaches to learn from these resources, training on combined data and training in a multi-task learning setup. Our experiments show that zero-shot scope resolution in clinical text is possible, and that combining available resources improves performance in most cases.

## introduction information extraction (ie) from clinical text, such as electronic health records (ehr) or clinical trial narratives, is a promising application of machine learning that can potentially benefit many areas of the health sector @xcite . ie systems are applied to facilitate administrative tasks by assigning medical codes @xcite , to extract phenotype information about patients @xcite , and to improve patient care by monitoring healthcare associated infections @xcite and adverse drug events (ade) @xcite . results are promising, but the majority of work focuses on clinical text in english @xcite . this puts at a disadvantage patients in countries where health narratives are recorded in languages other than english. this gap can be overcome by multilingual ie systems that are applicable to text in multiple languages. such multilingual systems can, on the one hand, lead towards improving healthcare within non-english speaking countries. on the other hand, they allow to gather information across countries, which is particularly interesting datasets are grouped according to the type of negation cue they annotate (syntactic (s), lexical (l), morphological (m), see further details in ). numbers for datasets not listed in section 4 (uam @xcite and uhu @xcite ) are taken from @xcite . nubes s counts only syntactic cues in nubes. for rare diseases with few cases per country, and can increase the statistical power of an analysis @xcite . with this paper, we contribute to improving ie from clinical text in languages other than english, which is a step towards improving healthcare for all. negation is a phenomenon that has received considerable attention in ie models for clinical and biomedical text, as there is large interest in identifying negated concepts @xcite @xcite , e.g. negated medical events @xcite or negated drug-drug interactions @xcite . absence of symptoms or the fact that chemical re-actions are not observable are crucial knowledge in the clinical and biomedical domain @xcite . this is reflected in figure 1 , which shows the percentage of negated sentences in clinical compared to non-clinical text. explicitly integrating negation information into machine learning (ml) models can improve performance for relation extraction @xcite and more general nlp tasks such as sentiment analysis @xcite and machine translation @xcite , which become increasingly popular for the clinical domain @xcite . even though pre-trained language models tailored to the biomedical and clinical domains @xcite @xcite now produce state-of-the-art results for several downstream tasks, such language models do not sufficiently capture the semantics of negation @xcite . one way of accessing negation information is to explicitly detect all negated words in a sentence, a task which is referred to as negation scope resolution @xcite . the task has recently been successfully approached by fine-tuning a pre-trained language model on labeled target data @xcite . for the clinical domain however, we cannot rely on multilingual labeled target data to be available. this is partly due to data privacy issues, which lead to few publicly available datasets in the clinical domain @xcite foot_0 , and partly due to the fact that other languages are underrepresented compared to english in clinical nlp @xcite . 2 in summary, when building a multilingual negation scope resolution system for clinical text, we are facing a lack of training data. our approach is hence to use the available data as best as possible to build a negation resolution model that works on data in multiple languages. negation scope resolution has also been considered for non-clinical text @xcite , and several datasets spanning various domains and including a small amount of non-english languages are available (see section 4). in this work, we investigate if and how these disparate data sources can serve as training resource to resolve negation scope in multiple languages in the clinical domain in a zeroshot setup. to enable transfer across languages we rely on multilingual bert (mbert) @xcite , a multilingual pre-trained language encoder that has proven capable of zero-shot cross-lingual transfer in other tasks foot_2 @xcite , and has recently been applied for zero-shot transfer to french clinical text @xcite . one challenge arising from the available scope resolution datasets is that they are annotated according to different annotation schemes (see section 3.3), raising the question if and how they can be combined as a training resource @xcite . we explore two strategies to handle disparities between the resources, a simple concatenation of datasets after a partial conversion of annotations, and a multi-task learning (mtl) setup, where each dataset is handled as a different task. the mtl setup also allows us to explore additional auxiliary tasks that can potentially help in resolving negation scope by making use of available resources. here, we consider a classification task for negated events in the clinical domain. ## related work recent work used large pre-trained language models, in particular bert, for negation scope resolution as a sequence labeling task @xcite and produced (1) el ojo derecho no completa procedure la aducción . ## negation scope resolution negation is a phenomenon in language that changes the truth value of a proposition. from a linguistic perspective, a negation signal or cue, i.e. an expression that indicates negation, is an operator, which has a scope over parts of a sentence. 7 in example sentence (1) in ## datasets in the following, we describe the clinical datasets as well as the training resources used in our experiments. a more detailed overview of annotation guidelines, negation types and negation components can be found in the survey of jiménez-zafra et al. ( 2020 ), as well as in the original works associated with the datasets. statistics on the available data can be found in ## experiments we split all datasets into 70% for training, 15% for validation, 15% for testing, and predict negation scope in the test splits of the three clinical datasets iula, nubes, and french. results are shown in for each model, we report the best combination of training datasets determined by the best average 17 note that our results are reported on the test sets with converted annotations, and hence not directly comparable to numbers reported on the original data. f1-score across the validation splits of the three clinical datasets. validation scores for other dataset combinations can be found in our code repository. as baselines, we report in-domain performance of mbert , which provides an upper baseline indicating the performance gain if annotated target data was available. we also include a punctuation baseline, as previous work has found that many test sentences are easy to label from cue token to next punctuation mark @xcite . the punctuation baseline labels all tokens following the negation cue until the next punctuation mark foot_12 as in scope. in all experiments, mbert is fine-tuned with the default hyperparameters for sequence labeling tasks implemented in mt-dnn, which means that we use the adamax optimizer with a learning rate of 5e-5 and a batch size of 8. the maximum sequence length is set to 512. 19 all models are trained for a maximum of 50 epochs, and the best model is selected according to f1-score on the bioscope validation split. ## conclusion negation is a frequent phenomenon in clinical text, and resolving its scope can benefit clinical ie tasks. as clinical ie has huge potential to improve healthcare, it should ideally be available regardless of language. availability in multiple languages, however, is hindered by a lack of annotated data for fine-tuning language models in task-specific data in the target language. in this paper, we show that even without labeled data in the target domain or target language, negation scope resolution in clinical text is possible, by fine-tuning a multilingual language model on available resources from other domains and languages. even though annotation schemes for negation scope differ, combining available resources for training improves performance in most cases.