GCN with External Knowledge for Clinical Event Detection (2021)

Abstract: In recent years with the development of deep learning and the increasing demand for medical information acquisition in medical information technology applications such as clinical decision support Clinical Event Detection has been widely studied as its subtask. However directly applying advances in deep learning to Clinical Event Detection tasks often produces undesirable results. This paper proposes a multi-granularity information fusion encoder-decoder frameworkthat introduces external knowledge. First the word embedding generated by the pre-trained biomedical language representation model (BioBERT) and the character embedding generatedby the Convolutional Neural Network are spliced. And then perform Part-of-Speech attention coding for character-level embedding perform semantic Graph Convolutional Network codingfor the spliced character-word embedding. Finally the information of these three parts is fusedas Conditional Random Field input to generate the sequence label of the word. The experimental results on the 2012 i2b2 data set show that the model in this paper is superior to other existingmodels. In addition the model in this paper alleviates the problem that “occurrence” event typeseem more difficult to detect than other event types.

## introduction electronic medical records are an inevitable product of medical information. the use of natural language processing (nlp) technology to effectively detect clinical events in electronic medical records becomes crucial as the number of electronic medical records rapidly grows. it has been widely studied because of its potential help in constructing clinical event lines, medical q & a, assisted diagnosis and other tasks. the task of clinical event detection (ced) is to identify the boundary of the event in the electronic medical record and determine its type. the event detection to identify the boundary and determine type is usually considered as a sequence labeling task. the emergence of deep learning has greatly improved the performance of the sequence labeling task model. bidirectional long short-term memory network (bilstm) is widely employed in sequence labeling tasks owing to its high power to learn the contextual representation of words. @xcite was the first to apply the bidirectional long short-term memory (bilstm) and the conditional random field (crf) to sequence labeling tasks. but bilstm needs to be processed sequentially over time, it cannot be calculated in parallel. instead, the transformer not only advantage in modeling the long-range context, but also fully make use of the concurrence power of gpus. @xcite found that due to its position coding problem, the performance of transformer in sequence labeling tasks is not as good as in other nlp tasks, and solves the position coding problem. however, due to the lack and particularity of clinical data, the performance of directly applying these technological advances to ced tasks is not ideal. for this reason, we need to use a large amount of medical information for tokens representation. in the past, word2vec @xcite or glove @xcite was used to train a large number of unlabeled clinical texts to generate word embedding. there are two problems with this method. on the one hand, it is difficult to obtain a large amount of clinical data. on the other hand, the obtained data is not standardized, many mistakes, various representations. @xcite proposed a pre-trained biomedical language representation model for biomedical text mining (biobert) in 2019, whose performance on various biomedical text mining tasks largely surpassed bert and previous advanced models. we use biobert for word embedding to solve the problem of poor model recognition performance caused by a large number of obscure professional terms in the medical field. in addition, the character-level features of words may show word features, for example, the beginning of "un" generally indicates negative characteristics. therefore, adding character-level encoding will also have an impact on improving the performance of the model, and can solve the problem of out of vocabulary(oov). @xcite , @xcite and @xcite have added character-level coding to the model of the english ner task and proved its effectiveness. this article uses convolutional neural network (cnn) as a character-level encoder for words @xcite . after the informatics integrated biology and bedside information (i2b2) sharing task was proposed in 2012 @xcite , we found that in order to better enable the event extraction task to better serve the later tasks such as disease diagnosis, the event types of the shared task also include "occurrence" type. however, the experimental results of the organizations that participated in the challenge in the past show that the "occurrence" type is more difficult to predict than other types of events, and it is not due to the small amount of data in this type. we analyzed the reasons and found a solution. (1) most of the events in this type are nouns or verbs. we use the part-of-speech generated by stanford corenlp tools as attention to help them identify. (2) the past practice always solves the problem of poor recognition based on the particularity of the medical field, while ignoring the language commonality between the medical field and the general field, and incidents of "occurrence" type is more inclined to event recognition in the general field. for this reason, this article introduces external knowledge to alleviates the problem that occurrence event type seem more difficult to detect than other event types. at the same time, these external knowledge are helpful to the recognition of the event span and improve the overall performance of the model. in general, the contributions of this paper are as follows: ## related work constructing the clinical timeline is crucial to the patient diagnosis and treatment. the 2012 informatics for integrating biology and the bedside (i2b2) shared task @xcite was the identification and linking of mentions of temporal expressions (tes) (eg, dates, times, durations, and frequencies) and clinically relevant events (eg, patients problems, tests, treatments) in narratives. for this task, previous research on deep learning methods is mainly based on recurrent neural networks (rnn) @xcite , convolutional neural networks (cnn) @xcite , bilstm @xcite and attention @xcite methods. ## model the model of this article mainly includes four parts: character-word embedding module, part-of-speech attention module, semantic gcn module, crf decoding module. figure 1 shows an overview of our model, where cnn is a character-level encoder for words, biobert is a word-level encoder for words, and the semantic analysis tool uses stanford corenlp. first, input a sentence to generate character-level embedding of words through cnn, and generate word-level embedding of words through biomedical pre-training language model biobert, the two are spliced together. and then perform part-of-speech attention coding for character-level embedding, perform semantic graph convolutional network coding for the spliced character-word embedding. finally, the information of these three parts is fused as crf input to generate the sequence label of the word. we will introduce each part in detail in the following chapters. figure 1 : the framework of our model. ## conclusion this paper proposes a multi-granularity information fusion encoder-decoder framework.this framework uses the pre-trained language model (biobert) to generate word-level features, and solves the problem of poor model recognition performance caused by obscure professional terms in electronic medical records. the graph convolutional network that introduces external knowledge improves the performance of the model in identifying "occurrence" type. further improve the overall performance of the model. experiments on the 2012 i2b2 challenge dataset show that our model achieves superior performance than other existing models.