Triple-view Event Hierarchy Model for Biomedical Event Representation (2024)

Abstract: “Biomedical event representation can be applied to various language tasks. A biomedical eventoften involves multiple biomedical entities and trigger words, and the event structure is complex.However, existing research on event representation mainly focuses on the general domain. Ifmodels from the general domain are directly transferred to biomedical event representation, theresults may not be satisfactory. We argue that biomedical events can be divided into three hierar-chies, each containing unique feature information. Therefore, we propose the Triple-views EventHierarchy Model (TEHM) to enhance the quality of biomedical event representation. TEHM ex-tracts feature information from three different views and integrates them. Specifically, due to thecomplexity of biomedical events, We propose the Trigger-aware Aggregator module to handlecomplex units within biomedical events. Additionally, we annotate two similarity task datasetsin the biomedical domain using annotation standards from the general domain. Extensive exper-iments demonstrate that TEHM achieves state-of-the-art performance on biomedical similaritytasks and biomedical event casual relation extraction.Introduction”

## introduction biomedical events consist of trigger words, biomedical entities, and specific actions of entities @xcite . a complex biomedical event often contains multiple simple biomedical events, making the structure of event texts quite complex. biomedical event representation refers to the process of converting biomedical event texts into machine-readable formats. event representation is applied in tasks such as event prediction, event relation extraction, and event information extraction @xcite , which can improve the performance of related tasks. therefore, obtaining a representation of biomedical events is necessary. currently, in the general domain, the works on event representation focus on simple events, such as "he was frightened football" which typically follow a subject-verb-object structure. these works usually rely on pre-trained language models (plms) for text encoding @xcite . however, biomedical events involve multiple biomedical entities, and the event structure is not a simple subject-verb-object format. for example, in the biomedical event "cooperation among nf-kappa b-, ap-1-and nf-atbinding sequences is required for induction of the gm-csf gene through pkc-and ca2+-signaling pathways downstream of t-cell activation" there are multiple verbs and the relationships between verbs are nested within each other @xcite . if we directly transfer models from the general domain for biomedical event representation, we cannot extract and integrate the structural features of biomedical events due to their complexity. currently, existing works lack biomedical event representation models capable of handling complex structures. we argue that although biomedical events are complex, they can be categorized into three hierarchies. as illustrated in figure 1 (a), we divide a complex biomedical event into complex units, simple units, and basic units. hierarchically organizing biomedical events and deriving biomedical event representations from these three hierarchies is an effective approach. therefore, based on this perspective, we propose the triple-views event hierarchy model (tehm). the tehm learns triple hierarchical features of biomedical events from three views. from the complex units view, we propose the trigger-aware aggregator module, which focuses on extracting features from complex units with the trigger word as the core. it is based on the architecture of graph neural networks and has been improved to better suit the characteristics of biomedical events for feature extraction. from the simple units view, we introduce the context-aware intensifier module, which integrates context information through prompt learning to explore implicit interaction features within simple events. from the basic units view, we present the entity relation integrator module, which obtains feature encoding matrices for biomedical entities. this module constructs an entity relation network and employs the algorithm proposed in this paper bioer to train entity feature matrices. after obtaining event representations, how to evaluate the quality of event representations is also a problem that needs to be addressed currently. due to the significant differences in datasets between the general domain (which consists of simple events) and the biomedical domain, there is currently a notable lack of effective evaluation tasks to directly validate the quality of biomedical event representations. to address this issue, we adopt the same annotation standards @xcite as those used for the tasks in the general domain and manually annotate two validation tasks relevant to assessing the quality of biomedical event representations: bio hard similarity task and bio transtive sentence similarity task. additionally, we validate the effectiveness of biomedical event representation on standard biomedical event causal relation extraction datasets. after extensive experiments, tehm achieves improvements of 25.9% (accuracy) in bio hard similarity and 0.045 (spearman correlation coefficient) in bio transitive sentence similarity. additionally, it improves 3.0% in the f1 score on hahn powell's dataset for biomedical event causal relation extraction, reaching the current sota. in summary, the contributions of this paper can be summarized as follows: ## triple-view event hierarchy model(tehm) figure 2 illustrates the framework of the tehm model. the input to the tehm model is the event text, which is simultaneously processed by the feature extraction modules of tehm to obtain features from three different views of the event. these feature vectors are then concatenated to obtain the representation of the final biomedical event. tehm employs a contrastive learning approach to training the model, which is a form of unsupervised learning. ## biomedical event representation evaluation dataset biot2e @xcite : this dataset provides structured biomedical event text information, including event triggers, biomedical entities, and entity-specific attributes. after filtering out texts without events and single-trigger events, we obtain a total of 36,662 biomedical events. ## related work due to the significant role of event representations in downstream applications, various methods for event representation have been introduced in current research @xcite . long short-term memory (lstm) @xcite effectively integrates the sequential information of events into a unified model. additionally, the fusion of external knowledge in jointly trained models provides a promising direction for capturing potential relationships among events @xcite . it's worth noting that the application of contrastive learning techniques in the field of computer vision, such as deep infomax @xcite , moco @xcite , and simcse @xcite , has significantly improved the performance of unsupervised image classification, even surpassing supervised methods. swcc @xcite and promptcl @xcite both use contrastive learning architecture. promptcl has achieved sota performance in event representation. recent studies @xcite have introduced dropout noise as a data augmentation technique, highlighting its effectiveness in nlp tasks compared to many traditional augmentation methods. ## conclusion we introduce the task of biomedical event representation for the first time and establish the evaluation tasks for biomedical event representation. we will make the relevant datasets publicly available for researchers to conduct further studies. due to the complexity of biomedical events, we propose a hierarchical structure for biomedical events and design the tehm model for biomedical event representation. tehm is capable of obtaining features of biomedical events from three perspectives and capturing subtle semantic features through contrastive learning. particularly, based on the characteristics of biomedical events, we propose a trigger-aware aggregator to extract relational features within complex biomedical events. additionally, we inject context information into the model using prompt learning. experimental results demonstrate that we achieve state-of-the-art performance in biomedical similarity tasks. we also validate the effectiveness of event representation in improving task performance in biomedical causal relation extraction, which also achieves state-of-the-art performance.