GDLLM : A Global Distance-aware Modeling Approach Based on Large Language Models for Event Temporal Relation Extraction (2025)

Abstract: In Natural Language Processing(NLP), Event Temporal Relation Extraction (ETRE) is to recognize the temporal relations of two events. Prior studies have noted the importance of language models for ETRE. However, the restricted pre-trained knowledge of Small Language Models(SLMs) limits their capability to handle minority class relations in imbalanced classification datasets. For Large Language Models(LLMs), researchers adopt manually designed prompts or instructions, which may introduce extra noise, leading to interference with the model’s judgment of the long-distance dependencies between events. To address these issues, we propose GDLLM, a Global Distance-aware modeling approach based on LLMs. We first present a distance-aware graph structure utilizing Graph Attention Network(GAT) to assist the LLMs in capturing long-distance dependency features. Additionally, we design a temporal feature learning paradigm based on soft inference to augment the identification of relations with a short-distance proximity band, which supplements the probabilistic information generated by LLMs into the multi-head attention mechanism. Since the global feature can be captured effectively, our framework substantially enhances the performance of minority relation classes and improves the overall learning ability. Experiments on two publicly available datasets, TB-Dense and MATRES, demonstrate that our approach achieves state-of-the-art (SOTA) performance.

## introduction in natural language processing (nlp), event temporal relation extraction (etre) aims to identify temporal connections between event pairs. as illustrated in figure 1 (a), in the given sentence, the relation between the target event1 continues and event2 grip is is_included. much of the existing studies pay attention to the crucial role of language models for etre, especially small language models @xcite . some research utilizes slms to form certain rules for temporal realtion @xcite @xcite . prior sota model mulco @xcite combines gnns and the model of bert variants via multi-scale knowledge distillation to enhance the performance of etre. however, the restricted pre-trained knowledge of slms limits their capability to handle minority class relations in imbalanced classification datasets @xcite . although some researchers have invested substantial effort in it @xcite @xcite , the performance of their models is still suboptimal on two popular datasets, ma-tres @xcite and tb-dense @xcite . as depicted in figure 1 (b), the relation "simultaneous" that refers to two events happening simultaneously only takes 1.5% in the tb-dense dataset, while "vague" has 47.7% @xcite . ## method in this section, we introduce the overall architecture of our proposed gdllm method, which is depicted in figure 2 . firstly, we formulate the etre task. secondly, we introduce the llmbased probability distribution prediction module. thirdly, we present the distance-aware graph attention module for capturing long-distance features and a temporal feature learning paradigm for learning short-distance features. finally, our framework will enhance the performance of minority relation classes in the imbalanced classification dataset by capturing the global feature. ## related work earlier studies for etre predominantly rely on machine learning @xcite . afterwards, some research integrates pre-trained language models to capture temporal semantics in the context @xcite @xcite . it is also worth noting that more and more studies focus on the special structure of event temporal relations. one of the widely employed graph-based methods is gnns. different gnn-based methods have been proposed to better learn the relation cues @xcite . differently, other researchers embed events in hyperbolic spaces for better hierarchical structure modeling @xcite . prior sota model mulco @xcite combines gnns and the model of bert variants via multi-scale knowledge distillation. there are also studies that tackle data scarcity or imbalance @xcite @xcite @xcite , while some work designs certain temporal rules @xcite @xcite . with the rapid development of llms, researchers pay great attention to the question-answer (qa) mechanism @xcite . similar to the zero-shot studies, another work proposes a variety of valuable prompt explanations @xcite or utilizes a unified framework @xcite . appendix c shows the performance of the previous llm-based methods versus our approach on the matres dataset. ## conclusion in this paper, we propose gdllm, a global distance-aware modeling approach based on llms. specifically, we present a distance-aware graph structure utilizing gat to assist llms in capturing long-distance dependency features. additionally, we design a temporal feature learning paradigm based on soft inference to augment the event relation extraction with a short-distance proximity band. our framework also substantially enhances the performance of minority relation classes and improves the overall learning ability. extensive parameters such as the dropout rates, class weights, and the learning rate. we initialize the search process with a set of randomly sampled hyperparameter points. for each iteration, the hebo algorithm calculates the acquisition function values for all points in the search space based on the current surrogate model. the hyperparameter point with the maximum acquisition function value is then selected and evaluated on the model. after obtaining the evaluation result, the surrogate model is updated to incorporate this new information. compared to traditional hyperparameter optimization methods such as random search and grid search, hebo can more efficiently explore the hyperparameter space by leveraging the information from previously evaluated points.