GL i M : Integrating Graph Transformer and LLM for Document-Level Biomedical Relation Extraction with Incomplete Labeling (2025)

Abstract: Document-level relation extraction (DocRE) identifies relations between entities across an entire document. However, as the number and complexity of entities and entity-pair relations grow, the problem space expands quadratically, causing incomplete annotations and frequent false negatives, especially in biomedical datasets due to high construction costs. This leads to low recall in real-world scenarios. To address this, we propose GLiM, a novel framework that reduces the problem space using a graph-enhanced Transformer-based model and leverages large language models (LLMs) for reasoning. GLiM employs a cascaded approach: first, a graph-enhanced Transformer processes entity-pair relations with finer granularity by dynamically adjusting the graph size based on the number of entities; then, LLM inference handles challenging cases. Experiments show that GLiM boosts average recall and F1 scores by +6.34 and +4.41, respectively, outperforming state-of-the-art models on biomedical benchmarks. These results demonstrate the effectiveness of combining graph-enhanced Transformers with LLM inference for biomedical DocRE. Code will be released at https://github.com/HaoFang10/GLiM.

## introduction relation extraction (re) plays a crucial role in information extraction (ie), aiming to identify relations between entities in a given text. documentlevel relation extraction (docre) extends this task by identifying relations between all entity pairs across an entire document. docre is of paramount importance for performance enhancement of downstream applications such as question answering, knowledge graph construction, and recommendation systems. *corresponding authors. compared to sentence-level relation extraction, which focuses on identifying entity relations within a single sentence, docre faces greater challenges in dataset quality. as the number of entities increases, the problem space grows quadratically, complicating the annotation process. moreover, docre needs to handle longer contexts and more complex relation types, leading to inevitable issues such as incomplete labeling and a high prevalence of false negatives. these challenges are especially prominent in biomedical docred datasets, such as chemdisgene. in the automatically annotated training set for chemdisgene, the average number of relations per document is only about one-third of that in the expert-annotated test set. consequently, models trained on such datasets often suffer from low recall in real-world applications, making this a critical issue to address. with the remarkable capabilities demonstrated by large language models (llms) across various domains, reframing relation extraction as a text generation task and leveraging llms has emerged as a promising approach to overcoming the aforementioned challenges. however, the relatively underwhelming performance of llms in re tasks is largely due to the massive problem space inherent to docre. therefore, our focus is on reducing the problem space while leveraging the intrinsic knowledge of llms to recover false negative relations without compromising model performance. in light of the above, we propose an innovative cascaded framework named graph llm integration model (glim). glim first employs a transformer-based model for initial processing, which not only produces preliminary results but also significantly reduces the problem space. this reduction allows subsequent llm to fully exploit its intrinsic knowledge, thereby improving both recall and f1 scores. furthermore, to enhance the first-step model's ability to capture finer-grained and complex entity relations, we incorporate adaptively-adjusted graph structures into the transformer-based model. this enhancement enables glim to achieve state-ofthe-art (sota) performance on the biomedical biored and chemdisgene benchmark datasets, even without utilizing llms. when llms are used, the framework achieves further significant performance gains. an example of glim is shown in figure 1 . the main contributions of this paper include: ## conclusion and future work in this paper, we propose glim, a novel framework that combines transformer-based pre-trained model with llm inference. this approach significantly improves recall and overall f1 scores for biomedical document-level relation extraction. notably, by incorporating a graph transformer into the transformer-based model to enhance finegrained relation handling, glim surpasses the performance of current sota models. furthermore, its effectiveness is validated through experiments on three benchmark datasets. for future work, we plan to further explore the relation reasoning capabilities of llm, with a focus on improving precision while maintaining high recall to achieve a higher f1 score.