IPED : An Implicit Perspective for Relational Triple Extraction based on Diffusion Model (2024)

Abstract: Relational triple extraction is a fundamental task in the field of information extraction, and a promising framework based on table filling has recently gained attention as a potential baseline for entity relation extraction. However, inherent shortcomings such as redundant information and incomplete triple recognition remain problematic. To address these challenges, we propose an Implicit Perspective for relational triple Extraction based on Diffusion model (IPED), an innovative approach for extracting relational triples. Our classifier-free solution adopts an implicit strategy using block coverage to complete the tables, avoiding the limitations of explicit tagging methods. Additionally, we introduce a generative model structure, the block-denoising diffusion model, to collaborate with our implicit perspective and effectively circumvent redundant information disruptions. Experimental results on two popular datasets demonstrate that IPED achieves state-of-the-art performance while gaining superior inference speed and low computational complexity. To support future research, we have made our source code publicly available online.

## introduction the extraction of relational triples has been an important and fundamental task in knowledge graph construction @xcite , aiming to recognize triples in the form of (head entity, relation, tail entity) from unstructured text. current research in information extraction can be categorized into two main approaches: the joint extraction models, which utilize a simultaneous style, and the pipeline models, which utilize a twoencoder methodology to extract entities and relations. while the pipeline framework is criticized for serious error propagation and lack of interaction between its two subtasks @xcite , leading to performance decline, many recent joint extraction models have begun to thrive due to their enhanced capability to deal with complex scenarios such as single entity overlap (seo), entity pair overlap (epo), and subject object overlap (soo). among these popular joint extraction methods, one baseline, known as the table-filling method, has gained favor in recent research. compared to a multi-task joint structure, this method features a table of token pair units that are to be filled and decoded in a single step. in this way, it avoids exposure bias and error propagation, challenges that most methods cannot fully overcome. particularly for recently proposed models @xcite @xcite , these can employ a novel table-filling strategy to simplify the decoding process and enhance information interaction. despite many unique advantages over tablefilling methods, some flaws still remain to be addressed. (1) the abundance of negative tagging in a table, which is significantly denser than positive tagging, leads to imbalanced labeling and redundant information @xcite . to the best of our knowledge, this is a universal issue across all table-filling models. this imbalance results in a bias towards negative tagging and heightened computational complexity. (2) many table-filling strategies fail to extract all scenarios of triples, leading to decreased recall @xcite . even in the recent significant work @xcite , entities consisting of a single token in a triple cannot be properly extracted due to conflicts arising from multiple labels in one element. (3) once a sentence contains multiple triples, the separate labels of different triples may intersect in a single element, causing confusion in decoding all ground-truth triples. many models @xcite employ de-coding algorithms that match labels based on the nearest-neighbor principle, which can lead to error associations within a triple. (4) a line of models, not limited to table-filling ones, exhibit poor learning performance on the webnlg dataset in contrast to the nyt dataset and they attribute it to the vast number of predefined relations in the former dataset @xcite . after conducting a detailed observation and analysis of their models, it is observed that all existing table-filling-based methods are consistently constrained by the approach of utilizing a classifier to tag each table element explicitly. mainly because of this, most of them can hardly escape the challenges mentioned above, despite attempts to introduce innovative labeling strategies and creative decoding algorithms. this constraint necessitates traversing each element of the table, consequently leading to a substantial number of negative samplings. this explicit way of assigning a fixed label to each element can not cope with scenarios when one element requires multiple labels, leading to the inability to recognize all triples and confusion in the regions where triple labels intersect. additionally, certain decoding strategies, designed in response to this approach, often result in incorrectly matched labels for a triple. to address the aforementioned issues at a fundamental level, instead of explicitly labeling all the elements, we formulate a fresh perspective to implicitly fill the tables using a block-covered approach. in this method, blocks defined by four edges @xcite and one level are refined within a three-dimensional table (multiple two-dimensional tables stacked together). in alignment with this implicit approach, we introduce a generative model designed to recover all blocks within the tables. specifically, our proposed blockdenoising diffusion model (blk-ddm) can progressively refine the edges and levels of the initialized blocks step by step through a reverse process, ensuring the blocks precisely cover the ground truth triples horizontally, vertically, and deeply. as a result, our model naturally disregards redundant information by leaving the negative spaces alone rather than classifying them. furthermore, our approach allows for the adequate recognition of all potential triples, as the proposed blocks can overlap implicitly. in contrast to previous decoding algorithms that match explicit labels, our proposed simple but effective parallel boundary emitting strategy (pbes) for decoding has the capability of extracting all triples accurately, circumventing error association challenges and significantly accelerating inference. additionally, our denoising diffusion process enables the gradual refinement of specific fine-grained relation types within triples, enhancing performance in large-relation datasets such as webnlg (demonstrated in section 4.8). experimental results on two datasets, nyt and webnlg, demonstrate that our model achieves state-of-the-art performance and exhibits superior efficiency in inference. ## methodology this section firstly introduces our implicit tablefilling strategy and its corresponding decoding algorithm. secondly, the formulation of the block-denoising diffusion model is presented. finally, the network architecture of our model is detailed. ## conclusion this paper proposes an implicit approach to relational triple extraction, diverging from the explicit tagging methods of prior table-filling methods, thereby addressing several prevailing issues. via denoising the edges and levels of noisy blocks, our introduced block-denoising diffusion model incrementally generates ground truth blocks, which can be swiftly and precisely converted into triples with our decoding algorithm pbes. moreover, our network architecture incorporates beneficial modules such as co-attention and biaffine, which promote the fusion of diverse representations. experimental results on public datasets demonstrate that our iped exceeds the performance of state-of-theart (sota) models, while also achieving significantly faster inference speeds.