READ : Improving Relation Extraction from an AD versarial Perspective (2024)

Abstract: Recent works in relation extraction (RE) have achieved promising benchmark accuracy; however, our adversarial attack experiments show that these works excessively rely on entities, making their generalization capability questionable. To address this issue, we propose an adversarial training method specifically designed for RE. Our approach introduces both sequence- and token-level perturbations to the sample and uses a separate perturbation vocabulary to improve the search for entity and context perturbations.Furthermore, we introduce a probabilistic strategy for leaving clean tokens in the context during adversarial training. This strategy enables a larger attack budget for entities and coaxes the model to leverage relational patterns embedded in the context. Extensive experiments show that compared to various adversarial training methods, our method significantly improves both the accuracy and robustness of the model. Additionally, experiments on different data availability settings highlight the effectiveness of our method in low-resource scenarios.We also perform in-depth analyses of our proposed method and provide further hints.We will release our code at https://github.com/David-Li0406/READ.

## introduction relation extraction (re) is an important subtask of information extraction and plays a crucial role in many other natural language processing (nlp) tasks like knowledge base construction @xcite and question answering @xcite . the goal of re is to determine the relationship between a head entity and a tail entity. for example, given the sentence "miettinen hired for wps champ sky blue.", the re models are supposed to predict the relation "employee-of " between the head entity "miettinen" and the tail entity "sky blue". with the recent advances in pre-trained * † corresponding author sentence prediction org miettinen hired for wps champ sky blue. employee-of adv miettinen hired for wps champ jeez blue. no-relation the recent success of re models sparks a growing interest in conducting more detailed analyses @xcite @xcite . a significant issue that arises in this context is to explore whether the re model learns from context or entities for relation prediction. analyzing this problem could reveal the underlying nature of re models and offer informative insights for their improvement. to address this issue, various methods are proposed such as information masking @xcite and counterfactual analysis @xcite . one drawback of these methods is they usually involve removing entities or context in the sample and observing the model's performance with the remaining part. that enables them to draw the conclusion about how much can the model learn from entity/ context when giving each of them individually. however, whether the model would prefer to learn from context or entities when both of them are given still remains unclear. we name this problem learning preference in re. to address this issue, we propose a novel approach read, a.k.a. improving relation extraction from an adversarial perspective. we begin by introducing the utilization of adversar-ial attacks @xcite as a means to investigate the model's learning preference and robustness. adversarial attacks in nlp are designed to deceive the model by making very few text substitutions. as the example shown in in our preliminary experiment applying adversarial attacks to re, we discovered a clear over-dependency on entities within the current re model. this is consistent with the previous works @xcite that re models tend to utilize shallow cues from entities to make predictions. our analysis revealed that this over-dependency is the underlying cause of the models' vulnerability to adversarial attacks and can also lead to poor generalization in clean samples. so the key to improving current re models is to mitigate this over-dependency on entities. one straightforward approach to bolster models' robustness is text substitution. however, the considerable time cost to generate adversarial samples with the text substitution method constrains it in scaling in large re datasets @xcite . also, in our preliminary experiments, we observed a performance drop in the clean test set with text substitution, which has also been reported by previous works @xcite foot_0 . so we shift our focus towards virtual adversarial training @xcite , which applies continuous perturbations at the embedding level during training, rendering it a more refined and efficient approach. our method builds upon the advancements of the current adversarial training methods in nlp @xcite and introduces both sequence-and token-level perturbations to the re sample. to facilitate perturbation searching, we devise a separate perturbation vocabulary that tracks the accumulated perturbation for entity and context respectively. furthermore, we propose a novel probabilistic strategy to encour-age the model to leverage relation patterns from the unperturbed context. through extensive experiments, we demonstrate the effectiveness of our method on both adversarial and clean test samples. we also observe significant improvements in lowresource settings, indicating the great potential of our method in scenarios with limited data. we conduct a series of in-depth analyses to give more hints about read. the contribution of our work could be summarized as follows: ## adversarial attack for re in this section, we start by analyzing the stateof-the-art (sota) re models' performance under textual adversarial attacks. then, through further analysis, we expose the over-dependency and nongeneralization on entities in the current re models. ## adversarial training for re to improve the robustness and generalization of the re models, read employs an entity-aware virtual adversarial learning method. in this section, we first give a brief illustration of the virtual adversarial training (vat) process, then we will ## experiment in this section, we design experiments to test our entity-aware vat's performance on both clean and adversarial samples. ## further analysis in this section, we conduct further experiments to give in-depth analyses of the mechanism of our proposed method. ## conclusion in this work, we present read, a novel method that leverages an adversarial perspective for analyzing and enhancing re models. our adversarial attacks experiment on current sota re models reveals their excessive reliance on entities for relation prediction. through our analysis, this overdependency is the underlying cause of the models' non-robustness to adversarial attacks and can limit the model's generalization. to tackle this issue, we propose an entity-aware virtual adversarial training method. experiment results show our method's effectiveness in improving the performance in both adversarial and clean samples.