RE -Matching: A Fine-Grained Semantic Matching Method for Zero-Shot Relation Extraction (2023)

Abstract: Semantic matching is a mainstream paradigm of zero-shot relation extraction, which matches a given input with a corresponding label description. The entities in the input should exactly match their hypernyms in the description, while the irrelevant contexts should be ignored when matching. However, general matching methods lack explicit modeling of the above matching pattern. In this work, we propose a fine-grained semantic matching method tailored for zero-shot relation extraction. Guided by the above matching pattern, we decompose the sentence-level similarity score into the entity matching score and context matching score. Considering that not all contextual words contribute equally to the relation semantics, we design a context distillation module to reduce the negative impact of irrelevant components on context matching. Experimental results show that our method achieves higher matching accuracy and more than 10 times faster inference speed, compared with the state-of-the-art methods.

## introduction relation extraction (re) is a fundamental task of natural language processing (nlp), which aims to extract the relations between entities in unstructured text. benefitting from high-quality labeled data, neural relation extraction has achieved superior performance @xcite @xcite . however, it is expensive and even impractical to endlessly label data for a fast-growing number of new relations. in order to deal with the emerging new relations that lack labeled data, zero-shot relation extraction (zerore) has recently attracted more attention. @xcite frame the zerore as a slot-filling task solved in a question-answering way. each relation is associated with a few question templates. figure 1 : an example of the matching pattern of relational data. the input sentence contains a given entity pair, which should match the corresponding hypernyms (usually the entity type). the context describes the relations between entities, containing relation-irrelevant redundant information, which should be ignored when matching. however, the templates are expensive and timeconsuming to build @xcite . obamuyide and vlachos (2018) simplify the templates to readily available relational descriptions and reformulate zerore as a semantic matching task. recently, pretrained model based zerore methods have achieved great success. siamese scheme and full encoding scheme are two mainstream methods for matching semantics. the siamese scheme separately encodes the input and description @xcite . therefore, the encoded representations of descriptions can be both stored and reused for each input, resulting in a fast inference. however, insufficient interaction during encoding also limits the matching performance. by contrast, the full encoding scheme performs selfattention over the pair to enrich interaction @xcite , although the performance increase comes with a computational overhead. (for m inputs and n descriptions, the siamese scheme requires m + n encodings, while the number is m × n for full encoding). an approach that combines the advantages of both can be attractive. unlike ordinary sentence pairs, relational data has a unique matching pattern, which is not explicitly considered by general matching methods. as shown in fig. 1 , the entities in the input should exactly match their hypernyms in the description (e.g., apple and organization). meanwhile, not all contextual words contribute equally to the relation semantics. for example, the clause "is a great company" is only used to modify apple instead of expressing the relationship between apple and california. such redundant components should be ignored when matching. due to the lack of explicit annotations to the redundant components, it is nontrivial for the model to learn to identify them. in this work, we propose a fine-grained semantic matching method that improves both the accuracy and speed over the current state-of-theart. specifically, we decouple encoding and matching into two modules. while the encoding module follows a siamese scheme for efficiency, the matching module is responsible for the finegrained interaction. following the matching pattern of relational data, the sentence-level similarity score is decomposed into two: entity matching and context matching scores. to deal with the redundant components without explicit annotations, we design a feature distillation module. context features that maximize a classification loss are identified as relation-irrelevant features. then, the context representations are projected into the orthogonal space of the features to improve context matching. experimental results show that this method outperforms state-of-the-art (sota) methods for zerore, in terms of both accuracy and speed. our codes are publicly available * . the main contributions are three-fold: (1) we propose a fine-grained semantic matching method for zerore, which explicitly models the matching pattern of relational data; (2) we propose a context distillation method, which can reduce the negative impact of irrelevant components on context matching; (3) experimental results show that our method achieves sota matching f 1 score together with an inference speed 10 times faster. ## related works text semantic matching aims to predict a matching score that reflects the semantic similarity between a given pair of text sequences. the approaches used to calculate the matching score roughly fall into one of two groups. the first group uses a siamese scheme, which separately maps the pair of text sequences into a common feature space wherein a dot product, cosine, or parameterized * https://github.com/zweny/re-matching non-linearity is used to measure the similarity @xcite @xcite @xcite . the second group adopts a full encoding scheme, in which the concatenation of the text pair serves as a new input to a nonlinear matching function. neural networks with different inductive biases are used to instantiate the matching function @xcite @xcite . more recently, large-scale pretrained language models such as bert @xcite , roberta @xcite are introduced to yield richer interactions between the text pair, to improve semantic matching. unlike the general matching methods, the proposed method is designed to be specifically used for relational data. by explicitly modeling the matching pattern, our method achieves sota performance while decreasing the computational cost by an order of magnitude compared with a full encoding scheme (as the more candidate relations exist, the more cost decreases). ## approach in this work, we propose a fine-grained semantic matching method tailored for relational data. to facilitate inference efficiency, we adopt a siamese scheme to separately encode the input and the candidate relation descriptions. to explicitly model the matching pattern of relational data, we decompose the matching score into entities and contexts matching scores. in addition, we design a context distillation module to reduce the negative impact of irrelevant components in the input on context matching. ## conclusions in this work, we propose a fine-grained semantic matching method for zerore. this method explicitly models the matching pattern of relational data, by decomposing the similarity score into entity and context matching scores. we explore various ways to enrich entity description and thus facilitating high-quality entity representation. the context distillation module effectively reduces the negative impact of irrelevant components on context matching. experimental results show that our method achieves higher matching f 1 score and has an inference speed 10 times faster when compared with sota methods.