A lign RE : An Encoding and Semantic Alignment Approach for Zero-Shot Relation Extraction (2024)

Abstract: Zero-shot Relation Extraction (ZSRE) aims to predict unseen relations between entity pairs from input sentences. Existing prototype-based ZSRE methods encode relation descriptions into prototype embeddings and predict by measuring the similarity between sentence embeddings and prototype embeddings. However, these methods often overlook abundant side information of relations and suffer from a significant encoding gap between prototypes and sentences, limiting performance. To this end, we propose a framework named AlignRE, based on two Alignment methods for ZSRE. Specifically, we present a novel perspective centered on encoding schema alignment to enhance prototype-based ZSRE methods. We utilize well-designed prompt-tuning to bridge the encoding gap. To improve prototype quality, we explore and leverage multiple side information and propose a prototype aggregation method based on semantic alignment to create comprehensive relation prototype representations. We conduct experiments on FewRel and Wiki-ZSL datasets and consistently outperform state-of-the-art methods. Moreover, our method exhibits substantially faster performance and reduces the need for extensive manual labor in prototype construction. Code is available at https://github.com/lizehan1999/AlignRE.

## introduction relation extraction (re) extracts structured facts from unstructured text, aiming to predict a relation between two entities in a sentence. despite the impressive performance achieved by supervised re @xcite @xcite , in real scenarios, these methods have difficulty adapting to relations that are unseen during training @xcite . zero-shot relation extraction (zsre) aims to predict unseen relations by leveraging prior knowledge from seen relations. *corresponding author. in contrast to supervised re, zsre does not necessitate an abundance of labeled instances, which possesses the capacity to classify unseen relations not present in the training data @xcite . presently, the main strategies for addressing zsre can be roughly classified into three types: prototype-based, classification-based, and generation-based methods. as shown in figure 1 (a), traditional prototype-based methods (e.g., zs-bert @xcite ) obtain representations of different features of a sentence, concatenate them, and reduce dimensionality to encode sentence embeddings. then, they match it with prototype embeddings constructed from descriptions of relations. however, on one hand, prototype embeddings are encoded by computing the mean pooling of all tokens @xcite . when calculating the matching similarity between sentence embeddings based on concatenation and prototype embeddings based on mean pooling, differences in feature representation methods may lead to semantic matching gaps in feature space, possibly causing challenges in aligning embeddings and complicating model convergence. on the other hand, the previous prototype-based methods mainly rely on the quality of relation description information, and any noise in the description information has the potential to compromise the faithfulness of the prototype embeddings. other side information related to the relations, such as label names and aliases of relations, may contribute to making prototype embeddings more robust. in this work, we propose alignre, a simple yet effective zsre framework based on encoding schema alignment and prototype semantic alignment. firstly, we propose a new perspective based on encoding alignment to eliminate the gap between sentence and prototype embeddings. inspired by prompt-tuning @xcite , we devise a unified prompt containing @xcite to guide the model to capture more accurate semantic information at the positions of [mask] (figure 1 (b) ). on this basis, we obtain sentence representation using the same encoding schema as the prototype. secondly, to enhance the quality of prototypes, we propose a prototype aggregation method based on semantic alignment to aggregate multiple types of side information, such as label names, descriptions, and aliases of relations. we assign higher attention to more representative prototypes and then aggregate them to obtain a comprehensive relation prototype representation. we finally match the sentence and prototype embeddings after aligning the encoding and semantics to complete prediction. in summary, our contributions are as follows: zero-shot relation extraction (zsre) was first introduced by @xcite , and existing methods can be roughly categorized into three types. prototype-based methods transform relation descriptions into prototype embeddings. they minimize the distance between sentence embeddings and prototype embeddings, employing nearest neighbor search techniques to predict unseen relations @xcite . @xcite manually designs templates for each relation, facilitating more fine-grained semantic matching between sentences and prototypes. classificationbased methods @xcite @xcite integrate classification networks into their frameworks. however, as the unseen relations increases, the computational cost of these methods also rises @xcite . generation-based methods generate relation label directly for the input sentence @xcite . recently, large generative language models @xcite have injected new vitality into zsre tasks @xcite @xcite ), but they also come with elevated inference costs. moreover, the recent prompt-tuning techniques @xcite have been applied into the supervised re task @xcite , achieving promising results. recent work @xcite also effectively utilizes prompts in zsre. different from these methods, our approach is based on the prototype framework and for the first time considers the alignment of encoding schemas of sentences and prototypes. to this end, we construct only a unified prompt template with @xcite to accommodate all relations, avoiding the complexity of prompt design. further, we innovatively propose a method utilizing a weighted pooling mechanism of [mask] embeddings for generating sentence embeddings. we also propose a prototype aggre- ## conclusion in this paper, we propose a simple and effective zsre framework named alignre, based on encoding schema alignment and prototype semantic alignment. we for the first time propose the idea of aligning encoding schemas of sentences and prototypes, achieving this through well-designed prompttuning and careful token selection, bridging the encoding gap and significantly enhancing the performance of zsre. we also leverage multiple side information to construct prototypes for relations, and propose a aggregation method based on semantic alignment to obtain a comprehensive relation prototype. moreover, our method exhibits substantially faster performance and reduces the need for extensive manual labor in prototype construction. in future work, our goal is to further optimize prototype representations and extend the task to settings involving a wider range of unseen relations.