Efficient Information Extraction in Few-Shot Relation Classification through Contrastive Representation Learning (2024)

Abstract: Differentiating relationships between entity pairs with limited labeled instances poses a significant challenge in few-shot relation classification. Representations of textual data extract rich information spanning the domain, entities, and relations. In this paper, we introduce a novel approach to enhance information extraction combining multiple sentence representations and contrastive learning. While representations in relation classification are commonly extracted using entity marker tokens, we argue that substantial information within the internal model representations remains untapped. To address this, we propose aligning multiple sentence representations, such as the CLS] token, the [MASK] token used in prompting, and entity marker tokens. Our method employs contrastive learning to extract complementary discriminative information from these individual representations. This is particularly relevant in low-resource settings where information is scarce. Leveraging multiple sentence representations is especially effective in distilling discriminative information for relation classification when additional information, like relation descriptions, are not available. We validate the adaptability of our approach, maintaining robust performance in scenarios that include relation descriptions, and showcasing its flexibility to adapt to different resource constraints.

## introduction relation classification (rc) is an important subtask in the relation extraction, focusing on identifying the types of relations between pairs of entities within a given textual context. extracting relevant information is central to this task. to achieve this, rc models must distill rich information from sentences, including contextual cues, entity attributes, and relation characteristics. while language models are essential for extracting text representations, their use of vector space in sentence representations is suboptimal @xcite . to improve this, recent advances have enhanced sentence representations through various techniques like flowbased methods @xcite , whitening operations @xcite , prompting @xcite , and contrastive learning @xcite @xcite . relation extraction faces challenges with limited data for many relation types and disproportionate data acquisition costs @xcite . to address this challenge, few-shot rc trains models to quickly adapt to new relation types using only few labeled examples. common strategies include meta-learning and prototypical networks, leveraging representation similarity to match unseen query instances with few labeled support instances @xcite . recent studies have enhanced model representations using supplementary data. @xcite and @xcite incorporate information from external knowledge bases, augmenting entity-related knowledge. @xcite and @xcite utilize linguistic dependencies to integrate structural sentence information in models. textual relation descriptions also improve prototypical network performance by offering additional insights into relation types @xcite @xcite . given the inherent complexity of distinguishing between various relation types, rc applications commonly combine representations of entity marker tokens as sentence representations @xcite . recent work employs contrastive learning in few-shot rc for more discriminative representations @xcite @xcite . additionally, studies show that representing sentences with the [mask] token through prompting improves sentence representations @xcite . a key advantage in our approach is the efficient utilization of resources, since all representations are derived from a single forward pass. our method is adaptable to different resource constraints by varying the number of representations used in the sentence embeddings. this adaptability allows for a balance between performance and resource usage. additionally, we extend our approach to include additional information sources, such as relation descriptions. in summary, our contributions are: ## approach this section provides a detailed overview of our approach, as depicted in figure 1 . ## conclusion in this study, we propose aligning multiple sentence representations in few-shot rc through contrastive learning. this method efficiently distills discriminative information from these representations. our results emphasize its adaptability and effectiveness in diverse scenarios, particularly in low-resource and 1-shot environments, particularly when relation descriptions are unavailable. a key advantage of our approach lies in its resource efficiency, deriving all sentence representations from a single forward pass. additionally, we demonstrate that the performance of our method improves with an increasing number of representations, showcasing its resource-adaptability.