Generation-Augmented Retrieval: Rethinking the Role of Large Language Models in Zero-Shot Relation Extraction (2025)

Abstract: Recent advances in Relation Extraction (RE) emphasize Zero-Shot methodologies, aiming to recognize unseen relations between entities with no annotated data. Although Large Language Models (LLMs) have demonstrated outstanding performance in many NLP tasks, their performance in Zero-Shot RE (ZSRE) without entity type constraints still lags behind Small Language Models (SLMs). LLM-based ZSRE often involves manual interventions and significant computational overhead, especially when scaling to large-scale multi-choice data.To this end, we introduce RE-GAR-AD, which not only leverages the generative capability of LLMs but also utilizes their representational power without tuning LLMs. We redefine LLM-based ZSRE as a retrieval challenge, utilizing a Generation-Augmented Retrieval framework coupled with a retrieval Adjuster. Specifically, our approach guides LLMs through crafted prompts to distill sentence semantics and enrich relation labels. We encode sentences and relation labels using LLMs and match their embeddings in a triplet fashion. This retrieval technique significantly reduces token input requirements. Additionally, to further optimize embeddings, we propose a plug-in retrieval adjuster with only 2M parameters, which allows rapid fine-tuning without accessing LLMs’ parameters. Our LLM-based model demonstrates comparable performance on multiple benchmarks.

## introduction relation extraction (re) aims to select the relation from a predefined set for two target entities in a given sentence. traditional supervised re methods @xcite demand significant annotated data and struggle to handle @xcite , (b) sumask @xcite , and (c) our approach. in this work, we explore new possibilities for llm-based zsre and propose the re-gar-ad framework, which is based on the generation-augmented retrieval (gar) paradigm and incorporates a retrieval adjuster. considering the potential of llms as encoders for obtaining embeddings @xcite , we rethink and redefine llm-based zsre as an embedding-based retrieval problem. we input generation-augmented sentences and relation labels separately to llms to obtain their embeddings and conduct sentence and relation retrieval through matching scores. all candidate relation labels only need to be input once and kept fixed, requiring only token inputs at n + r level as shown in figure 1 (c ). this not only reduces the inference cost of large-scale multi-choice zsre data for llms but also demonstrates performance comparable to that of existing methods. specifically, we leverage the generation capability of llms to improve retrieval efficiency. considering that a sentence may contain more than one relation pattern, we distill valuable information for re from sentences using sentence distillation prompt, which includes the types of two target entities and their possible semantic relation. additionally, we guide llms to utilize their internal general knowledge to generate deep information about relation labels through the relation enrichment prompt, which explores possible entity types relevant to the relation. we then use llms to encode and obtain embeddings of sentences and relation labels, and achieve more accurate relation retrieval through triplet-based retrieval. furthermore, slms-based zsre fine-tuning typically involves learning prior knowledge from seen data, which may be computationally expensive for llms and impractical for api-based llms @xcite . therefore, we carefully design a plug-in retrieval adjuster for the framework. it aims to learn the ability to align sentence and relation label representations from seen data, and further adjust and optimize the embeddings of unseen data without accessing the full parameters of llms, thereby improving retrieval efficiency. our contributions are summarized as follows: ## conclusion we present re-gar-ad, a simple and effective zsre framework based on llms. we introduce a novel llm-based zsre paradigm of generation-augmented retrieval, leveraging llms' generative and representational capabilities. additionally, we train a plug-in retrieval adjuster to further optimize embeddings. extensively evaluated on common zsre benchmarks, re-gar-ad demonstrates comparable performance across various settings, especially in cases where entity types are excluded. furthermore, our method significantly improves the efficiency of llms in multi-choice relation extraction, which is advantageous for realworld zsre and opens up interesting avenues for future zero-shot classification efforts.