GL i REL - Generalist Model for Zero-Shot Relation Extraction (2025)

Abstract: We introduce GLiREL, an efficient architecture and training paradigm for zero-shot relation classification. Identifying relationships between entities is a key task in information extraction pipelines. The zero-shot setting for relation extraction, where a taxonomy of relations is not pre-specified, has proven to be particularly challenging because of the computational complexity of inference, and because of the lack of labeled training data with sufficient coverage. Existing approaches rely upon distant supervision using auxiliary models to generate training data for unseen labels, upon very large general-purpose large language models (LLMs), or upon complex pipelines models with multiple inference stages. Inspired by the recent advancements in zero-shot named entity recognition, this paper introduces an approach to efficiently and accurately predict zero-shot relationship labels between multiple entities in a single forward pass. Experiments using the FewRel and WikiZSL benchmarks demonstrate that our approach achieves state-of-the-art results on the zero-shot relation classification task. In addition, we contribute a protocol for synthetically-generating datasets with diverse relation labels.

## introduction recent advances in zero-shot nlp models for entity recognition have been enabled by large-scale synthetic training data generation using state-ofthe-art (sota) large language models (llms) @xcite ). an ongoing line of work achieves drastic improvements in accuracy and usability over previous approaches by using efficient architectures targeted at various nlp tasks @xcite @xcite . zero-shot named entity recognition (ner) models such as gliner @xcite do not operate on a fixed label set, only requiring textual labels to be specified at inference time, and can directly perform span classification using labels that are not observed during training. in contrast to generative models, targeted architectures for zero-shot span classification jointly predict all labels simultaneously, making them much more efficient than auto-regressive models @xcite . existing sota zero-shot relation classification foot_0 (zsrc) models achieve strong performance, but are inefficient because every entity pair and candidate label combination is treated as a separate input. existing methods do not scale to real-world use cases, where a large number of entity pairs are extracted from text, each of which must be classified against many candidate labels. glirel takes inspiration from recent successes in zero-shot ner and text classification, adapting these approaches to enable zsrc that is both efficient and accurate. while sota llms excel at information extraction (ie) tasks @xcite , there are major limitations to their scale and deployment patterns, including: the ability of llms to perform zero-shot inference with unconstrained output makes them very flexible, but for many tasks, their auto-regressive relation type/entity pair similarity matrix (dot product + sigmoid activation) token/word representation relation type embeddings entity pair embeddings [(3,5), (0,0)] factorization raises issues around reliability. for information extraction tasks in particular, the assignment of relationships between all entities found in a text is a requirement that cannot be reliably achieved with llms without auxiliary models @xcite and/or at-runtime data augmentation mechanisms @xcite . in contrast to the disadvantages above, llms excel at unconstrained classification and labeling tasks, and can be effectively utilized to produce large-scale, diverse datasets for training downstream models with task-specific inductive biases. synthetic dataset generation using general-purpose llms is a critical component in the recent success of zero-shot nlp models @xcite . this paper includes a protocol for generating a large-scale dataset for relationship classification which enables training zero-shot models. the key contributions of our work are: the paper is organized as follows: section 2 discusses important related work, section 3 discusses the glirel model architecture, section 4 presents our experimental results, and sections 5, 6 and 7 provide discussion, analysis, and examinations of the limitations of this work. all code is publicly available. 2 2 background joint vs independent ner and relation classification joint entity and relationship classification (eberts and ulges, 2019; zaratiana et al., 2024) can enhance performance through task transfer and global optimization, but increases decoding complexity and reduces flexibility, often requiring bespoke architectures that may not generalize to other tasks. in contrast, traditional ie pipelines use independent models (e.g., spacy (honnibal et al., 2020)), offering flexibility but making relationship classification dependent on static upstream ner components. our work assumes entities are provided by an upstream component and focuses on detecting relationships between these entities using zero-shot relation labels, thus maintaining pipeline flexibility and allowing classification between any number of entities from diverse sources. zero-shot relation extraction zero-shot relation extraction is an appealing avenue of research because of the flexibility and simplicity of the inference and training paradigms. mc-bert (lan et al., 2023) can use previously-unseen relation type labels to classify entity pairs by treating the task as a multiple-choice problem. tmc-bert (möller and usbeck, 2024) extends upon this by incorporating entity types and relation label descriptions. mc-bert and tmc-bert construct a template for each entity pair for each candidate label in an instance. this results in a large number of inputs from a relatively small sample size, making it unsuitable for scaling. relationprompt (chia et al., 2022) generates synthetic training examples at inference time using gpt2, requiring a large number (@xmath0, which is resource-intensive. dsp uses a discriminative prompting strategy to classify both entities and relations in a zero-shot setting. zs-ska (gong and eldardiry, 2024) performs zsrc by using templates to augment data and incorporating an external knowledge graph. zsre (tran et al., 2023) encodes text and relation labels separately, computing semantic correlation for each entity pair and label combination, leading to inefficiency in realworld scenarios where many entities are present. in contrast, glirel supports any relation labels at inference time without lengthy descriptions or entity type information (see figure 1 ). multiple entity pairs can be classified in a single input, making our approach more efficient than models that require multiple rounds of inference or at-runtime data generation. additionally, glirel processes relation labels and input text simultaneously, capturing interactions between all labels and entity pairs. llms for relation classification llms have been leveraged for relation classification, achieving strong zero-and few-shot performance using meta in-context learning and synthetic data generation @xcite . some approaches reformulate zero-shot relation classification as a question-answering task @xcite . in document-level re (docre), finetuning llama2 with lora shows significant improvements, especially when a pretrained language model first classifies whether an entity pair expresses a relationship before passing it to the llm @xcite . genrdk @xcite uses chain-of-retrieval prompts with chatgpt to generate synthetic data for finetuning llama2. alternatively, @xcite finetune an llm to propose head and tail entities given a document and relation label, outperforming other llm-based baselines. zaratiana et al. (2023) and bogdanov et al. (2024) showed that a straightforward and efficient model architecture can achieve excellent performance on the zero-shot ner task, given high-quality, large scale training data. opensource llms have enabled the creation of this kind of training data, through simple and scalable protocols which prompt a model to label the entities in a short text with any type label @xcite . importantly, labels are not constrained to a particular taxonomy, and the generative model is free to assign any representative label to entities in the text. in the case of gliner @xcite , training on the pile-ner dataset (created by zhou et al. ( 2024 )) enabled a new sota in zero-shot ner. ## method the glirel architecture has three main components: the architecture encodes relation labels and entity pair embeddings in the same latent space to compute their similarity. the overall architecture is illustrated in figure 2 . we choose deberta v3-large as the encoder model due to its excellent performance on downstream tasks @xcite . ## conclusion we have shown that glirel is a flexible and highly performant approach to zero-shot relation classification (zsrc), which achieves sota results on challenging benchmarks. unlike other highperforming zsrc models, glirel can classify multiple entity pairs and relation labels in a single input, making it more efficient. additionally, we have presented a paradigm for generating highquality, large-scale synthetic datasets for zero-shot relation classification, as well as an effective training protocol. we hope these methods inspire future work in the area of relation classification.