Topic-Oriented Open Relation Extraction with A Priori Seed Generation (2024)

Abstract: The field of open relation extraction (ORE) has recently observed significant advancement thanks to the growing capability of large language models (LLMs). Nevertheless, challenges persist when ORE is performed on specific topics. Existing methods give sub-optimal results in five dimensions: factualness, topic relevance, informativeness, coverage, and uniformity. To improve topic-oriented ORE, we propose a zero-shot approach called PriORE: Open Relation Extraction with a Priori seed generation. PriORE leverages the built-in knowledge of LLMs to maintain a dynamic seed relation dictionary for the topic. The dictionary is initialized by seed relations generated from topic-relevant entity types and expanded during contextualized ORE. PriORE then reduces the randomness in generative ORE by converting it to a more robust relation classification task. Experiments show the approach empowers better topic-oriented control over the generated relations and thus improves ORE performance along the five dimensions, especially on specialized and narrow topics.

## introduction relation extraction (re) @xcite , which aims to recognize semantic relations between entities under certain contexts, is essential for various downstream tasks such as knowledge graph construction @xcite and question answering @xcite . traditional re mostly focuses on the closed setting (closed re), where the task is to choose the best relation from a pre-defined relation set. another line of work proceeds to the open relation extraction (ore) task that extracts relations without being constrained by a pre-defined set. despite these developments, existing ore approaches often give sub-optimal results on specific * equal contribution topics. take figure 1 as an example. given a passage under a chemical topic redox reactions, existing methods fail to robustly extract the desired relation (oxidized by) between the entities magnesium and oxygen. closed re fails when the desired relation is not in the pre-defined set. extractive ore is restricted by the expression of the relation in contexts. it thus can be easily misled by topicirrelevant relations (e.g., exposed to is not directly relevant to redox reactions) or implicit expressions. the generative method mostly utilizes the general knowledge in llms to interpret the context, leading to the generation of overly general relations (e.g., forms compound with). an important yet less studied direction is how to make ore more aligned with the users' needs under the specific topics (e.g., redox reactions, electric vehicle batteries). on the one hand, existing domain-specific ore work @xcite usually relies heavily on domain features and can ## related work open re traditional open re work can be categorized primarily into sequence labeling and clustering-based methods. sequence labeling methods use syntactic or semantic features to extract relational phrases from the text as relations @xcite @xcite . the relations can be limited in expressiveness, and this line of work can hardly capture global context. clustering-based approaches model the relation representations and cluster them into relation types with masked language modeling @xcite @xcite or relational feature extraction @xcite . although enhancing the coherence of expressions, they are sensitive to data quality, dataset size, and distribution of relations, especially for narrow topics. generative re more recent works take advantage of generative models for ore by formulating ore into sequence-to-sequence tasks @xcite , whose generalization to new topics is limited by training data. with the development of llms, wadhwa et al. (2023a) and jiang et al. (2024) show the effectiveness of llms on ore. other llm-based re methods focus on the closed-domain setting, with the paradigm of filter-then-rerank @xcite , the formulation of summarization @xcite , or data synthesis @xcite . both generative and extractive methods for topic-oriented ore may suffer from redundant/insufficient information and incoherent expressions. to align with users' needs and reduce ambiguity, some may apply relation canonicalization after extraction based on token-level statistics, entity information, triplet-level structure information, or external knowledge bases along with clustering methods @xcite @xcite . instead of the postextraction canonicalization, which is sensitive to data volume and distribution, our work can extract high-quality relations by a priori seed generation to convert the ore task to a more robust relation classification task. ## conclusion in this paper, we propose priore, a zero-shot approach for the task of topic-oriented ore. existing methods lead to inferior results on the dimensions of factualness, topic relevance, informativeness, coverage, and uniformity. by generating seed relations with llms for the topic in an a priori way, we reduce the randomness in generative llms by converting the ore task into a more robust relation classification task. experiments show we can outperform state-of-the-art generative llms along the five dimensions. our advantage is particularly remarkable on specialized topics along the informativeness, coverage, and uniformity dimensions.