Conservative Bias in Large Language Models: Measuring Relation Predictions (2025)

Abstract: Large language models (LLMs) exhibit pronounced conservative bias in relation extraction tasks, frequently defaulting to no_relation label when an appropriate option is unavailable. While this behavior helps prevent incorrect relation assignments, our analysis reveals that it also leads to significant information loss when reasoning is not explicitly included in the output. We systematically evaluate this trade-off across multiple prompts, datasets, and relation types, introducing the concept of Hobson’s choice to capture scenarios where models opt for safe but uninformative labels over hallucinated ones. Our findings suggest that conservative bias occurs twice as often as hallucination. To quantify this effect, we use SBERT and LLM prompts to capture the semantic similarity between conservative bias behaviors in constrained prompts and labels generated from semi-constrained and open-ended prompts.

## introduction recent advancements in llms have shown impressive ability to capture rich semantic knowledge and excel in tasks like text generation and question answering @xcite . as these models are increasingly deployed for complex natural language processing tasks, including relation extraction, distinct behavioral patterns have emerged that warrant careful examination. one such pattern is hallucination, where llms generate content (or relations) beyond the provided context (or available options). this phenomenon has attracted enormous attention within the llm community @xcite , as it is often perceived as a limitation in most applications. however, hallucination also presents opportunities for innovation, particularly in domains that benefit from creative generation such as image synthesis and other generative ai applications @xcite . given the substantial research on hallucination detection @xcite , we have observed a reduction in hallucination rates. this reduction has led us to explore other emergent behaviors of llms that may have significant downstream effects. we focused on relation extraction tasks using llms, where we initially anticipated some degree of hallucination. however, our findings revealed minimal occurrences of such behavior. instead, we observed a distinct pattern where llms consistently exhibit a systematic bias towards classifying instances as no_relation even when a more appropriate relation is available, presenting a "hobson's choice" scenario. we attribute this behavior to alignment strategies designed to reduce hallucinations by reinforcing contextual adherence while suggesting external alternatives. we define this preference for overly cautious responses as conservative bias (cb). in relation extraction tasks, llms exhibit a distinct cb, defaulting to the least incorrect classification when faced with uncertainty. unlike hallucination, this bias leads to a unique form of information loss by creating "hobson's choice" scenarios, where models favor safe but uninformative labels even when more suitable alternatives exist. our figure 1 example shows a llm's response to relation extraction task, especially when the available options do not perfectly match the true relationship. the example demonstrates the model's strategies in "hobson's choice" scenarios and conservative bias (cb) behavior. the "hobson's choice" effect occurs when the llm must select from predefined options that do not accurately capture the relationship, defaulting to no_relation as the least incorrect choice. the llm output also shows its conservative bias behavior, acknowledging that a more accurate relation like owner_of or shareholder_of would be preferable if available. this conservative approach helps avoid incorrect assertions but can lead to a loss of valuable information. this work seeks to address three key research questions: [rq1] how can we capture and quantify this cb? @xcite what is its relationship to hallucination prevention? [rq3] how can we leverage this behavior to improve relation extraction tasks? ## related work recent studies have highlighted emergent behaviors in llm, such as sycophancy and hallucination, which impact their reliability and trustworthiness in downstream applications @xcite . sycophancy refers to the tendency of models to align their responses with user views, regardless of objective correctness @xcite . this behavior is most prevalent in models that have been fine-tuned using human feedback, but it can be mitigated through the use of synthetic data @xcite . hallucinations have received much attention from the research community @xcite @xcite . among proposed hallucination mitigation methods, @xcite investigated llm hallucinations in entity/relation extraction tasks proposing mitigating techniques. advances in prompt engineering @xcite have also mitigated hallucinations by constraining responses to given contexts @xcite . as prompt engineering advances, new emergent behaviors in llms may arise. to the best of our knowledge, conservative bias behavior has not been explored in existing literature. ## method our research aims to analyze cb in llms. we investigate the frequency with which llms default to the least incorrect labels from a list of options, as opposed to generating hallucinated relations. we analyze the rates of hallucination and cb across multiple prompt iterations. we also explore practical applications where cb can be utilized to refine relation classification, potentially expanding existing relations. formally, cb is detected in an output when the following conditions are met: (i.) the model recognizes that a valid relation exists, as outputted in the reasoning. (ii.) the correct relation type is not available in the option set. (iii.) the model chooses to default to no_relation or selects the least incorrect (suboptimal) option. (iv.) the model demonstrates awareness of the correct relation through reasoning, suggesting it when appropriate to preserve the integrity of extracted relations. see the example in figure 1 . for evaluation purposes, we designed three types of prompts: constrained prompt, semi-constrained prompt, and open-ended prompt and assessed performance using four measures: hobson's choice rate (hcr), conservative bias rate (cbr), hallucination rate (hr) and new relation rate (nrr). we adopted a multi-tiered approach to prompt design, where each level offers varying degrees of specificity to the llms. this approach explores how different levels of constraint affect the llms' ability to generate and select appropriate relations. the prompt categories are defined as follows: ## experiments and results data for our experiment, we focus on two datasets: refind @xcite and tacred @xcite . 1 refind is a large financial dataset, consisting of 28,676 instances and 22 relation types across 8 entity pairs, with data sourced from the quarterly and annual reports of publicly traded companies. tacred, is a large-scale relation extraction dataset with 106,264 examples derived from newswire and web text and features 41 relation types. for our analysis, we focus on subset of data where gold_relation is labeled as no/other relation or no_relation which constitutes 45% of the refind and 79.5% of the tacred dataset (statistics shown in app a.2). we also explored, but did not include, biored @xcite , a biomedical relation extraction dataset, because this dataset has no no_relation option. models we leveraged gpt-4, llama3.1-8b-instruct, and mistral-7b-instruct-v0.3 as our models. for each model, we utilized two low temperature settings, specifically 0.2 and 0.5, in order to simulate multiple data annotators while maintaining high level of determinism. we captured output consistency per temperature by conducting multiple iterations of each prompt. our selection of llm models was based on those accessible within our organization that are on the llm leaderboard. prompt setup prompts are structured in a hierarchical manner, allowing us to evaluate how varying level of constraints can affect llms' responses. while all prompts share the same basic structure, they differ in their option list setup. among these, only the constrained prompt is prone to hallucination. outputs from semi-constrained and openended prompts will be used to validate the cb behavior in the constrained prompt. ## discussion our findings confirm the presence of cb tendencies in llms during relation extraction. while gpt-4 demonstrates strong hallucination resistance under constrained conditions (0.02-0.04% hr), it also shows a much higher frequency of conservatism. this pattern persists in the semi-constrained design, suggesting a fundamental tension between innovation and accuracy in llm behavior. in contrast, llama3.1 shows less cb but a higher hr. this indicates that as models become more resistant to hallucinations, they tend to exhibit increased cb [rq2], presenting a crucial trade-off in model behavior that requires careful consideration in application design. there are significant differences in output quality between gpt-4 and llama3.1 when using identical prompts. llama3.1 generated noisier outputs, often returning meta-responses such as "please specify title example", resulting in substantial data loss during the cleanup process. this disparity in output quality highlights the importance of model selection and prompt engineering in relation extraction tasks. to mitigate this limitation, our research indicates that detailed prompting strategies incorporating step-by-step reasoning are essential. this finding is particularly relevant in specialized professional contexts; for example -a boutique law firm employing ai for litigation analysis. without structured reasoning steps in the prompting strategy, these systems risk returning conclusions that may be either overly conservative or inappropriately broad, potentially missing crucial legal nuances within the established constraints. ## conclusion & future work this study explored the conservative bias (cb) in llms during relation extraction, where models default to no_relation when a correct option is unavailable. our experiment confirmed an inverse relationship between conservative bias rate (cbr) and hallucination rate (hr), highlighting a trade-off between accuracy and innovation. future research should focus on developing prompting strategies that balance cb with the need for novel relation identification, potentially by refining prompt designs and integrating external knowledge bases.