D e CAP : Context-Adaptive Prompt Generation for Debiasing Zero-shot Question Answering in Large Language Models (2025)

Abstract: While Large Language Models (LLMs) excel in zero-shot Question Answering (QA), they tend to expose biases in their internal knowledge when faced with socially sensitive questions, leading to a degradation in performance. Existing zero-shot methods are efficient but failto consider context and prevent bias propagation in the answers. To address this, we propose *DeCAP*, a method for debiasing LLMs usingContext-Adaptive Prompt Generation. *DeCAP* leverages a *Question Ambiguity Detection* to take appropriate debiasing actions based on the context and a *Neutral Answer Guidance Generation* to suppress the LLMs make objective judgments about the context, minimizing thepropagation of bias from their internal knowledge. Our various experiments across eight LLMs show that *DeCAP* achieves state-of-the-art zero-shot debiased QA performance. This demonstrates *DeCAP*’s efficacy in enhancing the fairness and accuracy of LLMs in diverseQA settings.

## introduction large language models (llms) outperform in question answering (qa) tasks, which require answering user questions by utilizing the given information and their internal knowledge. however, generating answers to socially sensitive questions poses the risk of exposing bias in their internal knowledge @xcite @xcite . recently, zero-shot approaches have been proposed to reduce biases in qa, such as adding predefined debiasing instructions to the prompt @xcite @xcite or self-debiasing, which relies on the llm to explain potential biases @xcite . these methods have gained attention because they are more efficient and cost-effective compared to approaches that require model retraining or modification @xcite @xcite @xcite . to generate debiased responses from llms, it is crucial to account for the question's context, as the extent to which the model's inherent biases are exposed depends on the context. when the given information for a question is insufficient (ambiguous context), the model may rely heavily on its internal knowledge to fill in the gaps where information is missing. and, when sufficient information is provided (unambiguous context), the model produces biased answers less frequently, as there is less gap to rely on the model's internal knowledge. for example, if the following sentences are given: donna and williams are interviewing an engineering candidate for a software company. who was able to ask technical interview questions? the first sentence contains the "context" information, while the second sentence is the "question", which the llm is expected to answer considering the context. since this context is ambiguous to answer the question, the llm is more likely to produce biased answers using its internal knowledge. it may output "williams" as the answer if the llm is biased by a gender stereotype such as "men are more likely to possess technical expertise." however, existing zero-shot qa techniques do not effectively consider the context of the question. for example, debiasing instructions and selfdebiasing result in a performance trade-off depending on the question's context as shown in debiasing instructions uses a fixed prefix without considering the context of questions, therefore, it is not enough to help the llm fully understand the context of the question and mitigate the bias @xcite . although self-debiasing methods aim to use a context-aware approach to help llms focus on the context, their performance is unpredictable due to the potential bias in the llm's internal knowledge. since the method relies on the llm to explain potential biases, if the explanation itself is biased, it can lead to biased answers, ultimately degrading performance. to address these limitations of existing zero-shot debiasing methods, a novel approach is required that enables the llm to consider the context and minimizes the propagation of bias from internal knowledge into the generated answers. in this paper, we propose decap, debiasing llms using context-adaptive prompt generation. our method is a zero-shot debiasing technique that enhances the qa performance of llms while mitigating bias without any additional training. decap has two main processes: question ambiguity detection and neutral answer guidance generation. we first classify the type of questions based on their level of ambiguity and provide appropriate instructions. this approach enhances the llm's ability to respond adaptively based on the question's context. if a question type is classified as ambiguous, the llm is instructed not to answer based on assumptions. on the other hand, if a question is classified as unambiguous, the llm is directed to provide a precise answer based on the context. we also generate neutral answer guidance to suppress llms' internal biased knowledge by guiding the llms to make objective judgments about the context. to do this, we utilize external knowledge to make the guidance generator produce neutral information. by incorporating context-aware instruction and neutral answer guidance, the llms can respond in a context-aware, enabling them to answer questions objectively without depending on their internal biases. we evaluate the llm's debiasing performance across two qa benchmarks and various llms, with decap significantly outperforming existing zero-shot methods through comprehensive experiments. furthermore, the performance analy-sis across various bias categories on different llms validates that our method consistently achieves bias mitigation. ## method in this section, we explain our method, contextadaptive prompt generation for debiasing zero-shot qa in llms. as shown in figure 1 , decap consists of two main processes: question ambiguity detection and neutral answer guidance generation. section 3.1 explains the process of selecting the prefix instruction, and section 3.2 describes the process of generating the neutral answer guidance. ## experimental results and analyses in this section, we demonstrate decap's outperformance through various experimental results. section 5.1 provides overall results across llms. sections 5.2 and 5.3 evaluate decap's compo-nents and their effectiveness in mitigating bias while addressing trade-offs between question types. section 5.4 discusses the ambiguity detector and how our context-adaptive prompt is effective. section 5.5 evaluates the quality of neutral answer guidance, and section 5.6 provides effectiveness across bias categories on various llms, further proving decap's superiority. ## conclusions we addressed the limitations of existing zero-shot debiasing methods by proposing a context-adaptive prompt generation approach that improves the qa performance of llms while mitigating bias without additional training. our method effectively addressed performance trade-offs and mitigates varying biases that arise depending on the question type, enabling llms to provide objective answers without relying on their internal biases. through various experiments and analyses, we demonstrated that decap consistently achieves outperformance across various llms and proves to be effective across a wide range of bias categories.