Can We Instruct LLM s to Compensate for Position Bias? (2024)

Abstract: Position bias in large language models (LLMs) leads to difficulty in accessing information retrieved from the retriever, thus downgrading the effectiveness of Retrieval-Augmented Generation (RAG) approaches in open-question answering. Recent studies reveal that this bias is related to disproportional attention across the context. In this work, we examine how to direct LLMs to allocate more attention towards a selected segment of the context through prompting, aiming to compensate for the shortage of attention. We find that language models do not have relative position awareness of the context but can be directed by promoting instruction with an exact document index. Our analysis contributes to a deeper understanding of position bias in LLMs and provides a pathway to mitigate this bias by instruction, thus benefiting LLMs in locating and utilizing relevant information from retrieved documents in RAG applications. The code and data in our study have been made publicly available.

## introduction rag is an established method for enabling continuous knowledge updates @xcite @xcite and reducing hallucination @xcite through retrieving and adding relevant documents to the prompt of llms @xcite . however, recent research has discovered that increasing the number of documents in the context may distract the model and degrade performance @xcite @xcite , even when they contain accurate and relevant information @xcite . task instruction: [task] + [attention instruction] search results: [documents] question: [question] answer: prompt multi-media sources retrieval sorted documents task instruction: [task] search results: [documents] question: [question] answer: prompt question bias @xcite @xcite . this bias cause models to favor the beginning or end text within the context @xcite leading to the "lost-inthe-middle" problem. for example, figure 1 illustrates this problem in the rag pipeline for the open question answering task, where multiple retrieved documents are added to the prompt. by grouping and averaging the attention scores of tokens across the 3 retrieved documents, we observe that the second document consistently receives less attention scores, irrespective of the gold document's position, which aligns with previous works @xcite @xcite . this bias can lead to incorrect answers when the gold document is in the middle. to address position bias, many researchers have explored either finetuning @xcite @xcite or modifying position embeddings @xcite @xcite . however, finetuning-based approaches lack adaptabil-ity and require additional computation, whereas embedding-based approaches require multiple rounds of inferencing or hyperparameter search, which is inefficient. in this study, we focus on instructing llms to attend to specific positions within the context, thereby compensating for position bias. in particular, we design two types of attention instructions that instruct llms to adjust their attention using either relative position words or absolute document indexes. we conduct comprehensive experiments with these two types of attention instructions on six open-sourced llms and one closed-source llm based on the multi-document question answering (mdqa) task. our investigation focuses on the feasibility of mitigating position bias in llms through attention instructions. in summary, our findings are as follows: ## experimental setup we design attention instructions, which are two-sentence prompts that guide llms to focus on a selected segment, thereby preventing the overlooking of crucial information. to test the effectiveness of the attention instructions, we conduct a series of experiments on the mdqa task @xcite under the setting that only one document contains the gold answer, namely the gold document. the position of the gold document is referred to as the gold document position. by controlling the gold document position and attention segment specified in the instructions, we aim to evaluate the llms' ability to follow attention instructions accurately. an overview of the input prompt and some toy examples can be seen in figure 2. ## result and analysis probe the relative position awareness of llms with relative attention instructions as described in §2.1, we virtually split the search results into three parts and represent these parts with relative positions words beginning, midsection, and tail. by placing the gold document at different positions among all the documents and refer to each position in the relative attention instruction, we create a 3x3 accuracy heatmap for each model. the heatmaps' y-axis represents the gold document position, while the x-axis represents the selected attention segment. it is worth noting that diagonal cells in the heatmap reflect instances where the attended segments align with the positions of the gold documents. we present the accuracy heatmaps of meta-llama-3-8b, and mistral-7b-instruct-v0.2 with 3 documents in figure 3 . in general, the top row in each heatmap outperforms the other rows, which is consistent with the findings of @xcite , indicating that the model is biased to the beginning (3-5% higher than the midsection and tail). there is no significant improvement in diagonal cells across different positions in both models, indicating that llms do not effectively adhere to relative attention instructions and lack awareness of relative position. this limitation is also observed in the closed-source model gpt-4o-mini, which similarly demonstrates 3 https://platform.openai.com/docs/models/gpt-4o-mini a deficiency in relative position awareness. 4 figure 3 : accuracy heatmaps of meta-llama-3-8b and mistral-7b-instruct-v0.2 when using relative attention instruction in no-index setting. in each cell of the heatmaps, the accuracy value is shown in % and the +-indicates the performance difference compared to without using attention instruction. the darker the color of the cell, the higher the accuracy. ## conclusion and future work we empirically study how sensitive llms are to attention instructions via a series of systematic experiments. we find that llms can be prompted to pay more attention to a document or region through direct indexing. however, we also find that models are not capable of locating a document or a region in the context based on its relative position. our results and analyses provide new insights into solving the position bias through semantic instructions and a potential pathway to achieve more effective rag by distributing attention based on relevance scores or source information confidence.