Split and Merge: Aligning Position Biases in LLM -based Evaluators (2024)

Abstract: Large language models (LLMs) have shown promise as automated evaluators for assessing the quality of answers generated by AI systems. However, LLM-based evaluators exhibit position bias, or inconsistency, when used to evaluate candidate answers in pairwise comparisons, favoring either the first or second answer regardless of content. To address this limitation, we propose PORTIA, an alignment-based system designed to mimic human comparison strategies to calibrate position bias in a lightweight yet effective manner. Specifically, PORTIA splits the answers into multiple segments, taking into account both length and semantics, and merges them back into a single prompt for evaluation by LLMs. Extensive experiments with six LLMs on 11,520 answer pairs demonstrate that PORTIA markedly enhances the consistency rates for all models and forms of comparison tested, achieving an average relative improvement of 47.46%. It also enables PORTIA-enhanced GPT-3.5 to achieve agreement rates with humans comparable to GPT-4 and elevates GPT-4’s consistency rate up to 98%. Subsequent human evaluations indicate that the PORTIA-enhanced GPT-3.5 model can even surpass standalone GPT-4 in terms of alignment with human evaluators, highlighting PORTIA’s ability to correct position bias, improve LLM consistency, and boost performance while keeping cost efficiency.

## introduction recent advances in large language models (llms) have achieved remarkable results on various tasks, sometimes even exceeding human performance @xcite . however, assessing the quality of llm-generated answers poses challenges. specifically, n-gram matching metrics like bleu (papineni et al., 2002) † corresponding authors. recently, researchers have investigated the use of powerful llms like @xcite to evaluate the quality of text generated in response to open-ended questions @xcite . notably, robust llm evaluators such as gpt-4 have been shown to align remarkably well with both controlled and crowdsourced human preferences, achieving over 60% agreement @xcite . these studies suggest that llms can emulate human evaluations, offering a scalable and transparent alternative to the expensive and time-intensive human assessment of text quality. while llms have advanced capabilities, they are not flawless evaluators and have been identified to possess certain biases. one notable bias is the position bias @xcite , in which an llm might prefer either the first or second answer in a pairwise comparison, regardless of its content, as illustrated in figure 1 . even the state-of-the-art gpt-4 model is not immune to position bias @xcite @xcite , and the behavior of its various versions can be inconsistent over time @xcite . moreover, owing to pronounced position biases in less-powerful gpt models, much of the prior research @xcite has been compelled to use the expensive gpt-4 for llm evaluations, emphasizing the necessity for a more cost-effective approach to large-scale assessments. to address these limitations, we propose por-tia 1 , an alignment-based system designed to calibrate position bias. inspired by human long-text reading strategies @xcite , portia splits the answers into multiple segments, aligns similar content across candidate answers, and then merges them back into a single prompt to feed to llm evaluators. specifically, portia first identifies possible split positions at sentence boundaries within each answer. it then conducts a length alignment between the candidates to generate segments of roughly equal length across answers. if this length alignment does not yield a consistent verdict, portia further undertakes an iterative semantic alignment to identify the optimal split positions, enabling the merging of segments across candidates. since this lightweight approach does not require changes to the models themselves, portia is readily adaptable to enhance a variety of llm evaluators for improved evaluation consistency. we conducted comprehensive experiments using six llms as evaluators to assess 11,520 answer pairs across three prevalent pairwise comparison forms. our results show that portia markedly boosts consistency rates for all the tested models and templates, achieving an average relative improvement of 47.46% and rectifying an average of 62.31% of the initially inconsistent cases. furthermore, portia addresses between 36% and 86% (over 80% for two-thirds of the comparison templates) of the position bias occurrences within the gpt-4 model, elevating its consistency rate up to 98%. moreover, efficiency and cost evaluations indicate that portia enables the less advanced gpt-3.5 model to achieve 88% agreement with the state-of-the-art gpt-4 model at merely 9.57% of the cost. additionally, a user study involving five human participants demonstrated enhanced agreement between portia-optimized evaluators and human evaluators. remarkably, the agreement of human evaluators with portia-enhanced gpt-3.5 even exceeds that with the standalone gpt-4. a subsequent ablation study suggests that portia's two key components -length alignment and semantic alignment -are beneficial for improving consistency across different comparison forms. pairwise comparison presents two answers sideby-side and asks evaluators to select the superior one. in particular, pairwise comparison methods can be further categorized into three forms: scorebased, likert-based, and relation-based. in scorebased comparison, evaluators assign a score to each answer and then compare these scores to determine the better answer. the likert-based method @xcite requires evaluators to score answers on a likert scale @xcite , where lower scores indicate a strong preference towards the first answer, middle scores represent a close tie, and higher scores signal a preference for the second answer. additionally, the relation-based comparison solicits direct inputs from the evaluators about their preference for one answer over another. this approach aims to avoid the use of potentially arbitrary scores, guiding evaluators to make relative comparisons between answers instead. the details of these three forms are shown in appendix k.1. position bias in pairwise comparison. despite the generally encouraging performance of pairwise comparison methods, we note that llm evaluators are not perfect and can exhibit certain biases. a primary concern is the position bias @xcite , whereby the llm may favor the first (or second) answer in a pairwise comparison, regardless of its content. in fact, llms have shown notable sensitivity to small changes in prompts @xcite . for clarity, we provide a formal definition of position bias as well as the consistency. as illustrated in ## the portia system design intuition. it is worth noting that both human evaluators and llms encounter difficulties in making consistent evaluations when faced with lengthy and intricate answers @xcite . a common cognitive approach among individuals is to decompose information into smaller units, thereby simplifying the comparison process (ratnasari, 2023). inspired by this observation, portia is designed to split candidate answers into segments, merge specific segments across candidates that share "comparable" content, and eventually align them. based on this intuition, portia seeks to mimic effective human comparison procedures, aiming to calibrate position bias and enhance the consistency of llm evaluators. that said, for each question, the verdicts of portia should be consistent with any permutation of the answers, i.e., - moreover, portia is specifically designed to address the position bias issue, rather than aligning the reasoning capabilities of llm-based evaluators. therefore, if an llm evaluator lacks the capability to adequately judge a pair of semantically different responses, this falls outside the scope of portia's intended purpose. portia is solely focused on rectifying inconsistencies that arise when the order of a pair of semantically different responses is altered, leading to a change in the judgment of the llm evaluator. in this context, portia is generalizable to handling open-ended answers because, regardless of their semantic differences, we can always mix the two previously individual answers. such a mixing operation effectively eliminates the position bias in pairwise llm-based evaluation. ## ethical considerations use of human annotations we protect the privacy rights of workers and pay them above the local minimum wage. all five annotators are paid 35$ per hour for the given 80 samples. careful instruction is given to ensure that the annotators understand the task and are not exposed to harmful content. study scope. our work aims to improve the consistency of llm-based evaluators, which can be utilized to assess the quality of ai-generated answers. more consistent llm-based evaluators can provide human-like evaluations at a lower cost, supplying feedback to reduce biases during training. notably, our work enhances the evaluation consistency of open-source models like llama2, enabling their use as reliable evaluators and making research on llms more accessible to the broader community. however, we recognize that malicious actors could exploit these methods to intentionally train models that go against human values. the open-source llms could be leveraged as consistent evaluators to guide the training of harmful models such as worm-gpt (wor). while our work targets constructive applications, we caution that like any technology, consistent llm evaluators could potentially be misused. researchers should consider ethical implications and preventative measures. overall, our current focus is on addressing the position bias of llm evaluators, thereby making them more consistent and reliable in supporting large-scale automatic evaluation processes. ## limitations context window length. although we do not identify any new biases introduced by portia the maximum context window length of the llm evaluator poses a challenge, as it requires the llm to process the entire input prompt, which comprises the original question and two candidate responses. if these responses are excessively lengthy, the merged prompt may exceed the maximum context window length of the llm, thereby violating the crucial principle of content preservation discussed in section 3.1. while we did not encounter this issue in our experiments, it could potentially limit the effectiveness of llms with shorter context window lengths. we believe that this can be addressed by increasing the maximum context window length of llms @xcite or simply selecting an llm with a longer context window length, such as claude2 for 100k tokens. excessive llm alignment. although portia works well under most scenarios, it is not perfect. one limitation is that portia is not able to handle the case where the llm evaluators refuse to make a verdict, which usually occurs on llms with advanced alignment techniques such as gpt series models. we find that these models would become too conservative to make a verdict, no matter how the answers are split and aligned. for example, gpt-3.5 often refuses to give any meaningful response when the question is in "roleplay" category.