Strengthened Symbol Binding Makes Large Language Models Reliable Multiple-Choice Selectors (2024)

Abstract: Multiple-Choice Questions (MCQs) constitute a critical area of research in the study of Large Language Models (LLMs). Previous works have investigated the selection bias problem in MCQs within few-shot scenarios, in which the LLM’s performance may be influenced by the presentation of answer choices, leaving the selection bias during Supervised Fine-Tuning (SFT) unexplored. In this paper, we reveal that selection bias persists in the SFT phase , primarily due to the LLM’s inadequate Multiple Choice Symbol Binding (MCSB) ability. This limitation implies that the model struggles to associate the answer options with their corresponding symbols (e.g., A/B/C/D) effectively. To enhance the model’s MCSB capability, we first incorporate option contents into the loss function and subsequently adjust the weights of the option symbols and contents, guiding the model to understand the option content of the current symbol. Based on this, we introduce an efficient SFT algorithm for MCQs, termed Point-wise Intelligent Feedback (PIF). PIF constructs negative instances by randomly combin- ing the incorrect option contents with all candidate symbols, and proposes a point-wise loss to provide feedback on these negative samples into LLMs. Our experimental results demonstrate that PIF significantly reduces the model’s selection bias by improving its MCSB capability. Remarkably, PIF exhibits a substantial enhancement in the accuracy for MCQs.

## introduction multiple-choice questions (mcqs) are ubiquitously employed in the realm of large language models (llms). they typical comprise a query accompanied by an array of potential options, wherein the model's assignment is to discern and select the most appropriate solution. given the significance, numerous studies @xcite @xcite have explored the challenges faced in the few-shot phase of mcqs. one of the main challenges is the selection bias @xcite , which refers that llms are sensitive to variations in the arrangement of options within mcqs. figure 1 demonstrates an example of selection bias. foundation llms such as llama2-7b can seldom acquire data-sensitive domain knowledge. for such scenarios, it is crucial to perform supervised fine-tuning (sft) on llms. thus we hope llms to ensure accuracy and also robustly select reliable options in mcqs during sft. unfortunately, we find that the selection bias still exists during the sft phase. following the methodology of @xcite , we conduct "answer-moving attack" experiment by always moving the golden option content to a specific symbol, and the results are displayed in why do llms show selection bias during the sft stage? we propose a hypothesis that "strengthened symbol binding makes large language models reliable multiple-choice selectors". we utilize multiple choice symbol binding (mcsb) capability @xcite to represent the llm's ability to bind option symbols and their corresponding option contents, and employ proportion of plurality agreement (ppa) metric to compare the relative mcsb ability of two llms. through comprehensive experimental validation, we discover that improving the llms' performance on the ppa metric alleviates the llms' performance on selection bias. based on the relationship between the llms' selection bias and its mcsb capabilities, we expect to mitigate selection bias by enhancing the model's mcsb capability. we first incorporate option contents into the loss function, guiding the model to understand the content of the current symbol. however, the results are less than satisfactory. considering that label words are anchors @xcite , we adjust the weights of the option symbols and contents in the optimization objective, termed reweighting symbol-content binding (rscb). subsequently, inspired by reinforcement learning from human feedback @xcite , we propose point-wise intelligent feedback (pif), in which we construct negative samples by randomly combining the contents of incorrect options with all option symbols, and design a pointwise loss to feedback these negative samples into sft. finally, pif not only ensures the stability of model performance but also enhances it. in summary, our contributions are as follows: (1) we conduct experiments to demonstrate that there is still the selection bias when performing sft on llms for mcqs. we hypothesize that strengthened symbol binding makes large language models reliable multiple-choice selectors. in section 2.3, we validate this hypothesis by investigating the correlation between the mcsb capability and the selection bias. (2) we mitigate selection bias by enhancing the model's mcsb capability. we propose the point-wise intelligent feedback (pif) method, which constructs negative samples by randomly combining the content of incorrect options with all candidate symbols and designing a point-wise loss to provide feedback on these negative samples into llms. (3) we conduct extensive experiments to validate that our pif method can significantly alleviate the selection bias of llms during the sft phase for mcqs. meanwhile, the experimental results prove that pif could also enhance the accuracy performance of the model. ## methodology according to the previous analysis, we can mitigate the selection bias of llms during sft by enhancing their mcsb abilities. ## related work large language models in recent years, the scale of the model parameters has progressively increased with the rapid development of deep learning, from 1.5 billion in 2018 @xcite to 540 billion in 2022 @xcite . a significant number of large language models have swiftly surfaced, especially after the emergence of chatgpt(openai, 2022), such as llama @xcite , vicuna @xcite , and bloom @xcite . these large language models have a vast amount of parameters. after being trained with massive data using generative methods, they possess the impressive ability of natural language understanding and can follow hu-man instructions effectively @xcite . ## conclusion this paper highlights that selection bias persists in the sft of llms in mcqs. to explore why llms suffer from selection bias, we suppose that "symbol binding makes large language models reliable multiple-choice selectors". we utilize mcsb capability to represent the llms' ability to bind option symbols and the corresponding content and design experiments to establish the relationship between mcsb capability and the selection bias. finally, we eliminate selection bias by enhancing the model's mcsb capability. we propose pif, constructing negative samples by randomly combining the content of incorrect contents with all candidate symbols and designing a point-wise loss to resume these negative samples. comprehensive experimental results demonstrate that pif significantly reduces selection bias and substantially improves the accuracy of llms for mcqs.