An Empirical Analysis on Large Language Models in Debate Evaluation (2024)

Abstract: In this study, we investigate the capabilities and inherent biases of advanced large language models (LLMs) such as GPT-3.5 and GPT-4 in the context of debate evaluation. We discover that LLM’s performance exceeds humans and surpasses the performance of state-of-the-art methods fine-tuned on extensive datasets. We additionally explore and analyze biases present in LLMs, including positional bias, lexical bias, order bias, which may affect their evaluative judgments. Our findings reveal a consistent bias in both GPT-3.5 and GPT-4 towards the second candidate response presented, attributed to prompt design. We also uncover a lexical bias in both GPT-3.5 and GPT-4, especially when label sets carry connotations such as numerical or sequential, highlighting the critical need for careful label verbalizer selection in prompt design. Additionally, our analysis indicates a tendency of both models to favor the debate’s concluding side as the winner, suggesting an end-of-discussion bias.

## introduction prior research in automatic debate evaluation has predominantly relied on pre-trained encoders and the modeling of argument relations and structures @xcite @xcite . a significant drawback of these approaches is their dependency on feature engineering and extensive data training, limiting their generalizability across diverse datasets. the advent of advanced large language models (llms) such as gpt-3.5 and gpt-4 @xcite has marked the beginning of a new era in automating a wide spectrum of complex tasks @xcite @xcite @xcite evaluation: pro side is the winner llms positional biases, lexical biases, order biases, etc. figure 1 : large language models presents various biases during the evaluation of long debates. @xcite . these models have been increasingly utilized as automatic evaluators @xcite @xcite @xcite . leveraging llms for debate evaluation presents more challenges, including the extended duration of debates, evolving argument dynamics, and the necessity for evaluators to rely on comprehensive knowledge and reasoning that extend beyond the immediate scope of the debate. our research delves into the utilization of llms for debate evaluation, uncovering their zero-shot capabilities that parallel human evaluators and surpass all existing state-of-the-art (sota) methods finetuned on ample data @xcite . we further investigate potential biases in gpt-3.5 and gpt-4 within the context of debate evaluation. while previous research has identified various biases in llms, such as persona bias @xcite , political bias @xcite , and positional bias @xcite , our investigation uniquely concentrates on biases affecting debate evaluation performance, a relatively unexplored domain. specifically, upon comparing outcomes between scenarios where the positions of candidate responses are switched, persistent bias has been observed in both gpt-3.5 and gpt-4 toward the second candidate response presented, a positional bias induced by the prompt design. beyond this, both models also display significant lexical biases, particularly when label sets carry connotations such as sequential or magnitude, underscoring the importance of careful selection of label verbalizers in prompt design to mitigate unintended biases @xcite . moreover, our study reveals that both gpt-3.5 and gpt-4 exhibit a tendency to favor the concluding side of a debate as the winner, pointing to a potential end-of-discussion order bias. interestingly, after all the identified biases are eliminated, gpt-3.5 still demonstrates a consistent bias, while this residual bias is less obvious for gpt-4. these insights highlight the nuanced nature of biases in llms and the complexity of designing fair and unbiased evaluation methodologies for debate evaluation. ## methodology llms' capability for debate evaluation. as illustrated in for each debate topic, two labels are assigned to represent the pro and con sides, respectively. these labels are then inserted into the designated slots of the evaluation template, creating a prompt t (@xmath0, @xmath1. this tailored prompt is subsequently employed to query the llm for its judgment on the debate. experiments with other templates are presented in the appendix a.4. llms' biases in debate evaluation. analyzing errors and biases in llms is complex due to their multifaceted decision-making processes, which are influenced by inherent orders, stances, and the nuances of prompt design. to separate llm-induced biases from dataset biases, we use a balanced dataset, ensuring class representation equality and mitigating skewness from overrepresentation. this method focuses on llm-specific biases within a controlled environment, minimizing dataset composition effects. we explore llm sensitivity to label choice from alphabetic (a/b, ## experiments dataset. we utilize ddo dataset @xcite , which comprises 77,655 debates from 23 topics on debate.org, structured into rounds with a single utterance from each of the pro and con side. we focus on debates of 3 to 5 rounds, defining winners by audience vote differences exceeding two, and exclude debates with forfeits to maintain analysis integrity, following previous works' setting @xcite . the length of these debates aligns well with the input length capacities of current llms, making it more suitable than other datasets derived from transcribed debate videos. we present experiments on an additional dataset in the appendix a.6, which demonstrate consistent findings. the dataset exhibits a win bias towards the con side across 3 to 5-round debates (36.9% vs. 63.1%, 44.9% vs. 55.1%, 37.9% vs. 62.1%, respectively), likely due to a concluding side bias with con frequently concluding debates. to evaluate llms in debate assessment, we propose two settings: balanced and unbalanced. the unbalanced setting replicates the original dataset's distribution, sampling 500 debates for each round count (totaling 1500). conversely, the balanced setting aims to examine llms' inherent bias by ensuring equal representation of four scenarios-pro or con initiating and winning or losing-with 125 debates each for 3 and 4 rounds, and due to data constraints, 75 debates each for 5 rounds, resulting in 500 debates for 3 and 4 rounds and 300 for 5 rounds. evaluation metrics. in addition to the accuracy reported by previous works, we measure weighted f-1 score to accommodate the imbalance between pro win and con win in the original data distribution, aiming for a more comprehensive understanding. ## discussion our research demonstrates that llms outperform current sota models in evaluating debates but are influenced by specific word choices, affecting their efficacy. we highlight llms' embedded biases-positional, lexical, and order-offering insights for future llm training enhancements. despite attempts to neutralize positional bias by shuffling labels in figs 3a and 3d , gpt-3.5 still exhibits a pro bias, contradicting its lexical preference for 'b'('con'). this might suggest a con-firmation bias-like tendency in gpt-3.5, favoring agreement with the debate topic. we further conduct experiments shuffling a/b with b/a and 1/-1 with -1/1 label sets, where each label randomly represents pro or con in 50% of cases, with positions also shuffled. despite eliminating lexical and positional biases, results indicate a persistent pro bias, detailed in appendix figure 6 , pointing to an underlying tendency warranting further investigation. ## limitations the insights from our investigation, based on the examination of gpt-3.5 and gpt-4, indicate that the discerned behavioral patterns might be unique to these specific models and not necessarily extend to other language models with divergent architectures or training approaches. with the relentless advancement in language model technology and the anticipation of updated versions, the biases detected in gpt-3.5 and gpt-4 could become obsolete in subsequent iterations. highlighting the significance of prompt types and training techniques on the efficacy of models, our research underlines the imperative for continued research to identify the optimal prompt types for various scenarios and the optimal training methods for reducing bias. although various biases may interact and potentially counterbalance each other, leading to improvements, the intensity of distinct bias types can vary significantly across different contexts. consequently, a prompt that appears to exhibit balanced bias in one scenario may manifest more pronounced bias under slightly altered conditions.