Primacy Effect of C hat GPT (2023)

Abstract: Instruction-tuned large language models (LLMs), such as ChatGPT, have led to promising zero-shot performance in discriminative natural language understanding (NLU) tasks. This involves querying the LLM using a prompt containing the question, and the candidate labels to choose from. The question-answering capabilities of ChatGPT arise from its pre-training on large amounts of human-written text, as well as its subsequent fine-tuning on human preferences, which motivates us to ask: Does ChatGPT also inherit humans’ cognitive biases? In this paper, we study the primacy effect of ChatGPT: the tendency of selecting the labels at earlier positions as the answer. We have two main findings: i) ChatGPT’s decision is sensitive to the order of labels in the prompt; ii) ChatGPT has a clearly higher chance to select the labels at earlier positions as the answer. We hope that our experiments and analyses provide additional insights into building more reliable ChatGPT-based solutions. We release the source code at https://github.com/wangywUST/PrimacyEffectGPT.

## introduction humans tend to recall information presented at the start of a list better than information at the middle or end. this phenomenon is known as the primacy effect @xcite , which is a cognitive bias that relates to humans' attention spans @xcite , rehearsal @xcite , and memory systems @xcite . similarly, in advertisement systems and search engines, humans tend to interact with items in higher positions regardless of the items' actual relevance @xcite . primacy effect influences humans' behaviors to make unfair decisions. similarly, if it exists in machine learning models, it may lead to worse performance. recently, instruction-tuned large language models (llms), represented by @xcite , have received wide attention on their capabilities of imitating humans in question-answering and problem-solving. however, this underlying behavioral similarity between chatgpt and humans naturally leads to an intriguing question: is chat-gpt also affected by the primacy effect? ## primacy effect of chatgpt in this section, we first introduce the general prompt design of chatgpt in discriminative natural language understanding (nlu). then, we analyze the primacy effect of chatgpt using label shuffling in prompts. ## experiments we analyze the primacy effect based on the aforementioned strategies using three relation extraction datasets and an intent detection dataset. ## related work large language models (llms) @xcite @xcite , such as gpt-3 @xcite , lamda @xcite and palm @xcite , refer to large scale pretrained models that contain more than a hundred billion parameters. based on the highly parallelizable transformer architecture @xcite , these large language models have shown powerful capability to produce reasonable results with very few samples or task descriptions as input. a key milestone in the development process of llms is chatgpt, which is developed by ope-nai based on instructgpt @xcite . chatgpt is able to interact with humans through multiple turns of dialogue, understand user intent, accomplish instructions, and return human-like responses. this attracts huge attention from research field, motivating numerous recent work @xcite @xcite @xcite to utilize chat-gpt to different tasks. as chatgpt is a proprietary model, and openai does not disclose its training specifics, researchers are actively investigating its associated implications and capabilities. there has been some work analyzing the performance, robustness, faithfulness, and explain-ability of chatgpt @xcite @xcite . for example, @xcite investigates the educational integrity of chatgpt and evaluates the chatgpt's abilities to solve assignments of various levels in computer security specialization. @xcite and @xcite investigate the ethical risks of chatgpt. before chatgpt, llms' inference has been accompanied by in-context learning (icl) which adds a few demonstrations in the prompt @xcite . accordingly, some work investigates the effects of demonstration orders for the llms before chatgpt @xcite . @xcite finds the majority label, recency, and common token biases of llms' icl. different from the above work, we focus on a new phenomenon of chatgpt: the primacy effect, which is the tendency of selecting the first labels as the answer. the primary effect seriously influences chatgpt's fairness. collectively, our findings provide a new understanding of how chatgpt works given the instructional prompts. ## conclusion while previous work often takes chatgpt as a universal method applicable to all text-related tasks, we argue that its flexibility comes with the risk of inheriting human's cognitive biases. these biases lead to unfair judgments which can affect the performance of the machine learning model. this work studies a cognitive bias of chatgpt: primacy effects. we propose a simple yet effective label shuffling method to analyze the influence of label orders on chatgpt. we discover the primacy effect of chatgpt and finds that it highly influences the fairness of chatgpt in nlu. our work contributes to a better understanding of the behaviors of chatgpt and building more faithful solutions with chatgpt in nlu applications.