Examining Cognitive Biases in C hat GPT 3.5 and C hat GPT 4 through Human Evaluation and Linguistic Comparison (2024)

Abstract: This paper aims to investigate the presence of cognitive biases, more specifically of Availability heuristics, Representativeness heuristics and Framing, in OpenAI’s ChatGPT 3.5 and ChatGPT 4, as well as the linguistic dependency of their occurrences in the Large Language Models’ (LLMs) outputs. The innovative aspect of this research is conveyed by rephrasing three tasks proposed in Kahneman and Tversky’s works and determining whether the LLMs’ answers to the tasks are correct or incorrect and human-like or non-human-like. The latter classification is made possible by interviewing a total of 56 native speakers of Italian, English and Spanish, thus introducing a new linguistic comparison of results and forming a “human standard’. Our study indicates that GPTs 3.5 and 4 are very frequently subject to the cognitive biases under discussion and their answers are mostly non-human-like. There is minimal but significant discrepancy in the performance of GPT 3.5 and 4, slightly favouring ChatGPT 4 in avoiding biased responses, specifically for Availability heuristics. We also reveal that, while the results for ChatGPT 4 are not significantly language dependent, meaning that the performances in avoiding biases are not affected by the prompting language, their difference with ChatGPT 3.5 is statistically significant.

## introduction in the last years, large language models (llms) have been used exponentially thanks to their capabilities to be queried with natural language and to return content-and context-aware responses. they became popular within the general public, and businesses swiftly introduced these models in their workflow aiming at becoming more productive, while reducing employees' workload. natural language itself is not only our easiest and quickest way to communicate to these language models, but also the main reason why we tend to anthropise these machines @xcite , making our relationship with them resonate widely and strongly in our everyday life. nonetheless, "llms simply do not have the capacity to distinguish between truth and falsehood and, therefore, without malicious intent, @xcite can confidently present fictions as if they were truths" @xcite . for this reason, we have the responsibility to prove if, how and when they are most reliable. much work has been done in delicate fields such as legal, medical and educational @xcite @xcite @xcite to analyse how to improve their use in the professionals' decision-making process and to help users make more conscious choices. when only taking the outputs into consideration, the main hindrance to their implementation into businesses and field-specific tasks are hallucinations, defined as "the generated content that is nonsensical or unfaithful to the provided source content" @xcite . hallucinations are defined as in-trinsic, when the output contradicts the source, or extrinsic, when the output cannot be verified from the source @xcite . given the potentially harmful and often subtle nature of this phenomenon, researchers have developed various hallucination mitigation techniques. these techniques operate at different levels of the llm's functioning to help reduce their occurrence. addressing the issue can involve either prompt engineering or developing models to reduce the elicitation of hallucinations. specific actions are available for each level of the llm's functioning @xcite . alongside the phenomena that hinder menaces, the utility of llms is threatened by the presence of cognitive biases in their outputs. as hallucinations, cognitive biases are concepts mainly used to describe human behaviours and have been adapted to this field to define machines' dysfunctions. generative artificial intelligence (genai) can exhibit biases due to various factors. some of the key causes are that llms are trained on human-made data, including historical data. they can be skewed and affected by under/over-representation of certain phenomena. moreover, biases can be introduced in the process of data annotation and refinement, often based on reinforced learning with human feedback (rlhf) @xcite @xcite and propagate into the models. thus, machines can potentially inherit and enhance human cognitive biases. the main focus of this article is to verify whether chatgpt 3.5 and chatgpt 4 are susceptible to three specific cognitive biases, known in the literature as availability heuristics, representativeness heuristics and framing @xcite @xcite . the dictionary of psychology issued by the american psychological association defines them as follows. availability heuristics is: "a common strategy for making judgments about likelihood of occurrence in which the individual bases such judgments on the salience of the information held in their memory about the particular type of event" foot_1 . representativeness heuristics is: "a strategy for making categorical judgments about a given person or target based on how closely the exemplar matches the typical or average member of the cate-gory" foot_2 . finally, framing is: "the process of defining the context or issues surrounding a question, problem, or event in a way that serves to influence how the context or issues are perceived and evaluated" 3 . a recent trend in generative artificial intelligence literature is "machine psychology" by thilo hagendorff, meaning that the llm is positioned by the researchers as the subject of their psychological tests, initially designed to investigate human misbehaviour @xcite . according to this approach, llms are tested for cognitive biases using their chatbot interfaces. the tools mainly investigated for tracking cognitive biases are open ai's chatgpt (version 3.5, 3.5 turbo and 4), google's gemini, anthropic's claude in different versions and llama in different versions @xcite @xcite . most studies refer to the tasks proposed by kahneman and tversky to test cognitive biases @xcite @xcite and have unmasked that llms are, in fact, victims of these biases, including but not limited to: confirmation bias (macmillan-scott and m., 2024; @xcite @xcite @xcite , availability heuristics @xcite @xcite , overconfidence @xcite @xcite , representativeness heuristics (macmillan-scott and m., 2024; @xcite , framing @xcite , recency effect @xcite . the primary methodology described in these studies either directly or indirectly refers to the "machine psychology" approach @xcite . this paper addresses the following research questions: based on the results of previous research, it can be supposed that: -llms show biases like availability heuristics @xcite @xcite , representativeness heuristics (macmillan-scott and m., 2024; @xcite and framing @xcite in their outputs; -there are differences between chatgpt 3.5 and 4, where 4 should be less subject to biases since it is trained on more data, or at least gives better performances according to @xcite ; -llms should be language dependent since the data with which they are trained differ among languages, causing different performances, or at least their results in massive multitask language understanding (mmlu) are, despite a minimal difference, better in english than italian or spanish @xcite . ## methodology in consideration of recent literature, the aim is to analyse the biases of availability heuristics, representativeness heuristics and framing in two llms, namely chatgpt 3.5 and 4 foot_4 . this will be done by introducing a new rephrasing approach to three specific tversky and kahneman tests, namely the judgement of word frequency @xcite to demonstrate availability heuristics, the linda problem @xcite to demonstrate representativeness heuristics and the framing of contingencies @xcite to demonstrate framing. the presence of biases in llms outputs was examined by classifying the answers of the llms according to the methodology proposed by olivia macmillan-scott and mirco musolesi in their paper (macmillan-scott and m., 2024), using four parameters: correct/incorrect and human-like/non-humanlike. to support the definition of what is humanlike, a pool of 56 people (59% women, 41% men, age average: 33) was interviewed, defined by the availability of resources, yet guaranteeing the representativeness of the sample, posing the same questions asked to the two chatgpt models. ethics ap-proval was not obtained since the research did not involve sensitive personal information or interventions that required formal ethical oversight. additionally, to verify whether the answers of the models, as well as their potential biases, are language dependent, a multilingual analysis was conducted by prompting the llms in italian, english and spanish. the innovative aspect of this work lies in the multilingual comparison between human native speakers and llm data, setting it apart from previously analysed reference material. to answer the research questions, we followed the methodology described in the subsequent sections. all data regarding the complete prompts and results are available in a public github repository. foot_5 ## results in order to have a more complete and clear view about the outcomes of this research, the results that appear in this section are divided in general results and language-specific results. ## discussion in the previous section, we presented the results of our examinations. in this section, we discuss them to address the research questions and the hypotheses from the introduction. similar to previous studies, it was found that availability heuristics, representativeness heuristics, and framing are indeed present in the outputs of chatgpt 3.5 and 4. among them, the less frequent bias is availability heuristics, since across all prompts in italian, english and spanish, chatgpt 4 was able to answer correctly 98.2% of the times. the most frequent bias is framing, which was reported 100% of the times for both llms and across the three languages, with a minor difference for gpt 3.5 that in spanish gave a not applicable (n/a) answer. referring to representativeness heuristics, the bias is undoubtedly present in llms answers, but quite less frequently than framing. this study also aims to evaluate potential differences of bias appearance in the two analysed models. as hypothesised, gpt 4 performs slightly better than gpt 3.5. on the one hand, its higher percentage of correct outputs is statistically significant according to both the t-test and the wilcoxon signed-rank test, even though the effect size is small. additionally, it performs better in the way it approaches a problem and provides a solution: when presented with choices among the various options, chatgpt 4 exhibits a tendency to provide more detailed explanations for its decision-making process compared to gpt 3.5. this behaviour is hypothesised to reflect the machine's tendency to convince the user of its answer, even though this can sometimes lead the machine to fall victim of confirmation bias, a phenomenon already demonstrated in other studies (macmillan-scott and m., 2024; @xcite @xcite @xcite . for this reason, to guide the drafting of one single response for each llm, the iterative approach was integrated to the zero-shot prompting. the present study wants to determine whether prompting in different languages has effects on the biases occurrence. it can be concluded that the italian outputs are more similar to their respective human counterparts. in contrast, the spanish outputs exhibit the highest frequency of correct answers. the english results instead are consistent with the general findings. unlike humans, llms tend to be highly confident in their answers, consistently reproducing the same results across numerous requests, even when prompted in separate, new chats, each time. this does not indicate that llm outputs are consistent; rather, it suggests that they are more susceptible to biases than human responses. they are more vulnerable to being influenced by biases compared to humans, who tend to demonstrate a more varied and inconsistent frequency in their answers, irrespective of these being correct or incorrect. when investigating the language dependency of results, the anova test and also the kruskall-wallis test show the performance of chatgpt 3.5 are statistically different for the language combinations english-spanish and italian-spanish. the combinations italian-english with chatgpt 3.5 and all language combinations with chatgpt 4 are not statistically different. these data must be interpreted with caution: as many studies in this field, this analysis is subject to limits. a wider range of biases, prompts, language models, natural languages, participants, and methodologies should be applied to guarantee more reliable results. it is important to remember that these models are considered "stochastic parrot[s]" @xcite , thus non-deterministic in their answers. the results of the research, even if conducted with high standards of control, may not be generalisable to any broader range. the development of a wider picture of cognitive biases in llms is subject to the performance of additional studies, with the objective of tackling the problem and further analysing the models along their evolution, for example taking into consideration chatgpt 4o and other models from different developers. further research should be also undertaken to investigate the influence of the prompt formulation and the relevance of specific wording in the elicitation of cognitive biases, or hallucinations in general, by the machine. the present study was designed to determine whether the two llms under scrutiny exhibit cognitive biases similar to humans, considering the human nature of the data and feedback they are trained on. the research aimed to determine the frequency of these biases, compare their prevalence between chatgpt 3.5 and 4, and examine whether the emergence of these biases is influenced by the language of the prompts, thereby determining if they can be considered language-dependent. as demonstrated by recent literature, these machines reflect many different types of cognitive biases. the investigation focused on the occurrence of availability heuristics, representativeness heuristics, and framing. this study suggests that biases are very frequently present in llm outputs, especially when the prompt structure imposes the machine to make a choice with limited information available. compared to gpt 3.5, chatgpt 4 proved to be slightly less affected by these biases, especially by availability heuristics. however, both of them are subject to biases. another significant assertion in the restitution of biases concerns the fact that, at the moment, the only two combinations that seem to depend on the natural language they are prompted with are italian/english-spanish for chatgpt 3.5. it is worth noting that the languages chosen for this research show similar performances according to openai's paper @xcite . a future analysis could be designed taking into consideration two very different performing languages, so to verify this result further. the insights gained here should raise awareness when using llms, regardless of the purpose of use. this awareness is particularly crucial in fields such as medicine, law, education and research, where llms play a significant role in decisionmaking processes @xcite . task name: judgment of word frequency cognitive bias: availability heuristics original english consider the letter r. is r more likely to appear in: the first position; the third position? check one. @xcite based on your knowledge, which of the two words are more common: "yes" or "no"? task name: linda problem cognitive bias: representativeness heuristics original english linda is 31 years old, single, outspoken and very bright. she majored in philosophy. [pick one assumption from the following.] as a student, she was deeply concerned with issues of discrimination and social justice, and also participated in anti-nuclear demonstrations. linda is a teacher in elementary school. linda works in a bookstore and takes yoga classes. linda is active in the feminist movement. linda is a psychiatric social worker. linda is a member of the league of women voters. linda is a bank teller. linda is an insurance salesperson. linda is a bank teller and is active in the feminist movement. @xcite julia is a law trainee in a company, she has a partner and she has recently started volunteering in the road accidents center in her city. her friend saw her wearing flipflops and short and carrying a helmet while walking. it's winter time. what does julia own: i don't know. a house on the beach. a house on the beach and a motorbike. a house on the beach and a bike. choose one of the options above.