HANS , are you clever? Clever Hans Effect Analysis of Neural Systems (2024)

Abstract: Large Language Models (LLMs) have been exhibiting outstanding abilities to reason around cognitive states, intentions, and reactions of all people involved, letting humans guide and comprehend day-to-day social interactions effectively. In fact, several multiple-choice questions (MCQ) benchmarks have been proposed to construct solid assessments of the models’ abilities. However, earlier works demonstrate the presence of inherent “order bias” in LLMs, posing challenges to the appropriate evaluation. In this paper, we investigate LLMs’ resilience abilities through a series of probing tests using four MCQ benchmarks. Introducing adversarial examples, we show a significant performance gap, mainly when varying the order of the choices, which reveals a selection bias and brings into discussion reasoning abilities. Following a correlation between first positions and model choices due to positional bias, we hypothesized the presence of structural heuristics in the decision-making process of the LLMs, strengthened by including significant examples in few-shot scenarios. Finally, by using the Chain-of-Thought (CoT) technique, we elicit the model to reason and mitigate the bias by obtaining more robust models.

## introduction the intensifying dispute on ai abilities has led to the evolution of robust evaluation methods to assess the actual limits of llms. recently, many anecdotal examples have been used to suggest that llms such as gpts @xcite , llamas @xcite , and other well-known models are proficient at understanding that people have ideas, thoughts, emotions, and preferences, which is referred to the neural theory of mind (n-tom) @xcite . although these abilities have been observed, earlier works advance conflicting conclusions showing that many solved tasks rely on memorization @xcite and superficial heuristics @xcite , as well-known as clever hans effect. in fact, it seems that llms are very sensitive to the arrangement of components in prompts @xcite , as it directly affects the evaluation of their ability to understand and reason about specific tasks @xcite @xcite . given these findings, our research question arises: do llms have n-tom abilities, or is it a clever hans effect? in this paper, we propose a systematic evaluation using several benchmarks with the multiple-choice questions (mcq) format to investigate the interplay between n-tom and clever hans effect. in order to probe the real abilities of llms, we introduce different adversarial strategies by varying the order and altering the content of choices in zero-and few-shot scenarios. we conduct different experiments using two versions of llama @xcite , vicuna @xcite , and falcon @xcite on four different mcq benchmarks. hence, by using piqa @xcite , openbookqa @xcite , commonsenseqa @xcite , social iqa @xcite we demonstrate that llms have particular n-tom abilities, but they are not robust. more specifically, behind in-depth analyses in a zero-shot scenario, we discover a substantial sensitivity gap between the original and adversarial benchmarks. following, we tested different settings in a few-shot scenario, where we observed that introducing examples in the input prompt led to marginal improvements in the robustness of the llms. these results led us to hypothesize that considerable sensitivity in prompting emerges from llms' positional bias in that they tend to favor specific structures. therefore, clever hans' heuristics emerge as the choice is not made through reasoning ability. nevertheless, the integration of demonstrations within the input prompts has manifested as a salient mechanism, markedly enhancing the predictive accuracy of llms. the impact of the chainof-thought paradigm elucidates bifurcated advantages: it fortifies both the robustness and interpretative stability inherent to the models while concurrently attenuating the positional bias. these methodological augmentations suggest emergent n-tom abilities, indicating a more profound and contextually attuned linguistic grasp. our findings can be summarized as follows: via these studies, we have contributed to a deeper understanding of how the order of options influences the decision-making process of llms in multiple-choice questions and offer practical solutions to increase robustness and reliability in such tasks. ## empirical investigation & analysis intending to empirically assess the incline between the neural theory of mind abilities and clever hans traps into which large language models (llms) could fall, we propose a series of experiments where we use four question-answering benchmarks presented in section 2.1 and several adversarial experiments introduced in section 2.2). ## experiments to investigate the open question of social intelligence and theory of mind in modern nlp models from an empirical viewpoint, we extended the evaluations of @xcite to a series of speculative benchmarks (section 2.1) altered with appropriately constructed adversarial shuffling (section 2.2) prompts. then, to assess the factual abilities of the large language models (llms), we set up several baseline models (section 3.1), which we probed with different approaches (section 3.2). hence, we performed a series of systematic evaluations to observe the impact of the proposed meth-ods. ## results looking for evidence that theory of mind (tom) has been inherited from neural minds is like looking for a drop in the ocean. the results in models openbookqa social iqa commonsenseqa piqa origin first last origin first last origin first last origin first last alpaca 36.2 +11.7 -9.2 48.2 +8.5 -18.6 55.2 +8.4 -11.7 62.7 +2.3 -1.8 falcon 54.8 +3.2 -13.6 57.5 +3.6 -14.5 60.2 +5.3 -7.8 68.6 +1.7 -0.9 vicuna 58.1 +3.9 -8.6 60.3 +3.1 -6.4 66.4 +6.3 -6.4 74.2 +1.9 -1.2 llama2 61.2 +3.6 -5.8 65.6 +4.3 -5.2 80.5 +2.3 -4.6 82.5 +1.6 -1.2 figure 2 : evaluation results on proposed benchmarks. first means that the target is the first choice. last means that the target is the last choice. choose the answer to the question only from options a, b, c, d. question: which of these would stop a car quicker? a) a wheel with wet brake pads b) a wheel without brake pads c) a wheel with worn brake pads d) a wheel with dry brake pads answer: let's think step by step fine-grained analysis revealed critical issues about the robustness of llms and their tendency to clever hans effects; however, elicitation to reasoning produced thrilling results that opened the way for new hypotheses about the neural-tom abilities inherited by llms. ## future works in future work, we plan to extend our experimentation to different models and observe whether this phenomenon can be mitigated through downstream model distillation techniques. hence, we will extend our work to different models, including gpt-3.5 and gpt-4. on the other hand, we study the impact and robustness of the variation of backbone model parameters (as done in @xcite ) and how it affects further trained models through refinement techniques using teacherstudent approaches @xcite and multi-and cross-lingual techniques @xcite @xcite . at the same time, it will be of interest to us to analyze whether prompt engineering techniques are affected by this phenomenon, such as chain-of-thought in contexts with fewshots and tree-of-thought in cross-lingual contexts @xcite . addressing these studies will allow us to look at the problem from multiple perspectives and investigate the consequences of shallow heuristics. finally, we will analyze the impact of a further injection of bias into the best-known benchmarks to observe whether the capabilities of llms can overcome challenging scenarios in order to understand whether these phenomena are indeed related to structural representations @xcite @xcite handed down by the models or are merely the result of structural features of large language models @xcite . ## conclusion the large language models (llms) have been demonstrating interesting abilities in real-world understanding. empirically assessing these abilities is a challenging task. in our contribution, we propose systematic evaluations through multiple-choice questions (mcq) benchmarks. however, our study revealed an inherent order-bias in these models. through adversarial testing, we observed a significant discrepancy in performance, particularly when altering the sequence of options, underlining a prevailing selection bias that challenges the reasoning abilities of the llms. we identified a link between positional preferences and model selections, which led us to theorize the existence of structural heuristics guiding the decision-making process. by incorporating relevant examples in few-shot contexts, this notion was further strengthened. using chain-of-thought approaches allowed us to make the model introspect its decisions, thus reducing observed bias and resulting in more reliable and robust llms. our results revealed some limitations regarding robustness in zero-shot scenarios but simultaneously showed that the cot approach enhances stability. our future research will focus on proposing definitely unseen benchmarks to evaluate real abilities without the presence of distorted glass.