Do LLM s Overcome Shortcut Learning? An Evaluation of Shortcut Challenges in Large Language Models (2024)

Abstract: Large Language Models (LLMs) have shown remarkable capabilities in various natural language processing tasks. However, LLMs may rely on dataset biases as shortcuts for prediction, which can significantly impair their robustness and generalization capabilities. This paper presents Shortcut Suite, a comprehensive test suite designed to evaluate the impact of shortcuts on LLMs’ performance, incorporating six shortcut types, five evaluation metrics, and four prompting strategies. Our extensive experiments yield several key findings: 1) LLMs demonstrate varying reliance on shortcuts for downstream tasks, which significantly impairs their performance. 2) Larger LLMs are more likely to utilize shortcuts under zero-shot and few-shot in-context learning prompts. 3) Chain-of-thought prompting notably reduces shortcut reliance and outperforms other prompting strategies, while few-shot prompts generally underperform compared to zero-shot prompts. 4) LLMs often exhibit overconfidence in their predictions, especially when dealing with datasets that contain shortcuts. 5) LLMs generally have a lower explanation quality in shortcut-laden datasets, with errors falling into three types: distraction, disguised comprehension, and logical fallacy. Our findings offer new insights for evaluating robustness and generalization in LLMs and suggest potential directions for mitigating the reliance on shortcuts.

## introduction the field of natural language processing (nlp) is experiencing rapid advancements, driven by the emergence of large language models (llms) such as gpt @xcite , gemini @xcite , and llama @xcite series. these models have been pivotal in revolutionizing a wide array of tasks by leveraging techniques like in-context learning (icl) @xcite and chain-of-thought (cot) promptings @xcite , demonstrating exceptional capabilities without parameter updates. despite these advances, the research on the robustness and generalization ability of llms across different contexts remains limited. models with poor robustness and generalization may rely on "shortcut learning," where they develop decision rules that perform well on standard benchmarks but fail to transfer to more challenging testing conditions, such as real-world scenarios @xcite . therefore, evaluating llms performance in the face of shortcut information is crucial for understanding their robustness and generalization capabilities. a recent study investigates the reliance of llms on shortcuts or spurious correlations within prompts @xcite . however, this research falls short of providing an exhaustive evaluation across a broad spectrum of llms and varied prompting contexts, focusing solely on icl experiments. furthermore, it only considers relatively simple shortcuts such as letters or signs. consequently, its evaluation lacks comprehensiveness and granularity. to address this, we introduce shortcut suite, an in-depth test suite designed to evaluate the performance of different llms across six shortcuts, five metrics, and four prompt settings. extensive experiments on shortcut suite reveal that llms tend to capture spurious correlations between source text and particular labels, indicating a prevalence of shortcut learning. for example, as shown in figure 1 , gemini-pro resorts to matching subsequences (the professor recommended the bankers) in a natural language inference (nli) task rather than comprehending the clause structure or delving into the sentence's semantic content. this tendency of llms to capture spurious correlations can significantly impair their performance. in this paper, we conduct a comprehensive evaluation of llms' behavior concerning shortcut learning from the following perspectives. first, to identify the reliance of llms on shortcuts in downstream tasks, we collect six datasets containing different shortcuts and analyze the accuracy of llms on these datasets. we find a notable performance drop across various shortcuts, especially constituent and negation shortcuts, in some cases by more than 40%. moreover, in the position dataset, llms demonstrate a propensity for shortcut learning behavior by prioritizing the beginning of sentences while neglecting the end, revealing a vulnerability to additional information within sentences. furthermore, an analysis of the distribution of llms' predictions revealed inherent biases, with the llms favoring certain labels over others even in a balanced standard dataset. second, we perform comprehensive evaluation metrics to assess the impact of shortcuts on llms. in addition to accuracy, we introduce three novel metrics to assess the explanatory power of llms: semantic fidelity score (sfs), internal consistency score (ics), and explanation quality score (eqs). our analyses using these metrics reveal that llms' explanations often contain contradictions. furthermore, we prompt llms to report their confidence levels and consistently find that they are overconfident in their predictions. third, we compare the performance of different llms and different prompting strategies in shortcut learning. closed-source and some opensource llms excel on standard datasets but falter on those with shortcuts. surprisingly, larger llms are more prone to utilize shortcuts under zero-shot and few-shot icl prompts. we find that llms are less affected by shortcuts under cot settings than others. notably, llms often demonstrate inferior performance in few-shot scenarios compared to zero-shot scenarios. finally, we summarize three error types of llms in shortcut learning by checking their cot responses: distraction, disguised comprehension, and logical fallacy. these errors predispose llms to adopt shortcuts, undermining their robustness. shortcut learning in llms. @xcite provided a review of recent developments that address the robustness challenge of llms. the most related work was the study investigating the reliance of llms on shortcuts within in-context learning @xcite . our work differs from it in the following ways: first, their experiments were conducted on a limited model scope (gpt2 ## shortcut suite as nli is well positioned to serve as a benchmark task for research on nlp and can encapsulate the entire spectrum of the six identified shortcuts, we mainly anchor our framework on it. we also explore other tasks in appendic c. building on previous research, we create six datasets with different shortcuts and develop five metrics to investigate llms' shortcut learning behavior and understand their robustness generalization capabilities. ## experimental results we conduct our experiments based on the shortcut suite and observe that llms tend to exploit various shortcuts in downstream tasks, resulting in a notable decrease in performance. in this section, we present a comprehensive analysis. ## conclusion in this study, we proposed shortcut suite, a test suite designed to evaluate the performance of llms in shortcut learning across several nlp tasks. shortcut suite encompasses six types of shortcuts: lexical overlap, subsequence, constituent, negation, position, and style, and evaluates performance using five metrics: acc, sfs, ics, eqs, and cfs, across four prompt settings: zero-shot, few-shot icl, zero-shot cot, and fewshot cot. our extensive experiments on diverse llms demonstrated that llms frequently rely on shortcuts in downstream tasks. we explored the impact of different models, types of shortcuts, and prompting strategies. our analysis then extended to explanation quality, label distribution, confidence score and error analysis. our findings offer new perspectives on llms' robustness and present new challenges for reducing their shortcut reliance, paving the way for future advancements in this field.