Focus on Your Question! Interpreting and Mitigating Toxic C o T Problems in Commonsense Reasoning (2024)

Abstract: Large language models exhibit high-level commonsense reasoning abilities, especially with enhancement methods like Chain-of-Thought (CoT). However, we find these CoT-like methods lead to a considerable number of originally correct answers turning wrong, which we define as the Toxic CoT problem. To interpret and mitigate this problem, we first utilize attribution tracing and causal tracing methods to probe the internal working mechanism of the LLM during CoT reasoning. Through comparisons, we prove that the model exhibits information loss from the question over the shallow attention layers when generating rationales or answers. Based on the probing findings, we design a novel method called RIDERS (Residual decodIng and sERial-position Swap), which compensates for the information deficit in the model from both decoding and serial-position perspectives. Through extensive experiments on multiple commonsense reasoning benchmarks, we validate that this method not only significantly eliminates Toxic CoT problems (decreased by \textbf{23.6} %), but also effectively improves the model’s overall commonsense reasoning performance (increased by \textbf{5.5} %).

## introduction with the increase in scale, large language models (llms) have demonstrated outstanding performance in different tasks @xcite @xcite , among them, commonsense reasoning has received significant attention due to its importance for general intelligence @xcite @xcite . in this task, researchers have proposed a series of chain-of-thought (cot) like techniques to elicit models' potential abilities (e.g. self-consistency @xcite , least-to-most @xcite , reflexion @xcite ). the bald eagle is a symbol of america. it is the national bird of the united states. metal is used to make what? (1) junkyard ( 2 ) ore (3) instruments (4) metal fabrication shop (5) bowls metal can be used to make various things, but the most common use of metal is to make tools and machines. what kind of status is the bald eagle given? (1) outside ( 2 ) world (3) protection (4) colorado ( 5 ) america ## tracing information flow in rationale in this section, we aim to verify the hypothesis 1 by tracing the information flow in the rationale generation stage. to this end, we start by describing our attribution tracing method ( §3.1). through this method, we conduct comparative experiments between the correct reasoning and the drifting one, figuring out the mechanism behind the issue ( §3.2). at last, we use the attention score to validate our findings from another perspective ( §3.3). ## tracing information flow in answer in this section, our goal is to verify the information loss based on hypothesis 2. to achieve this, we first introduce the main tracing method in this section, which is called the causal tracing method ( §4.1). next, by employing it, we trace the information flow in the answer generation stage and identify the mechanism behind the answer drift issue through comparative experiments ( §4.2). at last, we apply the attribution tracing method to verify our hypothesis from another perspective ( §4.3). ## mitigating toxic cot problem in this section, we propose a novel method called riders (residual decoding and serial-position swap) to address the toxic cot problem. we first introduce the two components in it, which are designed based on hypothesis 1 and 2, respectively ( §5.1). then, we conduct experiments on commonsense reasoning benchmarks, demonstrating the effectiveness of our method ( §5.2 and §5.3). at last, we conduct extra experiments to further emphasize the contribution of our approach ( §5.4). ## conclusion in this paper, we find a problem named toxic cot, which results in the model's reasoning deviating from the original correct answer when utilizing cot-like prompting. through tracing the internal information flow of the llm with attribution tracing and causal tracing methods, we prove that this problem is mainly caused by the model's lack of information from the question in shallow attention layers when generating rationales or answers. based on this result, we propose the riders method to mitigate the toxic cot problem from both decoding and serial position perspectives. through extensive experiments on multiple commonsense reasoning datasets, we verify the effectiveness of our approach in mitigating toxic cot problems and enhancing the model's overall commonsense reasoning capabilities.