Rationales Are Not Silver Bullets: Measuring the Impact of Rationales on Model Performance and Reliability (2025)

Abstract: Training language models with rationales augmentation has been shown to be beneficial in many existing works. In this paper, we identify that such a prevailing view does not hold consistently. We conduct comprehensive investigations to thoroughly inspect the impact of rationales on model performance as well as a novel perspective of model reliability. The results lead to several key findings that add new insights upon existing understandings: 1) Rationales can, at times, deteriorate model performance; 2) Rationales can, at times, improve model reliability, even outperforming their untrained counterparts; 3) A linear correspondence exists in between the performance and reliability improvements, while both are driven by the intrinsic difficulty of the task. These findings provide informative regulations on the broad utilization of rationales and raise critical implications on the procedure of explicitly aligning language models with implicit human thoughts. Codes can be found in this anonymous link: https://anonymous.4open.science/r/rationales-CEE8.

## introduction it is widely acknowledged that the capabilities of large language models can be significantly enhanced when they are given time to think, a process where a rationale is generated first to mimic human inner thought before arriving at the final answer @xcite . although this concept was initially identified and established in the context of prompting large language models (llms), it has also been extensively explored in training language models (lms) as well. in general, rationales have profoundly influenced our understanding and utilization of lms. the answer is 624 pages. the answer is 2 pages. james writes a 3-page letter to each of his 2 friends. for each friend, he writes 3 pages, that's 3 pages × 2 @xmath0. he does this twice a week. so, in a week, he writes 6 pages × 2 = 12 pages. there are 52 weeks in a year. so, in a year, he writes 12 pages × 52 @xmath1. ## impacts of rationales on performance and reliability we record the accuracy and ece difference of models finetuned with and without rationales as follows: note that we use a negative increase of ece as lower ece means better calibration. ## analyze impacts of rationales with difficulty it is notable that improvements are more significant for intuitively harder tasks, e.g. arc-challenge compared to arc-easy. thus we assume the impacts of rationales on performance and calibration are both related to the inherent difficulty of tasks, which can explain the linear correspondence. however, correctly defining the intrinsic difficulty of a task is non-trivial. in this section, we explore three alternative metrics to approach a reasonable characterization of task difficulty as follows. ## robustness of the impacts brought by rationales as model performance and calibration can be affected by several factors, we conduct ablations to verify the robustness of our conclusions. ## explanatory analysis in this section we try to explain the impacts observed above. we first exclude the formality of raft and the extra computation as potential causes through a blank rationale experiment and then conduct a qualitative analysis to identify situations where rationales cause performance drop. furthermore, despite the difficulty of fully explaining our conclusions from the perspective of inner mechanisms of raft, we provide some hypotheses and a discussion in the appendix a.1. ## related work rationales are free-text reasoning steps produced by human or language models. much attention has been paid to leveraging rationales to enhance model performance @xcite @xcite . recently llms have been capable of generating reasoning steps through chain-of-thought @xcite , which inspires a bunch of works studying rationale-augmented finetuning (raft) using llms @xcite @xcite @xcite . these works focus more on proposing new methods of extracting and utilizing rationales on certain target tasks. there are also works studying the effectiveness of rationales @xcite @xcite . yet none of the above has studied the effect of rationales widely on different tasks nor do they study the effect of rationales on calibration (while @xcite introduces similar idea to ours, they mainly focuses on zero-shot setting, and do not study the behavior of calibration). our study works in this untraveled direction. confidence calibration is first proposed for determining how a weather forecaster is reliable @xcite . research on confidence calibration of statistical machine learning methods has a long history @xcite , and later calibration of neural networks is also researched on @xcite @xcite . @xcite points out that finetuned neural networks are over-confident, which is potentially caused by over-minimizing the loss. recently, calibration of language models also received attention. @xcite study the calibration of pretrained language models and point out that they are well calibrated. similarly, finetuning has been found to harm the calibration of lms @xcite @xcite . while previous works have studied the calibration of pretrained and finetuned language models, we expand the calibration measurement to the rationale-augmented finetuning setting. ## conclusion in this work, we systematically examine the impact of rationale-augmented finetuning (raft) on model performance and reliability, and bring out several key findings that add new insights to current understandings. it sometimes deteriorates model performance, while alleviating calibration error caused by finetuning over-fitting. we also identify a significant linear correlation between the impacts on performance and reliability, both are driven by the intrinsic difficulty of the task. with exploratory analysis and discussions, this paper implies future directions to continually delve into the underlying mechanism of rationales, pursuing better alignment between the explicit reasoning process of auto-regressive language models and implicit human thought structure.