Analysing Chain of Thought Dynamics: Active Guidance or Unfaithful
  Post-hoc Rationalisation?
Authors: name: Samuel Lewis-Lim, name: Xingwei Tan, name: Zhixue Zhao, name: Nikolaos Aletras

Recent work has demonstrated that Chain-of-Thought (CoT) often yields limited
gains for soft-reasoning problems such as analytical and commonsense reasoning.
CoT can also be unfaithful to a model's actual reasoning. We investigate the
dynamics and faithfulness of CoT in soft-reasoning tasks across
instruction-tuned, reasoning and reasoning-distilled models. Our findings
reveal differences in how these models rely on CoT, and show that CoT influence
and faithfulness are not always aligned.