Steering When Necessary: Flexible Steering Large Language Models with
  Backtracking
Authors: name: Jinwei Gan, name: Zifeng Cheng, name: Zhiwei Jiang, name: Cong Wang, name: Yafeng Yin, name: Xiang Luo, name: Yuchen Fu, name: Qing Gu

Large language models (LLMs) have achieved remarkable performance across many
generation tasks. Nevertheless, effectively aligning them with desired
behaviors remains a significant challenge. Activation steering is an effective
and cost-efficient approach that directly modifies the activations of LLMs
during the inference stage, aligning their responses with the desired behaviors
and avoiding the high cost of fine-tuning. Existing methods typically
indiscriminately intervene to all generations or rely solely on the question to
determine intervention, which limits the accurate assessment of the
intervention strength. To this end, we propose the Flexible Activation Steering
with Backtracking (FASB) framework, which dynamically determines both the
necessity and strength of intervention by tracking the internal states of the
LLMs during generation, considering both the question and the generated
content. Since intervening after detecting a deviation from the desired
behavior is often too late, we further propose the backtracking mechanism to
correct the deviated tokens and steer the LLMs toward the desired behavior.
Extensive experiments on the TruthfulQA dataset and six multiple-choice
datasets demonstrate that our method outperforms baselines. Our code will be
released at https://github.com/gjw185/FASB.