LookAhead Tuning: Safer Language Models via Partial Answer Previews
Authors: name: Kangwei Liu, name: Mengru Wang, name: Yujie Luo, name: Lin Yuan, name: Mengshu Sun, name: Ningyu Zhang, name: Lei Liang, name: Zhiqiang Zhang, name: Jun Zhou, name: Huajun Chen

Fine-tuning enables large language models (LLMs) to adapt to specific
domains, but often undermines their previously established safety alignment. To
mitigate the degradation of model safety during fine-tuning, we introduce
LookAhead Tuning, which comprises two simple, low-resource, and effective
data-driven methods that modify training data by previewing partial answer
prefixes. Both methods aim to preserve the model's inherent safety mechanisms
by minimizing perturbations to initial token distributions. Comprehensive
experiments demonstrate that LookAhead Tuning effectively maintains model
safety without sacrificing robust performance on downstream tasks. Our findings
position LookAhead Tuning as a reliable and efficient solution for the safe and
effective adaptation of LLMs. Code is released at
https://github.com/zjunlp/LookAheadTuning.