Fine-Tuning Lowers Safety and Disrupts Evaluation Consistency (2025)

Abstract: Fine-tuning a general-purpose large language model (LLM) for a specific domain or task has become a routine procedure for ordinary users. However, fine-tuning is known to remove the safety alignment features of the model, even when the fine-tuning data does not contain any harmful content. We consider this to be a critical failure mode of LLMs due to the widespread uptake of fine-tuning, combined with the benign nature of the “attack”. Most well-intentioned developers are likely unaware that they are deploying an LLM with reduced safety. On the other hand, this known vulnerability can be easily exploited by malicious actors intending to bypass safety guardrails. To make any meaningful progress in mitigating this issue, we first need reliable and reproducible safety evaluations. In this work, we investigate how robust a safety benchmark is to trivial variations in the experimental procedure, and the stochastic nature of LLMs. Our initial experiments expose surprising variance in the results of the safety evaluation, even when seemingly inconsequential changes are made to the fine-tuning setup. Our observations have serious implications for how researchers in this field should report results to enable meaningful comparisons in the future.

## introduction recent work has reported the concerning phenomenon that fine-tuning a large language model (llm) for a specific task can significantly impact the safety guardrails on the base model -even in cases where the fine-tuning dataset does not contain any harmful content @xcite . since fine-tuning is a popular and accessible way to leverage general-purpose llms for specialized use cases, understanding and mitigating this safety risk is crucial. however, as we began work in this direction, we were confronted with a more fundamental problem: how do we compare the safety of two models in a meaningful way? given the stochastic nature of llms, are safety measurements repeatable? are they reproducible across minor inconsequential variations in the fine-tuning process (e.g., different random seeds)? are the conclusions similar whether the fine-tuning proceeds for one epoch, or two, or more? a clear understanding of how different parameters impact the final safety evaluation is necessary before being able to assess the usefulness of any proposed mitigation. in this paper, we present the results of our initial experiments, comprising two base models, two finetuning datasets, and a total of 150 fine-tuned model checkpoints. we investigate: ## discussion our initial experiments suggest several methodological considerations for work moving forward. ## conclusion to make progress on understanding the security vulnerabilities of llms, we need to take a rigorous and principled approach to safety evaluation. in this study, we have examined the effect of factors which might be reasonably deemed inconsequential, such as repeated measurements on the same model, random seed during fine-tuning, the specific content of two identically-sized generaldomain instruction-tuning datasets, or the number of epochs used for fine-tuning. in some cases, the variance can be quite substantial, implying that finetuning not only degrades safety, but also disrupts evaluation consistency. unreliable measurements can make it difficult to interpret whether a difference between two safety evaluations is actually meaningful. we therefore emphasize the importance of reporting all training and generation parameters and making repeated measurements whenever possible, to advance our collective scientific understanding of llm behaviour.