Hallucination Detox: Sensitive Neuron Dropout (SeND) for Large Language
  Model Training
Authors: name: Shahrad Mohammadzadeh; name: Juan David Guerra; name: Marco Bonizzato; name: Reihaneh Rabbany; name: Golnoosh Farnadi
Published: 2024-10-20 18:18:23

Abstract: As large language models (LLMs) become increasingly deployed across various
industries, concerns regarding their reliability, particularly due to
hallucinations-outputs that are factually inaccurate or irrelevant to user
input-have grown. Our research investigates the relationship between the
training process and the emergence of hallucinations to address a key gap in
existing research that focuses primarily on post hoc detection and mitigation
strategies. Using models from the Pythia suite (70M-12B parameters) and several
hallucination detection metrics, we analyze hallucination trends throughout
training and explore LLM internal dynamics. We introduce SEnsitive Neuron
Dropout (SeND), a novel training protocol designed to mitigate hallucinations
by reducing variance during training. SeND achieves this by deterministically
dropping neurons with significant variability on a dataset, referred to as
Sensitive Neurons. In addition, we develop an unsupervised hallucination
detection metric, Efficient EigenScore (EES), which approximates the
traditional EigenScore in 2x speed. This efficient metric is integrated into
our protocol, allowing SeND to be both computationally scalable and effective
at reducing hallucinations. Our empirical evaluation demonstrates that our
approach improves LLM reliability at test time by up to 40% compared to normal
training while also providing an efficient method to improve factual accuracy
when adapting LLMs to domains such as Wikipedia and Medical datasets.