Neural sequence models can generate highly fluent sentences  but recent studies have also shown that they are also prone to hallucinate additional content not supported by the input , which can cause a lack of trust in the model. To better assess the faithfulness of the machine outputs , we propose a new task to predict whether each token in the output sequence is hallucinated conditioned on the source input, and collect new manually annotated evaluation sets for this task. We also introduce a novel method for learning to model hallucination detection, based on pretrained language models fine tuned on synthetic data that includes automatically inserted hallucinations . Experiments on machine translation and abstract text summarization demonstrate the effectiveness of our proposed approach -- we obtain an average F1 of around 60 across all the benchmark datasets. Furthermore, we demonstrate how to use the token-level hallucination labels to define a fine-grained loss over the target sequence in the low-resource machine translation and achieve significant improvements over strong baseline methods. We will release our annotated data and code to support future research.
Neural sequence models can generate highly fluent sentences , but recent studies have also shown that they are also prone to hallucinate additional content not supported by the input . These variety of fluent but wrong outputs are particularly problematic, as it will not be possible for users to tell they are being presented incorrect content. To detect these errors, we propose a task to predict whether each token in the output sequence is hallucinated (not contained in the input) and collect new manually annotated evaluation sets for this task. We also introduce a method for learning to detect hallucinations using pretrained language models fine tuned on synthetic data that includes automatically inserted hallucinations Experiments on machine translation (MT) and abstractive summarization demonstrate that our proposed approach consistently outperforms strong baselines on all benchmark datasets. We further demonstrate how to use the token-level hallucination labels to define a fine-grained loss over the target sequence in low-resource MT and achieve significant improvements over strong baseline methods. We also apply our method to word-level quality estimation for MT and show its effectiveness in both supervised and unsupervised settings. Codes and data available at URL
A
,
R
, which can cause a lack of trust in the model. To better assess the faithfulness of the machine outputs ,
. These variety of fluent but wrong outputs are particularly problematic, as it will not be possible for users to tell they are being presented incorrect content. To detect these errors,
D
new
R
conditioned on the source input,
(not contained in the input)
D
novel
R
model hallucination detection, based on
detect hallucinations using
D
.
R
and abstract text summarization demonstrate the effectiveness of
(MT) and abstractive summarization demonstrate that
R
-- we obtain an average F1 of around 60 across all the
consistently outperforms strong baselines on all
R
Furthermore, we
We further
D
the
R
machine translation
MT
R
will release our annotated data and code to support future research.
also apply our method to word-level quality estimation for MT and show its effectiveness in both supervised and unsupervised settings. Codes and data available at URL