Before revision: Neural sequence models can generate highly fluent sentences  but recent studies have also shown that they are also prone to hallucinate additional content not supported by the input , which can cause a lack of trust in the model. To better assess the faithfulness of the machine outputs , we propose a new task to predict whether each token in the output sequence is hallucinated conditioned on the source input, and collect new manually annotated evaluation sets for this task.
After revision: Neural sequence models can generate highly fluent sentences but recent studies have also shown that they are also prone to hallucinate additional content not supported by the input . These variety of fluent but wrong outputs are particularly problematic, as it will not be possible for users to tell they are being presented incorrect content. To detect these errors, we propose a new task to predict whether each token in the output sequence is hallucinated conditioned on the source input, and collect new manually annotated evaluation sets for this task.