Training Language Models on the Knowledge Graph: Insights on
  Hallucinations and Their Detectability
Authors: name: Jiri Hron; name: Laura Culp; name: Gamaleldin Elsayed; name: Rosanne Liu; name: Ben Adlam; name: Maxwell Bileschi; name: Bernd Bohnet; name: JD Co-Reyes; name: Noah Fiedel; name: C. Daniel Freeman; name: Izzeddin Gur; name: Kathleen Kenealy; name: Jaehoon Lee; name: Peter J. Liu; name: Gaurav Mishra; name: Igor Mordatch; name: Azade Nova; name: Roman Novak; name: Aaron Parisi; name: Jeffrey Pennington; name: Alex Rizkowsky; name: Isabelle Simpson; name: Hanie Sedghi; name: Jascha Sohl-dickstein; name: Kevin Swersky; name: Sharad Vikram; name: Tris Warkentin; name: Lechao Xiao; name: Kelvin Xu; name: Jasper Snoek; name: Simon Kornblith
Published: 2024-08-14 23:34:28

Abstract: While many capabilities of language models (LMs) improve with increased
training budget, the influence of scale on hallucinations is not yet fully
understood. Hallucinations come in many forms, and there is no universally
accepted definition. We thus focus on studying only those hallucinations where
a correct answer appears verbatim in the training set. To fully control the
training data content, we construct a knowledge graph (KG)-based dataset, and
use it to train a set of increasingly large LMs. We find that for a fixed
dataset, larger and longer-trained LMs hallucinate less. However, hallucinating
on leq5% of the training data requires an order of magnitude larger model,
and thus an order of magnitude more compute, than Hoffmann et al. (2022)
reported was optimal. Given this costliness, we study how hallucination
detectors depend on scale. While we see detector size improves performance on
fixed LM's outputs, we find an inverse relationship between the scale of the LM
and the detectability of its hallucinations.