Attention Satisfies: A Constraint-Satisfaction Lens on Factual Errors of
  Language Models
Authors: name: Mert Yuksekgonul; name: Varun Chandrasekaran; name: Erik Jones; name: Suriya Gunasekar; name: Ranjita Naik; name: Hamid Palangi; name: Ece Kamar; name: Besmira Nushi
Published: 2023-09-26 17:48:55

Abstract: We investigate the internal behavior of Transformer-based Large Language
Models (LLMs) when they generate factually incorrect text. We propose modeling
factual queries as Constraint Satisfaction Problems and use this framework to
investigate how the model interacts internally with factual constraints.
Specifically, we discover a strong positive relation between the model's
attention to constraint tokens and the factual accuracy of its responses. In
our curated suite of 11 datasets with over 40,000 prompts, we study the task of
predicting factual errors with the Llama-2 family across all scales (7B, 13B,
70B). We propose SAT Probe, a method probing self-attention patterns, that can
predict constraint satisfaction and factual errors, and allows early error
identification. The approach and findings demonstrate how using the mechanistic
understanding of factuality in LLMs can enhance reliability.