Attention Heads of Large Language Models: A Survey
Authors: name: Zifan Zheng; name: Yezhaohui Wang; name: Yuxin Huang; name: Shichao Song; name: Bo Tang; name: Feiyu Xiong; name: Zhiyu Li
Published: 2024-09-05 17:59:12

Abstract: Since the advent of ChatGPT, Large Language Models (LLMs) have excelled in
various tasks but remain largely as black-box systems. Consequently, their
development relies heavily on data-driven approaches, limiting performance
enhancement through changes in internal architecture and reasoning pathways. As
a result, many researchers have begun exploring the potential internal
mechanisms of LLMs, aiming to identify the essence of their reasoning
bottlenecks, with most studies focusing on attention heads. Our survey aims to
shed light on the internal reasoning processes of LLMs by concentrating on the
interpretability and underlying mechanisms of attention heads. We first distill
the human thought process into a four-stage framework: Knowledge Recalling,
In-Context Identification, Latent Reasoning, and Expression Preparation. Using
this framework, we systematically review existing research to identify and
categorize the functions of specific attention heads. Furthermore, we summarize
the experimental methodologies used to discover these special heads, dividing
them into two categories: Modeling-Free methods and Modeling-Required methods.
Also, we outline relevant evaluation methods and benchmarks. Finally, we
discuss the limitations of current research and propose several potential
future directions. Our reference list is open-sourced at
https://github.com/IAAR-Shanghai/Awesome-Attention-Heads.