Towards Uncovering How Large Language Models Work: An Interpretability Perspective
Authors: Haiyan Zhao, Fan Yang, Bo Shen, Ali Payani, Himabindu Lakkaraju, Mengnan Du


Reviewer paper summaries: The paper is a survey on interpretability approaches that investigate the internal mechanisms of language models (LMs). Specifically, it reviews interpretability research addressing the following questions: 
1. How is knowledge encoded in LMs? 
2. What knowledge is encoded in the activations of LMs? 
3. How is generalization ability achieved during training? 
4. The applications of the insights gained from studying (1), (2), and (3)?
Summary of strengths: The paper provides a comprehensive overview of the studies being conducted to understand the internal mechanisms of language models (LMs), spanning from mechanistic interpretability to representation engineering. It is well-organized around four clear research questions, helping readers grasp the overall motivation behind the discussed topics.
Summary of weaknesses: **Unclear writing**: 
1. The paper doesn’t define important concepts such as features, neurons, and circuits (section 2.1) before using them to describe other topics. 
2. There is an inconsistent description of section 3 in the Abstract: "Then, we summarize how knowledge is embedded in LLM representations by leveraging probing techniques and representation engineering" (lines 14-17) and "What Knowledge is Encoded in Intermediate Representations?" (lines 319-320). 

**Possibly incorrect writing**: 
1. Monosemanticity is presented as the underlying cause of polysemanticity alongside superposition in line 117, which is not correct. 
2. The phrase "dictionary learning where features are pre-defined" is unclear. 
 
**Missing discussions**: 
1. The discussion on other important components of LMs such as feed-forward layers (FFN), layer normalization, embedding layer, unembedding layer, etc. 
2. A large number of papers on interpreting circuits and attention heads have not been covered in the survey. 
3. Other phase changes observed in training dynamics besides grokking, such as the phase change discovered by Olsson et al., 2022. 
4. The discussion on LM generation steering as the application of the interpretability insights.
Comments, suggestions and typos: Including a formal description of the transformer architecture in the background section would make the paper more self-contained.
Overall assessment scores: 2.5