PSALM: Pre-Substitution Symbol-Concept Alignment in Language Modeling
Authors: Fuju Rong, Zheng GONG, Liang Pang, Weihao Gao, Zhuo Deng, chucheng chen, Zhiyuan Niu, FANG LI, Lan Ma


Reviewer paper summaries: Next token prediction in current language models is done by associating hidden representations to symbols (i.e. words of the alphabet) rather than semantic concepts. Since words can be polysemous, this paper claims that this is a limitation because a single word might be related to multiple concepts. To mitigate this the authors proposed a pre-substitution step in which instead of considering a single next symbol several symbols are considered and substituted.
Summary of strengths: Explicitly addressing and understanding how polysemy is handled by current LLMs seems relevant.
Summary of weaknesses: 1- I found the paper hard to follow, the motivation is relatively well established. But I found it hard to understand how the proposed solution mitigates the polysemy limitation. Maybe some concrete examples would help to clarify the proposed approach.

2- The improvements in most tasks seem rather minor. The paper could benefit from a more in-depth analysis of the results.
Comments, suggestions and typos: 1- Some illustrative examples could help to clarify the proposed approach.
Overall assessment scores: 2.5