Paper: Transformer-Based Large Language Models Are Not General Learners: A Universal Circuit Perspective (ICLR-2024)

Reviewer comment (Experiments):
Claims about instance memorization are very speculative. The authors show that a poly(n)-size transformer can in principle memorize poly(n) instances, but jumping from this to suggesting that instance-based memorization explains T-LLM success in practice is not well justified. Two key issues are:  
1. Experiments with trained LMs would be needed to find evidence of memorization. Empirical evidence shows LMs may memorize n-grams but not full examples (see McCoy et al.).  
2. There are theoretical reasons why T-LLMs may struggle to memorize instances from single examples, due to large batch training, which can obscure instance-specific information. This ties to known lower bounds on learning certain functions (see Barak et al.).  
The claim could be further toned down and, if speculation is retained, supported with related work showing practical and theoretical limitations of instance memorization.

Author response:
Thank you for suggestion. Our current claim is that from the perspective of expressive, it is possible that the model could achieve something by memorization. We agree that it requires more discussion on the learnability and the training dynamics to further investigate whether learning to memorize is feasible and accounts for the success of LLMs (at least for some problems).

Reviewer comment (Theory):
There is an open conceptual question of whether we really want models that can express universal circuits; constraints on expressive power might actually benefit the inductive bias of models. This potential benefit could be discussed in the limitations section.

Author response:
We think that constraints on the expressive power of the hypothesis class may benefit the learning of the model. Some classic results in learning theory  (e.g., VC dimension) have shown that a limited hypothesis class is important for learnability. Despite that these results may not directly apply to the LLMs, we think the idea that a limited hypothesis class  benefits learnability could be insightful. We believe it is an interseting direction for future work to reveal the underlying mechanism (if exists) how the idea works for LLMs.