Paper: Inductive Transformers: How Large Language Models Form Concepts, And How to Make Them Even Better At It (ICLR-2024)

Reviewer comment (Presentation):
The paper is severely lacking clarity. Most of the details needed to understand the model are hidden in the appendix. It's not at all clear how this model works without digging through the appendix, and even then it takes a long time to figure out how the different components come together. This is especially puzzling given that the authors have so much extra space left in the main paper. The introduction, I feel, can be condensed a lot and the content in the supplement should be moved into the main paper such that readers of the main paper should understand how this architecture is implemented.

Author response:
The paper is severely lacking clarity. Most of the details needed to understand the model are hidden in the appendix. Its not at all clear how this model works without digging through the appendix, and even then it takes a long time to figure out how the different components come together. 

--> We have now completed a lot of edits to hopefully make the paper more accessible. We agree that it would have been nice to be able to include (a) a high-level figure for the production and statistical model without all of the factor graph details, (b) a comparison table with more space to explain the comparison of each layer type (c ) the factored probability distribution for vanilla and for inductive transformers. (The factored probability distributions for the vanilla and for inductive transformers are actually identical to one another, so it would be the same equation twice.) (4) inline much of the appendices.  In the archival version, where we are afforded more space, we will make these changes.

This is especially puzzling given that the authors have so much extra space left in the main paper. 
--> We checked into this and confirmed that in ICLR we are limited to 9 pages for the main body of the paper.  We are right up against the limit.

The introduction, I feel, can be condensed a lot 
-->  We actually assembled a draft exactly like this, and to be honest it really did not seem to help.  Bringing in slivers of appendices into the main body seemed to be both too little additional explanation and also too much detail, both at the same time.  Even if we eliminated the introduction almost entirely, it would not afford enough space for all but a sliver of the appendices.  

and the content in the supplement should be moved into the main paper such that readers of the main paper should understand how this architecture is implemented.

--> We completely agree! The content in the supplement should be moved into the main paper. In an archival version where space allows, we will do that.  Unfortunately, ICLR severely limits the available space to a strict limit of 9 pages.  That said, we do feel that the reader who hopefully gains from the main body a high-level appreciation of the story, can open the appendices side by side with the main body and refer back and forth fairly efficiently.  In other words, all of the supporting material is, at the end of the day, contained in the submission. 

--> Since the initial submission, we have also done a tremendous amount of further editing on both the main body and the appendices  in order to try to make them more understandable and to tie it all together more seamlessly.

Reviewer comment (Experiments):
The model is only implemented on a very simple toy dataset (two word sentences). It would be better to see how this model can be used on a larger dataset that is more on the level of what transformers are trained on.

Author response:
The model is only implemented on a very simple toy dataset (two word sentences). It would be better to see how this model can be used on a larger dataset that is more on the level of what transformers are trained on.

--> Two thoughts here:

(1) We agree that a key next step will be to scale this up! That said, we see it as future work. We intend this first paper to be analogous to theoretical physicists proposing a new mathematical model with simulations that show that the math works. Future work then typically involves a larger team of experimentalists. We have now joined with an experimental team to do just that, but we would humbly submit that this paper represents a publishable contribution in terms of new ideas, novel mathematical derivations, and code that implements the model and runs.

(2) Scaling laws for large language models are critical for the field. That said, models scale up, but they do not scale down. There is a minimum data set size below which all previous transformer architectures do not learn anything. Ours does. We now state explicitly in the text that "We design a transformer that successfully learns even as the data set size scales to zero."

Reviewer comment (Novelty):
A lot of components of the model seem highly similar to clone structured cognitive graphs (George et al. 2021). What are the similarities/differences to their work?

Author response:
Dear Reviewer,

Regarding your question "A lot of components of the model seem highly similar to clone structured cognitive graphs (George et al. 2021). What are the similarities/differences to their work?"

Thank you for pointing out this interesting read. There are indeed several similarities between this paper and ours.
In particular, clone-structured cognitive graphs (CSCGs) are focused on learning higher-order relational representations from sequence data. Similarly, we aim to organize higher-level abstract concepts in inductive transformers. 
There are also several notable differences:
1. The training methodology differs significantly - we introduce custom activation functions, connectivity patterns, and synthetic training data. 
2. Inductive transformers have direct ties to the underlying transformer architecture. CSCGs seem more standalone, or in this paper more closely tied to biology.
3. Inductive transformers connect model form to recursive marginalization in transformers. CSCGs have a domain-specific probabilistic graph formulation.
4. This leads to very different mathematical groundings for the two approaches.

Let us know if there is any point you'd like us to further clarify.

Thank you again for your feedback!

Reviewer comment (Theory):
It's very hard to understand the equivalence between the proposed generative model and the vanilla transformer.

Author response:
We completed a lot of edits to hopefully make the paper more accessible   We agree that it would have been nice to be able to include (a) a high-level figure for the production and statistical model without all of the factor graph details, (b) a comparison table with more space to explain the comparison of each layer type (c ) the factored probability distribution for vanilla and for inductive transformers.  (The factored probability distributions for the vanilla and for inductive transformers are actually identical to one another, so it would be the same equation twice.)  In the archival version, where we are afforded more space, we can include these.

Reviewer comment (Writing):
Can you provide more explanations on the "concept" meaning in both the generative model and vanilla transformer.

Author response:
In response to your comment, we added this explanation: "We define a ``concept'' as a sub-network that can generate many different but synonymous token sequences (e.g. ``tiny canine'')."

Reviewer comment (Theory):
What's the time complexity of the propose generative model's training/inference process?

Author response:
In response to your comment, we added this explanation: "The inductive transfomer is a more focused version of the vanilla transformer, and will therefore generalize similarly.  The time and space complexity is identical."

Reviewer comment (Experiments):
Is it possible to apply the proposed generative model on large scale dataset?

Author response:
Two thoughts here:

(1) We agree that a key next step will be to scale this up!  That said, we see it as future work. We intend this first paper to be analogous to theoretical physicists proposing a new mathematical model with simulations that show that the math works. Future work then typically involves a larger team of experimentalists. We have now joined with an experimental team to do just that, but we would humbly submit that this paper represents a publishable contribution in terms of new ideas, novel mathematical derivations, and code that implements the model and runs.

(2) Scaling laws for large language models are critical for the field. That said, models scale up, but they do not scale down. There is a minimum data set size below which all previous transformer architectures do not learn anything. Ours does. We now state explicitly in the text that "We design a transformer that successfully learns even as the data set size scales to zero."

Reviewer comment (Writing):
The paper does a poor job of explaining its ideas. While the introduction is reasonably accessible, I struggled to understand the probabilistic model, the production system, and the need for Bernoulli/categorical variables. I feel like this could have been framed much more clearly using terminology / frameworks that are more common in the machine learning community. For example, this could have been framed in terms of probabilistic graphical models, or graph grammars, etc.

Author response:
--> We have now done a tremendous amount of edits and revisions to hopefully make the paper more accessible.

--> We actually explicitly tried to avoid this.  Although it might make the paper more accessible to a specific sub-field with the particular vocabulary, many in deep learning are not familiar with probabilistic graphical models, belief propagation, graph grammars, etc.  We have been trying to restrict ourselves to only using simple probability theory and self-contained explanations.

Reviewer comment (Experiments):
The empirical results were unconvincing. The training set was too simplistic, and the simple lesion done does not confirm the hypothesis of structured / compositional knowledge.

Author response:
--> Three thoughts here:

(1) We also agree that a key step will be to scale this up! We see that as future work. We intend this first paper to be analogous to theoretical physicists proposing a new mathematical model with simulations that show that the math works. Future work then typically involves a larger team of experimentalists. We have now joined with an experimental team to do just that, but we would humbly submit that this paper represents a publishable contribution in terms of new ideas, novel mathematical derivations, and code that implements the model and runs.

(2) Scaling laws for large language models are critical for the field. That said, models scale up, but they do not scale down. There is a minimum data set size below which all previous transformer architectures do not learn anything. Ours does. We now state explicitly in the text that "We design a transformer that successfully learns even as the data set size scales to zero."

(3) This paper essentially shows how to efficiently learn a PCFG using belief propagation as inference and back-prop for learning.  For the first time PCFG structure learning does not require using inside-outside, E-M, and all that.  We do it with tensors in PyTorch.  The literature on grammar learning at smaller scales and less scalable algorithms, however, does demonstrate that learned PCFGs are structured and compositional, and we cite, for example, Goodman et. al. who have demonstrated machine learning of compositional and structural PCFGs to grade student programming assignments.  Space did not allow for a full treatment of grammar learning literature.   For the deep learning reader, we show the startling conclusion that the vanilla transformer actually IS a grammar learner.  Seen from that perspective, this work in some sense unifies PCFG learning and transformers into a single mathematical framework.

Reviewer comment (Experiments):
While the paper provides an illustrative example to demonstrate the inductive transformer's potential, it lacks empirical evaluation with large-scale experiments and/or diverse datasets, which reduces its impact in my opinion. A more comprehensive evaluation would provide a clearer understanding of its capabilities and limitations.

Author response:
Two thoughts here:

Scaling laws for large language models are critical for the field.  That said, models scale up, but they do not scale down.  There is a minimum data set size below which all previous transformer architectures do not learn anything.  Ours does.  We now state explicitly in the text that "We design a transformer that successfully learns even as the data set size scales to zero."

We also agree that a key step will be to scale this up!  We see that as future work.   We intend this first paper to be analogous to theoretical physicists proposing a new mathematical model with simulations that show that the math works.  Future work then typically involves a larger team of experimentalists.  We have now joined with an experimental team to do just that, but we would humbly submit that this paper represents a publishable contribution in terms of new ideas, novel mathematical derivations, and code that implements the model and runs.

Reviewer comment (Evaluation):
The introduction of additional inductive biases could potentially result in an increased model complexity and training cost. It would be helpful to discuss the trade-offs between model performance and computational resources, as well as practical scalability issues that may arise.

Author response:
Inductive bias always reduces the size of the overall search space.  It can, however, make the search space more difficult to traverse which can be costly in compute.  That is not the case here.  As we show by reproducing the training run 100 times, the convergence is efficient and repeatable.  it does not, for example, get stuck in local minima.