Paper: The Expressive Power of Transformers with Chain of Thought (ICLR-2024)

Reviewer comment (Writing):
The paper is not very well self-contained and relies on the reader being familiar with two previous papers.

Author response:
1. **Self-Containedness:** We will add more specific details of the papers we build upon where appropriate. We will also provide a reference to a new comprehensive survey paper on formal language results about transformers (https://arxiv.org/abs/2311.00208).

Reviewer comment (Theory):
The last paragraph in the conclusion should be clearly marked as limitation of this theoretical bounds (the lower bounds and upper bounds might not have any relecance in practice).

Author response:
We appreciate your suggestion and plan to add a limitations section if accepted. The most high-priority item to discuss here will be the current caveat paragraph about lower bounds mentioned in the conclusion.

Reviewer comment (Theory):
For the upper bounds: Doesn’t the same reasoning as for the lower bounds apply? For fixed data distributions, complexity classes are irrelevant. How does this mismatch fit into the theory work?

Author response:
The reasoning for lower bounds actually does not apply for the upper bounds. The reasoning for the lower bounds is: just because a transformer can express a solution to a problem, it does not mean that gradient descent will find that solution. In the other direction, if we know that a transformer cannot express a solution to a problem, we know that gradient descent will not find a solution to the problem, because no solution exists. We will clarify this.

Reviewer comment (Experiments):
For the audience of ICLR, it would be interesting to have some results analysing aspects related to learnability of such models.

Author response:
Thanks for your review! We appreciate that you found our theoretical results interesting.

We agree that questions around learnability are interesting but they lie outside the scope of this work. For future work, we are thinking about whether it may be possible to say something about learnability in the PAC learning framework by extending ideas from [this recent paper](https://arxiv.org/pdf/2309.06979.pdf).

Reviewer comment (Writing):
The core idea of using norm-hash is not illustrated well. Using examples to explain the intuition will help in improving the readability of the paper.

Author response:
We will add an example problem like the following the better illustrate the layer norm hash. Potentially, if space allows, we would like to have a figure illustrating it visually.

**Problem:** Imagine we describe the state of a 1D grid world with x’s and o’s and then specify a sequence of +/- moves in that world: `xoooxo+-+-++-`. We then want a transformer to add up the +/- moves and return whether the grid cell is `x` or `o`.

The natural algorithm is to compute the current position with c = #(`+`) - #(`-`) and then attend to that position and see whether it’s x or o. A transformer with causal masking can compute c/n, but it’s messy to use this to retrieve the grid cell at position c because that cell is represented as 1/c. The layer-norm hash lets implement this hard attention cleanly by attending with query phi(c/n, 1/n) = phi_c and key phi(1/c, 1) = phi_c!

We hope this helps make the idea more concrete, and it captures the way we later use the layer-norm hash for automata and Turing machines in a simpler setting.

Reviewer comment (Writing):
The paper assumes a lot in terms of prior knowledge of the reader in the formal language theoretic modeling of transformers.

Author response:
Finally, we would like to address your suggestion to give readers more context for formal language results about transformers. Given the space limitations, we think the best way to provide a gentler introduction is to add a reference to a [new survey paper](https://arxiv.org/abs/2311.00208) on transformers and formal language theory.

Reviewer comment (Writing):
Please explain the intuition behind norm-hash using examples. Otherwise, its introduction appears too abrupt. This is an important concept for the proofs, and hence it should have been explained more clearly with examples.

Author response:
We will add an example problem like the following the better illustrate the layer norm hash. Potentially, if space allows, we would like to have a figure illustrating it visually.

**Problem:** Imagine we describe the state of a 1D grid world with x’s and o’s and then specify a sequence of +/- moves in that world: `xoooxo+-+-++-`. We then want a transformer to add up the +/- moves and return whether the grid cell is `x` or `o`.

The natural algorithm is to compute the current position with c = #(`+`) - #(`-`) and then attend to that position and see whether it’s x or o. A transformer with causal masking can compute c/n, but it’s messy to use this to retrieve the grid cell at position c because that cell is represented as 1/c. The layer-norm hash lets implement this hard attention cleanly by attending with query phi(c/n, 1/n) = phi_c and key phi(1/c, 1) = phi_c!

We hope this helps make the idea more concrete, and it captures the way we later use the layer-norm hash for automata and Turing machines in a simpler setting.

Reviewer comment (Theory):
Why is it necessary to consider a multi-tape Turing machine when showing that a move of a Turing machine can be simulated by one move of a transformer with intermediate steps? Wouldn't showing this for a single tape Turing machine suffice?

Author response:
Regarding the Turing machine simulation, indeed, you’re correct that it would suffice to simulate a single tape to prove Turing completeness.We consider the multitape model because this is the standard model used to define finegrained complexity classes within P (e.g., TIME(n)). The reason for this is only having just a single tape can add polynomial overhead compared to multitape TM. Thus, if we want to conclude TIME(t) \subseteq CoT(t) as in Corollary 2.1, we need to use the multitape model.

Reviewer comment (Presentation):
The result concerning characterization of P is not adequately explained in the paper. For example, there is no subsection devoted to it. It is only mentioned as a bullet on page 3. Please elaborate on this a bit more, like the other results.

Author response:
> The result concerning characterization of P is not adequately explained in the paper. For example, there is no subsection devoted to it.

Thanks for pointing this out. We will add text explaining that this follows from Corollary 2.1 (TIME(t) in CoT(t)) and Theorem 3 (CoT(t) in softTIME(t^2)). Together, these results show poly(n) CoT can express any problem in P and vice versa.

Reviewer comment (Writing):
The paper presumes a lot about prior knowledge of the reader in the formal language theoretic modeling of transformers. A more gentle introduction would help the readability of the paper.

Author response:
Finally, we would like to address your suggestion to give readers more context for formal language results about transformers. Given the space limitations, we think the best way to provide a gentler introduction is to add a reference to a [new survey paper](https://arxiv.org/abs/2311.00208) on transformers and formal language theory.