Paper: Logical Languages Accepted by Transformer Encoders with Hard Attention (ICLR-2024)

Reviewer comment (Theory):
On p.6, the paper says, "Observe that $(\bar{w}, i) \models \phi U \psi $ if and only if $(\bar{w}, j_i) \models \psi$." I wonder what happens if there exists $i \leq j^\prime < j_i$ such that $(\bar{w}, j^\prime) \models \psi$"? Following the definition on page 5, I think $(\bar{w}, i) \models \phi U \psi$ holds if there exists such $j^\prime$.

Author response:
Many thanks for your nice comments about the paper! 

Regarding your question: You are right, there is a small problem with the way in which j_i is defined. The proof, however, works verbatim after a slight modification of the definition of j_i. Please see the modified pdf. Thanks for noticing this!

Reviewer comment (Theory):
In section 2.2 paragraph 2, it is unclear why you set \(T(\bar{w})\) to \(\langle \mathbf{t},\mathbf{v}_{0}\rangle\).

Author response:
We define the acceptance of a language using the content of the first position of the last layer, more specifically, we take a linear function (with values in R) of the value in this position, and we require that this function is positive for words in the language, and negative for words not in the language.

Reviewer comment (Writing):
[Minor Comment] Page 6, "reverses the third coordinate" is not a very precise statement.

Author response:
We meant ``multiplies the third coordinate by (-1)’’. We have restated this (see new version of the pdf).

Reviewer comment (Theory):
When is the notion of \(T(\bar{w}) > 0\) (as introduced in section 2.2) used as the criterion in the proof on page 6 and page 7? From this proof, it just appears that you can perform LTL operations on input strings, but it is unclear how this shows that a string in the language will never be mapped to a string outside the language.

Author response:
We show that for any LTL language, there is a transformer encoder that maps strings from the language to 1 and strings not from the language to 0 (when we look at the first position at the last layer). To satisfy our acceptance criterion, it is enough to make an additional affine position-wise transformation x \mapsto x - ½, which maps 1 to ½  and 0 to -½. We should have added this remark and we, of course, will do that.

Reviewer comment (Theory):
Does Kamp's theorem give any bounds on the length of an equivalent FO(Mon) equivalent formula in LTL(Mon)?

Author response:
Translations from FO to LTL are non-elementary (which is unavoidable given known results on the satisfiability problem for such logics over finite words). In general, if the quantifier depth of the FO formula A is k, then an equivalent LTL formula is of size k-th exponential on the size of A.

Reviewer comment (Theory):
Precision of the number processed by the transformer. The paper does not include any restriction on the precision of the numbers processed by the transformer. This could make the model unrealistically expressive as discussed in many related works (e.g., proving Turing completeness of RNNs requires relaxations on the numerical precision). In related works, a realistic assumption could be log-precision transformers, i.e., the number of floating/fixed-point bits scale logarithmically with the sequence length.

Author response:
Firstly, we would like to point out that even infinite precision cannot take UHAT outside AC^0, as shown by Hao et al. (2022).

Our current positional encoding needs polynomially many bits of precision. In fact, we have an alternative construction where instead of cosines in the positional encoding we simply use 2^{-i} and 2^{-2i} in the ith position. For our construction we need an attention that, at position i, is strictly smaller for any j < i than for any j >= i, and moreover, which strictly decrease for j = i, i+1, … An example of such attention is -(2^{-i} - 2^{j})^2 which is a bilinear of form in the positional encoding mentioned above. It seems open how to achieve the same result with logarithmically many bits of precision.

Reviewer comment (Theory):
No assumptions have been made about the number of transformer layers. Prior work usually assumes constant depth or logarithmic depth (with respect to sequence length). Related to this assumption, it seems that the proof of Proposition 2 constructs a Transformer whose number of layers depends on the form of input LTL. This makes it particularly important to make the correct assumption.

Author response:
Our Transformers are of constant depth wrt the sequence length. You are right that the depth of the transformer that represents a given LTL formula phi depends on the structure of phi. We will make this more explicit in the final version of the paper.

Reviewer comment (Theory):
Structure of the model. The model does not include residual connections and only uses single-head attention. Also, the ReLU layer only applies ReLU to a single element of the input vector. Although these might be adaptable in the proof, it would still be preferable to make these assumptions as practical as possible.

Author response:
We could include these features into the model, but note that this would have made our main result – that transformers accept every language in FO(Mon) – only weaker.  This is because we are showing that even without these features transformers can still do that. As for the lower bound (that transformer encoders cannot accept every language in AC^0), it relies on the sensitivity technique of Hahn, which is valid in the case with constant number of attention heads and arbitrary activation and attention functions. In other words, the lower bound is quite robust and does not depend on the technical details of the model.

As for the ReLU – it is true that in practice as activation one uses feed-forward networks, not just application of ReLU to a single element. Still, one might note that in the context of acceptance of formal language it does not matter – both variants just mean that position-wise, we can use arbitrary compositions of affine transformations and max-function.

We agree, however, that it is better to elaborate more on how these assumptions actually look at practice, and we will definitely do that.

Reviewer comment (Novelty):
It would be nice to discuss the relation between this paper and the missed related work mentioned in the weaknesses section.

Author response:
Here we were referring to "logical languages", i.e., languages that are defined by means of logical formulas (which do not always have natural circuit complexity classes counterparts, e.g., our FO(Mon) and LTL(+,C)). At the time of writing, the paper by Chiang et al. (2023) was the only example we were aware of, which we mentioned in Related Work. A new article on logical languages accepted by transformers appeared on arXiv last month (after our submission):

Dana Angluin, David Chiang, Andy Yang. Masked Hard-Attention Transformers and Boolean RASP Recognize Exactly the Star-Free Languages. (https://arxiv.org/abs/2310.13897)

An excellent survey also recently appeared on arXiv after the submission of our paper that discusses logical languages accepted by transformers:

https://arxiv.org/abs/2311.00208

We will discuss these in detail in the final version of our paper.

None of [1]-[3] deal with logical languages, but rather connections of different classes of transformers to circuit complexity, like the earlier papers by Hahn (2020) and Hao et al. (2022), which we already discussed in detail in the introduction. [Note also that [3] appears after the submission of our paper, so we could not possibly cite it.] We have added [1]-[3] in the introduction.

Reviewer comment (Writing):
In Section 1 Introduction, the paper claims that "the expressive power of transformer encoders has not been fully elucidated to date." I am curious about that. What do you mean they are not fully elucidated?

Author response:
We mean there is currently no precise characterization of which languages are accepted by different kinds of transformer encoders with infinite precision. We have rephrased this in the pdf to make it more understandable.

Reviewer comment (Novelty):
I am very confused about the challenges of studying the circuit language. I could not find any information to discuss the existing challenges and related works, which makes it hard to understand the motivation for this work.

Author response:
The question of what can and cannot be expressed by various classes of transformers lies at the heart of the foundation of the area. The works in the past five years (e.g. Hahn (2020) and Hao et al. (2022)) have underlined such an intimate connection between circuit complexity classes and transformer encoders, that results on circuit complexity (e.g. the inability of AC0 circuits to "count", i.e., solve PARITY) have been used to show an inherent limitation of various classes of transformer encoders, prominently Unique Hard Attention Transformers (UHAT) and Average Hard Attention Transformers (AHAT). Despite these, the precise expressivity of transformer encoders belongs to some of the most important open problems in the field, e.g., see Hao et al. (2022), as well as the following new excellent survey that has appeared on arXiv after the submission of our paper:

https://arxiv.org/abs/2311.00208

More precisely, any UHAT (resp. AHAT) transformer was known to be able to be simulated by AC0 (resp. TC0) circuits. Note that AC0 and TC0 are major complexity classes, and after decades of research the expressive power of these circuit classes is understood fairly well. This leads to the open problem of whether any AC0 (resp. TC0) language can also be recognized by UHAT (resp. AHAT). If not, point out which ones, as well as a natural subclasses of AC0 and TC0 languages that can be captured by UHAT and AHAT, respectively.

Reviewer comment (Writing):
What are your contributions to this work? I could not find any conclusions about contributions after reading this section, or even the whole manuscript.

Author response:
In relation to our answer to the previous question, our results have significantly improved our understanding of what UHAT and AHAT transformer classes are (in)capable of. We have provided a concrete "counting language" in AC0 and showed that it cannot be recognized by UHAT, consequently UHAT is a strict subclass of AC0. We have also provided a natural subclass of AC0 that can be captured by UHAT, namely, a logical counterpart L of AC0 but restricted to monadic numerical predicates. We have shown similar results as well for AHAT by providing a natural subclass of TC0 (in terms of an extension of L that naturally incorporates counting capabilities of TC0, in particular the majority function) that can be captured by AHAT. As a corollary of these results, we showed that AHAT has an extremely powerful counting capability (e.g. the language of strings of a's, b's, and c's with the same number of occurrences of these letters).

Reviewer comment (Presentation):
I would suggest that an illustration figure be provided to clearly show the main idea of this work.

Author response:
Thanks. We will think about how to implement this idea in the final version of the paper.

Reviewer comment (Writing):
The last sentence of the second paragraph of section 2.2 makes no sense. f is a function from Sig -> R^d, T: Sig^+ -> R et the last sentence says that T gets an input sequence that types as a sequence of vectors of R"e. The value of T(w) = (t, v0) only depends on t and v0, which is plain weird as it gives the impression it depends on a constant (t) and v0, which is f(a_0) + p(0, n). The whole paragraph is thus buggy.

Author response:
Q: The last sentence of the second paragraph of section 2.2 makes no sense. f is a function from Sig -> R^d, T: Sig^+ -> R et the last sentence says that T get an input sequence that type as a sequence of vectors of R"e ...

A: In our view, the main point of confusion arises from the fact that we are abusing notation and using T to denote both the transformer, which transforms sequences of vectors in R^d to sequences of vectors in R^e, and the function that given a word \bar w outputs a real number based on T. We have modified this and updated the pdf accordingly. Regarding your comment that the value of T(w) only depends on t and v0: please notice that v0 is not f(a_0) + p(0, n), but the first vector in the output of T when presented with the input f(a_0) + p(0, n), …, f(a_n) + p(n,n). We believe there is no mistake in this case.

Reviewer comment (Theory):
About your open question: Additionally, does there exist a language in the circuit complexity class TC0, the extension of AC0 with majority gates, that cannot be recognized by AHATs? I would go for Dyck language. Sounds hard for AHATs.

Author response:
Q: About your open question: Additionally, does there exist a language in the circuit complexity class TC0, the extension of AC0 with majority gates, that cannot be recognized by AHATs: I would go for Dyck language. Sounds hard for a AHATs.

A: Thanks for the suggestion! In fact, [Yao et al. 2021] show that Dyck languages (even with more than one type of brackets) can be accepted by a transformer with soft attention. But they are actually using only uniform weights, which can also be done in AHAT. That being said, they are also using layer normalization. It seems unknown whether Dyck with more than one type of brackets can be done by AHATs without layer normalization.