On the Pathological Path-star Task for Language Models (2024)

Abstract: The recently introduced path-star task is a minimal toy task designed to exemplify limitations to the abilities of language models (Bachmann and Nagarajan, 2024). It involves a path-star graph where multiple arms radiate from a single starting node and each node is unique. Then, given the start node and a specified target node which ends one of the arms, the task is to generate the arm containing that target node. This is straightforward for a human but surprisingly difficult for a language model, which they found failed to predict above chance. They hypothesized this is due to a deficiency in teacherforcing and next-token prediction paradigm. In this extended abstract, we demonstrate that the task is learnable using teacher-forcing in alternative settings and that the issue is (partially) due to representation. We analyze situations when the models fail to solve the task which leads us to introduce a regularization technique where we pack each training batch with multiple instances of the same graph but with differing target nodes to prevent overfitting. Initial results indicate this helps in solving the task.

text: Proceedings of the 9th Workshop on Representation Learning for NLP (RepL4NLP-2024), pages 274–284 August 15, 2024 ©2024 Association for Computational Linguistics
In this extended abstract, we demonstrate that the task is learnable using teacher-forcing in alternative settings and that the issue is (partially) due to representation. We analyze situations when the models fail to solve the task which leads us to introduce a regularization technique where we pack each training batch with multiple instances of the same graph but with differing target nodes to prevent overfitting. Initial results indicate this helps in solving the task.

heading: 1 Introduction
text: Recently, language models (LMs) have become increasingly capable of solving a variety of complex tasks (Brown et al., 2020; Zoph et al., 2022; Bubeck et al., 2023). This has led to increased interest in determining why this is and the limits to these abilities (Chen et al., 2024). Language models can do many spectacular things, which makes it all the more shocking when they fail on simple tasks. Recently, Bachmann and Nagarajan (2024) introduced one such seemingly simple task designed to showcase pathological behaviour of causal (decoder-only) autoregressive (AR) LMs trained via teacher-forcing. The task is simple by design and thus failure of AR models is both surprising and informative. We begin by describing the task in Sec. 1.1, before analyzing why it is hard for LMs in Sec. 2.

heading: 1.1 The Path-star Task
text: We need to describe the path-star graph, G, i.e. the data meant to be manipulated, the problem specification or question, Q, i.e. the prompt specifying the desired manipulation, and their tokenization.
Let N be the set of unique nodes forming G. A path-star graph contains one central starting node s ∈ N and D radial arms each of length M (inclusive of s), s.t. |N | = D(M − 1) + 1. s has degree D, all final nodes which end an arm, F ⊂ N s.t. |F | = D, have a single degree, and all others have a degree of 2. See Fig. 1 for an example.
Given G, and a task specification, Q, containing s and a target node t ∈ F , the task is to generate the unique arm, Rt, as a sequence of nodes starting from s until t. i.e. Rt = sort({ r ∈ N | ∀f∈F dist(r, t) ≤ dist(r, f)}).1 Let L be the set of possible leading nodes which are adjacent to s i.e. L = { l ∈ N |dist(l, s) = 1}. The challenge of the task is predicting the correct leading node lt ∈ L ∩ Rt from all other leading nodes. By design, there is a uniform 1/D chance of this given only G. Prediction over chance should be possible by inferring the correct target arm and thus lt given t in Q. Note, as all nodes are unique, all non-leading nodes are deterministic given their preceding neighbour (closer to s).
When generating the dataset, the nodes in a single graph are uniformly sampled from a set of possible nodes, V , without replacement. G is tokenized as a series of D(M − 1) edges where each edge is internally ordered by distance to s and marked by special token ‘|’ so that a given edge, (u, v), is a three token sequence ‘u v |’. Q is tokenized as a sequence of four tokens with special tokens marking the beginning and end of Q as ‘/ s t =’. Special beginning- and end-of-sequence tokens are also used, making the final vocabulary size |V |+5.
1‘dist’ is graph distance. We abuse notation by treating Rt as a set and G and Q as sequences after having been tokenized.
274

heading: 1.2 Autoregressive models and training
text: A causal or decoder-only AR LM models the joint probability of a T -length sequence, y, as a factorized product of local probabilities, as in
p(y1, y2, . . . , yT | y0, ) = ∏ p(yj | y<j). (1)
Here, we model the path-star task as
p(r1, . . . , rM | [G, Q]) = M∏
j=1
p(rj | [G, Q, r<j ]),
(2) where x = [G, Q, r<j ] is the concatenation of the tokenized graph and problem specification along with the partial ground-truth sequence, r<j , forming the given conditioning input to the model. Such a model is trained by via maximum likelihood training, generally referred to as ‘teacher-forcing’ in the context of language models, as the partial groundtruth sequence is used to condition the model during training instead of the model’s own predictions as done during inference (Williams and Zipser, 1989). We minimize −∑r∈Rt log p( . |x). Thus the loss is only over the target sequence Rt and not on tokens in the prefix [G, Q]. This is because the node ids forming G are random and Q necessarily must be provided, thus both are not predictable and can only be used to condition the model.
During inference, G and Q are provided. We consider a non-traditional ‘teacher-forced’ inference procedure where, instead of generating the arm autoregressively, it is conditioned on r<j . Thus inference exactly matches the training procedure and prevents any potential training-inference bias.
We focus on transformer models (Vaswani et al., 2017), where the causal parameterization of AR models is enforced via an attention mask which
prevents the token at any step j from depending on any token at step > j. This causal restriction applies across the entire input x. Positional embeddings make each token unique. To prevent learning a trivial answer based on position, as a data preprocessing step, the edges in G are shuffled, which can be seen as a random permutation applied to the edge order of tokenization of G.

heading: 1.3 Failure to learn: Clever Hans hypothesis
text: Bachmann and Nagarajan (2024) empirically demonstrated three different LMs – finetuned GPT2, a smaller GPT2 trained from scratch, and a state-space model, Mamaba – all fail to predict above 1/D chance, even in settings as small as D = 2 and M = 5 (Radford et al.; Gu and Dao, 2023). They hypothesized this was caused by teacher-forcing. The idea being that there are two possible modes of predictions which the model can learn. The first is the desired mode which learns to represent the entire path between s to t. This mode is necessary for predicting lt. Whereas, the second mode makes trivial predictions about the next node in the arm given the previous node. This mode only needs to lean superficial information about edge structure but not graph structure and is, by design of the task, sufficient for predicting all non-leading nodes given the correct preceding node.
Bachmann and Nagarajan (2024) argued that teacher forcing will result in learning the second mode, referred to as the Clever Hans cheat (CHC). This is because teacher-forcing conditions on the correct ground-truth, which in this case is the correct preceding node in the arm. Also, when applied to AR models, it is restricted to making a single next-token prediction and hence precludes learning any long term planning. Then, once the CHC is learnt, it will discourage learning the desired mode necessary for predicting lt. Their intuition, which admittedly is not proven, is that, sequence modelling relies on the intermediate training steps across the sequence to form a coherent representation of the overall sequence. In our case, that would be a representation of the entire arm structure, however, here those intermediate steps do not participate in learning such a structure but are rather absorbed into learning the trivial CHC, resulting in a loss of this intermediate training signal.
They presented empirical evidence for the CHC hypothesis by considering the overall sequence accuracy when provided with the correct preceding predictions (i.e. teacher-forced generation). Here,
all non-leading tokens are learnt with 100% accuracy and the leading token is only predicted at 1/D chance, leading to an overall sequence accuracy of 1/D (See their Fig. 3 and our Fig. 3).
Interestingly, a trivial solution to the task exists if the model can look-ahead M tokens to the end of the arm as the model just needs to find and match the correct target token. Once done, it can apply the CHC in reverse order to determine the arm. This led them to provide two additional supporting empirical arguments as to why they believe the issue stems from teacher-forcing. First, they modified the task to require that the arm be generated in reverse order. This makes task trivial as the CHC can just be applied in reverse order via supervision.
Second, they introduced a ‘teacher-less’ model (Monea et al., 2023). This works by using M masked tokens, m, to make make all M predictions in independently of the ground-truths i.e. x = [G, Q, m1, . . . , mM ]. This completely removes teacher-forcing as it removes all dependencies between target-tokens during prediction. Of the 15 reported experiments, this method allows the model to solve the task in 5 instances: for the D = 2 experiment using the small GPT2, and for D ∈ {2, 3, 4} (but not D = 5) when using large GPT2. Thus this method did not work consistently.
Importantly, they establish that, 1), the failure is not due to the amount of training data, 2), that the failure is in-distribution, and 3), that it is not due to any exposure bias or other differences between the training and inference procedures (Bengio et al., 2015; Ranzato et al., 2016; Arora et al., 2022). This allows them to disclude these alternative explanations and conclude that the CHC causes the learning problem which is itself a consequence of teacher-forcing and next-token prediction. This leads to a discussion concerning possible fundamental limitations to the next-token prediction paradigm, with the path-star task being offered as a counter example to the paradigm being sufficient to learn any task.

heading: 2 Methods and Results
text: We solely focus on the small LM setting under the belief that such models should be able to learn such a simple task and that the biases from the pretrained data and any emergent abilities of LLMs will just obfuscate the root problem. We implement our models using Fairseq (Ott et al., 2019). Our AR model have 6 layers and 200 dim. embeddings.
Each layer has a feed-forward dim. of 800 and 8 heads. We use Adam (Kingma and Ba, 2014) with a learning rate of 0.0005, a dropout rate of 0.1 and a weight decay of 0.01. We train with 16-bit precision and a batch size of 1024. Each model is given a maximum of 100 epochs and stop training if the validation loss drops below 0.001 (See Fig 2). Note this is smaller than both GPT2 models used by Bachmann and Nagarajan (2024) which used 36 and 12 layers with larger embeddings.
Following Bachmann and Nagarajan (2024), |V | = 100 and M = 5. Each dataset for D ∈ {2, 3, 4, 5} is made up of 2,000,000 training and 20,000 test samples of randomly generated G, Q pairs without any overlap. Unlike them, we randomly permute G at every epoch instead of just once in an attempt to prevent overfitting.
We present our work as an investigation over a series of hypotheses and corresponding experiments to get at the heart of the path-star mystery. As such, we report intermediate results and describe new methodology as it becomes motivated. We try to present results in order of our findings, however, we need to give some post-hoc explanations for our methodology in order for the reader to understand the contents of Tables (1, 2, 3, 4). First, in our initial experimentation (using D = 2), we found that the models would be able to solve the task seemingly at random (under modified task conditions). This motivated the use of running multiple trials for each experiment under different random seeds. For all listed experiments we consider the percent of trials that correctly succeed in learning the task across 11 trials. Second, we found it was necessary to set attention dropout to zero, which makes sense given the task requires routing node information across positions. Third, we also found that we required learned positional embeddings instead of sinusoidal embeddings. We suspect that the later results in too strong of a positional bias when randomly permuting the edges in G. As an aside, we also found that for the decoder-only model, not using any positional embeddings could also work. This is because positional information will arise out of the asymmetry induced by the causal masking.

heading: 2.1 A reproduction of empirical results
text: As the results of Bachmann and Nagarajan (2024) are surprising, we independently verify them as an initial step. Experiment (exp.) 1 of Table 1, confirms that the task is not learnable under the initial conditions. Exp. 2 confirms that reversing
the arm results in a trivial 100% success rate.

heading: 2.2 Simplifying the task
text: Having confirmed the results in the original task setting, our method to investigate the issue will be to simplify the task until it becomes consistently solvable. We begin by considering the target-side.

heading: 2.2.1 Evidence against the CHC hypothesis
text: Under one interpretation of the CHC hypothesis, lt is indecipherable due to the model being overwhelmed by the CHC. As such, we should be able to learn a simplified version of the task where we only predict lt directly instead of the entire arm Rt. Exp. 3 of Table 1, shows this produces the same negative result as when predicting the entire arm.
The more charitable interpretation of the CHC hypothesis is that the core issue concerns the fact that the CHC removes necessary intermediate training signal for learning the task. Under this interpretation, we have not disproved the core hypothesis, but have shown that the entire CHC aspect is irrelevant to the underlying issue. That is, if lack of intermediate supervision is the core issue, we should just explicitly remove it from the task description and cut out the red-herring of the CHC.
However, we do not believe that lack of intermediate supervision is the real issue and take this result as a sign that something else is at play. To solve this task, all the model needs to do is 1) determine the final node, 2) trace back each arm from the final node to its leading node, and 3) predict that leading node. Importantly, this requires that we can correctly represent the arms in the graph. This motivates us to experiment with simplifying the source-side of the task instead of the target-side.

heading: 2.2.2 Alternative hypothesis: representation issues due to the permutation of G
text: Our first hypothesis as to what is preventing learning the solution is that it is a representation issue due to randomly permuting the edges of G. This will corrupt the arm structure with the model seemingly unable to recover the structure. In particular, when using a causal model, all information can only be routed forward in ‘time’ and this may induce difficulties when trying to recover and route information across the arm structure. Not only does permutation make routing information across the arms harder, or even impossible, but the difficulty in learning might also be due to the assumptions we make when we decompose the joint probability as in Eq. 1. Specifically, we are parameterizing the model to a specific decomposition (Yang et al., 2019; Liao et al., 2020). However, by permuting the arms, we are forcing the model to learn an exponential number of possible decompositions. This may be a challenge, even when using an overparameterized model like a transformer and may explain the difficulty of the task.
Thus we can simplify the task where we retain the arm structure by only permuting the order of the arms relative to each other (but not the internal order of the edges). Refer to this change as Edgev.s. Arm-wise permutation. If this is solvable, then we know that the issue lies in the corruption of the arm information via permuting the edges. Exp. 4 of Table 1 shows this improves the results, with D = 2 being consistently solved, but with a diminishing success-rate as D increases. These partial improvements lead us to a related hypothesis.

heading: 2.2.3 Alternative hypothesis: representation issues due to the order of G and Q
text: If we can only route information into the future, maybe our representation issue stems from that fact that we have placed the problem specification after the graph during tokenanization. That is, we have placed the information needed to specify what to do with the data after the actual data. This means that the latent representation of G formed by the LM can not depend on Q. Thus instead we form our input as x = [Q, G, r<j ]. Refer to this as Q’s position being either Start v.s End. Exp. 5 of Table 1 demonstrates that this consistently solves the task when combined with permuting the arms only, but goes back to being completely unsolved when combined with permuting the edges (Exp. 6). While this shows that the task is solvable, it is unsatisfying as we require stronger supervisory information in this setting. This also begs the question as to why placing Q after G is at times solvable, even if we understand why it makes the task harder.
As the causal constraint of decoder-only models potentially induces these issues, we are motivated to change the model specification to see if abandoning this constraint will solve the task.

heading: 2.3 Changing the model parameterization

heading: 2.3.1 Encoder-decoder model, or, alternative hypothesis: it’s the causal constraint
text: Here we use encoder-decoder model with a 6-layer encoder with a 3-layer decoder with tied embeddings. Removing the causal constraint on the source-side encoding of [G, Q] makes the relative position of Q to G irrelevant. If this model can consistently solve the task, it will demonstrate that the underlying issue is that the causal constraint prevents the decoder-only model from recovering the arm structure with edge-wise permutation.
Exp. 9 in Table 2 demonstrates that using a noncausal encoder representation does not solve the problem with edge-wise permutation. This motivates us to revisit the ‘teacher-less’ methodology as it has been shown to partially work and is an alternative non-causal methodology.

heading: 2.3.2 Non-autoregressive models
text: Bachmann and Nagarajan (2024) reported that ‘teacher-less’ models where unable to solve the task in the small LM setting. Here we attempt to improve their results. We begin by modify their their ‘teacher-less’ model as it was designed to modify an LM post-hoc, which is not applicable here (Monea
et al., 2023). Instead, note this is actually just a kind of non-autoregressive model (NAR) (Gu et al., 2018; Wang et al., 2018; Gu and Kong, 2021).
NAR models treat all targets as independent in order to make multiple predictions in parallel instead of sequentially. This is achieved by removing the causal constraint i.e. attention mask. In the case of (fully) NAR models, full independence is assumed. However, this can lead a poor model as it limits the ability to learn from dependencies inherent in the sequence (Lee et al., 2018; Qian et al., 2021). This lead to the development of iterative autoregressive models (IAR)2 which assume partial dependencies, both during training and inference – except in the first generation step (Lee et al., 2018; Ghazvininejad et al., 2019). Importantly, IAR models assume no order-of-generation. This allows for the model to potentially learn the reverse order solution without supervision.
To train an IAR model, an order-permutation of the sequence is sampled, along with a time-step, j such that model conditions on the permuted or ‘unmasked’ ground-truths prior to step j. This is equivalent to the MLM objective with a dynamically sampled masking rate, where the the uniform masking is acting as the permutation (Devlin et al., 2019; Lee et al., 2018; Ghazvininejad et al., 2019). Thus use teacher-forcing, but it is just applied to a permuted sequence order. We use CMLM (Ghazvininejad et al., 2019) for an encoder-decoder IAR model and an encoder-only model using the same hyper-parameters as the decoder-only model (i.e same model but without causal masking).
We evaluated these models both in the NAR and IAR generation setting using either 1 or M iterative steps (results not reported). Both settings produced the same results. That is, once, the model learnt the solution, it could generate the entire arm in one step just as well as over M steps. In principle, the IAR models should be able to first generate t in the last position, condition on it, and then just generate the arm in reverse order via CHC – which should be much easier than learning the true solution. This did not happen. It should be disconcerting for practitioners of IAR models that the trivial generation order does not seem to be found.
We demonstrate that small IAR models are capable of learning the task in Tables 3 and 4. Exp. 11 and 14 may indicate that IAR models are not as performative on the task, however, this is not true
2Often called iterative NAR models, which is a misnomer.
once contrastive samples are used (Sec 2.4).

heading: 2.4 Contrastive samples
text: Our observations of the failures of the above models lead us to conclude that, in the instances where the model failed to solve the task, the model would overfit. See the top graph of Fig 2 with D = 2. Here all trials end up successfully learning the task, which can be seen when the validation accuracy branches off from chance. However, the last example nearly overfits. In the third graph, we see the same experimental setting but with D = 4. Here only a single trial succeeded and the rest maintained a stagnant validation accuracy at chance while the training and validation losses diverge.
To prevent this, we experimented with standard methods to combat overfitting such as lowering learning rate, increasing batch size or L2 regularization, etc. without success. This lead to developing an alternative method where we supplement the training data with multiple instances of G but with different target nodes, and hence, different Q and target arms to be generated. This was achieved via expanding each batch with one or more of these contrastive samples per original G. These extra instances should act as interference on any spurious training signal. Note, apart from sampling, these are treated as independent and are not part of a contrastive loss. This can be viewed as extra supervisory information applied at the batch-level.
Results of exp. 7, 13, and 16 show that this prevents overfitting and leads to improved success rate across models. The arm-wise decoder-only exp. 7 shows improved rates at D ∈ {4, 5}. This can be seen in the first and third plots in Fig 2 in contrast to their corresponding second and forth plots where we see the validation loss tracks the training loss instead of diverging when provided
with contrastive samples. This lead to learning the solutions in less epochs in the D = 2 case and lead to 10/11 instead of 1/11 of the trials succeeding in the D = 4 case.

heading: 3 Limitations and Future work
text: The major limitation to this extended abstract is that the experiments with contrastive samples are in progress. We included partial and preliminary results of this method in order to show the future direction of this work. Despite this, we have already demonstrated a number of interesting findings: 1) We independently reproduced the empirical results of Bachmann and Nagarajan (2024). 2) We show that the CHC is a red-herring, if the problem is due to loss of intermediate training signal. 3) We show that retaining the arm structure leads to improved results and that combining this with a better ordering of Q and G makes the task solvable. We take this as strong evidence that poor representation is a key factor in why the task is difficult. 4) We show that NAR/IAR models can successfully solve the task in limited settings, and describe a surprising failing to these models. Finally, 5), we show that overfitting is a key issue and provide initial positive results in tackling this.
We set out the hypothesis that trials where the training and validation losses do not diverge will succeed given sufficient training time. However, that has yet to be shown. It is an open question if contrastive samples will always lead to finding the solution given sufficient time or if this scales with D and M . More importantly, it is an open question as to why the models prefer to overfit instead of finding the correct solution. On lead we have is that the task hinges on a single token which determines the necessary latent representation of G, and this seems to have large consequences on the behaviour
In plot 1, all trials succeed, however, when D is increased to 4, only 1/11 trials succeed as shown in plot 3. Here we see that the training and validation losses diverge shortly after epoch 20, resulting in overfitting. In plot 4, the use of contrastive samples prevents this divergence, leading to 10/11 trials succeeding, with the remaining trial not finding the solution within the 100 epoch limit.
of the learning algorithm and the difficulty of learning a correct solution. This indicates that the issue may stem from the sensitivity of the task to the target token (Hahn et al., 2021; Chen et al., 2023; Chakraborty et al., 2023; Bhattamishra et al., 2023; Hahn and Rofin, 2024).
A large and important part of the scientific process is testing hypotheses and putting forth counter arguments. If, as we believe, the CHC hypothesis is incorrect, then the broader discussion of Bachmann and Nagarajan (2024) concerning limitations of next-token prediction is not be supported by their findings. This is the main contribution of this work, even if, we do not have a full solution or replacement hypothesis. Even if the CHC hypothesis is wrong, the path-star task is still a seemingly trivial but deceptively difficult problem, making it worthy of study. In addition to questioning the CHC hypothesis, we make headway into the mystery of the path-start task by demonstrating multiple simplifications or alternatives of the task which make it (more) solvable. Finally, we introduce a contrastive method which has early indications of helping to solve the task, however, open questions remain as to why this method works and why it is necessary in the first place.

heading: 3.1 Post-submission findings
text: Between submission and acceptance, new findings have come to light, which we summarize here.
1) Constrastive samples are necessary but not sufficient to solve the task consistently. We find that, as each example is randomly sampled, there will be |V |D(M−1)+1 × D possible graph-target pairs to sample from. Thus it is easy to see why the models would overfit as they can learn to make many possible spurious correlations between any unique node or combination of nodes in the sourceside and the targets. Contrastive samples will alleviate these spurious correlations by indicating that the targets depend on a single token only. Above, we wondered if contrastive samples will always lead to finding the solution given sufficient time. We found this is not true. However, we also con-
duct an analysis of the task using RASP (described below), which lead to a counter-intuitive result that, even though overfitting is an issue, we need to increase the size of the models to solve the task.
2) The RASP programming language is a formal computation model used to verify if a transformer is capable of solving a given symbolic (nonnumerical) task where the existence of a valid RASP program proves there exists at least one transformer which can (Weiss et al., 2021; Zhou et al., 2024). We conduct a RASP analysis which lead us to find to many new insights to the task. We can formally prove, via the existence of RASP programs, that the task is solvable via transformers both using non-causal and causal models. Due to the graph structure, we find that the simplest RASP programs which solve the task with edgewise permutation require O(M) number of layers in order to route information about leading nodes to final nodes (or vise versa). We show a O(logM) algorithm exists, but conjecture than no O(1) exists. This would mean that the task will not be generalizable to higher values of M . However, the problem is easily solvable with a O(1) algorithm when using arm-wise permutation, which explains why the task is solvable under the simpler conditions demonstrated above. We also find that using a causal decoder makes the task harder as it can only route information into the future, which requires different rules depending on if the connecting edge is before or after a current edge when routing.
3) Given the RASP analysis we increased the number of layers of each model. This lead to worse in overfitting in all models, except the encoder-only model, which then is able to consistently solve the task. Both this model and the CMLM (enocderencoder) model employ the IAR training method. The fact that it is just the encoder-only model which works indicates that the reason why it works is not due to the training method. It is an open mystery why it is only this model which can consistently solve the task.
Please look for an updated pre-print (available soon) or contact arvie@cs.toronto.edu.

heading: Acknowledgments
text: We thank our reviewers as well as Siavash Kazemian, Gagandeep Singh, and Shunan Zhao for their insightful feedback.
Resources used in preparing this research were provided, in part, by the Province of Ontario, the Government of Canada through CIFAR, and companies sponsoring the Vector Institute

heading: A The Clever Hans Phenomenon
text: In Fig 3 we show how the CHC appears during training. Here we see that the first token to fit to 100% accuracy is the given start node, s. The next token is the given target node, t. While this might seem strange as it is generated at the end of the sequence, this token is actually easily predicable since the model can infer that the target token should always be placed in the M th position. This is because there is no requirement that predictions be generalizable to different arm lengths and hence the target token is always in the M th position. This is explicitly done to maintain that the test data is in-domain with the training data. Next we see that all other non-leading nodes fit via the CHC. As no trials succeeded in this experiment, the validation accuracy of the leading token becomes stagnant at chance, while the training accuracy improves.

heading: Future Work
text: We also find that using a causal decoder makes the task harder as it can only route information into the future, which requires different rules depending on if the connecting edge is before or after a current edge when routing. 3) Given the RASP analysis we increased the number of layers of each model. This lead to worse in overfitting in all models, except the encoder-only model, which then is able to consistently solve the task. Both this model and the CMLM (enocderencoder) model employ the IAR training method. The fact that it is just the encoder-only model which works indicates that the reason why it works is not due to the training method. It is an open mystery why it is only this model which can consistently solve the task. Please look for an updated pre-print (available soon) or contact arvie@cs. toronto. edu