Meta-learning from demonstrations improves compositional generalization
Authors: Sam Spilsbury
Alexander Ilin
Venue: Submitted to ICLR 2023 (2023)
Keywords: meta-learning
grounded language learning
compositional generalization

Abstract: We study the problem of compositional generalization of language-instructed agents in gSCAN. gSCAN is a popular benchmark which requires an agent to generalize to instructions containing novel combinations of words, which are not seen in the training data. We propose to improve the agent’s generalization capabilities with an architecture inspired by the Meta-Sequence-to-Sequence learning approach (Lake, 2019). The agent receives as a context a few examples of pairs of instructions and action trajectories in a given instance of the environment (a support set) and it is tasked to predict an action sequence for a query instruction for the same environment instance. The context is generated by an oracle and the instructions come from the same distribution as seen in the training data. In each training episode, we also shuffle the indices of the actions and the words of the instructions to make the agent figure out the relations between the actions and the words from the context. Our predictive model has the standard transformer architecture. We show that the proposed architecture can significantly improve the generalization capabilities of the agent on one of the most difficult gSCAN splits: the ``adverb-to-verb” split H.

Under review as a conference paper at ICLR 2023
FROM DEMONSTRATIONS IMPROVES COMPOSITIONAL GENERALIZATION
Anonymous authors Paper under double-bling review
ABSTRACT
We study the problem of compositional generalization of language-instructed instructed agents in gSCAN. gSCAN is a popular benchmark which requires an agent generalize to instructions containing novel combinations of words, which are not seen in the training data. We propose to improve the agent's generalization capabilities with an architecture inspired by the Meta- www---uuuenceto-Seueeeee 2019). The agent receives as a context a few examples of pairs of instructions and action trajectories in a given instance of the environment (a support set) and it is tasked to predict an action sequence for a query instruction for the same environment instance. The context is generated by an oracle and the instructions come from the same distribution as seen in the training data. In each training episode, we also shuffle the indices of the actions and the words of the instructions to make the agent figure out the relations between the actions and the words from the context. Our predictive model has the standard Transformer architecture. We show that the proposed architecture can significantly improve the generalization capabilities of the agent on one of the most difficult gSCAN splits: the "adverb-to- verb" Split H.
learning approach (Lake)
INTRODUCTION
We want autonomous agents to have the same compositional understanding of language that humans do (Chomsky [957 Tenenbaum 2018). Without this understanding, the sample complexity required to train them for wide range of compositions of instructions would be very high (Sodhani et al. 2021 Jang et al. 202T). Naturally, such compositional generalization has received interest from both the language and reinforcement learning communities. "Comppsitional Generalization" can be divided into several different sub-skills, for example being able to reason about object properties compositionally (Chaplot et al. 2018 Qiu et al. 2021), composing sub-instructions into sequence (Logeswaran et al. 2022] Min et al. (021) or generating novel action sequences according to novel instructions made up of familiar components (Lake & Baroni 2018). In this work, examine this latter challenge in more detail using the gSCAN environment (Ruis et al.) 2020).
gSCAN is testing environment for manguage-grounder agents, consisting of 6-by-6 grid-world where each episode has a unique combination of objects, an initial agent position (in this work, the state) and some language instruction. The objects are one of circle, cylinder or triangle, can be five different sizes and come in the colors red, blue, purple and yellow. We encode each cell in the state is encoded bag of words, similar BabyAl AI (Chevalier-Boisver) et al., 2019). The components are the object size, color, shape, and possible agent occupancy and direction. Available instruction action words are push, pull and walk to and available instruction adverbs are while spinn ing, whi le z zigzaggi ng, hes tant ly and caut i ous ly. Actions which the agent must produce are the set of WALK, STAY, LTURN, RTURN, PUSH and PULL. In this work, when we refer to an action that is repeated more than once, place the repeat count in parenthesis, like LTURN (4) A success happens when the agent exactly matches the target actions for an episode.
The instructions follow template of "actioon a size? color? object adverb". where indicates that a token is optional. Certain combinations of instructions, actions, objects and adverbs not found in the training set. They instead found in certain held out "Splits". Split A is an in in-distribution validation set split, containing instructions, target objects and target locations which can be found in the training set. Splits B F are nut-of-distributiin sets where the target object has

---

Under review as a conference paper at ICLR 2023
an unseen description made up of combinations of familiar terms (for example red square in Split C) or is at a location not seen before during training (for example, southwest of the agent in Split D). These test splits, except Split D, have been solved by using a Transformer (Qiu et al. 2021). Split G is a "meta-learning" split, where an example of "cautiously" is seen only & times during training. Split H, otherwise known the "adverb-to-verh" split, contains only instructions following the template "pull a size? color? ob jece while spinning". This requires the agent to walk towards the target object and pull it the required number of times, while at the same time performing actions LTURN (4) after each WALK and PULL. Solving the problem requires the agent to generate the unseen action trajectory based on a compositional understanding of the instructions. In this work, we mainly focus on Split H.
We hypothesize that a promising approach is Meta Siguence-to-Sequenee Learning (meta-seq2seqq) proposed by Lake (2019). We think the reason why this approach works well is the permutations applied to the supports and target instructions, which ensures that an agent does not overfit to particular sequences of symbols in the output space and instead forces meta-learning to determine what the true output actions should be for a given episode. Extending this approach to language grounding environments was flagged possible future work direction by Ruis et al. (2020). In this work we propose to do exactly that.
Our contributions are: first, we describe an extension of meta-seq2seq with state-relevant supports, second we report promising success rate performance on gSCAN Split H, third we demonstrate the importance of generating relevant supports and show how different procedures for generating supports affect performance, and fourth we motivate how this approach aligns with intuitions about human compositional problem solving.
RELATED WORK
Compositional Generalization There is long line of work on the challenge of compositional
generalization in deep learning. Initial works show that sequence models such as RNNs cannot solve these problems well [Tenenbaum 2018: Loula al.) 2018)). Datasets such as SCAN (Lake Baroni 2018). Ogendata (Geiger et al. 2019). COGS (Kim & Linzen 2020), and PCFG (Hupkes et al.
2020) serve as benchmarks to measure progress. Various approaches have been proposed to improve compositional generalization performance, including data augmentation (Andreas 2020) [hi et al.)
2021 Guo et al. 2020al Qiu et al. 2022). problem-specific inductive biases (Russin et al. 2020
Guo et 2002b Yin et al. 202 1| C Chakravarthy et al. 2022 Spilsbury & Ilin 2022). increased data diversity (Patel et [2222][Andred
2020). transfer learning (huu et 2001 ) and modular networks (Andreas 2016 Ruis & Lake 2022). These approaches can perform very well, but usually
require prior assumptions about underlying data. In computer vision and multimodal domains, the Transformer architecture has been shown to solve some compositional generalization tasks (Vaswani eet al. 2017 |Hudson& & Zitnick 2021: Chen et al. (2020). Transformer's ' success on token-level tasks also promising but still limited (Power al. 2022]] Bhattamishra al.) 2020 Qiu et al. 1021
2022 Sikarwar et al.) (022). Meta-learning (Conklin et 2021 Yang et al. 2022] Mitchell et al. 2021 and group equivariance (Gordon al. 2020) have also shown promise on such problems.
Grounded Environments Many language grounding environments exist, such as BabyAI
(Chevalier- Boisverrt et al., 2019). ALFRED (Shridhar et al. 2020). VizDoom (Chaplot et al. 2018)
and SILG (Zhog al. 2021). gSCAN and its derivatives (Ruis et al.) 2020: Wu al.
2021
specifically focus on task compositional generalization in an interactive world. The various splits of
gSCAN are still not completely solved. Various approaches including graph networks (Gao et al.
attention (Kuo et al.. 2021). symbolic reasoning Nye et al. (202)) auxiliary tasks Jiang& & Bansal (202T) Transformers QQiu et al., (02T), modular networks (Heinze-Deml &
2020), linguistic-assiste
Bouchacouri, 20200 Ruis & Lake 2022). data augmentation (Setzler et al. 2022 Ruis & Lake 2022
have been proposed. Splits D, G and remain challenging to solve with a general approach.
In-context and model-based learning We take inspiration from a long line of work on in-context meta-learning, starting with RL2 (Duan et al. 2016) for RNNs and the extension to Transformers with TrML (Melo) 2022). Also related are the ideas of retrieval for in-context learning (Goyal et al.)
2022: Borgeaud et al. (022)) and proposing goals and planning in an imagination of the world (Nair
al. Sh18-Chane-SSe Sane et al. 2021: Hafner et al. 2020 Deac et al. 2021)
2

---

Under review as a conference paper at ICLR 2023
3 METHOD
3.1
MOTIVATION
Many existing solutions this problem build autoregressive model of the generated target sequences. Our hypothesis is that those models overfit to particular distribution of actions seen in the training data.
The nature of the problem can be seen in the one-step-ahead action conditional probability distri- butions in Table [T While these one-step-ahead conditional probabilities do not capture the full picture, since there are other variables such as other previous actions as well as the instruction, they do give snapshot of the nature of the distribution shift problem Split H. In the training split, both P(L.URN/PULLL) = 0 and P(PULL| [LTURN) = O. On the contrary, and unlike the other the test splits of gSCAN (see for example the "red squares" split C), P(PULL P(LTURN(PULL) 0 in Split H.
and
Table 1: One-step ahead conditional distributions in various gSCAN splits for targets. Shown is P(a a3-1) where a₂ is down the rows and a--1 is along the columns. Bolded are conditional probabilities notably different from the training data.
(a) Training data
(b) Test Split A
PUSH
FULL
0.52
0.33
0
0.15
PULL
0.52
0.32
0.16
PUSH
0.27
0.26
0.26
0
0.21
POSH
0
0.28
0.25
0.27
0.21
0.19
0.07
LIURN
0.01
0.01
0.73
0
0.24
1203H
0
0.02
0.02
0.73
0.23
RIURN
0
0.18
0.14
0
0.68
8TURN
0
0.20
0.13
0.67
0.20
0.35
0.17
0.28
0
0.19
0.33
0.19
0.29
(c) Test Split
(d) Test Split H
PULL
PUSH
S:AY
17URN
STURN
WALK
DULL
PUSH
STAY
LTURN
RTURN
MALK
FUSH
0.00
0.23
0.26
0.26
0.00
0.25
PUSH
0
STAY
0.20
0.04
0.00
0.00
0.00
0.76
STAY
0
0
0
0
STURN
0.00
0.00
0.18
0.14
0.00
0.69
RCUSH
0
1.00
0
MALK
0.00
0.00
0.20
0.34
0.17
0.28
MALK
0
0 0.89
0.11
0
This poses a problem for any model conditioned on past actions, because giving any probability mass to token where P(az (4=1)) = 0 in the training data will result in some error according to the whole calls for some level of
cnossyyntropy loss function. At the same time, the gSCAN problem timestep-dependen behaviour, so modelling P(a: (ot-)) is required. We believe that any approach which performs well on these two splits will need handle this problem in some way.
3.2 META- POTTA-SEQUENCE-TT TO-NNNCCE-OO-SEUUUUUEESE TO-SQQUENCE LEARNING
Meta Sequence-to-Scquuee
learning (meta-seq2scg Lake 2019h) is type of meta-learning introduced
to handle certain classes of sequence-to-sequee compostional generalization problems like the one described in Section 3.1 In few-shot meta-learning problem context, the objective is to use some examples of a particular class of problem called supports in order to estimate the target for query. Better known examples of this framework include MAML Finn al. (211), RL2 Finn DDunn et al. 2016) and TrML (Melo 2022). The supports consist of suppori instructions .... query instruction 1
and corresponding support targets A1. A...... An (in this case the actions). is given and the model must predict the corresponding query targets I The critical part of the framework is that the token-symbol mapping for both the supports and query inputs and targets re- re-permuted on every episode. A consistent permutation is applied for both the queries and supports. Typically in mequence-modelling problems, a tokenizer or dictionary used to map words or tokens to integer symbols. The model operates at the level of symbols, then the user typically translates the output back into tokens. The main idea behind meta-seq2seq is learn a model that can handle permutations in the token-symbol mapping by providing supports of how given permutation of symbols are used to solve other related problems. The permutations make the task unsolvable without the supports, forcing the model to make use of them and also not overfit particular
3

---

Under review as a conference paper at ICLR 2023
1
In
A
a1. am
Permuter
Permuter
S
T
S
targets
A1
TE
Permuter
V1 K
Kn
Attention
TD
V
An
TE
S
T
Permuter
I
SOS, 01 ... 0m-1
Figure 1: A schematic showing our proposed extension of meta-seq2seq Lake 2019). For a given episode with the initial state S. oracle function generates state-relevant in-distribution support instructions sequences I,,-., In. which contain parts of the query instruction, but not the query itself. The action generator, which could be an oracle function an existing in-distribution model, generates the corresponding support target sequences AI, A, for those instructions in the environment. Permuter blocks shuffle the token-symbol indices for both the instructions and targets on each episode. S is encoded and each of !1.... In are decoded using a Transformer Encoder-Decoder (T), summarized into vectors in the same way using a Transformer Encoder (without S). 1 and S are encoded using the same T, except that the decoded 7 sequence taken as the output as opposed to the decoded [CLS] token. Attention between each decoded symbol in 1 in and each decoded support instruction vector /1, .... L provides mixing weights for each of the encoded support target vectors in A1, .... An, resulting in output sequence the same length I A Transformer Decoder (TD) takes right-shifted causally masked ground truth permuted targets as input and the Attention block output context and predicts the permuted targets. At testing time, no permutation is applied.
vector by using [CL.]] token in the decoder sequence. A A,,... A,, are encoded
T pull green big cylinder while spinning
AQ = LTURN (41 WALK LTURN LTURN KALK LTURN NALK STURN (4)
NALK LTURN (1) PULL LIURN (4) PULL LTURN(4) PULL LTURN (1) PULL
1 walk to a green big cylinder while spinning
A = LTURN (4) MALK LIURN (4) LTURN WALK (40 KALK LTUUR(4) NALK
L2 push green big cylinder while spinning
Ag = LTURN (4) WALK (4) LTURN WALK LTJRN (4) WALK LTURN( KALK
LTU3N
2TURS
I3 = pull green big cylinder while zigzagging
A3 I.TURN
PULL PULL PULL
44 = pull green big cylinder hesitantly
A4 RAIX STAY LTURN WALK STAY NALK STAY NALK STAY PULL STAY PULL
STAY PULL STAY FULL STAY
Is = pull green big cylinder
Aş WALK LTURN NALK MALK WALK PULL
PULL
Figure 2: Example of the context generated by the oracle for episode of training. Above: State S. query instruction 1 and target action sequence AR, Below: Support instructions I,,.., Is and action sequences A,,... As.
4

---

Under review as a conference paper at ICLR 2023
symbol sequences. In the meta-seq2seq framework, the permutations are applied using a Permuter block during the episode generation process (see Appendix [E] for examples of permutations). At testing time, permutations are applied. The meta-seq2seq architecture encodes each I,,... I and each query symbol in 7 computes attention between the two, then uses attention between 1 and I,,...,,, I, to mix the representations for each encoded support target A1,..., A,, into sequence of the same length as 19, which is the input sequence to a decoder used to predict the query targets.
Meta-seq2seq differs from other contemporary approaches compositional generalization in lan- guage such as "Good Enough Compositional Augmentation" (GECA, Andreas (2000). GECA is an augmentation method which recogmizes template fragments in text, then realizes those templates with other possible substitutions. Following the example in that work, if a dataset contains "she picks the wug up in Fresno" and "she puts the wug down in Tempe", , then the augmentation method generates samples of puts down substituted into sentences containing picks up. For example the sentence "Pat picks cats up" can be augmented to "Pat puts cats down". :. GECA relies on being able to identify templates containing discontiguous fragments which contain at least two tokens. In the case of JUMP" from the concatenated instruction-acto action pair "jump thrice JUMP JUMP JUMP" and substitute it into "walk around right thrice WALK RTURN WALK RTURN WALK RTURN" such that it is augmented into "jump around right thrice JUMP RTURN JUMP RTURN JUMP RTURN". As noted by Andreas (2020), the time and space complexity of GECA can be quite large and scales with the number of recognized templates and fragments. In contrast to GECA, meta-seq2seq does not rely on finding templates within the dataset to generate augmentations. Instead, the input and target word-symbol mappings are permuted on every episode, which generates a much wider diversity of data.
SCAN, GECA might identify the fragment "jump ... JUMP
JUMP
The meta-seq2seq architecture is attractive, first because of its strong performance on the sequence generation tasks requiring compositional generalization in SCAN, but more importantly because of its non coun-ommin-sppciine does not require any domain specific information for generating new sentences. was shown in Lake (2019) that the compositional generalization can be achieved if the context tasks resemble the query task.
architecture. It only requires permutation of token-symbol mappings and
However, it has until now remained open question how such approach could be applied to state-conditional problems (Ruis et al.) 2020). The main challenge is that both meta-seq2seq rely on retrieval from the training data for either their supports or fragments used to generate additional training data. As observed by Mitchell et al. (022)) this is problematic because the supports need to be relevant to the task at hand and they will not be relevant at the point where there is a mismatch between the support state and the query state. We hypothesize that the supports have to be from the same state, which means that they should be generated as such.
The architecture we use in this work extends on Lake (2019) in two ways. The first is general mod- ernization of the architecture, replacing RNN encoders and decoders with Transformer counterparts. which we do to make our work more comparable to recent works on gSCAN (Qiu et al. 2021 Sikar war et al., 2022). The second, and more critical extension, is to generate supports in the same state for each meta- training episode. In this work, we do that through the use of an oracle function (example shown in Figure which generates support token instructions and actions by replacing verbs and adverbs with other valid combinations seen in the training data, then generating the corresponding support targets for the new instruction in that environment using the generation procedure provided by Ruis et al. (2020). Support instructions which are in Split H are never generated by the oracle. It is possible that a query with the same symbols for pull... ... while spinning is generated after permutation during training, however the probability of this happening is low. We measured that for a single pass through the training data, approximately 3% of permuted query instructions matched pull ... whi le spinning, 0.3% of the permuted query targets matched PULL actions followed by four LTTURN instructions, and their intersection was 0.001 % of the data.
We believe that this architecture is also simple and intuitive to understand through the lens of how humans might think about compositional problems. When faced with an unfamiliar instruction (19), the agent thinks of similar instructions (L,, ...,,,4. In) and their solutions (Ai, A..) in the same environment state. The agent then thinks about how those solutions can be composed in light of the current instruction.
5

---

Under review as a conference paper at ICLR 2023
4 EXPERIMENTS
We ran experiments to determine the performance of our approach. The Transformer blocks use an embedding size (dmodel) of 128 units and fully-connected layer size (dFF) of 512 units is used. We use 16 layers for each of the state/instruction Transformer (T), 8 layers for the action supports Transformer Encoder (TE) and and layers for the Transformer Decoder (TD), about 13.2M parameters. The learning rate is 10-5, we have an effective batch size of 4096, and training iteration count of 30,000. During training, dropout is not used and weight decay is set to 10-3 with the AdamW optimizer. Beta values are left at their defaults, 81 0.9 and B2 0.999. Learning rate warmup is used up to step 5000 to peak learning rate of 10-5, then decayed on a log-linear schedule from steps 5000 to 50000 to 10 Gradient norms are clipped at 0.2 to improve training stability. We use 16-bit precision during training and make use of gradient accumulation in order to simulate large batch sizes where memory is limited.
Runs are over 10 seeds and the bottom 3 seeds are excluded by ranking maximum performance on the in-distribution Split A. Since the other splits are in-principle unseen, show performance on splits B-H by taking a checkpoint for each seed at its best Split A performance, then measuring performance on all the other splits that checkpoint, shown in Ours(o, A). For completeness we also show the performance on all splits on a checkpoint at the best Split H performance at Ours(o).
Table 2:
Our approach compared to other recent works on the gSCAN dataset. Numbers mean success rate over seeds standard deviation. Additional comparisons, including with the original baseline (Ruis et al. 2020), GECA (Andreas) 2020), LCGN (Gao et al. 2020) and Recursive Decoding (Setzler et al., (022) can be found Appendix B Not that splits B-F are not directly comparable due to the way that generate supports, see the later analysis in this work.
ViLBERT
Modular Ruis Lake 2022
Role-guided
Transformer
Ours(o, A)
Ours(o)
Qiu et al. 2021
Kuo et al. 2021
Ours
Ours
Ours
#purams
3M
13.2M
13,2M
13.2M
99.95 0.02
%6.4 0.28
96.73 + 0.58
1.0 0.0
0.96 0.0
0.95 1 0.01
99.90 0.06
59.66 23.76
94.91 # 1.30
0.91 0.15
0.96 0.01
0.96 0.01
99.25 0.91
32.09 9.79
67.72 10.83 10.83
0.86 0.19
0.97
0.97 = 0.01
D
0.0 + 0.0
0.0 a: 0.0
11.52 à: 8.18
0.79 0.0 0.0
0.35 + 0.06
0.35 : 0.05
E
99.02 # 1.16
49.34 # 11.60
76.83 # 2.32
0.98 = 0.01
0.97 = 0.01
99.98 0.01
94.16 = 1.25
98.67 + 0.05
1.0 + 0.0
0.97 0.01
0.97 + 0.01
H
22.16 0.01
76.84 + 26.94
20.98 + 1.38
0.08 + 0.09
0.81 + 0.03
0.86 + 0.02
We include three recent related works for comparison. ViLBERT QQu et al. 2021 is general architecture and currently the state-of-the-art on many of the gSCAN splits. It uses a convolutional encoder for the states, cross-attention with Transformer-like model between instructions and states, then uses this as context for Transformer decoder to generate actions autoregressively.
"Modular" is recent work by Ruis & Lake 2022). It uses a specialized decomposition into Perception, Interaction, Navigation and Transformation Modules, each of which are trained independently with their own training targets, then connected together test time. The modular decomposition gives a prior on how the problem should be solved (for example by decomposition into egocentric and allocentric plans). The work also describes how data augmentation can be used to improve the model, but we show the results coming from use of the modular architecture alone. This approach can get good performance on Splits and H. Performance on other splits is either slightly improved or comparable to the baseline in Ruis et al. (2020), which is likely due to the use of a similar underlying architecture of RNNs and CNNs as feature encoders.
"Role-guided" Kuo et al. 2021 uses linguistic priors to decompose the parsing problem and specify how sub-parsers are connected. can achieve some level of performance on Split and comparable performance on Split H to the Transformer.
"Transformer" a baseline Transformer model, different and more vanilla than the ViLBERT in Qiu et al.) (2021). The Transformer follows an encoder-decoder structure, where the state and instruction are jointly encoded and the actions are decoded with causal masking. The Transformer uses 28 layers on both the encoder and the decoder, in order match the parameter count of the meta-seq2seg model. At the same parameter budget and with similar training hyperparameters as meta-seq2seq. is still not able solve Split H.

---

Under review as a conference paper at ICLR 2023
The results are shown in Table 2 We achieve very strong performance on Split H. comparable to the recent work Ruis & Lake (2022). We provide an analysis of the remaining failure cases in Section 4.2
There are also a few observations to be made about the results that we report. The first is that the results Splits B to F. the results are not exactly comparable, because a path the goal object is indirectly shown through the supports. For example in split C, the challenge might be to "walk" to the unseen "red square" and the supports show series of actions which satisfy "push" the "red square" or "walk" to the "red square while spinning". Its important to note that the indirect leak does not reveal the true solution, only components which could be composed together into the real solution. The leak also does not happen during training; only during inference on the test set. In practice, we show in Section 3 that the indirect leak does not mean much for practical applicability since the support actions can be generated by Transformer trained only on the training set. In that light, retained high performance on these splits is impressive, since the model can perform a form of few-shot compositional meta-learning based on generated support targets. The second observation is that we do not consider Split G (the "few-shot-learning split) our work. Initial experiments showed that our approach does not solve this task when the number of related examples is small. This might seem surprising for a meta- leeta-lernnig architecture, but we believe the reason for this is that Split G requires RTURN, LTURN(3) 3) RTURN after each step, and for any two target actions indices (") and a!2), it is not likely that you will see the sequence (2)) in the training data. There is good discussion by Ruis & Lake (2022) about a data- augmentation method which can help to solve this split. The final observation is that while the results reported by Ruis et al. (2020) when using GECA are reproduced in Appendix B the comparison may not be fair one, since the underlying baseline architecture is quite different (RNNs as opposed to Transformers) and GECA was only applied to the instructions and state and not to the target commands like we apply permutations for meta-seq2seq.
4.1 RELAXING ASSUMPTIONS ON THE ORACLE
The assumption that oracle is able generate perfect examples of related instructions and actions in the same environment state is a strong one, so we also provide some analysis of how extending meta-seq2seq in this way performs when these assumptions are relaxed. We also want to check if the oracle behaviour that we rely on is important for the agent's 's ability to perform well on Split H. Success rates for a similar experimental setup are shown in Table
No Permutations When we remove the Permuter blocks in our architecture, performance on Splits D and H drop to become are comparable to a Transformer. This confirms our hypothesis about overfitting to particular sequences of symbols and also a similar result (Lake 2019).
Distractor supports Ideally we also want to be robust irrelevant distractor supports if we are to replace the oracle in the future with a generative model that generates superset of the relevant instructions. Irrelevant distractor supports are those which refer non-target object, or to an object which does not exist. We perform the same meta-learning experiment, but this time the oracle generates three additional distractor instructions. In this case, performance is slightly worse, but this mainly due to higher variance in convergence rate between seeds.
Irrelevant Instructions When adjusting the oracle to only generate irrelevant supports, perfor- mance on all splits drops significantly. This is consistent with the analysis of Mitchell et al. (202T)
We also tested generating relevant instructions and making demonstrations on different state to the query state by retrieving a state for the query instruction from the training data. The states are included as part of the support inputs. Performance in this case also drops.
Retrieval
Support Targets generated by Transformer (Transformer Actions) Since our ultimate aim is to replace the oracle with a generative model in future work, we also tested scenario the action supports come from a baseline Transformer which receives the current state and the support instructions generated by the oracle function. In this case the Transformer was trained the gSCAN training set and had a 95% autoregressive generation success rate on Split A. In this scenario we no longer doing "few show compositional meta-learning" for Splits A-F, but instead zero-shot
7

---

Under review as a conference paper at ICLR 2023
Table 3: Different types of oracle behaviour. Numbers are success rates standard deviation over top 3 of 10 seeds by Split A performance, Ablations measured at 30,000 iterations at the best validation checkpoint on Split A.
Ours(o, A)
No permutations
Transformer Actions
Distractors
Inrelevant Instructions
Retrieval
0.96 0.0
0.97 = 0.0
0.96 0.0
0,92 0.05
0.27 0.01
0.28 0.02
0.96 0.01
0.98 0.0
0.9
0.94 # 0.04
0.27 = 0.02
0.2 0.03
0.97 0.01
0,98 0.0
0.97
0,93 # 0.04
0.27 0.0
0.01 + 0.0
D
0.35 0.06
0.03
0.0
0.25
0.02 0.02
0.0
E
0.98
0.9
0.97
0,94 0.06
0.28 0.02
0.04 + 0.02
0
0.90
0.97
0,92 0.06
0
0.37 # 0.02
H
0.81 4.81
0.07
0.03
0.74 0.09
0.0
0.08 + 0.02
generalization. We can still get decent performance all splits except Split D. This is because the Transformer cannot predict any support sequence to reach an object south-west of the agent, whereas in for other splits, the Transformer can predict path to the previously unseen object in the context of the instruction. Notably, we match the performance of the macleegeneeated actions on Split H.
4.2
FAILURE CASE ANALYSIS
Vodel
15:0
Mede
meta seq2aez
pes2bve tranoformer
2500
tranaformer
3000
1200
2500
1000
2000
3500
600
500
200
(a) Edit-distance frequency distribution
(b) 2 Pulls
Model
175
Vide mato, Staibe toanffommer
transformee mer
2500
125
2000
120
2300
20
30
Edlt Disianic
Edit Detance
(c) 2-7 Pulls
(d) 8-15 Pulls
Figure 3: Failure case analysis. In (a), we show the edit distance frequency distribution over all failures. In (b), (c) and (d), we show the edit distance distribution as well as edit distance as function of the number PULL instructions in the target sequence. Models that generalize poorly will have a larger edit distance for more complex target instructions
We investigated whether the remaining failures on Split H were due our approach being unable to solve the fundamental generalization problem in Split H or due to some other issue. To diagnose this, we examined edit distances between ground truth sequences in Split H and sequences generated by Ours(o) and the Transformer when run without teacher forcing on the failed examples. If the edit distance length scales with the number of PULL and LTURN (4) combinations required, then it means that the model is likely either generating one of the two instructions, but not both in an interleaved sequence. For example, might generate actions corresponding to the neighbours 'push while spinning' 'pull hesitantly'. In practice we found that the edit distances for Ours(o) follow power law, whereas for a Transformer they scale with the instruction complexity, shown in Figure In the majority of cases, only small number to 3) errors made throughout the whole sequence,
8

---

Under review as a conference paper at ICLR 2023
and only in very small number of cases do make large number of emror.. The edit distance does not scale with the complexity of the target instruction. This indicates that Ours(o) correctly generalizes to the expected behaviour, but makes some other error, different from the Transformer which does not generalize to the expected behaviour and has a edit distance distribution that scales with the target complexity.
Over the same remaining failure cases, we observed four types of failure:
Did not turn (78.22% %) The agent "missed" a turn instruction when generating an instruction path that requires one, for example, because the target object is not in the same the agent. In this case, WALK WALK generated as opposed to LTURN WALK or RTURN WALK.
Spurious Pull (33.1 %) The agent generates a PULL instruction where it should not generate one.
Missed Pull (8.09%) The agent does not generate a POLL instruction where should generate onc.
Other reason (0.058%) The failure is complex multi-facete than be attributed to the above reasons.
Note that multiple failures can happen in single example, so percentages do not add to 100.
The majority of failures can be attributed to "Did not turn". To compute whether the agent is erroneously picking WALK but is uncertain, we compute the mean entropy of the prediction logits in all cases where there failure turn. A high amount of uncertainty should correspond to value of 0.5. The mean value of the entropy plus-minus standard deviation in this case is 0.22 + 0.24. This indicates that the agent is somewhat certain about its decision, but there are cases where is completely certain (in error) or quite uncertain, where further training may improve the situation.
There are also few cases where the agent generates PULL instruction does not generate one when it is expected to ("Spurious Pull" and "Missed Pull"). We hypothesized that this may be because of an asymmetry between the actions seen for "push while spinning" in the context and "pull while spinning" in the target (the number of PUSH or PULL actions can differ depending the location of the target object and surroundings), but we did not see any concrete relationship here.
DISCUSSION AND CONCLUSION
The extension of meta-seq2seq we present in this paper has promising results in the challenging Split H of the gSCAN benchmark. Split D and G remain challenging to solve with general approach that does not rely on data augmentation or a problem-specific architecture. There also the splits proposed in (Qiu al. 202T T) which require relative reasoning about objects, where Transformer can only solve half of the test examples. Finally there are also the test splits in ReaSCAN (Wu et al.) 202I) which are comprised of very complex instructions requiring multiple steps of reasoning find the target object. See Sikarwar et al. (2022) for recent progress on these splits.
At present, an important limitation of this work is that oracle function generates I,,..., In and Ai.... An for given 12 We hope to extend this work by replacing the oracle with generative model, which generates in in-distribution query instructions and their corresponding actions with reference to the generative distribution of the training data. There is also slight degradation of performance on the other splits, including the in-distribution Split A, compared to a Transformer. In suggest that applications
light of the limitations, example by distillation (Hinton et al. 2015) or by using this approach as a sort "system-2" fallback for when "system-1" model is uncertain about its inputs (Goyal & Bengio 2020).
this work in conjunction with bascline, for
Generalization to unseen instruction compositions remains a challenging problem. Our hypothesis was that veetsge is promising general approach and could be extended to grounded language scenarios by an agent which generates a relevant meta-learning context. Our preliminary results show that such extension has promise and an area for future work.

---

Under review as a conference paper at ICLR 2023
REFERENCES
Jacob Andreas. Good-enough compositional data augmentation. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, July 5-10, 2020, pp- 7556-7566, 2020.
Jacob Andreas, Marcus Rohrbach, Trevor Darrell, and Dan Klein. Neural module networks. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 39-48, 2016.
Satwik Bhattamishra, Kabir Ahuja, and Navin Goyal. On the ability and limitations of transformers to recognize formal languages. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, Online, November 16-20, 2020, pp. 7096-7116 16, 2020.
Sebastian Borgeaud, Arthur Mensch, Jordan Hoffmaann, Trevor Cai, Eliza Rutherford, Katie Mil- lican, George van den Driessche, Jean-Ban Baptiste Lespiau, Bogdan Damoc, Aidan Clark, Diego de Las Casas, Aurelia Guy. Jacob Menick, Roman Ring, Tom Hennigan, Saffron Huang, Loren Maggiore, Chris Jones, Albin Cassirer, Andy Brock, Michela Paganini, Geoffrey Irving, Oriol Vinyals, Simon Osindero, Karen Simonyan, Jack W. Rae, Erich Elsen, and Laurent Sifre. Improv- ing language models by retrieving from trillions of tokens. In International Conference on Machine Learning, ICML 2022, 17-23 July 2022, Baltimore, Maryland, USA, volume 162 of Proceedings of Machine Learning Research, pp. 2206-2240, 2022.
Ayush K. Chakravarthy, Jacob Labe Russin, and Randall C. O'Reilly. Systematicity emerges in transformers when abstract grammatical roles guide attention. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguisties: Human Language Technologies: Student Research Workshop, NAACL-HIIT 2022, Hybrid Event / Seattle, WA, USA, July 10-15, 2022, pp. 1-8, 2022.
Elliot Chane-Sane, Cordelia Schmid, and Ivan Laptev. Goal-conditioned reinforcement learning with imagined subgoals. In Proceedings of the 38th International Conference Machine Learning, ICML 2021, 18-24 July 2021, Virtual Event, volume 139 of Proceedings of Machine Learning Research, pp.
2021.
Devendra Singh Chaplot, Kanthashree Mysore Sathyendra, Rama Kumar Pasumarthi, Dheeraj Rajagopal, and Ruslan Salakhutdinov. Cattedattentionn architectures for task-oriented language grounding. In Proceedings of the Thirty-Secone AAAI Conference on Artificial Intelligence, (8),I-18), the 30th innovative Applications of Artificial Intelligence (IAAI-18), and the 8th AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI-18), New Orleans, Louisiana, USA, February 2-7, 2018, pp. 281 2819-2826. 2018
Yen-Chun Chen, Linjie Li, Licheng Yu, Ahmed El Kholy, Faisal Ahmed, Zhe Gan, Yu Cheng, and Jingjing Liu. UNITER: universal image-text representation learning. In Computer Vision ECCV 2020 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XXX, volume 12375 of Lecture Notes in Computer Science, pp. 104 120, 2020.
Maxime Snevalier-Boisverr Dzmitry Bahdanau, Salem Lahlou, Lucas Willems, Chitwan Saharia, Thien Huu Nguyen, and Yoshua Bengio. Baby A platform to study the sample efficiency of grounded language learning. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019, 2019.
Noam Chomsky, Syntactic Structures. 1957.
Henry Conklin, Bailin Wang, Kenny Smith, and I Titov. Meta-learning to compositionally gener- alize. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, ACIJUCCLP 2021,
(Volume 1: Long Papers), Virtual Event, August 1-6, 2021, pp. 3322-3335, 2021.
Andreea Deac, Petar Velickovic, Ognjen Milinkovic, Pierre-Luc Bacon, Jian Tang, and Mladen Nikolic. Neural algorithmic reasoners are implicit planners. In Advances in Neural Information Processing Systems 34: Anmual Conference on Neural Information Processing Systems 2021, NeurlPS 2021, December 6-14, 2021, virtual, pp. 15529-15542, 2021.
10

---

Under review as a conference paper at ICLR 2023
Yan Duan, John Schulman, Xi Chen, Peter L. Bartlett, llya Sutskever, and Pieter Abbeel. RIS"2S: Fast reinforcement learning via slow reinforcement learning. arXiv, 1611.02779, 2016.
Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adaptation of deep networks. In Doina Precup and Yee Whye Teh (eds.), Proceedings of the 34th Interna- tional Conference on Machine Learning, ICML 2017, Sydney, NSW, Australia, 6-11 August 2017, volume 70 of Proceedings of Machine Learning Research, pp- 1126-11355 PMLR, 2017. URL http::/proceeddnngg .mw....mmm...................... mir.. wwtps//wo//aenn//////
Tong Gao, Qi Huang, and Raymond J. Mooney. Systematic generalization on gSCAN with language conditioned embedding. In Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing, AACL/IJCNLP 2020, Suzhou, China, December 4-7, 2020, pp. 491-503, 2020.
Atticus Geiger, Ignacio Cases, Lauri Karttunen, and Christopher Potts. Posing fair generalization tasks for natural language inference. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP 2019, Hong Kong, China, November 3-7, 2019, pp. 4484 4494, 2019.
Jonathan Gordon, David Lopez-Paz, Marco Baroni, and Diane Bouchacourt. Permutation equivariant models for compositional generalization in language. In 8th International Conference Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020, 2020.
Anirudh Goyal and Yoshua Bengio. Inductive biases for deep learning of higher-level cognition. arXiv, 2011.15091, 2020.
Anirudh Goyal, Abram L. Friesen, Andrea Banino, Theophane Weber, Nan Rosemary Ke, Adrià Puig- domènech Badia, Arthur Guez, Mehdi Mirza, Peter C. Humphreys, Ksenia Konyushkova, Michal Valko, Simon Osindero, Timothy P. Lillicrap, Nicolas Heess, and Charles Blundell. Retrieval- augmented reinforcement learning. In International Conference on Machine Learning. (CML2022, 17-23 July 2022, Baltimore, Maryland, USA, volume 162 of Proceedings of Machine Learning Research, pp. 7740-7765, 2022.
Demi Guo, Yoon Kim, and Alexander M. Rush. Sequence-level mixed sample data augmentation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, Online, November 16-20, 2020, pp. 5547-5552, 2020a.
Yinuo Guo, Zeqi Lin, Jian-Guang Lou, and Dongmei Zhang. Hierarchical poset decoding for compositional generalization in language. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual, 2020b.
Danijar Hafner, Timothy P. Lillicrap, Jimmy Ba, and Mohammad Norouzi. Dream to control: Learn- ing behaviors by latent imagination. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020, 2020.
Christina Heinze-Demt and Diane Bouchacourt. Think before you act: A simple baseline for compositional generalization. arXiv:2009. 000220009.3993 /3962, 2009.13962, 2020.
Geoffrey E. Hinton. Oriol Vinyals, and Jeffrey Dean. Distilling the knowledge in a neural network. arXiv, 1503.02531, 2015.
Drew A. Hudson and Larry Zitnick. Generative adversarial transformers. In Proceedings of the 38th International Conference on Machine Learning, ICML 2021, 18-24 July 2021, Virtual Event, volume 139 of Proceedings of Machine Learning Research, pp. 4487-4498 2021.
Dieuwke Hupkes, Verna Dankers, Mathijs Mul, and Elia Bruni Compositionalitt decomposed: How do neural networks generalise? J. Artif. Intell. Res., 67:757-795, 2020.
Eric Jang. Alex Irpan, Mohi Khansari, Daniel Kappler, Frederik Ebert, Corey Lynch, Sergey Levine, and Chelsea Finn. BC-Z: zero-shot task generalization with robotic imitation learning. In 5th Annual Conference on Robot Learning, 8-11 November 2021, London, UK, pp. 991-1002, 2021.
11

---

Under review as a conference paper at ICLR 2023
Yichen Jiang and Mohit Bansal. Inducing transformer's compositional generalization ability via auxiliary sequence prediction tasks. In
Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih (eds.), Proceedings the 2021 Conference Empirical Methods in Natural Language Processing, EMNLP 2021, Virtual Event / Punta Cana, Dominican Republic, 7-11 November, 2021, pp. 6253-6260 353-6265..4 Association for Computational Linguistics, 2021. doi: 10.1865.. emnlp moanlp-main
505
Najoung Kim and Tal Linzen. COGS: A compositional generalization challenge based on semantic interpretation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing. EMNLP 2020, Online, November 16-20, 2020, pp. 9087-9105, 2020.
Yen-Ling Kuo, Boris Katz, and Andrei Barbu, Compositional networks enable systematic general- ization for grounded language understanding. In Findings of the Association for Computational Linguistics: EMNLP 2021, Virtual Event / Punta Cana, Dominican Republic, 16-20 November, 2021, pp. 216 226.226,222 2021.
Brenden M. Lake. Compositional generalization through meta sequence-to-sequeee sequence learning. In Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurlPS 2019, December 8-14, 2019, Vancouver, BC, Canada, pp-
2019.
Brenden M. Lake and Marco Baroni. Generalization without systematicity: On the compositional skills of grquence-to-sqqueee recurrent networks. In Proceedings of the 35th International Conference on Machine Learning, ICML. 2018, Stockholmsmüssan, Stockholm, Sweden, July 10- 15, 2018, volume 80 of Proceedings of Machine Learning Research, pp. 2879-2888 2018.
Lajanugen Logeswaran, Yao Fu, Moontae Lee, and Honglak Lee. Few-shot subgoal planning with language models. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACI 2022, Seattle, WA, United States, July 10-15, 2022, pp. 5493-5506, 2022.
João Loula, Marco Baroni, and Brenden M. Lake. Rearranging the familiar: Testing compositional generalization in recurrent networks. In Proceedings of the Workshop: Analyzing and Interpreting Neural Networks for NLP, BlackboxNLP EMNLP 2018, Brussels, Belgium, November 1, 2018, pp. 108-114, 14, 2018
Luckeciano C. Melo. Transformers are meta-reinforcemeen learners. In International Conference on Machine Learning, ICML. 2022, 17-23 July 2022, Baltimore, Maryland, USA, volume 162 of Proceedings of Machine Learning Research, pp. 15340-15359, 2022.
So Yeon Min, Devendra Singh Chaplot, Pradeep Ravikumar, Yonatan Bisk, and Ruslan Salakhutdinov. FILM: following instructions in language with modular methods. arXiv, 21 10.07342, 2021.
Eric Mitchell, Chelsea Finn, and Christopher D. Manning. Challenges of acquiring compositional inductive biases via meta-learning. In AAAI Workshop on Meta-Learning and MetaDL Challenge, MetaDL AAAI 2021, virtual, February 9, 2021, volume 140 of Proceedings of Machine Learning Research, pp. 138-148, 2021.
Ashvin Nair, Vitchyr Pong, Murtaza Dalal, Shikhar Bahl, Steven Lin, and Sergey Levine. Visual reinforcement learning with imagined goals. In Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurIPS 2018,
December 3-8, 2018, Montréal, Canada, pp. 9209-9220, 2018
Maxwell Nye, Michael Henry Tessler, Joshua B. Tenenbaum, and Brenden M. Lake. Improving coherence and consistency in neural sequence models with dual-system, neuro-symbolic reasoning. In A. Beygelzimer, Y. Dauphin, P. Liang, and J. Wortman Vaugban (eds.), Advances in Neural Information Processing Systems, 2021. URL https: fttEs:://operevviie
P P7GUAXXS3ym.
Arkil Patel, Satwik Bhattamishra, Phil Blunsom, and Navin Goyal. Revisiting the compositional generalization abilities of neural sequence models. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). ACL 2022, Dublin, Ireland, May 22-27, 2022, pp. 424 434, 2022.
12

---

Under review as a conference paper at ICLR 2023
Alethea Power, Yuri Burda, Harrison Edwards, Igor Babuschkin, and Vedant Misra. Grokking: Generalization beyond overfitting on small algorithmic datasets. arXiv, 2201.02177, 2022.
Linlu Qiu, Hexiang Hu, Bowen Zhang, Peter Shaw, and Fei Sha. Systematic generalization on gSCAN: What is nearly solved and what next? In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, EMNLP 2021, Virtual Event / Punia Cana, Dominican Republic, 7-11 November, 2021, pp. 2180-2188, 2021.
Linlu Qiu, Peter Shaw, Panupong Pasupat, Pawel Krzysztof Nowak, Tal Linzen, Fei Sha, and Kristina Toutanova. Improving compositional generalization with latent structure and data augmentation. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguisties: Human Language Technologies, NAACI 2022, Seattle, WA, United States, July 10-15, 2022, pp. 4341-4362, 2022.
Laura Ruis and Brenden M. Lake. Improving Systematic Generalization Through Modularity and Augmentation. In 44th Annual Conference of the Cognitive Science Society, 2022.
Laura Ruis, Jacob Andreas, Marco Baroni, Diane Bouchacourt, and Brenden M. Lake. A benchmark for systematic generalization in grounded language understanding. In Advances in Neural Informa- tion Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, Virtual Event, 2020.
Jacob I.. Russin, Jason Jo. Randall C. O'Reilly, and Yoshua Bengio. Systematicity in a recurrent neural network by factorizing syntax and semantics. In Proceedings of the 42th Annual Meeting of the Cognitive Science Society- Developing a Mind: Learning in Humans, Animals, and Machines, CogSci 2020, virtual, July 29 August 1, 2020, 2020.
Matthew Setzler, Scott Howland, and Lauren A. Phillips. Recursive decoding: A situated cognition approach to compositional generation in grounded language understanding. arXiv, 2201. 1766, 2022.
Haoyue Shi, Karen Livescu, and Kevin Gimpel. Substructure substitution: Structured data augmenta- tion for NLP. In Findings of the Association for Computational Linguistics: ACL/IJCNLP 2021, Online Event, August 1-6, 2021, volume ACL/JJCNPP 2021 of Findings of ACL, pp. 3508, 2021.
Mohit Shridhar, Jesse Thomason, Daniel Gordon, Yonatan Bisk, Winson Han, Roozbeh Mottaghi, Luke Zettlemoyer, and Dieter Fox. ALFRED: A benchmark for interpreting grounded instructions for everyday tasks. 2020 IEEE/CVF Conference on Computer Vision and Puttern Recognition, CVPR 2020, Seattle, WA, USA, June 13-19, 2020, pp. 10737-10746, 2020.
Ankur Sikarwar, Arkil Patel, and Navin Goyal. When can transformers ground and compose: Insights from compositional generalization benchmarks. arXiv, 2210.12786, 2022.
Shagun Sodhani, Amy Zhang, and Joelle Pineau. Multi-task reinforcement learning with context- based representations. In Proceedings of the 38th International Conference on Machine Learning, ICML 2021, 18-24 July 2021, Virtual Event, pp. 9767-977777 2021.
Sam Spilsbury and Alexander Ilin. Compositional generalization in grounded language learning via induced model sparsity. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Student Research Workshop, NAACL-HILT 2022, Hybrid Event / Seattle, WA, USA, July 10-15, 2022, pp. 143-155, 2022.
Josh Tenenbaum.
Building machines that learn and think like people. In Proceedings of the 17th International Conference on Autonomous Agents and MultiAgent Systems, AAMAS 2018, Stockholm, Sweden, July 10-15, 2018, pp. 5, 2018.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in Neural Information Processing Systems 30: Anmual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, pp. 5998-6008, 2017.
13

---

Under review as a conference paper at ICLR 2023
Zhengxuan Wu, Elisa Kreiss, Desmond C. Ong. and Christopher Potts. Reascan: Compositional reasoning in language grounding. arXiv, 2109.08994, 2021.
Jingfeng Yang, Haoming Jiang, Qingyu Yin, Danqing Zhang, Bing Yin, and Diyi Yang. Seqzero: Few-shot compositional semantic parsing with sequential prompts and zero-shot models, arXiv, 2205.07381, 2022.
Pengcheng Yin, Hao Fang, Graham Neubig, Adam Pauls, Emmanouil Antonios Platanios, Yu Su, Sam Thomson, and Jacob Andreas. Compositional generalization for neural semantic parsing via span- level supervised attention. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLLT 2021, Online, June 6-11, 2021, pp. 2810-281888880 2823, 2021.
Victor Zhong, Austin W. Hanjie, Sida I. Wang, Karthik Narasimhan, and Luke Zettlemoyer. SILG: the multi-domain symbolic interactive language grounding benchmark. In Advances in Neural Information Processing Systems 34; Annual Conference on Neural Information Processing Systems 2021.
2021, NeurlPS 2021, December 6-14, 2021, virtual, pp.
Wang Zhu, Peter Shaw, Tal Linzen, and Fei Sha. Learning to generalize compositionally by transfer- ring across semantic parsing tasks, arXiv, 2111.05013, 2021.
14

---

Under review as a conference paper at ICLR 2023
A COMPUTATIONAL RESOURCE USAGE AND REPRODUCIBILITY REQUIREMENTS
Experiments
internal GPU cluster. Running meta-learning experiment 30,000
iterations takes about 3 days on a NVIDIA Tesla V100 GPU. For different experiment runs with 10 seeds each, the total compute time about 210 GPU-days, though in practice the experiments can be run in parallel.
The batch size (4096)
quite large and does fit in GPU memory for a consumer grade GPU. In order to achieve these batch sizes, we use gradient accumulation, so the batch size for each backward step might be 256, but then gradients are averaged over 16 steps make effective batch size of 4096. This trades training time for optimization stability.
B ADDITIONAL COMPARISONS
We show additional related work comparisons and hyperparameters in Tables and
Sacg (Ruis
GECA
Fil M
RelNer
LCGN
Planning
RD Random/RI
Chrs(o)
2020
X020
2020
V5.15
Kr.67
95.33
V8.6
98. U.17
50.05 26.76
34.92 39.90
94.04 7.41
99.08
87.31 4,38
62.19 24.08
0.96 0:01
C
29.79 a 17.70
78.77 1 6.63
60.12 8.81
19.92
80.31 a 24.51
81.07 a 10.12
56.52 a 29.70
0.97 a 0.01
D
0.00 x 0.00
0.00 0.00
0.00 0.00
0,00
0.16
43.60 6.00 6.05
0.35 0:05
42.17
$
19.04
11.83 0.31
11.71 2.34
18.26 1.24
33.6 20.81
21.95 0.03
0.86 0.02
Table 4: Additional related work comparisons.
Modular Ruis Lake,
Role-guided
Transformer
Ours(o,
Ours(o)
2021
Kuo al. 2021
Ours
Ours
Leaming Rate
0.0013
0.001
0.001
0.0001
0.0001
0.0001
Batch Size
128
200
200
4096
4096
4096
Steps
11 14..6KK 3M
73K
150K
35K
35K
35K
#params
13.2M
13.2M
13.2M
Table Hyperparameters used in the related work comparisons of Table
15

---

Under review as a conference paper at ICLR 2023
C
ORACLE FUNCTION
Environment
Query Instructions
Target Actions
Supports
push small circle hesitandy
LIORN LIURN
WALK STAY
STAY WALK
STAY WALK STAY
RIJRN WALK
STAY PUSH STAY
PUSH STAY POSH
STAY PUSH STAY
pull small circle hes- itandy
LT3RN LIURN
WALK STAY
WALK STAY WALK
STAY WALK STAY
RIJRN WALK
PULL STAY
PULL STAY PULL
walk small circle hesitantly
LTURN (2) KALK STAY
STAY PULL STAY
WACK STAY WALK STAY
walk small circle while spinning
LTJRN (4) LIURN
WALK STAY RTURN
LTURN WALK
WAIK STAY
LTJRM (4) WALK
LTURN WATK
LTJRM 14) WALK
LTURN (4)
RTTURN
WALK
walk small
LIJHN LIURN
while zigzagging
WALK RTURN
JRN
WALK WALK WALK
small
JIJ
RALK WALK WALK
JI.K
RALK
walk to cylinder while zigrag. ging
MALK RTORN
WALK LTURN
JIJ
WALK LTURM
pull red small cylin- der while zigzagging
WALK RTURN
JIJHH
RALK BTUR
RALK
push a cylin- der while spinning
LIJBNN
push a red small ey lin- while sigzagging
NACK RITURN WALK WALK RIURN LTURN WALK
LTJRN (4)
WALK
LTJRN
WALK
LTJRN (4) RIURN
LTURN
RALK
push red small cylin- der besitantly
NALK
STAY WAT. STAY
RTJRN
STAY NAL. STAY
push red small der
NALK
RTORN MATK
Table Examples of what supports the oracle function generates for given query instruction and environment state. These two examples are from the training data. Note that we never generate the same instruction as the query instruction the supports, and we also never generate any Split H instruction in the supports. Also note that in some cases, the environment makes pushing or pulling an object impossible, even though it is in the instruction, see the second row for an example this.
The oracle function generates relevant instructions by the use of a templating mechanism, which replaces verbs and adverbs in the sentence with other verbs and adverbs, such that the whole combination is still in distribution, but not the same as the query instruction. The rules of the system are:
Replace "pull" with "push" and "walk to"
Replace "walk to" with "push" and "pull" (but not "while spinning" is the adverb)
Replace "push" with "walk to" and "pull" (but not if "while spinning" is the adverb) Replace "while zigzagging" with "hesitantly", nothing and "while spinning" (but not "push" is the verb)
16

---

Under review as a conference paper at ICLR 2023
Replace "hesitantly" with "while zigzagging", nothing and "while spinning" (but not if "push" is the verb)
Replace "while spinning" with "hesitantly", " "while zigzagging" and nothing
Examples of what the oracle function generates for given query instruction and environment be found in T
Actions generated by using the same procedure provided in Ruis et al. (2020). 'he instruction generated by the oracle is given to the demonstration generation procedure and a demonstration
generated by that. A demonstration can also be generated by providing the oracle-generated
instruction and current state representation as input to a Transformer model trained on the provided training set.
D DICTIONARIES
Word
Symbol
Action
Symbol
PULL
0
'big",
PUSH
I
"blue"
2
STAY
2
cauuioussl"
3
LTTRN
3
"cirele"
RTURN
4
cylinder'
NALK
5.
"green'
'hesitantly'
'pull'
'push
"red'
10
smalll
11
square"
12
'walk'
"while spinning'
"while zigzagging' 16
Table 7: Default mapping of words and actions to symbols
17

---

Under review as a conference paper at ICLR 2023
E
PERMUTER BLOCKS
Original
Encuded words
encodiin
walk to blue small cylinder
*0
bi 17, blue2)
141302 II5
21430511
cauriously(3) 16, circle(4) cylinder(5))
11, green(()) - 10, hesitanll(7)
15, pall
13. push(9) + 4, rod(10) 6, smali(1
square( (2) - 12, to(13) 14, walk(14)
2.
while spioning(15) 9, while zigzagging(16)
1. yellowwr (7) -
pall a green cylinder
a(O)
big(l) - bluc(2)
8063
177113
cautiously()) 15, circle()) cylindert5)
5. green((6) 11, hesitanly(7)
pall(8)
17. push()) red(10) + 12, small
square(12) 10, to(13) + walk(14)
while spinningt15) -, while rigragging(16)
ycllow((7)-+
push small square while spinning
cautiously()) 1..cirele() blue(2)
a(0
big(1)
9.06 11 12
1607 1343
cylinder(5)
17. green()) hesitntly(]] 12, pull(8)
2. push(9) rod(10) 16, small
square(12)
1o(13)
w4)k(14)
while spinning(15) while zigzagging(16)
11, yellow(())) -3
pull yellow smmall cylinder besi- lanty
a(0) - 6, big(1) - 14, blue(2) +
17.
80
4 1209
cautiously()) - 2,,ccrc - cylindert5)
0. green(6) + hesitantl(()) + pull(8)
push(9) 8, red(10) 10, small( 12.
squarc(12)
to(13) + walk(14) 11,
while spinning(15) while zigzagging(
ycllow(17) 4.
push big cylinder while spinning
a(0 17. big(1) + 0. blue(2)
901151
817036
cantiousy(3)) 14, cincle(4) 2, cylinder(S))
3. green(6)
besituntl(())
16, pull(8)
7. push(9) red(10) 15, smalk(l) 11.
square(12) - 13, to(13) - 10, walk(14) 12.
while spinning(11) while zigzagging(16)
5. yellew((7) -
Table 8: Instructions and possible mapping permutations generated by the permuter block.
Original actions
Permuration
Encoded actions
Permured eacoding
WALK(5) RTURN WALK(S)
PULL(U) O, PUSH(I) STAY(2)
5(5) 4 5(5)
4(5) 3 4:5)
LTURN(3) 1, RTURN(4) - 3, WALK(S)
4.
RTURN WALK(3)
PULLO) - 0. PUSIRT) - 2, STAY(2) -5 3,
4
4T33
LTURN(() 5, RTURN(4)
WALK(S)
[TURN(4) WALK LTURN(4)
PULLZO) PUSH(T STAY(2)
3(4) 3() 3(5
4(4) 2(4) 255
WALK LTURN(S) WALK
LTURN(3) - 2, RTURN(4) - 3, WALK(S)
3(4) 3(4) 5 3(4)
2(4) 2(4) 2(4) 1
(TURN(4) WALK LTURN(4)
3(4)
2(4)
WALK LTURN(4) WALK
(TURN(4) WALK
LTURN WALK STAY WALK
PULLÓ)) - 3, PUSH() - 0, STAY() +
352325232
312121212
STAY WALK STAY WALK STAY
LTURRN(3 5, RTURN(4) WALK(S)
1.
LTURN WALK STAY
+
STAY(2)
35252
$1414
LTURN(3) + 5, RTURN(4) - 2, WALK(S)
LTURN(4)
WALK LTURN(4)
PULLO + PUSIIO) + 4, STAY2) + LTURN(3) RIURN(4) WALKIS) 2.
3(4) 3() 3(4
1(4) 2 1(4) [(4]
LTURN(4)
1(4)
LTURN(4) RTURN
WALK
3(4) 5 3(4)
1(4) 1(4)
WALK
LTURN(4)
WALK
WALK
LTURN WALKZJ
PULLOP 1, PUSIIO) + O. STAY() +
35(2
322)0
LTURN() 3, RTURN(4) WALK(() 2.
Table 9: Actions and possible mapping permutations generated by the permuter block.
The permuter block shuffles the indices mapping words to symbols in the dictionary given in Table Tables 8 and give example of how the permuted sequences might look to the encoders. Essentially the individual symbols no longer hold any special meaning without reference to the demonstrations, only conditional autoregressive probabilities up to permutation hold meaning.
18


Reviews:
[{"review_id": "qWdOzEI9EGz", "reviewer": "Reviewer_xpEo", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper works on compositional generalization of language-instructed agents. \nThey apply meta-seq2seq method to this setting. \nThey conduct experiments on the gSCAN benchmark. ", "summary_of_the_review": "The paper aims to address an interesting problem and their general technical idea (using meta-learning) is reasonable. \n\nBut the paper is poorly written and it is hard to give a good judgement on its current version. ", "strength_and_weaknesses": "Strength \n\nThey study an important problem. \n\nUsing meta-learning for compositional generalization is reasonable. \n\nThe results on Split H are positive and they also conducted a range of ablation and error analysis. \n\nWeakness \n\nThe paper is poorly organized and very hard to follow. This is the key weakness, but this makes me find it really difficult to judge the overall technical quality and significance. \n\nE.g., after reading the related sections a few times, I still do not understand how meta-seq2seq is applied in their setting and how the technical components handle the gSCAN environments. It seems that many important introductions and remarks are missing. So I suggest that the authors give a technical introduction of the framework and more precisely discuss how it can solve the problem of interest, possibly with visual illustrations. \n\nSome important concepts are repeatedly used without a definition. E.g., what is a \"support\"? The paper has \"support set\" and \"support instructions\" at many places but it is unclear to me what it actually means. I also read the original gSCAN paper but they didn't use this term at all. Similarly, what is an \"oracle\"? What is \"in-distribution\" instructions and how would an instruction from non-oracle look to an instruction from oracle? \n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "gnAnv2BZ-c", "reviewer": "Reviewer_wsCX", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper addresses the generalization ability of language-instructed agents of gSCAN. Based on the Meta-Sequence-to-Sequence learning approach and meta-seq2seq architecture of Lake 2019, they issue the statistical action-bias of gSCAN split-H and extend the Meta-Sequence-to-Sequence learning approach to the specially case of split-H. They introduce “oracle templating function” tune the model on the different in-distribution set of split-H. They also provide the model improvements, such as replacing RNNs with Transformers. They succeeded to tune their models to some specific test sets of Split-D and Split-H sacrificing performance in other testsets, using the preliminary known information of the testsets.\n \nEntirely authors include both the contribution of the proposed “oracle templating function” in Sec 3.2 and the comparably minor model improvement (e.g., swapping RNNs with Transformer and minor neural network modification / engineering). It is regrettable that authors mix two parallel contributions and hence it becomes less clear what is the major contribution of the paper. Please clarify the differences from Lake 2019 concisely in the manuscript. In the current paper, authors raised many incremental improvements one by one in Sec 3.2 seemingly irreverent to the main contribution of “oracle templating function.” The details and insights of “oracle templating function” are not clear. Overall, the arguments of this paper are not still convincing yet.", "summary_of_the_review": "Overall, the paper arguments are still not clear nor convincing yet. The motivation doesn't sound. I hope authors have further discussion on the motivation and the next directions of the entire paper possibly in other conferences or workshops.", "strength_and_weaknesses": "- The strength of this paper is that they obtained better performance than the previous models with “oracle templating function” in limited test splits, although there are following problems as written in weakness.\n \n- The first problem is that the proposed model uses the periluminal knowledge of some testsets such as the different distributions of verbs and adverbs mentioned in Sec 3.2. It is prohibited in the previous studies compared in Table 2. The details of the contribution “oracle templating function” is written in Appendix, not in the main paper. It still lacks details.\n \n- The second problem is that the contribution / observation is limited. In this paper, authors proved that tuning Meta-Sequence-to-Sequence learning approach on the limited testset of gSCAN is effective. However, this is not so surprising because some testsets of gSCAN are designed for different test situations. Therefore, we can tune our models for some specific test sets if we preliminary know the information of the testset, such as the different distributions of verbs and adverbs. When we turn some models into the specific distribution, we obviously sacrifice the performance in other test sets with different distributions. This is the observation of the main score Table 2 and is not a novel thing in the learning of the out-of-domain adaptation. Hence the overall paper contribution looks not obscure. In short, optimizing models just for a few good test sets is not a good contribution.\n \n- The third problem is indeed the largest problem: the motivation to access the statistics/bias of gSCAN test set and taking advantage of the existing bias in tuning model is not discussed well in this paper. I’d like to know concrete applications or explanations how the proposed models / approaches for the limited OOD test sets are useful in the following studies or real applications. If the motivation in this paper relies on human compositional problem solving as written in the introduction, how the observations in the experiments, especially for the results of “oracle templating function,” have contributed to it?\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "Q8dNk0GftjX", "reviewer": "Reviewer_TWtT", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposes an analysis of a solution for achieving compositional generalization on the grounded SCAN dataset, in particular on its compositional split \"H\", i.e. the novel adverb-verb combination setting. The model is required to generalize to creating commands for instructions such as `pull X while spinning`, but has never seen `pull` and `while spinning` together during training. The authors approach this problem inheriting the Meta-Seq2Seq approach from Lake et al. (2019) where a Seq2Seq model is trained to use a batch of support instructions and target commands pairs to predict the output command for a given query instruction. They create the support set using an oracle function (which is actually a manual instantiation of GECA (Andreas, 2020)) and show that permuting the mapping from each instruction to the command (not clear how the permutation was done, but I can imagine by reading the Meta-Seq2Seq paper). They show that the resulting model can outperform the baselines on split H (and D) and provide an ablation of the most important components, namely the use of permutations and the oracle function to create support examples.\n", "summary_of_the_review": "I do overall appreciate the relevance of the problem the paper is trying to solve and the ablations therein. Lack of novelty and limited scope indicate this paper might be more suited for a workshop. In its present form, the paper lacks clarity, it is not self-contained and thus it might not be ready for publication just yet.", "strength_and_weaknesses": "Strengths:\n- *Strong performance on split H*: The paper presents a model that show promise in improving performance on split H for GScan. Improvements over recent baselines are promising.\n\n- *Nice ablation study*: The ablation study and the analysis are interesting to understand the failures of the method. I particularly appreciate when a particular problem is studied in detail and ablations are provided.\n\nWeaknesses:\n- *Limited clarity*: The paper could be more self-contained: I found it hard to figure out what is going on without re-reading the relevant literature. For example the GScan dataset and the original Meta-Seq2Seq model have not been introduced.\n\n- *Limited scope*: In-depth analysis of a particular generalization failure in only one setting, GScan. A more impactful paper would offer perspective on how the method can be fruitful to solve also other compositional generalization datasets (CFQ? Natural Questions?)\n\n- *Limited novelty*: An application of the Meta-Seq2Seq model from Lake et al. 2019 to GScan. I cannot identify whether there exist an intrinsic modelling novelty brought forward by this work, apart from the design of the oracle function that generates support sets. The oracle function resembles the methodology introduced by GECA.\n\n- *Limited applicability*: Ablation offered the key insight that permutations are important to solve split H, but afaicu, permutations require knowing a lot about the structure of the problem (e.g. that walk maps to I_WALK). Is this realistic for other datasets?\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "3LDLC_SoDQ", "reviewer": "Reviewer_EJmn", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper expands the Meta-Sequence-to-Sequence learning approach proposed by B. Lake in 2019, adapting it to be used with a modern transformer architecture and applying it to the gSCAN benchmark, utilizing the idea of random symbol/token permutation as a remedy against overfitting to specific sequences in the training set. The authors show that performance on two challenging splits of the dataset is improved, compared to previous works.", "summary_of_the_review": "I enjoyed reading the paper which addresses a highly important and ambitious problem of compositional generalization. I believe that the proposed method is interesting and has the potential to be generalized to model abstraction and generalization in a wider array of applications. Unfortunately, in its present state, I feel that the combination of relatively limited novelty with the relatively narrow range of applications puts this paper below the threshold of acceptance.", "strength_and_weaknesses": "Strengths:\n- The paper addresses a highly relevant problem\n- The proposed method is elegant and might allow to model abstraction and generalization\n- The proposed method shows promising results\n\nWeaknesses:\n- The method is very closely related to the work \"Compositional generalization through meta\nsequence-to-sequence learning\" by Lake (2019), which it builds upon. While adapting it to transformer architecture and a new task is not trivial, it does limit the conceptual novelty of the contribution.\n- Empirical results, while promising, are confined to a relatively narrow domain. This is especially unfortunate, since when novelty is relatively limited, the breadth and impact of empirical results becomes paramount.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject