On the generalization capacity of neural networks during generic multimodal reasoning
Authors: Takuya Ito
Soham Dan
Mattia Rigotti
James Kozloski
Murray Campbell
Venue: ICLR 2024 poster (2024)
Keywords: compositional generalization
compositionality
representation learning
out-of-distribution generalization
multimodal reasoning
Primary area: applications to neuroscience & cognitive science
TL;DR: We introduce a flexible multimodal compositional generalization benchmark, and evaluate a variety of base neural architectures on this task.

Abstract: The advent of the Transformer has led to the development of large language models (LLM), which appear to demonstrate human-like capabilities. To assess the generality of this class of models and a variety of other base neural network architectures to multimodal domains, we evaluated and compared their capacity for multimodal generalization. We introduce a multimodal question-answer benchmark to evaluate three specific types of out-of-distribution (OOD) generalization performance: distractor generalization (generalization in the presence of distractors), systematic compositional generalization (generalization to new task permutations), and productive compositional generalization (generalization to more complex tasks with deeper dependencies). While we found that most architectures faired poorly on most forms of generalization (e.g., RNNs and standard Transformers), models that leveraged cross-attention mechanisms between input domains, such as the Perceiver, fared better. Our positive results demonstrate that for multimodal distractor and systematic generalization, cross-attention is an important mechanism to integrate multiple sources of information. On the other hand, all architectures failed in productive generalization, suggesting fundamental limitations of existing architectures for specific types of multimodal OOD generalization. These results demonstrate the strengths and limitations of specific architectural components underlying modern neural models for multimodal reasoning. Finally, we provide *Generic COG* (gCOG), a configurable benchmark with several multimodal generalization splits, for future studies to explore.

Published as a conference paper at ICLR 2024
ON THE GENERALIZATION CAPACITY OF NEURAL NET- WORKS DURING GENERIC MULTIMODAL REASONING
Takuya Ito, Soham Dan, Mattia Rigotti, James Kozloski, & Murray Campbell TJJ. Watson Research Center, IBM Research (takuya to soham. dan @ bm. com, mrg zurici ch. bmm com [kozloski @ucameeus. i'om. com
ABSTRACT
The advent of the Transformer has led to the development large language mod- els (LLM), which appear to demonstrate human-like capabilities. To assess the generality this class of models and a variety of other base neural network ar- chitectures to multimodal domains, we evaluated and compared their capacity for multimodal generalization. We introduce a multimodal question-ansee answer bench- mark to evaluate three specific types of onttof--distiibution (OOD) generalization performance: distractor generalization (generalization in the presence of distrac- tors), systematic compositional generalization (generalization to new task per- mutations), and productive compositional generalization (generalization to more complex tasks structures). We found that across model architectures (e.g., RNNs, Transformers, Perceivers, etc.), models with multiple attention layers, mod- els that leveraged cross-attention mechanisms between input domains. fared bet- ter. Our positive results demonstrate that for multimodal distractor and systematic generalization, either cross-modal attention or models with deeper attention lay- ers are key architectural features required to integrate multimodal inputs. On the other hand, neither of these architectural features led to productive generalization, suggesting fundamental limitations of existing architectures for specific types of multimodal generalization. These results demonstrate the strengths and limita- tions of specific architectural components underlying modern neural models for multimodal reasoning. Finally, we provide Generic COG (gCOG), a configurable benchmark with several multimodal generalization splits, for future studies plore.
INTRODUCTION
Recent LLMs appear to exhibit remarkable cognitive abilities. On the surface, these models perform exceptionally well on cognitive tasks, such language comprehension, mathematical reasoning, and coding (Bubeck et al.) 2023]] Webb et al.] 2023). However, many of these demonstrations are limited single modality (e.g., language). Moreover, the mechanisms driving performance among these models are opaque. Even when both the model and dataset are publicly available, the sheer scale of the model and training data make it difficult to isolate which architectural mechanisms influence generalization behavior, in part due to the difficulty in controlling for confounding factors in large pretraining datasets (Kim et al., 2022). To quantify the relationship between mechanism and generalization on multimodal cognitive tasks, we sought to evaluate a set of base neural network architectures (RNN, GRU, Transformer, Perceiver) on carefully controlled multimodal task.
Recent studies have suggested that the impressive behaviors exhibited by LLMs are due to superficial data interpolation, rather than "emergent cognition" (Schaeffer et al. 2023 Kim et al. 2022 Wu
et al. 2023). One approach to adjudicate between possibilities is to first curate carefully controlles
training and test sets generated from compositional task experiments (Keysers et al. 2020 Dziri
et al. 2023; Bahdanau al. 2020 Ontañón et al. 2022, Csordás et al. 2022 Hupkes et al. 2020
Lake & Baroni 2018 Yang et al. 2018 Kim et al., (022). 2). By design, compositional tasks enable experimenters to measure OOD generalization the ability to perform tasks beyond the training distribution - by programmatically composing a test set using novel task components. Indeed, when controlling for confounding factors in a training set, studies have shown that neural models cannot

---

Published as a conference paper at ICLR 2024
However, most prior demonstrations have been limited to evaluating tasks within a single modality (c.g., natural language). Thus, remains unclear to what extent previous unimodal models generalize to multimodal reasoning tasks.
generalize OOD (Kim et al. 2022 Dzin al. 2023
Given recent studies demonstrating that pretraining can actually degrade downstream systematic generalization (Kim et al., ).22); and that OOD generalization performance inversely scale with model size (McKenzie et al. (023), we focused on training models from scratch rather than fine-tuning large pretrained models. This ensured full experimental control over training data and architectural choices. We introduce Generic COG (gCOG), a task abstracted from the previous COG task (Yang al., 2018). Our variant employs generic feature sets that are tied any specific modality, and relaxes previous experimental constraints to broaden its capacity to test compositional generalization splits (Fig. D This design allowed us comprehensively evaluate variety of model architectures on tasks that test for three different forms of OOD generalization: 1) Distractor generalization (generalization in the presence positional generalization (generalization to new permutations of task structures, i.e., combinatorial generalization), and 3) Productive compositional generalization (generalization to task structures of greater complexity). We find that models that integrate multimodal inputs with either deeper atten- tion layers or Pross-attention mechanisms (such as Perceiver- like models) performed best, and were capable of excellent distractor generalization, reasonably good systematic compositional generaliza- tion, yet (as with all models tested) no productive compositional generalization. Our results illustrate the successes and fundamental limitations of modern neural architectures' nascent multimodal rea- soning abilities, and we provide configurable multimodal reasoning benchmark for future studies build upon.
different noise distribution), 2) Systematic com-
A
Task Operators
Get Location
Get Shape
Sum
Sum Odd
Product Even
Product Odd
Exist
Color
Even
B
Example task, depth
Stimulus, 5 distractors
Example task, depth Instruction Exist pink "7"
Stimulus, 30 distractors
Instruction Exist
If-Then-Else
orange
then
else
Get Location blue "z"
Get Shape
green object
pink exists Then get the location of the blue "2" Else, got the shape of the green object.
Does an orange 'I' exist?
Answer:
"True"
Answer:
Location:
(8,8)
Figure I: gCOG task. We adapted the gerviously-develoee COG task (Yang et al. 2018 Our modifications to COG included different task operators, the ability to use categorical tokens to allow for generic testing of multimodal reasoning, and the ability to allow for arbitrarily long task instructions to allow for the evaluation of compositional productivity. A) Task operators and objects operator (e.g., Exiss)) paired with specific feature combination (e.g., orange + "("). Feature categories correspond to shape (i.e., letters "a" through "z") and color (i.e., 10 discretely coded colors), but can be naturally extended. B) At minimum, a task must comprise of one specific task operator (e.g., Exist) and feature combination (e.g., orange "t"). An arbitrary number of stimuli (e.g.. images) can be constructed on-the-fiy combined with
the units of the gCOG task. A task
satisfy this task instruction (i.e., produce TRUE FALSS. response). C) Tasks task complexity. This enables
conditional operator (c.g., an TF-THEN-BI.S conditional) to increase construction of arhitrarily complex tasks. While the original COG task explored only task trees of depth (i.e., a single conditional), we relaxed this constraint to allow for arbitrarily long task trees, Dataset: https: //github. com/ IBM/geog
1.1 RELATED WORK
number of recent studies have evaluated the compositional generalization properties of LLMs,
documenting several of their generalization shortcomings (Dziri et al. 2023 Keysers et al. 2020
2

---

Published as a conference paper at ICLR 2024
Kim et al. 2022 Wu et al. 2023). However, evaluations were limited to massive pretrained lan- guage models, making it difficult (if not impossible) to assess the interplay between training data, architecture, and generalization,
noted by Kim et al. (2022). Smaller scale studies demonstrated the limitations of modern neural network architectures (e.g.. Transformers, RNNs) on compositional
tasks, but have been focused primarily
single modality (e.g.. language) (Coordás et al.] 2022
Ontañón et al. 20222 Hupkes et al., 2020: Lake & Baroni 2018 Andreas, 2019 Bahdanau et al,
2019 Furrer al. 2021). Other neural models trained on multimodal question question-answer -nsswr datasets like CLEVR (Johnson et al. 2017) also failed to generalize systematically when proper training and test splits were curated (e.g.. CLOSURE) (Bahdanau et al., 2020). Related hybrid neuro- suro-symboolc approaches have demonstrated great success at compositional generalization tasks (Klinger et al.) 2023 Nye et al. 2020; Bahdanau et al., 2019; Shaw et al. 202T), although these approaches require customized symbolic architectures, possibly rendering them difficult to scale. Extending these stud- ies, our aim is to evaluate the performance of base neural models on generic multimodal reasoning.
Our approach to constructing arbitrarily complex compositions of simple tasks is similar to gSCAN (Ruis et al. 2020). However, it differs in three key ways. First, we focus on question-answer tasks (which require encoder-only architectures), rather than sequence-to-sequenee learning tasks (which require decoder architectures, e.g.. QQu et al. (202I)). Sequence decoding tasks introduce the added complexity of requiring autoregressive responses, which susceptible to fundamental statistical challenges, such as exposure bias (Wang & Sennrich, (2020). Second, gCOG includes a distractor generalization split, in addition to systematic and productive compositional generalization splits. Finally, we methodically characterize different forms of generalization using simpler under- lying abstractions (i.c., without the explicit use of image pixels). Indeed, the experimental design is most similar to the original COG (Yang et al. 2018), SQOOP (Bahdanau et al. 2019) and CLO- SURE (Bahdanau et al., 2020) in that is multimodal and can generate synthetic trials on-the-fly. fly- However, those previous tasks did not include productive compositional generalization benchmarks (i.e., evaluation of arbitrarily complex task commands), systematic compositional generalization benchmarks deeper task trees (c.g., task trees of depth 3), and explicit splits for OOD distractor generalization. In gCOG, we uniquely designed splits that target these three distinct forms of OOD generalization. gCOG therefore provides scalable design (e.g., more than two modalities can be straightforwardly accommodated) to broadly evaluate multimodal generalization (Fig. !
1.2 CONTRIBUTIONS
Specific contributions center around the configurability and flexibility of gCOG for three forms of OOD generalization, as well the comprehensive evaluation of variety of base neural network architectures on this task. We highlight three principal contributions of this study:
1. A configurable dataset that complements and extends prior tasks (i.e., gSCAN, SQOOP, COG) multimodal compositional generalization include productivity splits, system- aticity splits on deeper task trees, and OOD distractor generalization splits.
2. A comprehensive evaluation of commonly base neural models (RNNs, GRUs, Trans- formers, Perceivers) on distractor, systematic, and productive generalization splits. We find that for distractor and systematic generalization, including a cross-attention mechanism across input modalities is important. However, all models fail on the productivity split.
3. A comprehensive evaluation of how scaling standard encoder-only Transformer models improves distractor and systematic generalization, but not productive generalization.
Finally, we include analysis of internal model representations in Appendix A. revealing the influ- ence of base neural architecture internal model representations.
EXPERIMENTAL DESIGN
2.1 GCOG FOR MULTIMODAL AND COMPOSITIONAL EVALUATION
gCOG is configurable question-answer dataset, originally inspired from COG (Yang et al. 2018), that programmatically composes task instructions, and then generates synthetic stimuli to satisfy those instructions on-the-fly (Fig. 1.. The primary modifications in gCOG 1) differences in the
3

---

Published as a conference paper at ICLR 2024
set of task operators, 2) the ability to categorical tokens to allow for generic testing of mul- timodal reasoning. and 3) the ability to allow for arbitrarily long task trees to assess productive compositional generalization, in addition to distractor and systematic generalization (e.g., Ap- pendix Fig. [). Importantly, the original COG task did not allow for tasks with more than a single conditional statement, e.g., task tree of depth 3, making it difficiult to evaluate productive com- positional generalization. The use of categorical stimulus tokens and instruction tokens generically tests the capacity of neural architectures to maintain, manipulate, and generalize novel composi- tions. Importantly, if models are unable to generalize using simple categorical encodings, then it is unlikely that these models will generalize when presented with the same task in modality -speciif domain. The total number of unique individual tasks (i.e., task trees of depth 1) is 8 operators 26 shapes 10 colors- = 2080 unique individual task commands, but can straightforwardly extended with modifications to the configuration file. The number of total unique task structures explodes ex- ponentially when task trees exceed depth 1 (c.g., 5,624,320,00 of depth 3). We additionally provide functionality in the dataset that allows the choice to generate samples using either categorical task encodings, or task encodings with image pixels and natural language instructions. (All evaluations performed in this paper used categorical task encodings.)
unique task structures for task trees
The original COG dataset formulated tasks as directed acyclic graphs (DAGs). To simplify this representation (and ensure a unique, topologically sorted solution of operators), we constrained tasks to binary trees. Unless otherwise stated, stimuli were mapped from a 10x10 spatial grid to a sequence of 100 binary tokens, where each token represented specific location in the grid. If location contained an object, the embedding was encoded as the specific attributes of that object. All task rule tokens were appended with an EOS statement, which was previously demonstrated to improve generalization (Csordás et al. 2022). (See Appendix for additional details on experimental design A.2 and how the task inputs were configured for model training and inference AA.3
2.2 MODEL ARCHITECTURES
All models were trained to perform classification in a supervised manner. Outputs were projected to vector with 138 ele- ments, with each element in the vector representing a True/False boolean or a feature label (e-g.. the color "Red", letter "a", or the spatial location (2, 1)). Models that included a Transformer compo- nent in the main figures used absolute positional encoding (Vaswani et al.) 2017), though we also report results in the Appendix with Transformers that used relative positional encoding (Shaw et al.) 2018 Huang et al., 2018) (Appendix Fig. 9 and Fig. I0). Importantly, there were no discernible differences between these choices of positional encoding. Finally, we report additional evaluations on deeper and larger SSTfmr models (i.e., BERT-style models) in Fig. 6] all generalization splits. Those results demonstrate that improved distractor and systematic generalization performance can be achieved by scaling models (i.e., increasing encoder depth and attention heads), but not produc- tive compositional generalization.
We evaluated performance six encoder-only model architectures (Fig. 2
RNNs and GRUs. We trained both RNNs and GRUs with 512 units on gCOG (Fig. 2 Task trees (i.e., instructions) were presented as a sequence of token embeddings, one for each node in the binary tree. The end of the rule sequence was marked with an BOS token. Stimuli were flattened from 10 x 10 D matrix presented simultaneously (i.e., not tokenized).
100 x D matrix (where D the embedding dimension, and were
Single stream Transformer (SSTfmr). We trained a single stream Transformer, where the task instructions and stimuli were concatenated into a single matrix (with prro-padding), then passed through a shared Encoder block (Vaswani et al., 2017)) (Fig. [p)). Thus, in the Transformer block, self-attention was applied on both rule and stimulus embeddings simultaneously. The output of the Transformer was then processed through a 2-layer MLP before projection to the output layer.
Dual stream Transformer (DSTímr). We used Transformer-base) model to process task in- structions and stimuli separately through modality-specific parallel Transformer blocks (Fig. (bb) Outputs from the Transformers were subsequently processed through a shared MLP.
Transformers with Cross Attention (CrossAttm), Like the DSTfmr, this Transformer-bawee model processed task instructions and stimuli separately through modality-seecifie Transformers. How- ever, the outputs of the parallel Transformers were integrated through a cross-attention mechanism (Fig. 2). Specifically, cross-attention was estimated by computing the query from the output of
4

---

Published as a conference paper at ICLR 2024
RNN
GRU
B
Single Stream Transformer
Dual Stream Transformer
Task vector
Stimulus vector
Task vector
Stimulus vector
Task vector
Stimulus vector
Transformer
Transformer
Transformer Encoder Block
512 units
Encoder Block
Encoder Block
MLP
MLP
MLP
Output vector
Outpul vector
LayerNorm + MLP
D
E
Transformers with Cross Attn
Perceiver
Outpul vector
Task vector
Stimulus
Task vector
Stimulus vector
F
Number of parameters
Transformer Encoder Block
Transformer Encoder Block
Transformer Encoder Block
Transformer
per model
Encoder Block
Query
Key
Vallue
Key
Value
Key
Value
Parameters
Query
Query
3.3e406
Latent Transformer
CrossAttn+
LayerNorm
1.7e406
1e1006
Latent Transformer
MLP
NN8
CrosAAi
Perceiier
200
Output vector
Output vector
Figure 2: Model architectures. We evaluated generalization across six base neural network architectures. A) RNNs and GRUs with 512 hidden units. B) Single Stream Transformer (SSTfmrr), which processes task rules and stimuli a single Transformer, applying self-attention former (DSTfmr). In contrast to the SSTfmr, two parallel Transformer blocks process nule and image tokens separately, and then process them together in a shared MLP. D) Transformers with cross-attention (CrossAttm). The outputs of two parallel Transformer blocks are processed with cross-attention mechanism, where the out- put of the task rule Transformer produces query, and then the stimulus Transformer block produces a key and value matrix. E) A Perceiver-like like architecture, which integrates both task and stimulus output information in a latent Transformer through cross-altention (Jaegle et al. 2021). F) The number of parameters for each model.
Transformer block. C) Dual Stream Trans-
the task rule Transformer, and the key and value matrix computed from the stimulus Transformer output. The cross- cross-attention attention output was processed through a LayerNorm, and then an MLP.
Perceiver-like architecture (Perceiver). Finally, we included Perceiver-lie like model (Jaegle al. 2021], an architecture designed to generically process multimodal inputs (Fig. [2p). The Perceiver architecture contained latent Transformer, which uses cross- coss-attention process information from input modalities. Specifically, the latent Transformer produces a query for each modality. The out- puts of each modality-specific Transformer produced keys and values, which were subsequently processed through cross-attention with the latent Transformer. The latent Transformer also con- tained a standard self-attention Transformer block, followed by an MLP.
RESULTS
3.1 DISTRACTOR GENERALIZZTION
Experimental setup. Distractor generalization evaluates the degree to which a model can generalize a task to a stimulus or environment with more distractors than it was trained on. For example, good distractor generalization requires that model can correctly discern if a "red a" exists, independent of the number or configuration of distractors presented. We evaluated distractor generalization on an independent and identically distributed (IID) split and an OOD split. The IID split tests for generalization to stimuli with the same number of distractors that the model was trained on, but with different configuration. The OOD split evaluates generalization to stimuli with more distractors than observed during training. Models were trained on individual task operators for simplicity (i.c., task trees of depth 1). Stimuli in the training set were randomly generated with minimum of one
5

---

Published as a conference paper at ICLR 2024
distractor and maximum of five distractors. Models trained the same number of training steps (Appendix A.3). All models converged to greater than 94% training set accuracy (Fig. --)))
A Train set (up 5 distractors)
Test set (10, 20, 30, 40 distractors)
5 distractors
20 distractors
40 distractors
P
Training loss TeNN
GRU
CrcesAtti
generalizo
Perceiver
5000000
Train steps
C
Distractor generalization
E
OOD vs. IID distractor generalization
100
100
100
10..6
Training accuracy
180
80
80
Accuracy
60
68.8
0ID
51.4
5
30,000
30.5
(000)
30 (000)
20
110
5000000
Train steps
RNN
GRU
DST
SSTfmr
CrossAttn
Perceiver
RNN
GRU
DSTfmr
SSTTMm
CrossAttn
Perceiver
Figure 3: Distractor generalization. A) Experimental evaluation for distractor generalization. We trained mod- els on individual task operators (c.g., "Exist red d')) on stimuli that included 5 distractors, and then evaluated OOD generalization performance stimuli with 10, 20, 30, and 40 distractors. B) Loss and C) accuracy. D)
accuracy trajectories during training Distractor generalization performance for each model. We assessed IID distractor generalization (novel stimuli, but with or distractors), and OOD distractor gencralization (10, 20, 30, or 40 distractors). For most models, performance reduced the number of distractors inereased. E) We directly compared IID OOD distrac- tor generalization by averaging performance for IID OOD splits. Models incorporating cross cross-attention mechanism CrossAttn and Perceiver clearly exhibited best performance.
models. All models converged to greater
Generalization performance. While all base models performed IID generalization well, only mod- els that contained cross-attention mechanisms (CrossAttn and Perceiver models) exhibited excellent OOD distractor generalization (Fig. 3b). A related result also reported in Qiu et al. (2021) using cross-mes modal self-attention. Though the GRU, SSTfmr, and DSTfmr models were able to per- form some degree of OOD generalization (e.g., generalization 10 distractors), performance was markedly reduced as the number of distractors increased from 10 to 20 (Fig. 33d).
3.2 SYSTEMATIC COMPOSITIONAL GENERALIZATION
The evaluation of Transformer models on systematic generalization problems has been of recent interest (Ontañón et al. 2022; Csordás et al. 2022 Hupkes et al.) 2020 Keysers et al. 2020 Dziri tasks in a sin-
al. 2023). However, most evaluations have been limited to srquence-to-squeees gle modality (e.g.. natural language). Here we extend prior work, and provide an evaluation of systematic generalization in the gCOG task using encoder-only architectures.
Experimental setup. Evaluating systematic compositional generalization requires a test set that is novel recombination of previously seen tasks. gCOG, this train/test split can manifest in several ways. The simplest split is to evaluate generalization with objects (e.g., "red b"") that it has not been paired with before. For example, the model was trained on "Exi st blue a" and "Get Location red b". it would have to systematically combine the notion of "red b" with the Exist operator a configuration not seen in the training set (Fig. 4a).
individual task operators (c.g.. Exist)
Additionally, more challenging test of systematicity is to evaluate generalization performance more complex task tree structures. Prior question-ansen -nnweer benchmarks that evaluated systematic generalization typically were limited assessing systematicity individual task operations, rather

---

Published as a conference paper at ICLR 2024
A
Systematicity
Training loss
C
Individual operator generalization
individual operators
2
100
69.8
89.8
100,88.
102
train set
tast set
GRU
08.8
85.2
00.8
Exist
Exist
DSTtmr
Accuracy
80
Loss
85T
A
Crosskthn Perceiver
Get Location
Get Location
B
20
IID
OOD
generalize
0
Training steps 106
RNNN
GRU
DSTfm
SSTfmr
CrossAttn
Percever
Systematicity on
D
E
F
depth tasks
Depth 3 task generalization,
Training loss
train
14KU
PNN
Acouracy
1st
52
2220227
592
66
generalize
44.4
Loss
SSThhr
20
Crosshtt
34.1
Perceiver
test set
0
GOD
Training steps
1e7
RNN
GRU
SSTfmm DSTTmr
CrossAttn
Percen Perceiver
Figure 4: A) Systematicity on individual task operators, where specific objects (e-g., blue "a") are trained on a subset of operators, and then tested on distinct set of operators. This evaluates if the model can generalize to new operator and object combinations. B) Training trajectories. C) CrossAtin and Perceiver-like models exhibit excellent systematicity generalization, while other models performed reduced rates. D) Another test systematicity to train task trees depth and then test on novel combinations task trees of depth 3. E) Training trajectories. All models were able to efficiently learn this task variant. (Note that periodic spikes in the loss function are due to resampling of the training dataset due to model checkpointing and/or disruption compute job.) F) While overall generalization performance is lower (even on IID generalization), cross- attention models still perform systematic compositional generalization well above chance.
than task trees with greater dependencies (e.g., COG, SQOOP; Yang al. 2018;; Bahdanau cet al. [2019 2020). Here, we trained on a subset of task trees depth and 3, and then evaluated performance
novel combination of task structures of depth 3 (Fig. 4d). In this particular train/test split, we ensured that every individual task operator was previously trained on, but that a specific combination of task operators in task tree was novel.
Generalization performance. On the systematicity test with individual task operators (Fig. 41), all models converged the training with greater than 92% performance (Fig. 4b). All models hibited reasonably good performance (>78%; chance=33. 10%) on OOD systematic generalization (Fig. 4 (Note that chance was determined as the average probability of each output classifica- tion given the distribution of the training set.) Across all models, models containing cross-attention mechanisms performed the highest on systematic generalization, with the CrossAttn and Perceiver architectures exhibiting the highest OOD generalization performance (>97%; Fig. 4b).
On systematic generalization on depth 3 tasks (Fig. 4i), IID generalization was markedly reduced across the board, despite convergence on the training set for all models (all models achieved >988% accuracy on the training set) (Fig. fe.f). (Note, however, that increasing depth (encoder layers) to Transformers improves IID generalization on these splits; Fig. ( The Perceiver model outper- formed all other models, exhibiting 75.4% IID systematic generalization, and 65.7% OOD gener- alization. The next best performing models had 59.2% IID generalization performance (CrossAtttn model), and 54.1% OOD generalization performance (RNN). These results suggest that the Per- ceiver best suited for systematic multimodal generalization, indicating its promise generic, amodal architecture for multimodal systematic generalization.
3.3 PRODUCTIVE COMPOSITIONAL GENERALIZATION
Experimental setup. Productive compositional generalization involves generalizing to tasks of greater complexity (e.g., a task tree of depth 3 to task tree of depth 5). We evaluated productive

---

Published as a conference paper at ICLR 2024
generalization in the context of gCOG. We trained all models task trees of depth and depth 3, and then evaluated generalization performance to task trees of depth 5 and depth (Fig. (l)). (We also show in Appendix Fig. hoo models trained only on task tress of depth fail to generalize to task trees of depth 3, 5, and 7.)
A
Productive compositional generalization
depth 3
depth
TEST SET
generalize
depth
TRAIN SET
depth 5
B
C
D
Generalization performance
Training loss
Training accuracy
100
100
(UI)
RNN
2
GRU
(O00)
DSTfmr
Accuracy
Accuracy
7000
Loss
SSTfmr
CrossAttn
Percetver
40
Training steps
10/
Training steps
16/
RNN
GRU
DSTimr
SS
CrossAttn
Perceiver
Figure 5: Productive compositional generalization performance. A) OOD productivity performance of all models to novel tasks of greater complexity (i.c., deeper task trees). We trained models on task trees of depth I and depth 3, and then tested gencralization to task trees of depth 5 and 7. While the B) training loss and C) training accuracy converged for all models, D) all models failed to perform OOD productive compositional generalization more complex task trees. (Note that periodic spikes in the loss function due to resampling the training dataset due model checkpointing and/or disruption to compute job.)
Generalization performance. Though all models converged on the training dataset (Fig 5p,c), all models completely failed to OOD generalize task trees of 5 and 7 (Fig. (01 (performance was at or below chance for all OOD splits). Prior work has shown that transformer-basee models exhibit improved productivity generalization when using relative positional encoding (Csordás et al. 2022 Ontañón et al.] 2022). Though we used absolute positional encodings in our Transformer bbssed models in Fig. we found that using relative positional encodings did not improve productive generalization performance on gCOG (Appendix Fig. TOp. One possible explanation for the disparity between systematic and productive generalization in neural models is that systematicity requires the ability to exchange semantics (or tokens) from known syntactic structure (e.g., tree of certain depth). In contrast, productive gemeralization requires generalizing to an entirely new syntactic structure (e.g., task tree of different size or depth). This requires understanding the syntax how to piece together syntactic structures on-the-fly - requiring another level of abstraction. To our knowledge, there is no mechanism in modern Transformers that would enable this. Thus, productive compositional generalization remains a difficult capability for purely neural models to achieve.
3.4 IMPACT OF LAYER DEPTH AND ATTENTION HEADS ON GENERALIZATION
Our previous experiments evaluated the performance of base neural network models on gCOG. How- ever, these "base" models did not assess the impact of model scale (e.g., depth) on performance. To complement those previous experiments, we evaluated the impact scale (layer depth and attention heads) of a standard Transformer encoder model (e.g., BERT-style, and similar in base architecture to the SSTfmr; Devlin et al. (201)) on generalization. We assessed the influence of encoder layers (1, 2, 3, and 4 layers), and the number of attention heads per encoder layer (1, 4, and 8 heads). We found that increasing encoder layers improves generalization across distractor (Fig. (b..b) and systematic generalization (Fig. dde.gghh). Increasing attention heads per layer also marginally im-
8

---

Published as a conference paper at ICLR 2024
proved distractor and systematic generalization, but to lesser extent than adding layers (Fig. 6p.f.i). Importantly, the largest model we tested (a SERT-small-sizze model; 4 layers and attention heads) demonstrated excellent systematic and distractor generalization. However, model scale failed to il- lustrate any improvement on productive generalization (Fig. k).k. These results demonstrate that increasing scale of standard Transformer architectures may be sufficient for distractor and systematic generalization with a well-designed dataset, but that productive generalization remains significant challenge.
A
Distractor generalization
Enc layers
Attn heads
Accuracy
100
Acourasy
100
77.
50
75
25
000
80
1A
1L
1L4A
188A
21
14
2
AA
21 24
31 31 3A
14
BA
41 14
4l
AL4A
41
4LBA
Layers
2
Heads
D
Depth 1 systematicity
E
Enc layers
Altn heads
Accuracy ob
20.
200
10008
102
DO 130
ST
122
100
10
100
122
Acouracy
000
80
1A1A
1L4A
1L8A
1A
2L4A
2ABA
1
4A
3BA
41
1A
4L4A
41
4LBA
Layers
Heads
Accuracy @ 88 a
Depth systematicity
H
Enc layers
Attn heads
216
21
05.
115
Accuracy
75
50
000
1L
1L4A
1LBA
111 1A
2A4A
4A
2L8A
3L
1A
3A4A
388
41
74
4 4A
488A
2
Layers
Heads
Productivity
K
Accuracy
100
3.ID)
(0003)
Accuracy
Enc layers
35
Attn heads
30
30
20
25
1L1A 1L49 L88A
2L1A
2L4A
2L8A
3L
344A
3LA
4L 1A
444A
4L8A
1A
Layers
Heads
Figure 6: Evaluating generalization splits on BERT-like single-strean stream transformer models with varying layers (L.) and attention heads (A). Overall, increasing layers and attention heads can improve generalization across distractor and systematic generalization, but not productive generalization. A) Evaluation on distractor gener- alization across all model parameters. B) The effect of adding additional encoder layers on distractor general- ization performance (averaged across attention head configurations). C) effect of adding attention heads on distractor generalization performance (averaged across all layer depth configurations). D-F) Evaluation on systematicity for depth I lasks (generalization split in Fig. 4d. G-I) Evaluation systematicity for depth 3 tasks (generalization split in Fig. 41). J-L) Evaluation on productivity split (generalization split Fig. 5h).
CONCLUSION
Identifying neural architectures that can robustly generalize OOD is central goal in artificial intelli- gence. Compositional generalization benchmarks, which explicitly evaluate for generalization, pro- vide good testbed for measuring these capabilities. However, the most successful models for mul- timodal compositional generalization tend to be hybrid neuro-symbolic models rather than purely neural models (Bahdanau al. 2020). While useful for some applications, current neuro-symbolic models require a priori knowledge of what rules and operations to include, making them difficult to train end-to-end, and limiting their broader use and overall scalability. In this study, we sought to evaluate how different architectural mechanisms in purely neural models influence OOD multimodal generalization. We introduced gCOG, which provides explicit OOD generalization splits for generic multimodal reasoning that can be extended in future studies. Our experimental results demonstrate that while current neural models fall short of exhibiting any productive compositional generaliza- tion, increasing layer depths and/or targeted cross attention mechanisms between multiple domains provide paths towards improving systematic and distractor OOD generalization on multimodal tasks. Thus, hope this study inspires future work towards identifying the neural architectures capable of performing multimodal OOD generalization, with the goal advancing the broader reasoning capacities of modern AI systems.

---

Published as a conference paper at ICLR 2024
REPRODUCIBILITY STATEMENT
Code for this paper and dataset can be found at httttt/:://hiittttaaa details regarding the experimental design can be found in Appendix [A.2 Additional details regard-
com/ IBM/ gcog Additional
ing the model architectures and training can be found Appendix A.3 Additional details regarding representation analysis can be found in Appendix A.1
REFERENCES
Jacob Andreas. Measuring Compositionality in Representation Learning, April 2019. URL //axxiv.org/bb//10222 07181 arXiv:1902.0718: [cs, stat].
Dzmitry Bahdanau, Shikhar Murty, Michael Noukhovitch, Thien Huu Nguyen, Harm de Vries, and Aaron Courville. Systematic Generalization: What Is Required and Can It Be Learned?, April 2019.
/1811. 12889
arXiv:181 11tt18811,18888 12899 [cs].
Dzmitry Bahdanau, Harm de Vries, Timothy J. O'Donnell, Shikhar Murty, Philippe Beau- doin, Yoshua Bengio, and Aaron Courville. CLOSURE: Assessing Systematic Generaliza- tion of CLEVR Models, October 2020. URL http:://rxa: wttpv//.rrgab////eeoooooo /atps//991...55 arXiv:1912.05783 [es].
Sébastien Bubeck, Varun Chandrasekaran, Ronen Eldan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Peter Lee, Yin Tat Lee, Yuanzhi Li, Scott Lundberg, Harsha Nori, Hamid Palangi, Marco Tulio Ribeiro, and Yi Zhang. Sparks of Artificial General Intelligence: Early experiments with GPT-4, April 2023. URL http::/aaxxx /ttp///wwwvvoor/aa///3333000 2200 12712 1221: [cs].
Róbert Csordás, Kazuki Irie, and Jürgen Schmidhuber. The Devil is in the Detail: Simple Tricks Improve Systematic Generalization of Transformers, February 2022. URL http://arxiv, org httg/abb/21008 12284 arXiv:2108. 12284 [cs].
Jacob Devlin, Ming-Wes Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding, May 2019. URL http:///rxiv.
org/ bogg/abs/Di taabs/18100 04805 arXiv:1810.0480) [cs].
Nouha Dziri, Ximing Lu, Melanie Sclar, Xiang Lorraine Li, Liwei Jiang, Bill Yuchen Lin, Peter West, Chandra Bhagavatula, Ronan Bras, Jena D. Hwang. Soumya Sanyal, Sean Welleck, Xiang Ren, Allyson Ettinger, Zaid Harchaoui, and Yejin Choi. Faith and Fate: Limits of Trans- formers on Compositionalltt, June 2023. URL http::///aart..orrg/a/////33... arXiv:2305. 18654 [cs].
Daniel Furrer, Mare Zee, Nathan Scales, and Nathanael Schärli. Compositional Generalization in Semantic Parsing: Pre-training vs. Specialized Architectures, September 2021. URL http: 08970 arXiv:2007.08970 [cs].
//arxiv.org/abee www.or//gaaa//eeeeee//..e/eeeeee httpg//bb//00...oo /207.
Cheng-Zhi Anna Huang, Ashish Vaswani, Jakob Uszkoreit, Noam Shazeer, Ian Simon, Cur- tis Hawthorne, Andrew M. Dai, Matthew D. Hoffmman, Monica Dinculescu, and Douglas Eck. Music Transformer, December 2018. URL [cs, eess, stat].
04281
Dieuwke Hupkes, Verna Dankers, Mathijs Mul, and Elia Bruni. Compositionality Decomposed: How Neural Networks Generalise? Journal of Artificial Intelligence Research, 67:757-795, April 2020. ISSN 1076-9757. doi: 10. 10.1613/jair..11161 index. httpe//ppp/aaaaaarrrie//////////
https:/ //w:://wwwwwww jair org/
Andrew Jaegle, Felix Gimeno, Andy Brock, Oriol Vinyals, Andrew Zisserman, and Joao Car- reira. Perceiver: General Perception with Iterative Attention. In Proceedings of the 38th In- ternational Conference on Machine Learning, pp. 16644. PMLR, July 2021. ISSN: 2640-3498.
\/proceed ngs ml www.rr.rrassviie wttps/mmm...eeee////////eeee /
htm
10

---

Published as a conference paper at ICLR 2024
Justin Johnson, Bharath Hariharan, Laurens der Maaten, Li Fei-Fei, C. Lawrence Zitnick, and Ross Girshick. CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Vi- sual Reasoning. In IEEE Conference Computer Vision and Pattern Recognition (CVPR), pp. 1988-1997, Honolulu, HI, July 2017. IERE. ISBN 978-1-5386-045777 doi: 10. 109/CVPR. 2017.215. URL htt..httts:// hhtt:ps///eeexpp: ore ieee org httpee/e..rggeeeeeeeeee/////ee/0
Daniel Keysers, Nathanael Schärli, Nathan Scales, Hylke Buisman, Daniel Furrer, Sergii Kashubin, Nikola Momchev, Danila Sinopalnikov, Lukasz Stafiniak, Tibor Tihon, Dmitry Tsarkov, Xiao Wang, Mare van Zee, and Olivier Bousquet. Measuring Compositional Generalization: A Com- prehensive Method on Realistic Data, June 2020. URL
09713
Xrviv 19: 12.09713 [cs, stat].
Najoung Kim, Tal Linzen. and Paul Smolensky. Uncontrolled Lexical Exposure Leads to Overes- timation of Compositional Generalization in Pretrained Models, December 2022. URL http:
//arxivv.rrg/bb///2222. 10769 arXiv:2212.10769 [es].
Max Klabunde, Tobias Schumacher, Markus Strohmaier, and Florian Lemmerich. Similarity of Neural Network Models: A Survey of Functional and Representational Measures, May 2023. URL
httg//bs/23305. 06329 arXiv:2305.06329 [cs].
Tim Klinger, Qi Liu, Maxwell Crouse, Soham Dan, Parikshit Ram, and Alexander G. Gray. Compositional Program Generation for Systematic Generalization. June 2023. URL https:
//openreview. httperrvviewwee//ooruun...m ceeet/forumiid-iee 10t.forrmmmmddd111000000000----
Nikolaus Kriegeskorte, Marieke Mur, and Peter Bandettini. Representational similarity analy- sis connecting the branches of systems neuroscience. Frontiers in systems neuroscience, (November):4, 2008. ISSN 1662-5137. doi: 00389//ncuro000004...0000
Brenden Lake and Marco Baroni. Generalization without Systematicity: On the Compositional Skills of Sequence-to torgqquneet Recurrent Networks. In International Conference Machine Learming. pp. 2873-2882, PMLR, July 2018. URL http::/ppooceeiiggs mlr. p
v8C/lake18a html
ISSN: 2640-3498.
Ilya Loshchilov and Frank Hutter. Decoupled Weight Decay Regularization, January 2019. URL 05101 arXiv:1711.05 [cs, math].
http://arxiv. http:/aaaiii.rr..r/aaaa/aa
lan R. McKenzie, Alexander Lyzhov, Michael Pieler, Alicia Parrish, Aaron Mueller, Ameya Prabhu, Euan McLean, Aaron Kirtland, Alexis Ross, Alisa Liu, Andrew Gritsevskiy, Daniel Wurgaft, Derik Kauffman, Gabriel Recchia, Jiacheng Liu, Joe Cavanagh, Max Weiss, Sicong Huang, The Floating Droid, Tom Tseng, Tomasz Korbak, Xudong Shen, Yuhui Zhang. Zhengping Zhou, Najoung Kim, Samuel R. Bowman, and Ethan Perez. Inverse Scaling: When Bigger Isn't Better, June 2023. URL BKLL/Ettp///xxxv org/ abs/ 2200/3336.099 09479 urXiv:2306.09477 [cs].
Maxwell I. Nye, Armando Solar-Lezama, Joshua B. Tenenbaum, and Brenden M. Lake. Learning Compositional Rules via Neural Program Synthesis. arXiv:2003.055: 055662 [cs], March 2020. URL
http: org/ abs/ 2003. 05562 arXiv: 2003.05562.
Santiago Ontañón, Joshua Ainslie, Vaclav Cvicek, and Zachary Fisher. Making Transformers
Solve Compositional Tasks, March 2022. URL http: /ttp/vivvvv../////////a/////aavv wwtpp/////aaa.../.a// 04378
arXiv:2108.0 04377 [es].
Linlu Qiu, Hexiang Hu, Bowen Zhang, Peter Shaw, and Fei Sha. Systematic Generalization on What is Nearly Solved and What is Next? In Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih (eds.), Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 2180-2188, Online and Punta Cana, Dominican Republic, November 2021. Association for Computational Linguistics. doi: 10.18653/v1/200 1/201. org/ emnlp main. 166
gSCAN:
emnlp m6in.166. URL.nttps://ccaaattooooo
Laura Ruis, Jacob Andreas, Marco Baroni, Diane Bouchacourt, and Brenden M Lake. A Benchmark for Systematic Generalization in Grounded Language Understanding. In Ad- vances in Neural Information Processing Systems, volume 33, pp. 19861-19872. Curran As- sociates, Inc., 2020. URL ht URL wtweeediiinnnnnninneeee neurips wwwpaaipsscccppeeeeeeeeoooe htt.ppps...c/ppp//2///aaaaaa 901 82cc81 226

---

Published as a conference paper at ICLR 2024
Rylan Schaeffer, Brando Miranda, and Sanmi Koyejo. Are Emergent Abilities of Large Lan- guage Models a Mirage?, May 2023. URL pttt:://axxiv.org/aatt 15004 1tt.:230..150044 [es].
Peter Shaw, Jakob Uszkoreit, and Ashish Vaswani. Self-Attention with Relative Position Repre- sentations, April 2018. URL [es].
02155 arXiv:1803.02155
Peter Shaw, Ming- Wei Chang, Panupong Pasupat, and Kristina Toutanova. Compositional Gener- alization and Natural Language Variation: Can Semantic Parsing Approach Handle Both? In Chengqing Zong. Fei Xia, Wenjie Li, and Roberto Navigli (eds.), Proceedings of the 59th An- mual Meeting of the Association for Computational Linguistics and the 11th International Joint Online, Au-
Conference on Natural Language Processing (Volume 1: Long Papers), pp. gust 2021, Association for Computational Linguistics. doi: 10. N0.18853/v1/2222 lccl-long.75 URL
htt.pp::/aaclammboooooooooo
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention All you Need. In Ad- vances in Neural Information Processing Systems, volume 30. Curran Associates, www.inng...nn.ssseeeeeeeee.re..e neurips occ/papper/221/hast//
Inc., 2017. URL
Chaojun Wang and Rico Sennrich. On Exposure Bias, Hallucination and Domain Shift in Neural Machine Translation, May 2020. URL arXiv:2005.03642 [es].
Taylor Webb, Keith J. Holyoak, and Hongjigg Lu. Emergent analogical reasoning in large
language models. Nature Human Behaviour, pp. 1-16. July 2023. ISSN 2397-3374. doi: 100088884162-02--00000
URL https: http://www..tt.eeeeeeeeee naw.re.ttt....... www.aaateeeeeeaaeeeeeeeeeeeeeeee www. artmees/
641562-023-01655- 659-
Publisher: Nature Publishing Group.
Zhaofeng Wu, Linlu Qiu, Alexis Ross, Ekin Akyürek, Boyuan Chen, Bailin Wang, Najoung Kim, Jacob Andreas, and Yoon Kim. Reasoning or Reciting? Exploring the Capabilities and Limitations of Language Models Through Counterfactual Tasks, August 2023. URL http:
//arxiv. httcxii..org/ab//330... 02477 arXiv:2307.02477 [cs].
Guangyu Robert Yang, Igor Ganichev, Xiao-Jing Wang, Jonathon Shlens, and David Sus- sillo. A Dataset and Architecture for Visual Reasoning with Working Memory. pp. 714- 731, 2018. URL U Guangyu_Robert, Robert_ Yang A dataset and ECCV ECCV_2018 1attl__ppperrraa
pereeaaceesssteecc....ee.eee thecvf. com/ content ECCV 2018/htm1/
12

---

Published as a conference paper at ICLR 2024
A APPENDIX
A.1 REPRESENTATION ANALYSIS OF MODEL ARCHITECTURES
We performed representation analysis to identify the relationship between how a model's archi- tecture constrains its internal representations, and in turn, its generalization behavior. Originally developed to analyze and interpret neuroscience data, representational similarity analysis (RSA) facilitates the interpretability of model's internal actívation patterns (Kriegeskorte al., 2008 Klabunde et al. (023). In brief, RSA summarizes the geometry of representations in a neural net- work layer (or layers) by measuring the similarities (or distances) between the activation vectors during the presentation of different inputs. This explicitly measures the relations between input samples, facilitating insight into how this neural network layer processes different samples. More- over, transforming individual representations into cample-by-sampee matrix of representational similarities enables direct comparison between neural network layers both within and across net- works, since the matrix dimensions can be made equal. (In contrast, layer activations across models or even within models are hard to compare, due to the lack of a I-to-1 mapping of units.) In our specific case, we evaluated how different task features (stimulus input and target representaions) emerge in the penultimate layer of neural network models. This was done by computing the sim- ilarities of stimuli (the vectorized stimulus encoding) and target responses (one-hot codes for each target response), and then computing the distance of these similarity matrices to the representational similarity matrices of the model's penultimate layer.
In our analysis, we sought to understand why models with different architectures (e.g.. the presence of a cross-attention mechanism) better performed OOD distractor generalization. We focused our analysis to the distractor generalization benchmark, since we observed the strongest discrepancy in OOD generalization performance across models. To evaluate what task information was retained within the representations of each model during OOD distractor generalization, we randomly sam- pled 800 task samples with 40 to 50 distractors and quantified the vectorized cosine similarities between every pair of task stimuli (in the input space) (Fig. 7p.. (Note, however, that other distance metries can also be used Klabunde et al.) (2023).) This resulted in a stimulus similarity matrix, from which we could directly compare model's internal representational similarities using the same 800 samples. Next, we measured each model's s representational similarity matrix to the same set of stim- uli using activations in the models' penultimate layer (Fig. 7p.. To measure how much stimulus information the neural network's 's layer retained, we computed the L2 norm between matrices after aligning the two matrices through the orthogonal Procrustes transform. (The same procedure could be applied to measure the representational alignment to target resposne information (Fig. (P).) The orthogonal Procrustes transform between the representational similarity matrix, X, and stimulus (or target) matrix Y, was computed by solving for the orthogonal transformation
Q' = arg min QEO(D)
||XQ - -YIIF
where Q' is the best orthogonal transformation align X and Y. O(D) is the group of orthogonal transformations, and ||- r denotes the Frobenius norm. Then, to compute the alignment A between matrices X and Y, compute
A = |XQ' - -YlIF
We used this distance metric as it has been previously shown to be proper distance metric (i.e., it obeys the triangle inequality; Klabunde et al. (023). However, this measure alone does not control for potential biases in representational distances due architecture. Thus, control for the potential confound that some model architectures have different innate distances, we computed the relative alignment by computing the difference of A before and after training the model. Importantly, relative alignment can be negative, since it provides a metric of how the representations change relative to the randomly initialized model architecture. We use this measure of relative alignment to measure how much stimulus or target response information each model architecture retained in Figure 7 We found that the best performing models (CrossAttn and Perceiver models) on OOD distractor generalization, had the highest alignment to the stimuli, while the lowest performing models (e.g.. RNNs and GRUs), were least aligned to stimulus information (Fig. (7)) We found a similar trend when estimating the model's alignment to the correct response (i.c., target) (Fig. 7 Thus, the best OOD distractor generalization models were able to retain both stimulus and response information in the penultimate internal representations.
13

---

Published as a conference paper at ICLR 2024
B
Model alignment stimulus information
Model alignment target information
Jess alignment
Relative alligmment
Rolative alignment
500
250
compute
Representaation simiarilies
L2 norm
-250
sample
-300
greater alignment
RNN
GRU GRU
DS 1lms
SB ffrrr
CroesAtto AAttn
Perceiver
Gam
De Tims
SiTtmm
GrossAtt
Perceterr
Figure 7: Representation analysis across model architectures. A) To measure task information in cach model's internal representations, we measured the representational similarity matrices of the stimulus (or target) sam- ples, and compared those the models' representational similarity matrices during those tainment stimulus (or target) information norm between model and the task stimulus (or target) matrices. B) Model alignment to stimulus information. Models with cross- coss-attention mechanisms best retained stimulus information in the penultimate layer. C) Model alignment to target (response) information.
samples. Re- be measured as I.2
model's representations could
A.2 ADDITIONAL DETAILS ON TASK DESIGN
gCOG was modified from the original COG task enable the measurement of various OOD bench- marks (Yang et al. 2018). While we employed some similar task operations (c.g.. the Exist oper- ator), we also implemented a number of new task operators that were not in the original dataset. We also lifted the constraint of maximally including only single conditional (i.e., an IF-THEN-ELSS statement, originally referred to as "Switch" clause COG). allowing us to flexibly evaluate pro- ductive compositional generalization. A few example queries from the original COG task include: latest triangle? Point to latest red object. If square exists, then point
"What is the color to the current otherwise point to the last b."
Construction of task samples was algorithmically consistent with how samples were constructed in COG and CLEVR (Johnson et al., (017). Construction of task sample (i.e., task instructions, stimuli, and target) began by first constructing the task instruction, and then determining the specific task path (i.e., which operators will be encountered in the task tree). Depending what the target response is, a backward pass taken through the task path (i.e., bottom-up). Objects are then incrementally added to the stimulus to satisfy all task nodes from the bottom-up. This ensures that the stimulus will necessarily satisfy the chosen task path while guaranteeing a unique solution for unique object or object feature satisfy
each task node. More specifically, there will always task operator. (If the task operator is to Get the Color of the "a", the task algorithm guarantees that there will only be single "a" in the image. If the task operator Does the red "a" exist?, there is guaranteed to be maximum of one red "a".) Additional algorithmic details on task construction can found in Yang et al. (2018) and Johnson et al.] (017)). and the task code will be publicly released. We will additionally release specific benchmarks that were used in this study.
Note that the IID split for each test, it theoretically possible that the model will be tested on sample that it was trained on due to random sampling, though this is highly unlikely: the probability that two identical stimuli are presented is less than 10 14 due to the total number of distractor combinations.
A.2.1 TASK OPERATORS
Exist. The Exist operator is paired with object (i.c., color and shape feature combination), and asks whether that object exists in the stimulus. The correct response returns a boolean (True False). Example: Does the red "a" exist?
GetColor. The GetColor operator is paired with a shape feature, and asks to return the color of that shape. The correct response is color feature (1 out of 10 possible color features). Example: Get the color of the "a". (In the task construction, there is guaranteed to be a unique solution (i.c., one "a") in the image.
14

---

Published as a conference paper at ICLR 2024
GetShape. The GetShape operator is paired with color feature, and asks to return the shape of the object with that color. The correct response is a shape feature (1 out of 26 possible shape attributes). Example: Get the shape of the red object. (In the example, there will always be single red object.)
GetLocation. The GetLocation operator is paired with an object, and asks to return its location. The correct response is the (x. y) coordinates of that object (1 out of 100 possible locations in 10 by 10 grid). Example: Get the location of the red "a".
SumEven. The SumEven operator is paired with an object, and asks whether the sum of its x and y coordinate location is even. The correct response is True/Fa a se boolean. Example: Is the sum of the x and y coordinate values of the red" "a" even?
SumOdd. The SumOdd operator is paired with an object, and asks whether the sum of its and y coordinate location is odd. The correct response is a True/Fa alse boolean. Example: Is the sum of the x and y coordinate values of the red "a" odd?
P ProductEven. The P ProductEven operator is paired with an object, and asks whether the prod- uct of its and y coordinate location is even. The correct response is a True/F alse boolean. Example: ly the product of the x and y coordinate values of the red "a" even?
Productodd. The Poducttdd operator is paired with an object, and asks whether the product of its y coordinate location is odd. The correct response is True/False boolean. Example: Is the product of the x and y coordinate values of the red "a" odd?
Note that some of these operators can be used as terminals, i.e., task tree leaves with no chil- dren. This includes all task operators that do not return a boolean response (GetColor, Get.Shape, and GetLocations This is because boolean response is required to be input to an ME--THEN-BBEE clause.
A.3 ADDITIONAL DETAILS ON MODEL ARCHITECTURES AND TRAINING
Code associated with model architectures can be found here: https: petps://githute com/ IBM/ gcog Models were constructed using PyTorch version 2.0.0+cu 18. All models could be trained in under three days on an NVIDIA K80 GPU, and were trained on IBM's Cognitive Compute Cluster.
For a specific evaluation benchmark (e.g., distractor generalization), all models were trained on exactly the same number of samples (and training steps). This made it possible fairly evaluate and compare performance of different models. For distractor generalization (Fig. 33) all models were trained on 53, 33,9800000 samples. For systematic generalization on individual operators (Fig- 44), all models were trained on 47,980,000 samples. For systematic generalization on depth 3 tasks (Fig. 4J), all models were trained on 53,980,000 samples. For productive generalization (training on tasks with depth I and 3; Fig. 5) all models were trained on 59,980,000 samples.
All models were trained using the Adam W optimizer with a learning rate of 0.0001, and the loss was computed as the Cross Entropy between the target class and output vector.
A.3.1 MODEL ARCHITECTURES
The inputs to the models comprised of rule tokens (i.e., task instructions) and stimulus. For Transformer-based models (SSTfmr, DSTfmr, CrossAttn, and Perceiver) the 10x10 stimulus grid was presented as sequence of 100 tokens with absolute positional encoding. Each stimulus to- ken had three features (i.e., embedding dimensions) associated with it: color, shape, and a separate dimension indicating if the token flattened into an array (100 x 3). Rule tokens were embedded into a feature space containing an dimension for the task operator (e.g., Exist, GetCol or, etc.), and separate dimensions for the operator's object(s) features (c.g., color, shape, etc.). All models encoded rule tokens as sequence.
EOS token. For RNN and GRU models, stimuli were
RNN. Inputs to the RNN were a sequence of rule tokens and flattened stimulus vector, which were projected to 512 hidden units. Hidden unit activity, he was determined by the equation
h+ tanh(((n W.. br.t.) (I., WTT + by.n)) h4- W.T. + bub
15

---

Published as a conference paper at ICLR 2024
Example of task trees of varying depth
Depth 1
Depth
gelshape
getcolor
saament losee
red ?
object
Depth
0092
K-hon
prod comment green
cummnn yellow
i-hhn
5-twin
E-ther
gelodor
"adahape surrer
Detshaee summnt sépect
gensolor curvent
Surreneo cumrent
curmont yollow
Depth 7
edst carmen purple
i-che
cotet cument blue
OUITETE purple
i--hen
i6-hee
Prouutiwwn current green
surrent brown
ourTent orange 'x"?
current green
i1-taan
itthen
gotshape
Ddudddod
actcoror
productocd countyylllo
petshape surrent
Figure 8: Example task trees depth 1, and ourrent can be paired with up to 260 unique objects (26 letters 10 colors). However, some operators can only be used as tree leaves (i.c., they have no children the task tree). These include GetColor GetShape, and Get locat i on. This is because if task operator has child which is if-else-then clause, the task operator required return boolean.
Guiment
current
current design, there are task operators that
where 2r,0, WT. r.h+ and br,h represented the rule inputs, weights, and biases respectively, and 2,, WT. and be,n the stimulus inputs, weights, and biases. The stimulus vector was presented every time rule token was processed. This avoided the need for the RNN to "remember" the stimulus. Hidden units were initialized as a vector of zeros prior to each trial. The RNN hidden activation patterns were processed through LayerNorm after every iteration, and then subsequently projected to the output layer (with 138 units) for classification. A Softmax was applied prior to supervised training of the outputs. Model training was performed with the Adam optimizer at a learning rate of 0.0001 (Loshchilov & Hutter 2019), and the loss was computed as the Cross Entropy between the target class and the output vector.
GRU. Architecturally, the GRU was structured identically to the RNN. Input and outputs were for- matted and projected identically, and the placement of LayerNorms remained the same. The primary distinction the generation of the hidden unit activity he which determined by the equation
h4 (1 24) n4 24 3 h
16

---

Published as a conference paper at ICLR 2024
where ne. Res and rt correspond to the new, update, and reset gates, respectively, and are defined by the equations 14 = tanh(WWint + bin + rt O
(OW'WWn (Whnhe + bna))
24 = oWWi + biz + Whzhh-1 + bnæ))
F4 = o- + byr + Whhhh-1++ + bbr))
where is the sigmoid function, and @ is the Hadamard product. As in the case of the RNN, the GRU model was trained with the AdamW optimizer with a learning rate of 0.0001, and the loss was computed as the Cross Entropy between the target class and the output vector.
SSTfmr. Rule inputs the the SSTfmr were identical to the RNN, i.c., a sequence of rule tokens. Stimulus input was represented as sequence of tokens. Specifically, each location of the stim- ulus (i.e., each (x.y) coordinate) corresponded to a single token. Since there were 10 rows and 10 columns, there were total of 100 tokens. We zero- zero-padded concatenated both the rule tokens and the stimulus tokens into a single context window with zero-padding. Thus, the total number of input to- kens corresponded the sum of rule and stimulus tokens. The architecture of the SSTfmr followed a standard bidirectional Encoder-only Transformer, and is architecturally similar to BERT (Devlin) al. 2019) (without masking). Since multimodal inputs were concatenated before being fed into the SSTimr, self- self-attention was technically cross-modal. Note, however, that cross-modal self-attention distinct from cross-attention, since self-attention always produces a square (quadratic) attention matrix, while cross-attention can produce square matrix, depending on the number of tokens in the query matrix and the number tokens in the key and value matrices. Input tokens were linearly embedded into vector size of 256. For simplicity, we only included a single Transformer layer, and each layer only included a single attention head, though we do a more extensive evaluation of this type of architecture in Figure 6. We evaluated models with both absolute positional encoding (following Vaswani et al. (2077) and relative positional encoding (following [hhaw et al. (2018). The position-wist MLP portion of the Transformer block was 2-layer MIP with 512 units in each layer. The output of the Transformer block was subsequently processed through a 3-layer feedfor- ward MLP (512, 1024, and 512 units per layer). We applied a LayerNorm prior to projecting the activity to the output layer for classification. Model training was performed with the Adam W op- timizer at a learning rate of 0.0001, and the loss was computed as the Cross Entropy between the target class and the output vector.
The DSTfmr was urchitecturally similar to the SSTfmr, except rule and stimulus to- kens were processed independently in separate (and in parallel) encoder-only Transformer blocks, followed by separate 3-layer MI.Ps. (No zero- pro-padded concatenation was applied to the rule and stimulus tokens.) The output of the parallel MLPs (Fig. (2)) were then summed together, processed through a LayerNorm, and then passed through a 3-layer MLP (512, 1024, and 512 units per layer). Model training was performed with the AdamW optimizer at learning rate of 0.0001, and the loss was computed as the Cross Entropy between the target class and the output vector.
DSTfmr.
Inputs the CrossAttn model were processed identically to the DSTfmr model, i.e., in dual stream encoder-only Transformer blocks. Cross-attention was then computed from the outputs of the two Transformer blocks. Specifically, the query was computed from the output of the rule Transformer block, and the keys and values were computed from the output of the stimulus Trans- former block (Fig. 2). The cross-attention output was then processed through a LayerNorm (with skip connection from the rule Transformer block output), and a 2-layer MLP (both with 512 units). After a final LayerNorm, the activations were linearly projected training was performed with the Adam W optimizer at a learning rate of 0.0001, and the loss was computed as the Cross Entropy between the target class and the output vector.
CrossAttn.
the classification layer. Model
Perceiver. Inputs to the Perceiver- model were processed identically to the DSTfmr model, i.c., in dual stream encoder-only Transformer blocks. Outputs of the dual-stream Transformer blocks were processed through separate cross-attention mechanisms with the latent Transformer (Fig. ( The processing of inputs to the latent Transformer followed the structure of the Perceiver model (aegle et al. 2021). In our specific case, the latent Transformer contained 256 latent units, and was initialized to zero at the start of every trial. We computed cross-attention between the latent Transformer and the output of the rule Transformer first, and then between the latent Transformer and the output of the stimulus Transformer. Cross-attention involved computing the query from the latent units, and the keys and values from the modality Transformers. LayerNorms and
17

---

Published as a conference paper at ICLR 2024
residual connections were computed after every cross-attention computation, followed by self- attention computation. After self-attention was computed after integrating information, the latent activations were projected to the output layer for classification.
BERT-like SSTfmr. To assess the role of Transformer depth and attention head number in gen- eralization splits. performed additional experiments
range of BERT-like SSTfmr models (Fig. 6). This included systematically varying number of Transformer layers (1, 2, 3, and 4 layers), and attention heads per layer (I, 4, and 8 attention heads). The model with 4 encoder layers and 8 attention heads was architecturally identical to BERT-small, with the exception that the embed- ding dimensionality was limited to 256 (rather than 512). Moreover, unlike the SSTimr used in the main manuscript (described above), we removed the additional MLP that was included on top of the Transformer's encoder layer. The outputs of the final encoder layer were projected to the ouput layer for classification (using just linear layer with Softmax). For the experiments included in Fig. 6 all models used relative positional encoding (Shaw al,, 2018). BERT-like SSTfmr models were trained on a total of 11,980,000 samples before test set evaluation.
A.4 SUPPLEMENTARY RESULTS
Systematicity
depth tasks
Depth task generalization
Training loss
81.6
frain set
60
56
generalize
DSTTer
GRU
Aocuracy
45.4
38.2
$7.2
Loss
Chood//t Perceioo
test set
OOD
Training steps 1e7
RNN
GRU
DST
CrossAtin SSTImr
Perceiver
Figure 9: Systematic compositional generalization performance using relative positional encoding (haa et al.
2018 Huang al.) 2018) A) Systematicity evaluation task of depth 3. B) Training trajectories. C)
While we observe overall similar generalization performance patterns as observed using absolute positional en-
coding, we see some reduction in performance in the cross-attention models using relative positional encoding.
18

---

Published as a conference paper at ICLR 2024
A
Productive compositional generalization
depth 3
depth
generalize
TEST SET
depth
TRAIN SET
depth 5
B
C
D
Generalization performance
Training loss
Training accuracy
100
(UID)
RNN
GRU
80
(OOD)
DSTfmr
Accuracy
Accuracy
(000)
Logg
SSTImr
80
CrossAttn
Perceiver
20
Training steps
1c7
Training steps
167
RNN
GRU
DSTfmr
SSTImr Timr
CrossAftn
Percelver
Figure Productive compositional generalization performance using relative positional encoding (Sha et al. 2018 Huang et al. 2018) A) OOD productivity performance of all models to novel tasks of greater complexity (i.c., deeper task trees). We trained models on task trees of depth and depth 3, and then tested generalization task tress of depth 5 and 7. While the B) training loss and C) training accuracy converged for all models, D) all models failed perform OOD productive compositional generalization more complex task trees. The results with relative positional encoding are overall similar the results with absolute positional encoding (Fig.
A
Productive compositional generalization
depth
depth 1
TEST SET
generalize
depth
depth
TRAIN SET
B
Generalization performance
Training loss
Training accuracy
100
RNN CRU
80
DSTimr
Accuracy
Accuracy
(IID)
Lo8g
SSTire Time
80
(000)
CrossAttn
(OOD)
Perceiver
40
(OO0)
0.0
255
5.0
0.0
2.5
50
Training steps
1e6
Training steps
1e6
GRU
DSTfmr
SSTfm
CrossAttn
Perceiver
Figure 11: Productive compositional generalization performance, training on task trees of depth 1 (absolute positional encoding), and testing task trees of depth 3, 5, and 7. A) contrast to Figures 5 and 10 exclusively train on task trees of depth 1, and then assess generalization to task tress of depth 3. 5. and (i.e.. deeper task trees). Evaluation on task trees of 3, 5. and are therefore OOD. While the B) training loss and C) training accuracy converged for all models (>> 94%), D) all models performed poorly on OOD productive compositional generalization.
19


Reviews:
[{"review_id": "umBGrmnYm6", "reviewer": "Reviewer_DJb6", "summary": "The paper introduces a new multimodal question answering benchmark for out-of-distribution generalization, specifically covering task compositionality, robustness to distractors and combinatorial generalization. It uses this benchmark to evaluate various models and analyze their performance.", "questions": "- **COG task**: It will be useful to discuss the COG task (rather than just mentioning it) before describing the new gCOG one, so that it will be clearer to the reader what are new contributions of the new benchmark compared to COG and the degree of their importance. In the overview diagram I would also recommend showing a sample also from COG to make the differences clearer. \n- **Grid size / generalization**: It could be interesting to vary the size of the grid in training/evaluation and study its impact on model’s performance. \n- **Terminology**: I recommend changing the phrase “Distractor generalization” to one that better conveys it’s about changing the answer distribution. Maybe e.g. answer distribution shift. I also recommend changing the name “Systematic compositional generalization” to “combinatorial generalization”, to emphasize that the main point is the generalization to permutation, and also to better contrast it with the following “Productive generalization” (which could also be systematic).\n- **Figures**: Would be good to increase the size of the plots in Figure 3b. It will also be good the increase the distance and visual separation between the sub-figures in each figure throughout the paper. \n- In the introduction: “multimodal question-answer” -> “answering”.\n- “This design allowed us” -> “This design allow us”.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- **Topic**: The paper studies an important topic which in my opinion is underexplored in current deep learning research. Especially given the tendency these days to scale training up to vast amounts of data, I believe it is particularly important to design carefully controlled benchmarks that can: evaluate the model’s performance from a critical and cautious standpoint, point to their fundamental limitations (e.g. systematic generalization), and support further research about ways to overcome these.  \n- **Evaluation**: The paper offers both extensive extrinsic evaluation, with performance comparison of various models on the different generalization skills, as well as intrinsic analysis of their internal representations’ degree of alignment to the stimuli.\n- **Clarity**: The writing quality is good and the paper is clear and easy to follow. The paper is well-organized, claims and findings are clearly stated, and useful figures and diagrams are provided.\n- **Related Works**: It does a good job in providing the relevant context, motivation and related works. \n- **Contribution**: The empirical findings of the paper on the benefits and limitations of different inductive biases such as recurrent and attention-based are important and may be of broad interest to the community.", "weaknesses": "- **Pre-trained models** The paper focuses on models trained from scratch rather than pre-trained. This could be a strength and a weakness. On the one hand, it allows for isolating the contribution of the architectural choices from other factors of optimization, and training data. On the other hand, it has been observed that by training models at large enough scales enables the emergence of generalization capabilities, which we don’t see in smaller scales. I think it will be critical to also analyze the performance of pretrained models on the benchmark, in order to strengthen the paper.\n- **Visual Simplicity**: The visual side of the benchmark is quite rudimentary, featuring colorful letters. Extending it to a larger range of visual tokens/objects, that could have more than one property (color), and a broader set of elements and variations (than 26 letters), could be a straightforward extension that could help make it a bit more challenging visually.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "WaCqOkvd4I", "reviewer": "Reviewer_a4Su", "summary": "This paper proposes a new benchmark for assessing various forms of generalization in a multimodal setting named gCOG. The dataset includes several different splits intended to measure different aspects of generalization. The paper also compares several different model architectures on the dataset.", "questions": "* Why not try more layers and attention heads, e.g. following a standard hyperparameter setting for model size such as those of BERT-Base? Or even BERT-Small?\n* In Figure 2 (F) why does the single-stream Transformer have almost double the parameters of the double stream Transformer? For the other Transformers, do the encoder blocks used for the task vector and stimulus vector share parameters? \n* What optimizer and hyperparameters (e.g. learning rate) were used for training? How were these chosen? I didn't see these details in Appendix A.2. \n* Position embeddings - Since you are representing 10x10 grids as 1D sequences, 1D relative positions may not capture this structure well. On the other hand, absolute position embeddings seem potentially problematic in the case of the SSTrfmr model, since they will not be consistently assigned to the same grid position if the text sequence is first and has varying length. Mitigating this may be important to provide for a fairer comparison with the SSTrfmr model.\n* To what do you attribute the periodic loss spikes during training that are shown in Figure 4 (E)?\n* I found the usage of \"cross-attention\" a bit confusing. For example, the single stream Transformer features cross-modal attention as an implicit consequence of self-attention over the concatenated sequence. I thought this would commonly be referred to as an instance of \"cross-attention\" between modalities. \n* Does the dataset also contain visual renderings and natural language instructions to enable future work to study these tasks using familiar modalities?", "limitations": "", "rating": 3, "confidence": 3, "soundness": 1, "presentation": 2, "contribution": 2, "strengths": "* The paper introduces a new dataset, gCOG. While the dataset is conceptually similar to those from prior work, such as gSCAN, it supports different types of contexts and instruction types, including more compositional instructions. I'm aware of some prior work (e.g. [1], [2]) that studied compositional generalization in natural language tasks and found that gains on one synthetic task did not always transfer to other tasks, so increasing the diversity of such benchmarks for assessing compositional generalization and related challenges in the multimodal setting could be a potentially valuable contribution.\n\n[1] https://arxiv.org/abs/2007.08970\n[2] https://aclanthology.org/2021.acl-long.75/", "weaknesses": "* I'm concerned about the strength of the baselines used in the paper (see my related questions below). While the primary contribution of the paper is the dataset, it is also important to establish strong baselines for this new dataset and to ensure that the conclusions from the empirical results are valid. The appendix states that only a *single Transformer layer* with a *single attention head* was used. This is almost certainly not an optimal depth and number of attention heads. Relatedly, it looks like some models are potentially underfit, according to the figures. With >5M training examples and a relatively simple input space, I would have expected a reasonably sized Transformer model to achieve low training loss and reasonable IID generalization. If these models could have been applied to similar tasks such as gSCAN (even using symbolic tokens to represent the scene context), where they could be compared with comparable baselines from prior work, this would have helped establish that these are indeed reasonably strong baselines that have been well tuned.\n* The qualitative difference between gCOG and datasets from prior work such as gSCAN was not very clearly described. For example, one of the key claims seemed to be gCOG \"employs generic feature sets that are not tied to any specific modality\". However, it seems like it is a useful property for a multimodal dataset to have a clear relation to real-world multimodal tasks. Indeed, the authors provide interpretations of their tasks in the form of natural language instructions and visual scenes (e.g. in Figure 1), and these are very useful for understanding the task. Representing this dataset using familiar modalities (e.g. vision, natural language) could enable future work to study different research questions, e.g. the impact of pre-training. The ability to alternatively represent the task input as a sequence of tokens is also reasonable for studying certain research questions, but this also seems possible for datasets from prior work. For example, I understand that gSCAN includes both symbolic descriptions as well as visual renderings. Anyways, I think clarifying the motivation for this dataset (e.g. increasing diversity of available benchmarks, focusing on different generalization challenges, etc.) separately from how inputs are represented for the experiments in this paper (e.g. token sequence vs. images and natural language) would be useful.\n* Some of the main empirical conclusions (e.g. that generalization to greater \"depth\" is challenging for models such as Transformers) are generally known from prior work.\n\nnits:\n* Introduction paragraph 1 - \"on a carefully controlled generic multimodal reasoning tasks\" -> \"on carefully...\" or \"...task\"\n* Appendix A.2.1 - Maybe reference Tables 8 and 9 where you discuss different positional embeddings.\n* Consider discussing [3] in related work. [3] demonstrated the importance of cross-modal attention for gSCAN, and similarly studied the relative difficulty of various aspects of generalization, including distractors.\n\n[3] https://aclanthology.org/2021.emnlp-main.166/", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "c4bD4kpXHW", "reviewer": "Reviewer_skmj", "summary": "The paper studies multi-modal generalization in neural networks such as transformer-based models and recurrent networks. To do so, the authors propose Genertic COG, a modular benchmark with multi-modal splits to test for 3 types of generalization: 1) distractor (generalization to different noise distribution), 2) systemic compositional (generalization to new permutation of task structures) and 3) productive compositional (generalization to tasks of greater complexity) generalization. Experiments conducted by the authors showed that while cross-attention based transformers (e.g. CrossAttn and Perceiver) outperform other models and perform well on distractor and systemic compositional generalization, they fail at productive generalization when the depth of the task tree goes to out-of-distribution (>3). Representational analysis is done to show that cross-attention based transformers (e.g. CrossAttn and Perceiver) superior performance on distractor generalization might be due to their ability to better retain task-relevant (e.g. stimulus and response) information at the penultimate layer.", "questions": "What is the key architectural difference between dual stream transformer and transformers with cross attn that can explain their generalization performance? Is it only the lack of a cross attention between the different modalities?\n\nPossible typo:\n“Finally, we included a Perceiver-like model (Jaegle et al., 2021), an architecture designed to generically process multimodal inputs (Fig. 2f).”:  (Fig. 2f) > (Fig. 2e).\n\n\n==Post-Rebuttal==\nI appreciate the authors' response and decided to keep my score.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "+The paper studies a timely and critical question about the generalization capability of multimodal transformer-based models\n\n+The proposed benchmark dataset uncovers a limitation of current multimodal transformer-based models: productive generalization which can facilitate the development of more generalizable transformers/LLMs. \n\n+The paper is generally well-written and easy to follow", "weaknesses": "-While the paper’s studies show that certain designs (e.g. cross-attention) seem to confer multi-modal generalization, there are still some key questions that can be more thoroughly studied to uncover the reasons why this is the case.\n\n-Similarly, important discussions such as why the (cross-attention) transformers might fail at productive generalization is lacking.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)