Compositional generalization through abstract representations in human and artificial neural networks
Authors: Takuya Ito
Tim Klinger
Doug H Schultz
John D Murray
Michael Cole
Mattia Rigotti
Venue: NeurIPS 2022 Accept (2022)
Keywords: neuroscience
cognition
compositionality
generalization
neural coding
abstraction
representations
human
fMRI
artificial neural networks

Abstract: Humans have a remarkable ability to rapidly generalize to new tasks that is difficult to reproduce in artificial learning systems.
Compositionality has been proposed as a key mechanism supporting generalization in humans, but evidence of its neural implementation and impact on behavior is still scarce. Here we study the computational properties associated with compositional generalization in both humans and artificial neural networks (ANNs) on a highly compositional task. First, we identified behavioral signatures of compositional generalization in humans, along with their neural correlates using whole-cortex functional magnetic resonance imaging (fMRI) data. Next, we designed pretraining paradigms aided by a procedure we term primitives pretraining to endow compositional task elements into ANNs. We found that ANNs with this prior knowledge had greater correspondence with human behavior and neural compositional signatures. Importantly, primitives pretraining induced abstract internal representations, excellent zero-shot generalization, and sample-efficient learning. Moreover, it gave rise to a hierarchy of abstract representations that matched human fMRI data, where sensory rule abstractions emerged in early sensory areas, and motor rule abstractions emerged in later motor areas. Our findings give empirical support to the role of compositional generalization in humans behavior, implicate abstract representations as its neural implementation, and illustrate that these representations can be embedded into ANNs by designing simple and efficient pretraining procedures.

Compositional generalization through abstract representations in human and artificial neural networks
Takuya Ito Yale University taku. 1e1itto1100 meo1@gmail.com com
Tim Klinger IBM Research AI tklinger@us ibm com
Douglas H. Schultz
University of Nebraska-Lincoln
dhschultzounl. edu
John D. Murray
Michael W. Cole
Mattia Rigotti
Yale University
Rutgers University mi michael.co coc.aall.oleeruuterss.ee.
IBM Research Al mr 2r2666cco cuneiccoumaba..mm.mmmmam edu
ohn. wwrrmyyyaal...... edu
Abstract
Humans have remarkable ability to rapidly generalize to new tasks that is difficult to reproduce in artificial learning systems. Compositionality has been proposed as a key mechanism supporting generalization in humans, but evidence of its neural implementation and impact on behavior still scarce. Here we study the computa- tional properties associated with compositional generalization in both humans and artificial neural networks (ANNs) on highly compositional task. First, we identi- fied behavioral signatures of compositional generalization in humans, along with their neural correlates using whole-cortex functional magnetic resonance imaging (fMRI) data. Next, designed pretraining paradigms aided by procedure term primitives pretraining to endow compositional task elements into ANNs. We found that ANNs with this prior knowledge had greater correspondence with human behavior and neural compositional signatures. Importantly, primitives pretraining induced abstract internal representations, excellent zero-shot generalization, and sample-efficies learning. Moreover, it gave rise to hierarchy of abstract represen- tations that matched human fMRI data, where sensory rule abstractions emerged in early sensory areas, and motor rule abstractions emerged in later motor areas. Our findings give empirical support to the role of compositional generalization in human behavior, implicate abstract representations as its neural implementation, and illustrate that these representations can be embedded into ANNs by designing simple and efficient pretraining procedures.
Introduction
Humans efficiently transfer prior knowledge to novel contexts, ability commonly referred to as transfer learning- One proposed mechanism underlying transfer learning compositional generalization (or compositional transfer) the ability systematically recompose learned concepts into novel concepts (e.g.. "red" and "apple" can be combined to form the concept of a "red apple")
J 20 Indeed, it has been suggested that an algorithmic implementation of compositional generalization is one of the key missing ingredients that ANN models need in order to achieve human-like learning and reasoning capabilities RCC 28 Therefore, quantifying how compositional generalization is manifested in human behavior and investigating its underlying implementation in biological brains is natural first step to harness and deploy it in machine learning models.
Work done intern at IBM Research Al
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

Recent studies that investigated compositionality in machine learning have typically relied on architec- tures comprised of specialized modules. For instance, disentangled representation learning separates the independent factors underlying the structure of the input data into disjoint components of the
feature vector [7 18 35 16.. Program synthesis methods achieve state-of-the-an performance on systematic generalization 20 through model architectures built by combining specialized neural and
symbolic program modules interacting to search over a space of valid production rules 2 37 46
Complementing these studies, abstract representations have been recently proposed as vector repre- sentations that reconcile compositional generalization with distributed neural codes 2 In particular, parallel abstract representations - representations with high Parallelism Score as previously de- fined 2 support out-of- context generalization by encoding changes in individual variables as a linear shift in the representations. This notion of abstraction implies that these representations are compositionally additive; novel compositions are encoded as the vector sum of distinct abstract representations. This is similar to how word2vec embeddings solve relational analogy tasks [44] [] and generalizes disentangled representations by allowing for arbitrary affine transformations of disentangled codes. Crucially, this type of representation is operationally defined in way that can be quantified in neuroimaging data by computing the Parallelism Score metric defined in [21. In other words, parallel abstract representations are computationally promising candidate as neural substrate implementing compositional generalization, and are also measurable in the human brain by computing the Parallelism Score across fMRI voxels during neuroimaging experiments.
This work is motivated by the working hypothesis that parallel abstract representations support compositional generalization. Accordingly, we first characterized the behavioral signatures of compositional generalization in a task that systematically varied rule conditions across 64 contexts, showing that humans generalize better to tasks with greater similarity structure to previous tasks. We then analyze fMRI imaging data showing that parallel abstract representations are distributed across the entire cortex in content-specific way during the execution of the compositional task. This supports our working hypothesis that parallel abstract representations may implement compositional generalization in humans. To test this hypothesis in ANNs, we designed a pretraining paradigm for ANNs to emulate humans' prior knowledge about the compositional task elements, finding that ANNs pretrained in this way exhibit 1) more abstract representations, 2) excellent generalization performance, and 3) sample-efficien learning. This finding demonstrated that the degree of abstraction (induced through pretraining) directly impacted zero-shot compositional generalization performance in ANNs. Finally, we find that the layerwise organization of abstract representations in pretrained ANNs recapitulates the content-specifie distribution across the human sensory-to-mont cortical hierarchy. Together, these findings provide empirical evidence for the role of abstract representations in supporting compositional generalization in humans and ANNs.
1.1 Related work
Several recent studies in neuroscience have applied analytic tools to identify the neural basis of rapid generalization in biological neural networks. Such studies employed various measures- cross- condition gemeralization [2 40 ?
state-space projections of task-related compositional codes [51l quantify the generalizability and abstraction of representations. Prior work in neuroscience has primarily evaluated compositionality in limited context settings (e.g.. to 10 contexts), or without manipulating different types of features (e.g., higher-order vs. sensory/motor features). Moreover, these neuroscience studies used simple task paradigms due to limitations in either the model organism (rodents and monkeys are unable to perform complex tasks 2 or to isolate specific types of abstraction in humans (e.g., logical abstractions (40) Here we significantly expand on prior work by using a 64-context compositional task that systematically varies different types of task features (e.g., sensory, motor, and logical rules) to evaluate content-specift abstractions across the entire brain and multilayer ANNs. This work also complements related work in compositional generalization in machine learning 28
43 25), and Parallelism Score
46 20
29
45
50 19. However, those
studies primarily focused on building models that improve on current compositional generalization benchmarks on arbitrarily complex compositional tasks, such as SCAN (8), COG [50]. or GQA
19 Importantly, these studies did not directly benchmark ANN behavior (or representations) against human behavioral and neural data, making a direct comparison difficult. Here we leveraged a non-trivial 64-context compositional paradigm to investigate the representations that facilitate compositional generalization in both humans and ANNs.

---

2 Methods
2.1 The Concrete Permuted Rule Operations (C-PRO) task paradigm
We used the C-PRO paradigm (Fig. Mp during fMRI acquisition and ANN model training. Briefly, the C-PRO paradigm permutes specific task rules from three rule domains (logical decision, sensory semantic, and motor response) to generate dozens of novel task contexts. This creates context-rich dataset in the task context domain. The sensory rule indicates which stimulus feature should be attended to. The logic rule specifies a Boolean operation to be implemented on the stimulus feature set. The motor rule specifies a specific motor action (i.e., a button press with specific finger). One of 256 possible unique stimulus combinations could be presented with each task context. Visual dimensions included either horizontal or vertical bars with either blue or red coloring. Auditory dimensions included continuous (constant) or beeping high or low pitched tones.
A
C-PRO Cognitive Paradigm
Logic Rules
Task
Context 1 description: "If BOTH stimuli are VERTICAL, press your LEFT INDEX finger'
Bath
Both
Sensory nule? 2
Logic nule 1
VERTICAL
3. Either
Motor rule 1
LEFT INDEX
Neither
Answer: TRUE (Left index finger)
Encoding 3925 ms
Delay 1570
ms.
Stimul 2365 ms
Context 64 description: "If NEITHER stimulus is RED, press your LEFT INDEX finger [other finger, same hand false]"
1. Red
Sensory Rules
Vertical
High Pitch
Logic rule 4
Task 64
NEITHER RED LEFT INDEX
4. Constant
Sensory rule
Motor Rules
Motor
1. Loft Index
Encoding 3925
Delay
6280 ms.
Stimuli
(Left middle finger)
Answer: FALSE
Left Middle
Right Index Right Middle
2355
B
Task performance
C
D
E
Logic rules
Sensory rules
Motor rules
100
Accuracy
80
"EHEER" NETHERE
"BOT
VERTICAL HNPITCH
RED
LEFT
EFTINDDX
50
60
90TH
CONSTANT
INDE"" RICHT-MIDLE
Practiced
2-rule T-rule
10
15
15
Novel context
Novel context
Novel context
Task overlap
Figure 1: a) The C-PRO paradigm permutes 12 rules belonging to three different rule domains logical, sensory, and motor gating to generate up to 64 unique contexts. b) Human performance on novel task contexts four practice contexts prior to the test session). Moreover, subjects performed novel task contexts with more rule overlap with practiced contexts at higher accuracy. c-e) Task performance as a function of task trials for each rule (novel contexts only). Consistent with compositional generalization, participants had significant increase in task performance in 10/12 rules, even though each rule was used in a novel context. Shaded area around line plots (c-e) reflects the 95% confidence interval.
significantly lower than practiced contexts (participants were trained on
Each rule domain (logic, sensory, and motor) consisted of 4 specific rules (Fig. [t). A task context is comprised of one rule from each domain, for a total of 64 possible task contexts (4 logic x sensory x motor). Subjects were trained on 4/64 "practiced" contexts prior to the fMRI session. The 4 practiced contexts were selected such that all 12 rules were equally practiced. Subjects' mean performance across all trials was 84% (median=66%; chance=25%). See Appendix A.2 for detail
2.2 The geometry of abstract neural representations
Behavioral signatures of compositional generalization can be investigated by measuring behavioral performance as a function of task composition and prior learning. Neural signatures of generalization can be identified using analysis methods that characterize the geometry of neural activations during task generalization. In particular, prior work proposed the Parallelism Score (PS) [2] as a measure to evaluate the consistency of task variable representation across contexts. Intuitively, PS identifies coding axis (i.e., the parallel displacement vector) across task contexts that aids generalization.
'The 1MRI dataset is publicly available here: https::/ http:://pppeeerrroooogaaaatttatp hrg./aaaassttt//00003330
3

---

We posit that representations with high PS (the specific type of abstract representation we investigate) support compositional generalization in human behavior. We illustrate here how PS is reflected in the geometry of neural representations with respect to the rule domains of the C-PRO task. Let us consider a set of C-PRO contexts with logic rules BOTH or EITHER, and sensory rules with values RED or VERTICAL (Fig. 2 High PS in the logic rule domain indicates that the difference in activation vectors between contexts with BOTH and EITHER rules is the same when paired with either the RED or VERTICAL sensory rules. Thus, change from BOTH to EITHER results in the same parallel change irrespective of the sensory rule (Fig. 2 In contrast to unstructured nigh-dimensional representations (Fig. 2 and see 44 [12), this would afford high generalization, since the effect of changing the logic rule in either sensory rules automatically transfers to the other sensory rule.
A
High dimensional (low generalization)
B
Low dimensional
"Abstract"
(low decodability)
(high generalization/high parallelism)
BOTH RED LMID
EITHER vs. BOTH
EITHER vs. BOTH
BOTH
BOTH VERTICAL LMID
RED LMID
RED VERTICAL
BOTH RED LMID
EITHER LMID
VERTICAL LMID
X
EITHER RED LMID
RED
BOTH VERTICAL
LMID
EITHER VERTICAL
EITHER
X
X
VERTICAL
VERTICAL
LMDD
LMID
Figure 2: Hypothetical geometric configurations of neural activation space for "BOTH vs. EITHER" and "RED vs. VERTICAL" rule contrasts. a) nigh-dimensional representations of task activations lead to low PS (in addition to low generalizability across conditions) of rules. b) Low-dimensional representations lead overall low decodability, but some generalizability (across limited features). c) Parallel Abstract Representation of the neural activations leads to high generalizability.
2.3 Parallelism score
We generalize the definition of PS by [2] to tasks where variables can assume an arbitrary number of values (as opposed being binary) and applied it human fMRI and internal ANN activations. PS is defined as the cosine angle of the coding directions of the same rules in different contexts in the neural activation space (e.g., voxels neurons within a brain region) (Fig. B)-c). A cosine angle close to indicates coding directions that are highly parallel, despite differences in context. We compute the coding angle for specific rule dichotomy (e.g., the coding direction "BOTH" vs. "FITHER") by identifying all pairs of task contexts that had exactly the same secondary (sensory) and tertiary (motor) rules. For each pair, we subtracted the IMRI voxel activation vectors associated with each context to obtain the vector that represented that coding direction (see Fig. Bh. We did this for all other pairs in that coding direction. Defining v; as this coding vector for the ith pair, we computed the PS score for one dichotomy as PSa- TE Eijy cos(E, v, v;)), since there are 16 possible pairs for each rule dichotomy within the C-PRO task. To obtain the PS for a specific rule domain (c.g.. logic, sensory, or motor rules), PS₁ is computed for every coding direction, then averaged (e.g., for logic PS, the average of "BOTH" versus "EITHER". "BOTH" versus "NEITHER". etc.).
Statistical testing was performed using a non- oor-aarmettri procedure, where we shuffled labels within each rule domain 1000 times and re-caleulated PS to produce null distribution. We corrected for multiple comparisons (across brain regions) using non-parametric family-wise error correction (36
2.4
ANN construction and training
The primary ANN architecture had two hidden layers (128 units each) and an output layer that was
comprised of four units that corresponded each motor response (Fig, + see Appendix section A.7 for additional details). Training used a cross-entropy loss function and the Adam optimizer 27 The ANN transformed 28-element input vector into 4-element response vector with the equation Y = fRELU (X, W,, + b,). Weights and biases were initialized from a uniform distribution U(-V1/K, V1/k), where & is the number of input features from the previous layer.

---

A Parallelism Sensory rules Parallelism scores: Logic rules
Parallelism Motor rules
BOTH VERTICAL LMDD
VERTICAL LMID
RED LIND
EITHE RED
BOTH LMID
BOTH RED
PS=0 .99
bg
bg
X
PS+0.96
EITHER LMID
EITHER VERTICAL LMID
PS-0.95
BOTH VERTICAL LMID
EITHER VERTICAL
EITHER LMID
A=a
EITHER RED
mean]]A|]
D
E
F
SENSORY
LOGIC
MOTOR
0.20
0.48
0.50
Parallelism score
Figure 3: a-c) 2-D schematic visualization of PS estimation for the sensory, b) logic. and c) motor rule domains for a specific rule pair (e.g.. RED vs. VERTICAL). Intuitively, PS captures the geometry of the neural activation space (ANN fMRI data) by measuring the cosine angle between two linear decoders trained to distinguish two rule conditions in different task contexts. d-f) PS was calculated for each rule domain for every brain region
| PS was highest in association areas for logic rules, dorsal attention network regions for sensory rules, and somatomotor network for motor rules.
A
B
Task context input units
Motor output units
output
C
C
RIGHT INDEX C
NOT
BOTH
EITHER
RED
VERTICAL
HIGH PITCH
CONSTANT
LEFT MID
INFTX
RIGHT MID
RIGHT INDEX
LEFT MID
INDEX
RIGHT MID
Logic rules
Sensory rules
Motor rules
Sensory stimulus input units
CONSTANT C
CONSTANT L
RED
BLUE
VERTICAL
HORIZONTAL
HIGH PITCH
LOW PITCH
BEEPING
RED
BLUE
VERTICAL
HORIZONTAL
HIGH HIGH PITCH
LOW PITCH
BEEPING
Task
Sensory
1st stimulus
2nd stimulus
context
input
Figure 4: a) ANN architecture. We employed simple architecture with two hidden layers (128 units each) with rectified linear units. b) The ANN's input and output units. Task input units were modeled as one-hot vectors for each rule across all rule domains. Sensory stimulus input units were modeled as categorical one-hot vectors for each stimulus type. Separate vectors were modeled for first and second stimulus presentations. Motor outputs had unit corresponding each motor response. Noise was injected with input features when calculating
reduce the orthogonality of input features.
Training on the C-PRO task was performed in a sequential learning paradigm. mimic the human experiment, an arbitrary set of four practiced contexts was initially selected for training. (This was randomly selected across different ANN initializations.) Then, novel task contexts were incrementally added into the set of training contexts.
5

---

3 Results
3.1 Behavioral signatures of rapid compositional generalization in humans
We evaluated human behavioral compositional generalization by assessing performance on novel contexts in the C-PRO paradigm. Since adult humans have decades of prior knowledge, subjects were able to compositionally generalize to novel task contexts without any training (novel accu- racy=84.17%, chance=25% Wilcoxon signed-rank p=00.0011) However, subjects performed the four practiced contexts better than novel contexts (practiced=87.63) 87.66%% novel=84.177 17%;; p=0.003). We next assessed how performance on novel contexts changed as function of shared rule structure to the practiced contexts. Consistent with compositional transfer of previously learned rules, performance novel task contexts improved function of similarity to the practiced contexts (accuracy, 2-rule overlap=84.88 86%;; 1-rule overlap=83
practiced vs. 2-rule overlap. p=0.008; 2-rule vs. 1-rule overlap, p=0.03; Fig. I)). Though our findings are consistent with compositional transfer, we found that rapid transfer to novel contexts is more difficult. However, we found that increased exposure to specific rules improved performance on subsequent novel contexts using that same rule (all except for the "Both" and "Either" rules, likely due to ceiling effects, "DR-corrected p<0.05; Fig. - This suggests that even though performance in novel contexts is worse than practiced contexts, subjects can improve rule transfer with increased practice (or pretraining).
3.2 Spatial and content-specific topography of abstract representations in human cortex
We extended prior work to identify abstract representations using PS across the entire human cortex We calculated PS for each rule domain separately (Fig. 3a-c) using the vertices/voxels within each parcel (i.e., brain region) as activation vectors. We found topographic differences of sensory, logic. and motor rule abstractions tiled across human cortex (Fig. Bi-r). Specifically, we found that statistically significant sensory rule abstractions were primarily identified in higher order visual areas and the dorsal attention network (i.e., brain areas involved in the top-down selection visual stimuli) (PS of significant regions=O..1 family- -iiie error rule abstractions were more widely distributed, but primarily observed in frontoparietal areas (PS of significant regions=0.22: 0.:2: FWE-corrected p<0.05; Fig. Ge). Motor rule abstractions were primarily localized to somatomotor cortex (PS of significant regions=0.29; FWE-correeted p-c0.05; Fig. 3 Notably, regions with abstract representations form a subset of regions of those that contain rule information using standard decoding methods (SFig. D This ensures that high PS is accompanied by highly decodable representations (i.e., high dimensionality).
p<0.05; Fig. B... Logic
3.3 Embedding prior knowledge into ANNs with simple pretraining tasks
Human behavioral data suggested improved compositional generalization with increased task rule exposure, in addition to the years of "pretraining" from ordinary development (i.c., at least 18+ years). Thus, we sought to evaluate whether embedding prior knowledge of rules could improve compositional generalization in ANNs, while simultaneously investigating how prior knowledge impacts the geometry of ANNs' internal task representations. Given that the C-PRO task was specifically designed as compositional task that conjoined three task rules, we created pretraining paradigms designed to teach ANNs basic rule knowledge (Fig. see Appendix for full description).
We constructed a simple feedforward ANN with two hidden layers (Appendix A.7; Fig. This made it easier investigate the effects of pretraining on internal representations. rather than architectural choices. We designed two pretraining paradigms: Primitives (1-rule) and Simple task (2-rule) pretraining. Primitives pretraining trained on I-rule tasks that focused explicitly on learning the semantics of primitive rule features (Fig. 5p.c). This included distinguishing sensory stimuli, learning motor response mappings (e.g., "left index" rule would lead to a left index response), and abstract logical relations, which involved learning the boolean relations amongst logic rules. Simple task pretraining focused on learning 2-rule conjunctions (i.e., sensory and motor rule pairing logical and sensory rule pairing) (Fig. Lee) Importantly, these pretraining paradigms focused on learning primitive or 2-rule associations that were significantly simpler than the full C-PRO task (3-rule combination).

---

Primitives pretraining
Simple task pretraining
Stimuli
B
"True"
"Left Middle
VERTICAL
RED
INDEX
TTuee"
Instruction
"False"
Instruction
'Leff Index
Response
8OTH
Response
Stimuli
E
Instruction
LEFT INDEX
"Lefl Index
"True"
"False
Response
RIGHT MIDDLE
'Rightt Middlee
RED
"False"
Strmali
Response
Instruction
Hesponse
Figure 5: a) The logic rule primitives task involved teaching boolean relations among different logical operations. For example, when presented with the "BOTH" rule, the task was to distinguish two identical ("True") versus two different ("False"" stimuli (i.e., same vs. different). b) Sensory rules involved mapping sensory rules onto stimulus features. c) Motor rules involved mapping motor rules onto motor output units. d-e) Simple task pretraining (2-rule tasks) designed to teach the model how to perform simple (d) sensorimotor mappings and (e) logical-sensory gatings.
3.4 Pretraining induces abstractions, zero-shot performance, and sample efficiency
We measured the PS in ANNs trained with different pretraining routines: Primitives pretrained, Simple task pretrained, and Combined (Primitives + Simple task pretrained). Each pretraining condition was trained until ANNs achieved 99% accuracy (see A.8 for details). PS was calculated for each rule domain using the ANN's hidden layer activations (see A.9). Pretrained ANNs had higher PS than the Vanilla ANN (Primitives vs. Vanilla, 1(37)=8.46 p=le-11; Combined vs. Vanilla, 1(37)=3.03, p=0.003) (Fig. (l). Moreover, PS increased from Primitives to Simple task pretraining (1(37)=3.91, p=0.0002), though no significant increase in PS was observed in Combined vs. Simple task pretraining.
Vanilla (no pretraining),
Vanilla, t(37)=5.20 5.26, p=lp-=---- p=1c-05; Simple task
A
Overall PS hidden layers afler pretraining
B
Zero-shot performance
C-PRO lask (afler pretraining)
Sample efficiency
0.10
Number of samples
1e6
Accuracy (%)
Parallelism
0.05
0.00
Varilla
4,23x samples
wrivitve pretriliing
Simple task pretraming
-0.05
Combinsd
Pretraining
Pretraining
Vanvlaa
Trained pretraining
Combined
D
PS throughout
C-PRO Generalization performance
F
pretraining epochs
100
Parallelism
0.04
Accuracy (%)
Accuracy (%)
100
50
50
0 000.8 *8*
0.00
70
80
100
150
200
250
300
10
80
10p
180
200
200
300
0.03
Pretraining Epochs
Pretraining Epochs
Average PS
Figure 6: a) PS of hidden units averaged across all rule domains using noisy inputs. Zero-shot learning of all 64 C-PRO contexts. c) Sample efficiency of models (Combined and trained vanilla model were performance-nee matched), Total samples, including pretraining samples (if applicable). d) We computed the average PS of the ANN hidden layers throughout Combined pretraining. (Note that PS values are low due input features containing large amounts of noise.) e) Zero-shot performance to the unseen C-PRO trials throughout pretraining. c) The generalization performance and the PS scores were highly correlated with each other throughout pretraining, indicating close link between abstract representations and improved task generalization.
7

---

We next evaluated the zero-shot performance on the full C-PRO task after pretraining (Fig. 6p). As expected, the Vanilla ANN performed near chance mxcc=23.25% chance=25% one-sided 1(38)= p=0.98). Primitives pretraining marginally improved zero-shot performance (acc=31.511 1(38)=8.09 p<le-9). While Simple task pretraining exhibited significant improvement over Primitives pretrained models (acc=70.57% Simple task vs. Primitives, 1(37)=19.84, p<11e-31, Combined pretraining had excellent zero-shot performance on the entire C-PRO task (acc=92.15% Combined vs. Simple task pretraining, ((37)=10.85 p<1le-16)
We next sought to assess the impact of pretraining on learnin//sample efficiency. We therefore trained a Vanilla network (no pretraining) on 60/64 C-PRO contexts to match the zero-shot performance of the Combined pretraining model (i.e., at least 90% accuracy on the 60 context training set). We found that on the remaining test set (4/64 C-PRO contexts). the Vanilla trained model achieved 96.02% generalization performance, but required up to 4.23x training samples to match the performance of the Combined model (Fig. (G). Critically, the 4.23x more training samples included all possible samples (pretraining and C-PRO samples). Thus, pretraining afforded both zero-shot generalization and sample efficient learning.
Finally, we measured PS and generalization performance throughout Combined pretraining. We found that PS and zero-shot performance increased with pretraining (Fig. fil,e), and were highly correlated (r=0.88; p< 1<100--9 Fig. 6b. This illustrated that the abstract representations learned during pretraining directly facilitated zero-shot generalization, and is consistent with prior work demonstrating that the dimensionality of hidden representations altered throughout training t0
3.5
ANN pretraining leads to human-like compositional generalization
We evaluated the learning and generalization dynamics of ANNs with and without pretraining, after training ANNs on 4 of the full C-PRO contexts. This matched the human experiment, since humans were exposed to 4 "practice" contexts prior to performing the remaining 60 novel contexts (Fig. (D). (ANN training was stopped after achieving 90% performance on the 4 practiced contexts.) We found overall poor generalization on novel task contexts in the Vanilla model (accuracy, practiced=94.37% novel=28. p<0.0001; Fig. (7)). This suggested that ANNs with no prior knowledge cannot compositionally generalize. We subsequently compared generalization performance on ANNs after pretraining. We found that with Primitives pretraining, generalization performance significantly improved (57.97%; Fig, (Z)). We observed additional improvements with Simple task pretraining (86.79%; Fig. 7 achieving himan-like generalization performance (Fig. 7t).
A
B
C
Primitives pretraining
Simple task pretraining
Generalization performance
Vanilla
Practiced novel
***
Accuracy (%)
100
100
100
Difference (%)
50
general zution
50
VAN
50
50
Chance
Chance
Chance
Perfect generalizaion
Practiced
2-nule
1-rulo
Practiced
2-rule
1-tulo
Practiced
2-rule
1-nulo
Vanilla
Primitives
Simple
Human
Rule overlap
Human / madel type
Figure 7: a-c) We trained the ANN architecture on 4/64 C-PRO task contexts with (a) a Vanilla ANN with randomly initialized weights, (b) an ANN after Primitives pretraining, and (c) an ANN after Simple task pretraining. Training on the 4 practiced contexts was stopped after the model achieved 90% accuracy on those contexts. As in our empirical human data (Fig. [D), the ANN performed best on contexts in which it had previously seen (Practiced), followed by contexts with two overlapping (2-rule) and one overlapping (1-rule) rules. d) Generalization performance of ANNs with Primitives and Simple task pretraining was most similar to human generalization performance in novel contexts. (Lower values indicate better generalization.)
We next incrementally trained all ANN models on novel contexts, by adding one novel context into the training set at a time. We tested generalization performance on the held-out (test set) contexts until ANNs were trained on 63/64 contexts (SFig. 2 Generalization performance novel contexts
8

---

was significantly higher in ANNs with either pretraining routine (SFig. 2D). This was despite the fact that all ANNs had the same stopping criteria (i.e., 90% accuracy on the C-PRO training set). We ran an additional experiment where each of the ANNs were shown fixed number of C-PRO task samples during training, replicating our core finding (SFig. B. These findings suggest that the inductive biases formed during pretraining significantly improve downstream generalization performance.
3.6 Pretraining ANNS facilitates sample-efficient learning throughout novel task learning
We sought to evaluate how pretraining impacted sample efficiency. We found that pretrained ANNs became more sample efficient the training set expanded, even after accounting for total number of (pretraining and C-PRO) samples (SFig. (2). We quantified the generalization performance to sample efficiency ratio the generalization inefficiency, finding that after learning only C-PRO contexts, vanilla ANNs generalized worse than pretrained ANNs (SFig. 2 These findings support the notion that pretraining can simultaneously improve compositional generalization and sample efficiency.
3.7 Convergent hierarchy of abstract representations in humans and ANNS
Analysis of human fMRI data revealed that content-speciff abstraction was spatially heterogeneous across cortex. Recent neuroscience work has identified hierarchical gradients that organize along a sensory-to-me to-tmooor output axis in both resting-state 32] and multi-task fMRI data [21]. We therefore sought to quantify PS across this sensory-to-motor hierarchy in fMRI data, and compare changes in the feedforward hierarchy (i.e., layer-depth) in ANNs. We focused our analyses on the Combined pretrained model (which incorporates both Primitives and Simple task pretraining) due to its excellent zero-shot generalization (Fig. pp). In addition, extended model to include three hidden layers to make it easier to compare PS of different hidden-layer depths to the three cortical systems of interest: sensory, association, and motor systems (Fig.
A Sensory-to-motor hierarchy
B
PS across the sensory-motor hierarchy
C
PS ANN hidden layers
1-13 72
1:11.02 am
Sermorry Assooiation
peteo5
1.5
05 p-
1:
1:3.07
Normalized PS
1.0
Mokor
pHo.0035
p-o 2002 pcle-5
1:3.79
2-4.84
1.0
p-le-
0.5
0.5
0.0
Sensory "Layer
Association "Layer 2 'Layer 3"
Motar
Sensory
Logic
Motor
Sensory
Logic
Molo-
Rule domains
Rule domains
Figure 8: a) discretized sensory-to-mont motor hierarchy (see SFig. for discretization details). b) We computed the normalized PS (i.e., the PS of each brain region normalized by the maximum PS across all regions) for each rule domain across the discretized cortical systems. c) Same analysis as in b), but using the PS found in each ANN hidden layer.
We measured the PS for each rule domain for sensory, association, and motor systems. Sensory rule PS was highest in the sensory system, logic rule PS was highest in association systems, and motor rule PS was highest in the motor system (Fig. Bp). To observe whether similar hierarchical PS organization emerged in ANNs, we used the Combined pretrained model with three hidden layers, and plotted PS function of ANN depth. Since our ANN transformed sensory inputs into motor outputs, we analogized each ANN layer to the sensory. association, and motor cortical systems (Fig. [R)). We found a similar pattern in the ANN: sensory PS peaked in the first hidden layer; logic PS peaked in the second hidden layer, and motor PS peaked in the last two hidden layers (Fig. Rt). We corroborated these findings using a continuous sensory-motor hierarchical gradient map (without discretization) (SFig. 5 5 (In addition, see SFig. 7 for PS scores for all possible rule dichotomies
in human fMRI data and ANN activations.) These findings suggest that abstraction emerges as function of rule-dependent specialization and hierarchical organization.

---

4 Discussion, Limitations, Conclusions
We provide empirical support for the role of compositionality in human generalization, and implicate abstract representations as its neural implementation. In classic ANNs, which are known to perform poorly during systematic generalization [20 ( we found that computationally cheap pretraining paradigms embedded abstract representations that led to human-like generalization performance and sample efficient learning. When mapping abstract representations across cortex and ANN layers, we found converging patterns of rule-specific abstractions from early sensory areas/layers to late motor areas/layers across human and ANN hicrarchies. These results reveal the hierarchical organization of content-specific abstractions in the human brain and ANNs, while revealing the impact of these abstractions for compositional generalization in models.
Our pretraining approach directly leverages knowledge of task structure to design pretraining routines that embed task biases into ANNs Despite the sample efficiency of this approach, this pretraining approach requires the initial overhead of designing paradigms useful for downstream learning. related approach that similarly requires prior knowledge of task structure is "representational backpropagation" a regularization approach that aims to produce an idealized hidden representation 26.. However, there are other inductive bias approaches that do not require prior task knowledge. One approach constrains ANNs to produce abstract task representations by initializing ANN weights from a low-norm distribution
S However, initializing ANN weights this regime is computationally costly. Another approach is to initialize networks with built-in modular structures to facilitate the However, exactly how such networks disentangle representations has not yet been explored. Nevertheless, all these approaches are complementary to each other. will be important for future work to assess how these approaches may synergistically interact to optimize for sample-efficient generalization in multi-task settings.
re-use of network modules across tasks [3 4
Though we provide comprehensive evidence of the role of abstraction compositional generalization, there are several limitations in the present study that future research can explore. We found that the spatial topography of abstract representations was highly content-dependee were limited to cross-context manipulations of limited rule types (sensory, logic, and motor gating), without addressing the organization of other task components (e.g., reward or stimuli). Thus, future studies can explore how brains and ANNs represent the abstraction of other task components. Second, though we were able to explore cross-conted generalization across 64 contexts significantly more than previous empirical studies in neuroscience task type (i.e., the C-PRO paradigm). It will be critical to see the organization of abstraction in multi-task settings that go beyond 64 contexts. Finally, our ANN modeling approach revealed the computational benefits of pretraining. It will be important for future work to benchmark sample efficiency and generalization performance against other training paradigms (c.g.. in continual learning and/or meta-learning settings; [15 49
However, analyses
cross-contest analysis was limited to single
In conclusion, we characterized convergent hierarchical organization of abstract representations across the human cortex and in ANNs using a 64-context paradigm, and provided insight into the impact of abstract representations pretraining tasks efficiently embed abstract representations into ANNs, leading to improved systematic generalization similar human behavior. These findings provide a human-centric benchmark from which to understand compositional generalization in ANNs, paving the way for greater interpretability of compositionality in ANNs. Importantly, investigating compositionality through a human-centric framework (e.g.. by benchmarking ANNS against humans in the same task) creates a concrete target for interpreting the strengths and limitations of compositionality in ANNs. We hope these findings inspire further investigations into the comparison of compositionality in humans and ANNs.
generalization performance. Overall, we found that simple
Acknowledgments and Disclosure of Funding
T.I. acknowledges research internship sponsored by IBM Research and Swartz Foundation postdoctoral fellowship from Yale University. This project was supported in part by the US National Institutes of Health, under awards K99-R00 MH096901 and R01 MH109520 (M.WC..). The content solely the responsibility of the authors and does not necessarily represent the official views of any of the funding agencies.
10

---

References
[1] Yashar Behzadi. Khaled Restom, Joy Liau, and Thomas Liu. A component based noise correction method (CompCor) for BOLD and perfusion based fMRL. Neurolmage, 37(1);90-101, 2007. ISSN 1053-8119 19, doi: https:://oi..or// 1016/j http:///ooonrgoooo..ooo///mmmmmo
2007,04.44. URL httpp://ww..cccenccdirreee http://cweeeeecctecoeeeeeoeoeeee wwwpeeettt.ccccccteeeeeeeee coom/science/
[2] Silvia Bernardi, Marcus K. Benna, Mattia Rigotti, Jéröme Munucra, Stefano Fusi, and C. Daniel Salzman. The Geometry of Abstraction in the Hippocampus and Prefrontal Cortex. Cell, October 2020. ISSN 00928674, doi: 10.101 a11/50092864200312289
00. URL https: https://Linkinghubb elsevier wal.svierrccom/etrreeeee com/retrivee/
[3] Rastko Ciric, Daniel H Wolf, Jonathan D Power, David R Roalf, Graham L. Baum, Kosha Ruparel, Russell T Shinohara, Mark A Elliott, Simon B Eickhoff, Christos Davatzikos, Ruben C Gur, Raquel E Gur, Danielle S Bassett, and Theodore D Satterthwaite. Benchmarking of participant-level confound regression strategies for the control of motion artifact in studies of functional connectivity. Neurolmage, 154:174-187, 2017. ISSN 1053-8119. doi: https://doi.og// http:////doioo///.. 1016/j neuroimage.2211 07.00.00.. URL
http:://ww..cc www.enediieeet.t.....mmmmm com http://ccccccccreeeeaa http://ciiaaeee/atttcileaaapiaaa http:///ceeeee//111111111711110/
[4] Michael Cole, Joset A. Etzel. Jeffrey M. Zacks, Walter Schneider. and Todd S. Braver. Rapid Transfer of Abstract Rules to Novel Contexts in Human Lateral Prefrontal Cortex. Frontiers in Human Neuroscience, 5:142, November 2011. ISSN 1662-5161. doi: 10.3389/fnhum.201 2010..001111110 1.00122. URL p.ooural..frottian wwwpresiiinnggrrrreeeeeeeeeeeeee wttpg//rriceee//oo #389/fnhum... 2011 wtt.l001122asstte
[5] Michael W. Cole, Patryk Laurent, and Andrea Stocco. Rapid instructed task learning: new window into the human brain's 's unique capacity for flexible cognitive control. Cognitive, Affective, & &Behavioral Newroxcience, pages 1-22, 2012. ISSN 1530-7026. doi: 10.37588%13415-012-01335
[6] Ronald Boris Dekker, Fabian Otto, and Christopher Summerfield. Determinants human compositional generalization. Technical report, AsyArXiv, March 2022. URL httpa::/payyaxxiv.com qnpv6/ article.
type:
[7] Timo Flesch, Jan Balaguer, Ronald Dekker, Hamed Nili, and Christopher Summerfield. Comparing continual task learning in minds and machines. Proceedings of the National Academy of Sciences, 115(44); E10313-EB E10322. October 2018. doi: (0073//pas.10007555501 URL httph:/// pnas , org/doi/ doi/ Publisher: Proceedings of the National Academy of Sciences.
abs/10 100..0033/paas 1800755115
[8] Timo Flesch, Keno Juechems, Tsvetomira Dumbalska, Andrew Saxe, and Christopher Summerfield. Orthogonal representations for robust coneext-depnndet task performance in brains and neural networks. Neuron, 0(0), January 2022. ISSN 62733. doi: 10. www.:/nurro.o2.....00 00.0..0....... URL https:: : //www Publisher: Elsevier.
cell
6273 (22) 00005-8
[9] Steven M. Frankland and Joshua D. Greene. Concepts and Compositionality: In Search of the Brain's Language of Thought. Annual Review of Psychology, 71(1):273 303, January 2020. ISSN 0066-4308, doi: 10.1146/amurrev psych- 122216 000---1111 1829. URL (tRp:ttts:://ee wwwwalreviews..oooomoom
doi/100 w1144///nnee--eeee 122216-222226-- 011829
[10] K. J. Friston, A. P. Holmes, K. J. Worsley, J.-P. Poline, C. D. Frith, and R. S. J. Frackowiak. Statistical parametric maps in functional imaging: A general linear approach. Human Brain Mapping, 294):189-210, 1994. ISSN 10659471. doi: 10. (000///m.400020040
[11] Stefano Fusi, Wael F. Asaad, Farl K. Miller, and Xiao-Jing Wang. A Neural Circuit Model of Flexible Sensorimotor Mapping: Learning and Forgetting on Multiple Timescales. Neuron, 54(2)319-333, April 2007. ISSN 0896-6273, doi: 10..1166....oo... www.////....ro..2111 www..n.e00.....0..... 2007.03.017. URL URL httt:tttt///...... http://www..ciinccciieetettt. com/
[12] Stefano Fusi, Earl K Miller, and Mattia Rigotti. Why neurons mix: high dimensionality for higher cognition. Current Opinion Neurobiology, 37:66-74, 2016. ISSN 09594388, doi: 10. co.1006fj..mbb.2166. URL httl:/tttp//nniiii
elaevier
[13] Matthew F. Glasser, Timothy S. Coalson, Emma C. Robinson, Carl D. Hacker, John Harwell, Essa Yacoub, Kamil Ugurbil, Jesper Andersson, Christian F. Beckmann, Mark Jenkinson, Stephen M. Smith, and David C. Van Essen. A multi-modal parcellation of human cerebral cortex. Nature, pages 1-11, 2016. ISSN 0028-0888 doi: 10. 10.1ur//n 18933. URL http:://ww. nature. com www/tooifider//10...o.. 1038/ nature 18933

---

[14] Matthew F Glasser. Stephen M Smith, Daniel S Marcus, Jesper L R Andersson, Edward J Auerbach, Timothy E Behrens. Timothy S Coalson, Michael P Harms, Mark Jenkinson, Steen Moeller, Emma Robinson, Stamatios N Sotiropoulos, Junqian Xu, Essa Yacoub, Kamil Ugurbill and David Van Essen. The Human Connectome Project's neuroimaging approach. Nature neuroscience,, (9(9):1175 87, 2016. ISSN 1446-1726..0 doi: 10, 1038/00 4361. URL htttp://wan http:://wwwwnnatur.eemm nature. wwwp://wo//c//oooacaaoaaea http://urrrrooorra///////////// pdf /anf/nn.o 4361 wwtp///htttp:////waaa... ncbi nlm. nih.
[15] Raia Hadsell, Dushyant Rao, Andrei A. Rusu, and Razvan Pascanu. Embracing Change: Continual Learning in Deep Neural Networks. Trends in Cognitive Sciences, 24(12):0228-1440, December 2020. ISSN 1364-6613, 1879-307X, doi: 10. 10. 2020 010.00....0000 004. https://wwet http://wwwwwwlllllle./o.ooo. cell. coom/rends/
URL
Publisher: Elsevier.
[16] Irina Higgins, Loic Matthey, Arka Pal, Christopher Burgess, Xavier Glorot, Matthew Botvinick, Shakir Mohamed, and Alexander Lerehner. beta- VAE: Learning Basic Visual Concepts with Constrained Variational Framework. November 2016. http:pp:e//opeerreeeeeeeeee////e
Sy2t22UU
[17] Trina Higgins, David Amos, David Pfau, Sebastien Racaniere, Loic Matthey, Danilo Rezende, and Alexan- der Lerchner. Towards a Definition of Disentangled Representations. (rxiix/8/2.022..0 [cs, stat], December 2018. URL http:/ttt::/....ooo http://rrg/abbb//raeeeeeee/ wwt.aaabb..aaaa///a 02230
arXiv: 1812.02230.
[18] Irina Higgins, Le Chang, Victoria Langston. Demis Hassabis, Christopher Summerfield, Doris Tsao, and Matthew Botvinick. Unsupervised deep learning identifies semantic disentanglement in single inferotemporal face patch neurons. Nat Commun, 12(1) November 2021. ISSN 2041-1723, doi: 10. 00558/41467-021-2200 Number: 1 Publisher: Nature Publishing Group.
URL httpp://www. nature. wtt.///aaa/aeeea/aeeeooooee
021-
[19] Drew Hudson and Christopher Manning. GQA: A New Dataset for Real-World Visual Reasoning and Compositional Question Answering. a/Xiv:1902.09506 [cs]. May 2019. URL /tRp:/tt:///axt org// bbs/1192.co 09506
arXiv: 1902.09506.
[20] Dieuwke Hupkes, Verna Dankers, Mathijs Mul, and Elia Bruni Compositionality Decomposed: How do Neural Networks Generalise? Journal of Artificial Intelligence Research, 67:757-795, April 2020. ISSN 1076-9757.
doi: 10.1613/jains 1.1674,....1..4, URL https: //wp./ jair ora..dor//ndex..pop pttpp//p/aa/aaaaaic..e..... view/11674
[21] Takuya Ito and John D. Murray. Multi-task representations in human cortex transform along sensory- to-motor hierarchy. Technical report, November 2021. URL httpps://ww.. biorxiv org/ wrt./cntent//////////oomm/// content/
10. 10.1101/2021 11. 19.29.c... 470432v1 Company: Cold Spring Harbor Laboratory Distributor: Cold Spring Harbor Laboratory Label: Cold Spring Harbor Laboratory Section: New Results Type: article.
[22] Takuya Ito, Kaustubh R. Kulkarni, Douglas H. Schultz, Ravi D. Mill, Richard H. Chen, Levi I. Solomyak, and Michacl W. Cole. Cognitive task information is transferred between brain regions via resting-state network topology. Nat Commun, 8, October 2017. ISSN 2041-1723. doi: 10. 1017388/44147-0071-000000 URL (tLLhhtpp::///
www ncbi. www..iih....mmmmmmmmmm.mm..immmm gov
[23] Takuya Ito, Guangyu Robert Yang. Patryk Laurent, Douglas H. Schultz, and Michael W. Cole. Construct- ing neural network models from brain data reveals representational transformations linked to adaptive behavior. Nat Commun, 13(1):673, February 2022. ISSN 2041-1723, doi: (1098/441667-0222232277
URL htttp:://wwt www.wa.tuee... com/ www.aaaeeeeeeeeeeeeeeeee 28323-7 Publishing Group.
Number: Publisher: Nature
[24] Jie Lisa Ji, Marjolein Spronk, Kaustubh Kulkarni, Grega Repovš, Alan Anticevic, and Michael W Cole. Mapping the human brain's 's cortical-subcortical functional network organization. Neurolmage. 185: 35-57, 2019. ISSN 1053-8119. doi: https://doi.or//10.. http:///.....ooo.0..oooooeeooooo wtwp:///eeeooomaeeeee000.1... 20....00.00. URL http
//www www.ssceennedrret.cco http://///......caaaaaaaat//////
[25] W. Jeffrey Johnston and Stefano Fusi. Abstract representations emerge naturally in neural networks trained to perform multiple tasks. Technical report, October 2021. URI. httpp:////// biorxiv. org/ org//oooe content/
10. 1001/2222 10. 20. 20044651772 Company: Cold Spring Harbor Laboratory Distributor: Cold Spring Harbor Laboratory Label: Cold Spring Harbor Lahoratory Scction: New Results Type: article.
[26] Daniel R. Kepple, Rainer Engelken, and Kanaka Rajan. Curriculum learning as tool to uncover learning principles in the brain. September 2021. URL htt.aaapp//opppeeeeee..co http://www...e.eeooo///d//ttttoa __p0ppppp___
[27] Diederik P. Kingma and Jimmy Ba. Adam: A Method for Stochastic Optimization. arXiv:1412.6980 (cs]. January 2017. URL wtw.ttttt////////t httt:://aattiii wwwp://.rgga/bb//b/eeeeeee
hwt.aa/////4eee.. 6980 arXiv: 1412.6980.
12

---

[28] Brenden Lake and Marco Baroni. Generalization without Systematicity: On the Compositional Skills of Seruuence-toSSequeene Recurrent Networks. In International Conference on Machine Learning, pages 2873-2882. PMLR, July 2018. URL/attp://proceeding mlx. www.eppeessseseeeeeeeeee 18ke0// html ISSN:
2640-3498.
[29] Brenden M. Lake. Compositional generalization through meta sequence-to to-ee-ueneeeee learning. arXiv:906.05388 [es), Octoher 2019. httpp://///a http:///axiii.orraaa//////////// prg/aba/1906...../ 05381 arXiv: 1906.05381.
[30] Brenden M. Lake, Tomer D. Ullman. Joshua B. Tenenbaum. and Samuel J. Gersh- man. Building machines that learn and think like people. Behavioral and Brain Sei-
ences, 40, 2017. ISSN 0140-525X, 1469-1825. doi: 10.101 //50140//004 16001837.
URL https://wwe www.:/wwww....ee........m. wwtp.//orr//ouunaaa//aaaaaaaaaaa
and brain sciences/
article/ article/buildiii hawpee/obunndnnn--nannnneeeee-ee that-leaatttttttteeeea aaan-andd-hink-- likk---opee-pppeee
Publisher: Cambridge University Press.
[31] Omer Levy and Yoav Goldberg. Linguistic regularities in sparse and explicit word representations. In Proceedings of the eighteenth conference on computational natural language learning, pages 171-180, 2014.
[32] Daniel S. Margulies, Satrajit S. Ghosh, Alexandros Goulas, Marcel Falkiewicz, Julia Huntenburg. Georg Langs, Gleb Bezgin, Simon B. Eickhoff, F. Xavier Castellanos, Michael Petrides, Elizabeth Jefferies, and Jonathan Smallwood. Situating the default-mode network along a principal gradient of macroscale cortical organizaaion.. PNAS, 113(44):1254- 12574 November 2016. ISSN 0027-844 -227,8424, 1091-6490. doi:
10077//paa..100822211111 URL https:/ http:///wwwwpppa...a.aamaa ppa..org//aa...........aa..aappa wwwt://ttttttteecoocccceeccceett 113/ 44/12574
[33] Christian David Marton, Guillaume Lajoie, and Kanaka Rajan. Efficient and robust multi-task learning in the brain with modular task primitives. arXiv:2105. [cx, q-bioj. May 2021. URL http://arxii> arXiv: 14108.
htg/abb/21055 14108
[34] Tomáš Mikolov, Wen-tau Yih, and Geoffrey Zweig. Linguistic regularities in continuous space word representations. Proceedings of the 2013 conference of the north american chapter of the association for computational linguistics: Human language technologies, pages 746-751, 2013.
[35] Milton Llera Montero, Casimir JH Ludwig, Rui Ponte Costa, Gaurav Malhotra, and Jeffrey Bowers. The role of Disentanglement in Generalisation. September 2020. URL htttap:://pperrvveee.ettt//// Id-qbH974jK0N
[36] T Nichols and Andrew Holmes. Nonparametric Permutation Tests for Functional Neuroimaging Ex- periments: A Primer with examples. Human Brain Mapping, 15(1): 2001. ISSN 1065-9471, doi: 10. 1002/hbm. 1058. httpt:ttp:///ww/e interscience. wiley. wtt.coocccc// /gim betelbiin/abbee /ttp///bttratt/010444///
[37] Maxwell I. Nye, Armando Solar-Lezzama, Joshua B. Tenenbaum, and Brenden M. Lake. Learning Compositional Rules via Neural Program Synthesis. arXiv:000.05562 [es], March 2020. URL. http
//axivv. wwwg//oorooooee abaaabaa////// 220/.20033.000 05562
arXiv: 2003.05562.
[38] Steven T. Piantadosi. The Computational Origin of Representation. Minds & Machines, November 2020. ISSN 1572-8641. doi: 10.1007/s110233-000
URL httpa://doii fttpa:://doi.or/10... 1007/
311023-20-095400
[39] Stefano Recanatesi, Matthew Farrell, Guillaume Lajoie, Sophie Deneve, Mattia Rigotti, and Eric Shea-Brown. Predictive learning as network mechanism for extracting low-dimensional latent space representations. Nature Communications, 12(1):1417, March 2021. ISSN 2041-1723, doi: 107-//44467-021-2222
1. URL ltttps://www http://ww..ureeeeeeeeeet//ete wwwura. westureecoo//e wwt.o///aarcleees/aa1aaeee s414:////44477702011 021--2611-2-222--- Number: 1 Publisher: Nature Publishing Group.
[40] Carlo Reverberi, Kai Görgen, and John- -ylan Haynes. Compositionality of Rule Representations in Human Prefrontal Cortex. Cerebral Cortex, 226 June 2012. ISSN 1460-2199, 1047-3211. doi: 20000990/eecc.///12000.... 1093/ htt3///ccor/bbb2200
[41] Matria Rigotti, Omri Barak, Melissa R Warden, Xiao-Jing Wang. Nathaniel Daw, Earl K Miller, and Ste- Fusi. The importance mixed selectivity complex cognitive tasks. Nature, 497(7451):585-99, May 2013. ISSN 1476-4687. doi: 10. 0038/nauure111600 uttphttp:// /wwww nobi mlm nih, hww.m/mmw...oooobmmmemele.m
23685452http: : /www http:ttppp///w..puumeeeeeeeaall nih, hoop/aricceereeeee wwwpeennnerr.rf...eeeeee wtg.//artidddPM0441555.5
[42] Jesse Rissman, Adam Gazzaley, and Mark D' Esposito. Measuring functional connectivity during distinct stages of a cognitive task. Neurolmage, 23(2):752-766 763, 2004. ISSN 10538119, doi: 1..10166..euriinnne 2004.06.035.
13

---

[43] Reidar Riveland and Alexandre Pouget. A neural model of task compositionality with natural language instructions. Technical report, bioRxiv, February 2022. URL https://sww. wwwp:wwbw...i...mmmmoomoomm...w. org/ content/ Section: New Results Type: article.
10. N0.1101/2022.0 02. 22. 481293v1
[44] Clemens Rosenbaum, Tim Klinger, and Matthew Riemer. Routing Networks: Adaptive Selection of
Non-linear Functions for Multi-Tass Learning. arXiv:1711.0129/ [cs], December 2017. URL http'
wwwpivoorg//bb//////a 01239 arXiv: 1711,01239.
[45] Clemens Rosenbaum, Ignacio Cases, Matthew Riemer, and Tim Klinger. Routing Networks and the Challenges of Modular and Compositional Computation. 1rX4v 1e01199944.77444 [cs, stat], April 2019. URL
attp://arxisn org/ aba/ 1904. 12774
arXiv: 1904.12774 version: 1.
[46] Laura Ruis, Jacob Andreas, Marco Baroni, Diane Bouchacourt, and Brenden M Lake. Benchmark Systematic Generalization Grounded Language Understanding. In Ad- vances in Neural Information Processing Systems. volume 33. pages 19861-19872. Cur- ran Associates, Inc., 2020. URL ptttps://proceedii wwrnngg..eeriie www.:/eerrrr.cccccccce httppp..cpppppp///////aaaaa hastrettte-bttttt.hhhaae Abstract.html
[47] Luca Saglietti, Stefano Sarao Mannelli, and Andrew Saxe. An Analytical Theory of Curriculum Learning
in Teacher-Studen Networks, arXiv:2106.08M) [cond-mat, stat], June 2021. URL http::/arxiv......////
abs/2106. 08068
arXiv: 2106.08068.
[48] Walter Schneider, Amy Eschman, and Anthony Zuccolotto. E-Prime: User's guide. Psychology Software Incorporated, 2002.
[49] Jane X. Wang, Zeb Kurth- Nutth-Nelson, Dharshan Kumaran, Dhruva Tirumala, Hubert Soyer. Joel Z. Leibo, Demis Hassabis, and Matthew Botvinick. Prefrontal cortex as meta-reinforcemmet learning system. Nat Neumsci, 2 860-8866-868 June 2018. ISSN 1546-1726 doi: (038//415533-01-0000
_RI.bttps Number: 6 Publisher: Nature Publishing Group
//www.nature. nawure.
018--999--11--117711 0147-8
[50] Guangyu Robert Yang, Igor Ganichev, Xiao-Jing Wang. Jonathon Shlens, and David Sussillo. dataset and architecture for visual reasoning with working memory. arXiv preprint arXiv: 060022 2018.
[51] Guangyu Robert Yang. Madhura R. Joglekar, Francis Song, William T. Newsome, and Xiao-Jing Wang. Task representations in neural networks trained to perform many cognitive tasks. Nature Neuroscience, page 1, January 2019, ISSN 1546-1724 doi: 10. 00w0:/44111-----33111113 URL httph: htmps://wwnn //www. nature com
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's contribu- tions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes] see Section
(c) Did you discuss any potential negative societal impacts of your work? [N/A]
(d) Have you read the ethics review guidelines and ensured that your paper conforms them? [Yes] All human data has been de- identified and been publicly made available.
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [N/A]
(b) Did you include complete proofs all theoretical results? [N/A]
3. If you experiments..
(a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as URLY? [Yes] Public data URL is written in the text (Section 2.
(b) Did you specify all the training details (e.g., data splits, hyperparamcters, how they were chosen)?
[Yes]
(c) Did you report multiple timesy? [Yes] Applicable to Figures 1, 5, 6,
bars (e.g.. with respect to the random seed after running experiments
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [Yes]
4. If you using existing assets (e.g.. code, data, models) curating/releasing new assets...
14

---

(a) If your work uses existing assets, did cite the creators? [Yes] Used previously published
data 2
(h) Did you mention the license of the assets? [Yes] Published under a CCO license, mentioned in section A.I.
(c) Did you include any new either in the supplemental material URL? [No]
(d) Did you discuss whether and how consent was obtained from people whose data you're 'e us- ing/curating? [Yes] Yes, see section A.I.
(c) Did you discuss whether the data you using/curating contains personally identifiable informa- tion or offensive content? [Yss] Yes, section A. All data was previously de-identificd.
5. If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given participants and screenshots, if applicable? [Yes] see Figure A
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable? [Yes] IRB approval was mentioned in section A.1
(c) Did you include the estimated hourly wage paid to participants participant compensation? [No] This was a previously published dataset.
total amount spent on
15


Reviews:
[{"review_id": "u4EJPxur7sT", "reviewer": "Reviewer_h8RA", "summary": "Building off prior work proposing the role of “parallel abstract representations”—linearly additive vector representations of task dimensions—in compositional generalization, the authors set out to evaluate this framework through an analysis of existing behavioral and fMRI data as well as through ANN experimentation. The authors find that human task generalizability patterns are consistent with this factorized task representation framework and that parallel abstract representations are present for sensory, logic, and motor rule types in cortical systems that correspond to previous decoding work. They then propose a pretraining paradigm for ANNs on either task primitives or on simpler compositions of task primitives, and find that this improves zero-shot generalization to a more complex permuted rules task as well as sample efficiency, while resulting in representations which mirror some aspects of the neural encoding geometry and cortical hierarchy.", "questions": "The authors note (once again only in the appendix) that pretraining was stopped once pretraining tasks achieved 99%. To confirm, >99% accuracy on each task, e.g. until the worst task achieves 99% accuracy? How many training samples / iterations did this take under each pretraining paradigm?\n\nFor the Vanilla-ANN comparison, how is that generated? Is it just randomly initialized? From what distribution? (Okay I found this in appendix; add more details to main text.)\n\nAside from pretraining familiarizing the networks with task context rules, the networks also become more familiar with and learn information about the sensory stimulus input units. Do you think that this confound ends up being relevant for the observed increased sample efficiency, both for the primitives pretraining and especially for the simple task pretraining, which I assume was probably pretrained for more iterations to get to 99% due to the increased task complexity?\n\nIt would be helpful to see a better breakdown, in general, for both brain and ANN experiments, about which specific rules were more consistently encoded, e.g. how did PS scores differ across rules within each rule domain? Did certain dimensions drive mean PS scores more than others?\n\nOverall, this is an interesting study with great potential for publication, but the authors really need to be more explicit and detailed in their description of the ANN pretraining experiments, especially in the main text, so as to alleviate reader concerns on potential confounds.\n", "limitations": "The authors have addressed some limitations of this work. I agree that future work should attempt to replicate this using other task paradigms.", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 2, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "This paper addresses the interesting problem of compositional generalization, and does well to benchmark against human data. The extension of the “parallel abstract representations” framework to a new higher dimensional task and the resulting fMRI analysis is interesting, namely that these types of linearly additive vector representations of task abstractions appear to emerge in an expected and interpretable way.\n\nThe move to computational models and assessment of representational geometry in that regime is well motivated, and the results are very interesting, but in many respects, the details are drastically under-defined in the main text. In some cases, the reader is left struggling to understand critical aspects of what the authors actually did, until moving to the supplementary materials. For example, the precise exact input structure to the ANN is left to speculation until finding supplementary figure 8. This type of information needs to be expanded considerably in the main text.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "u-p530dYdwk", "reviewer": "Reviewer_ETpS", "summary": "Humans are good at generalizing previously acquired knowledge to find solutions to novel problems. In this way, they can learn a complex task by combining knowledge from different experiences instead of learning from scratch. This study suggests that neural networks can also afford this compositional generality by having task representations in a geometrical structure that allows a linear classifier to be used for generalization.\n", "questions": "Does the ANN learn different conditions in separate blocks for pretraining? How can the ANN learn a new condition while not modulating the weights learned from the pretraining of previous conditions (a.k.a. catastrophic forgetting)? Does the order of pretraining conditions matter in the accurate generalization? I think this part is the key to this paper but I can't find how this part was made.", "limitations": "Humans intuitively know which features of sensory stimuli can be combined and which features cannot be combined (e.g. red vertical stimulus can be shown but red blue stimuli cannot be presented). Therefore, they can generalize the representation of 'vertical' to 'blue' stimuli even only when they were trained with 'red' or when the colors of training stimuli are covered with their shape. However, the proposed ANN would not yet be able to overcome these kinds of biases. On the other hand, humans might have more experiences to answer to 'both' and 'either' than their negation, this would make it harder to respond to 'neither' or 'not both' conditions, while the ANN might not have  (or might not able to capture) these biases. ", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strength,\nThis work provides clear evidence that the representational structures (parallelism scores) are closely related to the ability for efficient cognitive control and generalization while the representations and cognitive control have been examined largely in separate domains. The effects of training facilitating fast learning can be found not only in human participants but also in the neural networks, which is interesting because this may not be easily explained by reinforcement learning but supports the idea that learning entails building abstract task structure representations.\n\nPotential weaknesses,\nIt is not clear whether the neural representations were based on the data acquired at the time of the presentation of the conditioned cue, at the time of decision making (maybe ash the time of the second stimuli), or the mean activity across a mini block. If it is the former, this is before the stimuli presentations and decision making, which is not true in the ANN architecture (the ANN were made with the inputs of not only conditions but also stimuli). If it includes decision making, it may not be the rule but the activity associated with the decision making might be represented.\n\nIn the previous study (Bernardi 2020, et al.), the rule cannot be separated from the stimuli while the representational structure cannot be explained by the similarity between stimuli. That is, the structure was built based on associated rules for each of the visually independent stimuli. However, in this experiment, can some pairs of parallel representations also be explained by the (dis)similarity of input vectors? (e.g. the same pair of stimuli followed by the different action cues). How would the same condition be represented if they were paired with different stimuli pairs?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "iQlhqbB1ikT", "reviewer": "Reviewer_HvQw", "summary": "The paper presents an analysis of fMRI task data which seeks to uncover \"abstract representations\" in the brain. This same analysis is then applied to a simple artificial neural network.\n\nThe analysis is built around a \"parallelism score\" which measures displacement vectors between brain activations. Given three modalities, (sensory, logical, and motor), each modality is held fixed and the other two are varied. The parallelism score for the fixed modality is an average measure of how aligned the displacement vectors are, averaged across combinations of the other two modalities.\n\nThe authors find that the highest motor parallelism is found in the motor areas, the highest visual parallelism in the visual areas, and that the logical parallelism is distributed throughout the brain, but lies mostly in the temporal and frontal areas.\n\nThis same analysis is applied to an artificial neural network. It is found that pre-training the NN results in better parallelism scores and better zero-shot performance.", "questions": "- line 105: What is mean by \"coding axis\"? In the text that follows, I gather that it is a displacement vector associated with a given modality. Is this correct?\n- line 120: In this case, what is meant by \"neural activation space\"? The space of all fMRI scans?\n- line 124: What is the activation vector? An vector of fMRI voxel inensity?\n- Figure 3 caption: This is the first place where the activation vector is mentioned. I would move this into section 2.3.\n- Figure 3 caption: \"The cosine angle between two linear decoders\" --> Is this the ANN mentioned in line 135? That seems like a non-linear decoder. What does it mean to take the cosine angle between decoders? Does this refer to the classification boundary? Or the hidden state representation?\n- line 135: What is the input to the model? Is it the intensity for all voxels?\n- Section 2.3: The PS will be high if the coding directions are parallel, but it will also be high if the coding directions are identical. How do we know that the coding directions do not all lie in the same low dimensional space, i.e., as in Figure 2B. Importantly, it would be good to know, of the pairs $v_i$ and $v_j$ that have high cosine similarity, what is the $l_2$ distance?\n- Section 3.3: What is the input to these ANNs? A text description of the task?\n- Section 3.4: What does it mean to measure the PS of an ANN? What is the activation vector for the ANN?\n- line 217: Is there only one training example per context?\n- line 220: \"unlike humans (see Fig. 1b), ANNs with no prior knowledge cannot compositionally generalize.\" I understand the claim being made, but it should be noted that an adult human is not exactly without \"prior knowledge\"\n- Figure 6: In 6B Brain systems are on the y-axis and rule domains are shown in groups. In 6C rule domains are shown on the x-axis and layers are shown in groups. I suggest putting layers on the y-axis in 6C to make the figures easier to compare.\n\n## minor things\n- line 171: missing Figure 7\n- Section 3.5: missing Figure 9\n", "limitations": "Limitations are adequately addressed", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "## Strengths\n- Application of the parallelism score to neural networks is novel as far as I can tell\n- Identifying the cognitive mechanism behind compositionality is an important problem\n- Analysis of this particular fMRI dataset is novel and shows the promise of this sort of analysis\n\n## Weaknesses\n- My main concerns are about the significance of the machine learning results. The paper claims that it introduces a new pre-training scheme. But the model architecture being considered is a two layer feed forward network and the pre-training is similar to schemes that have already been proposed, e.g., gSCAN [1]. The simplicity of the architecture and setting make it hard to see whether the results about pre-training would still hold in general. \n- The particular task setting also makes the comparison between human behavior and ANNs hard to interpret. The paper claims that their trained ANN shows a human-like response pattern to tasks. That is, the lowest level of the ANN is activated for sensory modalities, the middle layer for logic modalities, and the last layer for motor modalities. But this cannot be used to support the claim that the ANN has converged on a human-like hierarchy of representations, because there is no analogous ordered layering in the brain. Rather, the human brain results show that the motor areas respond to the motor modalities, the logic areas to the logic modalities, and the sensory areas to the sensory modalities. \n- Some clarity in description would be helpful. In section 3.3, I am not sure what the model inputs and outputs are supposed to be.\n- The stories about the brain and about the ANN don't seem to interact much. This paper presents some findings about the human brain and some findings about pre-training ANNs, but beyond section 3.7, the two stories do not seem very related.\n- The paper talks about identifying abstractions, but the only abstractions that can be discovered using the parallelism score are ones in which the representations have a linear relationship. \n\n## Bibliography\n[1] Ruis, Laura et al. “A Benchmark for Systematic Generalization in Grounded Language Understanding.” ArXiv abs/2003.05161 (2020)\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "deVUaxnGOAx", "reviewer": "Reviewer_CF9z", "summary": "This work investigates compositional generalization in humans (using fMRI data) and ANNs. The working hypothesis of the paper is that parallel abstract representations - consistent variable representations across different contexts - support compositional generalization. The Parallelism Score, a measure from neuroimaging research, is used to measure the consistency of representations in humans and ANNs. In their experiments, the authors use the C-PRO paradigm to construct a 64-context compositional task for humans and ANNs. The paradigm enables systematically varying conditions to test generalization abilities. In their experiments, they expose humans and ANNs to limited contexts and test them in novel contexts. First they confirm that humans perform well on these tasks, but drop slightly in novel contexts. They observe parallel abstract representations in fMRI data, supporting their hypothesis that this enables generalization.  Based on their finding that for humans, increased exposure to a rule improves performance in novel contexts, they design pretraining tasks to improve generalization in ANNs. This includes 1-rule and 2-rule simplifications of the 3-rule tasks of C-PRO, enabling the ANN to learn prior representations for sub-rule variables. Without pretraining, ANNs struggle in novel contexts. Pretraining significantly improves performance in novel contexts, enables zero-shot performance, and more sample efficient learning. Pretraining enables compositional generalization in ANNs similar to humans and similarly increases parallel abstract representations. The authors also report similarities in the hierarchical Parallel Scores between human fMRI data and  ANNs. ", "questions": "1. What is your response to weakness (1)? Do you think the relatively small increase in Figure 5A undermines your hypothesis that parallel abstract representations are important for compositional generalization?\n\n2. Please elaborate on how you arrived at your pretraining method from your finding that in humans \"increased exposure to specific rules improved performance on subsequent novel contexts\". I understand the finding, and the pretraining method, but to me the link isn't obvious. The pretraining method does not simply expose the ANN to more examples of contexts in which the rule appears (that would simply be showing the ANN more 3-rule examples containing that rule). The pretraining method exposes the ANN to simplified contexts (1-rule or 2-rule examples), where it can focus on learning representations. How do you motivate the link?", "limitations": "The authors dedicate an entire section to discussing the limitations of their work. This is valuable and insightful.", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "I think this is an excellent paper. The authors apply ideas from neuroscience research to the growing field of study that is compositional generalization in ANNs. My only caveat to confidently recommending acceptance is that I am not qualified to assess the validity of the neuroscientific claims made in the paper. But it seems that the authors have followed a solid methodology throughout and do not overstate any claims, so I would feel comfortable still recommending acceptance. Furthermore, the findings of the paper related to ANNs are quite interesting and novel on their own.\n\n**Strengths**\n\n1. The ideas in this paper are novel and valuable in the study of compositional generalization in ANNs. I am not aware of previous work applying the idea of parallel abstract representations to ANNs*, and this seems useful for future research. Furthermore, the methodology followed by the authors to perform analogous experiments in neuroimaging and ANNs is unique and well thought out. A human-centric benchmark for compositional generalization at such a scale (64 contexts) is certainly valuable.\n\n2. For the most part, the claims made by the authors are supported and well motivated throughout the paper. Their pretraining method boosts generalization significantly and should be further developed in different deep learning contexts. I would have expected the PS measures on ANN hidden layers to be higher than those reported in Figure 5 A, but the zero-shot performance and sample efficiency makes a strong case that these pretraining methods are effective.\n\n3. The paper is excellently written and well structured. The ideas are presented clearly, without overstating any claims.\n\n\\*  The closest idea to this that I have read of is the concept of “localism” from the paper by Hupkes et al. titled Compositionality Decomposed: How do Neural Networks Generalise? (2020). It’s not the same idea, but is similar in that it expects compositional models to build consistent variable representations independent of context.\n\n**Weaknesses**\n\n1. There is one experiment where the results are not as convincing as one would wish. In Figure 5A the PS measure increases only slightly after pretraining. This shows that pretraining does in fact encourage parallel abstract representations, but it does not necessarily show that this is absolutely crucial to compositional generalization. The PS measures increase slightly, but the generalization ability increases significantly, so perhaps some other factor enabled by pretraining is enabling generalization. Since the working hypothesis of the paper is the link between parallel abstract representations and compositional generalization, it might be worth discussing this result more.\n\n2. The paper performs its ANN experiments with a simple 2-layer feedforward neural network. I understand that this isolates the effect of pretraining for the experiments, but in the process it ignores the neural network architectures that are currently state-of-the-art in most domains i.e. Transformers, CNNs, and LSTMs. Most existing studies on compositional generalization in ANNs have focused on these more advanced architectures and if this paper had done so it would have made a stronger case for the proposed methods and methodologies. As such, the findings of this paper are not directly comparable to leading edge research on the generalization capabilities of ANNs.\n\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept