Compositional generalization through abstract representations in human and artiﬁcial neural networks
Abstract
Humans have a remarkable ability to rapidly generalize to new tasks that is difﬁcultto reproduce in artiﬁcial learning systems.
Compositionality has been proposed asa key mechanism supporting generalization in humans, but evidence of its neuralimplementation and impact on behavior is still scarce.
Here we study the computational properties associated with compositional generalization in both humans andartiﬁcial neural networks (ANNs) on a highly compositional task.
First, we identiﬁed behavioral signatures of compositional generalization in humans, along withtheir neural correlates using whole-cortex functional magnetic resonance imaging(fMRI) data.
Next, we designed pretraining paradigms aided by a procedure weterm primitives pretraining to endow compositional task elements into ANNs.
Wefound that ANNs with this prior knowledge had greater correspondence with humanbehavior and neural compositional signatures.
Importantly, primitives pretraininginduced abstract internal representations, excellent zero-shot generalization, andsample-efﬁcient learning.
Moreover, it gave rise to a hierarchy of abstract representations that matched human fMRI data, where sensory rule abstractions emergedin early sensory areas, and motor rule abstractions emerged in later motor areas.
Our ﬁndings give empirical support to the role of compositional generalization inhuman behavior, implicate abstract representations as its neural implementation,and illustrate that these representations can be embedded into ANNs by designingsimple and efﬁcient pretraining procedures.
Introduction
Humans can efﬁciently transfer prior knowledge to novel contexts, an ability commonly referredto as transfer learning.
One proposed mechanism underlying transfer learning is compositionalgeneralization (or compositional transfer) – the ability to systematically recompose learned conceptsinto novel concepts (e.g., “red” and “apple” can be combined to form the concept of a “red apple”)[5, 8, 17].
Indeed, it has been suggested that an algorithmic implementation of compositionalgeneralization is one of the key missing ingredients that ANN models need in order to achievehuman-like learning and reasoning capabilities [27, 25].
Therefore, quantifying how compositional generalization is manifested in human behavior and investigating its underlying implementation inbiological brains is a natural ﬁrst step to harness and deploy it in machine learning models.
Recent studies that investigated compositionality in machine learning relied on ar- specialized modules instance disentangled representation learning separates the independent factors underlying the structure of the input data into disjoint componentsof the feature vector [14, 15, 32, 13].
Program synthesis methods achieve state-of-the-art performanceon systematic generalization [17] through model architectures built by combining specialized neuraland symbolic program modules interacting to search over a space of valid production rules [26, 34].
Complementing these studies, abstract representations have been recently proposed as vector representations that reconcile compositional generalization with distributed neural codes [2].
In particular,parallel abstract representations – representations with a high Parallelism Score as previously deﬁned [2] – support out-of-context generalization by encoding changes in individual variables as alinear shift in the representations.
This notion of abstraction implies that these representations arecompositionally additive; novel compositions are encoded as the vector sum of distinct abstractrepresentations.
This is similar to how word2vec embeddings solve relational analogy tasks [31, 28]and generalizes disentangled representations by allowing for arbitrary afﬁne transformations ofdisentangled codes.
Crucially, this type of representation is operationally deﬁned in a way that canbe quantiﬁed in neuroimaging data by computing the Parallelism Score metric deﬁned in [2].
Inother words, parallel abstract representations are a computationally promising candidate as neuralsubstrate implementing compositional generalization, and are also measurable in the human brain bycomputing the Parallelism Score across fMRI voxels during neuroimaging experiments.
This work is motivated by the working hypothesis that parallel abstract representations supportcompositional generalization.
Accordingly, we ﬁrst characterized the behavioral signatures ofcompositional generalization in a task that systematically varied rule conditions across 64 contexts,showing that humans generalize better to tasks with greater similarity structure to previous tasks.
We then analyze fMRI imaging data showing that parallel abstract representations are distributedacross the entire cortex in a content-speciﬁc way during the execution of our compositional task.
Thissupports our working hypothesis that parallel abstract representations may implement compositionalgeneralization.
We then design a pretraining paradigm for ANNs to emulate humans’ prior knowledgeabout the compositional task elements, ﬁnding that ANNs pretrained in this way exhibit 1) moreabstract representations, 2) excellent generalization performance, and 3) sample-efﬁcient learning.
Finally, we ﬁnd that the layerwise organization of abstract representations in pretrained ANNsrecapitulates the content-speciﬁc distribution in human cortex.
Together, these ﬁndings provideempirical evidence for the role of abstract representations in supporting compositional generalization.
1.1 Related work
Several recent studies in neuroscience have applied analytic tools to identify the neural basis ofrapid generalization in biological neural networks.
Such studies employed various measures –cross-condition generalization [2, 36, 7, 4], state-space projections of task-related compositionalcodes [44, 38, 22], and Parallelism Score [2] – to quantify the generalizability and abstraction ofrepresentations.
Prior work in neuroscience has primarily evaluated compositionality in limitedcontext settings (e.g., up to 10 contexts), or without manipulating different types of features (e.g.,higher-order vs. sensory/motor features).
Moreover, these neuroscience studies used simple taskparadigms due to limitations in either the model organism (rodents and monkeys are unable to performcomplex tasks [2]) or to isolate speciﬁc types of abstraction in humans (e.g., logical abstractions[36]).
Here we signiﬁcantly expand on prior work by using a 64-context compositional task thatsystematically varies different types of task features (e.g., sensory, motor, and logical rules) toevaluate content-speciﬁc abstractions across the entire brain and multilayer ANNs.
This work alsocomplements related work in compositional generalization in machine learning [25, 17, 26, 40, 43, 16].
However, those studies primarily focused on building models that improve on current compositionalgeneralization benchmarks on arbitrarily complex compositional tasks, such as SCAN [25], COG [43],or GQA [16].
Importantly, these studies did not directly benchmark ANN behavior (or representations)against human behavioral and neural data, making a direct comparison difﬁcult. Here we leverageda non-
Methods 2.1 C-PRO task paradigm
We used the Concrete Permuted Rule Operations (C-PRO) paradigm (Fig. 1a) during fMRI acquisitionand ANN model training.
Brieﬂy, the C-PRO paradigm permutes speciﬁc task rules from threedifferent rule domains (logical decision, sensory semantic, and motor response) to generate dozensof novel task contexts.
This creates a context-rich dataset in the task conﬁguration domain.
Thesensory rule indicates which stimulus feature the subject should attend to.
The logic rule speciﬁes a Boolean operation to be implemented on the stimulus feature set.
The motor rule speciﬁes a speciﬁcmotor action (i.e., a button press with a speciﬁc ﬁnger).
Visual stimuli include either horizontally orvertically oriented bars with either blue or red coloring.
Simultaneously presented auditory stimuliinclude continuous (constant) or non-continuous (i.e., high or low pitched beeping) tones.
Each rule domain (logic, sensory, and motor) consists of four speciﬁc rules (Fig. 1a) comprised of were selected such that all were practiced meanperformance across all trials was 84% (median=86%; chance=25%).
See Appendix for details.
2.2 The geometry of abstract neural representations
Behavioral signatures of compositional generalization can be investigated by measuring behavioralperformance as a function of task composition and prior learning.
Neural signatures of generalization can be identiﬁed using analysis methods that characterize the geometry of neural activations duringtask generalization.
In particular, prior work proposed the Parallelism Score (PS) [2] as a measureto evaluate the consistency of task variable representation across different contexts.
Intuitively, PSidentiﬁes a consistent coding axis across task contexts that beneﬁts generalization.
We posit that representations with high PS (the speciﬁc type of abstract representation we investigate)support compositional generalization in human behavior.
We illustrate here how PS is reﬂected inthe geometry of neural representations with respect to the rule domains of the C-PRO task.
Letus consider a set of C-PRO contexts with logic rules BOTH or EITHER, and sensory rules withvalues RED or VERTICAL (Fig. 2).
High PS in the logic rule domain indicates that the differencein activation vectors between contexts with BOTH and EITHER rules is the same when pairedwith either the RED or VERTICAL sensory rules.
Thus, a change from BOTH to EITHER resultsin the same parallel change irrespective of the sensory rule (Fig. 2c).
In contrast to unstructuredhigh-dimensional representations (Fig.
2a), this would afford high generalization, since the effect ofchanging the logic rule in either sensory rules automatically transfers to the other sensory rule.
[Figure] We generalize the deﬁnition of PS by [2] to tasks where variables can assume an arbitrary numberof values (as opposed to being binary) and applied it to human fMRI and internal ANN activations.
PS is deﬁned as the cosine angle of the coding directions of the same rules in different contexts inthe neural activation space (Fig.
3a-c).
A cosine angle close to 1 indicates coding directions thatare highly parallel, despite differences in context.
Speciﬁcally, we compute the coding angle fora speciﬁc rule dichotomy (e.g., the coding direction “BOTH” vs. “EITHER”) by identifying allpairs of task contexts that had exactly the same secondary (sensory) and tertiary (motor) rules.
Foreach pair, we subtracted the activation vectors associated with each context to obtain the vector thatrepresented that coding direction (see Fig. 3a).
We did this for all other pairs in that coding direction.
Deﬁning v i as this coding vector for the i th pair, we computed the PS score for one dichotomy as
PS k =
P 16 i 6 = j cos ( v i , v j )) , since there are 16 possible pairs for each coding direction within the C-PRO task.
To obtain the PS for a speciﬁc rule domain (e.g., logic, sensory, or motor rules), PS k iscomputed for every coding direction, then averaged (e.g., for logic PS, the average of “BOTH” versus“EITHER”, “BOTH” versus “NEITHER”, etc.).
Statistical testing was performed using a non-parametric procedure, where we shufﬂed labels withineach rule domain 1000 times and re-calculated PS to produce a null distribution.
We corrected formultiple comparisons (across brain regions) using non-parametric family-wise error correction [33].
2.4 ANN construction and training
The primary ANN architecture had two hidden layers (128 units each) and an output layer that wascomprised of four units that corresponded to each motor response.
Training used a cross-entropy lossfunction and the Adam optimizer [24].
(See Appendix for details.)
Training on the C-PRO task was performed in a sequential learning paradigm.
To mimic the human experiment, an arbitrary set of four practiced contexts was initially selected for training.
(This was randomly selected across different ANN initializations.)
Then, novel task contexts were incrementally added into the set of training contexts.
Results
3.1 Behavioral signatures of rapid compositional generalization in humans
We evaluated human behavioral compositional generalization by assessing performance on novelcontexts in the C-PRO paradigm.
Since adult humans have decades of prior knowledge, subjectswere able to compositionally generalize to novel task contexts without any training (novel accuracy=84.17%, chance=25%, Wilcoxon signed-rank p < 0.0001).
However, subjects performed the fourpracticed contexts better than novel contexts (practiced=87.67%, novel=84.17%; p=0.003).
We nextassessed how performance on novel contexts changed as a function of shared rule structure to thepracticed contexts.
Consistent with compositional transfer of previously learned rules, performanceon novel task contexts improved as a function of similarity to the practiced contexts (accuracy, 2-ruleoverlap=84.86%; 1-rule overlap=83.48%; practiced vs. 2-rule overlap, p=0.008; 2-rule vs. 1-ruleoverlap, p=0.03; Fig. 1b).
Though our ﬁndings are consistent with compositional transfer, we foundthat rapid transfer to novel contexts is more difﬁcult.
However, we found that increased exposure tospeciﬁc rules improved performance on subsequent novel contexts using that same rule (all except forthe “Both” and “Either” rules, likely due to ceiling effects, FDR-corrected p < 0.05; Fig.
1c-e).
Thissuggests that even though performance in novel contexts is worse than practiced contexts, subjectscan improve rule transfer with increased practice (or pretraining).
3.2 Spatial and content-speciﬁc topography of abstract representations in human cortex
We extended prior work to identify abstract representations using PS across the entire human cortex[2, 4, 36].
We calculated PS for each rule domain separately (Fig.
3a-c) using the vertices/voxelswithin each parcel (i.e., brain region) as activation vectors.
We found topographic differences ofsensory, logic, and motor rule abstractions tiled across human cortex (Fig.
3d-f).
Speciﬁcally, wefound that statistically signiﬁcant sensory rule abstractions were primarily identiﬁed in higher ordervisual areas and the dorsal attention network (i.e., brain areas involved in the top-down selectionof visual stimuli) (PS of signiﬁcant regions=0.15; family-wise error (FWE)-corrected p < 0.05; Fig.3d).
Logic rule abstractions were more widely distributed, but primarily observed in frontoparietalareas (PS of signiﬁcant regions=0.22; FWE-corrected p < 0.05; Fig. 3e).
Motor rule abstractions wereprimarily localized to somatomotor cortex (PS of signiﬁcant regions=0.29; FWE-corrected p < 0.05;
Fig. 3f).
Notably, regions with abstract representations form a subset of regions of those that containrule information using standard decoding methods (Fig. 7).
3.3 Embedding prior knowledge into ANNs with simple pretraining tasks
Human behavioral data suggested improved compositional generalization with increased task ruleexposure, in addition to the years of “pretraining” from ordinary development (i.e., at least 18+years).
Thus, we sought to evaluate whether embedding prior knowledge of rules could improvecompositional generalization in ANNs, while simultaneously investigating how prior knowledgeimpacts the geometry of ANNs’ internal task representations.
Given that the C-PRO task wasspeciﬁcally designed as a compositional task that conjoined three task rules, we created pretrainingparadigms designed to teach ANNs basic rule knowledge (Fig. 4; see Appendix for full description).
We constructed a simple feedforward ANN with two hidden layers (Fig.
This made it easier toinvestigate the effects of pretraining on internal representations, rather than architectural choices.
We designed two pretraining paradigms:
Primitives (1-rule) and Simple task (2-rule) pretraining.
Primitives pretraining trained on 1-rule tasks that focused explicitly on learning the semantics ofprimitive rule features (Fig.
4a-c).
This included distinguishing sensory stimuli, learning motorresponse mappings (e.g., “left index” rule would lead to a left index response), and abstract logicalrelations, which involved learning the boolean relations amongst logic rules.
Simple task pretrainingfocused on learning 2-rule conjunctions (i.e., a sensory and motor rule pairing / logical and sensoryrule pairing) (Fig.
4d-e).
Importantly, these pretraining paradigms focused on learning primitive 1- or2-rule associations that were signiﬁcantly simpler than the full C-PRO task (3-rule combination).
3.4 Pretraining induces abstractions, zero-shot performance, and sample efﬁciency
We measured the PS in ANNs trained with different pretraining routines: Vanilla (no pretraining),
Primitives pretrained, Simple task pretrained, and Combined (Primitives + Simple task pretrained).
PS was calculated for each rule domain separately, and then averaged across hidden layers.
Pretrained
ANNs had signiﬁcantly higher PS than the Vanilla ANN (Primitives vs. Vanilla, t(37)=5.26, p=1e-05;
Simple task vs. Vanilla, t(37)=8.46, p=1e-11; Combined vs. Vanilla, t(37)=3.03, p=0.003)
(Fig. 5a).
Moreover, PS increased from Primitives to Simple task pretraining (t(37)=3.91, p=0.0002), thoughno signiﬁcant increase in PS was observed in Combined vs. Simple task pretraining.
We next evaluated the zero-shot performance on the full C-PRO task after pretraining (Fig. 5b).
Asexpected, the Vanilla ANN performed near chance (acc=23.25%, chance=25%, one-sided t(38)=-2.17, p=0.98).
Primitives pretraining marginally improved zero-shot performance (acc=31.51%, t(38)=8.09,p<1e-9).
Simple task pretraining exhibited signiﬁcant improvement over Primitives pretrained models(acc=70.57%, Simple task vs. Primitives, t(37)=19.84, p < 1e-31).
Finally, we found that Combinedpretraining had excellent zero-shot performance on the entire C-PRO task (acc=92.15%, Combinedvs.
Simple task pretraining, t(37)=10.85, p < 1e-16).
Notably, we found that PS and zero-shotperformance monotonically increased with pretraining, illustrating that classic multilayer networkscan transfer abstract representations for systematic zero-shot generalization [17].
Finally, we sought to assess the impact of pretraining on learning/sample efﬁciency.
We thereforetrained a Vanilla network (no pretraining) on 60/64 C-PRO contexts to match the zero-shot performance of the Combined pretraining model (i.e., at least 90% accuracy on the 60 context trainingset).
We found that on the remaining test set (4/64 C-PRO contexts), the Vanilla trained modelachieved 96.02% generalization performance, but required up to 4.23x training samples to match theperformance of the Combined model (Fig. 5c).
Critically, the 4.23x more training samples includedall possible samples (pretraining and C-PRO samples).
This illustrated that pretraining afforded bothzero-shot generalization and sample efﬁcient learning.
3.5 Pretraining leads to compositional generalization in ANNs comparable to humans
We evaluated the learning and generalization dynamics of ANNs with and without pretraining, aftertraining ANNs on 4 of the full C-PRO contexts (matching the human experiment).
(Training wasstopped after achieving 90% performance on the 4 practiced contexts.)
We found overall poor generalization on novel task contexts in the Vanilla model (accuracy, practiced=94.37%, novel=28.79%;p < 0.0001; Fig. 9a).
This suggested that unlike humans (see Fig. 1b), ANNs with no prior knowledgecannot compositionally generalize.
We subsequently compared generalization performance on ANNsafter pretraining.
We found that with Primitives pretraining, generalization performance signiﬁcantlyimproved (57.97%; Fig. 9b).
We observed additional improvements with Simpliﬁed task pretraining(86.79%; Fig.
9c), achieving generalization performance on par with human performance (Fig. 9d).
We next incrementally trained all ANN models on novel contexts, by adding one novel context intothe training set at a time.
We tested generalization performance on the held-out (test set) contexts until
ANNs were trained on 63/64 contexts (Fig. 10a).
We found that generalization performance on novelcontexts was signiﬁcantly higher in ANNs with either pretraining routine (Fig. 10b).
This was despitethe fact that all ANNs had the same stopping criteria (i.e., 90% accuracy on the C-PRO trainingset).
We ran an additional experiment where each of the ANNs were shown an identical number of C-PRO task samples during training (i.e., ﬁxed number of samples), replicating our core ﬁnding (Fig.11).
These ﬁndings suggest that the inductive biases formed during pretraining signiﬁcantly improvedownstream generalization performance.
3.6 Pretraining ANNs facilitates sample-efﬁcient learning throughout novel task learning
We sought to evaluate how pretraining impacted sample efﬁciency.
We found that pretrained ANNsquickly became more sample efﬁcient as the training set expanded, even when accounting for totalnumber of (pretraining and C-PRO) samples (Fig. 10b).
We quantiﬁed the generalization performanceto sample efﬁciency ratio as the generalization inefﬁciency, ﬁnding that after learning only 7 C-PROcontexts, vanilla ANNs generalized worse than pretrained ANNs (Fig. 10c).
These ﬁndings supportthe notion that initial pretraining routines can simultaneously improve compositional generalizationand sample efﬁciency.
3.7 Convergent hierarchy of abstract representations in humans and ANNs
Analysis of human fMRI data revealed that content-speciﬁc abstraction was spatially heterogeneousacross cortex.
Recent neuroscience work has identiﬁed hierarchical gradients that organize alonga sensory input-to-motor output axis in both resting-state [29] and multi-task fMRI data [18].
Wetherefore sought to quantify PS across the sensory-to-motor hierarchy in fMRI data, and compare it to
PS changes in the feedforward hierarchy (i.e., layer-depth) in ANNs.
We focused our analyses on the
Combined pretrained model (which incorporates both Primitives and Simple task pretraining) due toits excellent zero-shot generalization (Fig. 5b).
In addition, we extended our model to include threehidden layers to make it easier to compare PS of different hidden-layer depths to the three corticalsystems of interest: sensory, association, and motor systems (Fig. 6a).
We measured the PS for each rule domain for sensory, association, and motor systems.
Sensoryrule PS was highest in the sensory system, logic rule PS was highest in association systems, andmotor rule PS was highest in the motor system (Fig. 6b).
To observe whether similar hierarchical PSorganization emerged in ANNs, we used the Combined pretrained model with three hidden layers,and plotted PS as a function of ANN depth.
Since our ANN transformed sensory inputs into motoroutputs, we analogized each ANN layer to the sensory, association, and motor cortical systems(Fig. 6a).
We found a similar pattern in the ANN: sensory PS peaked in the ﬁrst hidden layer;logic PS peaked in the second hidden layer; and motor PS peaked in the last two hidden layers (Fig.
6d).
We corroborated these ﬁndings using a continuous sensory-motor hierarchical gradient map(without discretization) (Fig. 12-13).
These ﬁndings suggest that abstraction emerges as a function ofrule-dependent specialization and hierarchical organization.
Discussion, Limitations, Conclusions
We provide empirical support for the role of compositionality in human generalization, and implicateabstract representations as its neural implementation.
In classic ANNs, which are known to performpoorly during systematic generalization [17, 6], we found that computationally cheap pretrainingparadigms embedded abstract representations that led to human-like generalization performance andsample efﬁcient learning.
When mapping abstract representations across cortex and ANN layers, wefound converging patterns of rule-speciﬁc abstractions from early sensory areas/layers to late motorareas/layers across human and ANN hierarchies.
These results reveal the hierarchical organizationof content-speciﬁc abstractions in the human brain and ANNs, while revealing the impact of theseabstractions for compositional generalization in models.
Our pretraining approach directly leverages knowledge of task structure to design pretraining routinesthat embed task biases into ANNs.
Despite the sample efﬁciency of this approach, this pretrainingapproach requires the initial overhead of designing paradigms useful for downstream learning.
A related approach that similarly requires prior knowledge of task structure is “representationalbackpropagation” – a regularization approach that aims to produce an idealized hidden representation[23].
However, there are other inductive bias approaches that do not require prior task knowledge.
Oneapproach constrains ANNs to produce abstract task representations by initializing ANN weights froma low-norm distribution [7].
However, initializing ANN weights in this regime is computationallycostly.
Another approach is to initialize networks with built-in modular structures to facilitate there-use of network modules across tasks [30, 39].
However, exactly how such networks disentanglerepresentations has not yet been explored.
Nevertheless, all these approaches are complementary toeach other.
It will be important for future work to assess how these approaches may synergisticallyinteract to optimize for sample-efﬁcient generalization in multi-task settings.
Though we provide comprehensive evidence of the role of abstraction in compositional generalization,there are several limitations in the present study that future research can explore.
We found that thespatial topography of abstract representations was highly content-dependent.
However, analyseswere limited to cross-context manipulations of limited rule types (sensory, logic, and motor gating),without addressing the organization of other task components (e.g., reward or stimuli).
Thus, futurestudies can explore how brains and ANNs represent the abstraction of other task components.
Second,though we were able to explore cross-context generalization across 64 contexts – signiﬁcantly morethan previous empirical studies in neuroscience – cross-context analysis was limited to a singletask type (i.e., the C-PRO paradigm).
It will be critical to see the organization of abstraction inmulti-task settings that go beyond 64 contexts.
Finally, our ANN modeling approach revealed thecomputational beneﬁts of pretraining.
It will be important for future work to benchmark sampleefﬁciency and generalization performance against other training paradigms (e.g., in continual learningand/or meta-learning settings; [12, 42]).
In conclusion, we characterized a convergent hierarchical organization of abstract representationsacross the human cortex and in ANNs using a 64-context paradigm, and provided insight into theimpact of abstract representations on generalization performance.
Overall, we found that simplepretraining tasks efﬁciently embed abstract representations into ANNs, leading to improved systematicgeneralization similar to human behavior.
These ﬁndings provide a human-centric benchmark fromwhich to understand and evaluate compositional generalization in ANNs, paving the way for greaterinterpretability of compositionality in ANNs.
Importantly, investigating compositional generalizationthrough a human-centric framework (e.g., by benchmarking ANNs against human behavior in thesame task) creates a concrete target for interpreting the strengths and limitations of compositionalityin ANNs.
We hope these ﬁndings inspire further investigations into the comparison and analysis ofcompositionality in humans and ANNs.