Compositional generalization through abstract representations in human and artificial neural networks Anonymous Author(s) AffiliationAddress email
Abstract
Humans have a remarkable ability to rapidly generalize to new tasks that is difficultto reproduce in artificial learning systems.
Compositionality has been proposed asa key mechanism supporting generalization in humans, but evidence of its neuralimplementation and impact on behavior is still scarce.
Here we study the computational properties associated with compositional generalization in both humans andartificial neural networks (ANNs) on a highly compositional task.
First, we identified behavioral signatures of compositional generalization in humans, along withtheir neural correlates using whole-cortex functional magnetic resonance imaging(fMRI) data.
Next, we designed pretraining paradigms aided by a procedure weterm primitives pretraining to endow compositional task elements into ANNs.
Wefound that ANNs with this prior knowledge had greater correspondence with humanbehavior and neural compositional signatures.
Importantly, primitives pretraininginduced abstract internal representations, excellent zero-shot generalization, andsample-efficient learning.
Moreover, it gave rise to a hierarchy of abstract representations that matched human fMRI data, where sensory rule abstractions emergedin early sensory areas, and motor rule abstractions emerged in later motor areas.
Our findings give empirical support to the role of compositional generalization inhuman behavior, implicate abstract representations as its neural implementation,and illustrate that these representations can be embedded into ANNs by designingsimple and efficient pretraining procedures.
Introduction
Humans can efficiently transfer prior knowledge to novel contexts, an ability commonly referredto as transfer learning.
One proposed mechanism underlying transfer learning is compositionalgeneralization (or compositional transfer) – the ability to systematically recompose learned conceptsinto novel concepts (e g., “red” and “apple” can be combined to form the concept of a “red apple”)[5, 8, 17].
Indeed, it has been suggested that an algorithmic implementation of compositionalgeneralization is one of the key missing ingredients that ANN models need in order to achievehuman-like learning and reasoning capabilities [27, 25].
Therefore, quantifying how compositional generalization is manifested in human behavior and investigating its underlying implementation in biological brains is a natural first step to harness and deploy it in machine learning models.
Recent studies that investigated compositionality in machine learning have typically relied on architec- tures comprised of specialized modules. For instance, disentangled representation learning separates the independent factors underlying the structure of the input data into disjoint components of thefeature vector [14, 15, 32, 13].
Program synthesis methods achieve state-of-the-art performance onsystematic generalization [17] through model architectures built by combining specialized neural andsymbolic program modules interacting to search over a space of valid production rules [26, 34, 41].
Complementing these studies, abstract representations have been recently proposed as vector representations that reconcile compositional generalization with distributed neural codes [2].
In particular,parallel abstract representations – representations with a high Parallelism Score as previously defined [2] – support out-of-context generalization by encoding changes in individual variables as alinear shift in the representations.
This notion of abstraction implies that these representations arecompositionally additive; novel compositions are encoded as the vector sum of distinct abstractrepresentations.
This is similar to how word2vec embeddings solve relational analogy tasks [31, 28]and generalizes disentangled representations by allowing for arbitrary affine transformations ofdisentangled codes.
Crucially, this type of representation is operationally defined in a way that canbe quantified in neuroimaging data by computing the Parallelism Score metric defined in [2].
Inother words, parallel abstract representations are a computationally promising candidate as neuralsubstrate implementing compositional generalization, and are also measurable in the human brain bycomputing the Parallelism Score across fMRI voxels during neuroimaging experiments.
This work is motivated by the working hypothesis that parallel abstract representations supportcompositional generalization.
Accordingly, we first characterized the behavioral signatures ofcompositional generalization in a task that systematically varied rule conditions across 64 contexts,showing that humans generalize better to tasks with greater similarity structure to previous tasks.
We then analyze fMRI imaging data showing that parallel abstract representations are distributedacross the entire cortex in a content-specific way during the execution of our compositional task.
Thissupports our working hypothesis that parallel abstract representations may implement compositionalgeneralization.
We then design a pretraining paradigm for ANNs to emulate humans’ prior knowledgeabout the compositional task elements, finding that ANNs pretrained in this way exhibit 1) moreabstract representations, 2) excellent generalization performance, and 3) sample-efficient learning.
Finally, we find that the layerwise organization of abstract representations in pretrained ANNsrecapitulates the content-specific distribution in human cortex.
Together, these findings provideempirical evidence for the role of abstract representations in supporting compositional generalization.
1.1 Related work
Several recent studies in neuroscience have applied analytic tools to identify the neural basis ofrapid generalization in biological neural networks.
Such studies employed various measures – crosscondition generalization [2, 36, 7, 4], state-space projections of task-related compositional codes [46,38, 22], and Parallelism Score [2] – to quantify the generalizability and abstraction of representations.
Prior work in neuroscience has primarily evaluated compositionality in limited context settings(e.g. , up to 10 contexts), or without manipulating different types of features (e.g., higher-order vs.sensory/motor features).
Moreover, these neuroscience studies used simple task paradigms due tolimitations in either the model organism (rodents and monkeys are unable to perform complex tasks[2]) or to isolate specific types of abstraction in humans (e.g., logical abstractions [36]).
Here wesignificantly expand on prior work by using a 64-context compositional task that systematically variesdifferent types of task features (e.g., sensory, motor, and logical rules) to evaluate content-specificabstractions across the entire brain and multilayer ANNs.
This work also complements related workin compositional generalization in machine learning [25, 41, 17, 26, 40, 45, 16].
However, thosestudies primarily focused on building models that improve on current compositional generalizationbenchmarks on arbitrarily complex compositional tasks, such as SCAN [25], COG [45], or GQA[16].
Importantly, these studies did not directly benchmark ANN behavior (or representations)against human behavioral and neural data, making a direct comparison difficult. Here we leverageda non-trivial 64-context compositional paradigm to investigate the representational principles thatfacilitate compositional generalization in both humans and ANNs.
Methods2.1 The Concrete Permuted Rule Operations (C-PRO) task paradigm
We used the C-PRO paradigm (Fig. 1a) during fMRI acquisition and ANN model training.
Briefly,the C-PRO paradigm permutes specific task rules from three rule domains (logical decision, sensorysemantic, and motor response) to generate dozens of novel task contexts.
This creates a context-richdataset in the task context domain.
The sensory rule indicates which stimulus feature should beattended to.
The logic rule specifies a Boolean operation to be implemented on the stimulus featureset.
The motor rule specifies a specific motor action (i.e., a button press with a specific finger). Oneof 256 possible unique stimulus combinations could be presented with each task context.
Visualdimensions included either horizontal or vertical bars with either blue or red coloring.
Auditorydimensions included continuous (constant) or beeping high or low pitched tones. Each rule domain (logic, sensory, and motor) consists of 4 specific rules (Fig. 1a).
task context is comprised of one rule from each domain, for a total of 64 possible task contexts (4 logic sensory x 4 motor). Subjects were trained on 4/64 “practiced” contexts prior to the fMRI session. The 4 practiced contexts were selected such that all 12 rules were equally practiced. Subjects’ meanperformance across all trials was 84% (median=86%; chance=25%).
See Appendix A.2 for details.
2.2 The geometry of abstract neural representations
Behavioral signatures of compositional generalization can be investigated by measuring behavioralperformance as a function of task composition and prior learning.
Neural signatures of generalizationcan be identified using analysis methods that characterize the geometry of neural activations duringtask generalization.
In particular, prior work proposed the Parallelism Score (PS) [2] as a measure toevaluate the consistency of task variable representation across contexts.
Intuitively, PS identifies acoding axis (i.e., the parallel displacement vector) across task contexts that aids generalization.
We posit that representations with high PS (the specific type of abstract representation we investigate)support compositional generalization in human behavior.
We illustrate here how PS is reflected inthe geometry of neural representations with respect to the rule domains of the C-PRO task.
Letus consider a set of C-PRO contexts with logic rules BOTH or EITHER, and sensory rules withvalues RED or VERTICAL (Fig. 2).
High PS in the logic rule domain indicates that the differencein activation vectors between contexts with BOTH and EITHER rules is the same when pairedwith either the RED or VERTICAL sensory rules.
Thus, a change from BOTH to EITHER resultsin the same parallel change irrespective of the sensory rule (Fig. 2c).
In contrast to unstructuredhigh-dimensional representations (Fig.
2a), this would afford high generalization, since the effect ofchanging the logic rule in either sensory rules automatically transfers to the other sensory rule. 2.3 Parallelism score
We generalize the definition of PS by [2] to tasks where variables can assume an arbitrary numberof values (as opposed to being binary) and applied it to human fMRI and internal ANN activations.
PS is defined as the cosine angle of the coding directions of the same rules in different contextsin the neural activation space (e.g., voxels or neurons within a brain region) (Fig.
3a-c).
A cosineangle close to 1 indicates coding directions that are highly parallel, despite differences in context.
We compute the coding angle for a specific rule dichotomy (e.g., the coding direction “BOTH” vs.“EITHER”) by identifying all pairs of task contexts that had exactly the same secondary (sensory) andtertiary (motor) rules.
For each pair, we subtracted the fMRI voxel activation vectors associated witheach context to obtain the vector that represented that coding direction (see Fig. 3a).
We did this forall other pairs in that coding direction.
Defining v i as this coding vector for the i th pair, we computedthe PS score for one dichotomy as PS k = 116 (cid:80) 16 i ̸
=
j cos ( v i , v j )) , since there are 16 possible pairs foreach coding direction within the C-PRO task.
To obtain the PS for a specific rule domain (e.g., logic,sensory, or motor rules), PS k is computed for every coding direction, then averaged (e.g., for logic PS, the average of “BOTH” versus “EITHER”, “BOTH” versus “NEITHER”, etc.).
Statistical testing was performed using a non-parametric procedure, where we shuffled labels withineach rule domain 1000 times and re-calculated PS to produce a null distribution.
We corrected formultiple comparisons (across brain regions) using non-parametric family-wise error correction [33].
2.4 ANN construction and training
The primary ANN architecture had two hidden layers (128 units each) and an output layer thatwas comprised of four units that corresponded to each motor response (see Appendix section A.7and Fig. 8 for additional details).
Training used a cross-entropy loss function and the Adamoptimizer [24].
The ANN transformed the trial input vector into a 4-element response vector with the equation Y = f ReLU ( X h W h + b h ) . Weights and biases were initialized from a uniform distribution U ( − (cid:112) 1 /k, (cid:112) 1 /k ) , where k is the number of input features from the previous layer.
Training on the C-PRO task was performed in a sequential learning paradigm.
To mimic the humanexperiment, an arbitrary set of four practiced contexts was initially selected for training.
(This wasrandomly selected across different ANN initializations.)
Then, novel task contexts were incrementallyadded into the set of training contexts.
Results
3.1 Behavioral signatures of rapid compositional generalization in humans
We evaluated human behavioral compositional generalization by assessing performance on novelcontexts in the C-PRO paradigm.
Since adult humans have decades of prior knowledge, subjectswere able to compositionally generalize to novel task contexts without any training (novel accuracy=84.17%, chance=25%, Wilcoxon signed-rank p < 0.0001).
However, subjects performed the fourpracticed contexts better than novel contexts (practiced=87.67%, novel=84.17%; p=0.003).
We nextassessed how performance on novel contexts changed as a function of shared rule structure to thepracticed contexts.
Consistent with compositional transfer of previously learned rules, performanceon novel task contexts improved as a function of similarity to the practiced contexts (accuracy, 2-ruleoverlap=84.86%; 1-rule overlap=83.48%; practiced vs. 2-rule overlap, p=0.008; 2-rule vs. 1-ruleoverlap, p=0.03; Fig. 1b).
Though our findings are consistent with compositional transfer, we foundthat rapid transfer to novel contexts is more difficult.
However, we found that increased exposure tospecific rules improved performance on subsequent novel contexts using that same rule (all except forthe “Both” and “Either” rules, likely due to ceiling effects, FDR-corrected p < 0.05; Fig.
1c-e).
Thissuggests that even though performance in novel contexts is worse than practiced contexts, subjectscan improve rule transfer with increased practice (or pretraining).
3.2 Spatial and content-specific topography of abstract representations in human cortex
We extended prior work to identify abstract representations using PS across the entire human cortex[2, 4, 36].
We calculated PS for each rule domain separately (Fig.
3a-c) using the vertices/voxelswithin each parcel (i.e., brain region) as activation vectors.
We found topographic differences ofsensory, logic, and motor rule abstractions tiled across human cortex (Fig.
3d-f).
Specifically, wefound that statistically significant sensory rule abstractions were primarily identified in higher ordervisual areas and the dorsal attention network (i.e., brain areas involved in the top-down selectionof visual stimuli) (PS of significant regions=0.15; family-wise error (FWE)-corrected p < 0.05; Fig.3d).
Logic rule abstractions were more widely distributed, but primarily observed in frontoparietalareas (PS of significant regions=0.22; FWE-corrected p < 0.05; Fig. 3e).
Motor rule abstractions wereprimarily localized to somatomotor cortex (PS of significant regions=0.29; FWE-corrected p < 0.05;
Fig. 3f).
Notably, regions with abstract representations form a subset of regions of those that containrule information using standard decoding methods (Fig. 7). This ensures that high PS is accompaniedby highly decodable representations (i.e., high dimensionality).
3.3 Embedding prior knowledge into ANNs with simple pretraining tasks
Human behavioral data suggested improved compositional generalization with increased task ruleexposure, in addition to the years of “pretraining” from ordinary development (i.e., at least 18+years).
Thus, we sought to evaluate whether embedding prior knowledge of rules could improvecompositional generalization in ANNs, while simultaneously investigating how prior knowledgeimpacts the geometry of ANNs’ internal task representations.
Given that the C-PRO task wasspecifically designed as a compositional task that conjoined three task rules, we created pretrainingparadigms designed to teach ANNs basic rule knowledge (Fig. 4; see Appendix for full description). logic rule primitives task involved teaching boolean relations among different logical operations. For example, when presented with the “BOTH” rule, the task was to distinguish two identical (“True”) versus two different (“False”) stimuli (i.e., same vs. different). b) Sensory rules involved mapping sensory rules onto stimulus features.
c) Motor rules involved mapping motor rules onto motor output units.
d-e) Simple task pretraining (2-rule tasks) was designed to teach the model how to perform simple (d) sensorimotor mappings and (e) logical-sensory gatings.
We constructed a simple feedforward ANN with two hidden layers (Appendix A.7; Fig.
This madeit easier to investigate the effects of pretraining on internal representations, rather than architecturalchoices.
We designed two pretraining paradigms:
Primitives (1-rule) and Simple task (2-rule)pretraining.
Primitives pretraining trained on 1-rule tasks that focused explicitly on learning thesemantics of primitive rule features (Fig.
4a-c).
This included distinguishing sensory stimuli, learningmotor response mappings (e.g., “left index” rule would lead to a left index response), and abstractlogical relations, which involved learning the boolean relations amongst logic rules.
Simple taskpretraining focused on learning 2-rule conjunctions (i.e., a sensory and motor rule pairing / logicaland sensory rule pairing) (Fig.
4d-e).
Importantly, these pretraining paradigms focused on learningprimitive 1- or 2-rule associations that were significantly simpler than the full C-PRO task (3-rulecombination).
3.4 Pretraining induces abstractions, zero-shot performance, and sample efficiency
We measured the PS in ANNs trained with different pretraining routines: Vanilla (no pretraining),
Primitives pretrained, Simple task pretrained, and Combined (Primitives + Simple task pretrained).
Each pretraining condition was trained until ANNs achieved 99% accuracy (see A.8 for details). PSwas calculated for each rule domain using the ANN’s hidden layer activations (see A.9).
Pretrained
ANNs had higher PS than the Vanilla ANN (Primitives vs. Vanilla, t(37)=5.26, p=1e-05; Simple taskvs.
Vanilla, t(37)=8.46, p=1e-11; Combined vs. Vanilla, t(37)=3.03, p=0.003)
(Fig. 5a). Moreover,
PS increased from Primitives to Simple task pretraining (t(37)=3.91, p=0.0002), though no significantincrease in PS was observed in Combined vs. Simple task pretraining.
We next evaluated the zero-shot performance on the full C-PRO task after pretraining (Fig. 5b).
As expected, the Vanilla ANN performed near chance (acc=23.25%, chance=25%, one-sided t(38)=-2.17,p=0.98).
Primitives pretraining marginally improved zero-shot performance (acc=31.51%, t(38)=8.09,p<1e-9).
Simple task pretraining exhibited significant improvement over Primitives pretrained models(acc=70.57%, Simple task vs. Primitives, t(37)=19.84, p < 1e-31).
Finally, we found that Combinedpretraining had excellent zero-shot performance on the entire C-PRO task (acc=92.15%, Combinedvs.
Simple task pretraining, t(37)=10.85, p < 1e-16).
Notably, we found that PS and zero-shotperformance monotonically increased with pretraining, illustrating that classic multilayer networkscan transfer abstract representations for systematic zero-shot generalization [17].
Finally, we sought to assess the impact of pretraining on learning/sample efficiency.
We thereforetrained a Vanilla network (no pretraining) on 60/64 C-PRO contexts to match the zero-shot performance of the Combined pretraining model (i.e., at least 90% accuracy on the 60 context trainingset).
We found that on the remaining test set (4/64 C-PRO contexts), the Vanilla trained modelachieved 96.02% generalization performance, but required up to 4.23x training samples to match theperformance of the Combined model (Fig. 5c).
Critically, the 4.23x more training samples includedall possible samples (pretraining and C-PRO samples).
This illustrated that pretraining afforded bothzero-shot generalization and sample efficient learning.
3.5 Pretraining leads to compositional generalization in ANNs comparable to humans
We evaluated the learning and generalization dynamics of ANNs with and without pretraining, aftertraining ANNs on 4 of the full C-PRO contexts. This matched the human experiment, since humanswere exposed to 4 “practice” contexts prior to performing the remaining 60 novel contexts.
(ANNtraining was stopped after achieving 90% performance on the 4 practiced contexts.)
We foundoverall poor generalization on novel task contexts in the Vanilla model (accuracy, practiced=94.37%,novel=28.79%; p < 0.0001; Fig. 9a).
This suggested that ANNs with no prior knowledge cannotcompositionally generalize.
We subsequently compared generalization performance on ANNs afterpretraining.
We found that with Primitives pretraining, generalization performance significantlyimproved (57.97%; Fig. 9b).
We observed additional improvements with Simplified task pretraining(86.79%; Fig.
9c), achieving generalization performance on par with human performance (Fig. 9d).
We next incrementally trained all ANN models on novel contexts, by adding one novel context intothe training set at a time.
We tested generalization performance on the held-out (test set)
contextsuntil ANNs were trained on 63/64 contexts (Fig. 10a).
Generalization performance on novel contextswas significantly higher in ANNs with either pretraining routine (Fig. 10b).
This was despite the factthat all ANNs had the same stopping criteria (i.e., 90% accuracy on the C-PRO training set).
We ranan additional experiment where each of the ANNs were shown a fixed number of C-PRO task samplesduring training, replicating our core finding (Fig. 11).
These findings suggest that the inductive biasesformed during pretraining significantly improve downstream generalization performance.
3.6 Pretraining ANNs facilitates sample-efficient learning throughout novel task learning
We sought to evaluate how pretraining impacted sample efficiency.
We found that pretrained ANNsbecame more sample efficient as the training set expanded, even when accounting for total number of(pretraining and C-PRO) samples (Fig. 10b).
We quantified the generalization performance to sampleefficiency ratio as the generalization inefficiency, finding that after learning only 7 C-PRO contexts,vanilla ANNs generalized worse than pretrained ANNs (Fig. 10c).
These findings support the notionthat pretraining can simultaneously improve compositional generalization and sample efficiency.
3.7 Convergent hierarchy of abstract representations in humans and ANNs
Analysis of human fMRI data revealed that content-specific abstraction was spatially heterogeneousacross cortex.
Recent neuroscience work has identified hierarchical gradients that organize alonga sensory input-to-motor output axis in both resting-state [29] and multi-task fMRI data [18].
Wetherefore sought to quantify PS across the sensory-to-motor hierarchy in fMRI data, and compare it to
PS changes in the feedforward hierarchy (i.e., layer-depth) in ANNs.
We focused our analyses on the
Combined pretrained model (which incorporates both Primitives and Simple task pretraining) due toits excellent zero-shot generalization (Fig. 5b).
In addition, we extended our model to include threehidden layers to make it easier to compare PS of different hidden-layer depths to the three corticalsystems of interest: sensory, association, and motor systems (Fig. 6a).
We measured the PS for each rule domain for sensory, association, and motor systems.
Sensoryrule PS was highest in the sensory system, logic rule PS was highest in association systems, andmotor rule PS was highest in the motor system (Fig. 6b).
To observe whether similar hierarchical PS organization emerged in ANNs, we used the Combined pretrained model with three hiddenlayers, and plotted PS as a function of ANN depth.
Since our ANN transformed sensory inputsinto motor outputs, we analogized each ANN layer to the sensory, association, and motor corticalsystems (Fig. 6a).
We found a similar pattern in the ANN: sensory PS peaked in the first hiddenlayer; logic PS peaked in the second hidden layer; and motor PS peaked in the last two hidden layers
(Fig. 6d).
We corroborated these findings using a continuous sensory-motor hierarchical gradientmap (without discretization) (Fig. 13-14). (In addition, see Fig. 15-17 for PS scores for all possiblerule dichotomies in human fMRI data and ANN activations.)
These findings suggest that abstractionemerges as a function of rule-dependent specialization and hierarchical organization.
Discussion, Limitations, Conclusions
We provide empirical support for the role of compositionality in human generalization, and implicateabstract representations as its neural implementation.
In classic ANNs, which are known to performpoorly during systematic generalization [17, 6], we found that computationally cheap pretrainingparadigms embedded abstract representations that led to human-like generalization performance andsample efficient learning.
When mapping abstract representations across cortex and ANN layers, wefound converging patterns of rule-specific abstractions from early sensory areas/layers to late motorareas/layers across human and ANN hierarchies.
These results reveal the hierarchical organizationof content-specific abstractions in the human brain and ANNs, while revealing the impact of theseabstractions for compositional generalization in models.
Our pretraining approach directly leverages knowledge of task structure to design pretraining routinesthat embed task biases into ANNs.
Despite the sample efficiency of this approach, this pretrainingapproach requires the initial overhead of designing paradigms useful for downstream learning.
A related approach that similarly requires prior knowledge of task structure is “representationalbackpropagation” – a regularization approach that aims to produce an idealized hidden representation[23].
However, there are other inductive bias approaches that do not require prior task knowledge.
Oneapproach constrains ANNs to produce abstract task representations by initializing ANN weights froma low-norm distribution [7]. However, initializing ANN weights in this regime is computationallycostly. Another approach is to initialize networks with built-in modular structures to facilitate there-use of network modules across tasks [30, 39]. However, exactly how such networks disentanglerepresentations has not yet been explored.
Nevertheless, all these approaches are complementary toeach other. It will be important for future work to assess how these approaches may synergisticallyinteract to optimize for sample-efficient generalization in multi-task settings. Though we provide comprehensive evidence of the role of abstraction in compositional generalization,there are several limitations in the present study that future research can explore. We found that thespatial topography of abstract representations was highly content-dependent.
However, analyseswere limited to cross-context manipulations of limited rule types (sensory, logic, and motor gating),without addressing the organization of other task components (e.g., reward or stimuli).
Thus, futurestudies can explore how brains and ANNs represent the abstraction of other task components. Second,though we were able to explore cross-context generalization across 64 contexts – significantly morethan previous empirical studies in neuroscience – cross-context analysis was limited to a singletask type (i.e., the C-PRO paradigm). It will be critical to see the organization of abstraction inmulti-task settings that go beyond 64 contexts. Finally, our ANN modeling approach revealed thecomputational benefits of pretraining.
It will be important for future work to benchmark sampleefficiency and generalization performance against other training paradigms (e.g., in continual learningand/or meta-learning settings; [12, 44]).
In conclusion, we characterized a convergent hierarchical organization of abstract representationsacross the human cortex and in ANNs using a 64-context paradigm, and provided insight into theimpact of abstract representations on generalization performance.
Overall, we found that simplepretraining tasks efficiently embed abstract representations into ANNs, leading to improved systematicgeneralization similar to human behavior.
These findings provide a human-centric benchmark fromwhich to understand compositional generalization in ANNs, paving the way for greater interpretabilityof compositionality in ANNs. Importantly, investigating compositionality through a human-centricframework (e.g., by benchmarking ANNs against humans in the same task) creates a concrete targetfor interpreting the strengths and limitations of compositionality in ANNs. We hope these findingsinspire further investigations into the comparison of compositionality in humans and ANNs. Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect
the
paper’s
contribu-
tions
and scope?
Yes
(b) Did you describe the limitations of your work?
[Yes] see Section 4.
(c) Did you discuss any potential negative societal impacts of your work?
[N/A]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
[Yes]
All human data has been de-identified and been publicly made available.
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[N/A]
(b) Did you include complete proofs of all theoretical results?
[N/A]
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experimental
results (either in the supplemental material or as a URL)?
[Yes] Public data URL is written in
the text (Section 2.1)
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
[Yes]
(c) Did you report error bars (e.g., with respect to the random seed after running experiments
multiple times)?
[Yes] Applicable to Figures 1, 5, 6,
(d) Did you include the total amount of compute and the type of resources used (e
g., type of GPUs,internal cluster, or cloud provider)?
[Yes]
[Table]
If you are using existing assets (e
g., code, data, models) or curating/releasing new assets...
(a)
If your work uses existing assets, did you cite the creators?
[Yes] Used previously publisheddata [19]
[Table]
(b) Did you mention the license of the assets?
[Yes] Published under a CC0 license, mentioned in
[Table]
section A.1.
[Table]
(c) Did you include any new assets either in the supplemental material or as a URL?
[No]
[Table]
(d) Did you discuss whether and how consent was obtained from people whose data you’re us-
ing/curating?
[Yes] Yes, see section A.1.
(e) Did you discuss whether the data you are using/curating contains personally identifiable
informa-
tion or offensive content?
[Yes] Yes, see section A.1.
All data was previously de-identified.5.
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if applicable?
[Yes] Yes – see Figure 1A
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB)approvals, if applicable?
[Yes] IRB approval was mentioned in section A.
(c) Did you include the estimated hourly wage paid to participants and the total amount spent onparticipant compensation? [No] This was a previously published dataset. A Supplementary MethodsA.1 Participants The following description is quoted with citation from a previous study that used the same dataset [20].
The 523 dataset was publicly published under a CC0 license.
All data on the public repository was previously deidentified.
Data were collected from 106 human participants across two different sessions (a behavioral and an imaging 526 session).
Technical error during MRI acquisition resulted in removing six participants from the study.
Fouradditional participants were removed from the study because they did not complete the behavior-only session.
528 fMRI analysis was performed on the remaining 96 participants (54 females).
All participants gave informedconsent according to the protocol approved by the Rutgers University Institutional Review Board.
The average 530 age of the participants that were included for analysis was 22.06, with a standard deviation of 3.84.
A.2 C-PRO task paradigm – additional details
The C-PRO cognitive paradigm permutes specific task rules from three different rule domains (logical decision, 533 sensory semantic, and motor response) to generate dozens of novel and unique task contexts.
Visual stimuliincluded either horizontally or vertically oriented bars with either blue or red coloring.
Simultaneously presented 535 auditory stimuli included continuous (constant) or non-continuous (non-constant, i.e., “beeping”) tones presentedat high (3000Hz) or low (300Hz) frequencies.
A given task context could be presented with 256 unique 537 stimulus combinations.
This is because a given task context was presented with two sequentially presentedaudiovisual stimuli, where each audiovisual stimulus varied in four dimensions: color (red/blue), orientation 539 (vertical/horizontal), pitch (high/low), continuity (continuous/beeping).
This led to 2 8 = 256 possible stimuluscombinations.
The paradigm was presented using E-Prime software version 2.0.10.353 [43].
Each rule domain (logic, sensory, and motor) consisted of four specific rules, while each task context was a 542 combination of one rule from each rule domain.
A total of 64 unique task contexts (4 logic rules x 4 sensory rulesx 4 motor rules) were possible, and each unique task set was presented twice for a total of 128 task miniblocks.
This meant that there were 256 ∗ 64 = 16384 unique trials (i.e., context-stimulus) combinations.
Identical taskcontexts were not presented in consecutive blocks.
Each task miniblock included three trials, each consisting of 546 two sequentially presented instances of simultaneous audiovisual stimuli.
A task block began with a 3925 msencoding screen (5 TRs), followed by a jittered delay ranging from 1570 ms to 6280 ms (2-8 TRs; randomly 548 selected).
Following the jittered delay, three trials were presented for 2355 ms (3 TRs), each with an inter-trialinterval of 1570 ms (2 TRs).
A second jittered delay followed the third trial, lasting 7850 ms to 12560 ms (10-16 550 TRs; randomly selected).
A task block lasted a total of 28260 ms (36 TRs). Subjects were trained on four of the64 task contexts for 30 minutes prior to the fMRI session. The four practiced rule sets were selected such that all 552 12 rules were equally practiced. There were 16 such groups of four task sets possible, and the task sets chosento be practiced were counterbalanced across subjects.
Subjects?
mean performance across all trials performed 554 in the scanner was 84% (median=86%) with a standard deviation of 9% (min=51%; max=96%).
All subjectsperformed statistically above chance (25%).
A.3 Analysis of human task performance data
The corresponding results described in this section can be found in Fig.
We calculated the average accuracy for each task miniblock (comprising three task trials).
Note that each task 559 miniblock had the same task context (three-rule combination) for all three trials.
This resulted in 128 taskaccuracy scores for every subject.
Task rule contexts were then sorted into three categories separately for every 561 subject ?
practiced, 2-rule overlap, and 1-rule overlap.
Practiced task contexts were defined as the four taskcontexts that were used to train participants on the C-PRO task outside of the MRI scanner. 2-rule overlap tasks 563 were task contexts that had at least 2 of the same rules overlapping with the previously seen practiced tasks, and1-rule overlap tasks were tasks with only a 1-rule overlap with practiced tasks. Note that there were no 0-rule 565 overlap tasks, since subjects were trained on every rule prior to the test session. Moreover, there were no ruleoverlaps across practiced task contexts.
Finally, every participant was provided with a randomly selected set of 567 practiced task contexts. Behavioral accuracy was computed for every task context group for every subject (Fig.1b). We next evaluated the successive miniblock performance of each rule presented in a novel task context during the 570 fMRI scanning session (see Fig. 1c-d).
This would capture a participant?s ability to use a previously seen rule ina novel context as a function of the number of times it used the rule previously. The performance of an individual 572 rule (e.g., “BOTH”) was calculated separately per participant as a function of each novel context seen. Thus,performance of each rule was calculated for exactly 15 miniblocks (each rule was presented 16 times, including 574 the practiced miniblock). For each rule, we then fit a linear regression model to assess whether performance(dependent variable) could be calculated as function of miniblock presentation (independent variable) with a positive coefficient (i.e., increasing slope).
A significant positive increase of performance (increasing positive 577 performance) was tested for significance by assessing the p-value of the beta coefficient (p < 0.05 threshold).
Thiscaptured compositional learning ? re-using a task rule (despite use in a novel context) indicated that participants 579 were learning to use previously learned rules in out-of-set novel contexts. A.4 fMRI acquisition The following fMRI acquisition details are taken from a previous study that used the same data set [20].
Whole-brain multiband echo-planar imaging (EPI) acquisitions were collected with a 32-channel head coil on a 583 3T Siemens Trio MRI scanner with TR=785 ms, TE=34.8 ms, flip angle=55 ◦ , Bandwidth 1924/
Hz/Px, in-plane
FoV read=208 mm, 72 slices, 2.0 mm isotropic voxels, with a multiband acceleration factor of 8.
Whole-brain 585 high-resolution T1-weighted and T2-weighted anatomical scans were also collected with 0.8 mm isotropicvoxels.
Spin echo field maps were collected in both the anterior to posterior direction and the posterior to anterior 587 direction in accordance with the Human Connectome Project preprocessing pipeline [11].
A resting-state scanwas collected for 14 minutes (1070 TRs), prior to the task scans.
Eight task scans were subsequently collected, 589 each spanning 7 minutes and 36 seconds (581 TRs). Each of the eight task runs (in addition to all other MRIdata) were collected consecutively with short breaks in between (subjects did not leave the scanner).  A.5 fMRI preprocessing The following details are quoted with citation from a previous study that used the same preprocessing scheme 593 [20].
Resting-state and task-state fMRI data were minimally preprocessed using the publicly available Human Connec- 595 tome Project minimal preprocessing pipeline version 3.5.0.
This pipeline included anatomical reconstruction andsegmentation, EPI reconstruction, segmentation, spatial normalization to standard template, intensity normaliza- 597 tion, and motion correction64.
After minimal preprocessing, additional custom preprocessing was conducted on
CIFTI 64k grayordinate standard space for vertex-wise analyses using a surface based atlas [11].
This included 599 removal of the first five frames of each run, de-meaning and de-trending the time series, and performing nuisanceregression on the minimally preprocessed data [3].
We removed motion parameters and physiological noise 601 during nuisance regression.
This included six motion parameters, their derivatives, and the quadratics of thoseparameters (24 motion regressors in total). We applied aCompCor on the physiological time series extracted 603 from the white matter and ventricle voxels (5 components each extracted volumetrically) We additionallyincluded the derivatives of each component time series, and the quadratics of the original and derivative time 605 series (40 physiological noise regressors in total). This combination of motion and physiological noise regressorstotaled 64 nuisance parameters, and is a variant of previously benchmarked nuisance regression models [3].
A.6 fMRI activation estimation
We performed a within-subject task GLM on the vertex-wise fMRI time series to estimate task rule-related 609 activations on the CIFTI grayordinate space.
To extract task activations for each task block, we performed a betaseries regression on every task miniblock [37].
Specifically, we fit an independent regressor to every encoding 611 period (3925ms, 5 TRs), resulting in 128 task regressors in total.
Fitting regressors on the encoding periodwas done primarily to isolate rule representations rather than the actual trial (stimulus-response) period.
Each 613 regressor was a boxcar function that was a vector of 0s every except for the specified encoding period.
Thisboxcar function was then convolved with the SPM canonical hemodynamic response function [9].
A single 615 activation estimate (beta coefficient) was extracted for every encoding block at every surface vertex.
A.7 ANN construction and batch training
The primary ANN architecture was comprised of two hidden layers, each with 128 units.
The output layer 618 was comprised of four units that corresponded to each motor response.
The ANN transformed the trial inputvector into a 4-element response vector with the equation Y = f ReLU ( X h W h
+ b h ) , where Y corresponds to 620 the output vector, W h is the weight matrix from the last hidden layer to the output layer, X h is the activation 621 vector of the last hidden layer, and b h is the bias vector.
The hidden unit activation vectors were defined as
X i = f ReLU ( X i − 1 W i − 1 + b
i − 1 ) , where X i is the activation vector for layer i .
Weights and biases wereinitialized from a uniform distribution U ( − (cid:112) 1 /k, (cid:112) 1 /k ) , where k is the number of input features from theprevious layer.
The ANN was optimized by minimizing the cross entropy between the outputs and the correct target output (a 626 one-hot vector).
Optimization was performed using Adam with a learning rate of 0.001 [24].
During training,we used dropout (with probability 0.2 from input to hidden, and 0.5 within hidden layers).
Training on the C-PRO task was performed in a sequential learning paradigm.
Initially, an arbitrary set of four 629 practiced contexts were selected with the constraint that no set of four practiced contexts had any overlapping rules.
(This was randomly selected across different ANN initializations).
Then, novel task contexts were 631 incrementally added (by 1) into the set of training contexts, and performance (and ANN analysis) was performedafter the addition of each task context.
Training stopped once all 64 task contexts had been fully trained on.
We 633 used batch training.
Each batch contained a single task context with all possible stimulus (256) combinations.
Thus, each batch contained 256 trials in total.
To stop training we set criteria for two different experiments.
The first experiment required that performance 636 on each task context (each batch) in the training set achieved a baseline performance accuracy or better (90%). Once this criterion was satisfied, a novel task context would be added into the training. The second experiment 638 kept fix the number of batches/gradient steps each task context took prior to adding a new task context. This wasset to 200 gradient steps per task context.
A.8 ANN pretraining description
There were two pretraining paradigms:
Primitives pretraining and Simple task pretraining.
Primitives pretraining focused on training individual rules within each rule domain, enabling ANNs learn 643 individual rule
Primitives (e.g., the notion of what “RED” is), and is consistent to how humans enter the full
C-PRO experiment – most participants already know what the primitives “RED”, “LEFT MIDDLE”, or “BOTH” 645 refer to.
While C-PRO task contexts activated 3 rules out of 12 possible task rules, Primitives tasks only activated1/12 rules.
Primitives pretraining was performed for each of the rule domains separately.
Motor Primitives pretraining involved making motor responses given a motor rule.
This involved activating 648 one motor rule in the input vector at a time.
If the “LEFT MIDDLE” rule was activated, a LEFT MIDDLEoutput response would be expected.
In addition, if the “NOT” unit was activated in conjunction with the “LEFT 650 MIDDLE” rule, then the “LEFT INDEX” output response would be expected, which is the analogous ruleinstruction that participants received prior to performing the C-PRO task.
Sensory Primitives pretraining involved making True/False statements on whether a specific sensory stimulus 653 feature was presented.
This involved activating one sensory rule and one sensory stimulus feature at a time.
Forexample, for the sensory rule “RED”, either a RED/BLUE sensory stimulus would activate.
During pretraining, 655 we included two additional output units that corresponded to True/False units.
If a “RED” stimulus was presentedin conjunction with “RED” sensory rule, the output should be “True”; otherwise, “False”.
In addition, we also 657 presented sensory rules with the “NOT” negation.
In other words, if the rule was “NOT RED” and the sensorystimulus presented was RED, then the network should produce “False”.
Logic Primitives pretraining involved learning the abstract logical relations between different logic rules.
660 The logic rules “BOTH” and “EITHER” could be equivalently interpreted as “AND” and “OR” logic rules,respectively.
The “NOT BOTH” and “NEITHER” rules were analogous to the negations of those rules.
This was 662 operationalized by presenting two stimuli from a specific feature domain (e.g., color).
Using stimuli from thecolor feature domain as an example, for “BOTH” Primitive training, if RED-then-RED stimuli were presented, 664 this would result in a “True” boolean.
In contrast, if RED-then-BLUE stimuli were presented, then this wouldresult in a “False” boolean.
This intuition was derived from conditional logic (where “BOTH” is equivalent to 666 the concept of “SAME”), where the statement x == x is True.
In contrast, “EITHER” (i.e., “OR”) was codedas “True” for any combination features of a color stimuli.
The negations “BOTH” and “EITHER” resulted in 668 the negation of the produced boolean (i.e., “NEITHER” produced “False” for all stimulus pairs).
The mostcritical point, however, is that the responses for all logic rules were distinct among each other, and described 670 equivalent logical relations between rules.
Previous theoretical work in cognitive science suggests that symboliccomputation emerges by learning the relational representations between symbolic operations [35].
This logic 672 Primitives pretraining approach therefore captures the relations amongst symbolic operations.
Simple task pretraining involved combining two-rule task context, rather than the full three-rule task context in 674 the C-PRO task.
We designed two variants of simple tasks: a sensorimotor task that combined a sensory andmotor rule, and a logical-sensory task that combined a logic and sensory rule.
The sensorimotor task activated a 676 sensory rule, motor rule, and one sensory stimulus unit.
The response output was a motor response.
For example,if the rules were “RED” and “LEFT MIDDLE”, and the stimulus was “RED”, ANNs were taught to respond 678 with the Left Middle response unit.
If the stimulus was anything other than “RED”, then ANNs were taught torespond with the Left Index response unit.
The sensorimotor task was used to teach ANNs simple mappings 680 between sensory input and motor responses in the simplest possible paradigm.
The logical-sensory task was designed to teach ANNs logical inferences over sensory stimuli in the simplest 682 possible manner.
Like the sensorimotor task, the logical-sensory task included two rules: a logic and sensoryrule.
However, unlike the sensorimotor task, it activated two stimulus units from the same feature domain (e.g., 684 the color domain).
For example, for the logic rule was “BOTH” and sensory rule “RED”, if the first stimuluswas red and second stimulus red, the ANN would be taught to respond with the True output unit.
If any of the 686 stimuli were not red (e.g., blue), then the ANN would be taught to respond with the False output unit.
Primitives pretraining was always performed prior to Simple task pretraining, except for Fig.
12, which 688 investigated the effect of reversing the order of pretraining tasks.
Pretraining procedures were blocked together,such that all conditions within the Primitives pretraining paradigm (i.e., Logic, Sensory, and Motor primitives 690 pretraining) were trained until all three tasks achieved 99% accuracy.
Simple task pretraining was subsequentlyperformed until both Logical-Sensory and Sensory-Motor tasks were performed at 99% accuracy.
This ordering 692 is consistent with prior work, suggesting that ANNs are more sample efficient when transitioning from easier tomore difficult tasks [42].
We also performed a simple control experiment demonstrating that when pretraining 694 was reversed in the Combined condition (i.e., Simple task pretraining followed by Primitives pretraining),generalization performance was reduced to chance (Fig. 12).
This suggests that the ordering of pretraining 696 paradigms is crucial for generalization performance and sample efficiency, which future work should explore.
After pretraining, the additional True/False output units were lesioned from the network.
Analysis of ANNs was carried out in a similar manner to how empirical fMRI data and behavior was analyzed. 700 ANN analysis was performed to infer how the structure of their internal representations was associated withtask generalization performance and sample efficiency. Task generalization performance was calculated as the 702 performance on novel contexts (i.e., novel recombinations of task rules).
This was independent of whether or not
ANNs had learned/seen individual rules previously. To estimate task sample efficiency, we calculated the number 704 of trial samples required to achieve a baseline task accuracy percentage (on contexts in the training set). Notethat for a fixed number of trial samples, the number of gradient steps were the same (batch sizes were always the 706 same). Generalization inefficiency was measured as the ratio of the number of samples trained on (normalizedbetween 0 and 1) and the generalization performance on novel contexts (normalized to 0 to 1 + a fixed constant).
PS in ANNs was calculated separately from the training procedure.
Since we were only interested in the PS of 709 rule representations, only input units associated with the task rules were activated, while stimulus inputs wereset to 0. This ensured that the hidden activations were not contaminated by stimulus-related activations when 711 calculating PS in the hidden layers. Otherwise, PS in ANNs was calculated in a similar manner to the empiricalfMRI data, where the spatial features (dimensions) were the units within a given hidden layer (like voxels within 713 a brain parcel). In addition, Figures 15, 16, and 17 illustrate that pairwise PS scores for all pairs of rule dichotomies in both 715 ANNs and human fMRI data.
A.10 Computing resources
fMRI analyses were carried out on a local server with 24 cores and 320GB of RAM. ANN training, while not 718 required, was performed on an Nvidia P100 GPU. A single ANN initialization can be successfully trained on a CPU in under 2 minutes.
B Supplementary Figures
Similar to Fig.
but we trained the model using a fixed number of samples per learned context (200 epochs per context) and evaluated generalization performance. a) We systematically trained each type of ANN on a range from 0-63 of the C-PRO task contexts, and assessed its generalization on the remaining (novel) contexts. We trained the model on a fixed number of samples per learned context. Learned contexts were sequentially introduced, and novel context performance was assessed on the remaining (excluded from training) contexts.
b) The number of total trials/samples shown to each model type. Since the number of samples shown to the model for each context was fixed, training samples linearly increased as a function of learned contexts. c)
As in Fig. 10c, we measured the generalization inefficiency of each model. Generalization inefficiency was measured as the ratio of the number of samples shown (normalized between 0 and 1) and the performance on novel tasks (normalized to 0 to 1 + a fixed constant). Thus, while the vanilla model was initially more efficient, pretrained models quickly learned more efficiently as evidenced by better generalization accuracies with fewer samples.
curriculum research that the of learning can significantly impact generalization performance we compared the Combined pretraining protocol to a Reverse-combined protocol, where Simple task pretraining was performed prior to Primitives pretraining.
a) We found that generalization performance on the unseen C-PRO tasks was significantly impaired in the Reverse-combined condition.
b) We also measured the sample efficiency of only the pretraining trials. We found that Reverse-combined pretraining required more samples than Combined pretraining, despite the stopping criterion for each pretraining task remaining the same. (ANNs were required to perform either Primitives pretraining or Simple task pretraining with at least 99% accuracy prior to moving on to the next pretraining task.) This suggests that ordering of pretraining tasks can significantly impact the learning and generalization dynamics of simple ANNs.
We compared the topographic differences in PS across cortex with a well-known sensory-to-motor hierarchical gradient identified during resting-state fMRI [29].
b) We compared how content-specific abstractions (PS) differed across this hierarchical gradient, finding that sensory rule abstractions were highest in the lower part of the gradient (sensory systems), logic rule abstractions were highest across association cortex, and motor rule abstractions were highest across motor cortex.
c)
For an analogous analysis, we plotted how domain-specific abstraction differed across different hidden layer depths in the pretrained ANN with 3 hidden layers.
We identified similar patterns of parallelism across thehidden layers for each of the rule domains.
match the number of layers in
, we created three discretized systems based on the functional network partition – sensory,
, and motor – that followed the sensory-to-motor hierarchy (Fig. 6) [29]. a) We computed the PS for every pairwise dichotomy in the Logic rule domain, and computed the average PS across brain regions within each cortical system (Sensory, Association, and Motor cortex). The average PS for each cortical system was computed by averaging the PS across all regions within that cortical system for every dichotomy. b) To compare how the pairwise dichotomies matched in pretrained ANNs (Combined pretrained only; no training on full C-PRO contexts), we computed the
PS for all dichotomies in each layer.
In this particular experiment we used an ANN with three hidden layers to compare with Sensory, Association, and Motor cortical systems. c) The sensory, association, and motor cortical systems can be analogized to hidden layer depths in the ANN. We computed the PS for every pairwise dichotomy in the Sensory rule domain, and computed the average PS across brain regions within each cortical system (Sensory, Association, and Motor cortex).
The average PS for each cortical system was computed by averaging the PS across all regions within that cortical system for every dichotomy. b) To compare how the pairwise dichotomies matched in pretrained ANNs (Combined pretrained only; no training on full C-PRO contexts), we computed the PS for all dichotomies in each layer. In this particular experiment we used an ANN with three hidden layers to compare with Sensory, Association, and Motor cortical systems.
c)
The sensory, association, and motor cortical systems can be analogized to hidden layer depths in the ANN. a) We computed the PS for every pairwise dichotomy in the Motor rule domain, and computed the average PS across brain regions within each cortical system (Sensory, Association, and Motor cortex). The average PS for each cortical system was computed by averaging the PS across all regions within that cortical system for every dichotomy. b) To compare how the pairwise dichotomies matched in pretrained ANNs (Combined pretrained only; no training on full C-PRO contexts), we computed the
PS for all dichotomies in each layer.
In this particular experiment we used an ANN with three hidden layers to compare with Sensory, Association, and Motor cortical systems. c) The sensory, association, and motor cortical systems can be analogized to hidden layer depths in the ANN.