Convergent Representations of Computer Programs in Human and Artificial Neural Networks
Anonymous Author(s)
AffiliationAddress email
Abstract
What aspects of computer programs are represented by the human brain duringcomprehension?
We leverage brain recordings derived from functional magneticresonance imaging (fMRI) studies of programmers comprehending Python code toevaluate the properties and code-related information encoded in the neural signal.
We first evaluate a selection of static and dynamic code properties, such as abstractsyntax tree (AST)-related and runtime-related metrics.
Then, to learn whetherbrain representations encode fine-grained information about computer programs,we train a probe to align brain recordings with representations learned by a suite of
ML models.
We find that both the Multiple Demand and Language systems–brainsystems which are responsible for very different cognitive tasks, encode specificcode properties and uniquely align with machine learned representations of code.
These findings suggest at least two distinct neural mechanisms mediating computerprogram comprehension and evaluation, prompting the design of code modelobjectives that go beyond static language modeling.
We make all the correspondingcode, data, and analysis publicly available.
Introduction
Computer code comprehension is a complex task which recruits multiple cognitive processes—fromsyntactic parsing to mentally simulating programs.
Despite the prevalence of this task, the representations of code processed in the human brain during code comprehension remain uninvestigated.
Isit possible that common code properties and program semantics are faithfully represented in brainactivity patterns when code is read and evaluated?
A few prior works have used data derived fromfunctional magnetic resonance imaging (fMRI) and electroencephalography (EEG) to locate physicalregions in the brain involved in code-related activities like code comprehension and debugging[Siegmund et al., 2017, Floyd et al., 2017, Peitek et al., 2018, Castelhano et al., 2019, Ivanova et al.
,2020, Liu et al., 2020, Ikutani et al., 2021, Peitek et al., 2021], code writing [Krueger et al., 2020,
Karas et al., 2021] and data structure manipulation [Huang et al., 2019].
These studies have helpeddetermine whether code-related activities join other activities supported by those brain regions, suchas working memory (processed by a set of brain regions known as the Multiple Demand system)or language processing (processed by the Language system–another set of brain regions).
Whilethese results improve our understanding of the brain regions involved in code comprehension, it stillremains unclear what specific code-related information these regions encode.
For example, does theresponse in a region seen during code comprehension encode specific syntactic or semantic codeproperties?
Do responses from multiple brain regions correspond to the same set of properties?
Or,are different code properties encoded in different regions?
A possible approach and its limitation.
One way to learn what information is encoded in the brainis to decode a code property of interest from recordings of brain signals when reading code (through fMRI or EEG).
Being able to decode the property accurately from a specific region of the brainestablishes that information related to that code property is faithfully represented in that brain region.
A question central to such a decoding analysis is the choice of the target code property–what codeproperties should be investigated?
We can hand-select a set of fundamental properties of code andtest if they can be decoded.
While helpful, such a set will not preclude other, more complex aspectsof code possibly being encoded.
ML models of code as a tool to reverse engineer what is encoded.
To address the limited scopeof hand-selected properties, we look to machine learning (ML) models trained on code.
Dubbedcode models , they are trained on large corpora of code, in an unsupervised manner, to learn MLmodel representations of computer programs.
Code models are increasingly being used in softwareengineering workflows [Allamanis et al., 2018a], and have been shown to perform well on tasks likecode summarization
[Alon et al., 2019], detecting variable misuse [Bichsel et al., 2016], and morerecently, code auto-completion [Chen et al., 2021].
These representations have been shown to encodeand describe complex code properties [Bichsel et al., 2016, Allamanis et al., 2018b, Srikant et al.
Successfully decoding these code model representations from brain activity data then allowsus to probe whether complex code properties are also encoded in the brain.
Besides serving as a tool to reverse engineer what is encoded in different brain regions, anotherdistinct advantage of decoding code model representations is its potential to serve as a tool to reverseengineer our cognitive processes.
It is currently unclear what mechanisms drive code comprehension.
If we find one class of ML models (say, masked language models) to be more predictive than another(say, autoencoders), it is reasonable to suspect that our brains optimize objectives more similarto that of masked-LMs than that of autoencoders when comprehending code.
As a corollary, apoor correspondence between the information encoded by our brains and ML models suggests thepossibility of unexplored neural architectures and objectives which may better model our cognition,which in turn may outperform extant ML models.
Yamins et al.
[2013] first showed how informationencoded in our visual system resembles what convolutional neural networks learn when trained torecognize images.
We attempt to establish a similar correspondence between code models and thehuman brain in comprehending code.
Why code comprehension?
We focus on code comprehension in this work because very littleof this important skill has been analyzed from a cognitive neuroscience perspective while steadyadvances are being made in training ML models to understand code and increase programmerproductivity [Hellendoorn and Sawant, 2021].
Extant ML models for understanding programmingare direct adoptions of the state-of-the-art in language processing research.
However, recent worksin neuroimaging like Liu et al.
[2020] and Ivanova et al.
[2020] suggest that code comprehensiondoes not share the same neural bases as natural language comprehension.
Do code models thenmimic human cognition of programs?
Do language models?
If not, can we consider other modelarchitectures and training objectives which are directly inspired by results from neuroscience?
Ourwork provides early considerations towards these directions.
Further, characterizing the representations of different code properties in the human brain canhelp learn more about the nature of human algorithmic problem solving more generally.
We cancharacterize signatures of brain activity corresponding to fundamental logic operators like iterativereasoning, conditional reasoning, retrieving calculations that were previously computed, etc . seen inindividual participants when solving carefully designed code comprehension tasks.
Such signatureswill then allow us to characterize other tasks which involve similar logic operators, but which are notnaturally described as computer programs e.g. general logic reasoning, problem solving, and moregenerally, our ability to employ abstractions and form concepts [Rule et al., 2020].
Our setup.
We introduce a framework to evaluate the code properties encoded in human brain representations of code by analyzing fMRI recordings of programmers comprehending Python programs.
We present two means of proceeding— one, probing of brain region representations for specific codeproperties, and two, analyzing the mapping of these representations onto various code models withdiffering model complexity.
We utilize the publicly available dataset from Ivanova et al.
[2020] forall our analyses as it offers high quality, granular brain response data on code comprehension stimulicontrolled across multiple code properties.
We train affine models on brain representations to predicthand-selected code properties which express syntactic and semantic behavior of programs.
Similarly,we predict code representations obtained from a suite of code models.
We investigate the effects that brain regions, the nature of code properties, and the complexity of the code models have on theaccuracies of these prediction tasks.
Key findings.
We explore two questions: (1) Do brain systems encode specific code properties?
Arethere differences in how well each brain system encodes each code property?
(2) Do brain systemsencode more complex properties of code, derived from the representations of code models?
We findthe Multiple Demand system and the Language system consistently encode both hand-selected codeproperties and data-derived code model representations, using features which cannot be explainedby only low-level visual characteristics of the code.
Within this set, the Multiple Demand systemmost effectively decodes runtime properties of code like the number of steps involved in a code’sexecution, while the Language system decodes syntax-related properties like the number of tokens ina program and the control structures present in it.
Our results improve the cognitive science domain of human program comprehension–by improvingour understanding of the functional organization of the brain systems we study, as well as objectivesemployed to train machine learning models of code–by offering new objectives for the design of codemodels.
We provide an open-source framework to replicate our experiments, and we release our data andanalysis publicly.
Link - https://github.com/anonmyous-author/anonymous-code .
This shouldenable authors from other neuroimaging studies or code model developers to collaborate and analyzedata across these works, which will also help amortize the high costs of carrying out such experiments.
Related Work
Of the prior works that have investigated the neural bases of programming through fMRI and EEGtechniques [Siegmund et al., 2017, Floyd et al., 2017, Peitek et al., 2018, Castelhano et al., 2019,
Huang et al., 2019, Krueger et al., 2020, Ivanova et al., 2020, Liu et al., 2020, Ikutani et al., 2021,
Peitek et al., 2021] and through behavioral studies [Prat et al., 2020, Casalnuovo et al., 2020, Crichtonet al., 2021], the following probe brain recordings for program properties encoded in them.
Floyd et al.
[2017] learn a linear model to successfully classify whether an observed brain activitycorresponds to reading code or reading text.
Ikutani et al.
[2021] study expert programmers and showthat it is possible to classify code into the four problem categories–math, search, sort, and stringfrom the brain activations corresponding to the code.
Similarly, Liu et al.
[2020] classify whether abrain signal corresponds to code implementing an if condition or not.
Peitek et al.
[2021] analyzecorrelations between brain recordings of participants reading code and a set of code complexitymetrics.
In testing for code properties, our work uses a similar methodology (a linear model trained onfMRI data), but we evaluate a larger set of static and dynamic code properties, often reflectingkey programming constructs like control flow.
Further, we perform these tests in the brain regionsidentified by Liu et al.
[2020] and Ivanova et al.
[2020] as being responsive specifically to codecomprehension, offering finer insight into the content of these specific regions’ representations.
Inaddition, we study representations generated by a suite of ML models with varying complexity andcompare those learned representations to brain representations.
Brain representations have also been studied in domains like natural language, vision, and motorcontrol.
Among related works in natural language, a domain that resembles programming languages,
Mitchell et al.
[2008], Pallier et al.
[2011], Brennan and Pylkkänen [2017], Jain and Huth [2018],
Gauthier and Levy [2019], Schwartz et al.
[2019], Wang et al.
[2020], Schrimpf et al.
[2021], Caoet al.
[2021], Caucheteux et al.
[2021], Toneva and Wehbe [2019] have studied brain representationsof words and sentences by relating them to representations produced by language models.
While thebroader tools we use to investigate these representations, like multi-voxel pattern analysis (MVPA),are similar to some of these prior works, our focus is on properties specific to code and not naturallanguage.
Background
We provide a brief background on fMRI signals as
Measuring brain activity with fMRI.
Functional magnetic resonance imaging (fMRI) is a brainimaging technique used to measure brain activity in specific brain regions.
When a brain region isactive, blood flows into the region to aid its processing.
An MRI machine measures this changein blood flow, and reports BOLD (blood oxygen level dependent) values sampled at the machine’sfrequency [Glover, 2011, usually 2 seconds].
The smallest unit of brain tissue for which BOLD signalis recorded is called a voxel (an equivalent of a 3D pixel); it comprises several cubic millimetersof brain tissue.
For our analyses, we select subsets of voxels belonging to specific brain systems.
Following common practice, the parameters of a general linear model, fit to time-varying BOLDvalues, are used as a measure of the overall activation in each voxel in response to a given input.
Itis the values of these parameters that, in concordance with common practices in the neurosciencecommunity, can be accessed as brain representations.
Brain systems.
A system of brain regions can span different areas of the brain but behaves as aholistic unit, showing similar patterns of engagement across a given cognitive task.
We probe thefollowing systems in our work: (a) Multiple Demand (MD) system: this system of regions is knownto engage in cognitively demanding, domain agnostic tasks like problem solving, logic, and spatialmemory tasks.
Liu et al.
[2020] and Ivanova et al.
[2020] reported that this system is active duringcode comprehension. (b) Language system (LS): this system responds during language productionand comprehension across modalities (speech, text) and languages (across 11 language families,including American sign language).
(c) Visual system (VS): these regions respond primarily tovisual inputs.
(d) Auditory system: these regions respond primarily to auditory inputs.
We probe the Visual system and Auditory system since they are involved in general perception.
While we do not expect activity in Auditory system, we expect the Visual system to reflect low-levelvisual properties of the code ( e.g. code length and indentation to reflect code-related properties.
See
Appendix A for a detailed description of these brain systems along with relevant references.
Brain and Model Representations
We describe in this section the method we follow to gather representations of code in the brain(Section 4.1), evaluate the different code properties they encode (Section 4.2), and how we comparebrain representations to those generated by ML models (Section 4.3).
4.1 Brain representations and decoding
We provide a summary of how we process activation signals in the brain elicited by code comprehension, to probe whether they encode any specific code properties.
We provide details in Appendix B.
Dataset.
We use the publicly available brain recordings released as part of the study by Ivanova et al.[2020] (MIT license).
It contains brain recordings of 24 participants, each of whom gave consentand is not personally identifiable according to Ivanova et al.
[2020] protocol requirements, reading72 programs from a set of 108 unique Python programs.
The 72 programs were presented in 12blocks of 6 programs each.
These programs were 3-10 lines in length and contained simple Pythonconstructs, such as lists, for loops and if statements.
A whole program was presented at once, and thetask required participants to read the code and mentally compute the expected output, press a buttonwhen done, and select one of four choices presented to them which matched their calculated output.
From dataset to brain representations.
The original dataset contains 3D images of the brain ofeach participant.
Each voxel value in these images is an estimate of the response strength in this voxelwhen a particular code (or sentence) problem is presented.
To determine which brain systems containinformation about particular code properties, we focus our analyses to four systems – MD, Language,
Visual, and Auditory (Section 3).
A vector of voxels’ activation values in each brain system is thentaken to constitute that system’s representation of a computer program and serves as an input to allour analyses.
See Appendix B for details on data processing and voxel selection.
Analyzing brain representations of code.
We probe brain representations from each participantseparately.
We do not average data across participants since the regions which respond to any task(comprehending code in our case) need not align anatomically.
For each of two experiments—decoding different code properties, and mapping to code model representations—we train ridgeregression/classification models which take as input normalized brain representations per participant.
We hence learn 24 different regression models, for each code property or code model (one perparticipant), and then report the mean performance of these models across participants.
This procedureis also referred to as multi-voxel pattern analysis (MVPA)
[Norman et al., 2006].
Linear models are conventionally preferred for probes into brain representations since there has been evidencesupporting the idea that other brain areas linearly map information from such brain representations[Kamitani and Tong, 2005, Kriegeskorte, 2011].
We choose a linear model primarily to control forover-fitting in light of the relatively small dataset.
A remark on data scarcity.
For each participant, we train a linear regression/classification modelwith L2-regularization for on unique cross-validated leave-one-run-out folds of the 72 programsthey attempted (or 48 programs when sentences are removed).
On the order of 1000 voxels wereselected from each brain region responding to any given program per participant, thus resulting in afeature set of dimensions 72 × 1000 .
Such dimensions allow for strong statistical tests to support therobustness of the predictions made.
As a baseline for the model predictions, we use the accuracy of anull permutation distribution generated from sampling 1000 random assignments of the labels.
4.2 Code properties
We attempt to decode the following code properties from brain representations.
See Appendix Bfor details.
(a) Code vs. sentences: classify whether an input stimulus is a code or an Englishsentences describing a code (referred to as sentences ).
See Figure B, Appendix B for an example.
predict static properties of aprogram like token count : number of tokens in the program, node count : number of AST nodes,cyclomatic complexity , and Halstead difficulty .
The latter two metrics have been used by softwareengineering practitioners to quantify the complexity of code, and to quantify the difficulty a humanwould experience when comprehending code respectively.
We defer predicting other advanced staticanalysis metrics such as tracking abstract interpretation joins, data flow analysis-related metrics,etc . to future work.
(f) Dynamic analysis: predict information about a code’s execution behaviorlike runtime steps : number of instructions executed in the program, and bytecode ops : number ofbytecode operations executed in running the program.
Program length as a potential confound.
Since the properties we examine can also potentially bedifferentiated using program length and other low-level code features, it is a potential confound inour experiments.
We measured the inter-correlations of these properties, and their correlation to thenumber of tokens in the program (program length; see Appendix H).
We expectedly found the fourstatic analysis properties to be highly correlated to each other and to bytecode ops .
We hence use onerepresentative metric each from the two categories of properties for the rest of our analysis– tokencount for static analysis , and runtime steps for dynamic analysis .
Importantly, the other propertieswe examine cannot be explained by program length alone, and therefore program length is not aconfound in our experiments.
Mapping to code properties.
The brain representations (Section 4.1) are mapped to each of thecode properties by training a ridge regression (for static analysis and dynamic analysis properties;continuous values) or a classification model each for every participant-property pair.
To evaluatemodel performance, we use classification accuracy when the predicted values are categorical ( e.g.string vs. numeric data types), and the Pearson correlation coefficient when the predicted values arecontinuous ( e.g. number of runtime steps).
We choose Pearson correlation over RMSE, the canonicaldistance metric for continuous values, for its simplicity and interpretability.
See Appendix J for the
RMSE results, which lead to the same conclusions as the correlation results.
When testing for thesignificance of these predictions, we perform false discovery rate (FDR) correction for the number ofbrain systems tested and the number of properties tested.
See Appendix B for a detailed descriptionof model hyper-parameters, cross-validation settings, and significance testing.
4.3 Model representations and decoding.
We evaluate a bench of unsupervised language models, spanning from count-based language modelsto transformer neural networks [Vaswani et al., 2017].
These models were all trained on large ( ∼ 1Mprograms)
Python datasets [Husain et al., 2019, Puri et al., 2021].
We use the output of the trainedencoders (raw logits) in each of the neural network models as representations of the code inputto the model.
We vary the general complexity of these models to test whether that variation ismeaningful in establishing the quality of brain to model fits.
Model complexity here is the number ofa model’s learnable parameters.
We evaluate the following models, ordered by their increasing modelcomplexity: simple frequency-based language models— bag-of-words , TF-IDF ; auto-encoder basedunsupervised models— seq2seq [Sutskever et al., 2014], CodeTransformer [Zügner et al., 2021],
CodeBERT [Feng et al., 2020], CodeBERTa [HuggingFace, 2020]; auto-regressive models withsimilar model complexity— XLNet
[Yang et al., 2019], CodeGPT [Microsoft, 2021].
Baseline: Token projection model.
We compare the results of the above models against an aggressivebaseline (relative to the null-distribution labeling baseline), a token projection model provided byusing unique Gaussian-distributed random vectors for the token embeddings in a vocabulary, andreturning the sum of these token embeddings across a program.
The resultant embedding is nottransformed by any model or any weights–it instead serves as a proxy for the tokens that appear inthe program.
The results of this baseline model should be interpreted as the level of performanceachievable from the presence of tokens alone with no semantic or syntactic information.
Mapping to code model representations.
The brain representations (Section 4.1) are mapped tocode representations by training another set of ridge regression models to learn an affine map, and aranked accuracy metric is used to compare outputs.
Ranked accuracy scores are commonly used ininformation retrieval where several elements in a range are similar to the correct one.
In our case, thetop-ranked prediction by the linear model indicates the closest fit (Euclidean distance) to the codemodel’s representation.
When reporting result significance, we perform false discovery rate (FDR)correction for the number of brain systems and the number of models.
See Appendix B for a detaileddiscussion of the implemented code models and the corresponding metrics.
All experiments in thiswork were run on a single 8-core laptop in under an hour following setup.
Experiments & Results
Our experiments address two research questions:
• Experiment 1.
How well do the different brain systems encode specific code properties?
Do they encode the same properties?
• Experiment 2.
Do brain systems encode additional code properties represented by computationallanguage models of code?5.1 Experiment 1 - How well do the different brain systems encode specific code properties?Do they encode the same properties?
We analyze the classification models and regressions trained on brain representations to predicteach of the code properties described in Section 4.2.
The results of our analyses are summarized in
Figure 2.
The classification and regression tests are marked on the x-axis of the left and right subplotsrespectively; the classification accuracy or Pearson correlation for each of the tasks is marked on they-axes.
We plot dynamic and static properties separately from the others because their baselines aredifferent due to a difference in the similarity metric (classification accuracy vs. Pearson correlation).
The baselines for the categorical code properties differ from each other due to variation in the numberof target classes.
Auditory and Visual systems.
The Auditory system and the Visual system serve as negative andbaseline controls for the other systems.
Since our code comprehension task is visual, we do notdecode any meaningful information between programs from Auditory system, although we doobserve decoding of code vs sentences , perhaps explained by previous work showing auditory cortexactivation during silent sentence reading [Perrone-Bertolotti et al., 2012].
The Visual system serves as a baseline for low-level visual features of the code (the layout and indentation of the code, thepresence of letters and alphabets in the programs, etc .).
In Analysis 2, we show that the MD and the
Language systems encode more than such visual features.
The MD and the Language systems yieldthe following observations.
Analysis 1 - How accurately are different properties predicted by MD and LS?
The MD and
LS together decode all the properties well above chance barring variable language.
The variablelanguage finding is consistent with Ivanova et al.
[2020], who show a lack of any significant differencein the aggregate neural activity between English and Japanese variables names—variable namesseem to not be encoded any differently in the context of programs we study.
To analyze the otherresults, we use paired two-sample t -tests ( p < 0 .
FDR-corrected) and examine whether for agiven property, any one brain system decodes it significantly more effectively than another (detailsin Table 8, Appendix E).
We find that the MD system decodes the dynamic analysis property betterthan the Language system.
We additionally test if any brain system has a preference for a specificcode property over another.
We find that the MD decodes the dynamic analysis property significantlybetter than the static analysis property.
These findings establish the role of the MD system inencoding code-simulation and execution related information—an important of aspect of codecomprehension, which was previously .
Analysis 2 - Multi-system partial regression analysis.
The decoding performance of the VS iscomparable to that of the MD and LS (Figure 2).
To assess the possibility that all three systems - MD,
LS, and VS encode the same properties (all potentially related to low level program features), weemploy a multi-system partial regression analysis.
For each brain system, MD and LS ( S i ), we traintwo models–one which decodes from VS, and another which decodes from S i + VS.
If the differencein the prediction accuracies between the two models is significant, we conclude that S i encodes atleast some information which is orthogonal to the information encoded by the Visual system.
Thismethod is similar to a variance-partitioning analysis which is often employed in encoding models,e.g. Deniz et al.
For all core properties, control flow , data type , dynamic analysis , and staticanalysis , we find the MD to encode information orthogonal to the VS.
For control flow and staticanalysis , the LS also encodes information orthogonal to the VS.
This suggests that low-level codeproperties are insufficient to explain the key results from Experiment 1.
Other combinations in theregression model reveal that the MD encodes information orthogonal to the LS when predicting codevs. sentences and dynamic analysis .
Detailed results are tabulated in Table 15, Appendix F.
Key learnings from Experiment 5.1.
All core code properties are decodable from the representationsof high-level brain systems, and this information is beyond that which can be explained from low-levelvisual information alone.
Although no property is exclusively encoded in any one brain system,the MD system significantly encodes dynamic analysis -related properties–more than what the LSencodes, and more than static analysis properties.
Similarly, we find evidence for the LS to alsosignificantly encode static analysis and control flow related properties.
These are new results on thenature of code properties different brain systems seem to process and encode.
To explore propertieswhich may not be specified by the set we investigate in this experiment, in the following section, we leverage code models as hypothesis-free proxy representations for code syntax and semantics, andsee if any one system preferentially encodes a code model.
5.2 Experiment 2 - Do brain systems encode additional code properties encoded bycomputational language models of code?
We train ridge regression models with brain representations of programs from specific brain systemsto predict code model representations of the same programs.
The set of results from this experimentare summarized in Figure 2.
Auditory and Visual systems.
Similar to Experiment 1, these systems perform as expected, withthe Auditory system exhibiting the lowest decoding performance across code models, and the Visualsystem acting as a proxy for low-level information (Table 10, Appendix E).
Analysis 1 - How well do brain representations in MD and LS map to code model representations?
We find that the MD and LS map to all the models in our suite significantly more accuratelythan the null permutation baseline (Figure 2).
Further, we find that the MD ranked accuracy ishigher than LS for CodeBERTa , CodeTransformer , seq2seq and TF-IDF (differences evaluated usingtwo-sample t -tests; p < 0 .
05 ; FDR-corrected.
See Table 10, Appendix E).
Analysis 2 - The effect of model complexity on decoding to code models.
We investigate the impactmodel complexity has on the performance of the mapping between brain and code representations.
We compare each of these code models against the Token Projection baseline model, which onlyencodes the presence of specific tokens, with no contextual or distributional information. We findthat the MD system maps to all the models more accurately than the Token Projection , but this is notobserved for the LS.
In a set of paired two-tailed t -tests ( p < 0 . 05 ;
FDR-corrected), we find that the MD maps to three models: CodeBERTa , CodeTransformer , and bag-of-words significantly moreaccurately than to the Token Projection model (Table 11, Appendix E).
Curiously, since all but 3of these mappings do not significantly surpass the model which can be explained using only tokenlevel information, these data suggest that the brain signals we access primarily encode token-levelinformation.
To investigate this further, we analyze the correspondence between brain representationsand code models in the context of the code properties they encode.
Analysis 3 - Code model and brain representations in the context of code properties. We firstevaluate whether the different code properties we investigate in this work can be decoded fromcode model representations. We find that all the code properties are strongly encoded in all models(Table 4, Appendix C.2). Since we have computed the mapping accuracies from code models →code properties (mentioned above), and from brain representations → code properties (Tables 1 & 2,
Appendix C.1)
, we compute how well brain representation probing results map to code model probingresults using Spearman rank correlation (details in Appendix G).
We find two clusters in decodingperformance across the code properties, each reflecting a distinct computational motif (Table 17,
Appendix G).
We see perfect rank correspondence between the z -scores of the probing results forthe MD and three transformer architectures: CodeBERTa , CodeBERT , and CodeTransformer , andbetween LS and two token-based models: bag-of-words and Token Projection .
Key learnings from Experiment 5.2.
The MD decodes representations from four models of codesignificantly more accurately than LS, providing evidence that some aspects of code are morefaithfully represented in the MD.
A follow up partial regression analysis, as in 5.1, reveals that formost models, MD encodes information orthogonal to the LS, and each system encodes informationabove low-level VS (Table 16, Appendix F).
Analyzing the correspondence between code modelsand brain representations based on the code properties they each encode reveals that the MD systemmaps preferentially to complex code models, encoding more than just token-level information.
Wediscuss these results further in the following section.
Discussion
Through this study, we learn what computer program-related information can be decoded from thebrain, and which brain systems primarily encode that information.
Brain representations and code properties.
We show that the MD system preferentially encodesdynamic analysis -related properties when compared to other brain systems and other properties.
Further, we find the Language system encodes syntax-related properties like control flow and staticanalysis .
These findings are complementary to the results from Liu et al.
[2020] and Ivanova et al.
[2020]–they show how the MD system is recruited in both mentally simulating code andcomprehending code.
They however do not find any consistent response in the Language system toeither code simulation or comprehension.
Our results show that despite not exhibiting significantaverage responses, these systems do encode code-specific properties, improving our understanding ofthese brain systems’ functional organization.
Brain representations and code model representations.
Another key contribution of our work,from Experiment 2, is demonstrating that it is possible to map brain representations to representationslearned by code models.
In particular, we observe encoding of the properties represented by codemodels in the MD and LS, with 4 models more accurately mapped from MD.
This is particularlynoteworthy since these models, which are trained on source code symbols, can be mapped morefaithfully from the representations of a network implicated in problem-solving than one associatedwith composition in languages.
We also considered model complexity as a relevant feature, and note that the MD and LS map toa combination of token projection embeddings almost as well as to complex models like XLNet .
One plausible explanation for this surprising result is that the program stimuli are simple enough toallow the different properties evaluated in our work ( control flow , data type , etc .) to be discernedfrom token level information alone (as validated in Experiment 1), which is likely why the Token
Projection model also predicts these properties very well (Table 4, Appendix C.2).
Taken together,these data suggest that the information being decoded from brain activations in these two regionsis driven at least by the information conveyed by tokens in the programs.
This finding is notablyconsistent with work in the field of natural language which show swaths of cortex are predictedprimarily by the token-level properties of sentence stimuli [Toneva et al., 2022].
While we see that information encoded in both the MD and the Language systems are driven bytoken-level information, a clearer trend emerges when evaluating brain-code model mapping in thecontext of specific code properties. We find that the MD shows decoding performance consistent withcomplex model architectures, suggesting that it may encode more than just token-related information,as complex code models capture contextual information as well. This is a new result, which alongwith Experiment 1, suggest distinct roles for the LS (purely token-level) and MD (more complexinteractions) with respect to computer code comprehension and execution.
Way forward.
Our findings have the potential to improve our understanding of the organization ofthe human brain, which can in turn lead to the design of better code models.
In computer vision,results by Tschopp et al.
[2018], Schrimpf et al.
[2020] show how deep network architectures thatmimic the visual system exhibit superior image classification rates on image recognition tasks.
Ourfindings prompt yet another reconsideration of the current design of ML models of code.
Extant codemodel architectures do not explicitly model the Multiple Demand system in any way–they only modelsyntactic information and infer dependency information from program ASTs.
Taking inspirationfrom the role of the Multiple Demand system we identified in this work, modeling dynamic runtimeinformation as well as static code structure should be explored.
We also provide initial results supporting the ability to decode specific and fundamental code-relatedprimitives like control flow information.
This sets us up to study other complex human cognitiveprocesses which involve such primitives, but which are not naturally described in terms of code, likegeneral problem solving, employing hierarchical and complex decision making strategies, etc .
Such astudy also promises to support the recent proposal of a child as a hacker Rule et al.
Limitations.
The average program in any software project exhibits non-trivial control and data dependencies, object manipulation, function calls, types, and state changes.
However, the programmingtasks in Ivanova et al.
[2020] are short snippets of procedural code with limited program properties.
Responses to longer programs, with more complex properties and across multiple languages, shouldbe studied on a larger number of participants in the future, in order to build on the trends provided byour work.
Further, while there are multiple equally important aspects to programming like designingsolutions, selecting appropriate data structures, and writing programs, here we have chosen to studyonly a single specific activity—code comprehension.
Future work should explore these other aspects.
Our results do not allow us to infer whether the MD and LS are driven by the same underlyingfeatures of code that are used to discriminate between code properties and code models, so futurework might consider an encoding analysis.
Finally, as with all neural decoding analyses, extractinginformation from the mental states of participants should be done with caution to ensure it is not usedfor any exploitative purpose.
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’scontributions and scope?
[Yes] Main claims are accurately reflected.
(b) Did you describe the limitations of your work?
[Yes] Limitations are noted in Section 6:Limitations.(c)
Did you discuss any potential negative societal impacts of your work?
[Yes] Societalimplications are noted in Section 6: Limitations.(d)
Have you read the ethics review guidelines and ensured that your paper conforms to them?[Yes]2.
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[N/A]
There are notheoretical proofs presented in this work.(b)
Did you include complete proofs of all theoretical results?
[N/A]
There are no theoreticalproofs presented in this work.3.
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experimentalresults (either in the supplemental material or as a URL)?
[Yes] All code and data is availableat the public link https://github.com/anonmyous-author/anonymous-code , mentioned on the first page of this draft.
A single command is provided to wrap the entire pipelinefrom environment setup to final plots and tables.(b)
Did you specify all the training details (e.g., data splits, hyperparameters, how they werechosen)?
[Yes] See Appendix B: MVPA cross-validation and hyper-parameters for all trainingdetails.(c)
Did you report error bars (e.g., with respect to the random seed after running experimentsmultiple times)?
[Yes] See Figure 2 for error bars.
(d) Did you include the total amount of compute and the type of resources used (e.g., type ofGPUs, internal cluster, or cloud provider)?
[Yes] See Section 4.3: Model representations anddecoding.
The experiments we present in this work are not compute-heavy and can be run on a single 8-core laptop in under an hour.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes] We use a public dataset byIvanova et al.
[2020] and cite the authors.(b)
Did you mention the license of the assets?
[Yes] We mention the license in 4.1: Dataset.(c) Did you include any new assets either in the supplemental material or as a URL?
[N/A] Nonew assets were included, except results derived from existing data.
All original links topublic data and code operating over them are made available and appropriately licensed to do so.
(d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[Yes] We mention participant consent in Section 4.1: Dataset, and link the original data source for more information.
(e) Did you discuss whether the data you are using/curating contains personally identifiableinformation or offensive content?
[Yes] We mention participant unidentifiability in Section4.1: Dataset, and link the original data source for more information.
No offensive content isincluded.5.
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if applicable?
[N/A]
We did not conduct research with human subjects, but this information is availablein Ivanova et al.
[2020](b)
Did you describe any potential participant risks, with links to Institutional Review Board(IRB) approvals, if applicable?
[N/A]
We did not conduct research with human subjects, butthis information is available in Ivanova et al.
[2020](c)
Did you include the estimated hourly wage paid to participants and the amount
A Brain regions
Regions of Interest (ROIs).
We investigate the representations of code in well-studied systems ofbrain regions–the MD system, Language system, Visual system, and the Auditory system (detailsbelow).
A region (also referred to as parcel ) here denotes a contiguous chunk of brain mass involvedin a cognitive task.
A system of regions (also referred to as a network ) can comprise multipledisjoint regions that exhibit shared activity patterns across a range of tasks.
For many cognitivetasks, a region marks only the approximate location of voxel populations involved in a cognitive task.
Functional ROIs (fROIs) are voxels within these broad regions that respond most strongly to a giventask (language, working memory, etc .).
The use of fROIs enables accounting for the exact anatomicallocations of these task-sensitive voxels, which vary across individuals.
The regions depicted are used as a starting point to functionally localize these systems in individual participants.
Multiple Demand (MD) system.
Located in the prefrontal and parietal areas of the brain, this system ofregions is active in a host of tasks requiring workingmemory and general problem solving skills, including math and logic [Duncan, 2010, Fedorenko et al. ,2013, Amalric and Dehaene, 2019].
Language system.
These regions have been identified to respond to both comprehension and productionof language across modalities (written, speech, signlanguage), respond to typologically diverse languages( 45 languages, from across 11 language families),form a functionally integrated system, reliably androbustly track linguistic stimuli, and have been shownto be causally important for language [Clark and Cummings, 2003, Fedorenko et al., 2010, Hu et al. ,2021, Blank and Fedorenko, 2017, Newman et al., 2015, Ayyash et al., 2021].
Visual system.
These regions in the occipital lobe of the brain respond to visual input, rangingfrom low-level features like lines and edges, through intermediate categories like shapes, letters, andnumbers, through higher order structures like faces and scenes.
[Hubel and Wiesel, 1959, Polk et al.
Epstein and Kanwisher, 1998, Kanwisher et al., 1997].
Auditory system.
The auditory system is located in the superior temporal region of the brain.
This region uniquely encodes pitch, speech, and music, but is not involved in high-level languagecomprehension and production [Norman-Haignere et al., 2015, 2019].
In our experiments pertainingto programming language comprehension, we use the activity seen in the auditory system as anegative control.
B Method - Details
Selecting brain representations.
For each trial in the fMRI experiment, stimulus responses in eachvoxel were extracted from the parameters of a General Linear Model (GLM) fit to the time-varying
BOLD signal in which each experimental condition was modeled with a boxcar function convolvedwith the canonical hemodynamic response function (HRF).
For the localizer experiments, conditionswere modeled as the entire block.
For the Python program comprehension experiment, individualprograms were modeled using the period from the onset of the code/sentence problem until thebutton press.
The predictors for the GLM included trial ID (equivalent to problem ID), run number,and motion regressors.
The voxels were then filtered using gray-matter masking and (for MD andthe Language systems) network localization.
Although fMRI measurements return whole brainresponses, only a thin layer of cortex dubbed gray matter contains BOLD signal of interest to theseanalyses.
Gray matter voxels were selected using a Bayesian segmentation of the anatomical brainimage into standard tissue types, and then returning the set of indices where the posterior gray matterprobability exceeds 0.
[Ashburner et al., 1999].
Next, these sets of voxels were filtered separatelyfor each of the brain regions outlined in Appendix A. For the visual and auditory networks, primarysensory areas were identified using an anatomical atlas [Rolls et al., 2020].
For the MD and the
Language systems, voxels were functionally localized as those containing the top 10% of responsesto their respective functional localizer tasks, as described in Ivanova et al.
See Fedorenkoet al.
[2010] for a discussion of the functional localization approach as it pertains to the languagenetwork.
GLM modeling and gray matter segmentation was performed using SPM12; functionallocalization was performed using the toolbox released by Ivanova et al.
Once voxel responseswithin each brain region were extracted for each trial of each run, some additional preprocessing wasrequired before finalizing the brain representations, and passing them to downstream models.
Dueto differences in MRI sensitivity across runs, each of the 12 runs of 6 programs for a given subjectappeared with nonuniform mean and variance.
In order to normalize these signals, so as to leveragedata across a participant’s entire scanning session, brain representations were z-transformed withineach run to achieve a common scale.
In order to avoid data-leakage from this preprocessing step, alldownstream analyses were cross-validated via a leave-one-run-out approach, such that no intra-runrescaling could be used for prediction.
MVPA cross-validation and hyperparameters.
For each brain system and each code propertyor code model, we run a separate MVPA analysis.
For each subject, we train a separate ridgeregression model on each cross-validated leave-one-run-out fold, and then predict the remainingtargets using the brain representations from the left-out run.
We use L 2 regularization to control formodel complexity, and employ a leave-one-out cross validation scheme to select λ.
We score resultingpredictions using a series of different metrics to maximize interpretability.
For the categorical codeproperties, we report classification accuracy of model estimates.
For the continuously scaled codeproperties, we report Pearson correlation between the model estimates and true targets.
For themodel representations, we report rank accuracy between model representation predictions and thetrue model representations of left-out samples.
We calculate rank accuracy as the percentile ofa predicted and true target pairing within the full set of possible pairings sorted by a Euclideandistance metric.
Rank accuracy was chosen for this experiment as it returns a more interpretable 50%baseline through which to evaluate the brain to model mappings.
Following metric calculation oneach cross-validation fold, we take the mean of those estimates to report an out-of-sample score overthe entire dataset for each subject.
We then take the mean of those scores across subjects to derivean overall performance measure.
As a baseline, we repeat the above process in its entirety 1000times for each MVPA analysis, to provide a null permutation distribution against which to comparescores.
These null distributions are fit with a univariate Gaussian and used to calculate z -statisticsfor MVPA scores.
For each statistical comparison, a single-tail one-sample z -test is performedbetween the MVPA score and the distribution fit to the null permutations.
Each of these p -valuesis then FDR-corrected for the set of comparisons in a given experiment, e.g., 4 brain networks∗ 9 code models =
36 comparisons for experiment 2 (Figure 2).
Following FDR-correction, thesignificance threshold for the resulting p -value is set at alpha = 0 .
001 , so decoding is deemedsignificant at p < 0 .
The code to run our MVPA analysis, and all resulting statistics, is availablein https://github.com/anonmyous-author/anonymous-code .
Code properties.
The code properties we consider are the following
• Token count.
The total number of tokens that appear in the program.
This is a sanity check tomeasure brain activity against, since it is natural to expect the activity in the brain to be correlated
to the length of a program, and in general, the amount of content being comprehended.
By theirdesign, the dataset had program lengths with a small standard deviation.• Node count.
The total number of AST nodes that appear in a program.
Similar to the number oftokens, we verify if brain activity correlates to a proxy for the amount of syntactic content in it.• Runtime steps.
We execute the programs and measure the number of instructions the programsteps through.
While two programs can have the same program length, the number of instructionsexecuted can differ ( e
g. for 1:10 vs. for 1:50 ).
We measure if any brain regions capture thenumber of mental operations needed to compute the output of a program.• Bytecode ops.
We execute the programs and count the number of bytecode operations performed.
[Figure]
This metric should mimic the number of mental operations performed needed to compute theoutput of a program.• Cyclomatic complexity.
This metric [McCabe, 1976] is used to measure the general complexityof software systems.
It is defined as a function of the number of nodes, edges, and the connectedcomponents in a program’s control flow graph.
While its efficacy as a metric to measure softwarecomplexity has been contested [Shepperd, 1988], we include it to see if brain activity is correlated [Figure]
to any explicit syntactic constructs of a program.
[Figure]
• Halstead difficulty .
This metric [Halstead, 1977] was defined to measure how difficult any pieceof software would be for a programmer to comprehend or write.
It is defined as a function of thenumber of tokens, operations, vocabulary that appears in a program.
The inter-correlations between these metrics have been tabulated in Table 19, Appendix H.
Code models - Model details.
We evaluate the following models in this work–
• bag-of-words , TF-IDF .
These are count-based language models.
They predict the likelihood of atoken appearing in a program based on vocabulary statistics and frequencies.
• seq2seq
[Sutskever et al., 2014], CodeTransformer [Zügner et al., 2021], CodeBERT [Fenget al., 2020], CodeBERTa [HuggingFace, 2020].
We evaluate these three autoencoder (AE) modelswith increasing model complexity.
AE based pretraining reconstructs the original program fromcorrupted input.•
XLNet
[Yang et al., 2019], CodeGPT [Microsoft, 2021].
We evaluate auto-regressive (AR) modelswith a model complexity similar to that of the other transformers.
AR language modeling estimatesthe probability distribution of a text corpus in one direction–either forward or backward, while AEmodels capture dependencies in both directions.
We use AR models to mimic a top-down, singlepass comprehension style by humans.•
Token Projection .
We compare the results of the above models against an aggressive baselineprovided by using unique Gaussian-distributed random vectors for the token embeddings in avocabulary, and returning the sum of these token embeddings across a program.
The resultantembedding is not transformed by any model or any weights–it instead serves as a proxy for thetokens that appear in the program.
This sets a higher bar than the null-distribution labeling baseline(Section 5.2).Code models - Configuration details.
In setting up our code model bench, we aimed to select acollection of models ranging in their complexity, namely bag-of-words , TF-IDF (Term frequencyInverse document frequency), seq2seq , XLNet , CodeTransformer , CodeGPT , CodeBERT , andCodeBERTa , as well as a Token Projection model.•
The bag-of-words , TF-IDF , seq2seq , and Token Projection models all use the same custom tokenizer defined by the current authors, whereas XLNet , CodeTransformer , CodeGPT , CodeBERT ,and CodeBERTa use tokenizers defined by the original training authors [Zügner et al., 2021,Microsoft,
• The tokenizer set up for the models trained by the current authors establishes a unique token in thevocabulary for each Python keyword, each Python builtin function, one token for all numeric types,one token for all string types, and N tokens for each of the N variables in a given program.•
In the case of the Token Projection baseline, this tokenizer was used to map each token in a pieceof source code to an index in the vocabulary, which in turn was used to index a Gaussian distributedrandom matrix ( D = 128 ).
The sum of these random token projections was calculated to return aunique embedding for each program.• For bag-of-words and TF-IDF , the validation split of the code-search-net Python dataset [Husainet al., 2019] was used to enumerate vocabulary and token occurrence statistics.
These data werethen used to transform each program to a vector where each dimension represented the raw( bag-of-words ) or document-weighted ( TF-IDF ) count of the unique items in the vocabulary.•
If a given set of programs viewed by a subject never included a specific token, leading that columnto equal the zero vector, then those dimensions would be filtered out.•
We used the dataset released by Project Codenet [Puri et al., 2021] to pretrain a seq2seq model[Sutskever et al., 2014].
The dataset contains Python programs that are solutions to olympiad-styleprogramming problems in data structures and algorithms.
We trained a seq2seq model with aGRU unit for 15 epochs, with a dropout probability of 0 . 2 , dot product attention, and a maximumsentence length of 500 .•
For XLNet , CodeTransformer , CodeGPT , CodeBERT , and CodeBERTa , each program was passedthrough that model’s tokenizer and model pipeline as defined by the original authors.
The representations of each program were extracted from the pretrained encoders.
XLNet , CodeTransformer ,and CodeGPT were originally trained on the Python subset
C Additional ResultsC.1 Experiment 1
Using representations from localized brain regions, we attempt to decode static and dynamic propertiesof comprehended code, and learn maps to code model representations of that same code.
We reportdecoding performance of each brain region to the original Ivanova et al.
[2020] code properties in
Table 1, the complexity-related code properties in Table 2, and the code model mappings in Table 3.
We find that the MD system and the Language system decode all code properties and code modelsexcept variable language significantly above baseline, as established through a null permutationtest.
The Visual system decodes 4 of the 6 code properties and 8 of the 9 code models, whereas the
Auditory system only decodes the code vs sentences property.
[Table] [Table]
[Table]
We additionally evaluate whether code models contain linearly decodable information about the codeproperties we explore in this work (Section 4.2).
The models decode with near perfect accuracy onmost classification tasks (avg.
accuracy = 0 .
97 ± 0 . 03 ), and with high correlations on the regressiontasks (avg.
r = 0 .
89 ± 0 . 05 ).
This is expected for the classification tasks, since the dataset containstokens which help with perfectly separable decision boundaries.
For example, for the control flowcode property, the dataset contains programs with if , for , or neither, but never both.
These resultssuggest that the code models we evaluate faithfully encode the set of properties we evaluate them on.
[Table]
D Analysis of Variance
We run a series of one-way ANOVA statistical tests to assess whether decoding performance variesacross brain regions for each code property and code model, and whether decoding performancevaries across code models for each brain region.
We report these results in Tables 5,
[Table]
and 7.
We findthat decoding performance varies across brain region for all code properties and code models, exceptthe variable language property and the Token Projection model, and decoding performance variesacross code models for the MD system and the Visual system.
[Table]
[Table] [Table]
E Pairwise Analysis
To extend the findings from the ANOVA analyses towards specific pairwise comparisons betweenbrain regions and code models, we compare scores between brain regions for each code propertyor code model, and compare scores between code models and a subset of code properties for eachbrain region, using paired two-tailed t -tests.
We report all pairwise brain region comparisons in
Tables 8 and 10, and report all pairwise code property and model comparisons in Tables 9 and 11.
To highlight a few key results from code properties, we find that the MD system decodes dynamicanalysis significantly better than the LS (Table 8).
Moving onto code models, we find that the MDsystem decodes CodeBERTa , CodeTransformer , seq2seq and TF-IDF significantly more accuratelythan the Language system (Table 10).
Finally, an investigation into model complexity reveals that
CodeBERTa , CodeTransformer , and bag-of-words are significantly more accurately decoded fromthe MD system than the Token Projection model.
[Table]
Is Significant
[Table]
[Table]
[Table]
Brain
[Table]
Brain Region
[Table]
t p p (corrected)
Is Significant?
[Table]
Bag Of Words
Language
Auditory 2.
9.22e
[Table]
2.36e-
[Table]
Of Words
MD
Auditory
[Table] [Table]
54e
[Table] [Table]
73e-
[Table]
Of Words
MD
Visual
[Table] [Table]
98e4.28eCodeBERT
Language
Auditory
[Table]
1.36e
[Table] [Table]
77eCodeBERT
MD
Auditory
[Table] [Table]
97e-
[Table]
14eCodeBERT
Visual
Auditory
[Table] [Table]
09e-
[Table]
95eCodeBERTa
[Table] [Table]
41e
[Table] [Table]
CodeBERTa
MD
Auditory
[Table] [Table]
26e
[Table] [Table]
80eCodeBERTa
MD
Language
[Table]
1.30e-
[Table]
05eCodeBERTa
Visual
[Table] [Table]
30e-
[Table]
77eCodeGPT
Language
Auditory
[Table] [Table]
41e
[Table] [Table]
00eCodeGPT
MD
Auditory
[Table] [Table]
73e-
[Table]
84eCodeGPT
Visual
[Table] [Table]
[Table]
10e4.14eCodeTransformer
Language
Auditory
[Table] [Table]
37e-
[Table]
73eCodeTransformer
MD
Auditory
[Table]
2.17e3.91eCodeTransformer
MD
Language
[Table] [Table]
29e-
[Table]
59eCodeTransformer
MD
Visual
[Table] [Table]
05e1.44e-
[Table]
[Table]
[Table]
[Table]
69e
[Table]
18e
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
22e
[Table]
[Table]
[Table]
58e
[Table]
[Table]
[Table]
51e
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
54e
[Table]
[Table]
[Table]
06e
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
77e
[Table]
[Table]
64e
[Table]
[Table]
[Table]
39e
[Table]
[Table]
[Table]
[Table]
[Table]
47e
[Table]
[Table]
83e
[Table]
[Table]
[Table]
11e
[Table]
[Table]
46e
[Table]
[Table]
CodeTransformer
[Table]
[Table]
77e
[Table]
[Table]
54e
[Table]
[Table]
[Table]
[Table]
21e
[Table]
[Table] 08e [Table] [Table]
IDF
[Table]
[Table]
60e
[Table]
[Table]
15e
[Table]
[Table]
[Table]
14e
[Table]
[Table]
73e
[Table]
[Table]
[Table]
-IDF 2.
6.49e5.19e0 MD
CodeBERTa
XLNet 2.
2.65e1.06e0 MD
CodeGPT
Bag Of Words -0.
5.81e8.20e0 MD
CodeGPT
Seq2Seq 0.
6.81e8.83e0 MD
CodeGPT
TF-IDF 0.
7.77e9.09e0 MD
CodeGPT
XLNet 0.
8.37e9.44e0 MD
CodeTransformer
Bag Of Words 1.
3.14e6.08e0 MD
CodeTransformer
CodeBERT 1.
1.65e3.91e0 MD
CodeTransformer
CodeGPT 2.
3.90e1.22e0 MD
CodeTransformer
Seq2Seq 1.
7.45e1.99e0 MD
CodeTransformer
TF-IDF 2.
{+2.73e1.06e
+}
MD
CodeTransformer
XLNet 2.
2.48e1.06e0 MD
Seq2Seq
Bag Of Words -0.
3.74e6.54e0 MD
Seq2Seq
TF-IDF -0.
8.29e9.44e0 MD
TF-IDF
Bag Of Words -1.
2.43e5.06e0 MD
Token Projection
CodeBERT -1.
2.14e4.60e0 MD
Token Projection
CodeGPT -1.
1.81e4.11e-
[Table] [Table] F Multi-system partial regression analysis: Scores and significance In order to investigate whether each of the brain systems included contribute unique information towards the 900 decoding tasks, we combine brain representations from different systems in paired combinations and evaluateeffects on downstream decoding performance across all experiments.
We find that the addition of MD or LS to 902 VS, relative to VS alone, improves downstream decoding of code vs sentences .
The same effect is observedfor the combination of MD and LS compared to either alone.
These data suggest that the MD system and the 904 Language system encode unique variance relevant to the decoding of code vs sentences , and this is above andbeyond the information encoded in the Visual system or each other individually.
Additionally, we observe that 906 the addition of MD to VS, relative to VS alone, improves downstream decoding of control flow , data type ,dynamic analysis , and static analysis .
This same effect is observed for the addition of LS to Visual system for 908 control flow and static analysis .
These data suggest that the MD system and the Language system encode uniqueinformation above and beyond information encoded in the Visual system for these decoding tasks (Table 15).
910 We repeat this process for Experiment 2, where we find that the addition of MD to VS improves decoding for 7models, the addition of MD to LS improves decoding for 7 models, and the addition of LS to Visual system 912 improves decoding for all 9 models (Table 16).
[Table] [Table]
[Table] [Table] [Table]
[Table] [Table] [Table]
[Table] [Table] [Table]
[Table] [Table]
[Table] [Table]
G Ranked correlations of brain and model decoding performance
[Table]
We investigate whether there exists any trends in the correspondence between how well each brain region and 915 each code model performs on each decoding task. For each brain region and code model, we extract the z -score of how well each hand-selected property is decoded. We then compare the performances of each brain region to each code model via a Spearman rank correlation. Under such a framework, if two representations have similar content, and perform similarly in how well or how 919 poorly they decode a battery of properties, they will show strong positive rank correlations.
On the other hand,if two representations show different patterns in how well they perform on the same tasks, they will be less 921 strongly correlated. Here, we find that the MD system shows perfect task performance rank correlation with 3 transformer ar- 923 chitectures: CodeBERTa , CodeBERT , and CodeTransformer , as well as TF-IDF (which contains broaderdistributional information). On the other hand, the Language system is consistent with bag-of-words and Token 925 Projection , which only reflect the presence of specific tokens. These data, in conjunction with the core analyses summarized
H All code properties: Scores and Correlations
We analyzed a series of code properties as part of the static and dynamic code analysis.
As several of these 931 properties ( token count , node count , Halstead difficulty , cyclomatic complexity , and bytecode ops ) wererevealed to be highly correlated in a post-hoc analysis, we report only one measure for this subset in Experiment 933 1, token count , but include all scores here for completeness (Table 18).
We also report the correlation matrixbetween all code properties that led us to select token count as the representative property for this subset (Table 935 19).
[Table]
[Table]
I Sensitivity of brain mapping to model output dimensions
Of potential interest to the decoding framework is not just the complexity of code models, but their intrinsic 938 dimensionality as well.
In order to investigate to what extent brain model mappings are robust to changes incode model dimensionality, and to assess which model representations are most sensitive to compression and 940 expansion, we rerun our current decoding framework while controlling for embedding size.
For each brain network to code model mapping task, prior to the MVPA analysis, we control for the dimen- 942 sionality of the code model embedding via projection through a Gaussian random matrix R d 1 × d 2 drawn from N (0 , 1 /d 2 ) where d 1 is the original code model embedding dimensionality and d 2 is the desired dimensionality.
We observe that, on average, models of lower complexity (e.g., bag-of-words , TF-IDF , seq2seq ) appear relatively 945 robust to compression, whereas the most complex models (e.g., CodeTransformer , CodeGPT , CodeBERTa )gain ample performance from higher dimensional expression, and suffer considerably when constrained.
These data suggest that higher dimensions of the encoder in complex models encode relevant neural informa- 948 tion.
Additionally, these effects appear to be most pronounced when decoding from the brain region whoserepresentations yield the strongest mappings, the MD system.
We note here however that we could be observing an interaction effect between model complexity and dimen- 951 sionality output in these results.
Since we cannot fully control for the complexity of these models (by makingthem all ‘equally complex’), this experiment alone cannot drive definitive conclusions.
These results instead constitute a preliminary exploration into the effects of code model dimensionality on brain 954 to model representation mappings, and suggest an avenue for future investigation.
J Robustness of Results to Regression Metric
For the decoding analysis of the continuous valued dynamic analysis and static analysis properties, it is reasonable 957 to ask why the Pearson correlation metric was chosen as opposed to RMSE , as is typically customary forregression tasks.
While we present the results using the Pearson correlation metric in the core results for 959 interpretability via the zero-baselne, here we confirm that the use of an RMSE metric leads to the sameconclusions.
As we see here, the MD, LS, and Visual system decode the dynamic analysis property, and MD 961 and LS decode the static analysis property with significantly lower RMSE than the null permutation baseline.
These results precisely confirm and mirror the patterns observed in Figure 2.
[Table]