Convergent Representations of Computer Programs in Human and Artificial Neural Networks
Anonymous Author(s)
AffiliationAddress email
Abstract
What aspects of computer programs are represented by the human brain duringcomprehension?
We leverage brain recordings derived from functional magneticresonance imaging (fMRI) studies of programmers comprehending Python code toevaluate the properties and code-related information encoded in the neural signal.
We first evaluate a selection of static and dynamic code properties, such as abstractsyntax tree (AST)-related and runtime-related metrics.
Then, to learn whetherbrain representations encode fine-grained information about computer programs,we train a probe to align brain recordings with representations learned by a suite of
ML models.
We find that both the Multiple Demand and Language systems–brainsystems which are responsible for very different cognitive tasks, encode specificcode properties and uniquely align with machine learned representations of code.
These findings suggest at least two distinct neural mechanisms mediating computerprogram comprehension and evaluation, prompting the design of code modelobjectives that go beyond static language modeling.
We make all the correspondingcode, data, and analysis publicly available.
Introduction
Computer code comprehension is a complex task which recruits multiple cognitive processes—fromsyntactic parsing to mentally simulating programs.
Despite the prevalence of this task, the representations of code processed in the human brain during code comprehension remain uninvestigated.
Isit possible that common code properties and program semantics are faithfully represented in brainactivity patterns when code is read and evaluated?
A few prior works have used data derived fromfunctional magnetic resonance imaging (fMRI) and electroencephalography (EEG) to locate physicalregions in the brain involved in code-related activities like code comprehension and debugging[Siegmund et al., 2017, Floyd et al., 2017, Peitek et al., 2018, Castelhano et al., 2019, Ivanova et al.
,2020, Liu et al., 2020, Ikutani et al., 2021, Peitek et al., 2021], code writing [Krueger et al., 2020,
Karas et al., 2021] and data structure manipulation [Huang et al., 2019].
These studies have helpeddetermine whether code-related activities join other activities supported by those brain regions, suchas working memory (processed by a set of brain regions known as the Multiple Demand system)or language processing (processed by the Language system–another set of brain regions).
Whilethese results improve our understanding of the brain regions involved in code comprehension, it stillremains unclear what specific code-related information these regions encode.
For example, does theresponse in a region seen during code comprehension encode specific syntactic or semantic codeproperties?
Do responses from multiple brain regions correspond to the same set of properties?
Or,are different code properties encoded in different regions?
A possible approach and its limitation.
One way to learn what information is encoded in the brainis to decode a code property of interest from recordings of brain signals when reading code (throughfMRI or EEG).
Being able to decode the property accurately from a specific region of the brainestablishes that information related to that code property is faithfully represented in that brain region.
A question central to such a decoding analysis is the choice of the target code property–what codeproperties should be investigated?
We can hand-select a set of fundamental properties of code andtest if they can be decoded.
While helpful, such a set will not preclude other, more complex aspectsof code possibly being encoded.
ML models of code as a tool to reverse engineer what is encoded.
To address the limited scopeof hand-selected properties, we look to machine learning (ML) models trained on code.
Dubbedcode models , they are trained on large corpora of code, in an unsupervised manner, to learn MLmodel representations of computer programs.
Code models are increasingly being used in softwareengineering workflows [Allamanis et al., 2018a], and have been shown to perform well on tasks likecode summarization
[Alon et al., 2019], detecting variable misuse [Bichsel et al., 2016], and morerecently, code auto-completion [Chen et al., 2021].
These representations have been shown to encodeand describe complex code properties [Bichsel et al., 2016, Allamanis et al., 2018b, Srikant et al.
Successfully decoding these code model representations from brain activity data then allowsus to probe whether complex code properties are also encoded in the brain.
Besides serving as a tool to reverse engineer what is encoded in different brain regions, anotherdistinct advantage of decoding code model representations is its potential to serve as a tool to reverseengineer our cognitive processes.
It is currently unclear what mechanisms drive code comprehension.
If we find one class of ML models (say, masked language models) to be more predictive than another(say, autoencoders), it is reasonable to suspect that our brains optimize objectives more similarto that of masked-LMs than that of autoencoders when comprehending code.
As a corollary, apoor correspondence between the information encoded by our brains and ML models suggests thepossibility of unexplored neural architectures and objectives which may better model our cognition,which in turn may outperform extant ML models.
Yamins et al.
[2013] first showed how informationencoded in our visual system resembles what convolutional neural networks learn when trained torecognize images.
We attempt to establish a similar correspondence between code models and thehuman brain in comprehending code.
Why code comprehension?
We focus on code comprehension in this work because very littleof this important skill has been analyzed from a cognitive neuroscience perspective while steadyadvances are being made in training ML models to understand code and increase programmerproductivity [Hellendoorn and Sawant, 2021].
Extant ML models for understanding programmingare direct adoptions of the state-of-the-art in language processing research.
However, recent worksin neuroimaging like Liu et al.
[2020] and Ivanova et al.
[2020] suggest that code comprehensiondoes not share the same neural bases as natural language comprehension.
Do code models thenmimic human cognition of programs?
Do language models?
If not, can we consider other modelarchitectures and training objectives which are directly inspired by results from neuroscience?
Ourwork provides early considerations towards these directions.
Further, characterizing the representations of different code properties in the human brain canhelp learn more about the nature of human algorithmic problem solving more generally.
We cancharacterize signatures of brain activity corresponding to fundamental logic operators like iterativereasoning, conditional reasoning, retrieving calculations that were previously computed, etc . seen inindividual participants when solving carefully designed code comprehension tasks.
Such signatureswill then allow us to characterize other tasks which involve similar logic operators, but which are notnaturally described as computer programs e.g. general logic reasoning, problem solving, and moregenerally, our ability to employ abstractions and form concepts [Rule et al., 2020].
Our setup.
We introduce a framework to evaluate the code properties encoded in human brain representations of code by analyzing fMRI recordings of programmers comprehending Python programs.
We present two means of proceeding— one, probing of brain region representations for specific codeproperties, and two, analyzing the mapping of these representations onto various code models withdiffering model complexity.
We utilize the publicly available dataset from Ivanova et al.
[2020] forall our analyses as it offers high quality, granular brain response data on code comprehension stimulicontrolled across multiple code properties.
We train affine models on brain representations to predicthand-selected code properties which express syntactic and semantic behavior of programs.
Similarly, we predict code representations obtained from a suite of code models.
We investigate the effectsthat brain regions, the nature of code properties, and the complexity of the code models have on theaccuracies of these prediction tasks.
Key findings.
We explore two questions: (1) Do brain systems encode specific code properties?
Arethere differences in how well each brain system encodes each code property?
(2) Do brain systemsencode more complex properties of code, derived from the representations of code models?
We findthe Multiple Demand system and the Language system consistently encode both hand-selected codeproperties and data-derived code model representations, using features which cannot be explainedby only low-level visual characteristics of the code.
Within this set, the Multiple Demand systemmost effectively decodes runtime properties of code like the number of steps involved in a code’sexecution, while the Language system decodes syntax-related properties like the number of tokens ina program and the control structures present in it.
Our results improve the cognitive science domain of human program comprehension–by improvingour understanding of the functional organization of the brain systems we study, as well as objectivesemployed to train machine learning models of code–by offering new objectives for the design of codemodels.
We provide an open-source framework to replicate our experiments, and we release our data andanalysis publicly.
Link - https://github.com/anonmyous-author/anonymous-code .
This shouldenable authors from other neuroimaging studies or code model developers to collaborate and analyzedata across these works, which will also help amortize the high costs of carrying out such experiments.
Related Work
Of the prior works that have investigated the neural bases of programming through fMRI and EEGtechniques [Siegmund et al., 2017, Floyd et al., 2017, Peitek et al., 2018, Castelhano et al., 2019,
Huang et al., 2019, Krueger et al., 2020, Ivanova et al., 2020, Liu et al., 2020, Ikutani et al., 2021,
Peitek et al., 2021] and through behavioral studies [Prat et al., 2020, Casalnuovo et al., 2020, Crichtonet al., 2021], the following probe brain recordings for program properties encoded in them.
Floyd et al.
[2017] learn a linear model to successfully classify whether an observed brain activitycorresponds to reading code or reading text.
Ikutani et al.
[2021] study expert programmers and showthat it is possible to classify code into the four problem categories–math, search, sort, and stringfrom the brain activations corresponding to the code.
Similarly, Liu et al.
[2020] classify whether abrain signal corresponds to code implementing an if condition or not.
Peitek et al.
[2021] analyzecorrelations between brain recordings of participants reading code and a set of code complexitymetrics.
In testing for code properties, our work uses a similar methodology (a linear model trained onfMRI data), but we evaluate a larger set of static and dynamic code properties, often reflectingkey programming constructs like control flow.
Further, we perform these tests in the brain regionsidentified by Liu et al.
[2020] and Ivanova et al.
[2020] as being responsive specifically to codecomprehension, offering finer insight into the content of these specific regions’ representations.
Inaddition, we study representations generated by a suite of ML models with varying complexity andcompare those learned representations to brain representations.
Brain representations have also been studied in domains like natural language, vision, and motorcontrol.
Among related works in natural language, a domain that resembles programming languages,
Mitchell et al.
[2008], Pallier et al.
[2011], Brennan and Pylkkänen [2017], Jain and Huth [2018],
Gauthier and Levy [2019], Schwartz et al.
[2019], Wang et al.
[2020], Schrimpf et al.
[2021], Caoet al.
[2021], Caucheteux et al.
[2021], Toneva and Wehbe [2019] have studied brain representationsof words and sentences by relating them to representations produced by language models.
While thebroader tools we use to investigate these representations, like multi-voxel pattern analysis (MVPA),are similar to some of these prior works, our focus is on properties specific to code and not naturallanguage.
Background
We provide a brief background on fMRI signals as a proxy for brain representations and describe thebrain systems that we probe in this work.
Measuring brain activity with fMRI.
Functional magnetic resonance imaging (fMRI) is a brainimaging technique used to measure brain activity in specific brain regions.
When a brain region isactive, blood flows into the region to aid its processing.
An MRI machine measures this changein blood flow, and reports BOLD (blood oxygen level dependent) values sampled at the machine’sfrequency [Glover, 2011, usually 2 seconds].
The smallest unit of brain tissue for which BOLD signalis recorded is called a voxel (an equivalent of a 3D pixel); it comprises several cubic millimetersof brain tissue.
For our analyses, we select subsets of voxels belonging to specific brain systems.
Following common practice, the parameters of a general linear model, fit to time-varying BOLDvalues, are used as a measure of the overall activation in each voxel in response to a given input.
Itis the values of these parameters that, in concordance with common practices in the neurosciencecommunity, can be accessed as brain representations.
The regions depicted are used as a starting point to functionally localize these systems in individual participants
Brain systems.
A system of brain regions can spandifferent areas of the brain but behaves as a holisticunit, showing similar patterns of engagement acrossa given cognitive task.
We probe the following systems in our work: (a) Multiple Demand (MD) system: this system of regions is known to engage incognitively demanding, domain agnostic tasks likeproblem solving, logic, and spatial memory tasks.
Liu et al.
[2020] and Ivanova et al.
[2020] reportedthat this system is active during code comprehension.(b) Language system (LS): this system responds during language production and comprehension acrossmodalities (speech, text) and languages (across 11language families, including American sign language).
c Visual system (VS): these regions respondprimarily to visual inputs
Auditory system: these regions respond primarily to auditory inputs.
We probe the Visual system and Auditory system since they are involved in general perception.
While we do not expect activity in Auditory system, we expect the Visual system to reflect low-levelvisual properties of the code ( e.g. code length and indentation to reflect code-related properties.
See
Appendix A for a detailed description of these brain systems along with relevant references.
Brain and Model Representations
We describe in this section the method we follow to gather representations of code in the brain(Section 4.1), evaluate the different code properties they encode (Section 4.2), and how we comparebrain representations to those generated by ML models (Section 4.3).
4.1 Brain representations and decoding
We provide a summary of how we process activation signals in the brain elicited by code comprehension, to probe whether they encode any specific code properties.
We provide details in Appendix B.
Dataset.
We use the publicly available brain recordings released as part of the study by Ivanova et al.[2020] (MIT license).
It contains brain recordings of 24 participants, each of whom gave consentand is not personally identifiable according to Ivanova et al.
[2020] protocol requirements, reading72 programs from a set of 108 unique Python programs.
The 72 programs were presented in 12blocks of 6 programs each.
These programs were 3-10 lines in length and contained simple Pythonconstructs, such as lists, for loops and if statements.
A whole program was presented at once, and thetask required participants to read the code and mentally compute the expected output, press a buttonwhen done, and select one of four choices presented to them which matched their calculated output.
From dataset to brain representations.
The original dataset contains 3D images of the brain ofeach participant.
Each voxel value in these images is an estimate of the response strength in this voxelwhen a particular code (or sentence) problem is presented.
To determine which brain systems containinformation about particular code properties, we focus our analyses to four systems – MD, Language,
Visual, and Auditory (Section 3).
A vector of voxels’ activation values in each brain system is thentaken to constitute that system’s representation of a computer program and serves as an input to allour analyses.
See Appendix B for details on data processing and voxel selection.
Analyzing brain representations of code.
We probe brain representations from each participantseparately.
We do not average data across participants since the regions which respond to any task(comprehending code in our case) need not align anatomically.
For each of two experiments—decoding different code properties, and mapping to code model representations—we train ridgeregression/classification models which take as input normalized brain representations per participant.
We hence learn 24 different regression models, for each code property or code model (one perparticipant), and then report the mean performance of these models across participants.
This procedureis also referred to as multi-voxel pattern analysis (MVPA)
[Norman et al., 2006].
Linear modelsare conventionally preferred for probes into brain representations since there has been evidencesupporting the idea that other brain areas linearly map information from such brain representations[Kamitani and Tong, 2005, Kriegeskorte, 2011].
We choose a linear model primarily to control forover-fitting in light of the relatively small dataset.
A remark on data scarcity.
For each participant, we train a linear regression/classification modelwith L2-regularization for on unique cross-validated leave-one-run-out folds of the 72 programsthey attempted (or 48 programs when sentences are removed).
On the order of 1000 voxels wereselected from each brain region responding to any given program per participant, thus resulting in afeature set of dimensions 72 × 1000 .
Such dimensions allow for strong statistical tests to support therobustness of the predictions made.
As a baseline for the model predictions, we use the accuracy of anull permutation distribution generated from sampling 1000 random assignments of the labels.
4.2 Code properties
We attempt to decode the following code properties from brain representations.
See Appendix Bfor details.
(a) Code vs. sentences: classify whether an input stimulus is a code or an Englishsentences describing a code (referred to as sentences ).
See Figure B, Appendix B for an example.
) Variable language: classify whether a program contains variable names written English English , node nodes,cyclomatic complexity , and Halstead difficulty
The metrics have been by softwareengineering practitioners to quantify the complexity of code, and to quantify the difficulty a humanwould experience when comprehending code respectively
We predicting other advanced staticanalysis metrics such as tracking abstract interpretation joins, data flow analysis-related metrics,etc . to future work
Dynamic code’s execution like runtime in
Program length as a potential confound.
Since the properties we examine can also potentially bedifferentiated using program length and other low-level code features, it is a potential confound inour experiments.
We measured the inter-correlations of these properties, and their correlation to thenumber of tokens in the program (program length; see Appendix G).
We expectedly found the fourstatic analysis properties to be highly correlated to each other and to bytecode ops .
We hence use onerepresentative metric each from the two categories of properties for the rest of our analysis– tokencount for static analysis , and runtime steps for dynamic analysis .
Importantly, the other propertieswe examine cannot be explained by program length alone, and therefore program length is not aconfound in our experiments.
Mapping to code properties.
The brain representations (Section 4.1) are mapped to each of thecode properties by training a ridge regression (for static analysis and dynamic analysis properties;continuous values) or a classification model each for every participant-property pair.
To evaluatemodel performance, we use classification accuracy when the predicted values are categorical ( e.g.string vs. numeric data types), and the Pearson correlation coefficient when the predicted values arecontinuous ( e.g. number of runtime steps).
We choose Pearson correlation over RMSE, the canonicaldistance metric for continuous values, for its simplicity and interpretability.
See Appendix I for the
RMSE results, which lead to the same conclusions as the correlation results.
When testing for thesignificance of these predictions, we perform false discovery rate (FDR) correction for the number ofbrain systems tested and the number of properties tested.
See Appendix B for a detailed descriptionof model hyper-parameters, cross-validation settings, and significance testing.
4.3 Model representations and decoding.
We evaluate a bench of unsupervised language models, spanning from count-based language modelsto transformer neural networks [Vaswani et al., 2017].
These models were all trained on large ( ∼ 1Mprograms)
Python datasets [Husain et al., 2019, Puri et al., 2021].
We use the output of the trainedencoders (raw logits) in each of the neural network models as representations of the code inputto the model.
We vary the general complexity of these models to test whether that variation ismeaningful in establishing the quality of brain to model fits.
Model complexity here is the number ofa model’s learnable parameters.
We evaluate the following models, ordered by their increasing modelcomplexity: simple frequency-based language models— bag-of-words , TF-IDF ; auto-encoder basedunsupervised models— seq2seq [Sutskever et al., 2014], CodeTransformer [Zügner et al., 2021],
CodeBERT [Feng et al., 2020], CodeBERTa [HuggingFace, 2020]; auto-regressive models withsimilar model complexity— XLNet
[Yang et al., 2019], CodeGPT [Microsoft, 2021].
Baseline: Token projection model.
We compare the results of the above models against an aggressivebaseline (relative to the null-distribution labeling baseline), a token projection model provided byusing unique Gaussian-distributed random vectors for the token embeddings in a vocabulary, andreturning the sum of these token embeddings across a program.
The resultant embedding is nottransformed by any model or any weights–it instead serves as a proxy for the tokens that appear inthe program.
The results of this baseline model should be interpreted as the level of performanceachievable from the presence of tokens alone with no semantic or syntactic information.
Mapping to code model representations.
The brain representations (Section 4.1) are mapped tocode representations by training another set of ridge regression models to learn an affine map, and aranked accuracy metric is used to compare outputs.
Ranked accuracy scores are commonly used ininformation retrieval where several elements in a range are similar to the correct one.
In our case, thetop-ranked prediction by the linear model indicates the closest fit (Euclidean distance) to the codemodel’s representation.
When reporting result significance, we perform false discovery rate (FDR)correction for the number of brain systems and the number of models.
See Appendix B for a detaileddiscussion of the implemented code models and the corresponding metrics.
All experiments in thiswork were run on
Experiments & Results
Our experiments address two research questions:
• Experiment 1.
How well do the different brain systems encode specific code properties?
Do theyencode the same properties?
• Experiment 2.
Do brain systems encode additional code properties represented by computational language models of code? 5.1 Experiment 1 - How well do the different brain systems encode specific code properties?Do they encode the same properties?
We analyze the classification models and regressions trained on brain representations to predicteach of the code properties described in Section 4.2.
The results of our analyses are summarized in
Figure 3.
The classification and regression tests are marked on the x-axis of the left and right subplotsrespectively; the classification accuracy or Pearson correlation for each of the tasks is marked on they-axes.
We plot dynamic and static properties separately from the others because their baselines aredifferent due to a difference in the similarity metric (classification accuracy vs. Pearson correlation).
The baselines for the categorical code properties differ from each other due to variation in the numberof target classes.
Auditory and Visual systems.
The Auditory system and the Visual system serve as negative andbaseline controls for the other systems.
Since our code comprehension task is visual, we donot decode any meaningful information between programs from Auditory system, although we doobserve decoding of code vs sentences , perhaps explained by previous work showing auditory cortexactivation during silent sentence reading [Perrone-Bertolotti et al., 2012].
The Visual system servesas a baseline for low-level visual features of the code (the layout and indentation of the code, thepresence of letters and alphabets in the programs, etc .).
In Analysis 2, we show that the MD and the
Language systems encode more than such visual features.
The MD and the Language systems yieldthe following observations.
Analysis 1 - How accurately are different properties predicted by MD and LS?
The MD and
LS together decode all the properties well above chance barring variable language.
The variablelanguage finding is consistent with Ivanova et al.
[2020], who show a lack of any significant differencein the aggregate neural activity between English and Japanese variables names—variable namesseem to not be encoded any differently in the context of programs we study.
To analyze the otherresults, we use paired two-sample t -tests ( p < 0 .
FDR-corrected) and examine whether for agiven property, any one brain system decodes it significantly more effectively than another (detailsin Table 8, Appendix E).
We find that the MD system decodes the dynamic analysis property betterthan the Language system.
We additionally test if any brain system has a preference for a specificcode property over another.
We find that the MD decodes the dynamic analysis property significantlybetter than the static analysis property.
These findings establish the unique role of the MD systemin encoding code-simulation and execution related information—an important of aspect of codecomprehension.
Analysis 2 - Multi-system partial regression analysis.
The decoding performance of the VS iscomparable to that of the MD and LS (Figure 3).
To assess the possibility that all three systems - MD,
LS, and VS encode the same properties (all potentially related to low level program features), weemploy a multi-system partial regression analysis.
For each brain system, MD and LS ( S i ), we traintwo models–one which decodes from VS, and another which decodes from S i + VS.
If the differencein the prediction accuracies between the two models is significant, we conclude that S i encodes atleast some information which is orthogonal to the information encoded by the Visual system.
For allcore properties, control flow , data type , dynamic analysis , and static analysis , we find the MD toencode information orthogonal to the VS.
For control flow and static analysis , the LS also encodesinformation orthogonal to the VS.
This suggests that low-level code properties are insufficient toexplain the key results from Experiment 1.
Other combinations in the regression model reveal thatthe MD encodes information orthogonal to the LS when predicting code vs. sentences and dynamicanalysis .
Detailed results are tabulated in Table 15, Appendix F.
Key learnings from Experiment 5.1.
All core code properties are decodable from the representationsof high-level brain systems, and this information is beyond that which can be explained from low-levelvisual information alone.
Although no property is exclusively encoded in any one brain system,the MD system significantly encodes dynamic analysis -related properties–more than what the LSencodes, and more than static analysis properties.
Similarly, we find evidence for the LS to alsosignificantly encode static analysis and control flow related properties.
These are new results on thenature of code properties different brain systems seem to process and encode.
To explore propertieswhich may not be specified by the set we investigate in this experiment, in the following section, weleverage code models as hypothesis-free proxy representations for code syntax and semantics, andsee if any one system preferentially encodes a code model.
5.2 Experiment 2 - Do brain systems encode additional code properties encoded by
computational language
of code We train ridge regression models with brain representations of programs from specific brain systemsto predict code model representations of the same programs.
The set of results from this experimentare summarized in Figure 3.
Auditory and Visual systems.
Similar to Experiment 1, these systems perform as expected, withthe Auditory system exhibiting the lowest decoding performance across code models, and the Visualsystem acting as a proxy for low-level information (Table 10, Appendix E).
Analysis 1 - How well do brain representations in MD and LS map to code model representations?
We find that the MD and LS map to all the models in our suite significantly more accuratelythan the null permutation baseline (Figure 3).
Further, we find that the MD ranked accuracy ishigher than LS for CodeBERTa , CodeTransformer , seq2seq and TF-IDF (differences evaluated usingtwo-sample t -tests; p < 0 .
05 ; FDR-corrected.
See Table 10, Appendix E).
Analysis 2 - The effect of model complexity on decoding to code models.
We investigate the impactmodel complexity has on the performance of the mapping between brain and code representations.
Model complexity here is the number of a model’s learnable parameters.
In analysis 1 above, wesaw that the MD system mapped to the representations of CodeBERTa and TF-IDF significantlymore accurately than the Language system.
These models vary substantially in their complexity, thussuggesting the lack of a specific relationship between model complexity and preferential mappingto one of these given brain systems. We also compare each of these code models against the Token
Projection baseline model.
In a set of paired two-tailed t -tests ( p < 0 . 05 ;
FDR-corrected), we findthat the MD maps to CodeBERTa , CodeTransformer , and bag-of-words significantly more accuratelythan to the Token Projection model (Table 11, Appendix E).
Key learnings from Experiment 5.2.
The MD decodes representations from several models ofcode more accurately than LS, providing evidence that some aspects of code are more faithfullyrepresented in the MD.
A follow up partial regression analysis, as in 5.1, reveals that for nearly everysingle model evaluated, MD encodes information orthogonal to the LS, and each system encodesinformation above low-level VS (Table 16, Appendix F).
Curiously, we also note that a non-trivialportion of these mappings can be explained using only token-level information, suggesting that thebrain signals we access largely encode token-level information.
We discuss this in the followingsection.
Discussion
Through this study, we learn what computer program-related information can be decoded from thebrain, and which brain systems primarily encode that information.
We show that the MD system preferentially encodes dynamic analysis -related properties whencompared to other brain systems and other properties.
Further, we find the Language system encodessyntax-related properties like control flow and static analysis .
These findings are complementaryto the results from Liu et al.
[2020] and Ivanova et al.
[2020]–they show how the MD system isrecruited in both mentally simulating code and comprehending code.
They however do not find anyconsistent response in the Language system to either code simulation or comprehension.
Our resultsshow that despite not exhibiting significant average responses, these systems do encode code-specificproperties, improving our understanding of these brain systems’ functional organization.
Another key contribution of our work, from Experiment 2, is demonstrating that it is possible tomap brain representations to representations learned by code models.
In particular, we observe ageneral preferential encoding of the properties represented by code models in the MD over LS.
Thisis particularly noteworthy since these models, which are trained on source code symbols, can bemapped more faithfully from the representations of a network implicated in problem-solving thanone associated with composition in languages.
We also considered model complexity as a relevantfeature, but do not observe any consistent trend across this dimension. In fact, the MD and LS mapto a combination of token projection embeddings almost as well as to complex models like XLNet .
One plausible explanation for this surprising result is that the program stimuli are simple enough toallow the different properties evaluated in our work ( control flow , data type , etc .) to be discernedfrom token level information alone (as validated in Experiment 1), which is likely why the Token
Projection model also predicts these properties very well (Table 4, Appendix C.2).
Taken together,these data suggest that the information being decoded from brain activations in these two regionsis driven at least by the information conveyed by tokens in the programs.
This finding is notablyconsistent with work in the field of natural language which show swaths of cortex are predictedprimarily by the token-level properties of sentence stimuli [Toneva et al., 2022a].
That said, theprecise causal relationship between stimulus properties and predictivity of cortical activations meritsfurther investigation, e.g. , via the approach of Toneva et al. [2022b].
Way forward.
Our findings have the potential to improve our understanding of the organization ofthe human brain, which can in turn lead to the design of better code models.
In computer vision,results by Tschopp et al.
[2018], Schrimpf et al.
[2020] show how deep network architectures thatmimic the visual system exhibit superior image classification rates on image recognition tasks.
Ourfindings prompt yet another reconsideration of the current design of ML models of code.
Extant codemodel architectures do not explicitly model the Multiple Demand system in any way–they only modelsyntactic information and infer dependency information from program ASTs.
Taking inspirationfrom the role of the Multiple Demand system we identified in this work, modeling dynamic runtimeinformation as well as static code structure should be explored.
We also provide initial results supporting the ability to decode specific and fundamental code-relatedprimitives like control flow information.
This sets us up to study other complex human cognitiveprocesses which involve such primitives, but which are not naturally described in terms of code, likegeneral problem solving, employing hierarchical and complex decision making strategies, etc .
Such astudy also promises to support the recent proposal of a child as a hacker Rule et al.
Limitations.
The average program in any software project exhibits non-trivial control and data dependencies, object manipulation, function calls, types, and state changes.
However, the programmingtasks in Ivanova et al.
[2020] are short snippets of procedural code with limited program properties.
Responses to longer programs, with more complex properties and across multiple languages, shouldbe studied on a larger number of participants in the future, in order to build on the trends provided byour work.
Further, while there are multiple equally important aspects to programming like designingsolutions, selecting appropriate data structures, and writing programs, here we have chosen to studyonly a single specific activity—code comprehension.
Future work should explore these other aspects.
Our results do not allow us to infer whether the MD and LS are driven by the same underlyingfeatures of code that are used to discriminate between code properties and code models, so futurework might consider an encoding analysis.
Finally, as with all neural decoding analyses, extractinginformation from the mental states of participants should be done with caution to ensure it is not usedfor any exploitative purpose.
Checklist
For all authors...
[Table] (a) Do the main claims made in the abstract and introduction accurately reflect the paper’scontributions and scope?
[Yes] Main claims are accurately reflected.(b)
Did you describe the limitations of your work?
[Yes] Limitations are noted in Section 6:Limitations.(c)
Did you discuss any potential negative societal impacts of your work?
[Yes] Societalimplications are noted in Section 6: Limitations.(d)
Have you read the ethics review guidelines and ensured that your paper conforms to them?[Yes]2.
If you are including theoretical results...
[Table]
(a) Did you state the full set of assumptions of all theoretical results?
[N/A]
There are notheoretical proofs presented in this work.(b)
Did you include complete proofs of all theoretical results?
[N/A]
There are no theoreticalproofs presented in this work.
[Table]
If you ran experiments...
[Table] (a) Did you include the code, data, and instructions needed to reproduce the main experimentalresults (either in the supplemental material or as a URL)?
[Yes] All code and data is availableat the public link https://github.com/anonmyous-author/anonymous-code , mentioned on the first page of this draft.
A single command is provided to wrap the entire pipelinefrom environment setup to final plots and tables.(b)
Did you specify all the training details (e.g., data splits, hyperparameters, how they werechosen)?
[Yes] See Appendix B: MVPA cross-validation and hyper-parameters for all trainingdetails.(c)
Did you report error bars (e.g., with respect to the random seed after running experimentsmultiple times)?
[Yes] See Figure 3 for error bars.(d)
Did you include the total amount of compute and the type of resources used (e.g., type ofGPUs, internal cluster, or cloud provider)?
[Yes] See Section 4.3: Model representations anddecoding.
The experiments we present in this work are not compute-heavy and can be run ona single 8-core laptop in under an hour.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes] We use a public dataset byIvanova et al.
[2020] and cite the authors.(b)
Did you mention the license of the assets?
[Yes] We mention the license in 4.1: Dataset.(c) Did you include any new assets either in the supplemental material or as a URL?
[N/A] Nonew assets were included, except results derived from existing data.
All original links topublic data and code operating over them are made available and appropriately licensed to doso.
(d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[Yes] We mention participant consent in Section 4.1: Dataset, and link theoriginal data source for more information.
(e) Did you discuss whether the data you are using/curating contains personally identifiableinformation or offensive content?
[Yes] We mention participant unidentifiability in Section4.1: Dataset, and link the original data source for more information.
No offensive content isincluded.5.
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if applicable?
[N/A]
We did not conduct research with human subjects, but this information is availablein Ivanova et al.
[2020](b)
Did you describe any potential participant risks, with links to Institutional Review Board(IRB) approvals, if applicable?
[N/A]
We did not conduct research with human subjects, butthis information is available in Ivanova et al.
[2020](c)
Did you include the estimated hourly wage paid to participants and the
A Brain regions
Regions of Interest (ROIs).
We investigate the representations of code in well-studied systems ofbrain regions–the MD system, Language system, Visual system, and the Auditory system (detailsbelow).
A region (also referred to as parcel ) here denotes a contiguous chunk of brain mass involvedin a cognitive task.
A system of regions (also referred to as a network ) can comprise multipledisjoint regions that exhibit shared activity patterns across a range of tasks.
For many cognitivetasks, a region marks only the approximate location of voxel populations involved in a cognitive task.
Functional ROIs (fROIs) are voxels within these broad regions that respond most strongly to a giventask (language, working memory, etc .).
The use of fROIs enables accounting for the exact anatomicallocations of these task-sensitive voxels, which vary across individuals.
Multiple Demand (MD) system.
Located in the prefrontal and parietal areas of the brain, this systemof regions is active in a host of tasks requiring working memory and general problem solving skills,including math and logic [Duncan, 2010, Fedorenko et al., 2013, Amalric and Dehaene, 2019].
Language system.
These regions have been identified to respond to both comprehension andproduction of language across modalities (written, speech, sign language), respond to typologicallydiverse languages ( 45 languages, from across 11 language families), form a functionally integratedsystem, reliably and robustly track linguistic stimuli, and have been shown to be causally important forlanguage [Clark and Cummings, 2003, Fedorenko et al., 2010, Hu et al., 2021, Blank and Fedorenko,2017, Newman et al., 2015, Ayyash et al., 2021].
Visual system.
These regions in the occipital lobe of the brain respond to visual input, rangingfrom low-level features like lines and edges, through intermediate categories like shapes, letters, andnumbers, through higher order structures like faces and scenes.
[Hubel and Wiesel, 1959, Polk et al.
Epstein and Kanwisher, 1998, Kanwisher et al., 1997].
Auditory system.
The auditory system is located in the superior temporal region of the brain.
This region uniquely encodes pitch, speech, and music, but is not involved in high-level languagecomprehension and production [Norman-Haignere et al., 2015, 2019].
In our experiments pertainingto programming language comprehension, we use the activity seen in the auditory system as anegative control.
B Method - Details
Selecting brain representations.
For each trial in the fMRI experiment, stimulus responses in eachvoxel were extracted from the parameters of a General Linear Model (GLM) fit to the time-varying
BOLD signal in which each experimental condition was modeled with a boxcar function convolvedwith the canonical hemodynamic response function (HRF).
For the localizer experiments, conditionswere modeled as the entire block.
For the Python program comprehension experiment, individualprograms were modeled using the period from the onset of the code/sentence problem until thebutton press.
The predictors for the GLM included trial ID (equivalent to problem ID), run number,and motion regressors.
The voxels were then filtered using gray-matter masking and (for MD andthe Language systems) network localization.
Although fMRI measurements return whole brainresponses, only a thin layer of cortex dubbed gray matter contains BOLD signal of interest to theseanalyses.
Gray matter voxels were selected using a Bayesian segmentation of the anatomical brainimage into standard tissue types, and then returning the set of indices where the posterior gray matterprobability exceeds 0.
[Ashburner et al., 1999].
Next, these sets of voxels were filtered separatelyfor each of the brain regions outlined in Appendix A. For the visual and auditory networks, primarysensory areas were identified using an anatomical atlas [Rolls et al., 2020].
For the MD and the
Language systems, voxels were functionally localized as those containing the top 10% of responsesto their respective functional localizer tasks, as described in Ivanova et al.
See Fedorenkoet al.
[2010] for a discussion of the functional localization approach as it pertains to the languagenetwork.
GLM modeling and gray matter segmentation was performed using SPM12; functionallocalization was performed using the toolbox released by Ivanova et al.
Once voxel responseswithin each brain region were extracted for each trial of each run, some additional preprocessing wasrequired before finalizing the brain representations, and passing them to downstream models.
Dueto differences in MRI sensitivity across runs, each of the 12 runs of 6 programs for a given subjectappeared with nonuniform mean and variance.
In order to normalize these signals, so as to leveragedata across a participant’s entire scanning session, brain representations were z-transformed withineach run to achieve a common scale.
In order to avoid data-leakage from this preprocessing step, alldownstream analyses were cross-validated via a leave-one-run-out approach, such that no intra-runrescaling could be used for prediction.
MVPA cross-validation and hyperparameters.
For each brain system and each code propertyor code model, we run a separate MVPA analysis.
For each subject, we train a separate ridgeregression model on each cross-validated leave-one-run-out fold, and then predict the remainingtargets using the brain representations from the left-out run.
We use L 2 regularization to control formodel complexity, and employ a leave-one-out cross validation scheme to select λ.
We score resultingpredictions using a series of different metrics to maximize interpretability.
For the categorical codeproperties, we report classification accuracy of model estimates.
For the continuously scaled codeproperties, we report Pearson correlation between the model estimates and true targets.
For themodel representations, we report rank accuracy between model representation predictions and thetrue model representations of left-out samples.
We calculate rank accuracy as the percentile ofa predicted and true target pairing within the full set of possible pairings sorted by a Euclideandistance metric.
Rank accuracy was chosen for this experiment as it returns a more interpretable 50%baseline through which to evaluate the brain to model mappings.
Following metric calculation oneach cross-validation fold, we take the mean of those estimates to report an out-of-sample score overthe entire dataset for each subject.
We then take the mean of those scores across subjects to derivean overall performance measure.
As a baseline, we repeat the above process in its entirety 1000times for each MVPA analysis, to provide a null permutation distribution against which to comparescores.
These null distributions are fit with a univariate Gaussian and used to calculate z -statisticsfor MVPA scores.
For each statistical comparison, a single-tail one-sample z -test is performedbetween the MVPA score and the distribution fit to the null permutations.
Each of these p -valuesis then FDR-corrected for the set of comparisons in a given experiment, e.g., 4 brain networks∗ 9 code models =
36 comparisons for experiment 2 (Figure 3).
Following FDR-correction, thesignificance threshold for the resulting p -value is set at alpha = 0 .
001 , so decoding is deemedsignificant at p < 0 .
The code to run our MVPA analysis, and all resulting statistics, is availablein https://github.com/anonmyous-author/anonymous-code .
Code properties.
The code properties we consider are the following
• Token count.
The total number of tokens that appear in the program.
This is a sanity check to measure brain activity against, since it is natural to expect the activity in the brain to be correlated
to the length of a program, and in general, the amount of content being comprehended.
By theirdesign, the dataset had program lengths with a small standard deviation.• Node count.
The total number of AST nodes that appear in a program.
Similar to the number oftokens, we verify if brain activity correlates to a proxy for the amount of syntactic content in it.• Runtime steps.
We execute the programs and measure the number of instructions the programsteps through.
While two programs can have the same program length, the number of instructionsexecuted can differ ( e
g. for 1:10 vs. for 1:50 ).
We measure if any brain regions capture thenumber of mental operations needed to compute the output of a program.• Bytecode ops.
We execute the programs and count the number of bytecode operations performed.
This metric should mimic the number of mental operations performed needed to compute theoutput of a program. • Cyclomatic complexity.
This metric [McCabe, 1976] is used to measure the general complexity of software systems.
It is defined as a function of the number of nodes, edges, and the connected components in a program’s control flow graph.
While its efficacy as a metric to measure softwarecomplexity has been contested [Shepperd, 1988], we include it to see if brain activity is correlated
to any explicit syntactic constructs of a program.
• Halstead difficulty .
This metric [Halstead, 1977] was defined to measure how difficult any piece of software would be for a programmer to comprehend or write.
It is defined as a function of thenumber of tokens, operations, vocabulary that appears in a program.
The inter-correlations between these metrics have been tabulated in Table 18, Appendix G.
Code models - Model details.
We evaluate the following models in this work–
• bag-of-words , TF-IDF .
These are count-based language models.
They predict the likelihood of atoken appearing in a program based on vocabulary statistics and frequencies.
• seq2seq
[Sutskever et al., 2014], CodeTransformer [Zügner et al., 2021], CodeBERT [Fenget al., 2020], CodeBERTa [HuggingFace, 2020].
We evaluate these three autoencoder (AE) modelswith increasing model complexity.
AE based pretraining reconstructs the original program fromcorrupted input.•
XLNet
[Yang et al., 2019], CodeGPT [Microsoft, 2021].
We evaluate auto-regressive (AR) modelswith a model complexity similar to that of the other transformers.
AR language modeling estimatesthe probability distribution of a text corpus in one direction–either forward or backward, while AEmodels capture dependencies in both directions.
We use AR models to mimic a top-down, singlepass comprehension style by humans.•
Token Projection .
We compare the results of the above models against an aggressive baselineprovided by using unique Gaussian-distributed random vectors for the token embeddings in avocabulary, and returning the sum of these token embeddings across a program.
The resultantembedding is not transformed by any model or any weights–it instead serves as a proxy for thetokens that appear in the program.
This sets a higher bar than the null-distribution labeling baseline(Section 5.2).Code models - Configuration details.
In setting up our code model bench, we aimed to select acollection of models ranging in their complexity, namely bag-of-words , TF-IDF (Term frequencyInverse document frequency), seq2seq , XLNet , CodeTransformer , CodeGPT , CodeBERT , andCodeBERTa , as well as a Token Projection model.•
The bag-of-words , TF-IDF , seq2seq , and Token Projection models all use the same custom tokenizer defined by the current authors, whereas XLNet , CodeTransformer , CodeGPT , CodeBERT ,and CodeBERTa use tokenizers defined by the original training authors [Zügner et al., 2021,Microsoft,
• The tokenizer set up for the models trained by the current authors establishes a unique token in thevocabulary for each Python keyword, each Python builtin function, one token for all numeric types,one token for all string types, and N tokens for each of the N variables in a given program.•
In the case of the Token Projection baseline, this tokenizer was used to map each token in a pieceof source code to an index in the vocabulary, which in turn was used to index a Gaussian distributedrandom matrix ( D = 128 ).
The sum of these random token projections was calculated to return aunique embedding for each program.• For bag-of-words and TF-IDF , the validation split of the code-search-net Python dataset [Husainet al., 2019] was used to enumerate vocabulary and token occurrence statistics.
These data werethen used to transform each program to a vector where each dimension represented the raw( bag-of-words ) or document-weighted ( TF-IDF ) count of the unique items in the vocabulary.•
If a given set of programs viewed by a subject never included a specific token, leading that columnto equal the zero vector, then those dimensions would be filtered out.•
We used the dataset released by Project Codenet [Puri et al., 2021] to pretrain a seq2seq model[Sutskever et al., 2014].
The dataset contains Python programs that are solutions to olympiad-styleprogramming problems in data structures and algorithms.
We trained a seq2seq model with aGRU unit for 15 epochs, with a dropout probability of 0 . 2 , dot product attention, and a maximumsentence length of 500 .•
For XLNet , CodeTransformer , CodeGPT , CodeBERT , and CodeBERTa , each program was passedthrough that model’s tokenizer and model pipeline as defined by the original authors.
The representations of each program were extracted from the pretrained encoders.
XLNet , CodeTransformer ,and CodeGPT were originally trained on the Python
C Additional ResultsC.1 Experiment 1
Using representations from localized brain regions, we attempt to decode static and dynamic propertiesof comprehended code, and learn maps to code model representations of that same code.
We reportdecoding performance of each brain region to the original Ivanova et al.
[2020] code properties in
Table 1, the complexity-related code properties in Table 2, and the code model mappings in Table 3.
We find that the MD system and the Language system decode all code properties and code modelsexcept variable language significantly above baseline, as established through a null permutationtest.
The Visual system decodes 4 of the 6 code properties and 8 of the 9 code models, whereas the
Auditory system only decodes the code vs sentences property.
[Table] [Table]
[Table]
We additionally evaluate whether code models contain linearly decodable information about the codeproperties we explore in this work (Section 4.2).
The models decode with near perfect accuracy onmost classification tasks (avg.
accuracy = 0 .
97 ± 0 . 03 ), and with high correlations on the regressiontasks (avg.
r = 0 .
89 ± 0 . 05 ).
This is expected for the classification tasks, since the dataset containstokens which help with perfectly separable decision boundaries.
For example, for the control flowcode property, the dataset contains programs with if , for , or neither, but never both.
These resultssuggest that the code models we evaluate faithfully encode the set of properties we evaluate them on.
[Table]
D Analysis of Variance
We run a series of one-way ANOVA statistical tests to assess whether decoding performance variesacross brain regions for each code property and code model, and whether decoding performancevaries across code models for each brain region.
We report these results in Tables 5,
[Table]
and 7.
We findthat decoding performance varies across brain region for all code properties and code models, exceptthe variable language property and the Token Projection model, and decoding performance variesacross code models for the MD system and the Visual system.
[Table]
[Table] [Table]
E Pairwise Analysis
To extend the findings from the ANOVA analyses towards specific pairwise comparisons betweenbrain regions and code models, we compare scores between brain regions for each code propertyor code model, and compare scores between code models and a subset of code properties for eachbrain region, using paired two-tailed t -tests.
We report all pairwise brain region comparisons in
Tables 8 and 10, and report all pairwise code property and model comparisons in Tables 9 and 11.
To highlight a few key results from code properties, we find that the MD system decodes dynamicanalysis significantly better than the LS (Table 8).
Moving onto code models, we find that the MDsystem decodes CodeBERTa , CodeTransformer , seq2seq and TF-IDF significantly more accuratelythan the Language system (Table 10).
Finally, an investigation into model complexity reveals that
CodeBERTa , CodeTransformer , and bag-of-words are significantly more accurately decoded fromthe MD system than the Token Projection model.
[Table]
Is Significant
[Table]
[Table]
Brain
[Table]
Brain Region
[Table]
t p p (corrected)
Is Significant?
[Table]
Bag Of Words
Language
Auditory 2.
9.22e
[Table]
2.36e-
[Table]
Of Words
MD
Auditory
[Table] [Table]
54e
[Table] [Table]
73e-
[Table]
Of Words
MD
Visual
[Table] [Table]
98e4.28eCodeBERT
Language
Auditory
[Table]
1.36e
[Table] [Table]
77eCodeBERT
MD
Auditory
[Table] [Table]
97e-
[Table]
14eCodeBERT
Visual
Auditory
[Table] [Table]
09e-
[Table]
95eCodeBERTa
[Table] [Table]
41e
[Table] [Table]
CodeBERTa
MD
Auditory
[Table] [Table]
26e
[Table] [Table]
80eCodeBERTa
MD
Language
[Table]
1.30e-
[Table]
05eCodeBERTa
Visual
[Table] [Table]
30e-
[Table]
77eCodeGPT
Language
Auditory
[Table] [Table]
41e
[Table] [Table]
00eCodeGPT
MD
Auditory
[Table] [Table]
73e-
[Table]
84eCodeGPT
Visual
[Table] [Table]
[Table]
10e4.14eCodeTransformer
Language
Auditory
[Table]
[Table] 37e-
[Table]
73eCodeTransformer
MD
Auditory
[Table]
2.17e3.91eCodeTransformer
MD
Language
[Table] [Table]
29e-
[Table]
59eCodeTransformer
MD
Visual
[Table] [Table]
05e1.44e-
[Table]
[Figure] [Table] [Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure] [Table] [Figure]
[Table] [Figure]
[Table]
[Figure] [Table] [Figure]
[Table] [Figure]
[Table]
[Figure] [Table] [Figure]
[Table] [Figure]
[Table]
[Figure] [Table] [Figure]
[Table]
[Figure] [Table] [Figure]
[Table]
[Figure] [Table] [Figure]
[Table]
[Figure] [Table] [Figure]
[Table]
[Figure] [Table] [Figure]
[Table] [Figure]
[Table]
[Figure] [Table] [Figure]
[Table]
[Figure] [Table] [Figure]
[Table] [Figure]
[Table]
[Figure] [Table] [Figure]
[Table]
[Figure]
[Table]
[Figure] [Table] [Figure]
[Table] [Figure]
[Table]
[Figure]
[Table] [Figure]
[Table] [Figure]
[Table]
[Figure] [Table] [Figure]
[Table]
[Figure]
[Table]
[Figure] [Table] [Figure]
[Table]
[Figure] [Table]
[Figure] [Table] [Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table] [Figure]
[Table]
[Figure]
[Table]
[Figure] [Table] [Figure]
[Table]
[Figure]
[Table]
[Figure] [Table] [Figure]
[Table]
[Figure] [Table] [Figure]
[Table]
[Figure] [Table] [Figure]
[Table]
[Figure]
[Table]
[Figure] [Table] [Figure]
[Table]
[Figure]
[Table]
[Figure] [Table] [Figure]
[Table]
[Figure]
[Table]
[Figure] [Table] [Figure]
[Table]
[Figure]
[Table]
[Figure] [Table] [Figure]
[Table] [Figure]
[Table]
[Figure] [Table] [Figure]
[Table]
[Figure]
[Table]
[Figure] [Table] [Figure]
[Table] [Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure] [Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[-7.45e1.99e0 MD CodeTransformer TF-IDF 2. 2.73e1.06e
-][Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table] [Figure]
F Multi system partial regression analysis: Scores and significance
In order to investigate whether each of the brain systems included contribute unique informationtowards the decoding tasks, we combine brain representations from different systems in pairedcombinations and evaluate effects on downstream decoding performance across all experiments.
Wefind that the addition of MD or LS to VS, relative to VS alone, improves downstream decoding ofcode vs sentences .
The same effect is observed for the combination of MD and LS compared to eitheralone.
These data suggest that the MD system and the Language system encode unique variancerelevant to the decoding of code vs sentences , and this is above and beyond the information encodedin the Visual system or each other individually.
Additionally, we observe that the addition of MDto VS, relative to VS alone, improves downstream decoding of control flow , data type , dynamicanalysis , and static analysis .
This same effect is observed for the addition of LS to Visual system forcontrol flow and static analysis .
These data suggest that the MD system and the Language systemencode unique information above and beyond information encoded in the Visual system for thesedecoding tasks (Table 15).
We repeat this process for Experiment 2, where we find that the additionof MD to VS improves decoding for 7 models, the addition of MD to LS improves decoding for 7models, and the addition of LS to Visual system improves decoding for all 9 models (Table 16).
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table]
[Table]
[Table]
G All code properties: Scores and Correlations
We analyzed a series of code properties as part of the static and dynamic code analysis.
As several ofthese properties ( token count , node count , Halstead difficulty , cyclomatic complexity , and bytecodeops ) were revealed to be highly correlated in a post-hoc analysis, we report only one measure for thissubset in Experiment 1, token count , but include all scores here for completeness (Table 17).
Wealso report the correlation matrix between all code properties that led us to select token count as therepresentative property for this subset (Table 18)
[Table]
[Table]
[Table]
H Sensitivity of brain mapping to model output dimensions
Of potential interest to the decoding framework is not just the complexity of code models, but theirintrinsic dimensionality as well.
In order to investigate to what extent brain model mappings arerobust to changes in code model dimensionality, and to assess which model representations are mostsensitive to compression and expansion, we rerun our current decoding framework while controllingfor embedding size.
For each brain network to code model mapping task, prior to the MVPA analysis, we control forthe dimensionality of the code model embedding via projection through a Gaussian random matrix 908 R d 1 × d 2 drawn from N (0 , 1 /d 2 ) where d 1 is the original code model embedding dimensionality andd 2 is the desired dimensionality.
We observe that, on average, models of lower complexity (e.g., bag-of-words , TF-IDF , seq2seq )appear relatively robust to compression, whereas the most complex models (e.g., CodeTransformer , CodeGPT , CodeBERTa ) gain ample performance from higher dimensional expression, and sufferconsiderably when constrained.
These data suggest that higher dimensions of the encoder in complex models encode relevant neuralinformation.
Additionally, these effects appear to be most pronounced when decoding from the brainregion whose representations yield the strongest mappings, the MD system.
We note here however that we could be observing an interaction effect between model complexityand dimensionality output in these results.
Since we cannot fully control for the complexity ofthese models (by making them all ‘equally complex’), this experiment alone cannot drive definitiveconclusions.
These results instead constitute a preliminary exploration into the effects of code model dimensionalityon brain to model representation mappings, and suggest an avenue for future investigation.
I Robustness of Results to Regression Metric
For the decoding analysis of the continuous valued dynamic analysis and static analysis properties,it is reasonable to ask why the Pearson correlation metric was chosen as opposed to RMSE , as istypically customary for regression tasks.
While we present the results using the Pearson correlationmetric in the core results for interpretability via the zero-baselne, here we confirm that the use ofan RMSE metric leads to the same conclusions.
As we see here, the MD, LS, and Visual systemdecode the dynamic analysis property, and MD and LS decode the static analysis property withsignificantly lower RMSE than the null permutation baseline.
These results precisely confirm andmirror the patterns observed in Figure 3.
[Table]