Representations of Computer Programs in the Human Brain

1 INTRODUCTION . A question of interest , as yet uninvestigated , is what aspects of computer programs , i.e . code , are encoded by the human brain when comprehending them . Is it possible that common code properties and the semantics of programs are encoded in brain activity patterns when code is comprehended ? A few prior works have investigated the neural bases of programming ( Siegmund et al. , 2017 ; Floyd et al. , 2017 ; Peitek et al. , 2018 ; Castelhano et al. , 2019 ; Huang et al. , 2019 ; Krueger et al. , 2020 ; Prat et al. , 2020 ; Ivanova et al. , 2020 ; Liu et al. , 2020 ; Ikutani et al. , 2021 ; Peitek et al. , 2021 ) . These works use analysis techniques like functional magnetic resonance imaging ( fMRI ) and electroencephalography ( EEG ) to investigate brain regions involved in coding tasks like code comprehension , code writing , and data structure manipulation . The primary quest of these works though has been to locate physical regions in the brain involved in these activities , with the goal of determining whether code-related activity joins other activities supported by those brain regions . It remains unclear what specific code-related information these regions encode . Do these regions encode specific syntactic or semantic code properties ? Further , are different regions involved in encoding different such properties ? To date , no systematic method exists to generate the evidence from which to draw answers . These questions form the focus of this work . One way to learn what information is encoded in the brain is by recording brain signals when reading code ( through fMRI or EEG ) , and then decoding from the recorded signals a code property of interest . Being able to decode the property accurately from a specific region of the brain establishes that information related to that code property is faithfully represented in that brain region . A question central to such a decoding analysis would be the choice of the target code property–what code properties should be investigated ? We can hand-select a set of fundamental properties of code and test if they can be decoded . While helpful , such a set will not preclude other , more complex aspects of code being encoded . To address the limitation of hand-selected properties , we look to recent advances in unsupervised machine learning ( ML ) models trained on code . Dubbed code models , they process large corpora of code to learn ML model representations of computer programs . These models are increasingly being used in software engineering workflows ( Allamanis et al. , 2018a ) , and have been shown to perform well on tasks like code summarization ( Alon et al. , 2019 ) , detecting variable misuse ( Bichsel et al. , 2016 ) , and more recently , code auto-completion ( Chen et al. , 2021 ) . These continuous representations likely encode syntax and semantics of programs ( Bichsel et al. , 2016 ; Allamanis et al. , 2018b ; Srikant et al. , 2021 ) . Beyond the fundamental , hand-selected properties , they may encode and describe more complex code properties . Decoding these continuous representations from brain activity data then promises to inform us whether such additional properties are also encoded 1Project URL - https : //github.com/anonmyous-author/anonymous-code in the brain . Further , comparing representations from code models to the code-related information encoded in the brain also informs us whether code models resemble humans in the knowledge they learn and encode . Yamins et al . ( 2013 ) first showed how information encoded in our visual system resembles what convolutional neural networks learn when trained to recognize images . A similar correspondence can possibly be established between code models and the human brain . In this work , we study code comprehension , and utilize the fMRI recordings dataset from Ivanova et al . ( 2020 ) to investigate human brain representations of code . We present two means of proceeding—probing of brain region representations for specific code properties , and analyzing the mapping of these representations onto various code models with differing model complexity . We learn affine maps from brain representations to predict hand-selected code properties which summarize the syntactic and semantic behavior of programs , and similarly predict code representations of different code models . We investigate the effects that brain regions , the nature of code properties , and the complexity of the code models have on the accuracies with which we decode brain representations . We find that we can decode code-related information from brain representations , and can map representations of code models from brain representations . We find the Multiple Demand system , followed by the Language system , to consistently encode specific code properties and map with machine learned representations of code . We provide an open-source framework to replicate our experiments , and we release our data and analysis publicly . Link - https : //github.com/anonmyous-author/anonymous-code . The framework allows both brain data and program metrics to be easily added to extend the current set of experiments . This should enable authors from other neuroimaging studies or code model developers to collaborate and analyze data across these works , which will also help amortize the high costs of carrying out such experiments . 2 RELATED WORK . Of the prior works that have investigated the neural bases of programming through fMRI and EEG techniques ( Siegmund et al. , 2017 ; Floyd et al. , 2017 ; Peitek et al. , 2018 ; Castelhano et al. , 2019 ; Huang et al. , 2019 ; Krueger et al. , 2020 ; Ivanova et al. , 2020 ; Liu et al. , 2020 ; Ikutani et al. , 2021 ; Peitek et al. , 2021 ) and through behavioral studies ( Prat et al. , 2020 ; Casalnuovo et al. , 2020 ; Crichton et al. , 2021 ) , the following probe brain recordings for program properties encoded in them . Floyd et al . ( 2017 ) learn a linear model to successfully classify whether an observed brain activity corresponds to reading code or reading text . Ikutani et al . ( 2021 ) study expert programmers and show that it is possible to classify code into the four problem categories–math , search , sort , and string from the brain activations corresponding to the code . Similarly , Liu et al . ( 2020 ) classify whether a brain signal corresponds to code implementing an if condition or not . Peitek et al . ( 2021 ) analyze correlations between brain recordings of participants reading code and a set of code complexity metrics . In testing for code properties , our work uses a similar methodology ( a linear model trained on fMRI data ) , but we evaluate a larger set of static and dynamic code properties . We also systematically test whether key programming constructs like control flow and data operations are encoded by these brain representations . A systematic analysis of multiple such properties has not been performed in any of these works . Further , we perform these tests in those brain regions identified by Liu et al . ( 2020 ) and Ivanova et al . ( 2020 ) as being responsive specifically to code comprehension . This gives us a finer insight into how different regions encode these properties . In addition to these tests , we study representations generated by a suite of ML models with varying model complexity and compare those representations to brain representations . Brain representations have also been studied in domains like natural language , vision , and motor control . Among related works in natural language , a domain that resembles programming languages , Mitchell et al . ( 2008 ) ; Pallier et al . ( 2011 ) ; Brennan & Pylkkänen ( 2017 ) ; Jain & Huth ( 2018 ) ; Gauthier & Levy ( 2019 ) ; Schwartz et al . ( 2019 ) ; Wang et al . ( 2020 ) ; Schrimpf et al . ( 2020 ) ; Cao et al . ( 2021 ) have studied brain representations of words and sentences by relating them to representations produced by language models . While the broader tools we use to investigate these representations , like multi-voxel pattern analysis ( MVPA ) , are similar to some of these prior works , our focus is on properties specific to code and not natural language . 3 BACKGROUND . We provide a brief background on fMRI signals as a proxy for brain representations and describe the brain systems that we probe in this work . Measuring brain activity with fMRI . Functional magnetic resonance imaging ( fMRI ) is a brain imaging technique used to measure brain activity in specific brain regions . When a brain region is active , blood flows into the region to aid its processing . An MRI machine measures this change in blood flow , and reports BOLD ( blood oxygen level dependent ) values sampled at the machine ’ s frequency ( Glover , 2011 , usually 2 seconds ) . The smallest unit of brain tissue for which BOLD signal is recorded is called a voxel ( an equivalent of a 3D pixel ) ; it comprises several cubic millimeters of brain tissue . For our analyses , we select subsets of voxels belonging to specific brain systems . Following common practice , the parameters of a general linear model , fit to time-varying BOLD values , are used as a measure of the overall activation in each voxel in response to a given input . It is the values of these parameters that , according the neuroscience community , are brain representations . Brain systems . A system of brain regions can span different areas of the brain but behaves as a holistic unit , showing similar patterns of engagement across a given cognitive task . We probe the following systems of regions in our work . See Appendix A for a detailed description of these systems along with relevant references . • Multiple Demand ( MD ) system . This system of regions is known to engage in cognitively demanding , domain agnostic tasks like general problem solving , logic , and spatial memory tasks . Liu et al . ( 2020 ) and Ivanova et al . ( 2020 ) reported that this system is active during code comprehension . • Language system ( LS ) . This system responds during language production and comprehension across modalities ( speech , text ) and languages ( across 10 language families , including American sign language ) . Siegmund et al . ( 2014 ; 2017 ) reported activity in regions resembling the LS during code comprehension , whereas Floyd et al . ( 2017 ) ; Liu et al . ( 2020 ) ; Krueger et al . ( 2020 ) concluded that brain regions processing language and code are distinct . Ivanova et al . ( 2020 ) reported moderate levels of activity within the LS in response to Python , but not to code in a visual programming language , ScratchJr . We also probe brain representations in two brain systems responsible for perception . • Visual system ( VS ) . These regions respond primarily to visual inputs . We expect activity in this system to reflect low-level visual properties of the code ( e.g . code length and indentation ) . • Auditory system . These regions respond primarily to auditory inputs . We do not expect activity in this system to reflect code-related properties . 4 BRAIN AND MODEL REPRESENTATIONS . We describe in this section the method we follow to gather representations of code in the brain ( Section 4.1 ) , evaluate the different code properties they encode ( Section 4.2 ) , and how we compare brain representations to those generated by ML models ( Section 4.3 ) .

Summary: In this work, the authors explore the relationship between 4 different brain regions (multi-demand network, language network, visual system, auditory system) and different features of program code. Specifically, they look at hidden state representations of code language models (seq2seq, CodeBERTa, CodeTransformer, XLNet), tf-idf and BoW representations of the input. For non-LM based features, they look at a code vs. sentence contrast, variable language (English vs. Japanese variable names), data types (strings vs. numerals) and control flow (for vs. if vs. no branching) To analyze relationships between the BOLD signal and stimulus features, they build linear classifiers/regressors from the BOLD activity of each system to each of the stimulus features. Models are evaluated using classification accuracy and linear correlation for regression. In the case of hidden-state features, model are evaluated using rank accuracy.  Overall, the authors find that the visual system is capable of significantly predicting several of the hand-crafted code features suggesting that these features are correlated with low-level stimulus properties like program length. While differences between MD & LS are not significant, these models successfully predict 5/6 hand-crafted features. In the code representation prediction task, the authors find that the MD, LS and Visual systems are able to rank significantly above change. However, the LS and visual systems do not beat a random token embedding baseline. Aside from CodeBERTa, the MD system is also not significantly above the random baseline.