Does injecting linguistic structure into language models lead to better alignment with brain recordings?
Authors: Mostafa Abdou
Ana Valeria González
Mariya K Toneva
Daniel Hershcovich
Anders Søgaard
Venue:  (2021)
Keywords: neurolinguistics
natural language processing
computational neuroscience

Abstract: Neuroscientists evaluate deep neural networks for natural language processing as possible candidate models for how language is processed in the brain. These models are often trained without explicit linguistic supervision, but have been shown to learn some linguistic structure in the absence of such supervision (Manning et. al, 2020), potentially questioning the relevance of symbolic linguistic theories in modeling such cognitive processes (Warstadt & Bowman, 2020). We evaluate across two fMRI datasets whether language models align better with brain recordings, if their attention is biased by annotations from syntactic or semantic formalisms. Using structure from dependency or minimal recursion semantic annotations, we find alignments improve significantly for one of the datasets. For another dataset, we see more mixed results. We present an extensive analysis of these results. Our proposed approach enables the evaluation of more targeted hypotheses about the composition of meaning in the brain, expanding the range of possible scientific inferences a neuroscientist could make, and opens up new opportunities for cross-pollination between computational neuroscience and linguistics.



Under review as a conference paper at ICLR 2021
DOES INJECTING LINGUISTIC STRUCTURE INTO LAN- GUAGE MODELS LEAD TO BETTER ALIGNMENT WITH BRAIN RECORDINGS?
Anonymous authors Paper under double-blind review
ABSTRACT
Neuroscientists evaluate deep neural networks for natural language processing as possible candidate models for how language is processed in the brain. These mod- els are often trained without explicit linguistic supervision, but have been shown to learn some linguistic structure in the absence of such supervision (Manning al.. 2020), potentially questioning the relevance of symbolic linguistic theories in mod- eling such cognitive processes (Warstadt & Bowman 2020). We evaluate across two fMRI datasets whether language models align better with brain recordings, if their attention is biased by annotations from syntactic or semantic formalisms. Using structure from dependency minimal recursion semantic annotations, we find alignments improve significantly for one of the datasets. For another dataset, we see more mixed results. We present an extensive analysis of these results. Our proposed approach enables the evaluation of more targeted hypotheses about the composition of meaning in the brain, expanding the range of possible scien- tific inferences a neuroscientist could make, and opens up new opportunities for cross-pollination between computational neuroscience and linguistics.
INTRODUCTION
Recent advances in deep neural networks for natural language processing (NLP) have generated excitement among computational neuroscientists, who aim to model how the brain processes language. These models are argued to better capture the complexity of natural language semantics than previous computational models, and are thought to represent meaning in way that is more similar to how it is hypothesized to be represented in the human brain. For neuroscientists, these models provide possible hypotheses for how word meanings compose in the brain. Previous work has evaluated the plausibility of such candidate models by testing how well representations of text extracted from these models align with brain recordings of humans during language comprehension tasks (Wehbel
et al. 2014|Jair & Huth 2018) Gauthier & Ivanova 2018 Gauthicr & Levy 2019; Abnar et al.
2019 Toneva & Wehbe 2019 Schrimpl al. 2020 Caucheleux & King 2020). and found some
correspondences.
However, modern NLP models are often trained without explicit linguistic supervision (Devlin et al. 2018, Radford et al., 2019). and the observation that they nevertheless learn some linguistic structure has been used to question the relevance symbolic linguistic theories. Whether injecting such symbolic structures into language models would lead to even better alignment with cognitive mcasurements, however, has not been studied. In this work, we address this gap by training BERT (3.1) with structural bias, and evaluate its alignment with brain recordings (3.2) Structure is derived from three formalisms- -UD, DM and UCCA (3..))which which come from different linguistic traditions, and capture different aspects of syntax and semanties.
Our approach, illustrated in Figure I allows for quantifying the brain alignment of the structurally- biased NLP models in comparison to the base models, as related to new information about linguistic structure learned by the models that is also potentially relevant to language comprehension in the brain. More specifically, in this paper, we:
(a) Employ a fine-tuning method utilising structurally guided attention for injecting structural bias into language model (L.M) representations.

---

Under review as a conference paper at ICLR 2021
BERT model
Sentence or word Representations
"Harry thought he would
Kalignment prlor to intervention
fMRI
Decoder
x alignment post
altered BERT
>>
O,
successful
encodling relevant
Figure 1: Overview of our approach. We use BERT as a baseline and inject structural bias in two ways. Through a brain decoding task, we then compare the alignment of the (sentence and word) representations of our baseline and our altered models with brain activations.
(b) Assess the representational alignment brain activity measurements of the fine-tuned and non-fine-tuned LMs.
(c) Further evaluate the LMs range of targeted syntactic probing tasks and a semantic tagging task, which allow us to uncover fine-grained information about their structure- sensitive linguistic capabilities.
Present an analysis of various linguistic factors that may lead to improved deteriorated brain alignment.
BACKGROUND: BRAIN ACTIVITY AND NLP
Mitchell et al. (2008) first showed that there is relationship between the co-occurrence patterns of words in text and brain activation for processing the semantics of words. Specifically, they showed that a computational model trained on co-occurrence patterns for a few verbs was able to predict (MRI activations for novel nouns. Since this paper was introduced, many works have attempted to isolate other features that enable prediction and interpretation of brain activity (Frank al. 2015 Brennan et al.) 2016) Lopopolo et al.) 2017] Anderson et al.) 2017: Pereira et al. 2018 Wang et al. 2020). Gauthier & Ivanova (2018) however, emphasize that directly optimizing for the decoding of neural representation limiting, as it does not allow for the uncovering the mechanisms that underlic these representations. The authors suggest that in order for better understand linguistic processing in the brain, we should also aim to train models that optimize for a specific linguistic task and explicitly test these against brain activity.
Following this line of work, Toneva & Wehbe (019 present experiments both predicting brain activity and evaluating representations on set linguistic tasks. They first show that using uniform attention in early layers of BERT (Devlin et al. 2018) instead of pretrained attention leads to better prediction of brain activity. They then use the representations of this altered model to make predictions on range of syntactic probe tasks, which isolate different syntactic phenomena (Marvin & Linzen 2019), finding improvements against the pretrained BERT attention. Gauthier & Levy (2019) present series of experiments in which they fine-tune BERT on variety of tasks including language modeling as well as some custom tasks such as scrambled language modeling and pfrt-of-speech- language modeling. They then perform brain decoding, where linear mapping is learnt from fMRI recordings to the fine-tuned BERT model activations. They find that the best mapping is obtained with the scrambled language modelling fine-tuning. Further analysis using structural probe method
2

---

Under review as a conference paper at ICLR 2021
confirmed that the token representations from the scrambled language model performed poorly when used for reconstructing Universal Dependencies (UD; Nivre et al. 2016) parse trees.
When dealing with brain activity, many confounds may lead to seemingly divergent findings, such as the size of fMRI data, the temporal resolution of IMRI, the low signal-to-noise ratio, as well as how the tasks were presented to the subjects, among many other factors. For this reason, it is essential to take sound measures for reporting results, such as -ooss-valigg models, evaluating on unseen test sets, and conducting a thorough statistical analysis.
APPROACH
Figure T shows high-level outline of our experimental design, which aims to establish whether injecting structure derived from variety of syntacto semattic formalisms into neural language model representations can lead to better correspondence with human brain activation data. We utilize fMRI recordings of human subjects reading set of texts. Representations of these texts then derived from the activations of the language models. Following Gauthier & Levy (2019), we obtain LM representations from BERT" for all our experiments. We apply masked language model fine-tuning with attention guided by the formalisms to incorporate structural bias into BERT's hidden-state representations. Finally, to compute alignment between the BERT-derived representations with and without structural bias-and the fMRI recordings, we employ the brain decoding framework, where a linear decoder is trained to predict the LM derived representation of word sentence from the corresponding fMRI recordings.
3.1 LM-DERIVED REPRESENTATIONS
BERT uses wordpiece tokenization, dividing the text to sub-word units. For a sentence S made up of P wordpieces, perform mean-pooling over BERT's final layer hidden-states [/1, hp] obtaining a vector representation of the sentence Smean = P. E, hp (Wu et al. 2016). In initial experiments, we found that this leads to a closer match with brain activity measurements compared to both max-pooling and the special [CLS] token, which is used by Gauthier & Levy (2019). Similarly, for a word W made up of P wordpieces, to derive word representations, we apply mean-pooling over hidden-states [h1,-.., hp], which correspond the wordpieces that make up W: W mean = E, hy-
For each dataset, DIM E dR= denotes a matrix of LM-derived word or sentence representations where dæ is BERT's hidden layer dimensionality (dæ 1024 in our experiments).
3.2 NEUROIMAGING DATASETS
We utilize two fMRI datasets, which differ in the granularity of linguistic cues to which human responses were recorded. The first, collected in Pereira et al. (2018)' 's experiment 2, comprises a single brain image per entire sentence. In the second, more fine-grained dataset, recorded by Weebbe et al. 2014), each brain image corresponds to 4 words. We conduct a sentence-level analysis for the former and word-level one for the latter
Pereira2018 consists of fMRI recordings from 8 subjects. The subjects were presented with stimuli consisting of 96 Wikipedia-style passages written by the authors, consisting of 4 sentences each. The subjects read the sentences one by one and were instructed to think about their meaning. The resulting data for each subject consists of 384 vectors of dimension 200,000; vector per sentence. These were reduced to 256 dimensions using PCA by Gauthier & Levy 2019). These PCA projections explain more than 95% of the variance among sentence responses within each subject. We use this reduced version in our experiments.
Wehbe2014 consists of fMRI recordings from 8 subjects as they read chapter from Harry Potter and the Sorcerer's 's Stone. For the 5000 word chapter, subjects were presented with words one by one for 0.5 seconds each. An fMRI image taken every 2 seconds, a result, each image corresponds
Specifically: bert - a rgg-uncased trained with whole-word masking. EEvn though the images are recorded at the 4-gram level of granularity, a word-level analysis is applied, as SShwwrtz et al. 2019).
3

---

Under review as a conference paper at ICLR 2021
1ad
Josking foreaz
bal
boen
boking
frward
learing
ne
Dan
aesturg do
von
lonring
25yting
do
(a) UD (above, orange), DM (below, bluc)
(b) UCCA
Figure 2: Manually annotated example graphs for a sentence from the Wehbe2014 dataset. While UCCA and UD attach all words, DM only connects content words. However, all formalisms capture basic predicate-argumeet
structure, for example, denoting that "more than anything else" modifies "looking forward" rather than "fly".
to words. The data was further preprocessed (i.e. detrended, smoothed, trimmed) and released by Toneva & Wehbe (2019). We use this preprocessed version to conduct word-level analysis, for which we use PCA to reduce the dimensions of the fMRI images from 25,000 to 750, explaining at least 95% variance for each participant.
3.3 FORMALISMS AND DATA
To inject linguistic structure into language models, we experiment with three distinct formalisms for representation of syntactic/semantic structure, coming from different linguistic traditions and capturing different aspects of linguistic signal: UD, DM and UCCA. An example graph for each formalism is shown in Figure 2 Although there are other important linguistic structured formalisms, including meaning representations such as AMR [Banarescu et al. 2013). DRS (Kamp & Reyle 1993; Bos al. 2017) and FGD (Sgall et al. 1986 Hajic et al.] 2012), select three relatively different formalisms as a somewhat representative sample. All three have manually annotated datasets, which use for experiments.
UD (Universal Dependencies; Nivre et al. 2020) is a syntactic bi-lexical dependency framework (dependencies are denoted arcs between words, with one word being the head and another the dependent), which represents grammatical relations according to a coarse sross-lingual scheme. For UD data, use the English Web Treebank corpus (EWT; Silveira et al. 2014), which contains 254,830 words and 16,622 sentences, taken from five genres of web media: weblogs, newsgroups, emails, reviews, and Yahoo! answers.
DM (DELPH-IN MRS Bi-Lexical Dependencies; Ivanova et al. 2012) is derived from the underspec- ified logical forms computed by the English Resource Grammar (Flickinger et al.) 2017 Copestake et all.] (005), and is one of the frameworks targeted by the Semantic Dependency Parsing SemEval Shared Tasks (SDP: Oepen et al. 2014 2015). We use the English SDP data for DM (Oepen et al.) 2016), annotated on newspaper text from the Wall Street Journal (WSJ), containing 802,717 words and 35,656 sentences.
UCCA (Universal Cognitive Conceptual Abnetation;AAbend & Rappoport 2013 based cognitive linguistic and typological theories, primarily Basic Linguistic Theory (Dixon 2010/2012). We use UCCA annotations over web reviews text from the English Web Treebank, and from English Wikipedia articles on celebrities. In total, they contain 138,268 words and 6,572 sentences. For uniformity with the other formalisms, which have a hierarchical constituency-like structure, to bi-lexical graphs with edges between words. This conversion keeps about 91 % of the information (Hershcovich et al. 2017).
bi-lexical approximation
convert UCCA graphs,
3.4 INJECTING STRUCTURAL BIAS INTO LMS
Recent work has explored ways of modifying attention in order to incorporate structure into neural models (Chen et al. 2016: Strubell et al. 2018 Strubell & McCallum 2018 Zhang et al. 2019
4

---

Under review as a conference paper at ICLR 2021
Bugliarello & Okazaki 2019). For instance, Strubell et al. (2018) incorporate syntactic information by training one attention head to attend to syntactic heads, and find that this leads to improvements in Semantic Role Labeling (SRL). Drawing
modify the BERT Masked Language Model (MLM) objective with an additional structural attention constraint. BERTLARGE consists of 24 layers and 16 attention heads. Each attention head head; takes in as input sequence of representations h = [hi,.... hp] corresponding to the / wordpieces in the input sequence. Each representation in hip is transformed into query, key, and value vectors. The scaled dot product is computed between the query and all keys and softmax function is applied to obtain the attention weights. The output of head; is matrix Oz, corresponding to the weighted sum of the value vectors.
these approaches,
For each formalism and its corresponding corpus, we extract an adjacency matrix from each sentence's parse. For the sequence S, the adjacency matrix As is a matrix of size x P. where the columns correspond to the heads in the parse tree and the rows correspond to the dependents. The matrix elements denote which tokens are connected in the parse tree, taking into account BERT's wordpiece tokenization. Edge directionality is not considered. We modify BERT to accept input matrix As as well as S; maintaining the original MLM objective. For each attention head head,, we compute the binary cross-entropy loss between O, and As and add that to our total loss, potentially down-welte weighted by a factor of a (a hyperparameter). BERT's 's default MLM fine-tum fine-tuning hyperparameters are employed and a is set 0.1 based on validation set perplexity scores in initial experiments.
Structural information can be injected into BERT in many ways, in many heads, across many layers. Because the appropriate level and extent of supervision is unknown priori, we run various fine- tuninig settings with respect to combinations of number of layers (l...., 24) and attention heads (1,3, 5, 7,9, 11, 12) supervised via attention guidance. Layers are excluded from the bottom up (e.g.: when 10 layers are supervised, it is the topmost 10); heads are chosen according to their indices (which are arbitrary). This results in a total of 168 fine-tuning settings per formalism. For each fine-tuning setting, we perform two fine- teeiintiing runs For each run of each fine-tuning setting f. we derive a set of sentence or word representations D,r E R=ndd from each fine-tuned model using the approach described in [3.1 for obtaining Pry.. the baseline set of representations from BERT before fine-tuning. We then use development se embedding space hubness- an indicator of the degree of difficulty of indexing and analysing data (Houle 2015) which has been used to evaluate 2014)- as an unsupervised selection criterion for the fine-tuned models, selecting the model with the lowest degree of hubness (per formalism) according to the Robin Hood Index (Feldbauer et al.) 2018). This yields three models for each of the two datasets -one per formalism--or which we present results below.
embedding space quality (Dinu et al.,
In addition to the approach described above, we also experiment with directly optimizing for the prediction of the formalism graphs (i.e., parsing) as a way of encoding structural information in LM representations. We find that this leads to a consistent decline in alignment of the LMs' representations to brain recordings. Further details can be found in Appendix
3.5 BRAIN DECODING
To measure the alignment of the different LM-derived representations the brain activity measure- ments, brain decoding is performed, following the setup described in Gauthier & Levy (2019) For each subject i's fMRI images corresponding to set of sentences or words, ridge regression model is trained to linearly map from brain activity B; E Raxda (n 384; dn 256 for Pereira2018 and n 4369; dØ = 750 for Wehbe2014) to a L--derived representation (Dyr or DI.M), minimizing the following loss:
Cifr = [B - D 11 + A ||G-,
where Gi-fr:: Rduxde xda is linear map, and À is a hyperparameter for ridge regularization. Nested 1 12-fold cross- cross-validation (Cawley & Talbot 2010) is used for selection of X, training and evaluation.
We find that the mean difference brain decoding score (Pearson's 's r) between two runs of the same setting (across all settings) is low (0.003), indicating that random initialization does not play a major part in our results. We, therefore, do not carry out more runs.
4 For Wehbe2014: 4: second chapter of Harry Potter. For Pereira2018: first sentences of English Wikipedia.
'Other methods for evaluating representational correspondence such as Representational Similarity Analysis Kriegeskorte et al. 2008) and the Centered Kernel Alignment similarity index (Kornblith et al. 2019) were also explored hut were found he either less powerful less consistent across subjects and datasets.
5

---

Under review as a conference paper at ICLR 2021
Percira at (2018)
Webbe al. (2014)
pretraned
0.30
0.30
0.25
0.25
0.20
0.20
0.10
0.10
0.05
0.05
om
om
formalism
formalism
Figure 3: Brain decoding score (mean Pearson's 's r; with 95% confidence intervals shown for subject scores) for models fine-tuned by MLM with guided attention on each of the formalisms, as well the baseline models: pretrained BERT (dotted line), and BERT fine-tuned by MLM on each formalism's training text without guided attention (domain-finee finetuned BERT, solid lines).
Evaluation evaluate the regression models, Pearson's 's correlation coefficient between the pre- dicted and the corresponding heldout true sentence word representations is computed. We find that this metrid is consistent across subjects and across the two datasets. We run 5000 bootstrap resampling iterations and a) report the mean correlation coefficient (referred to brain decoding score/performance) b) use paired bootstrap test to establish whether two models' mean (across stimuli) scores drawn from populations having the same distribution [ c) apply the Wilcoxon signed rank test [Wilcoxon] (992) to the by-subject scores to for evidence of strength of generalization over subjects. Bonferroni correction (correcting for 3 multiple comparisons) used adjust for multiple hypothesis testing. See Appendix(C for details.
4 RESULTS
To evaluate the effect of the structurally-guider attention, we compute the brain decoding scores for the guided-attention models corresponding to each formalism and IMRI dataset and compare these scores against the brain decoding scores from two baseline models: 1) domain--inetunee BERT (DF), which finetunes BERT using the regular MLM objective on the text of each formalism' training data, and a pretrained BERT. We introduce the domain--inetunnd baseline in order to control for any effect that finetuning using a specific text domain may have on the model representations. Comparing against this baseline allows to better isolate the effect of injecting the structural bias from the possible effect simply fine-tuning on the text domain. We further compare to a pretrained baseline order to evaluate how the structurally-guide attention approach performs against an off-the-shelf model that is commonly used in train-alignment experiments.
4.1 PEREIRA2018
Figure shows the sentence-level brain decoding performance on the Pereira2018 dataset, for the guided attention fine-tuned models (GA) and both baseline models (domain-finetune, and pretrained). We find that the
baseline (shown in Figure as solid lines) leads brain decoding
6 Appenddix shows results for the rank- base- metric reported in Gauthier Levy 2019). which we find to strongly correspond to Pearson's 's correlation. This metric evaluates representations hased on their support for contrasts between sentences/words which are relevant to the brain recordings. Other metrics for the evaluation of goodness of were found
less consistent.
7 Across fine-tuning runs, cross-validation splits, and bootstrap iterations.
"This applied per subject to test for strength of evidence of generalization sentence stimuli.

---

Under review as a conference paper at ICLR 2021
scores that are either lower than or not significantly different from the pretrained baseline. Specifically, for DM and UCCA, the DF baseline performs below the pretrained baseline, which suggests that simply fine-tuning on these corpora results in BERT's representations becoming less aligned with the brain activation measurements from Pereira2018. We find that all GA models outperform their respective DF baselines (for all subjects, p < 0.05). We further find that compared the pretrained baselines, with p < 0.05: a) the UD GA model shows significantly better brain decoding scores for 7 out of 8 subjects, b) the DM GA model for 4 out of 8 subjects, c) the UCCA GA shows scores not significantly different from or lower, for all subjects. For details see Appendix C
4.2
WEHBE2014
For Wehhe2014, where analysis conducted on the word level, we again find that Nomain-finetunee especially the one finetuned on the UCCA domain text text-achieve considerably lower brain decoding scores than the pretrained model, as shown in Figure 3 Furthermore, the guided-attention models for all three formalisms outperform both baselines by large, significant margin (after Bonferroni correction, p < 0.0001).
DISCUSSION AND ANALYSIS
Overall, our results show that structural bias from syntacto-semantic formalisms can improve the ability of a linear decoder to map the BERT representations of stimuli sentences to their brain recordings. This improvement is especially clear for Wehbe 2014, where token representations and not aggregated sentence representations (as in Pereira 2018) are decoded, indicating that finer-grain recordings and analyses might be necessary for modelling the correlates of linguistic structure in brain imaging data. To arrive at better understanding of the effect of the structural bias and its relationship to brain alignment, in what follows, we present an analysis of the various factors which affect and interact with this relationship.
The effect of domain Our results suggest that the domain of fine-tuning data and of stimuli might play significant role, despite having been previously overlooked: simply fine-tuning different domains leads to varying degrees of alignment to brain data. To quantify this effect, we compute the average word perplexity of the stimuli from both fMRI datasets for the pretrained and DF baselines on each of the three domain datasets If the domain of the corpora used for fine-tuning influences our results hypothesized, we expect this score be higher for the DF baselines. We find that this is indeed the case and that for those baselines (DF), increase in perplexity roughly corresponds lower brain decoding scores- see detailed results in Appendix [D] This finding calls to attention the necessity of accounting for domain match in work utilizing cognitive measurements and emphasizes the importance of the
data from
baseline in this study.
Targeted syntactic evaluation We evaluate all models on a range of syntactic probing tasks proposed by Marvin & Linzen (2019) 10 This dataset tests the ability of models distinguish minimal pairs of grammatical and ungrammatical sentences across a range of syntactic phenomena. Figure shows the results for the three Wehbe2014 models across all subject-verb agreement (SVA) tasks We observe that after attention-guided fine-tuning: a) the DM guided-attention model. and to a lesser extent the UD mnided-attention model have a higher score than the pretrained baseline and the comain-finetuned baselines for most SVA tasks and b) the ranking of the models corresponds to their ranking on the brain decoding task (DM > UD > UCCA) [2] Although all three formalisms annotate the bubject-verb-obeer or predicate-argument structure necessary for solving SVA tasks, appears that some of them do so more effectively, at least when encoded into a LM by GA.
NNte that this is not equivalent to the commonly utilised sequence perplexity (which can not be calculated for non-auto-regee regressvgee models) but suffices for quantifying the effect of domain shift.
Ussng the evaluation script from Goldberg (201)).
11 See Appendix F for the full set of results for both Wehbe2014 and for Pereira2018 with similar patterns. For reflexive anaphora tasks, these trends are reversed: the models underperform the pretrained baseline and their ranking is the converse of their brain decoding scores. Reflexive Anaphora, are not explicitly annotated for in any of the three formalisms. We find, however, that they occur in a larger proportion of the sentences comprising the UCCA corpus (1.4%) than those the UD (0.67%) or DM (0.64%) ones, indicating that domain might play role here too.
7

---

Under review as a conference paper at ICLR 2021
1.0
XARASII ugint LDVG Merthe
satwott sgen y21, Lamphamat,
vddenii agmit cosso snddeed
sil ugint stydde
AdbeoD AgTT AODO exsted Cari
persine
04
RccUPaKY score sysyyettc task
02
86-
9.4
UD
Formaliam
Figure 4: Accuracy per subject-verb agreement category of MaMavi & Linzen (2019) for the three Wehbe2014 models and each of the four baselines.
Effeet on semantics To evaluate the impact of structural bias on encoding of semantic information, consider Semantic Tagging (Abzianidze & Bos 2017). commonly used analyse the semantics encoded in LM representations (Belinkov et al. [2018] [Lu et al., 2019): tokens are labeled to reflect their semantic role in context. For each of the three guided attention Wehbe2014 models and the pretrained model, a linear probe is trained to predict a word's semantic tag, given the contextual representation induced by the model (see Appendix B for details). For each of the three GA models, Figure shows the change in test set classification FI Fl-score 13 relative to the pretrained baseline, per coarse- grained grouping tags We find that the structural bias improves the ability to correctly recogmize almost all of the semantic phenomena considered, indicating that our method for injecting linguistic structure leads to better encoding of broad range of semantic distinctions. Furthermore, the improvements are largest for phenomena that have a special treatment in the linguistic formalisms, namely discourse markers and temporal entities. Identifying named entities is negatively impacted by GA with DM, where they are indiscriminately labeled as compounds.
Content words and function words are treated differently by each of the formalisms: UD and UCCA encode all words, where function words have special labels, and DM only attaches content words. Our guided attention ignores edge labels (dependency relations), and so it considers UD and UCCA's attachment of function words just as meaningful as that of content words. Figure Appendix showss breakdown of brain decoding performance Wehbe2014. We find that: a) all GA models and the pretrained model show higher function than content word decoding score, b) large part of the decrease in decoding score of two of the three baselines (UD and DM) compared to the pretrained model is due to content words.
content and function words for
Note that the test set consists of 263.516 instances, therefore, the margin of change in number of instances here is considerable, e.g. 5652 0.6 40 instances for the DM and UCCA models the temporal category, which is the least frequent the test See
category frequencies in the appendix.
"The eight most frequent boarse-graine categories from an original set of ten are included -oneneeddrdrrrddd by frequency from left to right; exclude the UNKNOWN category hecause is uninformative and the ANAPHOR T category because it shows change from the baseline for all three models.
8

---

Under review as a conference paper at ICLR 2021
U3 UODA
64-
Chargee
00
-2-
ATTRSUUTE
UNIVEDENIIY
EXENT
ascóuuse
TEMPORAA
Semand:: category
Figure 5: Change in Fl-score per coarse-grained semantic class compared to the pretrained baseline for the three guided attention Wehhe2014 models.
Caveats The IMRI data used for both the sentence and word level analyses was recorded while participants read text without performing a specific task. Although we observe some correlates of linguistic structure, it possible that uncovering more fine-grained patterns would necessitate brain data recorded while participants perform targeted task. For future work it would be interesting to investigate if an analysis based on a continuous, naturalistic listening fMRI dataset (Brennan & Hale 2019) matches up to the results we have obtained. Regarding the different linguistic formalisms, there are potential confounds such as domain, corpus sizı 15 and dependency length, (i.c. the distance between words attached by relation), which depend both the formalism and on the underlying training set text. To properly control for them, a corpus annotated for all formalisms is necessary, but such corpus of sufficient size is not currently available.
Conclusions We propose a framework investigate the effect of incorporating specific structural biases in language models for brain decoding. We present evidence that inducing linguistic structure bias through fine-tuning using attention guided according to syntacto-semante improve brain decoding performance across two fMRI datasets. For each of the 3 investigated formalisms, we observed that the models that aligned most with the brain performed best at a range of subject-verb agreement syntactic tasks, suggesting that language comprehension in the brain, as captured by fMRI recordings, and the tested syntactic tasks may rely on common linguistic structure, that partly induced by the added attention constraints. Across formalisms, with attention guided by DM and UD consistently exhibited better alignment with the brain than UCCA for both fMRI datasets. Rather than concluding that DM and UD are plausible, controlled experiments, with fine-tuning on each annotated corpus as plain text, suggest that the text domain an important, previously overlooked confound. Further investigation needed using a common annotated corpus for all formalisms to make conclusions about their relative aptness.
formalisms, can
found that models
cognitively
Overall, our proposed approach enables the evaluation of more targeted hypotheses about the com- position of meaning in the brain, and opens up new opportunities for cross cross-pollination between computational neuroscience and linguistics. facilitate this, we make all code and data for our experiments available at: http: thub. com aoon///nommmmm zed
15 is interesting to note that decoding score rank for Wehbe2014 corresponds to fine-tuning corpus size for the GA models (DM > UD UCCA), but not the from this that dataset size might play role in the effective learning of structural bias.
models. A reasonable conclusion to draw

---

Under review as a conference paper at ICLR 2021
REFERENCES
Omri Abend and Ari Rappoport. Universal conceptual cognitive annotation (UCCA). In Proceedings of the s1 Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), pp. 228-238, Sofia, Bulgaria, August 2013. Association for Computational Linguistics. URL https: www. awwwwb. www.ww..og///aaaaea... httpb//oor.gaannoooo/o////-----/
Samira Abnar, Lisa Beinborn, Rochelle Choenni, and Willem Zuidema. Blackbox meets blackbox: Representational similarity and stability analysis of neural language models and brains. arXiv
preprint arXiv: arXiv: 19..00/996.009,, 2019.
Lasha Abzianidze and Johan Bos. arXiv:1709.10381, 2017.
Towards universal semantic tagging.
arXiv preprint
Andrew James Anderson, Jeffrey R Binder, Leonardo Fernandino, Colin J Humphries, Lisa Conant, Mario Aguilar, Xixi Wang, Donias Doko, and Rajeev DS Raizada. Predicting neural activity patterns associated with sentences using neurobiologically motivated model of semantic representation. Cerebral Cortex, 27(9):4379 4395, 2017.
Laura Banarescu, Claire Bonial, Shu Cai, Madalina Georgescu, Kira Griffitt, Ulf Hermjakob, Kevin Knight, Philipp Koehn, Martha Palmer, and Nathan Schneider. Abstract meaning representation for sembanking. In Proceedings of the 7th Linguistic Annotation Workshop and Interoperability with Discourse, pp. 178-186, Soñia, Bulgaria, August 2013. Association for Computational Linguistics. URL htt /www. aclweb wwtp/orggggaglgy//g/e/ 23322332
Yonatan Belinkov, Lluís Mårquez, Hassan Sajjad, Nadir Durrani, Fahim Dalvi, and James Glass. Evaluating layers of representation in neural machine translation part-of-specch and semantic tagging tasks. arXiv preprint arXiv:1801.07772. 2018.
Johan Bos, Valerio Basile, Kilian Evang, Noortje J Venhuizen, and Johannes Bjerva. The groningen meaning bank. In Handbook of linguistic annotation, pp. 463-496. Springer, 2017.
Jonathan R Brennan and John T Hale. Hierarchical structure guides rapid linguistic predictions during naturalistic listening. PloS one, 14(1)=0000741,,2g) 2019.
Jonathan R Brennan, Edward P Stabler, Sarah E Van Wagenen, Wen-Ming Luh, and John T Hale. Abstract linguistic structure correlates with temporal activity during naturalistic comprehension. Brain and language, 157:81-94, 2016.
Emanuele Bugliarello and Naoaki Okazaki. Enhancing machine translation with dependency-aware self-attention. arXiv preprint arXiv:1909.032 2019.
Charlotte Caucheteux and Jean-Rémi King. Language processing in brains and deep neural networks: computational convergence and its limits. BioRxiv, 2020.
Gavin C Cawley and Nicola LC Talbot. On over aver-fitting in model selection and subsequent selection bias in performance evaluation. The Journal of Machine Learning Research, 11 11:2079 2010.
Wanxiang Che, Longxu Dou, Yang Xu, Yuxuan Wang, Yijia Liu, and Ting Liu. Hit-scir at mrp 2019: A unified pipeline for meaning representation parsing via efficient training and effective encoding. In Proceedings of the Shared Task Cross-Framework Meaning Representation Parsing at the 2019 Conference on Natural Language Learning, pp. 76-85, 2019.
Yun-Nung Chen, Dilek Hakkani-Tur, Gokhan Tur, Asli Celikyilmaz, Jianfeng Gao, and Li Deng. Knowledge as a teacher: Knowledge guigge--guddddddde structural attention networks. arXiv preprint arXiv:1609.03286. 2016.
Ann Copestake, Dan Flickinger, Carl Pollard, and Ivan A Sag. Minimal recursion semantics: An introduction. Research language and computation, 3(2-3):281-332, 2005.
Jacob Devlin, Ming-Wor Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv: /8/0: 048005 2018.
10

---

Under review as a conference paper at ICLR 2021
Georgiana Dinu, Angeliki Lazaridou, and Marco Baroni. Improving zero-shot learning by mitigating the hubness problem. arXiv preprint arXiv:1412.6568, 2014.
Robert M. W. Dixon. Basic Linguistic Theory. Oxford University Press, 2010/2012.
Roman Feldbauer, Maximilian Leodolter, Claudia Plant, and Arthur Flexer. Fast approximate hubness reduction for large nigh-dimensional data. In 2018 IEEE International Conference Big Knowledge (ICBK), pp. 358-367. IEEE, 2018.
Dan Flickinger, Stephan Oepen, and Emily M Bender. Sustainable development and refinement of complex linguistic annotations at scale. In Handbook of Linguistic Annotation, pp. 353-377. Springer, 2017.
Stefan L Frank, Leun J Otten, Giulia Galli, and Gabriella Vigliocco. The erp response to the amount of information conveyed by words in sentences. Brain and language, 140:1-11, 2015.
Jon Gauthier and Anna Ivanova. Does the brain represent words? an evaluation of brain decoding studies of language understanding. arXiv preprint arXiv: 800 00591, 2018.
Jon Gauthier and Roger Levy. Linking artificial and human neural representations of language. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp. 529-539, 2019.
Yoav Goldberg. Assessing bert's syntactic abilities. arXiv preprint arXiv: 2019.
Jan Hajic, Eva Hajicová, Jarmila Panevová, Petr Sgall, Ondrej Bojar, Silvie Cinková, Eva Fucíková, Marie Mikulová, Petr Pajas, Jan Popelka, et al. Announcing prague czech-english dependency treebank 2.0. In LREC. pp. 3153-3160, 2012.
Daniel Hershcovich, Omri Abend, and Ari Rappoport. A transition-based directed acyelic graph parser for UCCA. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). pp. 1127-1138, Vancouver, Canada, July 2017. Association for Computational Linguistics. doi: 1...8663/11/117-1144 URL https: / /www wcwweb. /rg
antholegy/P17-11444
Daniel Hershcovich, Miryam de Lhoneux, Artur Kulmizev, Elham Pejhan, and Joakim Nivre. Køpsala: cansiition-based graph parsing via efficient training and effective encoding. In Proceedings of the 16th International Conference on Parsing Technologies and the IWPT 2020 Shared Task on Parsing into Enhanced Universal Dependencies, pp. 236-244, Online, July 2020. Association for Computational Linguistics. (ttttttps:/ /ttttp:://www.c.. acl wwt.//rtttogggggoooooooooo http:/llooyy//2.ooo..2.
Michael E Houle. Inlierness, outlierness, hubness and discriminability: an extreme-value-thhoeee foundation. National Institute of Informatics Technical Report NII-2015-002E, Tokyo, Japan, 2015.
Angelina Ivanova, Stephan Oepen, Lilja Øvrelid, and Dan Flickinger. Who did what to whom? Proceedings of the Sixth Linguistic
contrastive study of syntacto-semantic dependencies. Annotation Workshop. pp. 11, Jeju, Republic of Korea, July 2012. Association for Computational Linguistics. URL https: : www. aclweb. org/ ant hwwgg. /Wllccyyyyyllll11 -N02-3622
Shailee Jain and Alexander Huth. Incorporating context into language encoding models for fmri.
bioRxiv, pp. 327601, 2018
Hans Kamp and Uwe Reyle. From discourse to logic: introduction modeltheoretic semantics of natural language, formal logic and discourse representation theory. Studies in linguistics and philosophy, 1993.
Simon Kornblith, Mohammad Norouzi, Honglak Lee, and Geoffrey Hinton. Similarity of neural network representations revisited. arXiv preprint 190500000 2019.
Nikolaus Kriegeskorte, Marieke Mur, and Peter A Bandettini. Representational similarity analysis- connecting the branches of systems neuroscience. Frontiers in systems neuroscience, 2:4, 2008.
11

---

Under review as a conference paper at ICLR 2021
Nelson F Liu, Matt Gardner, Yonatan Belinkov, Matthew E Peters, and Noah A Smith. Linguistic knowledge and transferability of contextual representations. arXiv preprint DrXiv:1903.0830 2019.
Alessandro Lopopolo, Stefan Frank, Antal Van den Bosch, and Roel Willems. Using stochastic language models (slm) to map lexical, syntactic, and phonological information processing in the brain. PloS one, 12(5):e0777794 2017.
Christopher D Manning, Kevin Clark, John Hewitt, Urvashi Khandelwal, and Omer Levy. Emergent linguistic structure artificial neural networks trained by self-supervision. Proceedings of the National Academy of Sciences, 2020.
Rebecca Marvin and Tal Linzen. Targeted syntactic evaluation of language models. Proceedings of the Society for Computation in Linguistics, 2(1):373-374, 2019.
Tom M Mitchell, Svetlana V Shinkareva, Andrew Carlson, Kai-Min Chang, Vicente L Malave, Robert A Mason, and Marcel Adam Just. Predicting human brain activity associated with the meanings of nouns. science, 320(5880): 191-11999999 195, 2008
Joakim Nivre, Marie-Catherine De Marneffe, Filip Ginter, Yoav Goldberg, Jan Hajic, Christopher Manning, Ryan McDonald, Slav Petrov, Sampo Pyysalo, Natalia Silveira, et al. Universal de- pendencies v1: A multilingual treebank collection. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (I.REC' 16), pp. 1659-1666, 2016.
Joakim Nivre, Marie-Catherine de Marneffe, Filip Ginter, Jan Hajič, Christopher D. Manning, Sampo Pyysalo, Sebastian Schuster, Francis Tyers, and Daniel Zeman. Universal Dependencies v2: An evergrowing multilingual treebank collection. In Proceedings of The 12th Language Resources and Evaluation Conference, pp. 4034-4043, Marseille, France, May 2020, European Language Resources Association. ISBN 979- 10-95544--54-44444 https:ttp::/// httppp::/wwww.. www.ww.ww......o......... / mothology/2020... 1re.2020.00.1.1110
Stephan Oepen, Marco Kuhlmann, Yusuke Miyao, Daniel Zeman, Dan Flickinger, Jan Hajič, Angelina Ivanova, and Yi Zhang. SemEval 2014 task 8: Broad-coverage semantic dependency parsing. In Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014), pp. 63-72, Dublin, Ireland, August 2014. Association for Computational Linguistics. doi: 10.3115/v1/ S14-2008. URL https::/ /www aclweb. owg wwtp.///toooooy///ee---e-2222
Stephan Oepen, Marco Kuhlmann, Yusuke Miyao, Daniel Zeman, Silvie Cinková, Dan Flickinger, Jan Hajič, and Zdeňka Urešová. SemEval 2015 task 18: Broad-coverage semantic dependency parsing. In Proceedings of the 9th International Workshop
Semantic Evaluation (SemEval 2015), pp. 915-922 926, Denver, Colorado. June 2015. Association for Computational Linguistics. doi: 10.18653/v1/S55-2153. Utt.httss::/ www. a wwt.www...ooo....oo..oo.o..o.... org. waahh// ogy/s15-2/2
Stephan Oepen, Marco Kuhlmann, Yusuke Miyao, Daniel Zeman, Silvie Cinková, Dan Flickinger, Jan Hajic, Angelina Ivanova, and Zdenka Urešová. Semantic dependency parsing (sdp) graph banks release 1.0 Idc2016t10. Web Download, 2016.
Francisco Pereira, Bin Lou, Brianna Pritchett, Samuel Ritter, Samuel J Gershman, Nancy Kanwisher, Matthew Botvinick, and Evelina Fedorenko. Toward a universal decoder of linguistic meaning from brain activation. Nature communications, 111):1-13, 2018.
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. Language models are unsupervised multitask learners. OpenAl Blog, 188)99 2019.
Martin Schrimpf, Idan Blank, Greta Tuckute, Carina Kauf, Eghbal A. Hosseini, Nancy Kanwisher, Joshua Tenenbaum, and Evelina Fedorenko. Artificial neural networks accurately predict language processing in the brain. bioRxiv, 2020. doi: 10. 010.0//220.06.2..1777... 06. www.2222.....662....22.
URL htRpss:::: www
cont ceaetten///aaaee /tto:x/w..rr/ccoottttt//////////
Dan Schwartz, Mariya Toneva, and Leila Wehbe. Inducing brain-relevant bias in natural language processing models. In Advances in Neural Information Processing Systems, pp. 14123-14133 2019.
12

---

Under review as a conference paper at ICLR 2021
Petr Sgall, Eva Hajicová, and Jarmila Panevová. The meaning of the sentence and its semantic and pragmatic aspects. academia, 1986.
Natalia Silveira, Timothy Dozat, Marie-Catherine De Marneffe, Samuel R Bowman, Miriam Connor, John Bauer, and Christopher D Manning. gold standard dependency corpus for english. In LREC, pp. 2897-2904. Citeseer, 2014.
Emma Strubell and Andrew McCallum. Syntax helps elmo understand semantics: Is syntax still relevant in deep neural architecture for srl? arXiv preprint 1rXiv:/811.0477: 2018.
Emma Strubell, Patrick Verga, Daniel Andor, David Weiss, and Andrew McCallum. Linguistically- informed self-attention for semantic role labeling. In Proceedings of the 2018 Conference Empirical Methods in Natural Language Processing, pp. 5027-5038, 2018.
Mariya Toneva and Leila Wehbe. Interpreting and improving natural- language processing (in machines) with natural language-processing (in the brain). In NeurIPS, 2019.
Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman, Glue: A multi-task benchmark and analysis platform for natural language understanding. In Proceedings of the 2018 EMNLP Workshop BlackboxNI.P: Analyzing and Interpreting Neural Networks for NLP, pp. 353-355. 2018.
Shaonan Wang. Jiajun Zhang, Haiyan Wang. Nan Lin, and Chengqing Zong. Fine-graine neural decoding with distributed word representations. Information Sciences, 507:256-272, 2020.
Alex Warstadt and Samuel R Bowman. Can neural networks acquire a structural bias from raw linguistic data? arXiv preprint arXiv:2007.06761, 2020.
Leila Wehbe, Ashish Vaswani, Kevin Knight, and Tom M. Mitchell. Aligning context-based statistical models of language with brain activity during reading. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 233-243, Doha, Qatar,
October 2014. Association for Computational Linguistics. URL http: //t:://wwww /ww. aclweb. org/ anthology//14-1D00
Frank Wilcoxon. Individual comparisons by ranking methods. In Breakthroughs in statistics, pp. 196-202. Springer, 1992.
Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoe Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, et al. Google's neural machine translation sys- tem: Bridging the gap between human and machine translation. arXiv preprint arXiv: 1609.0814,, 2016.
Yue Zhang, Rui Wang, and Luo Si. Syntax-enhanced self-attention-bant semantic role labeling. arXiv preprint arXiv: 1204, 2019.
13

---

Under review as a conference paper at ICLR 2021
A INJECTING STRUCTURE BY PREDICTING PARSE
One way to encode structural information from each of these formalisms into language model representations is to directly optimize for the prediction of the formalism graphs, i.c., parsing. For DM and UCCA, we use the HIT-SCIR parser Che al. (2019), the best performing parser from the MRP 2019 Shared Task. For UD, we use the Køpsala parser Hershcovich et al. (2020) from the EUD Shared Task, which largely based on HIT-SCIR one. Both are transition-ban which fine-tune BERT during training: BERT takes in sequence S of P wordpieces and outputs sequence contextualized token representations [h...... hp]. which the parsers use as embeddings, fine-tuning the BERT model. Our assumption is that these representations training to better capture the linguistic distinctions made by each formalism. After fine-tunie tine-tuning on each formalism's ' respective corpus, extract sentence and word representations for all fine-tuned models as described above. Each of the parsers' default hyperparameters are employed.
parsers,
fine-tuned during parser
Model
Epoch 0
Epoch 1
Epoch 2
Pereira et al. (2018)
DM
0.278
0.201
0.167
UD
0.277
0.186
0.159
UCCA
0.277
0.189
0.161
PRE
0.277
Wehbe et al. (2014)
DM
0.225
0.126
0.083
UD
0.225
0.092
0.065
UCCA
0.225
0.110
0.077
PRE
0.226
Table 1: Brain decoding scores (Pearson's r) for each the BERT models fine-tuned via parsing, and for the pretrained baseline (PRE). Note that the latter is not fine-tuned.
Results for the models fine-tuned via parsing show divergence in brain decoding performance. Indeed, we find that as parsing performance (as measured by unlabeled undirected attachment scores (UUAS)) improves on the held-out development set, brain decoding performance declines. This finding is congruent with the results of Gauthier & Levy (2019), which show that fine-tuning on GLUE tasks Wang et al.) (2018) leads to decline brain decoding performance, until a ceiling point where eventually stabilizes. In experiments, after epoch of fine-tuning, decoding performance is cquivalent to the one achieved by the pretrained model. However, with more fine-tuning, the models consistently diverge, as shown in Table [ These results averaged over fine-tuning Understanding the learning dynamics that lead to such divergence an interesting avenue for future work.
B MEAN/MEDIAN RANK RESULTS
Table shows results for the Pearson's r metric reported in the main paper, alongside the mean and median rank metrics reported in Gauthier & Levy) (2019), which give the rank of a ground-truth sen- tence representation in the list nearest neighbors of a predicted sentence representation, ordered by increasing cosine distance. This metric evaluates representations based on their support for contrasts between sentences/words which are relevant to the brain recordings. The table shows that the models which have higher Pearson scores, also have a lower average ground truth word/sentence nearest neighbour rank iee. induce representations that better support contrasts between sentences/words which are relevant to the brain recordings.
14

---

Under review as a conference paper at ICLR 2021
Model
Pearson's s
Mean rank
Median rank
Percira
(2018)
DE-B DM
0.269
33.58
12.95
DF-B UD
0.277
32.91
13.03
DF-B UCCA
0.259
37.05
15.09
GA DM
0.280
32.66
12.39
GA UD
0.286
30.79
11.44
GA UCCA
0.268
34.54
13.77
PRE
0.276
32.18
12.13
Webbe et al. (2014)
DE-B DM
0,204
493.11
89.32
DE-B
0,206
497.24
81.69
DE-B UCCA
0.164
689.89
227.30
GA DM
0,343
172.45
10.96
GA
0,280
255.127
18.28
GA UCCA
0.261
315.73
25.78
PRE
0.225
436.70
53.13
Table 2: Brain decoding scores as measured via three metrics - Pearson's r, Mean rank, and Median Rank for each of the domain-finetuned baseline (DF-B) models, the guided attention models (GA), and the pretrained (PRE) model.
C SIGNIFICANCE TESTING
Bootstrapping The bootstrapping procedure described below. For each subject of m subjects:
1. There are stimuli sentences, corresponding n fMRI recordings. A linear decoder is trained to map each recording to its corresponding LM-extracted (PRE, DF-B,GA) sentence representation. This is done using 12-fold validation. This yields predicted a 'sentence representation" per stimuli sentence.
2. To compensate for the small size the dataset which might lead a noise estimate of the linear decoder's performance, we randomly resample datapoints (with replacement) from the full n datapoints.
3. For each resampling, our evaluation metries (pearson's v, mean rank, etc.) are computed between the sampled predictions and their corresponding 'gold representations? for all sets LM reps. We store the mean metric value (c.g. pearson I score) across the n 'sampled' datapoints. We run 5000 such iterations.
4. This gives us 5000 such paired mean (across the n samples, that is) scores for all models.
5. When comparing two models, e.g. GA DM vs. PRE, to test our results for strength of evidence of generalization over stimuli, we compute the proportion of these 5000 paired samples where c.g. GA DM's mean sample score greater than PRE. After Bonferroni correction for multiple hypothesis testing. is the p-value report. See 3 for these per subject p-values for Pereira 2018. For Wehbe 2014, comparisons between each of the GA models and the pretrained baseline lead greater than the pretrained baseline's 's mean score for all 5000 sets of paired samples), for all subjects. We, therefore, do include similar table.
0.000 (i.e. The GA model mean score is
6. We average over these 5000 samples per subject, and use these m subject means for the across-subject significance testing, which is described below.
Strength of generalization across subjects To test results for strength of generalization across subjects, we apply the Wilcoxon signed rank test (Wilcoxon) 1992) to the m by-subject mean scores (see above), comparing the GA models to the pretrained baselines. Since the lowest p-value is 0.0078 (if every subject's 's difference score consistently favors the GA model over the baseline vice versa).
for both datasets,
In the case of Pereira 2018: for PRE vs. GA UD we get a paluue of 0.0078 (0.0234 after Bonferroni correction); for PRE GA DM we get p-vuuue of 0.015 (0.045 after Bonferroni correction); for
15

---

Under review as a conference paper at ICLR 2021
PRE GA UCCA get p-value of 0.0078 (0.0234 after Bonferroni correction, here PRE GA UCCA for all subjects).
In the of Wehbe 2014: all comparisons yield p-value of 0.0078 (0.045 after Bonferroni correction), where the GA model the pretrained baseline.
Pereira al. (2018)
Model/Subject
M02
M04
M07
M08
M09
M14
M15
P01
GA UD
0.000
0.110
0.011
0.021
0.000
0,009
0.000
0.039
GA DM
0.132
0.216
0.031
0.014
0.000
0.417
0.186
0.085
GA UCCA
0.014
0.015
0.052
0.041
0.452
0.002
0.000
0.003
Table 3:
p-values resulting from paired bootstrap test described above, for each of the three GA models when compared to the pretrained baseline.
D THE DOMAIN EFFECT
Table shows average word perplexity scores for the pretrained model and the domain-finetuned models for each of the three text domains on the stimuli from Pereira2018 and Wehbe2014. Scores are averaged over the words sentence and the sentences (stimuli) in the datasets.
Pereira et al. (2018)
PRE
14.09
DF-B DM
19.11
DF-B UD
19.08
DF-B UCCA
20.67
GA DM
20.82
GA UD
17.15
GA UCCA
17.47
Wehbe et al. (2014)
PRE
34.79
DF-B DM
36.11
DF-B UD
38.41
DE-B UCCA
40.45
GA DM 33.24
GA UD
37.16
GA UCCA
33.60
Table 4: Average word perplexity scores for each of the domain-finetume baseline (DF-B) models, the guided attention models (GA), and the pretrained (PRE) model.
E SEMANTIC TAGGING
Probing details Representations for the probing task are derived as described in 3.1 for each sentence the development and testing sets from AoziAnnianiddze & Bos (2017). The development set is employed training set, because it is mostly manually annotated/correcte. (as opposed to the much noisier training set) and because it is already possible to train rather accurate semantic taggers which suffice for analysis with training set of that size (131337 instances). We report results for the official test set. Table [ shows the frequency of each semantic category we report scores for the test set. An 1.2 regularised logistic regression model is utilised.
16

---

Under review as a conference paper at ICLR 2021
Pereira
04
Accurwcy score in synectii tesk
Negatwe RACEL
MAE ayort
98-
0.-1
AgrrE king mood
AL//WN NOTI
1.0
0.2
0.1
1004
16
06
10
18750
DECA
Formalism
Figure Targeted syntactic evaluation accuracy scores per category for Pereira2018 models.
Further discussion We observe the largest improvements for the DISCOURSE and TEMPORAL categories. The former involves identifying subordinate, coordinate, appositional, and contrast relations. These relations are highly influenced by context, and correctly classifying them can often be contingent on longer dependencies, which the structural bias increases 'awareness' of. The TEMPORAL category, on the other hand, consists of tags such as clockt ime or t imee of day which are applied multi-wond expressions, e.g 27th December. Highlighting these dependencies by assigning more weight to the attention between their sub sub-parts is likely helpful for their accurate identification.
Category Frequency
Attribute
63763
Unamed Entity
48654
Logical
32973
Named Entity
29271
Event
25338
Tense and Aspect
15208
Discourse 9948
Temporal 5652
Table 5: Semantic category frequency the test set.
F TARGETED SYNTACTIC EVALUATION SCORES
Figures 6 and 7 show the performance of the Pereira2018 and Wehbe2014 models and the four baselines for each of the syntactic categories from Marvin & Linzen 2019).
17

---

Under review as a conference paper at ICLR 2021
(2014)
Accuracy score
0-
Tcfor Aracher:
Ancruge
0-6-
04
1.0
00CA
04
UOCA
0H
00
UCCA
0M
UCCA
D9
00
UCCA
Figure 7: Targeted syntactic evaluation accuracy scores per category for Wehbe2014 models.
G CONTENT WORDS AND FUNCTION WORDS ANALYSIS
Figure 8 shows the breakdown of brain decoding accuracy by content and function words for Wehbe2014. We consider content words as words whose universal part-of-speech according spaCy is one of the following: {ADJ, ADV, NOUN, PROPN, VERB, X, NUMJ. Out of total of 4369, 2804 considered content words and 1835 as function words.
18

---

Under review as a conference paper at ICLR 2021
Content Words
Function Words
0.40
0.40
pretrained
finetuned
0.35
0.35
0.30
0.30
0.25
0.25
Pearson r
Poarson
0.15
0.15
0.10
0.10
0.05
0.05
0.00
0.00
DM
UD Formalism
UCCA
DM
UD Formalism
UCCA
Figure 8: Content word and function word brain decoding score (mean Pearson's 's r)) for all models fine- tuned by MLM with guided attention on each formalisms (points), as well the four baselines: pretrained BERT, dotted line), and the domain finetundd BERT by MLM on each formalism's 's training text without guided attention (solid lines).
19


Reviews:
[{"review_id": "XGYA484ZAxt", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary of paper: the authors explore adding a soft structural attention constraint to BERT, by penalizing attention weights that are substantially different from a head–dependent \"adjacency\" matrix derived from dependency parses.  BERT is then fine-tuned with and without (\"domain-finetuned\") this constraint on corpus data for which fMRI recordings from participants during reading are available. A linear classifier from the final layer of BERT's embedding (mean-pooled) is then learned to the fMRI data.  Within this pipeline, domain-finetuned models are not an improvement over unfinetuned BERT, but fine-tuning with the structural attention constraint improves decoding to fMRI data, especially for word-level data (the Wehbe2014 dataset).\n\nAssessment: this is a nice paper that investigates an intuitive method of incorporating syntax-based, structural soft attention constraints into Transformer encoder models for language.  What makes the contribution fairly distinctive is evaluation on alignment with human fMRI recordings during comprehension of the texts.  The results show improvements in decoding relative to baseline models that involve no fine-tuning and/or domain-adaptation fine-tuning alone (no structural attention constraints), especially for fMRI data that are recorded below the sentence level.  The authors also evaluate the effect of fine-tuning on targeted syntactic evaluations from Marvin & Linzen; the results here are not particularly conclusive.  Overall, this is a potentially solid, if not ground-breaking, contribution.  However, there are a number of technical questions that are left unclear in the submission, and some of the results are cause for some concern.  These concerns need to be addresed in order for the submission to be fully satisfactory.\n\nThe single biggest concern is the extraordinarily high word perplexity scores in Table 2 for Wehbe2014 -- which get much, much worse after fine-tuning.  It is important to understand what's going on here in order to make sense of the core potential contribution of the paper, because it's only in the Wehbe2014 dataset where there seem to be appreciable improvements in decoding performance.  I would guess that the high perplexity comes from poor prediction of the proper nouns in the Harry Potter book chapter.  Maybe there needs to be some amount of fine-tuning of the models to the domain of the test-set corpus.  Overall, the paper needs more clarity on why it is only the Wehbe2014 dataset where the perplexity is so high and the fine tuning affects decoding performance so much.\n\nAdditional technical questions:\n\n1) How is the split of a word into word pieces handled in the adjacency matrix representing word–word dependencies?\n\n2) How are the adjacency matrix and each head's attention weight matrix converted into a distribution for computing cross-entropy loss?  Are the entries normalized globally? By row? By column?\n\n3) What are the perplexities like for domain-finetuned (no structural attention constraint) BERT?  These are missing from Table 2 (Appendix B), but are potentially important in interpreting your results.\n\n4) What words are pooled over for the Wehbe2014 analyses -- the four words in the 2-second window?\n\n5) Section 4.1 reports that UD and DM finetuned models are significantly better in brain decoding than the un-finetuned baseline, at p<0.0001, but the 95% confidence intervals for subject scores look very different.  And the difference in mean decoding performance for DM finetuning is barely visible.  How are you computing your confidence intervals and your p-values?  Why are they so different, and how are you getting such high confidence in improvements over the unfinetuned baseline here?\n\n6) How do your results compare to those using the best fine-tuning methods from Gauthier & Levy (2019), which involve scrambling the input sentences?\n\n7) Given that in Wehbe2014 each fMRI image corresponds to four words, most of which probably contain both function and content words, how is the content/function word analysis defined and performed?\n\nAdditional comments:\n\n* the authors write that \"increase in perplexity roughly corresponds to lower brain decoding scores\", but this doesn't look consistent with Table 2 and Figure 3.  For Wehbe2014, UCCA data yield the worst decoding accuracy but yield better perplexity than DM data, which yield decoding accuracy only slightly worse than the UD data.  The monotonicity is cleaner for Pereira2018 but the overall differences in decoding performance are much smaller.\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "IKPgvd-G2u", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper tests whether fine-tuning large pre-trained language models\n(LMs) with structural information can increase the correlation between\nthese representations and the representations of brain activity\nmeasured while processing the same stimuli. The injection of the\nstructural information is done through fine-tuning of the pre-trained\nmodel by \"guided attention\", which makes use of binary relations\nbetween the words according to three different syntactic or semantic\nformalisms. The authors map the brain activity to each of the\nalternative LM representations via a regression model, and measure the\nalignment by using correlation between the predicted (from brain\nactivity) and actual output of the alternative models. The results\nshow that under certain conditions representations learned through\nguided attention aligns better with the representations of brain\nactivity.\n\nIn general the investigates an interesting question which may be\n(eventually) relevant to both understanding the way humans process\nlanguage, and possibly building better computational models. The\nmethod followed in the study is (mostly) sound, and the paper is\nwritten well.\n\nA potential issue with the method is the direction of the prediction\nin \"brain decoding\" regression (section 3.5). Authors predict the\nmodel representations from the \"brain representation\" (this seems to\nbe based on earlier studies, but I did not verify). In my opinion the\nreverse is more meaningful. Since the invariant quantity in this study\nis the representations coming from the brain imaging. This is\nimportant, because the success of the regression is not only about the\namount of information in the predictors, but also simplicity of the\ntask. Hence, an alteration of the model representations that\nsimplifies them may result in better predictions, and hence, higher\ncorrelations\n\nExcept the above, I have some (mostly minor) comments:\n\n- I would be happier with a bit more explicit discussion of the main\n  results. After reading the articles, I am still not sure what to\n  take away from the main experiments. The effect on two different\n  data sets (also means representations at different levels/units) are\n  quite different - not allowing a clear conclusion. Side issues\n  discussed (the effects of the use of different formalisms, the\n  effect of domain, particular syntactic patterns, content vs.\n  function words are also relatively brief and far from being\n  conclusive). I think a clearer discussion of the main results, and\n  investigation of reasons for the discrepancy between the data sets\n  would make the paper stronger.\n\n- It would help if the data is explained slightly better.\n  Particularly, it would make the paper more self contained if the\n  authors specify whether any of the data sets (section 3.3) had\n  automatic annotation. On a somewhat related note, comparisons\n  between the formalisms seem to correlate with the data sizes, which\n  is not pointed out in the paper.\n\n- A few language/typography issues/suggestions: \n\n    - I am not sure about the ICLR guidelines, but avoiding citations\n      in the abstract is a good idea (abstracts should stand alone).\n    - Footnote marks should go after punctuation (footnote mark 8)\n    - Conclusions line 3: \"attention guided\" -> \"guided attention\" ?\n    - There are case (normalization) issues in the references:\n      \"groningen\", \"erp\", \"bert\" (not exhaustive, a through check is\n      recommended).\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "ErZ-t3zCjqu", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "An interesting paper that discusses whether injecting three types of syntactic and semantic formalisms lead to better alignment with how language is processed in the brain. The authors conduct experiments with the BERT model and two fMRI datasets and show that including linguistic structure through fine-tuning can improve brain decoding performance.\n\nThe paper would be improved by experimenting with language models other than BERT, as it is not clear at the moment whether the produced results are generalizable to different language models or are BERT-specific. For example, additional experiments with AlBert, distilBert and RoBerta would provide additional insights on the effect of size of the model, in terms of the number of parameters. Comparison of Bert to GPT and XLNet would emphasize the advantages/disadvantages of autoencoder-based vs autoregressive models and could potentially provide additional insight on how attention is represented in human brain.\n\nIt would also be interesting to read a discussion of semantic analysis, as currently the paper concentrates the most on syntactic formalism as represented in both BERT and fMRI data. Specifically, it would be interesting to know if the injection of syntax impacts the semantic representations. One of the possible methods to measure that would be probing for semantic classes (as in Yaghoobzadeh et al., 2019. Probing for Semantic Classes)\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "Dp9vC7JXi1Y", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper describes experiments that inject linguistic information (for example dependency structures) into BERT, then measure improvements in correlation with FMRI measurements of humans reading an underlying sentence (which is also analyzed by BERT). Linguistic information is incorporated by biasing attention heads to line up with dependency (or other) structures. \n\nPositives about the paper: it's an interesting experiment to try, and an important direction of work.\n\nNegatives:\n\n* It's a somewhat small increment over previous work, not sure it merits a full conference paper. As it stands the paper presents the approach and results, with little inspection of why improvements are seen. I would like the authors to go much deeper with the analysis. Are there particular syntactic constructions that are being better modeled? Is the new model much more sensitive to long range dependencies, as found in syntactic structures? Are particular classes of words effected more than others? Answering these questions will be challenging but would add a lot to the paper.\n\n* Most importantly, the evaluation metrics are unclear. The critical sentence in the paper is \"To evaluate the regression models, Pearson’s correlation coefficient between the predicted and the corresponding heldout true sentence or word representations is computed\". This is a terse description of a critical part of the approach, and I can't make sense of it.\n\nPart of my unease about the evaluation is the following. The matrix $D_{fr}$ is the output from BERT. Importantly in The definition of L_{ifr} this matrix is predicted from the \"brain\" matrix B_i. If $D_{fr}$ was all zeros it would be trivially predictable (and hence gameable). In the original Gauthier and Levy paper they appear to use metrics in addition to MSE. In this paper some variant of Pearson's correlation coefficient is used - but I can't understand what exactly this is, and my worry is that it is trivially gameable in the same way as MSE.  ", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper explores the effect on decoding accuracy (predicting hidden representations from fMRI datasets) from fine tuning models by injecting structural bias.  This paper specifically focuses the attention of BERT on syntactic features of the text, which (for one dataset) appears to improve the decoding performance.  The paper's motivation is strong, and complex concepts are communicated clearly.

The review period was very productive.  There were some questions about analyses, and the validity of the statistical tests, but through some very thorough back and forth with the reviewers, this seems to have been resolved.  There is a good amount of analysis done on the resulting language models to try and determine the impact of finetuning or attention on the models. However, the results on the fMRI two datasets appear to be very different, and it's unclear why (and isn't clearly related back to the extensive language model analyses).  We would have liked to have seen a more thorough analysis of the stark difference in performance, and some convincing explanations for the difference based on the analyses.  


P.s. A minor point, but the Wehbe paper uses Chapter 9 of Harry potter, not chapter 2.

Decision: Reject