Training language models to summarize narratives improves brain alignment
Authors: Khai Loong Aw
Mariya Toneva
Venue: ICLR 2023 notable top 25% (2023)
Keywords: language
nlp
neuroscience
fMRI
interpretability

Abstract: Building systems that achieve a deeper understanding of language is one of the central goals of natural language processing (NLP). Towards this goal, recent works have begun to train language models on narrative datasets which require extracting the most critical information by integrating across long contexts. However, it is still an open question whether these models are learning a deeper understanding of the text, or if the models are simply learning a heuristic to complete the task. This work investigates this further by turning to the one language processing system that truly understands complex language: the human brain. We show that training language models for deeper narrative understanding results in richer representations that have improved alignment to human brain activity. We further find that the improvements in brain alignment are larger for character names than for other discourse features, which indicates that these models are learning important narrative elements. Taken together, these results suggest that this type of training can indeed lead to deeper language understanding. These findings have consequences both for cognitive neuroscience by revealing some of the significant factors behind brain-NLP alignment, and for NLP by highlighting that understanding of long-range context can be improved beyond language modeling.

Published as a conference paper at ICLR 2023
TRAINING LANGUAGE MODELS TO SUMMARIZE NARRATIVES IMPROVES BRAIN ALIGNMENT
Khai Loong Aw1.2 Mariya Toneva!
Max Planck Institute for Software Systems Singaapoe Management University klaw. 2020@ smu. eduu sg mtoneva(mppi sws. org
ABSTRACT
Building systems that achieve a deeper understanding of language is one of the central goals of natural language processing (NIP). Towards this goal, recent works have begun to train language models on narrative datasets which require extracting the most critical information by integrating across long contexts. How- ever, it is still an open question whether these models are learning deeper under- standing of the text, or if the models are simply learning a heuristic to complete the task. This work investigates this further by turning to the one language process- ing system that truly understands complex language: the human brain. We show that training language models for deeper narrative understanding results in richer representations that have improved alignment to human brain activity. We further find that the improvements in brain alignment are larger for character names than for other discourse features, which indicates that these models are learning im- portant narrative elements. Taken together, these results suggest that this type of training can indeed lead to deeper language understanding. These findings have consequences both for cognitive neuroscience by revealing some of the significant factors behind brain-NLP alignment, and for NLP by highlighting that understand- ing of long-range context can be improved beyond language modeling.
INTRODUCTION
Language models trained to predict the next word over millions of text documents have led to large improvements range of benchmarks in natural language processing (NLP). However, re- searchers have shown that NLP models may rely on shallow heuristics to perform these tasks rather than on deeper language understanding (McCoy al. 2019, Min et al. 2020 Linzen 2020). To build systems with deeper language understanding, recent work proposed to train language models on narrative datasets which require extracting the most critical information by integrating across long contexts (Kryscinski et al. 2021 [aang et al.) 2022 Kočiský et al.) 2018). Does this approach truly lead to deeper understanding language? We investigate this by turning to the one language processing system that truly understands complex language: the human brain.
Prior work has used human brain recordings to interpret representations of pretrained language mod- els (LMs) (Søgaard) 2016. Toneva & Wehbe 2019: Abdou et al.) 2021). They evaluate how well representations obtained from a language model can predict representations sampled from the hu- man brain during language comprehension via a brain imaging device, such functional magnetic resonance imaging (IMRI). If this prediction performance, also known as brain alignment, is deter- mined to be significant via statistical test, then the LM and specific brain location timepoint thought to significantly align in their representations of language. Using these methods, researchers have shown that pretrained language models significantly predict large parts of the brain regions that are thought to underlie language comprehension (Wehbe el al. 2014b Jain & Huth 2018 Toneva & Wehbe 2019 Caucheteux & King 2022 Schrimpf et al. 2021 Goldstein et al.) (022).
In this work, we draw insights from the human brain to study whether language models are truly learning deeper language understanding. We analyze the effect of training LMs for narrative sum- marization on their alignment with fMRI recordings of human subjects reading a book chapter.
Code available /tttts:///ggt wwt.:ttwwb..omaacoo awwk /brain language summa r zat on

---

Published as a conference paper at ICLR 2023
We specifically investigate 4 pretrained language models (i.e., "base models") and 4 corresponding models obtained by training the base models on the BookSum dataset (Kryscinski et al. 2021) to improve the base language model's 's narrative understanding (i.e., "booksum models"), ). The Book- Sum dataset was selected because is a summarization dataset that requires understanding complex interactions across long narratives. The 4 models were selected because their architectures were designed to integrate information across long contexts. We evaluate the alignment of the base and booksum models with IMRI recordings of participants reading a chapter of popular book word- by-word, made publicly available by Wehbe et al.] (2014a). This dataset was chosen because it is one of the largest datasets of participants processing narrative story (5176 words which corresponds to approximately 1300 samples of fMRI recordings per participant).
Our main contributions are as follows:
1. In Section we show that training language models for deeper narrative understanding improves alignment to human brain activity. Also, when increasing the number of words fed to the models, up 500 words, brain alignment increases. Lastly, for each model, we identify the layers where these improvements in brain alignment occur.
2. In Section 5 we show that improved brain alignment in Section I is not due to improved language modeling (LM) ability, possible confounding factor. By disentangling LM abil- ity's s contribution to brain alignment, we present evidence that BookSum-trrinne models develop deeper language understanding.
3. In Section 6 we present a simple interpretability approach to study what brain-relevant information gained by language models after training for deeper language understand- ing. Our results reveal that these models are learning richer representations across all tested discourse features (Characters, Emotions, Motions). Furthermore, they learn more about Characters than Emotions and Motions. This indicates that discourse features are a promis- ing dimension study brain alignment and deep language understanding.
Combined, our contributions from Sections 4 5 and present evidence that models trained to sum- marize narratives indeed develop deeper language understanding. The first reason that improved alignment to human brains' deep understanding of characters, emotions and motions suggests the model has developed richer representations of these entities and concepts. Second, we focus on brain regions suggested by previous research to underlie language comprehension in humans. Hence, im- proved brain alignment is not spuriously related to non-language brain activities. Third, we show that brain alignment improves only when provide longer input contexts (20 to 1000 words) to the LMs, which may be important for deep contextual understanding
2 RELATED WORK ON BRAINS AND LANGUAGE
Our work relates to a growing body of research on disentangling the contributions of different types of information towards brain aligmment. Toneva et al. (2022a) present an approach to disentangle supra-word meaning from lexical meaning in language models (LMs) and show that supra-word meaning is predictive of fMRI recordings in two language regions (anterior and posterior temporal lobes). Caucheteux et al.) (021) and Reddy & Wehbe (202T) disentangle alignment due to syntactic and semantic processing. Toneva al. (2022b) examine if LM representations align with different language processing regions in different ways. Also, researchers have suggested that one contributor the alignment is the LM's ability to predict the next word, with positive relationship between next-word prediction ability and brain alignment across LMs (Schrimpf et al. 2021 Goldstein et al., 2022). However, more recent work shows that no simple relationship exists, and language modeling loss not perfect predictor of brain alignment (Pasquiou et al.) 2022 Antonello & Huth 2022)
Merlin& Toneva (2022) introduced perturbations to disentangle alignment due to next word pre-
diction and semantic knowledge. Our work contributes to this research area by disentangling the contributions of LM ability from deep language understanding towards brain-NLP alignment.
Some works investigated the alignment of fine-tuned language models with brain recordings. Schwartz et al. (2019) finetuned a pretrained BERT to predict fMRI and MEG recordings of peo- ple reading a book chapter, leading to improved prediction of previously unseen brain recordings, specifically in regions known to support language processing. However, it is not clear what informa- tion had been induced in the fine-tuned BERT model that contributed to improved brain alignment.
2

---

Published as a conference paper at ICLR 2023
Oota et al. 2022) investigate brain alignment of BERT fine-tuned variety of NLP tasks, such They show the best aligning task-tuned BERT depends whether the brain recordings correspond to reading or listening to the language stimuli. Our approach complements these works, but focuses on LMs trained to summarize narratives.
as co-reference resolution and
Additionally, prior work LM brain alignment LMs specifically designed integrate rich info across long contexts. Thus, they investigated relatively short contexts and observed that brain alignment worsened with increased sequence length for many models (Toneva & Wehbe (019). Differently, our work uses LMs trained to extract crucial info from long contexts. Brain alignment remains high even with sequence lengths up to 500 words, a finding that adds to prior works.
METHODS
3.1 BRAIN REPRESENTATIONS
We use publicly available brain dataset (Wehbe et al., 2014a) consisting of fMRI recordings of 8 participants reading chapter of the book Harry Potter and the Sorceror's Stone (Rowling et al.] 1998). The 5176 words in the chapter were presented to participants at fixed interval of 0.5 seconds, one word a time. The chapter was divided into four runs of approximately equal length, and participants were allowed short break at the end of each run. We use this specific dataset for two main reasons: (l) the story situated within a rich narrative world with characters, emotions and relationships. and (2) it is one of the largest publicly available datasets in terms of samples per participant, which improves the reliability of the brain aligmment estimate.
sampled at consistent interval, repetition time (TR). of seconds. At each TR, the activity level was recorded in each volume pixel (i.c., voxel) in the participant's 's brain. Hence, the brain representation for each participant is matrix Y; E R where is the number of TRs and iú is the number of voxels in the brain of participant i. The number of voxels differs for each human subject, ranging from 25, 000 31, 000.
The fMRI recordings
3.2 NLP REPRESENTATIONS
We extract NLP representations from total of 8 models. We refer to 4 models as "base models": BART-base, LED-base, BigBird-base and LongTT5-baee. We refer to the remaining models as "booksum models"; BART-booksum, LED-booksum, BigBird-booksun and LongT5-booksin
Base models. Our "base models" were trained with a language modeling (LM) objective. We use pretrained models from Hugging Face (Wolf et al. 2020). We provide more details in Appendix
BART (Lewis et al. 2019) uses a standard Transformer- based sequence-to- to-sequence architecture with 12 layers and dimedinma representations. can process sequence lengths up 1024 tokens. BART was pretrained by corrupting documents and learning to reconstruct the original text, on combination of books (narratives) and Wikipedia data.
BigBird (Zaheer et al. 2020) representations. It uses a sparse-attention mechanism that allows it to process sequence lengths up to 4096 tokens. We the version of BigBird that was pretrained with Masked Language Modeling on books and summarization of U.S. patent documents (BigPatent dataset).
Transformer-baser model with 32 layers and (024-dimensional
Longformer Encoder Decoder (LED) (Beltagy al., 2020 has an encoder-decoder Transformer architecture with 12 layers and 768-dimensional representations. It uses efficient local+global attention mechanism based on the Longformer, and can process up to 16384 tokens. LED was pretrained with the same procedure as BART above.
LongTS (Goo et al. 2021 is an encoder-decoder Transformer with 12 layers and 768-dimensional representations. can handle long input sequences up 16384 tokens. We use the version of LongT5 with Transient Global Attention (TGlobal). LongT5 was pretrained by masking and generating sentences on the C4 dataset, a collection of roughly 750 GB of English texts sourced from the public Common Crawl open repository.
Booksum models. Our 4 "booksum models" are first initialized from one of the "base models". All their parameters are then trained using the BookSum dataset (Kryscinski al. 2021). We describe
3

---

Published as a conference paper at ICLR 2023
the BookSum dataset in the paragraph below. We chose base and booksum models that would be useful for investigating our hypothesis. (1) Their architectures are designed to take advantage of long contexts, which may be useful for deep understanding. (2) The booksum models have good performance on BookSum, and the base models have good performance on other long-context benchmarks. (3) The base models, except LongT', were pretrained with language modeling on large narrative datasets, which reduces the differences in training domains between the base and booksum models, to allow for more fair comparisons. (4) Our models are sufficiently varied, and we study pairs of models in total, which improves reliability of our results.
BookSum dataset for long-range summarization. The BookSum dataset (Kryscinski et al. 2021) is a challenging dataset for summarizing long narratives. For training, the inputs are chapters from books and the outputs are human-written summaries. The input documents are relatively long, with an average of more than 5000 words, and contain long-range causal and temporal dependen- cies. The ground-truth output summaries have an average of roughly 500 words. To generate these summaries, NLP models must capture the most critical information by understanding complex in- teractions across long-range contexts. Some examples from the BookSum dataset are provided in Appendix B Furthermore, the dataset belongs to the narrative domain, which captures rich dis- course structures for characters, emotions, and relationships, which is also the case for the Harry Potter stimulus dataset that we include any books from the Harry Potter series, thus preventing train-test overlap.
evaluate brain alignment. The BookSum Dataset also does not
Extracting NLP representations. Our goal is to extract NLP representations for each of the 5176 words in our Harry Potter IMRI dataset. For each word, we feed the word and its preceding context to model, and extract its outputs at every layer for the word. We extract NLP representations for all layers of all 8 NLP models and vary the input sequence lengths between I to 1000 words. Each unique combination of (model, layer, input length) produces a different set of NLP representations, X E 1l167xdd where d is the embedding size for model m, layer I, sequence length s.
3.3 ALIGNING BRAIN AND NLP REPRESENTATIONS.
We follow a general alignment approach previously used in several works (Jain & Huth| 2018 Toneva & Wehbe 2019 Schrimpf et al. 2021). This approach learns function to predict the fMRI recordings at every voxel of each participant 2 using the NI.P representations that correspond to the same text read by the participant. Similarly to previous work, we parameterize this function linear function, regularized using the ridge penalty. We train this function in a cross-validated way and test its performance the data that was heldout during training. We select the ridge parameter via nested cross voossvvalidttion. Since the fMRI data was collected in 4 runs of approximately equal length, we use 4-fold cross-validation where each fold corresponds to holding out one run of fMRI data for testing. We also remove 10TRs of fMRI data and the corresponding NLP representations from the ends of each test run to avoid possible train-test overlap. We provide other details about the alignment approach in Appendix C
Evaluate brain alignment. We evaluate brain alignment using two evaluation metrics, computed between the predictions of heldout fMRI recordings Y; E R=xx and the true corresponding fMRI data Y; E paxi, Both evaluation metrics are used by prior works and each has its own advantages.
20v20 classification accuracy. This metric evaluates the 1MRI predictions by using them in a classification task on held-out data in the cross-validation setting. The classification task is to pre- dict which of two sets of words was being read, based on the corresponding NLP representations of these words (Mitchell et al. 2008 Wehbe et al. 2014b). The sets consist of 20 consecutive TRs and the classification is repeated 1000 times within each fold. The final 20v20 accuracy the average across the folds. This metric was proposed to boost the signal-to-noke noise ratio in estimating the brain alignment for single-trial data. The Harry Potter data is entirely single-trial (i.e., the participants read the book chapter only once and so each word appears only once in its context) and increasing the number of TRs that are classified together up to 20 improves the classification accuracy for this dataset (Wehbe et al., 2014a|bb b).
Pearson correlation. This metric evaluates the similarity between the fMRI predictions and the corresponding true fMRI data by computing the Pearson correlation between Hi, and yi, for participant and each voxel (1,...u:)- This produces vector R This metric can be
4

---

Published as a conference paper at ICLR 2023
layer
layer 10
bart
20v20 30/20 accuracy
55
BigBird
led
50
Dase
base booksum
........ booksum
layer
layer
bigbird
60
2Dv20 accuracy
long-t5
54
56
58
60
20v20 accuracy
600 Sequence longth
400
800
Sequence length
Figure 1: Left. Average 20v20 brain alignment for all 4 pairs base and booksum models. Aver- ages were computed over layers for each model, sequence lengths 1 to 1000, and all 8 participants. See Appendix G for similar Pearson correlation plots. Right. Brain alignment for BigBird-base and BigBird-booksum, across representative layers and sequence lengths. Each data point is an average across 8 participants. Training for deeper understanding improves brain alignment significantly in early to middle layers (layers 6- 6-14 out 32) (paired t-test, FDR corrected for multiple comparisons at significance level 0.05). Results for the remaining models (BART, LED, LongT5) also improve significantly, presented in Appendix H These results suggest that long-range and deep understand- ing is an important contributor to brain alignment.
computed efficiently and is widely used in cognitive neuroscience (Huth et al. 2016 Jain & Huth
2018 Caucheteux al. 2021: Schrimpf al. 2021 Goldstein al. 2022
TRAINING NLP MODELS FOR DEEP NARRATIVE UNDERSTANDING
Do NLP models trained for deeper understanding of long narratives have improved brain alignment over those trained for language modeling (L.M)? To investigate this, we evaluate the 20v20 brain alignment scores
booksum models (deeper understanding) against 4 base models (LMM,, as described in Section 33 We use all 8 NLP models, layers 1 to 12, sequence lengths I to 1000, and all 8 human subjects. In Figure !
we present average results for all 4 pairs of models, as well as results for representative set of layers for one of the models. Other results are in Appendix G and H
Booksum models are significantly better aligned to brain representations. We suggest that this due to deeper understanding of text, because of two key facts. (1) Booksum models were trained on the BookSum dataset, which designed with challenging tasks to learn deeper narrative under- standing. It requires understanding complex interactions across long-range contexts and generating summaries that capture the most critical information. (2) Furthermore, these models have architec- tures designed integrate information across long context lengths. This allowed them to develop deeper understanding when trained on BookSum.
Could the improved brain alignment of booksum models be due other confounding variables and not deeper language understanding? We tried to eliminate possible confounding variables in the following ways: (1) Both base and booksum models were trained with similar data: large narrative datasets. Hence, improved brain alignment is not simply because booksum models were trained on data from more similar domain the brain alignment dataset (Harry Potter). We also verified this by comparing booksum models against base models trained on BookSum with language modeling objective in Appendix M (2) The booksum models and their corresponding base versions support identical sequence lengths, and were also trained with similar input lengths: all the base models were trained with long input sequences between 1024 and 4096 tokens. Hence, improved brain alignment not because booksum models were trained longer input lengths. (3) In Section 5 show that training NLP models for deep understanding improves brain alignment, but does not improve LM ability. Hence, improved brain alignment is not because booksum models developed better
5

---

Published as a conference paper at ICLR 2023
language modeling ability through BookSum-training (4) In Appendix D show that the brain alignment of BART trained to summarize CNN news articles (BART-cnn) is not substantially higher than BART-base. Hence, the improved brain alignment is not simply due to the summarization training objective. Therefore, we argue that improved brain alignment for booksum models is likely not due to these confounding variables.
Brain alignment across models peaks around context length of 500 words. We further studied how brain alignment is affected by input context length to the models, which we varied from 1 to 1000 words. If brain alignment continues to increase as we increase context length, it would suggest that these models can indeed maintain information across long contexts. Prior work that used earlier language models such as BERT and Transformer-Xi found that brain alignment peaked around 50 words (Toneva & Wehbe 2019). We find 10-fold increase the sequence length that results in the peak of brain alignment, across both the hase and booksum models. Because this peak is consistent for both types of models, we attribute the 10-fold increase to the fact that the models we evaluate in this work are specifically designed to integrate longer context than previous models. However, we do not observe an additional benefit beyond 500 words of context, even though the models we analyze are supposed to incorporate more than 500 words of context (up to 16384 tokens for LED and LongTS).
Brain alignment improves only for longer input contexts (20 to 1000 words), but not short contexts (1 to 5 words). Importantly, this suggests: (1) improved brain alignment is likely due to deeper language understanding (which benefits from longer contexts), and (2) these models trained for deeper understanding have developed better ability to make use of long contextual information.
Booksum finetuning improves brain alignment for different layers across models. Finetuning has been shown to result in greater changes in deeper layers than earlier layers of models (Mosbach ct al.) 2020). Therefore one might expect that changes in brain alignment due to finetuning may emerge primarily in the deeper layers. However, we find that the brain alignment is significantly improved at different layer depths across the 4 models that we test. BigBird and LED improve their alignment with the brain significantly in early to mid layers (layers 6- 14 out of 32 in BigBird, and layers 2-8 out of 12 in LED), whereas LongT5 shows significant increases only in the deep layers (layers 8-12 out of 12). BART shows consistent significant difference throughout most of its layers (layers 2-11 out of 12). These results suggest that the belinntelevant changes may be distributed differently across different models.
RELATIONSHIP BETWEEN BRAIN ALIGNMENT, LANGUAGE MODELING AND DEEP UNDERSTANDING
Why do language models trained on BookSum have improved brain alignment, as our results show in Section 4 Is it due to the models developing deeper language understanding, or are there other confounding factors? One possible confounding factor is language modeling (LM) ability, as some works suggest that LM ability contributes brain-NIP alignment (Schrimpf et al., 202T Caucheteux & King 2022; Goldstein et al. 2022). This raises the possibility that booksum mod- els have greater brain aligument simply because they developed better language modeling ability through BookSum BookSum-training, We conduct experiments to further investigate this possibility.
Our goal is show that improved brain alignment is related deeper understanding, rather than improved LM ability. We show this by demonstrating that training LM models for deep understand- ing (booksum models) improves brain alignment, but simultaneously does not improve LM ability for 3 of the 4 models.
We evaluate the LM ability of all NLP models (4 base, 4 booksum) on the text corresponding to the same Harry Potter chapter which was used to obtain the brain recordings. First, we split our Harry Potter text dataset into a train and test set (75% and 25% of the text respectively). Next, for each model, we freeze all layers, add LM head, and train only the head to predict masked words in the train set. Finally, we evaluate each model's 's cross-entropy loss (LM ability) on the heldout test set. We froze all model weights (other than LM head) to ensure that the representations are preserved, identical to the ones used to evaluate brain alignment. We used the Harry Potter text dataset ensure that the same dataset is used for measuring both brain alignment and LM ability, to

---

Published as a conference paper at ICLR 2023
Cross-entropy Loss for various NLP models
BigBird
all
BART
PostTemp
AntTemp
AngularG
BigBird
IFG
MFG
LED
iFoorb
pCingulate
LongT5
base
dmpfc
base
booksum
Non-language
booksum
Cross-entropy Loss (lower is better)
20vs20 accuracy
Figure 2: Left. Language modeling (LM) loss (lower is better) on Harry Potter dataset. Booksum model has worse LM ability than base model for BigBird and LED, better for LongT's, and no sig- nificant difference for BART. Hence, while all 4 booksum models have better brain alignment, most do not have better LM ability. Thus, improved brain alignment booksum models (Figure [ is due to deeper understanding, and not language modeling ability. Right. 20v20 accuracy is significantly higher for the model trained for deeper understanding (booksum) than LM model (base), across all brain language regions (paired t-test, FDR corrected for multiple comparisons). Averages were computed over 8 layers for each model, sequence lengths to 1000, and all 8 participants. Similar results were obtained for all other models (BART, LED, LongT5) (Appendix []. This suggests that the function of deep understanding is not localized to any one particular region in the brain, but is instead distributed across language regions.
allow for consistent comparison. We perform 4-fold cross-validation to improve reliability, where each fold corresponds to consecutive 25% of the text. The results are presented in Figure 2
Improved brain alignment is not due to better language modeling (LM) ability. For BigBird and LED, the booksum model achieves significantly higher brain alignment but significantly worse LM performance than the base model. For BART, the booksum model achieves greater brain alignment but does not significantly differ in LM performance. For LongT5, the booksum model has both higher brain and LM performance. Hence, while all 4 booksum models have significantly better brain alignment, the majority of them do not have better LM performance. This shows that the improved brain alignment for at least 3 of the 4 models is due to deeper understanding and not to their ability predict the next word.
How does brain alignment differ across brain regions? We specifically investigate alignment within the brain regions in the language network Fedorenko al. (2010); Fedorenko & Thompson-Soon Schhll (2014), which are thought to underlie language comprehension, and two additional language re- gions Binder et al. (2009), which are thought support word semantics. For each brain region, we compare the average brain alignment for booksum versus base models. (Figure and Appendix [].
Across considered regions, brain alignment is significantly greater for booksum models. This suggests that brain-relevant information imparted by training towards deep understanding is not localized to any one particular brain region, but instead distributed across language regions. Furthermore, this shows that models with significantly better language modeling performance (base > booksum for BigBird and LED) do not achieve better alignment in any language region. This further supports the need complement language modeling with training for deep understanding.
INTERPRETING BRAIN ALIGNMENT ACROSS DISCOURSE FEATURES
What are NLP models learning when trained to summarize narratives? Are they learning deeper understanding of important discourse features in the text, such as Characters, Emotions, and Mo- tions? These discourse features have representations that go beyond the meaning of their individual words. Characters develop over the course of the entire book through their thoughts, actions, and interactions with other characters. Emotions depend who is experiencing them, what situation
7

---

Published as a conference paper at ICLR 2023
BigBird
Algorithm 1 Interpretability approach to compare brain alignment across discourse features
0.014
Characters
Input: fMRI predictions, fMRI activity
0.023
2: for all discourse features do
3: label words with discourse feature
0.012
Emotion
4: map labeled words to fMRI TRs
0.021
5: extract NLP predictions for discourse TRs
6: extract brain activity for discourse TRs
7: compute Pearson corr. for discourse TRs
Motion
0.007
booksum
8: Result: Pearson corr. for discourse feature
9: end for
-0.01
0.00
0.01
0.02
0.03
10: Result: Pearson corr. for all discourse features
Pearson correlation
Figure 3: Left. Interpretability approach to compare Pearson correlation brain alignment for fMRI samples corresponding to various discourse features. Right. Pearson correlation averages for three discourse features. Averages were computed over 8 layers for each model, sequence lengths 20 to 500, and all 8 participants. NLP models have greater brain alignment for Characters than other discourse features. When trained to summarize narratives, the models improve their brain align- ment significantly for all discourse features (paired t-test, FDR corrected for multiple comparisons). However, improves more for Characters than other discourse features. Note that the average cor- relations shown here are low in magnitude they include large number of brain voxels that may not be significantly involved in brain-NLP alignment or language processing, as well as many layers and sequence lengths.
they in, and why. Motions (actions) are situated in scene, contextualized by the actor, target, tionale of the action, and other people and objects in the environment. Hence, richer representations of these discourse features suggest deeper linguistic understanding of the text.
Our simple interpretability approach to compare brain alignment across discourse features. For each discourse feature, we compute an average Pearson correlation for booksum models, and another for base models. Different discourse features may be present at different frequency in the text. Hence, enable fair comparisons, we use the same number of samples to compute the corre- lation for each discourse feature. We present the results and the approach's 's pseudocode in Figure 3 Averages were computed over 8 layers for each the 8 models, sequence lengths 20 500, and 8 participants. We provide more details in Appendix D
NLP models align more with the brain for Characters than other discourse features, and also improve the most for Characters when training to summarize narratives. For Characters, Pear- son correlation averaged across all voxels improves significantly more than other discourse features for most models ((parred t-test, FDR corrected for multiple comparisons; Appendix []] Note that the correlations reported in Figure 3 are low in magnitude because they are an average that includes large number of brain voxels that may not be significantly involved in brain-NI.P alignment language comprehension, as well as over many layers and sequence lengths. Next, we compare the brain plots in rows D and E of Figure Character TRs improve in more voxels (orange regions) than the random TR sample. It is also surprising that booksum- booksum-training actually results in significantly worse alignment for some brain voxels (blue regions). One possible reason is that these specific re- gions may participate in prediction of future words. As we saw in Section the language modeling performance of BigBird declined due SookkSStttaining
Discourse features are promising dimension to study brain alignment. Training LMs to sum- marize narratives improved brain alignment (richer representations) for all discourse features. Our work contributes towards understanding what LMs learn when trained for deeper understanding. We show that brain alignment is higher for Characters than other discourse features. We hypothesize that, for Characters, language models integrate richer info from surrounding context, producing rep- resentations with greater alignment to brain representations. On the other hand, LM representations for other discourse features may be more shallow and less context-dependent. In future, we hope to dive deep into investigating this hypothesis, ultimately generating insights for a better understanding of NLP representations. Also, future work can explore additional discourse features.
8

---

Published as a conference paper at ICLR 2023
AG
A) Language ROls
ATL
DMPEC
booksum
0.6
B) Character TRs
C) Random sample TRs
0.0
booksum base
+0.2
D) Character TRs
E) Random sample TRs
-0.2
Figure 4: Pearson correlation averages between fMRI predictions from BigBird and true fMRI activity for one representative participant. Averages were computed over layers for each model, and sequence lengths 20 to 500. (A) Visualization of regions of interest (ROIs) thought to underlie language comprehension. (B) Pearson correlations for booksum models for Character TRs. (C) Same as B, but for equal number of TRs sampled randomly from all TRs. (D) Difference in Pearson correlations between booksum and base models, for Character TRs. (E) Same as D, but for the random sample. Comparing row B and C, there is greater brain alignment for Character TRs than for the random sample, both within and outside the language ROIs. Comparing row and E, Character TRs improve in more voxels than the random sample TRs. Only voxels with Pearson correlation significantly greater than O are plotted (one-sample t-test, FDR corrected for multiple comparisons significance level 0.05). Similar results for other participants Appendix K
7 CONCLUSIONS AND FUTURE WORK
Conclusions for brain-NLP interdisciplinary field. How and why do brain and NLP representa- tions align? This work contributes the following findings towards this question. (1) Understanding of characters and other discourse fcatures is a significant factor in brain-NLP alignment. (2) Deeper understanding of text contributes brain-NLP alignment independently from language modeling (LM) ability. Overall, our work helps to fill a knowledge gap in the brain-NLP field.
Conclusions for NLP. Language modeling (LM) has achieved great success and popularity. How- ever, is LM sufficient to develop NLP models with deep language understanding? We present unique approach to answer this question by drawing insights from human brains. Our results reveal that: (1) LM models achieve poorer representations for Characters and other discourse features com- pared deeper understanding models. (2) Existing training methods for narrative understanding are first step towards developing language models with deep language understanding. Using results from brain-NLP aligmment, we argue for the need to train NLP models with deeper understanding of long-range context, going beyond language modeling.
Future work. There are multiple exciting avenues to build on our work. (1) We showed that brain alignment differs across discourse features. Future work investigate why these differences exist. (2) We showed that deeper understanding in NLP models improves brain alignment. Future work explore the mechanisms behind deeper understanding of texts in NL. models.

---

Published as a conference paper at ICLR 2023
8 ACKNOWLEDGMENTS
We wish to thank Gabriele Merlin for valuable comments and discussions.
REPRODUCIBILITY STATEMENT
Code available at: In this repository, we provide the following: (1) Code used to reproduce the results in our paper. (2) README explaining how to install packages, preprocess data, run experiments, and plot visu- alizations of results. (3) our paper, we rum large number of experiments (many models, layers, sequence lengths, subjects, discourse features, brain ROIs, etc). Hence, we provide scripts to auto- mate the process of running experiments across the various models, layers, etc. We hope this will make it easy for others to use our code efficiently. (4) After running experiments to obtain data, you can also use our code for plotting the figures our paper, as well as additional graphs to visualize results, We hope this will allow others to use our code for future projects.
REFERENCES
Mostafa Abdou, Ana Valeria González, Mariya Toneva, Daniel Hershcovich, and Anders Søgaard. Does injecting linguistic structure into language models lead to better alignment with brain record- 2021.
ings? arXiv preprint
Richard Antonello and Alexander Huth. Predictive coding just feature discovery? an alternative account of why language models fit brain data. Neurobiology of Language, pp. 1-16, 2022.
Iz Beltagy, Matthew E Peters, and Arman Cohan. Longformer: The long-document transformer. arXiv preprint urXiv:2004.05/500 2020.
Jeffrey R Binder, Rutvik H Desai, William W Graves, and Lisa Conant. Where is the semantic system? critical review and meta-analy analysis of 120 functional neuroimaging studies. Cerebral cortex, 19(12):2767-276, 2009.
Charlotte Caucheteux and Jean-Rémi King. Brains and algorithms partially converge in natural language processing. Communications biology, 5(1):134, 2022.
Charlotte Caucheteux, Alexandre Gramfort, and Jean-Remi King. Decomposing lexical and com- positional syntax and semantics with deep language models. arXiv preprint arXiv:2103.01620, 2021.
E. Fedorenko, P.-J Hsieh, A. Nieto-Castanon, S. Whitfield-Gabrin and N. Kanwisher. New method for fMRI investigations of language: Defining ROIs functionally in individual subjects. Journal of Neurophysiology,
2010.
Evelina Fedorenko and Sharon L Shoipl Reworking the language network. Trends in cognitive sciences, 18(3):120-12) 126, 2014.
Ariel Goldstein, Zaid Zada, Eliav Buchnik, Mariano Schain, Amy Price, Bobbi Aubrey, Samuel A Nastase, Amir Feder, Dotan Emanuel, Alon Cohen, et al. Shared computational principles for language processing in humans and deep language models. Nature neuroscience, 25(3):369-380, 2022.
Mandy Guo, Joshua Ainslie, David Uthus, Santiago Ontanon, Jianmo Ni, Yun-Hsuan Sung, and Yinfei Yang. Longt5: Efficient text-to-text transformer for long sequences. arXiv preprint arXiv:2112.07916. 2021.
Alexander G Huth, Wendy De Heer, Thomas Griffiths, Frédéric Theunissen, and Jack Gallant. Natural speech reveals the semantic maps that tile human cerebral cortex. Nature, 532 (7600):453-455 2016.
Shailee Jain and Alexander Huth. Incorporating context into language encoding models for fMRL. In Advances in Neural Information Processing Systems, pp. (628-6637, 2018.
10

---

Published as a conference paper at ICLR 2023
Tomáš Kočiský, Jonathan Schwarz, Phil Blunsom, Chris Dyer, Karl Moritz Hermann, Gábor Melis, and Edward Grefenstette. The narrativeqa reading comprehension challenge. Transactions of the Association for Computational Linguistics, 6:317-328, 2018.
Wojeiech Kryscinski, Nazneen Fatema Rajani, Divyansh Agarwal, Caiming Xiong, and Dragomir Radev. Booksum: A collection of datasets for long-form narrative summarization. 2021.
Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Ves Stoyanov, and Luke Zettlemoyer. Bart: Denoising scquence-to-sequence pre- training for natural language generation, translation, and comprehension. arXiv preprint arXiv:1910.1361,, 2019.
Tal Linzen. How can we accelerate progress towards human-like linguistic generalization? In Proceedings of the 58th Anmual Meeting of the Association for Computational Linguistics, pp- 9210-5217, 2020.
Thomas McCoy, Ellie Pavlick, and Tal Linzen. Right for the wrong reasons: Diagnosing syntactic heuristics in natural language inference. arXiv preprint arXiv: 2019.
Gabriele Merlin and Mariya Toneva. Language models and brain alignment: beyond word-level semantics and prediction. arXiv preprint arXiv:2212.00596, 2022.
Junghyun Min, R Thomas McCoy, Dipanjan Das, Emily Pitler, and Tal Linzen. Syntactic data augmentation increases robustness to inference heuristics. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 2339-2352, 2020.
Tom M Mitchell, Svetlana V Shinkareva, Andrew Carlson, Kai-Min Chang, Vicente L Malave, Robert A Mason, and Marcel Adam Just. Predicting human brain activity associated with the meanings of nouns. science, 320(5880): 1991-1995 2008
Marius Mosbach, Anna Khokhlova, Michael Hedderich, and Dictrich Klakow, On the interplay between fine-tur fine-tuning and sentence-level probing for linguistic knowledge in pre-trained transform- ers. arXiv preprint arXiv:20/0.02616, 2020.
Shinji Nishimoto, An T Vu, Thomas Naselaris, Yuval Benjamini, Bin Yu, and Jack L Gallant. Re- constructing visual experiences from brain activity evoked by natural movies. Current biology, 21(199:1641-1646 2011.
Subba Reddy Oota, Jashn Arora, Veeral Agarwal, Mounika Marreddy, Manish Gupta, and Bapi Raju Surampudi. Neural language taskonomy: Which nlp tasks are the most predictive of fmri brain activity? arXiv preprint arXiv:2205.00 0140,,4400440000 2022.
Alexandre Pasquiou, Yair Lakretz, John Hale, Bertrand Thirion, and Christophe Pallier. Neural language models are not born equal to fit brain data, but training helps. In International Confer- ence on Machine Learning, pp. 17499-17516 PMLR, 2022.
Aniketh Janardhan Reddy and Leila Wehbe. Can fmri reveal the representation of syntactic structure in the brain? Advances Neural Information Processing Systems, 34:9843-9856, 2021.
J.K. Rowling, M. GrandPre, M. GrandPré, T. Taylor, Arthur A. Levine Books, and Scholastic Inc. Harry Potter and the Sorcerer's Stone. Harry Potter. A.A. Levine Books, 1998. ISBN 9780590353403, URL
Yisi Sang, Xiangyang Mou, Jing Li, Jeffrey Stanton, and Mo Yu. A survey of machine narrative reading comprchension assessments. arXiv preprint arXiv:2205.00294 2022.
Martin Schrimpf, Idan Asher Blank, Greta Tuckute, Carina Kauf, Eghbal A Hosseini, Nancy Kan- wisher, Joshua Tenenbaum, and Evelina Fedorenko. The neural architecture of language: In- tegrative modeling converges on predictive processing. Proceedings of the National Academy of Sciences, 118(, 2021.
Dan Schwartz, Mariya Toneva, and Leila Wehbe. Inducing brain-relevant bias in natural language processing models. Advances in neural information processing systems, 32, 2019.

---

Published as a conference paper at ICLR 2023
Anders Søgaard. Evaluating word embeddings with fmri and eye-tracking. In Proceedings of the 1st workshop on evaluating vector-space representations for NLP, pp. 116-121, 2016.
Mariya Toneva and Leila Wehbe. Interpreting and improving natural-Languaa processing (in ma- chines) with natural language-processing (in the brain). Advances in Neural Information Process- ing Systems, 32, 2019.
Mariya Toneva, Tom M Mitchell, and Leila Wehbe. Combining computational controls with natural text reveals aspects of meaning composition. Nature Computational Science, 2(11):745-757,
2022a.
Mariya Toneva, Jennifer Williams, Anand Bollu, Christoph Dann, and Leila Wehbe. Same cause; different effects in the brain. Causal Learning and Reasoning, 2022b.
Leila Wehbe. Brian Murphy, Partha Talukdar, Alona Fyshe, Aaditya Ramdas, and Tom Mitchell. Simultaneously uncovering the patterns of brain regions involved in different story reading sub- processes. PloS one, 9(1 x) 12575, 2014a.
Leila Wehbe, Ashish Vaswani, Kevin Knight, and Tom Mitchell. Aligning context-based statistical models of language with brain activity during reading. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, pp. 233-2433 2014b.
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, et al. Transformers: State-of-the-art natural language processing- In Proceedings of the 2020 conference empirical methods in natural language processing: system demonstrations, pp. 38-45, 2020.
Manzil Zaheer, Guru Guruganesh, Kumar Avinava Dubey, Joshua Ainslie, Chris Alberti, Santiago Ontanon, Philip Pham, Anirudh Ravula, Qifan Wang, Li Yang, et Big bird: Transformers for longer sequences. Advances in Neural Information Processing Systems, 11s17283-1729)) 2020.
12

---

Published as a conference paper at ICLR 2023
A NLP MODELS USED IN OUR WORK
For all 4 base models and 4 booksum models, we use pretrained models provided by Hugging Face
Wolf et al. 2020).
BART (Lewis et al. 2019)
Base model: https///wuugnnngfaeeccccocccbbb/
Booksum model: https//huggiinnaeeccco/mmm/llbba
LED (Beltagy et al. 2020
Base model: http:///ngggiigeeeccoooooolllaaa
Booksum model: http:///wwggggggocoooooo//////oa
BigBird (Zaheer et al. 2020
Base model:
http::/hhggunnggooooooooooh/ohhh bigpatent
Booksum model:
http://hwnggnngaccccoooo/mmmmmmp pegasus---------------ee large-K-booksumm
LLonTT (Guo et al. 2021
Base model: http://hngiiiggaaccogooggg//gggh
Booksum model: hutp://wwwggiigggooooooo///mmmmp bastle -as---basee-eee
B EXAMPLES FROM BOOKSUM DATASET
Here, describe the inputs and outputs used for training models on BookSum (Kryscinski al., 2021]. The input documents are chapters from books. The output summaries are numan-written obtained from various sources such as CliffNotes SparkNotes GradeSaveri etc. Below examples of summaries from BookSum.
Summary of Chapter 18 of "Sense and Sensibility" by Jane Austen.
Edward's reticence became more noticeable as his visit continued. On one occasion Marianne at- tempted to leave the couple alone together and met with the speedy withdrawal of Edward from the room where she had left them. One day when the sisters and Edward were having breakfast, Mari- anne noticed "a ring, with a plait of hair in the centre," " on one Edward's fingers. She asked him if it was Fanny's, and Edward answered in the affirmative. Elinor and Marianne, however, believed was Elinor's, s, although while Marianne thought was freely given, Elinor was "conscious must have been procured by some theft or contrivance unknown to herself." At noon, Sir John and Mrs. Jennings arrived, curious about the guest at the cottage. Realizing that was the mysterious man with a name beginning with "F" Sir John insisted that they must "drink tea" Barton Park that night and dine there the following day.
Summary of Chapter 4 of "The Picture of Dorian Gray" by Osear Wilde.
Over the next several years, Dorian becomes obsessed with the book given to him by Lord Henry. He buys multiple copies of the "first edition, and them bound in different colors that they might suit his moods." To Dorian, "the whole book...seemed to contain the story of his own life, written before he had lived it." Like the book's young hero, Dorian begins immersing himself in varied interests, including religion, mysticism, music, jewels, ancient tapestries, and the study of his own ancestors. Dorian is, however, quick change obsessions once they no longer interest him, following the whims of his desire with the passion of an artist. He clings to each current obsession fervently, studying it and acquiring as many fanciful examples it as he can find. He buys extravagent gowns covered in hundreds of pearls to feed his interest in jewels, and ancient, golden-threaded tapestries to nourish his curiosity about embroidery. mind, however, he drops it in favor of his next interest. For the next 18 years, capriciousness is way of life for Dorian...
soon as a given subject has exhausted itself in his
Summary of Chapter 6 of "The Brothers Karamazov, Volume 3" by Fyodor Dostoevsky.
When Alyosha enters his father's 's home, his father and brother Ivan are finishing dinner and the servants attend them in the dining room. Here we're given little background Stinking Lizaveta's s son. quiet, sullen child, Smerdyakov was discovered to have the "falling sickness" he would fall into fits every month For some reason Fyodor grew quite fond of the
Smerdyakov,
13

---

Published as a conference paper at ICLR 2023
boy after his illness discovered. One day, Smerdyakov was found picking through his soup, and Fyodor decided that Smerdyakov should be trained as a chef. After training in Moscow, Smerdyakov came back to be Fyodor's 's cook. Now 24, he is just as sullen and silent as ever.
C FURTHER DETAILS ABOUT BRAIN ALIGNMENT APPROACH
In this section, we describe additional details for aligning the word-level NL.P representations to human brain activity. We follow a general alignment approach that has been previously used in several works (Jain & Huth 2018 Toneva & Wehbe 2019 Schrimpf et al. 2021
We start with the following variables. NLP representations |R5166xd where is the embedding size. Brain activity R"x", where is the number of TRs and U2 is the number of voxels in the brain of participant i.
First, we reduce the dimensionality of the word-level NLP representations R-576xd using PCA and retain the top 10 principle components (more than 75% of the variance) to result in a matrix R1176x100 Next, construct TR-level NLP representations by down-sampling the word embed- dings (words presented at 0.5 seconds) to the TR rate (2 seconds) by averaging to corresponding word-level NLP representations result in a matrix E 1@ where n is the number of TRs. The final NLP representations are constructed by concatenating the TR-level NLP representations cor- responding to the previous 4 TRs. The previous TR-level embeddings are included account for the lag in the hemodynamic response that fMRI records. Because the response measured by fMRI an indirect consequence of brain activity that peaks about seconds after stimulus onset, predic- tive methods commonly include preceding time points (Nishimoto et al. (01 1; Wehbe et al., 2014aa] Huth et al. 2016). This allows for a data-driven estimation of the hemodynamic response functions (HRFs) for each voxel, which is preferable assuming one because different voxels may exhibit dif- ferent HRFs, These steps produce the final NLP inputs to the brain alignment stage X E Rox40 for model m, layer sequence length 8.
After reducing the dimensionality of NLP representations, we learn function that uses NL. repre- sentations X E x0= to predict brain activity E T=**, Similarly to previous work, we param- eterize this function as linear function, regularized using the ridge penalty. We train the function in cross- validated way and test its performance on the data that was heldout during training. We select the ridge parameter via nested cross- validation. Since the fMRI data was collected in 4 runs of approximately equal length, we use 4-fold cross-validation where each fold corresponds to holding out one run of fMRI data for testing. We also remove 10TRs of fMRI data and the corresponding NLP representations from the ends of each test run to avoid possible train-test overlap.
Finally, we evaluate the alignment between the NLP predictions and the true IMRI activity, as de- scribed in the main paper.
D INTERPRETABILITY APPROACH TO COMPARE BRAIN ALIGNMENT ACROSS VARIOUS DISCOURSE FEATURES
Algorithm 2 Interpretability approach to compare brain alignment across discourse features
1: Input: NLP encoding predictions, fMRI activity
> Both E R TRaxe; wozels
2: for all discourse features do
3: label words with discourse feature
> One-hot vector E R5176 words
4: map labeled words to fMRI TRs labeled with discourse feature One-hot vector R
5: extract NLP encoding predictions for TRs with the discourse feature
b R 160x
6: extract actual brain activity for TRs with the discourse feature
> R 160x
7: compute Pearson correlation between extracted NLP predictions and brain activity
b R
8: Result: Pearson correlation score for discourse feature, for each voxel
b R
9: end for
10: Result: Pearson score for discourse features
We present the pseudocode for our interpretability approach in Algorithm 2 and elaborate here.
14

---

Published as a conference paper at ICLR 2023
The Pearson correlation score is computed between pair of NLP encoding predictions and actual brain activity. Both E I= where n is the number of TRs and v the number voxels in the brain of participant (Line 1).
For each discourse feature (Line 2), we first label the words the Harry Potter chapter that corre- spond the discourse feature, producing one-hot vector E R6176 (Line 3) Then, apply the mapping from the words to fMRI TRs, because fMRI values are recorded once for every few words. This produces one-hot vector R", which labels the TRs which contain the discourse feature (Line 4).
Different discourse features will have different number of TRs labeled, e.g-, Characters has 236 TRs labeled, whereas Emotions has 170 TRs labeled. For each discourse feature, we sample 160 TRs of the TRs labeled. We do this ensure that we use the same number of TRs for each discourse feature, allowing fair comparison of Pearson correlation scores between discourse features. Hence, we have one-hot vector R 100
We extract the NLP encoding predictions and brain activity for only the TRs containing Characters, both 1660x (Line 5 and 6). Finally, we compute their Pearson correlation, producing vector I This produces one Pearson correlation score for each brain voxel (Line 7). To get the Pearson score for the Characters discourse feature, we take the average across v; voxels.
Earlier, we mentioned that for each discourse feature, words in the Harry Potter chapter have been la- beled. For example, the word "Draco" is marked with the Character feature, and the word "climbed" with the Motion feature. We used discourse feature labels from Wehbe et al. (2014a).
For each discourse feature, we computed the average Pearson correlation for booksum and base models. To compute these averages, 20 to 1000, and all 8 human participants. The results are reported in I6
all 8 models, layers for each model, sequence lengths
E STATISTICAL TESTS AND FALSE DISCOVERY RATE (FDR) CORRECTION USING THE BENAMINN-HOCHEER (BH) PROCEDURE
We perform t-tests to identify which results are statistically significant (not simply due to chance). We used paired t-test to compare brain alignment between booksum and base models (c.g.. Figure 1.. We used one-sample t-test identify brain voxels where model's pearson correlation brain alignment is significantly greater than 0 (e.g., Figure
As perform large number of statistical tests, obtain large number of p-values. Thus, some will have p-values below 0.05 purely by chance. To correct this problem of false discovery, we adjust the false discovery rate (FDR) using the Benjamini-Hochberg (BH) procedure. For each original p- value, this produces new, adjusted p-value. Finally, we compare the adjusted p-value against significance level of 0.05 identify statistically significant results. For our paper, we report only statistically significant results after FDR-BH correction.
F SELECTED LAYERS, SEQUENCE LENGTHS AND PARTICIPANTS
To compute 20v20 and Pearson averages, we could not run results for all layers and sequence lengths for all models, due computational constraints.
Layers. For each model, we selected 8 representative layers evenly spread across all its layers. For BART, LED and LongT5, these models contained 12 layers (1 to 12), and we selected layers (1, 4, 6, 8, 10, 11, 12). For BigBird, it contains 32 layers (1 to 32), and we selected 8 layers (2, 6, 10, 14, 18, 24, 28, 32).
Sequence lengths. We selected representative sequence lengths from 1 to 1000, listed here: 1, 5, 20, 100, 200, 300, 400, 500, 700, 1000. For BART and LED, we used only sequence lengths up to 500, due to their shorter input token limits.
Participants. We used fMRI results from all human participants.
15

---

Published as a conference paper at ICLR 2023
G AGGREGATED 20v20 AND PEARSON CORRELATION RESULTS FOR ALL 4 PAIRS OF MODELS
base
bart
bart
booksum
led
led
base
booksum
bigbird
bigbird
long-t5
long-t5
54
56
58
60
0.005
0.015
0.02
20v20 accuracy
Pearson correlation
Figure 5: Left. Aggregated 20v20 brain alignment results for all 4 pairs of models. Right. Aggre- gated Pearson correlation brain alignment results for all 4 pairs of models. Averages were computed over 8 layers for each model, sequence lengths to 1000, and all participants. In both figures, we see that for each of the pairs of models, the booksum model has significantly greater brain align- ment than the base model (paired t-test, FDR corrected for multiple comparisons at significance level 0.05). These results show that training language models for deeper understanding improves brain alignment.
16

---

Published as a conference paper at ICLR 2023
H RESULTS FOR 20v20 ACCURACY ACROSS SEQUENCE LENGTHS AND LAYERS
20v20 brain alignment for BART
layer
accuracy
60.0
$7.5
55.0
52.5
50.0
layer 8
layer
layer 11
layer 12
70,20 accurary
57.5
$5.0
$2.5
base
200
300
400
500
100
300 400
Sequence length
Sequence length
Sequence length
Sequence length
Figure 6: Comparing 20v20 classification accuracy between BART-base and BART-booksum, Av- erages were computed over all 8 human participants.
20v20 brain alignment for LED
layer
20x20 accuracy
$7.5
85.0
50.0
base
booksum
layer
layer 10
layer 11
layer 12
60.0
78v20 accuracy
$5.0
$2.5
47.3
Sequence length
Sequence langth
lengtt
Sequence length
Figure 7: Comparing 20v20 classification accuracy between LED-base and LED-booksum. Aver- ages were computed over all 8 human participants.
17

---

Published as a conference paper at ICLR 2023
20v20 brain alignment for BigBird
layer
layer
layer
layer
60
acquracy
35
base booksurn
30
45
24
accuracy
60
85
50
Sequence length
Sequence length
Sequence length
Sequence length
Figure 8: Comparing 20v20 classification accuracy between BigBird- and BigBird- Averages were computed over all 8 human participants.
20v20 brain alignment for LongT5
layer
layer
layer
layer
62
accuracy
60
688
86
booksum
layer
layer 10
layer 11
layer 12
20u20 accuracy
60
88
96
34
Sequence length
Sequence length
Sequence length
Sequence length
Figure 9: Comparing 20v20 classification accuracy between LongT5- and L5ngT5-boodd booksum.
Averages were computed over all 8 human participants.
18

---

Published as a conference paper at ICLR 2023
Paired t-test: Booksum greater than Base model (BART)
p-values after correction
Reject null hypothesis
1.0
0.8
0.6
-001
0.4
10
11
11
0.2
12
12
0.0
100
Figure 10: For BART, paired t-test was performed to identify the layers and sequence lengths where the booksum model has greater 20v20 brain alignment than the base model, with statistical signifi- cance. 20v20 averages were computed over all 8 human participants. Left. p-values obtained from paired t-test, after false discovery rate (FDR) correction using the Benjamini-Hochberg (BH) pro- cedure. Right. At significance level 0.05, the layers and sequence lengths labeled in black are those where the booksum model has significantly greater brain alignment than the base model.
Paired t-test: Booksum greater than Base model (LED) Reject null hypothesis
p-values after correction
1.0
0.8
0.6
10-1
0.4
10
10
11
12
12
10-2
20
100
300
400
500
700
1000
1
20
100
200
300
400
500
700
1000
Figure 11: For LED, paired was performed to identify the layers and sequence lengths where
the booksum model has greater 20v20 brain alignment than the base model, with statistical signifi- cance. 20v20 averages were computed over all 8 human participants. Left. p-values obtained from paired t-test, after false discovery rate (FDR) correction using the Benjamini- cedure. Right. At significance level 0.05, the layers and sequence lengths labeled in black are those where the booksum model has significantly greater brain alignment than the base model.
(BH) pro-
19

---

Published as a conference paper at ICLR 2023
Paired t-test: Booksum greater than Base model (BigBird)
p-values after correction
Reject null hypothesis
1.0
FL
0.8
10
10
0.6
12
14
10-1
18
18
0.4
22
24
28
28
0.2
32
32
10-2
0.0
1
100
200
300
400
500
700
1000
20
100
200
300
400
500
700
1000
Figure 12: For BigBird, paired t-test performed to identify the layers and sequence lengths where the booksum model has greater 20v20 brain alignment than the base model, with statistical significance. 20v20 averages were computed over all human participants. Left. p-values obtained from paired I-test, after false discovery rate (FDR) correction using the Benjamini-Hochberg (BH) procedure. Right. At significance level 0.05, the layers and sequence lengths labeled in black those where the booksum model has significantly greater brain alignment than the base model.
Paired t-test: Booksum greater than Base model (LongT5) Reject null hypothesis
p-values after correction
1.0
0.8
0.6
10-1
8
0.4
10
11
0.2
12
12
10-2
0.0
100
200
300
400
500
700
1000
20
100
200
300
400
500
700
1000
Figure 13: For LongT5, paired t-test performed to identify the layers and sequence lengths where the booksum model has greater 20v20 brain alignment than the base model, with statistical significance. 20v20 averages were computed over all human participants. Left. p-values obtained from paired t-test, after false discovery rate (FDR) correction using the Benjamini-Hochherg (BH) procedure. Right. At significance level 0.05, the layers and sequence lengths labeled in black those where the booksum model has significantly greater brain alignment than the base model.
20

---

Published as a conference paper at ICLR 2023
1
RESULTS FOR BRAIN REGIONS OF INTEREST (ROIs)
all
all
booksum
AntTemp
Antiemp
AngulerG
AngularG
MFG
MFG
pCingulate
pCingulate
dmpfc
base
dmptc
Non-language
booksum
Non-language
52
54
58
60
62
65
68
70
52
54
60
62
64
68
70
20vs20 accuracy
20vs20 accuracy
BigBird
LongT5
all
all
PostTemp
PostTemp
AntTemp
Antiampp
AngularG
AngularG
IFG
MFG
MFG
pCingulate
pCingulate
dmpfc
base
dmpfc
base
Non-language
booksur
Non-language
booksum
52
60
52
54
60
02
70
20vs20 accuracy
20vs20 accuracy
Figure 14: 20v20 accuracy averages for various brain Regions of Interest (ROIs), for each of the 4 models. Averages were computed over 8 layers for each model, sequence lengths 1000, and all 8 participants.
Paired t-test: Booksum greater than Base model for various brain ROIs p-values after correction
Reject null hypothesis
10-
PestTmm
10-24
AntTemp
10-39
AngularG
54
0.8
IGG..
MFG
10
F5orb
44
pCingulate
10
dmpfc
10-134
Non-language
0.0
bert
led
bighird
long-t5
bart
bigbird long-t5
For each of the models, paired t-test was performed identify brain ROIs where the booksum model has greater 20v20 brain alignment than the base model, with statistical significance. We also perform the I-test for all the models combined together, labeled "all". 20v20 averages were computed over 8 layers for each model, sequence lengths 1 to 1000, and all 8 participants. Left. p-values obtained from paired t-test, after false discovery rate (FDR) correction using the -oouuiinn-Hchhbrrgg (BH) procedure. Right. At significance level 0.05, the brain ROIs labeled in black are those where the booksum model has significantly greater brain alignment than the base model.
Figure 15:
21

---

Published as a conference paper at ICLR 2023
J
RESULTS FOR DISCOURSE FEATURES
BART
LED
Characters
Characters
0.022
0.008
-0.002
Emotion
Emotion
-0.001
-0.008
Motion
base booksum
Motion
base booksum
40.004
-0.01
0.00
0.01
0.02
0.03
0.02
-0.01
0.00
0.01
0.02
0.03
Pearson correlation
Pearson correlation
BigBird
LongT5
0.014
0.026
Characters
Characters
0.023
0.033
0.012
0.019
Emotion
Emotion
0.021
0.019
-0,000
0.008
Motion
base
Motion
base
0.007
booksum
0.015
booksum
-0.1
0,01
0.03
0.00
Pearson correlation
Pearson correlation
Figure 16: Pearson correlation brain alignment averages for various discourse features, for each of the 4 models. Averages computed over layers for each model, sequence lengths 20 500, and all 8 participants.
Paired t-test: Booksum greater than Base model various discourse features
p-values after correction
Reject null hypothesis
10-2
1.0
Characters
10
0.8
10-16
Emation
10-23
0.6
10-30
0.4
37
Notion
0.2
10-51
0.00
bart
led
bigbird
long-ts
bart
bigbird long-t5
Figure 17: For each of the models, paired t-test was performed to identify the discourse features where the booksum model has greater Pearson correlation brain alignment than the base model, with statistical significance. We also perform the 1-test for all the models combined together, labeled "all". Pearson correlation averages were computed over 8 layers for each model, sequence lengths 20 to 500, and all participants. Left. p-values obtained from paired t-test, after false discovery rate (FDR) correction using the Benjamini-Hochber the discourse features labeled black are those where the booksum model has significantly greater brain alignment than the base model.
(BH) procedure. Right. At significance level 0.05,
22

---

Published as a conference paper at ICLR 2023
Paired t-test: Characters improve (booksum base) more than other discourse features
p-values after correction
Reject null hypothesis
1.0
Emation
10-22
0.8
10
0.6
10-6
Motion
10
10-10
0.2
bart
bigbird long-t5
bart
bigbird long-t5
Figure 18: For each of the models, paired I-test was performed to identify whether there were greater improvements in Pearson correlation brain alignment for Characters than other discourse features, with statistical significance. We also perform the t-test for all the models combined to- gether, labeled "all". Pearson correlation averages sequence lengths 20 to 500, and all 8 participants. Left. p-values obtained from paired t-test, after false discovery (FDR) correction using the Benjamini-Hochherg (BH) procedure. Right. At significance level 0.05, the brain ROIs labeled in black are those where the booksum model has significantly greater brain alignment than the base model.
computed over 8 layers for each model,
23

---

Published as a conference paper at ICLR 2023
K
RESULTS FOR BRAIN PLOTS
Characters TRs
Booksum (extemal) BART
Random sample TRs (8 subjects)
(8 subjects)
0.0
0.6
across brain voxels on the external brain surfaces of 8 human participants. Averages were computed over 8 layers for each model and se- quence lengths 20 to 500. Only voxels with Pearson correlation significantly greater than are plotted (one-sample t-test, FDR corrected for multiple comparisons at significance level 0.05). Left. Pearson correlation averages computed for only TRs which contain Characters. Right. Same as Left, but for equal number of TRs sampled randomly from all TRs.
Figure 19: Pearson correlation averages of
Characters TRs subjects)
Booksum (extemal)
Random sample TRs (8 subjects)
LED
0.0
0.6
Figure 20: Pearson correlation averages of LED-booksum across brain voxels on the external brain surfaces of 8 human participants. Averages were computed over 8 layers for each model and se- quence lengths 20 to 500. Only voxels with Pearson correlation significantly greater than are plotted (one-sample t-test, FDR corrected for multiple comparisons at significance level 0.05). Left. Pearson correlation averages computed for only TRs which contain Characters. Right. Same as Left, but for an equal number of TRs sampled randomly from all TRs.
24

---

Published as a conference paper at ICLR 2023
Characters TRs (8 subjects)
Booksum (external) BigBird
Random sample TRs (8 subjects)
0.0
0.6
Figure 21: Pearson correlation averages of Big -bookdum across brain voxels the external brain surfaces of 8 human participants. Averages were computed over 8 layers for each model and sequence lengths 20 to 500. Only voxels with Pearson correlation significantly greater than 0 are plotted (one-sample t-test, FDR corrected for multiple comparisons at significance level 0.05). Left. Pearson correlation averages computed for only TRs which contain Characters. Right. Same as Left, but for an equal number of TRs sampled randomly from all TRs.
Characters TRs (8 subjects)
Booksum (external) LongT5
Random sample TRs (8 subjects)
0.0
0.6
Figure 22: Pearson correlation averages of LongT5 -oboookuu across brain voxels on the external brain surfaces of human participants. Averages were computed over 8 layers for each model and sequence lengths 20 to 500. Only voxels with Pearson correlation significantly greater than 0 are plotted (one-sample t-test, FDR corrected for multiple comparisons at significance level 0.05). Left. Pearson correlation averages computed for only TRs which contain Characters. Right. Same as Left, but for an equal number of TRs sampled randomly from all TRs.
25

---

Published as a conference paper at ICLR 2023
Characters TRs (8 subjects)
Booksum (internal) BART
Random sample (8 subjects)
0.0
0.6
DRIPEC
DNPPC
Figure 23: Pearson correlation averages of BART-booksum across brain voxels on the internal brain surfaces human participants. Averages were computed over layers for each model and se- quence lengths 20 to 500. Only voxels with Pearson correlation significantly greater than 0 plotted (one-sample t-test, FDR corrected for multiple comparisons at significance level 0.05). Left. Pearson correlation averages computed for only TRs which contain Characters. Right. Same as Left, but for an equal number of TRs sampled randomly from all TRs.
Characters TRs (8 subjects)
Booksum (internal) LED
Random sample TRs (8 subjects)
0.0
0.6
DRIPAC
DNPIC
Figure 24: Pearson correlation averages of LED-bboksuum across brain voxels on the internal brain surfaces human participants. Averages were computed over layers for each model and se- quence lengths 20 to 500. Only voxels with Pearson correlation significantly greater than 0 are plotted (one-sample t-test, FDR corrected for multiple comparisons at significance level 0.05). Left. Pearson correlation averages computed for only TRs which contain Characters. Right. Same as Left, but for an equal number of TRs sampled randomly from all TRs.
26

---

Published as a conference paper at ICLR 2023
Characters TRs (8 subjects)
Booksum (internal) BigBird
Random sample (8 subjects)
0.0
0.6
DRIPEC
DNPPC
Figure 25: Pearson correlation averages of BigBird- -bookd- across brain voxels on the internal brain surfaces of 8 human participants. Averages were computed over 8 layers for each model and sequence lengths 20 to 500. Only voxels with Pearson correlation significantly greater than 0 are plotted (one-sample t-test, FDR corrected for multiple comparisons at significance level 0.05). Left. Pearson correlation averages computed for only TRs which contain Characters. Right. Same as Left, but for an equal number of TRs sampled randomly from all TRs.
Characters TRs (8 subjects)
Booksum (internal) LongT5
Random sample TRs (8 subjects)
0.0
0.6
DNPPC
DNPIC
Figure 26: Pearson correlation averages of LongTS-booksung across brain voxels on the internal brain surfaces of human participants. Averages were computed over 8 layers for each model and sequence lengths 20 to 500. Only voxels with Pearson correlation significantly greater than 0 are plotted (one-sample t-test, FDR corrected for multiple comparisons at significance level 0.05). Left. Pearson correlation averages computed for only TRs which contain Characters. Right. Same as Left, but for an equal number of TRs sampled randomly from all TRs.
27

---

Published as a conference paper at ICLR 2023
Characters TRs subjects)
Booksum minus Base (external)
Random sample TRs (8 subjects)
BART
-0.2
+0.2
Figure 27: Difference in Pearson correlation averages between BART-booksum and BART-base across brain voxels on the external brain surfaces of 8 human participants. Averages were computed over 8 layers for each model and sequence lengths 20 to 500. Only voxels with significant Pear- son correlation differences between the booksum and base model are plotted (paired t-test, FDR corrected for multiple comparisons significance level 0.05). Left. Pearson correlation averages computed for only TRs which contain Characters. Right. Same Left, but for an equal number of TRs sampled randomly from all TRs.
Characters TRs subjects)
Booksum minus Base (external)
Random_sample TRs
(8 subjects)
LED
-0.2
+0,2
Figure 28: Difference in Pearson correlation averages between LED-booksum and LED-base across brain voxels on the external brain surfaces of 8 human participants. Averages were computed over 8 layers for each model and sequence lengths 20 to 500. Only voxels with significant Pearson correla- tion differences between the booksum and base model are plotted (paired t-test, FDR corrected for multiple comparisons significance level 0.05). Left. Pearson correlation averages computed for only TRs which contain Characters. Right. Same as Left, but for an equal number of TRs sampled randomly from all TRs.
28

---

Published as a conference paper at ICLR 2023
Characters TRs subjects)
Booksum minus Base (external)
Random sample TRs (8 subjects)
BigBird
-0.2
Figure 29: Difference in Pearson correlation averages between BigBird-booksee booksum and BigBird-bass across brain voxels on the external brain surfaces of 8 human participants. Averages were computed over 8 layers for each model and sequence lengths 20 to 500. Only voxels with significant Pear- son correlation differences between the booksum and base model are plotted (paired t-test, FDR corrected for multiple comparisons significance level 0.05). Left. Pearson correlation averages computed for only TRs which contain Characters. Right. Same Left, but for equal number of TRs sampled randomly from all TRs.
Characters TRs subjects)
Booksum minus Base (external) LongT5
Random_sample TRs (8 subjects)
-0.2
+0.2
Figure 30: Difference in Pearson correlation averages between LongT5- -oooossuu and LongTS- across brain voxels on the external brain surfaces of 8 human participants. Averages were computed over 8 layers for each model and sequence lengths 20 to 500. Only voxels with significant Pear- son correlation differences between the booksum and base model are plotted (paired t-test, FDR corrected for multiple comparisons significance level 0.05). Left. Pearson correlation averages computed for only TRs which contain Characters. Right. Same as Left, but for an equal number of TRs sampled randomly from all TRs.
29

---

Published as a conference paper at ICLR 2023
Characters TRs subjects)
Booksum minus Base (internal)
Random_sample TRs
(8 subjects)
BART
-0.2
+0,2
DIPPEC
DNPFC
Difference in Pearson correlation averages between BART-booksum and BART-base across brain voxels on the internal brain surfaces of 8 human participants. Averages were computed over 8 layers for each model and sequence lengths 20 to 500. Only voxels with significant Pear- son correlation differences between the booksum and base model are plotted (paired t-test, FDR corrected for multiple comparisons significance level 0.05). Left. Pearson correlation averages computed for only TRs which contain Characters. Right. Same Left, but for an equal number of TRs sampled randomly from all TRs.
Figure 31:
Characters TRs subjects)
Booksum minus Base (internal)
Random sample TRs (8 subjects)
LED
PC
-0.2
+0,2
DIPPIC
DRIPEC
Figure 32: Difference in Pearson correlation averages between LED-booksum and LED-base across brain voxels on the internal brain surfaces of 8 human participants. Averages were computed over 8 layers for each model and sequence lengths 20 to 500. Only voxels with significant Pearson correla- tion differences between the booksum and base model are plotted (paired t-test, FDR corrected for multiple comparisons significance level 0.05). Left. Pearson correlation averages computed for only TRs which contain Characters. Right. Same as Left, but for an equal number of TRs sampled randomly from all TRs.
30

---

Published as a conference paper at ICLR 2023
Characters TRs subjects)
Booksum minus Base (internal)
Random_sample TRs (8 subjects)
BigBird
PC
-0.2
DIPPEC
DNPFC
Figure 33: Difference in Pearson correlation averages between HigBird-bbokssng and BigBird-base across brain voxels on the internal brain surfaces of 8 human participants. Averages were computed over 8 layers for each model and sequence lengths 20 to 500. Only voxels with significant Pear- son correlation differences between the booksum and base model are plotted (paired t-test, FDR corrected for multiple comparisons significance level 0.05). Left. Pearson correlation averages computed for only TRs which contain Characters. Right. Same Left, but for equal number of TRs sampled randomly from all TRs.
Characters TRs subjects)
Booksum minus Base (internal) LongT5
Random sample TRs (8 subjects)
PC
-0.2
+0.2
DIPPIC
DRIPEC
Figure 34: Difference in Pearson correlation averages between LongT5- -booss-b and L-ngTS- across brain voxels on the internal brain surfaces of 8 human participants. Averages were computed over 8 layers for each model and sequence lengths 20 to 500. Only voxels with significant Pear- son correlation differences between the booksum and base model are plotted (paired t-test, FDR corrected for multiple comparisons significance level 0.05). Left. Pearson correlation averages computed for only TRs which contain Characters. Right. Same as Left, but for an equal number of TRs sampled randomly from all TRs.
31

---

Published as a conference paper at ICLR 2023
L
RESULTS FOR SUMMARIZATION MODELS ON NON-NARRATIVE DATASETS
base
booksum
bart
cnn
53
54
55
57 20v20 accuracy
56
58
59
60
Figure 35: 20v20 averages for BART models. Averages model, sequence lengths 1 to 500, and all 8 participants. The model trained to summarize long narratives (BART-booksum) has significantly greater brain alignment than the model pretrained with a language modeling objective (BART-base) (BART-cnn). Furthermore, BART-enn only performs on par with the pretrained BART-base at brain alignment. This shows that improved brain alignment in BART-booksum over BART-base is not simply due the summarization objective. We used paired t-tests, with FDR corrected for multiple comparisons at significance level 0.05.
computed over 8 layers for each
well as the model trained to summarize news articles
Our paper shows improved brain alignment for booksum models over base models. There are possible explanations: (1) similarity of narrative domain in train and test set, (2) summarization learning objective, (3) summarization on narrative domain, resulting in deeper understanding of characters, emotions and other discourse features. We argue that the third is the result: a deeper understanding emerges when training these models to summarize narrative texts.
In Appendix M show that improved brain aligmment is not simply due to the first possibility: domain similarity. In this appendix, we show that improved brain alignment not simply due the second possibility: summarization learning objective.
compared the brain alignment for models: BART-base, BART-booksum and BART-cnn. BART-base was pretrained by corrupting documents and learning to reconstruct the original text, on a combination of books (narratives) and Wikipedia data (see Appendix (A). BART-booksum ini- tializes weights of BART-base and further trained to summarize long narratives the BookSum summarization dataset. BART-con initializes weights of BART-base and was further trained to sum- marize news articles from the CNN Dailymail dataset
The model trained to summarize long narratives (BART-booksum) has significantly greater brain alignment than the model pretrained with language modeling objective (BART-base), as well as the model trained to summarize news articles (BART-cnn). Furthermore, BART-enn only performs on par with the pretrained BART-base at brain alignment. This shows that improved brain alignment in BART-booksum over BART-base is not simply due to the summarization objective.
Both the input documents and output summaries the BookSum dataset are substantially longer than those in the CNN Dailymail dataset, which promotes deeper long-range understanding. For the BookSum dataset, the input documents have an average of more than 5000 words, and output summaries have roughly 500 words on average. For the CNN Dailymail dataset, the input documents have 766 words on average while the summaries have 53 words on average.
The texts from the BookSum dataset contain richer discourse features. For example, characters develop over the course the entire book through their thoughts, actions, and interactions with other characters. We show in Section 6 that the booksum models developing richer representations of these discourse features.
32

---

Published as a conference paper at ICLR 2023
M RESULTS FOR TRAINING ON BOOKSUM DATASET WITH A LANGUAGE MODELING OBJECTIVE
Cross-entropy Loss on Harry Potter dataset
bart
BART
led
booksum
LED
LM-booksum
bigbird
BigBird
long-t5
base
LongTS
booksum
LM-booksum
54
58
60
20v20 accuracy
LM Cross-entropy (lower better)
Figure 36: Left. 20v20 brain alignment for base, booksum and LM-booksum models. LM-booksum models were trained on the BookSum dataset with a language modeling objective. Averages were computed over layers for each model, sequence lengths 1000, and all participants. Right. Language modeling cross-entropy loss of all 4*3=12 12 models on the Harry Potter fMRI dataset.
For 2 models (BART, LongT5), the booksum variant has substantially better brain alignment than the LM-booksum counterpart. For the other 2 models (LED, BigBird), there is similar brain alignment for the booksum and LMM-booksum variants. We used paired t-tests, with FDR corrected for multiple comparisons at significance level 0.05.
However, the LM-booksum variants have substantially better LM performance on the fMRI dataset than the booksum models. As LM performance has been shown to contribute to brain alignment (Schrimpf al., 202T CaucheCuu & King 2022 Goldstein et al. 2022). is difficult to disentangle the brain alignment contributions of "deeper understanding" and LM performance in these models. The LM-booksum models have greater LM performance than both the base and booksum models, whereas the base and booksum models have similar LM performance. Hence, we believe that the LM-booksum models may not be a better baseline than base models.
Combined, these results suggest that the improved brain alignment is not simply due to domain similarity.
33

---

Published as a conference paper at ICLR 2023
N
RESULTS FOR GPT-2 BOTH 20v20 AND PEARSON CORRELATION
gpt2
gpt2
bart
bart
led
base
booksum
led
base
booksum
bigbird
bigbird
long-t5
long-t5
54
56
60
0.005
0.01
0.015
0.02
20v20 accuracy
Pearson correlation
Figure 37: Left. Comparing average 20v20 accuracy for GPT-2 against the four pairs of base and booksum models. Right. Comparing average Pearson correlation for GPT-2 against the four pairs of base and booksum models. Averages were computed over 8 layers for each model, sequence lengths to 1000, and all participants.
We ran experiments compute the aggregated 20v20 and Pearson correlation for GPT-2, For 20v20, LongT5-bboossn has greater brain alignment than all other models, including GPT-2. For Pearson correlation, LonooSsboo and GPT-2 have similar performance, and both greater than the other models. We used paired t-tests, with FDR corrected for multiple comparisons at significance level 0.05.
Our work is focused on investigating how the training objective of summarizing narrative texts leads improved brain alignment, hence we compared 4 pairs of base and booksum models. Attaining model with SOTA brain alignment is not the goal of our work, hence we did not finetune GPT-2 on the booksum dataset. That outside the scope of our project, but may be an interesting direction for future research to pursue.
34

---

Published as a conference paper at ICLR 2023
O RESULTS FOR TRAINING ON BALANCED DISCOURSE FEATURES
BART
0.039
Characters
0.053
0.036
Emotion
0.045
0.022
Motion
base booksum
0.032
0.030
Others
0.039
0.035
Full
0.042
0.02
0.03
0.04
0.05
0.06
0.07
Pearson correlation
Figure 38: Pearson correlation averages for IMRI TRs corresponding to various discourse features. Averages were computed over layers for each model, sequence lengths 20 to 500, and all par- ticipants. "Others" refers to TRs which were not labeled with either "Characters", "Emotion" "Motion". "Full" refers to the full set of all TRs. These plots were generated by training BART on the last 500 TRs of the Harry Potter fMRI dataset, where the distribution of discourse features was more balanced. BART-base and BART-booksun achieve greater brain alignment for Characters than other discourse features. When training BART for deep understanding, brain alignment significantly improves for all discourse features. However, it improves more for Characters than other discourse features. We used paired t-tests, with FDR corrected for multiple comparisons at significance level 0.05.
There are roughly 1200 fMRI TRs the Harry Potter dataset where predictions are generated. Among this set, there are 242 Characters TRs, 174 Emotion TRs, 165 Motion TRs, and the remain- ing are TRs labeled as "Others". For each of the discourse features, their TRs are relatively evenly distributed across the 1200 TRs. All 3 discourse features' TRs are also distributed in similar ways.
In Section [ our results showed that Characters improved more (booksum minus base) than other discourse features when training models for deeper understanding. However, another possible cause of this result is that there are different number of TRs for each discourse feature in the training set (242 for Characters, 174 for Emotion, 165 for Motion).
Hence, in this section, we ran additional experiments where we trained the linear encoding models only the last 500 TRs, where the number of TRs for each discourse feature was more balanced (97 for Characters, 99 for Emotion, 72 for Motion). We then sampled an equal number TRs (74) for each discourse feature during testing the first 700 TRs. After controlling for equal number of TRs the training and test set across all discourse features, we still see the result that Characters improved more (booksum minus base) than other discourse features during training for deeper understanding.
35

---

Published as a conference paper at ICLR 2023
P RESULTS FOR TRAINING LARGER MODELS
base
booksum
long-t5-large
55
56
58 20v20 accuracy
57
59
60
Figure 39: 20v20 brain alignment for the larger variant of LongT5, with 24 layers. We report results Averages were computed over 8 layers
for both the base and booksum models of LongT5- for each model, sequence lengths 1 to 1000, and all 8 participants. The booksum model has greater brain alignment than the base model (paired t-test, with FDR corrected for multiple comparisons at significance level 0.05).
layer
20w20 accuracy
60
188
56
pretrained
54
booksum
1000
Figure 40: Comparing 20v20 brain alignment for large base against across different layers and input sequence lengths. Each data point is an average across 8 partici- pants. Brain alignment improvements occur in the middle and late layers (from layers 12 to 24) (paired t-test, with FDR corrected for multiple comparisons at significance level 0.05).
We ran experiments to evaluate the brain alignment for two model variants of LongT5-large with 24 layers. The first model the C4 dataset, collection of roughly 750 GB of English texts sourced from the public Common Crawl open repository. The second model model, and further trains the model to generate summaries for the BookSum narrative dataset. Our results show that the booksum model has greater brain alignment than the base model, and the brain alignment improvements occur in the middle and late layers (from layers 12 to 24).
was pretrained by masking and generating sentences on
initializes weights from the first
In BigBird, the brain alignment improvements occur in the middle layers. On the other hand, for both the 12-layer and 24-layer variants of LongT5, the brain alignment improvements occur in the middle and late layers. For different models (BART, LED, BigBird, LongT5), observed different pattern of brain alignment improvements across layers and sequence lengths. We find this observation fascinating, and hope explore in future work the reasons why different models improve brain alignment differently even though they are all trained for deeper understanding.
36

---

Published as a conference paper at ICLR 2023
Q
NOISE CEILING ESTIMATES FOR 20v20 METRIC
-0.70
0.66
0.63
0.59
0.57
0.63
0.63
0.61
0.72
0.72
0.66
0.66
0.66
0.65
Target subject
0.67
0.64
0.63
0.55
0.53
0.63
0.6
0.54
0.58
0.6
0.54
0.51
0.59
0.56
0.60
0.56
0.61
0.59
0.52
0.58
0.59
0.49
0.55
0.73
0.65
0.68
0.58
0.59
0.59
0.66
0.59
0.58
0.58
0.55
0.56
0.5
0.61
0.50
Source subject
Figure 41: 20v20 results for 8*7=56 pairs of (target subject, source subject).
20v20
0.66
0.62
0.69
Target
K-
0.56
0.57
0.57
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
20v20 noise ceiling, using bootstrap
Figure 42: 20v20 Noise ceiling estimates for each the 8 target subjects.
We ran experiments compute the noise ceiling estimates for all 8 subjects, for the 20v20 metric. To do this, we followed previous work on computing a noise ceiling for naturalistic data with only repetition of the stimulus (Schrimpf et al. (02T). We point the reader to this previous work for detailed description of the steps. Briefly, this noise ceiling estimation is based on the ability to predict data from one participant using data from a different participant. The prediction is done by training linear encoding models that use one source subject's fMRI response to predict different target subject's ' fMRI response, cross-validated manner.
The average noise ceiling across participants is 0.61 +/- 0.016 (sem) for 20v20. For sequence length 500, all BookSum models reach 98% of this noise ceiling (20v20 values of 0.6, for different layers across models).
37


Reviews:
[{"review_id": "yNf9SNnWyF-", "reviewer": "Reviewer_nyjm", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper aims to study whether particular large pre-trained transformers may align with human brain activity. This is taken as a solid hint that these pre-trained transformers are learning more than simple replicating the input. These models are domain adapted on book texts where they have to learn plots. \n", "summary_of_the_review": "The ideas in the paper are really interesting and fascinating. Indeed, the parallel between machines and brains has always been used to discover new insights in computer science as well as in understanding brain activity. This parallel has also been studied in:\nZanzotto, F.M., Croce, D. (2009). Reading What Machines “Think”. Brain Informatics. 2009.\nDell’Arciprete, et al (2012). Parallels between Machine and Brain Decoding. In: Brain Informatics. 2012. ", "strength_and_weaknesses": "PRO\n\n- The paper is a solid experimental paper which presents an apparently novel idea\n\nCONS\n\n- It is difficult to interpret results if the reader is not completely aware of the field\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "NTlagp5NEr", "reviewer": "Reviewer_Z82i", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper presents a neural network brain encoding analysis comparing the performance of Transformer architectures trained on language modeling objectives versus those trained on a narrative summarization objective. The authors argue that the narrative summarization task is a proxy for \"deeper understanding,\" and that the improved brain encoding performance of models optimized on the task (AKA BookSum models) demonstrates this point. They show that the improved brain encoding performance of BookSum models is not reducible to improved language modeling performance, and use a data-driven evaluation to further study the representational content that could be driving increased brain encoding performance.", "summary_of_the_review": "Overall I think the paper does a nice job of pairing performance gains with multiple attempts at explaining their cause. I'm not especially convinced by the final interpretability analysis, but I think these results are interesting and can help to start a good discussion in the community. I vote to accept, and encourage the authors to add the controls mentioned in the \"Quality\" section above.", "strength_and_weaknesses": "- The paper addresses a timely question about the sorts of language training objectives which produce brain-like representations. This question is of interest to several different fields within cognitive science/AI.\n\t- Note that this is not exactly the framing the authors themselves choose: they seem to believe that brain mapping performance can select between different types of models in deciding which models are \"deep\" in the same sense that human language processors are. I'm skeptical on this, but I think the methods and results have value in any case.\n- Good effort at interpreting the results with several creative follow-up experiments (sec. 5: is improved performance reducible to language modeling?; sec. 6: what sorts of representational content are driving improved performance?).\n- The notion of \"deeper understanding\" is never clearly defined *a priori* in the text, except as something that is instantiated in the BookSum task, and something which isn't reducible to language modeling.\n\t- A posteriori, it also seems to be something which yields model representations with improved brain encoding performance—across all tested ROIs!\n\t- For a paper at this conference, however, I don't think this is a critical issue. I'd encourage the authors to follow up this research and test their concept of \"deeper understanding\" by considering alternative tasks other than narrative summarization.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}, {"review_id": "KjND94hvTSB", "reviewer": "Reviewer_m5W3", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This work shows that after finetuning the pretrained language models on a book corpus, the models align better with brains though their language ability is not better. This better alignment seems general to all brain regions and is also shown in two different metrics. After further evaluating the alignment increase on different discourse features, the authors state that the increase in the Characters is the largest.", "summary_of_the_review": "This paper shows that after finetuning the pretrained language models on a book corpus, the models align better with brains though their language ability is not better. Although the results are exciting and inspiring, I think some more work (which should be doable during the rebuttal phase) is needed to make it acceptable.", "strength_and_weaknesses": "Strength:\n\nThis work shows that further training language models on some text corpus can improve their brain alignment, which is exciting and inspiring. This is especially interesting as the text corpus does not contain the data used in the brain data, making this improvement general. Showing that this improvement exists in both brain-alignment metrics and all four models trained makes the results general. The analysis presented by these authors is also interesting, like the discourse feature analysis. In addition, I highly appreciate the share of the code used in the paper and having many results in the supplementary information. \n\nWeaknesses:\n\nTwo significant issues need to be further improved for this paper. First, more results are required to make the results even stronger and more general. Second, the writing of this paper is confusing and needs some work, which will be mentioned in the next part. There are also several minor issues that would be great to be addressed.\n\nFor the result issue, the biggest question is whether these four base models tested in the paper are strong candidates compared to the current SOTA brain model, like GPT-2. In both Schrimpf et al. 2021 and Goldstein et al. 2022, GPT-2 seems to be the best language model for the brain. What is the 20vs20 and Pearson correlation for this model on this dataset? Even if it may not be possible for GPT-2 to be finetuned on the book corpus, I think having the performance of that model is still important to tell us whether the improved performance leads to a new STOA.\n\nFor the minor issues, I will start with interpreting the 20vs20 metric. From my understanding, this metric can also be applied to the ground truth fMRI responses, which may lead to some noise ceiling measures for this metric. But I don’t see any discussion or results on this. Do I misunderstand something? \n\nThen, about the results of the Character feature. Can the authors provide more measures to compare this feature group to others? For example, how many tokens are in this feature group and in other groups? Are the tokens belonging to different groups also distributed in different ways? I think this type of comparison is needed to show that the larger increase is not due to some other reasons.\nFinally, I wonder whether the authors tried the same finetuning on other datasets to show the specific usage of long context in the book corpus, like showing that finetuning on some typical corpus will not work.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}, {"review_id": "9cjLn3YXLO7", "reviewer": "Reviewer_ajVU", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper investigates the alignment between human brains and pretrained large language models (LMs), by learning linear mappings between LM representations and fMRI data on the same corpus. In addition, the authors propose to finetune on narrative text, and find that the alignment can be improved across models by such a process.", "summary_of_the_review": "This paper presents an intuitive finetuning strategy (i.e., finetuning on narrative text) to improve the alignment between LM representations and fMRI. Through comprehensive experiments, this paper presents fairly convincing findings on the linear alignment between pretrained language models and human brains, suggesting that further pretraining/finetuning on narrative corpora improves the alignment. There also exists some points that are not completely convincing, including lack of baselines for controlled experiment and the choices on the investigated NLP model. While I hereby give a score of 5, I am willing to raise my rating if the authors can address some or all of my concerns.", "strength_and_weaknesses": "\n## Strengths\n\n- This paper presents a reasonable finetuning strategy (i.e., finetuning on narrative text) to improve the alignment between LM representations and fMRI.\n\n- Through comprehensive experiments, this paper presents fairly convincing findings on the linear alignment between pretrained language models and human brains, suggesting that further pretraining/finetuning on narrative corpora improves the alignment. \n\n- The paper is generally clear and easy to follow.\n\n## Weaknesses \n\nThe major weakness of this paper, in my opinion, is that it doesn't involve sufficient baselines to support the claim. For example, I would expect a baseline that finetunes the language model on non-narrative corpora, e.g., Wikipedia or scientific papers: comparing the results with those by the BookSum models in this paper would make the claims much stronger.\n\nIn addition, the following points are not clear enough to me.\n\n- According to Wehbe et al. (2014), all 8 participants have read the Harry Potter books or watched the movies prior to the experiment. I'm not sure if the narrative models are in similar situations as well, i.e., have seen sufficient text relevant to Harry Potter in the pretraining process. If not, the story will be subtly slightly different from its current shape. I would suggest the authors to clarify this point.\n\n- Why did the authors choose models of a different size for BigBird? \nWhile all other investigated models have 12 layers, the selected BigBird model has 32 layers. There exists a [12-layer BigBird model](https://huggingface.co/google/bigbird-base-trivia-itc), and I wonder why the authors did not use it instead for a more uniform setting across investigated language models.\n\n  It's somewhat a commonsense in the NLP community that the base models (named following the HuggingFace convention for models that usually have 12 layers; not to be confused with the usage of \"base\" in this paper) usually have different behaviors from the large models (which usually have 24 or more layers) [1, *inter alia*]. While it's very interesting to see that the alignment get worse when the the depth of layer increases, I look forward to more evidences from experiments with other deeper models (e.g., `t5-large`).\n\nI look forward to the authors' clarification on the above points. \n\n[1] Tenney et al., ICLR 2019. https://openreview.net/pdf?id=SJzSgnRcKX\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Accept: notable-top-25%