Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 5: Marginally below acceptance threshold
review_content: This paper describes experiments that inject linguistic information (for example dependency structures) into BERT, then measure improvements in correlation with FMRI measurements of humans reading an underlying sentence (which is also analyzed by BERT). Linguistic information is incorporated by biasing attention heads to line up with dependency (or other) structures. 

Positives about the paper: it's an interesting experiment to try, and an important direction of work.

Negatives:

* It's a somewhat small increment over previous work, not sure it merits a full conference paper. As it stands the paper presents the approach and results, with little inspection of why improvements are seen. I would like the authors to go much deeper with the analysis. Are there particular syntactic constructions that are being better modeled? Is the new model much more sensitive to long range dependencies, as found in syntactic structures? Are particular classes of words effected more than others? Answering these questions will be challenging but would add a lot to the paper.

* Most importantly, the evaluation metrics are unclear. The critical sentence in the paper is "To evaluate the regression models, Pearson’s correlation coefficient between the predicted and the corresponding heldout true sentence or word representations is computed". This is a terse description of a critical part of the approach, and I can't make sense of it.

Part of my unease about the evaluation is the following. The matrix $D_{fr}$ is the output from BERT. Importantly in The definition of L_{ifr} this matrix is predicted from the "brain" matrix B_i. If $D_{fr}$ was all zeros it would be trivially predictable (and hence gameable). In the original Gauthier and Levy paper they appear to use metrics in addition to MSE. In this paper some variant of Pearson's correlation coefficient is used - but I can't understand what exactly this is, and my worry is that it is trivially gameable in the same way as MSE.  
review_title: Review of "DOES INJECTING LINGUISTIC STRUCTURE INTO LANGUAGE MODELS LEAD TO BETTER ALIGNMENT WITH BRAIN RECORDINGS?"

final_score: rating: 7: Good paper, accept
review_content: An interesting paper that discusses whether injecting three types of syntactic and semantic formalisms lead to better alignment with how language is processed in the brain. The authors conduct experiments with the BERT model and two fMRI datasets and show that including linguistic structure through fine-tuning can improve brain decoding performance.

The paper would be improved by experimenting with language models other than BERT, as it is not clear at the moment whether the produced results are generalizable to different language models or are BERT-specific. For example, additional experiments with AlBert, distilBert and RoBerta would provide additional insights on the effect of size of the model, in terms of the number of parameters. Comparison of Bert to GPT and XLNet would emphasize the advantages/disadvantages of autoencoder-based vs autoregressive models and could potentially provide additional insight on how attention is represented in human brain.

It would also be interesting to read a discussion of semantic analysis, as currently the paper concentrates the most on syntactic formalism as represented in both BERT and fMRI data. Specifically, it would be interesting to know if the injection of syntax impacts the semantic representations. One of the possible methods to measure that would be probing for semantic classes (as in Yaghoobzadeh et al., 2019. Probing for Semantic Classes)

review_title: Does it work the same way on other LM?

final_score: rating: 7: Good paper, accept
review_content: This paper tests whether fine-tuning large pre-trained language models
(LMs) with structural information can increase the correlation between
these representations and the representations of brain activity
measured while processing the same stimuli. The injection of the
structural information is done through fine-tuning of the pre-trained
model by "guided attention", which makes use of binary relations
between the words according to three different syntactic or semantic
formalisms. The authors map the brain activity to each of the
alternative LM representations via a regression model, and measure the
alignment by using correlation between the predicted (from brain
activity) and actual output of the alternative models. The results
show that under certain conditions representations learned through
guided attention aligns better with the representations of brain
activity.

In general the investigates an interesting question which may be
(eventually) relevant to both understanding the way humans process
language, and possibly building better computational models. The
method followed in the study is (mostly) sound, and the paper is
written well.

A potential issue with the method is the direction of the prediction
in "brain decoding" regression (section 3.5). Authors predict the
model representations from the "brain representation" (this seems to
be based on earlier studies, but I did not verify). In my opinion the
reverse is more meaningful. Since the invariant quantity in this study
is the representations coming from the brain imaging. This is
important, because the success of the regression is not only about the
amount of information in the predictors, but also simplicity of the
task. Hence, an alteration of the model representations that
simplifies them may result in better predictions, and hence, higher
correlations

Except the above, I have some (mostly minor) comments:

- I would be happier with a bit more explicit discussion of the main
  results. After reading the articles, I am still not sure what to
  take away from the main experiments. The effect on two different
  data sets (also means representations at different levels/units) are
  quite different - not allowing a clear conclusion. Side issues
  discussed (the effects of the use of different formalisms, the
  effect of domain, particular syntactic patterns, content vs.
  function words are also relatively brief and far from being
  conclusive). I think a clearer discussion of the main results, and
  investigation of reasons for the discrepancy between the data sets
  would make the paper stronger.

- It would help if the data is explained slightly better.
  Particularly, it would make the paper more self contained if the
  authors specify whether any of the data sets (section 3.3) had
  automatic annotation. On a somewhat related note, comparisons
  between the formalisms seem to correlate with the data sizes, which
  is not pointed out in the paper.

- A few language/typography issues/suggestions: 

    - I am not sure about the ICLR guidelines, but avoiding citations
      in the abstract is a good idea (abstracts should stand alone).
    - Footnote marks should go after punctuation (footnote mark 8)
    - Conclusions line 3: "attention guided" -> "guided attention" ?
    - There are case (normalization) issues in the references:
      "groningen", "erp", "bert" (not exhaustive, a through check is
      recommended).

review_title: An interesting paper with an interdisciplinary appeal. In general well-executed, well-written study, with some (minor) issues.

final_score: rating: 6: Marginally above acceptance threshold
review_content: Summary of paper: the authors explore adding a soft structural attention constraint to BERT, by penalizing attention weights that are substantially different from a head–dependent "adjacency" matrix derived from dependency parses.  BERT is then fine-tuned with and without ("domain-finetuned") this constraint on corpus data for which fMRI recordings from participants during reading are available. A linear classifier from the final layer of BERT's embedding (mean-pooled) is then learned to the fMRI data.  Within this pipeline, domain-finetuned models are not an improvement over unfinetuned BERT, but fine-tuning with the structural attention constraint improves decoding to fMRI data, especially for word-level data (the Wehbe2014 dataset).

Assessment: this is a nice paper that investigates an intuitive method of incorporating syntax-based, structural soft attention constraints into Transformer encoder models for language.  What makes the contribution fairly distinctive is evaluation on alignment with human fMRI recordings during comprehension of the texts.  The results show improvements in decoding relative to baseline models that involve no fine-tuning and/or domain-adaptation fine-tuning alone (no structural attention constraints), especially for fMRI data that are recorded below the sentence level.  The authors also evaluate the effect of fine-tuning on targeted syntactic evaluations from Marvin & Linzen; the results here are not particularly conclusive.  Overall, this is a potentially solid, if not ground-breaking, contribution.  However, there are a number of technical questions that are left unclear in the submission, and some of the results are cause for some concern.  These concerns need to be addresed in order for the submission to be fully satisfactory.

The single biggest concern is the extraordinarily high word perplexity scores in Table 2 for Wehbe2014 -- which get much, much worse after fine-tuning.  It is important to understand what's going on here in order to make sense of the core potential contribution of the paper, because it's only in the Wehbe2014 dataset where there seem to be appreciable improvements in decoding performance.  I would guess that the high perplexity comes from poor prediction of the proper nouns in the Harry Potter book chapter.  Maybe there needs to be some amount of fine-tuning of the models to the domain of the test-set corpus.  Overall, the paper needs more clarity on why it is only the Wehbe2014 dataset where the perplexity is so high and the fine tuning affects decoding performance so much.

Additional technical questions:

1) How is the split of a word into word pieces handled in the adjacency matrix representing word–word dependencies?

2) How are the adjacency matrix and each head's attention weight matrix converted into a distribution for computing cross-entropy loss?  Are the entries normalized globally? By row? By column?

3) What are the perplexities like for domain-finetuned (no structural attention constraint) BERT?  These are missing from Table 2 (Appendix B), but are potentially important in interpreting your results.

4) What words are pooled over for the Wehbe2014 analyses -- the four words in the 2-second window?

5) Section 4.1 reports that UD and DM finetuned models are significantly better in brain decoding than the un-finetuned baseline, at p<0.0001, but the 95% confidence intervals for subject scores look very different.  And the difference in mean decoding performance for DM finetuning is barely visible.  How are you computing your confidence intervals and your p-values?  Why are they so different, and how are you getting such high confidence in improvements over the unfinetuned baseline here?

6) How do your results compare to those using the best fine-tuning methods from Gauthier & Levy (2019), which involve scrambling the input sentences?

7) Given that in Wehbe2014 each fMRI image corresponds to four words, most of which probably contain both function and content words, how is the content/function word analysis defined and performed?

Additional comments:

* the authors write that "increase in perplexity roughly corresponds to lower brain decoding scores", but this doesn't look consistent with Table 2 and Figure 3.  For Wehbe2014, UCCA data yield the worst decoding accuracy but yield better perplexity than DM data, which yield decoding accuracy only slightly worse than the UD data.  The monotonicity is cleaner for Pereira2018 but the overall differences in decoding performance are much smaller.

review_title: Provocative paper, but several technical concerns

Meta-review: This paper explores the effect on decoding accuracy (predicting hidden representations from fMRI datasets) from fine tuning models by injecting structural bias.  This paper specifically focuses the attention of BERT on syntactic features of the text, which (for one dataset) appears to improve the decoding performance.  The paper's motivation is strong, and complex concepts are communicated clearly.

The review period was very productive.  There were some questions about analyses, and the validity of the statistical tests, but through some very thorough back and forth with the reviewers, this seems to have been resolved.  There is a good amount of analysis done on the resulting language models to try and determine the impact of finetuning or attention on the models. However, the results on the fMRI two datasets appear to be very different, and it's unclear why (and isn't clearly related back to the extensive language model analyses).  We would have liked to have seen a more thorough analysis of the stark difference in performance, and some convincing explanations for the difference based on the analyses.  


P.s. A minor point, but the Wehbe paper uses Chapter 9 of Harry potter, not chapter 2.

Decision: Reject