  Neuroscientists evaluate deep neural networks for natural language processing
as possible candidate models for how language is processed in the brain. These
models are often trained without explicit linguistic supervision, but have been
shown to learn some linguistic structure in the absence of such supervision
(Manning et al., 2020), potentially questioning the relevance of symbolic
linguistic theories in modeling such cognitive processes (Warstadt and Bowman,
2020). We evaluate across two fMRI datasets whether language models align
better with brain recordings, if their attention is biased by annotations from
syntactic or semantic formalisms. Using structure from dependency or minimal
recursion semantic annotations, we find alignments improve significantly for
one of the datasets. For another dataset, we see more mixed results. We present
an extensive analysis of these results. Our proposed approach enables the
evaluation of more targeted hypotheses about the composition of meaning in the
brain, expanding the range of possible scientific inferences a neuroscientist
could make, and opens up new opportunities for cross-pollination between
computational neuroscience and linguistics.
