Representation and Pre-Activation of Lexical-Semantic Knowledge in Neural Language Models (2021)

Abstract: In this paper, we perform a systematic analysis of how closely the intermediate layers from LSTM and trans former language models correspond to human semantic knowledge. Furthermore, in order to make more meaningful comparisons with theories of human language comprehension in psycholinguistics, we focus on two key stages where the meaning of a particular target word may arise: immediately before the word’s presentation to the model (comparable to forward inferencing), and immediately after the word token has been input into the network. Our results indicate that the transformer models are better at capturing semantic knowledge relating to lexical concepts, both during word prediction and when retention is required.

## introduction a wide variety of natural language processing (nlp) tasks have been improved dramatically by the introduction of lstm @xcite and transformer-based @xcite neural language models, which can encode the meanings of sentences in such a way that facilitates a range of language tasks @xcite @xcite . furthermore, both recurrent and transformer networks have been shown to capture a broad range of semantic phenomena and syntactic structure @xcite @xcite @xcite @xcite @xcite . although such models clearly learn aspects of lexical semantics, it remains unclear whether and how these networks capture semantic features associated with conceptual meaning. some work has demonstrated that word embeddings do reflect conceptual knowledge captured by property norming studies @xcite @xcite , in which human participants produce verbalisable properties for concepts, such as is green or is an amphibian for concepts such as frog @xcite . such features correspond to stereotypic tacit assumptions @xcite ; common-sense knowledge we have about the real world. there is some evidence that language models implicitly encode such knowledge @xcite ; however, coverage of different types of knowledge may be inconsistent, with evidence to suggest that these models fail to capture some types of semantic knowledge such as visual perceptual information @xcite , as well as questions about the completeness of such empirical studies @xcite @xcite . in general, there has been only limited work that attempts to investigate whether these neural language models activate lexico-semantic knowledge similarly to humans, further restricted by the fact that such knowledge probing is only performed on latent representations that have received the target concept, ignoring theories of language comprehension and acquisition that emphasise the importance of prediction @xcite @xcite . in this paper, we contribute to the analysis of neural language models by evaluating latent semantic knowledge present in the activation patterns extracted from their intermediate layers. by performing a layer-by-layer analysis, we can uncover how the network composes such meaning as the information propagates through the network, eventually emerging as a rich representation of semantic features that facilitates conditional next word prediction, which is directly dependent on the past knowledge. we perform our layer probing analysis at two temporal modalities. that is, we investigate the hidden layer activations of the nnlms both before the concept word occurs (which facilitates next word prediction), and after the concept word has been explicitly given to the model. in this way, we determine how richly these latent representations capture real-world perceptual and encyclopaedic knowledge commonly associated with human conceptual meaning. ## related work the recent popularity of interpretability in nlp has resulted in strong progress on understanding both recurrent @xcite and transformerbased networks @xcite . a number of these studies rely on probing techniques, where supervised models are trained to predict specific linguistic phenomena from model activations @xcite @xcite . there exists some work that analyses semantic knowledge in such networks, though to date this has been more limited than investigations of syntax. @xcite focused on the recurrent layers of lstm and gru networks and attempted to interpret their semantic content by using a set of decoders to predict the previous network inputs. @xcite devised a set of psycholinguistic diagnostic tasks to evaluate language understanding in bert, demonstrating that some phenomena such as semantic role labelling and event knowledge are well-inferred, though others such as negation are less so. similar to our work, @xcite mined sentences with words in context to demonstrate that context representations are highly anisotropic, while @xcite built static word embeddings from contextual representations using pooling methods, analysing their performance on semantic similarity benchmarks. language models have also been successfully employed for predicting activation patterns in the brain during human language comprehension @xcite . such work is particularly relevant from the perspective of predictive coding theories of human language com-prehension @xcite , which posits that high-level representations of an unfolding utterance facilitate active prediction of subsequent lexical content in the sentence. neurolinguistic studies provide evidence that such predictions can be of wordform identity @xcite , or of the semantic features that are expected for the upcoming word (for example, whether the upcoming word is animate or not; @xcite . ## neural language models due to the compatibility issues, we limit our investigation to left-to-right language models that are trained to perform conditional next word prediction, as other sota models such as bert @xcite fail to capture the desired criterion that facilities similar mechanisms in language comprehension. for the lstm-based network, we make use of a very large-scale and influential neural language model developed by @xcite , which we refer to as jlm foot_0 . the model's architecture consists of character-level embeddings with cnns, followed by a two-layer lstm with projection layers to reduce dimensionality and a final linear layer with softmax activation. the vocabulary of the output layer consists of 800000 words, and the model is trained using the one billion word corpus @xcite . for the transformer-based model, we make use of the gpt-2 (345m) model @xcite , which consists of 24 multi-head attention layers. ## evaluation tasks for our empirical analysis, we first analyse these layers on classic intrinsic benchmarks that determine their ability to explain human semantic judgments scores on word association, to first determine how well these networks capture the semantic content of the word. we then probe these layers to determine whether they capture a rich set of semantic features related to upcoming concepts and whether such representations are retained by the network for functional use on the prediction task. ## conclusion in this paper, we present a novel approach to gaining a better understanding of the kinds of semantic information encoded within the layers of largescale language models. our analysis allows us to peer inside the hidden state representations of neural language models, and examine how semantically relevant information is encoded in each layer of the networks. we examine the language models on their ability to capture semantic meaning from two perspectives, when the network is predicting the target word, and when the target word is the most recent input. the results demonstrate that the transformer model is much better at capturing attributive features than the lstm model, whilst both models are able to retain rich semantic representations of the concept after the concept has been given to the network.