Scaffolded input promotes atomic organization in the recurrent neural network language model (2021)

Abstract: The recurrent neural network (RNN) language model is a powerful tool for learning arbitrary sequential dependencies in language data. Despite its enormous success in representing lexical sequences, little is known about the quality of the lexical representations that it acquires. In this work, we conjecture that it is straightforward to extract lexical representations (i.e. static word embeddings) from an RNN, but that the amount of semantic information that is encoded is limited when lexical items in the training data provide redundant semantic information. We conceptualize this limitation of the RNN as a failure to learn atomic internal states - states which capture information relevant to single word types without being influenced by redundant information provided by words with which they co-occur. Using a corpus of artificial language, we verify that redundancy in the training data yields non-atomic internal states, and propose a novel method for inducing atomic internal states. We show that 1) our method successfully induces atomic internal organization in controlled experiments, and 2) under more realistic conditions in which the training consists of child-directed language, application of our method improves the performance of lexical representations on a downstream semantic categorization task.

## introduction in order to reproduce the full range of human language behavior, a language system needs to be able to represent sequences of words, while simultaneously being capable of forming semantically organized representations of individual words. to date, researchers have had difficulty specifying a model or system that is good at both tasks. on the one hand, static word embedding models are better at representing individual word meanings @xcite , but do not naturally lend themselves to multi-word sequences. on the other hand, language models based on recurrent neural networks @xcite or transformers @xcite are well-suited for dealing with word sequences, but usually perform worse than static word embedding models on word-level tasks @xcite and are challenging to extract word embeddings from @xcite . in this paper, we diagnose why recurrent neural networks (rnns) learn sub-optimal representations of individual words. in short, we argue that the language modeling objective encodes information relevant to individual words by spreading it across representations of neighboring words when they provide partially redundant information, a phenomenon related to entanglement @xcite . although this supports prediction of words in context, this impedes the formation of stand-alone lexical representations that are useful for tasks requiring de-contextualized information about individual words. in this work, we propose a simple strategy for protecting against this kind of context-sensitivity, enabling the learning of representations that, while still good for sequence prediction, are also better at a lexical semantic tasks. first, we demonstrate the extent of entanglement in rnns using a set of artificial corpora consisting of words from sets of (artificially constructed) semantic categories. we show that the extent of entanglement between lexical representations learned by rnns is caused by the availability of partially correlated contextual cues. next, we demonstrate that pre-training an rnn on a sample of the same artificial grammar without correlated cues protects it against entanglement when trained subsequently on input with correlated cues. contrary to expectations based on catastrophic forgetting @xcite , biasing the network to start with atomic organization helped the rnn retain that organization when faced with language with redundant information. lastly, we examined whether our pre-training strategy could be applied to a corpus of natural language. we conclude by discussing implications of this work, both for language models and for theories of human language learning and representation. ## discussion language users represent both the meanings of sentences and individual words @xcite . in this work, we examined whether a recurrent neural network (rnn) language model can do the same. because next-word prediction requires extreme sensitivity to lexical context, we hypothesized the rnn acquires lexical representations that are not suitable for downstream tasks in which lexical representations must stand on their own -without being supported by the contexts in which they occurred during training. in section 3.1, we compared the internal organization of rnn language models trained on samples from an artificial grammar, and showed that nonatomic lexical representations are acquired by models trained on input in which neighboring items provide redundant information about an upcoming word. instead, the rnn learned to offload information relevant to one set of words to the representations of neighboring words -resulting in entanglement between the two words' representations. this entanglement makes the rnn good at next-word prediction, but lowers the effectiveness of its lexical representations for tasks that operate on single words. next, we argued that an inductive constraint might protect lexical representations from irrelevant (i.e. redundant) information provided by neighboring words. we reasoned that by establishing processing dynamics during pretraining on input without redundant information, the rnn could no longer minimize the language modeling objective by predicting redundant items, and would therefore "learn to ignore" the redundant information provided by neighboring items. in section 3.2, we observed that our pre-training strategy could induce a long-lasting inductive bias for acquiring lexical representations that are robust two groups of 10 rnns were trained on approximately 3m steps of child-directed language randomly sampled from ao-childes, but with different initial knowledge and experience. in the atomicity-bias condition, the rnns were first trained with 37k steps on simple artificial 2-item sequences (of the form x i y i ) to induce perfect knowledge of the target semantic category structure and with perfect atomicity. in the baseline condition, the rnns were first trained with 37k steps on sequences of child-directed input -which neither provides perfect knowledge of the target task nor strong atomicity. right panel: results of experiment 3b. rnns were first trained for 1m steps (13 epochs) on ao-childes, and then separated into one of two groups (time of separation is indicated by ). in the atomicity-bias condition ( ), all the parameters of an rnn were randomly re-initialized except for lexical representations of probe words. in the baseline condition ( ), training on ao-childes continued uninterrupted for another 2m steps (27 epochs). to conclude, our demonstrations reveal a fundamental trade-off between sequence-level and wordlevel learning dynamics; while greater utilisation of contextual information is useful for modeling sequences, individual lexical items become increasingly reliant on information provided elsewhere in the system and are thus less useful as input to downstream lexical tasks. does this mean we are better off using different models for sequence versus lexical tasks? this is unsatisfying from the perspective of a unified account of language acquisition. an alternative is to extend models operating on sequences with inductive biases that encourage more atomic organization. our demonstration in section 3.3 showed that induction of an atomicitybias is a promising first step in that direction. finally, it is possible that the lack of atomicity is a limitation unique to recurrent or other networks that force sequential order during processing of word sequences. work by @xcite suggests that word embeddings extracted from non-recurrent transformer-based language models, which represent word order in the parameters of the model rather than being strictly enforced as a result of sequential processing, outperform static word embeddings on a variety of lexical tasks. therefore, an important next step is to quantify and compare atomicity of language models that vary in how strictly word order is enforced during processing (e.g. recurrence vs. self-attention).