Semantic Training Signals Promote Hierarchical Syntactic Generalization in Transformers (2024)

Abstract: Neural networks without hierarchical biases often struggle to learn linguistic rules that come naturally to humans. However, neural networks are trained primarily on form alone, while children acquiring language additionally receive data about meaning. Would neural networks generalize more like humans when trained on both form and meaning? We investigate this by examining if Transformers—neural networks without a hierarchical bias—better achieve hierarchical generalization when trained on both form and meaning compared to when trained on form alone. Our results show that Transformers trained on form and meaning do favor the hierarchical generalization more than those trained on form alone, suggesting that statistical learners without hierarchical biases can leverage semantic training signals to bootstrap hierarchical syntactic generalization.

## introduction language learners encounter sentences through their surface forms: linear sequences of words. however, syntactic rules are sensitive to sentences' underlying hierarchical structure. what evidence lets learners determine that syntactic rules operate on hierarchical structure, rather than linear order? some contend that the evidence children receive is insufficient for a learner without a hierarchical bias to generalize hierarchically @xcite @xcite ). an alternative hypothesis (e.g., @xcite is that learners require no innate hierarchical bias: the input children get includes sufficient cues for hierarchical generalization. both sides, however, tacitly assume that the data relevant for hierarchical generalization is form alone (i.e., words and their linear order), rather than form and meaning. since meanings involve hierarchical dependencies which often correspond to syntactic structure @xcite , they may provide additional cues to the hierarchical syntactic generalization. the rise of neural networks seems to suggest that the focus on form is warranted: networks trained on form alone perform well on syntactic evaluations (e.g., @xcite @xcite @xcite . however, when networks' input aligns more closely with the sentences children get, models fail to generalize hierarchically, suggesting that attaining hierarchical generalization from form alone requires stronger priors than those of standard neural architectures @xcite . in this work, we test the hypothesis that learners without a hierarchical bias can generalize hierarchically when trained on form and meaning. we train transformers @xcite , an architecture known to prefer linear rules @xcite , to translate form to meaning, then test for hierarchical generalization. following @xcite , our testbed for hierarchical generalization is yes/no question formation, exemplified by the relationship between declarative sentence (1a) and yes/no question (1b). we train models on question formation data like (1) which is consistent with a hierarchical and a linear rule (see figure 1 ). to test for hierarchical generalization, we evaluate models on examples like (2), where the hierarchical rule predicts (2a), and the linear rule, ( 2b ). we find that transformers trained on form and meaning display stronger preferences for the hierarchical rule than transformers trained on form alone. our results support the hypothesis that semantic training signals help statistical learners without hierarchical biases learn hierarchical syntactic rules. 1 2 background meaning representations are not observable in the mind. forms, however, are observable. this asymmetry makes reasoning about the effect of form on generalization simpler than reasoning about meaning's effect by lessening the need to make assumptions about unobservable representations. focusing on forms, @xcite observes that although english speaking adults acquire the hierarchical rule for yes/no question formation, children's input likely lacks the evidence ruling out the linear rule. @xcite conjectures that even a child who never encounters such disambiguating examples (e.g., (2a)) would generalize hierarchically, and argues that a innate hierarchical bias is thus necessary. empirically, @xcite find that children do behave consistently with the hierarchical rule, and rarely with the linear rule @xcite , while disambiguating evidence is very uncommon in children's input @xcite . though such work makes a hierarchical bias seem necessary for child-like generalization, domain general biases may suffice for hierarchical generalization. to explore this possibility, several studies have investigated how artificial learners generalize from form alone. some argue that their results support an innate hierarchical bias @xcite , while others argue against this conclusion @xcite @xcite , and still others do not take a strong stance @xcite @xcite . the direct precursors to our work also vary in their conclusions. @xcite and @xcite show that neural networks with-out hierarchical biases trained on form alone in a sequence-to-sequence setup generalize to the linear rule of question formation. these results support the claim that hierarchical generalization requires a hierarchical bias. however, @xcite and @xcite find that models trained in a language modeling setup on the @xcite and @xcite data generalize linearly early on, but 'grok' the hierarchical generalization after training far beyond saturation on in-domain performance. though the role of semantic information in the acquisition of syntax has long been theorized (cf. @xcite ; @xcite ), fewer studies have explored semantic signals' effect on hierarchical generalization. studying children, @xcite find evidence against @xcite hypothesis about how meanings aid hierarchical generalization, but leave open the possibility that meanings help in other ways. @xcite find that visual context aided the acquisition of constituent structure in adult learners of an artificial language, though no more so than adding explicit cues to constituent structure to the forms. this finding with adults, however, does not address how children generalize hierarchically during first language acquisition. using computational modeling, @xcite show that networks with built-in linguistic knowledge generalize hierarchically when trained to map meaning to form, and @xcite explore how semantic training signals help a learner acquire syntactic rules but use statistical modeling techniques that presuppose that syntactic structures must be hierarchical. it is an open question if learners without such built-in knowledge generalize hierarchically when trained on both form and meaning. the hypothesis we test in this work is in the spirit of the semantic bootstrapping hypothesis: that children leverage sentences paired with structured meaning representations to acquire syntactic rules @xcite . in this work, we generalize semantic bootstrapping to the problem of determining that syntactic rules must be sensitive to hierarchical structure rather than linear order. ## experiments in this work, we train models to form yes/no questions in two ways. in exp. 1 (section 4), we train neural networks in a sequence-to-sequence setup on the objective of translating declarative sentences to their yes/no question counterparts. we use this setup to enable comparison to @xcite and @xcite . we then test if models' generalization behavior is more consistent with the linear or the hierarchical rule. in exp. 2 (section 5) we explore grokking: we train models longer and track how training on form and meaning changes models' training dynamics. we additionally train neural networks on the task of language modeling (predicting the next word at every point in a sentence), since grokking may depend on this training objective @xcite . in exp. 3 (sections 6 and 7) we vary the representation of meaning and the translation task to investigate several possible causes of the benefit of training models to map form to meaning. crucially, all the training instances for the question formation task are consistent with both the hierar-chical and the linear generalizations (figure 1 ). to evaluate what models learn from the training data, we test models on examples like (2), for which the hierarchical rule produces well-formed questions, like (2a), while the linear rule produces ill-formed questions, like (2b). ## experiment 2: grokking recent work suggests that transformer models display structural grokking: when trained past saturation on in-domain accuracy, out-of-domain generalization continues to improve, and eventually hierarchical generalization is achieved @xcite . this raises the possibility that early stopping caused the lack of hierarchical generalization in experiment 1. to explore the interaction between grokking and semantic training signals, we train models on the datasets from experiment 1 (namely form alone and form & meaning), but far past the point of perfect in-domain accuracy. ## experiment 3: why do the meaning representations help? we found in experiments 1 and 2 that adding meaning to models' training data resulted in a stronger preference for hierarchical generalization. in this experiment, we explore the following questions: we build seven new datasets: three designed to address question (5a) (section 6.1), and four to address question (5b) (sections 6.2 and 6.3). the ιx. newt newt(x) sleep doesn't sleep 6.1 specifics of the meaning representation? one reason that the data in experiments 1 and 2 may lead to hierarchical generalization is that the relationship between the main auxiliary and the first word of the question strongly parallels the relationship between the negation's location in the declarative and in the meaning representation. for instance, though models do not receive questions like (6a), they do receive meanings like (6b), where the relationship between the main auxiliary (don't in ( 6 )) and the negation (¬ in (6b)) closely corresponds to the relationship between that main auxiliary in the declarative and the first word in the question in (6a). see figure 4 for an illustration. to test whether this close parallel is responsible for the models' preference for hierarchical generalization, we remove the negation from the dataset, transition to an event semantic representation where the element of meaning corresponding to the auxiliary is no longer directly at the front of the meaning representation, and introduce the necessary variability in the auxiliary using tense. 8 for example, (7a) translates to ( 7b now the element of meaning corresponding to the auxiliary is no longer directly at the front of the meaning representation, due to '∃e :'. however, the relationship between (7a) and (7b) still bears a close correspondence to the hierarchical question formation rule. as (8) shows, the tense predicate (past) corresponding to the main auxiliary (did) appears near the front of the meaning representation. for this reason, we explore an equivalent semantics in which the tense predicate corresponding to the main auxiliary is located at the end of the meaning representation, as demonstrated in ( 9 ) and ( 10 ), which are translations of ( 7a ) and (8a), respectively. if we see an equivalent boost in preference for hierarchical generalization using this representation scheme, this suggests that the similarity of the hierarchical question formation rule and the placement of negation at the front of the meaning representation is not the source of hierarchical generalization in experiments 1 and 2. ## experiment 3.5: ablating agreement in mccoy et al.'s (2020) grammar, the subject agrees in number with the hierarchically determined main auxiliary, as shown in examples (13), in which the number of the subject (newt vs. newts) determines the form of the auxiliary (does vs. do). ## discussion across three experiments and several training setups, we find that training models to translate from form to meaning enables stronger hierarchical generalization than training on form alone. these results suggest that one avenue to hierarchical generalization in learners without a hierarchical bias is leveraging semantic signals in the training data. for transformers specifically, our results show that transformers generalize more like humans when trained to translate forms to meanings than when trained on form alone. in practice, this takeaway must be tempered by the possibility that the large quantities of form that large language models receive make the benefits of training to translate form to meaning inconsequential. however, recent results suggest that translations from forms to meaning-like representations may provide benefits even to language models trained at scale: @xcite find that large language models trained additionally on code performed better on a entity tracking task. this benefit may be due to the the presence of translations from natural language sentences to code in the training data. further work is necessary to disentangle this possibility from other possible contributing factors. for child acquisition, our results suggest that one possible source of hierarchical generalization is the relationship between forms and meanings, so long as children can construct logical meaning representations either innately or develop this capacity sometime before making hierarchical generalizations. the early development of such logical capabilities is consistent with recent work on the logic in infants (e.g., cesana-arlotti et al. 2018), though this line of research is still in its early stages. though in this work we focus on meaning, it may be that other language-external cues also facilitate hierarchical generalization. for instance, prosody (morgan and demuth, 2014) may also provide a hierarchical signal of a similar nature to meanings, and visual information @xcite may provide information about lexical semantics that is useful to determine how meanings must combine. future work should also better align the input with what children get, perhaps following the lead of @xcite and using a corpus of child-directed speech as model training data, to strengthen the inferences about the innate biases necessary for children to acquire language.