The Role of Learning Regime, Architecture and Dataset Structure on Systematic Generalization in Simple Neural Networks

1 INTRODUCTION . Humans frequently display the ability to systematically generalize , that is , to leverage specific learning experiences in diverse new settings ( Lake et al. , 2019 ) . For instance , exploiting the approximate compositionality of natural language , humans can combine a finite set of words or phonemes into a near-infinite set of sentences , words , and meanings . Someone who understands “ brown dog ” and “ black cat ” also likely understands “ brown cat , ” to take one example from Szabó ( 2012 ) . The result is that a human ’ s ability to reason about situations or phenomena extends far beyond their ability to directly experience and learn from all such situations or phenomena . Deep learning techniques have made great strides in tasks like machine translation and language prediction , providing proof of principle that they can succeed in quasi-compositional domains . However , these methods are typically data hungry and the same networks often fail to generalize in even simple settings when training data are scarce ( Lake & Baroni , 2018 ; Lake et al. , 2019 ) . Empirically , the degree of systematicity in deep networks is influenced by many factors . One possibility is that the learning dynamics in a deep network could impart an implicit inductive bias toward systematic structure ( Hupkes et al. , 2020 ) ; however , a number of studies have identified situations where depth alone is insufficient for structured generalization ( Lake & Baroni , 2018 ; Niklasson & Sharkey , 1992 ; Pollack , 1990 ; Phillips & Wiles , 1993 ) . Another significant factor is architectural modularity , which can enable a system to generalize when modules are appropriately configured ( Vani et al. , 2021 ; Phillips , 1995 ) . However , identifying the right modularity through learning remains challenging ( Bahdanau et al. , 2019 ) . A third possibility builds on Iterated Learning ( IL ) , a method in which generations of agents train briefly on a ‘ language ’ produced by their parent , and then generate a new language for their child ( Kalish et al. , 2007 ) . If systematic components are easier to learn than non-systematic ones , this process can successively refine a language toward a systematic structure , a process which has been hypothesized to be the cause of the compositional nature of natural language ( Kirby , 2001 ; Kirby et al. , 2008 ) . In spite of these ( and many other ) possibilities for improving systematicity ( Hupkes et al. , 2020 ) , it remains unclear when standard deep neural networks will exhibit systematic generalization ( Dankers et al. , 2021 ) , reflecting a long-standing theoretical debate stretching back to the first wave of connectionist deep networks ( Rumelhart & McClelland , 1986 ; Fodor & Pylyshyn , 1988 ; Smolensky , 1991 ; 1990 ; Hadley , 1994 ) . In this work we theoretically investigate the acquisition of systematic knowledge by simple neural networks ( NNs ) . We introduce a simple space of datasets that contain systematic and non-systematic features , and examine the impact of implicit biases , architectural modularity , and iterated learning on the learned input-output mappings of shallow and deep linear networks . In particular , • We derive exact training dynamics for shallow and deep linear networks as a function of the dataset parameters . • We show that for all datasets in the space , despite the possibility of learning a fully systematic mapping , neither shallow nor deep networks do so under gradient descent dynamics . • We show that modular network architectures can learn fully systematic network mappings , but only when the modularity segregates systematic and non-systematic features . • We show that iterated learning can converge to a fully systematic ‘ language ’ when combined with the weaker architectural constraint of output modularity . In Section 7 we consider how our findings , which rely on a simplified setting for mathematical tractability , generalize to more complicated datasets and non-linear architectures by training a convolutional neural network to label handwritten digits between 0 and 999 . Overall , our results help clarify the diverse factors impacting systematic behavior in neural networks , and suggest that iterated learning can improve systematicity . 2 BACKGROUND . Systematic generalization has been proposed as a key feature of intelligent learning agents which can generalize to novel stimuli in their environment ( Hockett & Hockett , 1960 ; Fodor & Pylyshyn , 1988 ; Hadley , 1993 ; Kirby et al. , 2015 ; Lake et al. , 2017 ) . In particular , the closely related concept of compositional structure has been shown to have benefits for both learning speed ( Ren et al. , 2019 ) and generalizability ( Lazaridou et al. , 2018 ) . There are , however , counter-examples which find only a weak correlation between compositionality and generalization ( Andreas , 2018 ) or learning speed ( Kharitonov & Baroni , 2020 ) . In most cases neural networks do not manage to generalize systematically ( Ruis et al. , 2020 ) , or systematic generalization occurs only with the addition of modular architectures , explicit regularizers or a degree of supervision of the learned features . Neural Module Networks ( NMNs ) ( Andreas et al. , 2016 ; Hu et al. , 2017 ; 2018 ) have become one successful method of creating network architectures which generalize systematically . By ( jointly ) training individual neural modules on particular subsets of data or to perform particular subtasks , the modules will specialize . These modules can then be combined in new ways when an unseen datapoint is input to the model . Thus , through the composition of the modules , the model will systematically generalize , assuming that the correct modules can be structured together . Bahdanau et al . ( 2019 ) show , however , that strong regularizers are required for the correct module structures to be learned . Thus , without regularizers , compositional mappings do not emerge with NMNs . Iterated learning ( IL ) approaches suggest that systematicity can emerge over generations of agents that learn from each other . In some works , based primarily on the topological structure of language ( Brighton & Kirby , 2006 ) , compositional structure does emerge from IL ( Ren et al. , 2019 ) . When paired with NMNs , IL has previously been used to refine the module structures to be compositional ( Vani et al. , 2021 ) by treating the module structure as an output “ language ” . Then each “ word ” in the language represents a particular module composed to form a “ sentence ” which describes the model architecture ( the relative arrangement of the modules in Polish notation ) . Thus , by phrasing the problem as a language learning task , IL is able to refine this language of architectures to become modular . This work also highlights the fact that IL is applicable outside of traditional language learning tasks . IL has also been successfully used with NNs as part of a larger pipeline . In particular Seeded Iterated Learning ( SIL ) ( Lu et al. , 2020a ) uses IL to counter language drift . Language drift occurs when a pretrained language model is then used to complete a language based task . While training on the task , the model begins to lose structure in the learned language that is not useful for performing the task . By training a “ teacher ” model on task completion and then using the teacher to supervise a pretrained “ student ” , Lu et al . ( 2020a ) show that the student network is able to learn to complete the task while maintaining the initially learned language . SIL is extended by Lu et al . ( 2020b ) to include supervised selfplay in the teacher task-learning step . This is known as Supervised SIL ( SSIL ) . Thus , parts of the data used to pretrain the teacher are also replayed while the teacher learns to perform the task . This also aids in avoiding language drift . While these previous works reflect the influence of certain initial data properties , regularizers or complex architectures on the emergence of compositional structure , the necessity of structured architectures with modular designs remains unclear . 3 A SPACE OF DATASETS WITH COMBINATORIAL SYSTEMATICITY . The notion of systematic generalization is broad , and has been assessed using a variety of datasets and paradigms ( Hupkes et al. , 2020 ) . Here we introduce a simple setting reminiscent of learning to refer to objects in an environment , as depicted in Fig . 1 . For instance , we might see a particular car ( Fig . 1a ) , extracting various perceptual input features such as whether it has a stripe , is red , or has a particular hood ornament . This scenario affords several ways in which we could refer to the car . We could systematically map individual input features to individual outputs by saying a sentence like “ My car is the red one with the white stripe ; ” or we could choose a single word to refer to some combination of properties , saying something like “ My car is the Ferrari. ” This task has a type of combinatorial systematic structure , because individual input features can link independently to individual output features . More generally , an agent first observes an object and forms an internal “ logical form ” . The task is then to map from this logical form to a “ phonetic form ” by providing a name for each object ( Brighton & Kirby , 2006 ) . As illustrated in this example , real world settings often allow redundant expressions and mappings , such that the same object could be individuated with a variety of expressions that vary in systematicity . To formalize this setting , we define a parametric space of datasets with input and output matrices X = [ Ωx Γx ] T and Y = [ Ωy Γy ] T respectively , where nx , ny , kx , ky , r ∈ R+ are the parameters that define a specific dataset . Figure 1b visualizes one dataset in the space . The systematic input feature matrix Ωx ∈ { −1 , 1 } nx×2 nx consists of all binary patterns with nx bits . Here nx is a key parameter determining the number of bits in the systematic input structure . Overall , the dataset contains 2nx examples . The systematic output feature matrix Ωy ∈ { −1 , 1 } ny×2 nx is a sampling of ny features ( rows ) from Ωx , and is the systematic component of the output matrix . Intuitively , the binary combinatorial structure in Ωx and Ωy is meant to reflect systematic mappings from input features ( the color red or blue ) to output features ( the words “ red ” or “ blue ” ) . Next , the non-systematic input feature matrix Γx = [ rI1 ... rIkx ] consists of kx scaled identity matrices , Ii ∈ { 0 , 1 } 2 nx×2nx . Similarly , the non-systematic output matrix Γy = [ rI1 ... rIky ] has ky scaled identity matrices , with scale factor r. These identity matrices provide a single feature for each pattern which is only on for that pattern . Intuitively , these features are meant to reflect idiosyncratic features ( like hood ornament , a distinctive scratch , or a proper noun name ) or features reflecting specific nonlinear combinations of systematic features ( red-striped-2-door ) . Together kx , ky and r control the frequency and intensity ( for example the size or perceptual prominence of the brand ’ s logo ) of the non-systematic features , which are both factors that can promote non-compositional language being used by humans ( Rogers et al. , 2004 ) . This space of datasets is consistent with numerical notions of compositionality in previous works ( Andreas , 2018 ) , which define compositionality as a homomorphism between the observation space and the naming space ( since the input-output mappings are linear they are homomorphic ) . The amount of systematic structure can be titrated by adjusting nx and ny ; and the prevalence of nonsystematic structure by adjusting kx , ky , and r. Importantly , and novel to our analysis , datasets in this space allow redundant solutions : the systematic output features can be generated based on systematic input features alone , but they can equally be generated using non-systematic features alone , or some mixture of the two . Exploiting this fact , we now ask how reliance on systematic or non-systematic structure is influenced by implicit biases , architecture , and learning regime .

Summary: The submission studies what properties of a training setup impact systematicity of a neural network. The theoretical setting uses the deep linear neural networks setup of Saxe et al. (2014; 2019) and derives the effect of the degree of systematicity of a dataset on a model's systematic behavior. The theoretical results demonstrate that systematicity is difficult to guarantee, but that structural modularity and iterated learning are helpful to improve systematicity.