The Role of Learning Regime, Architecture and Dataset Structure on Systematic Generalization in Simple Neural Networks

1 INTRODUCTION . Humans frequently display the ability to systematically generalize , that is , to leverage specific learning experiences in diverse new settings ( Lake et al. , 2019 ) . For instance , exploiting the approximate compositionality of natural language , humans can combine a finite set of words or phonemes into a near-infinite set of sentences , words , and meanings . Someone who understands “ brown dog ” and “ black cat ” also likely understands “ brown cat , ” to take one example from Szabó ( 2012 ) . The result is that a human ’ s ability to reason about situations or phenomena extends far beyond their ability to directly experience and learn from all such situations or phenomena . Deep learning techniques have made great strides in tasks like machine translation and language prediction , providing proof of principle that they can succeed in quasi-compositional domains . However , these methods are typically data hungry and the same networks often fail to generalize in even simple settings when training data are scarce ( Lake & Baroni , 2018 ; Lake et al. , 2019 ) . Empirically , the degree of systematicity in deep networks is influenced by many factors . One possibility is that the learning dynamics in a deep network could impart an implicit inductive bias toward systematic structure ( Hupkes et al. , 2020 ) ; however , a number of studies have identified situations where depth alone is insufficient for structured generalization ( Lake & Baroni , 2018 ; Niklasson & Sharkey , 1992 ; Pollack , 1990 ; Phillips & Wiles , 1993 ) . Another significant factor is architectural modularity , which can enable a system to generalize when modules are appropriately configured ( Vani et al. , 2021 ; Phillips , 1995 ) . However , identifying the right modularity through learning remains challenging ( Bahdanau et al. , 2019 ) . A third possibility builds on Iterated Learning ( IL ) , a method in which generations of agents train briefly on a ‘ language ’ produced by their parent , and then generate a new language for their child ( Kalish et al. , 2007 ) . If systematic components are easier to learn than non-systematic ones , this process can successively refine a language toward a systematic structure , a process which has been hypothesized to be the cause of the compositional nature of natural language ( Kirby , 2001 ; Kirby et al. , 2008 ) . In spite of these ( and many other ) possibilities for improving systematicity ( Hupkes et al. , 2020 ) , it remains unclear when standard deep neural networks will exhibit systematic generalization ( Dankers et al. , 2021 ) , reflecting a long-standing theoretical debate stretching back to the first wave of connectionist deep networks ( Rumelhart & McClelland , 1986 ; Fodor & Pylyshyn , 1988 ; Smolensky , 1991 ; 1990 ; Hadley , 1994 ) . In this work we theoretically investigate the acquisition of systematic knowledge by simple neural networks ( NNs ) . We introduce a simple space of datasets that contain systematic and non-systematic features , and examine the impact of implicit biases , architectural modularity , and iterated learning on the learned input-output mappings of shallow and deep linear networks . In particular , • We derive exact training dynamics for shallow and deep linear networks as a function of the dataset parameters . • We show that for all datasets in the space , despite the possibility of learning a fully systematic mapping , neither shallow nor deep networks do so under gradient descent dynamics . • We show that modular network architectures can learn fully systematic network mappings , but only when the modularity segregates systematic and non-systematic features . • We show that iterated learning can converge to a fully systematic ‘ language ’ when combined with the weaker architectural constraint of output modularity . In Section 7 we consider how our findings , which rely on a simplified setting for mathematical tractability , generalize to more complicated datasets and non-linear architectures by training a convolutional neural network to label handwritten digits between 0 and 999 . Overall , our results help clarify the diverse factors impacting systematic behavior in neural networks , and suggest that iterated learning can improve systematicity . 2 BACKGROUND . Systematic generalization has been proposed as a key feature of intelligent learning agents which can generalize to novel stimuli in their environment ( Hockett & Hockett , 1960 ; Fodor & Pylyshyn , 1988 ; Hadley , 1993 ; Kirby et al. , 2015 ; Lake et al. , 2017 ) . In particular , the closely related concept of compositional structure has been shown to have benefits for both learning speed ( Ren et al. , 2019 ) and generalizability ( Lazaridou et al. , 2018 ) . There are , however , counter-examples which find only a weak correlation between compositionality and generalization ( Andreas , 2018 ) or learning speed ( Kharitonov & Baroni , 2020 ) . In most cases neural networks do not manage to generalize systematically ( Ruis et al. , 2020 ) , or systematic generalization occurs only with the addition of modular architectures , explicit regularizers or a degree of supervision of the learned features . Neural Module Networks ( NMNs ) ( Andreas et al. , 2016 ; Hu et al. , 2017 ; 2018 ) have become one successful method of creating network architectures which generalize systematically . By ( jointly ) training individual neural modules on particular subsets of data or to perform particular subtasks , the modules will specialize . These modules can then be combined in new ways when an unseen datapoint is input to the model . Thus , through the composition of the modules , the model will systematically generalize , assuming that the correct modules can be structured together . Bahdanau et al . ( 2019 ) show , however , that strong regularizers are required for the correct module structures to be learned . Thus , without regularizers , compositional mappings do not emerge with NMNs . Iterated learning ( IL ) approaches suggest that systematicity can emerge over generations of agents that learn from each other . In some works , based primarily on the topological structure of language ( Brighton & Kirby , 2006 ) , compositional structure does emerge from IL ( Ren et al. , 2019 ) . When paired with NMNs , IL has previously been used to refine the module structures to be compositional ( Vani et al. , 2021 ) by treating the module structure as an output “ language ” . Then each “ word ” in the language represents a particular module composed to form a “ sentence ” which describes the model architecture ( the relative arrangement of the modules in Polish notation ) . Thus , by phrasing the problem as a language learning task , IL is able to refine this language of architectures to become modular . This work also highlights the fact that IL is applicable outside of traditional language learning tasks . IL has also been successfully used with NNs as part of a larger pipeline . In particular Seeded Iterated Learning ( SIL ) ( Lu et al. , 2020a ) uses IL to counter language drift . Language drift occurs when a pretrained language model is then used to complete a language based task . While training on the task , the model begins to lose structure in the learned language that is not useful for performing the task . By training a “ teacher ” model on task completion and then using the teacher to supervise a pretrained “ student ” , Lu et al . ( 2020a ) show that the student network is able to learn to complete the task while maintaining the initially learned language . SIL is extended by Lu et al . ( 2020b ) to include supervised selfplay in the teacher task-learning step . This is known as Supervised SIL ( SSIL ) . Thus , parts of the data used to pretrain the teacher are also replayed while the teacher learns to perform the task . This also aids in avoiding language drift . While these previous works reflect the influence of certain initial data properties , regularizers or complex architectures on the emergence of compositional structure , the necessity of structured architectures with modular designs remains unclear . 3 A SPACE OF DATASETS WITH COMBINATORIAL SYSTEMATICITY . The notion of systematic generalization is broad , and has been assessed using a variety of datasets and paradigms ( Hupkes et al. , 2020 ) . Here we introduce a simple setting reminiscent of learning to refer to objects in an environment , as depicted in Fig . 1 . For instance , we might see a particular car ( Fig . 1a ) , extracting various perceptual input features such as whether it has a stripe , is red , or has a particular hood ornament . This scenario affords several ways in which we could refer to the car . We could systematically map individual input features to individual outputs by saying a sentence like “ My car is the red one with the white stripe ; ” or we could choose a single word to refer to some combination of properties , saying something like “ My car is the Ferrari. ” This task has a type of combinatorial systematic structure , because individual input features can link independently to individual output features . More generally , an agent first observes an object and forms an internal “ logical form ” . The task is then to map from this logical form to a “ phonetic form ” by providing a name for each object ( Brighton & Kirby , 2006 ) . As illustrated in this example , real world settings often allow redundant expressions and mappings , such that the same object could be individuated with a variety of expressions that vary in systematicity . To formalize this setting , we define a parametric space of datasets with input and output matrices X = [ Ωx Γx ] T and Y = [ Ωy Γy ] T respectively , where nx , ny , kx , ky , r ∈ R+ are the parameters that define a specific dataset . Figure 1b visualizes one dataset in the space . The systematic input feature matrix Ωx ∈ { −1 , 1 } nx×2 nx consists of all binary patterns with nx bits . Here nx is a key parameter determining the number of bits in the systematic input structure . Overall , the dataset contains 2nx examples . The systematic output feature matrix Ωy ∈ { −1 , 1 } ny×2 nx is a sampling of ny features ( rows ) from Ωx , and is the systematic component of the output matrix . Intuitively , the binary combinatorial structure in Ωx and Ωy is meant to reflect systematic mappings from input features ( the color red or blue ) to output features ( the words “ red ” or “ blue ” ) . Next , the non-systematic input feature matrix Γx = [ rI1 ... rIkx ] consists of kx scaled identity matrices , Ii ∈ { 0 , 1 } 2 nx×2nx . Similarly , the non-systematic output matrix Γy = [ rI1 ... rIky ] has ky scaled identity matrices , with scale factor r. These identity matrices provide a single feature for each pattern which is only on for that pattern . Intuitively , these features are meant to reflect idiosyncratic features ( like hood ornament , a distinctive scratch , or a proper noun name ) or features reflecting specific nonlinear combinations of systematic features ( red-striped-2-door ) . Together kx , ky and r control the frequency and intensity ( for example the size or perceptual prominence of the brand ’ s logo ) of the non-systematic features , which are both factors that can promote non-compositional language being used by humans ( Rogers et al. , 2004 ) . This space of datasets is consistent with numerical notions of compositionality in previous works ( Andreas , 2018 ) , which define compositionality as a homomorphism between the observation space and the naming space ( since the input-output mappings are linear they are homomorphic ) . The amount of systematic structure can be titrated by adjusting nx and ny ; and the prevalence of nonsystematic structure by adjusting kx , ky , and r. Importantly , and novel to our analysis , datasets in this space allow redundant solutions : the systematic output features can be generated based on systematic input features alone , but they can equally be generated using non-systematic features alone , or some mixture of the two . Exploiting this fact , we now ask how reliance on systematic or non-systematic structure is influenced by implicit biases , architecture , and learning regime .

Summary: The paper studies the relation of learning dynamics and architecture structure to the acquisition of systematic knowledge in shallow and deep linear neural networks. To this end the authors study the network behavior on a space of parametric datasets composed of systematic and non-systematic features and assess the impact of training regime, architecture, and iterated learning on the learned by the network functions. It is observed that a fully systematic mapping, thought theoretically possible to learn in such a regime, is not achieved by the shallow nor deep networks, and is not encouraged by a modular architecture (except for the most trivial, fully separated case).  Authors also observe that, in the studied framework, the iterated learning can lead to systematic functions when combined with the output modularity constraint.