Gradient Descent Resists Compositionality

In this paper , we argue that gradient descent is one of the reasons that make compositionality learning hard during neural network optimization . We find that the optimization process imposes a bias toward non-compositional solutions . This is caused by gradient descent , trying to use all available and redundant information from input , violating the conditional independence property of compositionality . Based on this finding , we suggest that compositionality learning approaches considering only model architecture design are unlikely to achieve complete compositionality . This is the first work to investigate the relation between compositional learning and gradient descent . We hope this study provides novel insights into compositional generalization , and forms a basis for new research directions to equip machine learning models with such skills for human-level intelligence . The source code is included in supplementary material . 1 INTRODUCTION . Compositional generalization is the algebraic capacity to understand and produce many novel combinations from known components ( Chomsky , 1957 ; Montague , 1970 ) , and it is a key element of human intelligence ( Minsky , 1986 ; Lake et al. , 2017 ) to recognize the world efficiently and create imagination . Broadly speaking , compositional generalization is a class of out-of-distribution generalization ( Bengio , 2017 ) , where the training and test distributions are different . A sample in such a setting is a combination of several components , and the generalization is enabled by recombining the seen components of the unseen combination during inference . For example , in the image domain , an object is a combination of multiple parts or properties . In the language domain , a sentence is a combination of syntax and semantics . Each component of an output depends only on the corresponding input component , but not on other variables . We call this the conditional independence property , and will formally introduce in Section 3 . People hope to design machine learning algorithms with compositional generalization skills . However , conventional neural network models generally lack such ability . There have been many attempts to equip models with compositionality ( Fodor & Pylyshyn , 1988 ; Bahdanau et al. , 2019 ) , and most efforts focus on designing neural network architectures ( Graves et al. , 2014 ; Andreas et al. , 2016 ; Henaff et al. , 2016 ; Shazeer et al. , 2017 ; Li et al. , 2018 ; Santoro et al. , 2018 ; Kirsch et al. , 2018 ; Rosenbaum et al. , 2019 ; Goyal et al. , 2019 ) . Recently , multiple approaches showed progress in specific tasks ( Li et al. , 2019 ; 2020 ; Lake , 2019 ; Russin et al. , 2019 ) , but we still do not know why standard approaches seldom achieve good compositionality in general . In this paper , we argue that there is a bias to prevent parameters from reaching compositional solutions , when we use gradient descent in optimization ( please see Figure 1 for illustrations ) . This is because gradient seeks the steepest direction , so that it uses all possible and redundant input information , which contradicts to the conditional independence property of compositionality . This problem is not due to how gradient is computed , such as back propagation , but caused by the essential property of gradient . We derive theoretical relation between gradient descent and compositionality with information theory . We also provide examples and visualization to show the detailed process of how gradient resists compositionality . Based on the finding , we propose that compositionality learning approaches with model structure design ( manual or searching ) alone are not likely to achieve complete compositionality . We hope this research provides new insights and forms a basis for new research directions in compositional generalization , and helps to improve machine intelligence towards human-level . The contributions of this paper can be summarized as follows . • The novelty of this work is to find the relation between compositional learning and gradient descent in optimization process , i.e. , gradient descent resists compositionality . • We theoretically derive the result and explain why standard approaches with architecture design alone do not address compositionality . 2 RELATED WORK . Compositionality Humans learn language and recognize the world in a flexible way by leveraging systematic compositionality . The compositional generalization is critical in human cognition ( Minsky , 1986 ; Lake et al. , 2017 ) , and it helps humans to connect limited amount of learned concepts for unseen combinations . Though deep learning has many achievements in recent years ( LeCun et al. , 2015 ; Krizhevsky et al. , 2012 ; Yu & Deng , 2012 ; He et al. , 2016 ; Wu & et al , 2016 ) , compositional generalization has not been well addressed ( Fodor & Pylyshyn , 1988 ; Marcus , 1998 ; Fodor & Lepore , 2002 ; Marcus , 2003 ; Calvo & Symons , 2014 ) . There are observations that current neural network models do not learn compositionality ( Bahdanau et al. , 2019 ) . Most recently , multiple approaches are proposed to address compositionality in neural networks ( Li et al. , 2019 ; 2020 ; Lake , 2019 ; Russin et al. , 2019 ) for specific tasks . However , we are still not sure why compositionality is hard to achieve in general cases , and this work discusses about this problem from optimization perspective . Another line of related work is independent disentangled representation learning ( Higgins et al. , 2017 ; Locatello et al. , 2019 ) . Its main assumption is that the expected components are statistically independent in training data . This setting does not have transferring problem in test , because all combinations have positive joint probabilities in training ( please refer to Section 3 ) . Compositionality is applied in different areas such as continual learning ( Jin et al. , 2020 ; Li et al. , 2020 ) , question answering ( Andreas et al. , 2016 ; Hudson & Manning , 2019 ; Keysers et al. , 2020 ) , and reasoning ( Talmor et al. , 2020 ) . Gradient Descent Gradient descent is a powerful and general purpose optimization tool for solving large scale problems in deep neural networks . It is usually used in a stochastic way ( Stochastic Gradient Descent ) with mini-batches , and has many variations such as Momentum ( Rumelhart et al. , 1986 ) , averaging ( Polyak & Juditsky , 1992 ) , AdaGrad ( Duchi et al. , 2011 ) , AdaDelta ( Zeiler , 2012 ) , RMSProp ( Tieleman & Hinton , 2012 ) , Adam ( Kingma & Ba , 2014 ) . Most of the previous work focus on faster reduction of loss and theoretical convergence analysis of SGD ( Bottou et al. , 2018 ; Luo , 1991 ; Reddi et al. , 2018 ; Chen et al. , 2018 ; Zhou et al. , 2018 ; Zou & Shen , 2018 ; De et al. , 2018 ; Zou et al. , 2018 ; Ward et al. , 2018 ; Barakat & Bianchi , 2019 ) . In particular , this work focuses on investigating why standard neural network training only achieves limited level of compositionality by studying the relationship between gradient descent and compositionality . 3 CONCEPTS FOR COMPOSITIONALITY AND GRADIENT DESCENT . We first formulate compositionality using the conditional independence property , and define compositional generalization . We then review properties of gradient for the derivation in the next section . Conditional Independence Property for Compositionality When multiple hidden variables live in the same representation , and can not be separated by simply splitting the representation , then these variables are entangled in the representation . For example , color and shape are two hidden variables and they share the same representation of image . Also , syntax and semantics are two hidden variables and they share the same representation of sentence . When we extract the hidden variables from their shared representation , we disentangle them . We then consider a prediction problem , where input X and output Y have multiple entangled components that are not labeled in data , and they are aligned . For example in machine translation , X1 is input syntax , and X2 is input semantics . Y1 is output syntax , and Y2 is output semantics . The syntax of output Y1 depends only on the syntax of input X1 , and the semantics of output Y2 depends only on the semantics of X2 . We can formalize the alignments as conditional independence property : Yi depends only on Xi . ∀i : P ( Yi|X1 , . . . , XK , Y1 , . . . , Yi−1 , Yi+1 , . . . , YK ) = P ( Yi|Xi ) . When a model fits this property , we say it has compositionality . Note that this can be understood as a kind of sparseness property ( Bengio , 2017 ) , because it restricts effective connection between input and output components . Compositional Generalization In compositional generalization , each sample in either training or test is a combination of several components . A test sample has a combination that does not appear in training , but each component of the test sample appears in training . We need to recombine the seen components to generalize to the test sample . We can define compositional generalization probabilistically as follows . In train , In test , ∀i : P ( Xi ) > 0 , P ( X1 , . . . , XK ) = 0 , P ( X1 , . . . , XK ) > 0 , ∀i : P ( Yi|Xi ) is high . P ( Y1 , . . . , YK |X1 , . . . , XK ) is predicted high . Compositionality bridges the gap between training and test distributions to achieve compositional generalization . We first apply chain rule , and then use compositionality as follows . P ( Y1 , . . . , YK |X1 , . . . , XK ) = K∏ i=1 P ( Yi|X1 , . . . , XK , Y1 , . . . , Yi−1 ) = K∏ i=1 P ( Yi|Xi ) . When P ( Yi|Xi ) are all high , their product should also be high . Therefore , a model with compositionality—satisfying this conditional independence property—addresses compositional generalization . Property of Gradient For a function f ( x1 , . . . , xK ) , the gradient∇f is the steepest direction to change the function ’ s value . Generally , gradient descent methods estimate ∇f using low-order local estimation . By definition , it is the vector of partial derivatives with respective to the inputs . ∇f = ∂f ∂x1 , . . . , ∂f ∂xK We will use the following definition in later arguments . Definition 1 ( Partial derivative ) . Partial derivative for an input is the derivative assuming other inputs are constant . ∀i = 1 , . . . , K : ∂f ( x1 , . . . , xK ) ∂xi = df ( c1 , . . . , xi , . . . , cK ) dxi 4 GRADIENT DESCENT RESISTS COMPOSITIONALITY . We focus on the early phase of training to show that gradient descent causes a model to use the redundant information to compute output when it has information to reduce the loss . We develop the arguments step by step . We first analyze the influence of the input on an output variable . We then consider the case of entangled inputs and one output . Finally , we discuss the case with entangled inputs and entangled outputs . The gradient is used to reduce loss , so we aim to relate loss reduction and the influence from input to output . To do that , we use mutual information to describe the influence , and use knowledge from information theory ( Definition 2 , Theorem 1 and Theorem 2 ) . We also study the impact of gradient descent to the influence , so we relate mutual information with gradient with Proposition 1 . Definition 2 ( Markov chain ( Cover , 1999 ) p.34 ) . Random variables X , Y , Z are said to form a Markov chain in that order ( denoted by X → Y → Z ) if the conditional distribution of Z depends only on Y and is conditionally independent of X . Theorem 1 ( Data-processing inequality ( Cover , 1999 ) p.34 ) . If X → Y → Z , then I ( X ; Y ) ≥ I ( X ; Z ) . Theorem 2 ( Chain rule for information ( Cover , 1999 ) p.24 ) . For random variables X , Y , Z , I ( X , Y ; Z ) = I ( Y ; Z|X ) + I ( X ; Z ) . Proposition 1 . When ∂Y∂X is defined , I ( X ; Y ) > 0 ⇐⇒ ∂Y ∂X 6= 0 Proof . I ( X ; Y ) > 0 means X and Y are not independent , which means Y is not invariant to X .

Summary: the paper investigates what neural networks learn when trained with gradient descent, in case parts of the inputs are only partially relevant to the output. The main claim is that GD is what prevents compositionality. In a set of synthetic experiments it is shown that indeed GD learns to use all information in the input, which results in poor generalization ood when only a subset of it was relevant.