An extensive body of empirical research has revealed remarkable regularities in the acquisition, organization, deployment, and neural representation of human semantic knowledge, thereby raising a fundamental conceptual question: what are the theoretical principles governing the ability of neural networks to acquire, organize, and deploy abstract knowledge by integrating across many individual experiences? We address this question by mathematically analyzing the nonlinear dynamics of learning in deep linear networks. We find exact solutions to this learning dynamics that yield a conceptual explanation for the prevalence of many disparate phenomena in semantic cognition, including the hierarchical differentiation of concepts through rapid developmental transitions, the ubiquity of semantic illusions between such transitions, the emergence of item typicality and category coherence as factors controlling the speed of semantic processing, changing patterns of inductive projection over development, and the conservation of semantic similarity in neural representations across species. Thus, surprisingly, our simple neural model qualitatively recapitulates many diverse regularities underlying semantic development, while providing analytic insight into how the statistical structure of an environment can interact with nonlinear deep learning dynamics to give rise to these regularities. semantic cognition | neural networks | hierarchical generative models Abbreviations: SVD, singular value decomposition H uman cognition relies on a rich reservoir of semantic knowledge enabling us to organize and reason about our complex sensory world [1][2][3][4]. This semantic knowledge allows us to answer basic questions from memory (i.e. "Do birds have feathers?"), and relies fundamentally on neural mechanisms that can organize individual items, or entities (i.e. Canary, Robin) into higher order conceptual categories (i.e. Birds) that include items with similar features, or properties. This knowledge of individual entities and their conceptual groupings into categories or other ontologies is not present in infancy, but develops during childhood [1, 5], and in adults, it powerfully guides the deployment of appropriate inductive generalizations. The acquisition, organization, deployment, and neural representation of semantic knowledge has been intensively studied, yielding many well-documented empirical phenomena. For example, during acquisition, broader categorical distinctions are generally learned before finer-grained distinctions [1, 5], and long periods of relative stasis can be followed by abrupt conceptual reorganization [6, 7]. Intriguingly, during these periods of developmental stasis, children can strongly believe illusory, incorrect facts about the world [2]. Also, many psychophysical studies of performance in semantic tasks have revealed empirical regularities governing the organization of semantic knowledge. In particular, category membership is a graded quantity, with some items being more or less typical members of a category (i.e. a sparrow is a more typical bird than a penguin). The notion of item typicality is both highly reproducible across individuals [8, 9] and correlates with performance on a diversity of semantic tasks [10][11][12][13][14]. Moreover, certain categories themselves are thought to be highly coherent (i.e. the set of things that are Dogs), in contrast to less coherent categories (i.e. the set of things that are Blue). More coherent categories play a privileged role in the organization of our semantic knowledge; coherent categories are the ones that are most easily learned and represented [8, 15, 16] . Also, the organization of semantic knowledge powerfully guides its deployment in novel situations, where one must make inductive generalizations about novel items and properties [2, 3]. Indeed, studies of children reveal that their inductive generalizations systematically change over development, often becoming more specific with age [2, 3, [17] [18] [19] . Finally, recent neuroscientific studies have begun to shed light on the organization of semantic knowledge in the brain. The method of representational similarity analysis [20, 21] has revealed that the similarity structure of neural population activity patterns in high level cortical areas often reflect the semantic similarity structure of stimuli, for instance by differentiating inanimate objects from animate ones [22] [23] [24] [25] [26] . And strikingly, studies have found that such neural similarity structure is preserved across human subjects, and even between humans and monkeys [27, 28] . This wealth of empirical phenomena raises a fundamental conceptual question about how neural circuits, upon experiencing many individual encounters with specific items, can over developmental time scales extract abstract semantic knowledge consisting of useful categories that can then guide our ability to reason about the world and inductively generalize. While a diverse set of theories have been advanced to explain human semantic development, there is currently no analytic, mathematical theory of neural circuits that can account for the diverse phenomena described above. Interesting non-neural accounts for the discovery of abstract semantic structure include for example the conceptual "theory-theory" [2, [16] [17] [18] , and computational Bayesian [29] approaches. However, neither currently proposes a neural implementation that can infer abstract concepts from a stream of specific examples. In contrast, much prior work has shown, through simulations, that neural networks can gradually extract semantic structure by incrementally adjusting synaptic weights via error-corrective learning [4,[30][31][32][33][34][35][36][37]. However, in contrast to the computational transparency enjoyed by Bayesian approaches, the theoretical principles governing how even simple artificial neural networks extract semantic knowledge from their ongoing stream of experience, embed this knowledge in their synaptic weights, and use these weights to perform inductive generalization, remains obscure. In this work, our fundamental goal is to fill this gap by employing an exceedingly simple class of neural networks, namely deep linear networks. Surprisingly, we find that this model class can qualitatively account for a diversity of phenomena involving semantic cognition described above. Indeed, we build upon a considerable neural network literature [30][31][32][33][34][35][36][37] addressing such phenomena through simulations of more complex nonlinear networks. We build particularly on the integrative, simulation-based treatment of semantic cognition in [4], often using the same modeling strategy in a simpler linear set- ting, to obtain similar results but with additional analytical insight. Thus, in contrast to prior work, whether conceptual, Bayesian, or connectionist, our simple model is the first to permit exact analytical solutions describing the entire developmental trajectory of knowledge acquisition and organization, and its subsequent impact on the deployment and neural representation of semantic structure. In the following, we describe each of these aspects of semantic knowledge acquisition, organization, deployment, and neural representation in sequence, and we summarize our main findings in the discussion. Here we consider a framework for analyzing how neural networks extract abstract semantic knowledge by integrating across many individual experiences of items and their properties, across developmental time. In each experience, given an item as input, the network is trained to correctly produce its associated properties or features as output. Consider for example, the network's interaction with the semantic domain of living things, schematized in Fig. 1A . If the network encounters an item, such as a Canary, perceptual neural circuits produce an activity vector x ∈ R N 1 that identifies this item and serves as input to the semantic system. Simultaneously, the network observes some of the item's properties, for example that a canary Can Fly. Neural circuits produce an activity feature vector y ∈ R N 3 of that item's properties which serves as the desired output of the semantic network. Over time, the network experiences many individual episodes with a variety of different items and their properties. The total collected experience furnished by the environment to the semantic system is thus a set of P examples x i , y i , i = 1, . . . , P , where the input vector x i identifies item i, and the output vector y i is a set of features to be associated to this item. The network's task is to predict an item's properties y from its perceptual representation x. These predictions are generated by prop- To sample one feature's value across items, the root node is randomly set to ±1; next this value diffuses to children nodes, where its sign is flipped with a small probability e. The leaf node assignments yield the value of this feature on each item. To generate more features, the process is repeated independently N times. (b) Analytically derived input singular vectors (up to a scaling) of the resulting data, ordered top-to-bottom by singular value. Mode 1 is a level 0 function on the tree, mode 2 is level 1, 3 and 4 are level 2, while modes 5 through 8 are level 3. Singular modes corresponding to broad distinctions (higher levels) have the largest singular values, and hence will be learned first. (c) The output covariance of the data consists of hierarchically organized blocks. address the output data yµ, µ = 1,...,P. Each yµ is an Ndimensional feature vector where each feature i in example µ takes the value y µ i = ±1. The value of each feature i across all examples arises from a branching diffusion process occurring on a tree (see e.g. Fig. 4A ). Each feature i undergoes its own diffusion process on the tree, independent of any other feature j. This entire process, described below, yields a hierarchical structure on the set of examples µ = 1,...,P, which are in one-to-one correspondence with the leaves of the tree. The tree has a fixed topology, with D levels indexed by l = 0,...,D 1, with Ml total nodes at level l. We take for simplicity a regular branching structure, so that every node at level l has exactly Bl descendants. Thus Ml = M0Pl 1 k=0Bl . The tree has a single root node at the top (M0 = 1), and again P leaves at the bottom, one per example in the dataset (MD 1 = P). Given a single feature component i, its value across P examples is determined as follows. First draw a random variable h(0) associated with the root node at the top of the tree. The variable h(0) takes the values ±1 with equal probability 1 2 . Next, for each of the B0 descendants below the root node at level 1, pick a random variable h i takes the value h(0) with probability 1 e and h(0) with probability e. The process continues down the tree: each of Bl 1 nodes at level l with a common ancestor at level l 1 is assigned its ancestor's value with probability 1 e, or is assigned the negative of its ancestor's value with probability e. Thus the original feature value at the root, h(0), diffuses down the tree with a small probability e of changing at each level along any path to a leaf. The final values at the P leaves constitute the feature values y µ i for µ = 1,...,P. This process is repeated independently for N feature components. In order to understand the dimensions of variation in the feature vectors, we consider the inner product, or overlap, between two example feature vectors. This inner product, normalized by the number of features N, has a well-defined limit as N ! •. Furthermore, due to the hierarchical diffusive process which generates the data, the normalized inner product only depends on the level of the tree at which the first common ancestor of the two leaves associated with the two examples arises. Therefore we can make the definition where again, the first common ancestor of leaves µ1 and µ2 arises at level k. It is possible to explicitly compute qk for the generative model described above, which yields It is clear that the overlap qk strictly decreases as the level k of the last common ancestor decreases (i.e. the distance up the tree to the last common ancestor increases). Thus pairs of examples with a more recent common ancestor have stronger overlap than pairs of examples with a more distant agating activity through a three layer linear neural network (Fig. 1B) . The input activity pattern x in the first layer propagates through a synaptic weight matrix W 1 of size N2 × N1, to create an activity pattern h = W 1 x in the second layer of N2 neurons. We call this layer the "hidden" layer because it corresponds neither to input nor output. The hidden layer activity then propagates to the third layer through a second synaptic weight matrix W 2 of size N3 × N2, producing an activity vectorŷ = W 2 h which constitutes the output prediction of the network. The composite function from input to output is thus simplyŷ = W 2 W 1 x. For each input x, the network compares its predicted outputŷ to the observed features y and it adjusts its weights so as to reduce the discrepancy between y andŷ. To study the impact of depth, we will contrast the learning dynamics of this deep linear network to that of a shallow network that has just a single weight matrix, W s of size N3 × N1 linking input activities directly to the network's predictionsŷ = W s x. At first inspection, it may seem that there is no utility whatsoever in considering deep linear networks, since the composition of linear functions remains linear. Indeed, the appeal of deep networks is thought to lie in the increasingly expressive functions they can represent by successively cascading many layers of nonlinear elements [38, 39]. In contrast, deep linear networks gain no expressive power from depth; a shallow network can compute any function that the deep network can, by simply taking W s = W 2 W 1 . However, as we see below, the learning dynamics of the deep network is highly nonlinear, while the learning dynamics of the shallow network remains linear. Strikingly, many complex, nonlinear features of learning appear even in deep linear networks, and do not require neuronal nonlinearities. As an illustration of the power of deep linear networks to capture learning dynamics even in nonlinear networks, we compare the two learning dynamics in Fig. 2 . Fig. 2A shows a low dimensional visualization of the simulated learning dynamics of a multilayered nonlinear neural network trained to predict the properties of a set of items in a semantic domain of animals and plants (details of the neural architecture and training data can be found in [4]). The nonlinear network exhibits a striking, hierarchical progressive differentiation of structure in its internal hidden representations, in which animals versus plants are first distinguished, then birds versus fish, and trees versus flowers, and finally individual items. This remarkable phenomenon raises important questions about the theoretical principles governing the hierarchical differentiation of structure in neural networks. In particular, how and why do the network's dynamics and the statistical structure of the input conspire to generate this phenomenon? In Fig. 2B we mathematically derive this phenomenon by finding analytic solutions to the nonlinear dynamics of learning in a deep linear network, when that network is exposed to a hierarchically structured semantic domain, thereby shedding considerable theoretical insight onto the origins of hierarchical differentiation in a deep network. We present the derivation below, but for now, we note that the resemblance between Fig. 2A and Fig. 2B suggests that deep linear networks can form an excellent, analytically tractable model for shedding conceptual insight into the learning dynamics, if not the expressive power, of their nonlinear counterparts. We now begin an outline of the derivation that leads to Fig. 2B . The incremental error corrective process described above can be formalized as online stochastic gradient descent; each time an example i is presented, the weights W 2 and W 1 are adjusted by a small amount in the direction that most rapidly decreases the squared error y i −ŷ i 2 , yielding the standard back propagation learning rule where λ is a small learning rate. We show the extraction of such abstract domain structure is possible provided learning is gradual, with a small learning rate λ. In this regime, many examples are seen before the weights appreciably change, so learning is driven by the statistical structure of the domain. We imagine training is divided into a sequence of learning epochs. In each epoch the above rule is followed for all P examples in random order. Then averaging [ 1 ] over all P examples and taking a continuous time limit gives the mean change in weights per learning epoch, where is an N3 × N1 input-output correlation matrix, and τ ≡ 1 P λ (see SI for detailed derivation). Here, t measures time in units of learning epochs; as t varies from 0 to 1, the network has seen P examples corresponding to one learning epoch. These equations reveal that learning dynamics in even in our simple linear network can be highly complex: the second order statistics of inputs and outputs drives synaptic weight changes through coupled nonlinear differential equations with up to cubic interactions in the weights. Explicit solutions from tabula rasa. These nonlinear dynamics are difficult to solve for arbitrary initial conditions on synaptic weights. However, we are interested in a particular limit: learning from a state of essentially no knowledge, which we model as small random synaptic weights. To further ease the analysis, we shall assume that the influence of perceptual correlations is minimal (Σ x ≈ I). Our fundamental goal, then, is to understand the dynamics of learning in (2)-(3) as a function of the input-output correlation matrix Σ yx . The learning dynamics is closely related to terms in the singular value decomposition (SVD) of Σ yx (Fig. 3A) , which decomposes any matrix into the product of three matrices. Each of these matrices has a distinct semantic interpretation. For example, the α'th column v α of the N1 ×N1 orthogonal matrix V can be thought of as an object analyzer vector; it determines the position of item i along an important semantic dimension α in the training set through the component v α i . To illustrate this interpretation concretely, we consider a simple example dataset with four items (Canary, Salmon, Oak, and Rose) and five properties (Fig. 3) . The two animals share the property can Move, while the two plants do not. Also each item has a unique property: can Fly, can Swim, has Bark, and has Petals. For this dataset, while the first row of V T is a uniform mode, the second row, or the second object analyzer vector v 2 , determines where items sit on an animal-plant dimension, and hence has positive values for the Canary and Salmon and negative values for the plants. The other dimensions identified by the SVD are a bird-fish dimension, and a flower-tree dimension. The corresponding α'th column u α of the N3 × N3 orthogonal matrix U can be thought of as a feature synthesizer vector for semantic distinction α. It's components u α m indicate the extent to which feature m is present or absent in distinction α. Hence the feature synthesizer u 2 associated with the animal-plant semantic dimension has positive values for Move and negative values for Roots, as animals typically can move and do not have roots, while plants behave oppositely. Finally the N3 × N1 singular value matrix S has nonzero elements sα, α = 1, . . . , N1 only on the diagonal, ordered so that s1 ≥ s2 ≥ · · · ≥ sN 1 . sα captures the overall strength of the association between the α'th input and output dimensions. The large singular value for the animal-plant dimension reflects the fact that this one dimension explains more of the training set than the finer-scale dimensions like bird-fish and flower-tree. Given the SVD of the training set's input-output correlation matrix in (4), we can now explicitly describe the network's learning dynamics. The network's overall input-output map at time t is a timedependent version of this SVD decomposition (Fig. 3B) ; it shares the object analyzer and feature synthesizer matrices of the SVD of Σ yx , but replaces the singular value matrix S with an effective singular value matrix A(t), where the trajectory of each effective singular value aα(t) obeys Eqn. 6 describes a sigmoidal trajectory that begins at some initial value a 0 α at time t = 0 and rises to sα as t → ∞, as plotted in Fig. 3C . This solution is applicable when the network begins as a tabula rasa, or an undifferentiated state with little initial knowledge, corresponding small random initial weights (see SI for derivation), and it provides an accurate description of the learning dynamics in this regime, as confirmed by simulation in Fig. 3C . This solution also gives insight into how the internal representations in the hidden layer of the deep network evolve. An exact solution for W 2 and W 1 is given by where Q is an arbitrary N2 × N2 invertible matrix (SI Appendix). If initial weights are small, then the matrix Q will be close to orthogonal, i.e., Q ≈ R where R T R = I. Thus the internal representations are specified up to an arbitrary rotation R. Factoring out the rotation, the hidden representation of item i is [ 8 ] Thus internal representations develop over time by projecting inputs onto more and more input-output modes as they are learned. The shallow network has a solution of analogous form, bα(t) u α v αT , but now each singular value evolves as In contrast to the deep network's sigmoidal trajectory, Eqn. 9 describes a simple exponential approach from the initial value b 0 α to sα, as plotted in Fig. 3D . Hence depth fundamentally changes the dynamics of learning, yielding several important consequences below. Rapid stage like transitions due to depth. We first compare the time-course of learning in deep versus shallow networks as revealed in Eqns. (6) and (9). For the deep network, beginning from a small initial condition a 0 α = , each mode's effective singular value aα(t) rises to within of its final value sα in time in the limit → 0 (SI Appendix). This is O(1/sα) up to a logarithmic factor. Hence modes with stronger explanatory power, as quantified by the singular value, are learned more quickly. Moreover, when starting from small initial weights, the sigmoidal transition from no knowledge of the mode to perfect knowledge can be arbitrarily sharp. Indeed the ratio of time spent in the sigmoidal transition regime to the ratio of time spent before making the transition can go to infinity as the initial weights go to zero (see SI Appendix). Thus rapid stage like transitions in learning can be prevalent even in deep linear networks. By contrast, the timescale of learning for the shallow network is which is O(1) up to a logarithmic factor. Hence in a shallow network, the timescale of learning a mode depends only weakly on its associated singular value. Essentially all modes are learned at the same time, with an exponential rather than sigmoidal learning curve. Progressive differentiation of hierarchical structure. We are now almost in a position to explain how we analytically derived the result in Fig. 2B . The only remaining ingredient is a mathematical description of the training data. Indeed the numerical results in Fig from a toy-training set, making it difficult to understand which aspects the data were essential for the hierarchical learning dynamics. Here, we introduce a probabilistic generative model for hierarchically structured data, in order to move beyond toy datasets to extract general principles of how statistical structure impacts learning. We use a generative model (introduced in [40]) that mimics the process of evolution to create a dataset with explicit hierarchical structure (see SI Appendix). In the model, each feature diffuses down an evolutionary tree (Fig. 4A) , with a small probability of mutating along each branch. The items lie at the leaves of the tree, and the generative process creates a hierarchical similarity matrix between items such that items with a more recent common ancestor on the tree are more similar to each other (Fig. 4B) . We analytically computed the SVD of this hierarchical dataset and we found that the object analyzer vectors, which can be viewed as functions on the leaves of the tree in Fig. 4C respect the hierarchical branches of the tree, with the larger (smaller) singular values corresponding to broader (finer) distinctions. Moreover, in Fig. 4A we have artificially labelled the leaves and branches of the evolutionary tree with organisms and categories that might reflect a natural realization of this evolutionary process. Now, inserting the singular values in Fig. 4D (and SI Appendix) into the deep learning dynamics in Eq. 6 to obtain the time-dependent singular values a α (t), and then inserting these along with the object analyzers vectors v α in Fig. 4C into Eq. 8, we obtain a complete analytic derivation of the evolution of internal representations over developmental time in the deep network. An MDS visualization of these evolving hidden representation then yields Fig. 2B , which qualitatively recapitulates the much more complex network and dataset that led to Fig. 2A . In essence, this analysis completes a mathematical proof that the striking progressive differentiation of hierarchical observed in Fig. 2 is an inevitable consequence of deep learning dynamics, even in linear networks, when exposed to hierarchically structured data. The essential intuition is that dimensions of feature variation across items corresponding to broader (finer) hierarchical distinctions have stronger (weaker) statistical structure, as quantified by the singular values of the training data, and hence these dimensions are learned faster (slower), leading to waves of differentiation in a deep, but not a shallow network. Such a pattern of hierarchical differentiation has long been argued to apply to the conceptual development of infants and children [1,[5][6][7]. Illusory Correlations. Another intriguing aspect of semantic development is that children sometimes attest to false beliefs (i.e. worms have bones [2]) that could not have been learned through direct experience. These errors challenge simple associationist accounts of semantic development that would predict a steady, monotonic accumulation of information about individual properties [2, 16, 17, 41]. Yet as shown in Fig. 5 , the network's knowledge of individual properties exhibits complex, non-monotonic trajectories over the course of learning. The overall prediction for a property is a sum of contri- Fig. 5A , the first two modes make a positive contribution while the third makes a negative one, yielding the inverted U-shaped trajectory. Indeed, any property-item combination for which u α m v α i takes different signs across different α will exhibit a non-monotonic learning curve, making such curves a frequent occurrence even in a fixed, unchanging environment. In a deep network, two modes with singular values that differ by ∆ will have an interval in which the first is learned but the second is not. The length of this developmental interval is roughly O(∆) (SI Appendix). Moreover, the rapidity of the deep network's stage-like transitions further accentuates the non-monotonic learning of individual properties. This behavior, which may seem hard to reconcile with an incremental, errorcorrective learning process, is a natural consequence of minimizing global rather than local prediction error: the fastest possible improvement in predicting all properties across all items sometimes results in a transient increase in the size of errors on specific items and properties. Every property in a shallow network, by contrast, monotonically approaches its correct value and therefore shallow networks provably never exhibit illusory correlations (SI Appendix). We now turn from the dynamics of learning to its final outcome. When exposed to a variety of items and features interlinked by an underlying hierarchy, for instance, what categories naturally emerge? Which items are particularly representative of a categorical distinction? And how is the structure of the domain internally represented? Category membership, typicality, and prototypes. A long observed empirical finding is that category membership is not simply a logical, binary variable, but rather a graded quantity, with some objects being more or less typical members of a category (i.e. a sparrow is a more typical bird than a penguin). Indeed, such graded judgements of category membership are both highly reproducible across individuals [8, 9] and moreover correlate with performance on a range of tasks: subjects more quickly verify the category membership of more typical items [10, 11], more frequently recall typical examples of a category [12], and more readily extend new information about typical items to all members of a category [13, 14]. Our theoretical framework provides a natural mathematical definition of item typicality that both explains how it emerges from the statistical structure of the environment and improves task performance. Indeed, a natural notion of the typicality of an item i for a categorical distinction α is simply the quantity v α i in the corresponding object analyzer vector. To see why this is natural, note that after learning, the neural network's internal representation space has a semantic distinction axis α, and each object i is placed along this axis at a coordinate proportional to v [ Hence under our definition of typicality, an item i that is more typical than another other item j will have |v α i | > |v α j |, and thus will necessarily have a larger response magnitude under Eq. (12). Any performance measure which is monotonic in the response will therefore increase for more typical items under this definition. Thus our definition of item typicality is both a mathematically well defined function of the statistical structure of experience, through the SVD, and proveably correlates with task performance in our network. Several previous attempts at defining the typicality of an item involve computing a weighted sum of category specific features present or absent in the item [8, 15, [42][43][44]. For instance, a sparrow is a more typical bird than a penguin because it shares more relevant features (can fly) with other birds. However, the specific choice of which features are relevant-the weights in the weighted sum of featureshas often been heuristically chosen and relied on prior knowledge of which items belong to each category [8, 43]. Our definition of typicality can also be described in terms of a weighted sum of an object's features, but the weightings are uniquely fixed by the statistics of the entire environment through the SVD (see SI Appendix): which holds for all i, m, and α. Here, item i is defined by its feature vector o i ∈ R N 3 , where component o i m encodes the value of its m th feature. Thus the typicality v α i of item i in distinction α can be computed by taking a weighted sum of the components of its feature vector o i , where the weightings are precisely the coefficients of the corresponding feature synthesizer vector u α (scaled by the reciprocal of the singular value). The neural geometry of Eq. 13 is illustrated in Fig. 6 when α corresponds to the bird-fish categorical distinction. In many theories of typicality, the particular weighting of object features corresponds to a prototypical object [3, 15] , or the best example of a particular category. Such object prototypes are often obtained by a weighted average over the feature vectors for the objects in a category (i.e. averaging together the features of all birds, for instance, will give a set of features they share). However, such an average relies on prior knowledge of the extent to which an object belongs to a category. Our theoretical framework also yields a natural notion of object prototypes as a weighted average of object feature vectors, but unlike many other frameworks, it yields a unique prescription for the object weightings in terms of the statistical structure of the environment through the SVD (SI Appendix): [ 14 ] Thus the feature synthesizer u α , can itself be thought of as a category prototype for distinction α, as it can be obtained through a weighted average of all the object feature vectors o i , where the weighting of object i is none other than the typicality v α i of object i in distinction α. In essence, each element u α m of the prototype vector signifies how important feature m is for distinction α (Fig. 6) . Fig. 6 . The geometry of item typicality. For a semantic distinction α (in this case α is the bird-fish distinction) the object analyzer vector v α i arranges objects i along an internal neural representation space where the most typical birds take the extremal positive coordinates, and the most typical fish take the extremal negative coordinates. Objects like a rose, that is neither a bird nor a fish, are located near the origin on this axis. Positions along the neural semantic axis can also be obtained by computing the inner product between the feature vector o i for object i and the feature synthesizer u α as in (13). Moreover u α can be thought of as a category prototype for semantic distinction α through (14). an MDS visualization of the evolution of its internal representations reveals 3 clusters. (C) By computing the SVD of the product of synaptic weights W 2 W 1 , we can extract the network's object analyzers v α and feature synthesizers u α , finding 3 with large singular values sα, for α = 1, . . . , 3. Each of these 3 object analyzers v α and feature synthesizers u α takes large values on a subset of items and features respectively, and we can use them to reorder the rows and columns of (A) to obtain (C). This re-ordering reveals underlying structure in the data corresponding to 3 disjoint categories, such that if a feature and item belong to a category, the feature is present with a high probability p, whereas if it does not, it appears with a low probability q. In summary, a beautiful and simple duality between item typicality and category prototypes arises as an emergent property of the learned internal representations of the neural network. The typicality of an item is determined by the projection of that item's feature vector onto the category prototype in (13). And the category prototype is an average over all object feature vectors, weighted by their typicality in (14). Moreover, in any categorical distinction α, the most typical items i and the most important features m are determined by the extremal values of v Category coherence. The categories we naturally learn are not arbitrary, but instead are in some sense coherent, and efficiently represent the structure of the world [8, 15, 16] . For example, the set of things that are are Red and cannot Swim, is a well defined category, but intuitively is not as coherent as the category of Dogs; we naturally learn, and even name, the latter category, but not the former. When is a category learned at all, and what determines its coherence? An influential proposal [8, 15] suggested that coherent categories consist of tight clusters of items that share many features, and moreover are highly distinct from other categories with different sets of shared features. Such a definition, as noted in [3, 16, 17] can be circular: to know which items are category members, one must know which features are important for that category, and conversely, to know which features are important, one must know which items are members. Thus a mathematical definition of category coherence, as a function of the statistical structure of the environment, that is proveably related to the learnability of categories by neural networks, has remained elusive. Here we provide such a definition for a simple model of disjoint categories, and demonstrate how neural networks can cut through the Gordian knot of circularity. Our definition and theory is motivated by, and consistent with, prior network simulations exploring notions of category coherence through the coherent covariation of features [4]. Consider for example, a dataset consisting of No objects and N f features. Now consider a category consisting of a subset of K f features that tend to occur with high probability p in a subset of Ko items, whereas a background feature occurs with a lower probability q in a background item p when either are not part of the category. For what values of K f , K0, p, q, N f and N0 can such a category be learned, and if so, how accurately? Fig. 7A-D illustrates, for example, how a neural network can extract 3 such categories buried in the background noise. We see in Fig. 7E that the performance of category extraction increases as the number of items K0 and features K f in the category increases, and also as the signal-to-noise ratio, or SNR ≡ increases. Through random matrix theory (SI Appendix), we show that performance depends on the various parameters only through a category coherence variable Fig. 9 . Representation of explicit structural forms in a neural network. Each column shows a different structure. The first four columns correspond to pure structural forms, while the final column has cross-cutting structure. First row: The structure of the data generating probabilistic graphical model (PGM). Second row: The resulting item covariance matrix arising from either data drawn from the PGM (first four columns) or designed by hand (final column). Third row: The input-analyzing singular vectors that will be learned by the linear neural network. Each vector is scaled by its singular value, showing its importance to representing the covariance matrix. Fourth row: MDS view of the development of internal representations. 6 When the performance curves in Fig. 7E are re-plotted with category coherence C on the horizontal axis, all the curves collapse onto a single universal performance curve shown in Fig. 7F . We derive a mathematical expression for this curve in SI Appendix. It displays an interesting threshold behavior: if the coherence C ≤ 1, the category is not learned at all, and the higher the coherence above this threshold, the better the category is learned. This threshold is strikingly permissive. For example, at SNR = 1, it occurs at K0K f = N0N f . Thus in a large environment of No = 1000 objects and N f = 1600 features, as in Fig. 7A , a small category of 40 objects and 40 features can be easily learned, even by a simple deep linear network. Moreover, this analysis demonstrates how the deep network solves the problem of circularity described above by simultaneously bootstrapping the learning of object analyzers and feature synthesizers in its synaptic weights. Finally, we note that the definition of category coherence in Eq. (15) is qualitatively consistent with previous notions; coherent categories consist of large subsets of items possessing, with high probability, large subsets of features that tend not to co-occur in other categories. However, our quantitative definition has the advantage that it proveably governs category learning performance in a neural network. Basic categories. Closely related to category coherence, a variety of studies of naming have revealed a privileged role for basic categories at an intermediate level of specificity (i.e. Bird), compared to superordinate (i.e. Animal) or subordinate (i.e. Robin) levels. At this basic level, people are quicker at learning names [45, 46], prefer to generate names [46], and are quicker to verify the presence of named items in images [11, 46]. We note that basic level advantages typically involve naming tasks done at an older age, and so need not be inconsistent with progressive differentiation of categorical structure from superordinate to subordinate levels as revealed in preverbal cognition [1,[4][5][6][7] 47]. Moreover, some items are named more frequently than others, and these frequency effects could contribute to a basic level advantage [4]. However in artificial category learning experiments where frequencies are tightly controlled, basic level categories are still often learned first [48]. What statistical properties of the environment could lead to this basic level effect? While several properties have been put forth in the literature [11,42,44,48], a mathematical function of environmental structure that proveably confers a basic level advantage to neural networks has remained elusive. Here we provide such a function by generalizing the notion of category coherence C in the previous section to hierarchically structured categories. Indeed, in any dataset containing strong categorical structure, so that its singular vectors are in one to one correspondence with categorical distinctions, we simply propose to define the coherence of a category by the associated singular value. This definition has the advantage of obeying the theorem that more coherent categories are learned faster, through Eq. (6). Moreover, we show in SI Appendix that this definition is consistent with that of category coherence C defined in Eq. (15) for the special case of disjoint categories. However, for hierarchically structured categories as in Fig. 4 , this singular value definition always predicts an advantage for superordinate categories, relative to basic or subordinate. Is there an alternate statistical structure for hierarchical categories that confers high category coherence at lower levels in the hierarchy? We exhibit two such structures in Fig. 8 . More generally, in the SI Appendix, we analytically compute the singular values at each level of the hierarchy in terms of the similarity structure of items. We find these singular values are a weighted sum of within cluster similarity minus between cluster similarity for all levels below, weighted by the fraction of items that are descendants of that level. If at any level, between cluster similarity is negative, that detracts from the coherence of superordinate categories, contributes strongly to the coherence of categories at that level, and does not contribute to subordinate categories. Thus the singular value based definition of category coherence is qualitatively consistent with prior intuitive notions. For instance, paraphrasing Keil (1991) , coherent categories are clusters of tight bundles of features separated by relatively empty spaces [17] . Also, consistent with [3, 16, 17] , we note that we cannot judge the coherence of a category without knowing about its relations to all other categories, as singular values are a complex emergent property of the entire environment. But going beyond past intuitive notions, our quantitative definition of category coherence based on singular values enables us to prove that coherent categories are most easily and quickly learned, and also proveably provide the most accurate and efficient linear representation of the environment, due to the global optimality properties of the SVD (see SI Appendix for details). Discovering and representing explicit structures. While we have focused on hierarchical structure, the world may contain many different types of abstract structures. How are these different structures learned and encoded by neural networks? A convenient formalization of environmental structure can be specified in terms of a probabilistic graphical model (PGM), defined by a graph over items (Fig. 9 top) that can express a variety of structural forms underlying a domain, including clusters, trees, rings, grids, orderings, and hierarchies. Features are assigned to items by independently sampling from the PGM (see [29] and SI Appendix), such that nearby items in the graph are more likely to share features. For each of these structural forms, in the limit of a large number of features, we computed the item-item covariance matrices (Fig. 9 second row) , object analyzer vectors ( Fig. 9 third row) and singular values of the resultant input-output correlation matrix, and we employed them in our learning dynamics in Eq. 6 to compute the development of the network's internal representations through Eq. 8. These evolving hidden representations are shown in (Fig. 9 bottom) . Overall, this approach yields several insights into how distinct structural forms, through their different statistics, drive learning in a deep network, as summarized below: Clusters. Graphs that break items into distinct clusters give rise to block-diagonal constant matrices, yielding object-analyzer vectors that pick out cluster membership. Trees. Tree graphs give rise to ultrametric covariance matrices, yielding object-analyzer vectors that are tree-structured wavelets that mirror the underlying hierarchy [49, 50]. Rings and Grids. Ring-structured graphs give rise to circulant covariance matrices, yielding object-analyzer vectors that are Fourier modes ordered from lowest to highest frequency [51]. Orderings. Graphs that transitively order items yield highly structured, but non-standard, covariance matrices whose object analyzers encode the ordering. Cross-cutting Structure. Real-world domains need not have a single underlying structural form. For instance, while some features of animals and plants generally follow a hierarchical structure, other features like male and female can link together hierarchically disparate items. Such cross-cutting structure can be orthogonal to the hierarchical structure, yielding object-analyzer vectors that span hierarchical distinctions. In essence, these results reflect an analytic link between two very popular, but different, methods of capturing structure: PGM's and deep networks. This general analysis transcends the particulars of any one dataset, and shows how different abstract structures become embedded in the internal representations of a deep neural network. Over the course of development, the knowledge acquired by children powerfully reshapes their inductions upon encountering novel items and properties [2, 3]. For instance, upon learning a novel fact (e.g., "a canary is warm-blooded") children extend this new knowledge to related items, as revealed by their answers to questions like "is a robin warm-blooded?" Studies of inductive projection have shown that children's answers to such questions change over the course of development [2, 3, [17] [18] [19] , generally becoming more specific with age. For example, young children may readily project the novel property of warm-blooded to distantly related items, while older children will only project it to more closely related items. How could such changing patterns of inductive generalization arise in a neural network? Here, building upon previous network simulations of inductive projection [4, 31], we show analytically that deep networks exposed to hierarchically structured data, naturally yield progressively narrowing patterns of inductive projection across development. Consider the act of learning that a familiar item has a novel feature (e.g. "a pine has property x"). To accommodate this knowledge, new synaptic weights must be chosen between the familiar item pine and the novel property x (Fig. 10A) , without disrupting prior knowledge of items and their properties already stored in the network. This may be accomplished by adjusting only the weights from the hidden layer to the novel feature so as to activate it appropriately. With these new weights established, inductive projections of the novel feature to other familiar items (e.g. "does a rose have property x?") naturally arise by querying the network with other inputs. If a novel property m is ascribed to a familiar item i, the inductive projection of this property to any other item j is given by (see SI Appendix), This equation implements a similarity-based inductive projection of the novel property to other items, where the similarity metric is precisely the Euclidean similarity of hidden representations of pairs of items (Fig. 10B) . In essence, being told "a pine has property x," the network will more readily project the novel property x to those familiar items whose hidden representations are close to that of the pine. A parallel situation arises upon learning that a novel item possesses a familiar feature (e.g., "a blick can move," Fig. 10C ). Encoding this knowledge requires new synaptic weights between the item and the hidden layer. Appropriate weights may be found through standard gradient descent learning of the item-to-hidden weights for this novel item, while holding the hidden-to-output weights fixed to preserve prior knowledge about features. The network can then inductively project other familiar properties to the novel item (e.g., "Does a blick have legs?") by simply generating a feature output vector in response to the novel item as input. Under this scheme, a novel item i with a familiar feature m will be assigned another familiar feature n through the equation (SI Appendix), where the α th component of hn is h can be thought of as the hidden representation of feature n at developmental time t. In parallel to (16), this equation now implements similarity based inductive projection of familiar features to a novel item. In essence, being told "a blick can move," the network will more readily project other familiar features to a blick, if those features have a similar internal representation as that of the feature move. Thus the hidden layer of the deep network furnishes a common, semantic representational space into which both features and items can be placed. When a novel feature m is assigned to a familiar item i, that novel feature is placed close to the familiar item in the hidden layer, and so the network will inductively project this novel feature to other items close to i in neural space. In parallel, when a novel item i is assigned a familiar feature m, that novel item is placed close to the familiar feature, and so the network will inductively project other features close to m in neural space, onto the novel item. This principle of similarity based generalization encapsulated in Eqns. 16 and 17, when combined with the progressive differentiation of internal representations over developmental time as the network is exposed to hierarchically structured data, as illustrated in Fig. 2B , then naturally explains the shift in patterns of inductive projection from broad to specific across development, as shown in Fig. 10E . For example, consider specifically the inductive projection of a novel feature to familiar items (Fig. 10AB) . Earlier (later) in developmental time, neural representations of all items are more similar to (different from) each other, and so the network similarity based inductive projection will extend the novel feature to many (fewer) items, thereby exhibiting progressively narrower patterns of inductive projection that respect the hierarchical similarity structure of the environment (Fig. 10E) . Thus remarkably, even a deep linear network can provably exhibit the same broad to specific changes in patterns of inductive projection that are empirically observed in many works [2, 3, 17, 18] . Representational similarity matrix Σ h . Bottom row: Behavioral similarity matrix Σŷ. For small-norm, but not large-norm weight initializations, representational similarity is conserved and behavioral similarity mirrors neural similarity.  Compared to previous models which have primarily made behavioral predictions, our theory has a clear neural interpretation. Here we discuss implications for the neural basis of semantic cognition. Similarity structure is an invariant of optimal learning. An influential method for probing neural codes for semantic knowledge in empirical measurements of neural activity is the representational similarity approach (RSA) [20, 21, 28, 52], which examines the similarity structure of neural population vectors in response to different stimuli. This technique has identified rich structure in high level visual cortices, where, for instance, inanimate objects are differentiated from animate objects [22] [23] [24] [25] [26] . Strikingly, studies have found remarkable constancy between neural similarity structures across human subjects, and even between humans and monkeys [27, 28] . This highly conserved similarity structure emerges despite considerable variability in neural activity patterns across subjects [53,54]. Indeed, exploiting similarity structure enables more effective across-subject decoding of fMRI data relative to transferring a decoder based on careful anatomical alignment [55]. Why is representational similarity conserved, both across individuals and species, despite highly variable tuning of individual neurons and anatomical differences? Remarkably, we show that two networks trained in the same environment must have identical representational similarity matrices despite having detailed differences in neural tuning patterns, provided that the learning process is optimal, in the sense that it yields the smallest norm weights that solve the task (see SI Appendix for a derivation). One way to get close to the optimal manifold of synaptic weights of smallest norm after learning, is to start learning from small random initial weights. We show in Fig. 11AB that two networks, each starting from different sets of small random initial weights, will after training learn very different internal representations (Fig. 11AB top row) but will have nearly identical representational similarity matrices ( Fig. 11AB middle row) . Such a result is however, not obligatory. Two networks starting from large random initial weights not only learn different internal representations, but also learn different representational similarity matrices (Fig. 11CD top and middle rows) . This pair of networks both learn the same composite input output map, but with suboptimal large-norm weights. Hence our theory, combined with the empirical finding that similarity structure is preserved across humans and species, may speculatively suggest that all these disparate neural circuits may be implementing an approximately optimal learning process in a common environment. When the brain mirrors behavior. In addition to matching neural similarity patterns across subjects, experiments using fMRI and single unit responses have also documented a correspondence between neural similarity patterns and behavioral similarity patterns [21] . When does neural similarity mirror behavioral similarity? We show this correspondence again emerges only in optimal networks. In particular, denote byŷi the behavioral output of the network in response to item i. These output patterns yield the behavioral similarity matrix Σŷ ij =ŷ T iŷj . In contrast, the neural similarity matrix is Σ h ij = h T i hj where hi is the hidden representation of stimulus i. We show in the SI Appendix that if the network learns optimal smallest norm weights, then these two similarity matrices obey the relation [ 18 ] Moreover, we show the two matrices share the same singular vectors. Hence behavioral similarity patterns share the same structure as neural similarity patterns, but with each semantic distinction expressed more strongly (according to the square of its singular value) in behavior relative to the neural representation. While this precise mathematical relation is yet to be tested in detail, some evidence points to this greater category separation in behavior [27] . Given that optimal learning is a prerequisite for neural similarity mirroring behavioral similarity, as in the previous section, there is a match between the two for pairs of networks trained from small random initial weights (Fig. 11AB middle and bottom rows) , but not for pairs of networks trained from large random initial weights (Fig. 11CD middle and bottom rows) . Thus again, speculatively, our theory suggests that the experimental observation of a link between behavioral and neural similarity may in fact indicate that learning in the brain is finding optimal network solutions that efficiently implement the requisite transformations with minimal synaptic strengths. In summary, the main contribution of our work is the analysis of a simple model, namely a deep linear neural network, that can, surprisingly, qualitatively capture a diverse array of phenomena in semantic development and cognition. Our exact analytical solutions of nonlinear learning phenomena in this model yield conceptual insights into why such phenomena also occur in more complex nonlinear networks [4,[31][32][33][34][35][36][37] trained to solve semantic tasks. In particular, we find that the hierarchical differentiation of internal representations in a deep, but not a shallow, network (Fig. 2) is an inevitable consequence of the fact that singular values of the input-output correlation matrix drive the timing of rapid developmental transitions (Fig. 3 and Eqns. (6) and (10
2. We consider REF as the closest related work, we re-use their notion of simple deep linear neural network, that we call two-layer neural networks, and use some elements of their proofs to extend their results.
A mathematical theory of semantic development in deep neural networks
ArXiv
ArXiv
Computer Science Medicine Biology Mathematics