Nonlinear dynamics of localization in neural receptive fields
Authors: Leon Lufkin
Andrew M Saxe
Erin Grant
Venue: NeurIPS 2024 spotlight (2024)
Keywords: localization
receptive fields
learning dynamics
emergence
Primary area: neuroscience_and_cognitive_science
TL;DR: We analyze the dynamics of emergent localization in receptive fields of a nonlinear neural network, which enables us to bind emergence to specific higher-order statistical properties of the input data.

Abstract: Localized receptive fields—neurons that are selective for certain contiguous spatiotemporal features of their input—populate early sensory regions of the mammalian brain. Unsupervised learning algorithms that optimize explicit sparsity or independence criteria replicate features of these localized receptive fields, but fail to explain directly how localization arises through learning without efficient coding, as occurs in early layers of deep neural networks and might occur in early sensory regions of biological systems. We consider an alternative model in which localized receptive fields emerge without explicit top-down efficiency constraints—a feed-forward neural network trained on a data model inspired by the structure of natural images. Previous work identified the importance of non-Gaussian statistics to localization in this setting but left open questions about the mechanisms driving dynamical emergence. We address these questions by deriving the effective learning dynamics for a single nonlinear neuron, making precise how higher-order statistical properties of the input data drive emergent localization, and we demonstrate that the predictions of these effective dynamics extend to the many-neuron setting. Our analysis provides an alternative explanation for the ubiquity of localization as resulting from the nonlinear dynamics of learning in neural circuits

Nonlinear dynamics of localization in neural receptive fields
Leon Lufkin Yale University
Andrew Saxe Gatsby Unit & SWC, UCL a. saxe@ucl. ac.
Erin Grant Gatsby Unit & SWC, UCL erin. Faan.grant@ucl. ac uk
leon. www.kiinnallle.ee.
Abstract
Localized receptive ficlds-neurons that are selective for certain contiguous spa- tiotemporal features of their input-populate early sensory regions of the mam- malian brain. Unsupervised learning algorithms that optimize explicit sparsity or independence criteria replicate features of these localized receptive fields, but fail to explain directly how localization arises through learning without efficient coding, occurs in early layers of deep neural networks and might occur in early sensory regions of biological systems. We consider an alternative model in which localized receptive fields emerge without explicit top-down efficiency constraints feed- forward neural network trained on a data model inspired by the structure of natural images. Previous work identified the importance of non-Gaussian statistics localization in this setting but left open questions about the mechanisms driving dy- namical emergence. We address these questions by deriving the effective learning dynamics for a single nonlinear neuron, making precise how higher-ordet statistical properties of the input data drive emergent localization, and demonstrate that the predictions of these effective dynamics extend to the many- setting. Our analysis provides an alternative explanation for the ubiquity of localization resulting from the nonlinear dynamics of learning in neural circuits."
1 Introduction
A striking feature of peripheral responses in the animal nervous system is localization-tia that is, the linear receptive fields of simple-cell neurons often respond contiguous regions much smaller than their full input domain. In vision, retinal ganglion cells approximate localized surerr- filters that tile the input [Dac+00; Doi+12; KK78], and simple cells downstream primary visual cortex have localized filters that are selective for spatial frequency and orientation [HW59; HW68; RT95; NS08; W WMG11: I; RSH02; Rin02]. In primary somatosensory cortex, neurons respond to stimulation of restricted regions of skin [Cro+11] and in primary auditory cortex, spatiotemporal receptive fields are typically localized in both time and frequency domains (DW703; HDZ08); see Fig. (left).
By contrast, artificial learning systems do not always learn localized filters. Principal component analysis tends fit weights that span the entire input signal, do unregularized autoencoder neural network architectures and restricted Boltzmann machines [Sax+1 I]. This difference has prompted the search for artificial learning models that can learn localized receptive fields from naturalistic stimuli, the most notable of which are sparse coding [OF96; OF97] and independent component analysis [ICA; BS97; vHvdS98]. Sparse coding, ICA, and related compression methods that produce localized receptive fields from naturalistic data share a top-down approach- they find an efficient representation of the input signal by optimizing an explicit sparsity criterion, or an independence criterion that necessitates sparsity in critically parameterized regime [Fie99; Sax+11].
Code to replicate experiments and figures at http:////gtttttoooooo//o/lt///ar
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

[Rin02] [dBM98]
[Sin+18]
[KSHI2]
[HO00]
M1 SCM
Figure 1: (Left) Localization in spatial receptive fields (RFs) measured from non non-human primate (NHP) primary visual cortex [Rin02, Fig. 2] and in spatiotemporal RFs measured from NHP [dBM98, Fig. 2] and ferret [Sin+ 18, Fig. 2] primary auditory cortex. (Center) Half-slice of the localized first-layer kernels of AlexNet trained for ImageNet classification [KSH12]. (Right) Localized receptive fields learned from the task of Section 2.3 2-D using ICA [HO00] and soft committee machine (SCM; M1
fixed second-layer weights) of Section 2.1. Localization- Localization-spataa and/or temporal selectivity-appear across settings, as measured by response maximization in biological systems (left) and by inspecting linear filters in artificial systems (center, right).
Though sparsity is appealing potentially unifying explanation for localization, localization also emerges naturally in networks trained to perform classification tasks without any explicit sparsity regularization [KSH12: ZF13; Yos+15; Sen+18]; see Fig. (center) for an example. Ingrosso and Goldt [IG22] distilled such examples of emergent localization by demonstrating that localized receptive fields emerge in simple feedforward neural networks trained on data model with properties meant approximate natural visual input, in particular, locality structure (statistical independence of non non-collocated dimensions) and non-Gr non-Gaussianity (higher-order cumulants are non-null). In simulations, Ingrosso and Goldt [IG22] tie the dynamical emergence of localization to increased tuning to higher-order statistics of the input, and demonstrate that even a single neuron sufficient to learn localized receptive field in this setting.
this work, we build on the demonstration of Ingrosso and Goldt [IG22] with the aim of describing the mechanisms behind the emergence of a localized receptive field in this minimal setting. The higher-order input statistics that drive localization are challenging to analyze with existing tools that exploit implied Gaussianity [Gol+20]. By separating two stages of learning, equations for the effective carly-time learning dynamics of the single neuron model that learns a localized receptive field from idcalized naturalistic data. Our analytical model identifies a concise description the higher-order statistics that drive emergence, and we validate both positive and negative predictions of this analytical model via simulations with many neurons; Fig. 1 (right). These findings suggest an alternative path to account for the ubiquity of localization in early neural responses resulting from the interaction of the nonlinear dynamics of learning in neural eircuits and naturalistic data with higher-order statitistical structure, rather than an explicit efficiency criterion.
are able to derive
Modeling approach
We extend the setting of Ingrosso and Goldt [IG22], minimal example of a neural network that learns localized receptive fields from idealized naturalistic data. We analyze the dynamics of learning in this setting in Section 3 and validate our analytical model with simulations in Section 4.
2.1 Neural network architecture and learning algorithm
We consider two-layer feedforward neural network with nonlinear activation and scalar output. While simple, this architecture is highly expressive, capable of approximating arbitrary integrable univariate functions with appropriate scaling [Bar93; Pin99], and exhibits rich feature learning dynamics that underlie the performance of models at scale [Woo+20], making this architecture the ongoing subject of theoretical neural network analyses [MMN18; Gol+19 Vei+22]. We denote two-layer network with N- N-dimensional input, M hidden units, and one-dimensional scalar output as

---

Model 1 (many-neuron architecture).
(MI)
n(x) = ( + Σ m=1 Wmm
(2) a (Iml (w!l) x))
where o R + R is pointwise nonlinearity such as the rectified linear unit (ReLU) or sigmoid function, w!!h E R." and w!! E RR are learnable weights, b ((2) IR are learnable bias terms, and (1.) denotes the standard Euclidean inner (dot) product on R When the second-layer parameters fixed, this model known as a soft-committee machine (SCM; SS95], which [IG22] notes learns less noisy receptive fields but exhibits similar localization behavior. The many-neuron architecture in M1 is the focus of our simulations (Section 4), but the dynamics of this model are too complex to analyze directly, even for the idealized naturalistic data model considered here. In order derive analytical results (Section 3), we consider the simplest neural network exhibiting the desired localization phenomenon, single hidden neuron without bias and with rectified linear unit activation, written as
Model 2 (single-neuron architecture).
(M2)
jxx) ReLU ((w,x))
where (eLU(z) = max(x, 0), applied pointwise to vectorial input. As Ingrosso and Goldt [IG22] demonstrate, the localized receptive fields learned by the many- and single-neuron models defined in M1 and M2 are qualitatively similar up to spatial translation, which permits us generalize insights from analyzing the learning dynamics of the single-neuron M2 to the many-neuron MI. For simulations, we initialize the weights and biases independent draws from an isotropic Gaussian distribution with scaled variance, and train with batch gradient descent with fixed learning rate on the mean-squared error (MSE) evaluated input-output pairs from the task; see Section 2.3 for task sampling procedures.
2.2 Stimulus properties
The data model of Ingrosso and Goldt [IG22] can be shown to satisfy three conditions that enable the analysis give in Section 3. We consider several other data models that share the below properties but differ in generative mechanism order to probe the effect of these properties on localization.
In particular, we consider data X sampled from distributions p on Rº satisfying the following: Stimulus properties 1-3 (idealization of natural images).
(S1) (Positional) weak dependence: for any fixed p E (0, 1), as N +00
((N)= A
sup
E A) E B)] + 0, IP(,, E A, E B
BCR AN
(S2) Translation invariance: p(X x) = p({ = Sx) for E R.N. where S the
circular shift operator, and
(S3) Sign symmetry: p(X = x) p -x) for all E
Properties S1 and S2 are defining characteristics of natural image data [HHH09]] Property S3 can also be seen to hold for natural images after centering and is convenient analytically because it implies that [ O. Property SI assumess that p implicitly parameterized by N in order state that the statistical dependence between entries of X vanishes as their separation increases."
We denote the covariance X by Σ A Cov[X], the square of grincipa-ddiagne entries (the variance of each entry of X) by a² and the i-th row by d; Weak dependence (S1) implies that entries far from the principal diagonal of > will be O, while translation-invariann (i.e,, entries along each diagonal are equal) and thus identifiable by a single row; see Fig. 2 (center).
(S2) implies that Σ is circulant
"The weak dependence condition in S1 is based on strong a-mixing, a notion first introduced by [Ros56] to obtain a generalization of the central limit theorem, which we employ later on. We choose m-mixing because it is easy to interpret and verify, but alternative definitions of weak dependence [e.g.. Bar+08] can be substituted.

---

dimemiion
PCX.)
Ising
ippt dmmeriii dimemiin
004
Mmm h0m
wh Loan Mr pullyp
NLGP((0.01)
Upper
000
020
may mmmrrrs ham Impur dimension
ptX;)
(5)
Laper
0.0
Ieput dimession
isput dimession
Isput valoe
Figure 2: From left: Long- and short-lengthscale samples covariances E for one lengthscale, and marginals p(X;) for the data models described in Section 2.3: Ising (with J 1.2, 0.3 for left, right samples), the nonlinear Gaussian process INLGP; IG22], and the controllable kurtosis model, Kur (with g = 5, for left, right samples). Each model generates samples centered about zero and with covariances that can be constrained to be similar, but with differing higher-order statistics, as can be seen from the dümension-wis marginals.
2.3 Lengthscale discrimination task
Ingrosso and Goldt [IG22] develop a minimal task for which localization emerges in a feedforward neural network: binary discrimination between inputs from two distributions that differ in the lengthscale of the correlations between their entries. This lengthscale discrimination task can seen pretext task for self supervised learning [KZB19; Che+20] of representations [cf. unsupervised: OF96; BS97]. More precisely, we generate data (X, Y) for supervised training according to
X Y
(X:
(I)
where is to be defined, Σ, are distinct covariance matrices for each and we sample Y uniformly among a set of increasing lengthscale correlation classes y E {0, .... which correspond the strength of correlation between distant positions. For instance, in the case of two classes (y = 0, 1), we take Eg to be closer to T²T than Σ, where Ty is the N x N identity matrix and is fixed value. This construction isolates, via distinct covariance matrices per class, the second-order statistics, which we will see below enter into the learning dynamics separately from other properties of p(X), including, most critically, the implied marginal distributions, p(X;).
Ising. The first distribution we consider is the one-dimensional Ising model. It is of interest as distribution that satisfies SI to S3 with marginals p(X,) with extreme support on (+1), making the distribution that promotes localization most strongly, of an external field, the Ising distribution is
will see in Section 3. In the absence
Pising (X x) = Piasig (X, = Zi. X = IN) = EN J2 /2,
(2)
where J is chosen pairwise interaction strength, Z is the normalizing constant, and enforce periodic boundary constraint via v+++1 = Tt. As J increases, the lengthscale of the correlations in X also increases. For simulations, sample from Plaing using Gibbs sampler [GG84]. Diserimination tasks in the simulations in Section 4 use J, = 0.7 (for 1) and Jo = 0.3 (for 0).
(g) We also consider the data model used in Ingrosso and Goldt [IG22], the nonlinear Gaussian process (NLGP), which enables one to interpolate between distributions that do and do not yield localization via single parameter, g. sample X Y y from the NLGP constructed by first sampling Gaussian Z Y y ~ N Σ.)) and then transforming it via
x, ( Z(g)
IsisN,
(3)
where erf the Gauss error function, 2 is normalization constant to ensure that the variances of X; and Z, are the same, and Ê., is covariance matrix for Z, where we use (L,,,; = exp(-(i --) //²/5) for lengthscale parameter & [IG22]. If y x 0 (where localization is not observed), nZ, will tend to lie in the linear regime of erf, so Z, will be untransformed, i.e., is Gaussian. However, as g - oo (where localization is observed), g will tend to saturate erf, so X; will have support (=1).
(5,))=eexpp

---

(ur(k). The final family we consider is chosen to give us flexibility over the kurtosis K of the marginals p(X,). the Ising model, the excess kurtosis (x (x-3) of the marginals is fixed at -2, while in (L)P(g), it varies from -2 to O. This family allows us vary the excess kurtosis from negative through positive values. We sample X = y from this family via inverse transform sampling to vary the marginals while enforcing dependence via Gaussian copulas. More concretely, we sample Z - ~ (0,, Eg) and then transform it via
X, f
1SiSN,
where ē is the standard deviation of Z; T is the standard Gaussian cumulative distribution function (CDF), f is the CDF of the desired marginal distribution for X,, and Z is a normalization constant, which we compute numerically. We define S.y as for NLGP. We choose f to the generalized algebraic sigmoid function (see Appendix A.2) for k >> 0 to make use of its tractable inverse, simplifying the procedure in Eq. (4). We denote the corresponding distribution by Kur(k). Though we are able to continuously vary excess kurtosis, we lack explicit form; however, numerical computation shows that for k S 5.8, excess kurtosis is positive, while for 5.9, it is negative.
3 Theoretical results
We derive an analytical model for the localization dynamics of the single-neuron architecture in M2. This result establishes necessary and sufficient conditions for localization under Properties S1 to S3 for the minimal case of binary response, i.e., y = 0, 1. The conditions for localization the single- neuron architecture in M2 are demonstrated in Section to also hold empirically for the many-neuron architecture in M1. Further, we use this model to derive a negative prediction about localization, that the architectures in M1 and M2 fail to learn localized receptive field on elliptical distributions despite their non-Gaune Sonn-Gussian-ban particular, significantly positive partosis-statisticc [cf. positive kurtosis as an objective or diagnostic for localization, HO00; IG22].
3.1 An analytical model for the dynamics of localization in a single neuron
Previous approaches obtain analytical dynamics in the architectures in MI and M2 have studied the gradient flow under the assumption that the preactivation (w, X) is approximately Gaussian [Gol+20; Ger+20; Gol+22], but this assumption fails to capture the propagation of higher-order statistics through a neural network that promotes localization [IG22]. Happily, the idcalized visual input setting set out in S1 to S3 permits some simplification. In particular, the translation translation-invariance of the data X under S2 and the architecture of M2 allow us to work with the marginal distributions of each input dimension, X, rather than the full joint distribution of X.
We now give the analytical simplifications that allow us to derive an analytical model for the localization dynamics of the single neuron architecture in M2, namely two assumptions on X X; for all E {I,...,NY ] as a well as a mild condition on the weights that is satisficd at initialization. These are, where of to denotes the i-th row of Ey:
Analytical simplifications 1-3 (early-time, limiting dynamics).
(A1) X X; 2, Y = v) x. i.e., the conditional mean scales linearly with 2,-
(A2) Cov X = = x, y E, o² i.e., the conditional covariance is smaller near i, but independent of the exact value of zạ.-
(A3) Lindeberg's 's condition holds for the sequence X XN X Iz as N + for all Vi.-
Our motivation for Assumptions Al to A3 is that they replicate the kurtosis of the marginal distribu- tions X; (discussed further below) of two important and distinct limiting cases where localization does and does not appear, respectively: when X has support on the vertices of the hypercube {=1}" (satisfied by Ising for any J), and when X is Gaussian (satisfied by NLGP with g 0).
The gradient flow Lemma 3.1 also relies on Assumption A3 that Lindeberg's condition holds for the sequence W;X;, which ensures that no single term W;X; in the sequence can dominate. If this holds,
5

---

then we can conclude that (w.X) X, is approximately Gaussian. As we discuss in Appendix C.2, this is almost always satisfied for Gaussian initialization of w, and for slight deviations therefrom, and is satisfied by the settings of Ingrosso and Goldt [IG22]. Using this fact, we obtain an explicit form for the gradient flow early in training, stated in Lemma 3.1.
Lemma 3.1. Under Assumptions AI and A2, the gradient flow for the single ReLU neuron in M2 early in training with = 0, trained using MSE loss is
)
2 dw
Eyw
= 80 (E,W, W.
- (20 + 2/ + ON
(5)
T dt
where oN (1) vanishes as N + co, and where up : (-1, R is defined (x1
p(a) = Ex, - [x
alg alg (a)/v2)
(6)
and alg -1 (xx x/ x/VV x, the inverse of the algebraic sigmoid function alg(x) = x/ V 2
Lemma 3.1 reduces the study of higher-order statistics to the marginal distributions, 1,, where, by translation invariance, all marginals have the same distribution, so refer to X without loss of generality, While Lemma 3.1 technically only holds early in training and breaks down if w becomes localized due to violation of A3, the gradient flow in Eq. (5) holds sufficiently long detect the emergence of localization in the weights w. In particular, numerically integrating Eq. (5) yields localized weights w as t corresponds closely to the actual peak of the weight, when we observe localization; see Section 4.2 for empirical validation of this fact. The primary difference observed is that the localized bump from Eq. (5) is less peaked than when computed exactly; see Fig. 3 for a comparison between experimentally observed localized receptive fields and theoretical predictions.
Moreover, the location of the peak of final weights from Eq. (5)
3.2 Necessary and sufficient conditions for emergent localization
To establish an exact threshold at which localization emerges requires solving Eq. (5), which is not possible exactly for general nonlinear differential equations. Nevertheless, the form of Eq. (5) reveals that localization is driven solely by the first term. Indeed, the second term depends only on the second-order statistics of the data, and so be held fixed as X is varied from a distribution that induces localization to one that does not. Secondly, one can see that the first term in Eq. (5) does not change as is scaled, in contrast to the second term. As such, the second term in Eq. (5) serves to constrain the scale of w, distinct from localization, while the first is primarily concerned with the shape of w, and thus localization. This further motivates the first term, and thus p. which we will refer to as the amplifier and which itself depends on properties of the data distribution p( (X,, as a focus of study for understanding localization.
We present an analysis of y in Appendix B.I that reveals the role of the marginal distribution of the data in driving localization. For each marginal, (a(a Rs (/2/T)a for a Re O. For larger a, & depends more strongly the data distribution and can be super-linear (sub-linear), i.e. greater (smaller) than (v 22/T))a. Super-linear to encourage entries in w that are large in some neighborhood to grow faster than those that are smaller, yielding localization. Linear and sub-linear yp are the opposite, encouraging oscillatory flat weights by suppressing neighhorhoods in w. However, super- and sub-linearity may not hold uniformly, as can be both over its domain (see Fig. 3, bottom row, black line). As an approximation, we consider a third-order Taylor expansion (red lines in Fig. 3. second column), which reveals that for the canonical setting of a² = 1, negative excess kurtosis of the marginals yields super-linearins while positive excess kurtosis yields sub-lineariny see Appendix B. This leads us to the following claim, which is validated by our simulations in Section 4:
Claim 3.2. For sufficiently large N, if the data X E N satisfies conditions SI to S3 and has marginal distributions with sufficiently negative excess kurtosis, then Model M2 will learn localized receptive fields. Conversely, if the excess kurtosis is sufficiently positive, it will not.
As minimal positive example, the distribution with the most negative excess kurtosis is the symmetric Bernoulli, with a value of 2. In our setting, this corresponds to a data vector X with support on
We discuss partial differential equation limit that faces similar intractabilities in Appendix B.3.

---

Ising
1=-100 12
NOGP(0.01)
2.95
Kur
K 03
25 18
input
dimension
dimension
Figure 3: From and for same Ising, NLGP, and Kur data models as in Fig. marginals pf X x)), the amplifier y defined in Theorem 3.1 and kurtosis and the evolution of simulated receptive fields for the single-neuron model (M2) trained on its data, and lastly the receptive field given by numerically integrating Eq. (5) with yp expanded to third-order Taylor approximation for the data; training or evolution time indicated by line color (blue early-time; red late-time). See Section for exposition.
the vertices of the hypercube, (#1)% As mentioned above, it can be seen from the law of total covariance combined with sign-symmetry that Al and A2 hold exactly. Note that is the same for all such distributions, which leads to Claim 3.2 that any distribution satisfying conditions S1 to S3 whose marginals are maximally concentrated will induce a localized receptive field in M2. Importantly, this claim includes the limiting case of Ingrosso and Goldt [IG22], - in NLGP. also includes the Ising model as another example, corroborating an observation for restricted Boltzmann machines [Har+20] that Ising data induces localization in learning model. These claims are validated for the single-neuron model in Fig. and in Section 4 for the many-neuron model.
3.3 Case study: Elliptical distributions fail to produce localization
Above, we assume weak dependence (S1) as it enables a focus on how the marginals control localization. As first investigation into departures from this regime, we consider data X sampled from an elliptical distribution, where weak dependence may not hold. We specialize the definition of an elliptical distribution [Fra04] our setting of multiple class labels and sign-symmetry:
Def 1. Samples X E T satisfy elliptical distribution if we write X. Y y RyA U,
where R. is a nonnegative random variable, Ay E NxD is such that A T Ey and U is independent of Ry and uniformly distributed on the D-dimensional sphere.
The class of elliptical distributions is broad, imposing only the constraint that the contours of the density be ellipses; the multivariate Gaussian and Student-t distributions are examples. As such, they can vary greatly in measures of non- Gaussianity, including kurtosis, while maintaining enough structure for analytical convenience. Proposition 3.3 states that training elliptical data prevents localization the single ReL.U neuron model.
Proposition 3.3. Assume the data X. are sign-symmetric (S3), an elliptical distribution such that the MSE on the task in Section 2.3 is always finite. If Zig 2n are such that the ratio of their i-th eigenvalues, (x(2)) (,,n//((2)) (x,))), assumes a particular value for at most two distinct i, then the steady states of the weight of M2 are sinusoids, i.e., not localized.
(S2), and follow
The condition on the number i such that the ratio of the i-th eigenvalues are same constrains number of Fourier components that can be non-zero in the steady state of w. While opaque, this requirement seems always hold in practice, as even slight increases in length-scale correlation can dramatically change the spectrum of Ey.
7

---

(v 3)
elliptical shell
elliptical
magniiud
666
magnindd
mappinuk
3.64-
102
632
dimension & of weight w
dimension of weight w
dimension of weight w
Figure 4: Evolution of receptive fields learned by the single-neuron model (M2), along with sinusoids fit final states (red dashes) when trained on data from three elliptical distributions: (40 =3) (left), the surface of an ellipse (middle), and a custom elliptical distribution that places mass near outside of an ellipse (right). In all cases, the learned receptive field is oscillatory (a sinusoid), as predicted by Proposition 3.3. The l distances between the fitted oscillatory weights and empirieal RFs, as ratio of the fg norm of the empirical RFs, are (left) 9.77%, (center) 3.75%, and (right) 4.14%. See Section 4.3 for exposition.
The proposition is surprising because reveals that the kurtosis of the preactivation not an appropriate metric for explaining localization. Consider the example of the N N-dimensional Student-t distribution with degrees of freedom, ty (v). If X ty (z), then (w.X) X) t (v). Note the kurtosis of ( (v) is non-zero, and can be very large or even infinite for small v. This prediction is validated in Section 4.3. The condition also reveals that not all symmetries the data (here, elliptical symmetry) induce structure in the trained model weights, if localization is to be seen as sparsity more structured than oscillatory weights [cf. God+23]; indeed, translational symmetry (S2) is more relevant for localization than elliptical symmetry.
Experimental results
We describe experiments to validate the generalizability of the analytical results from Section 3. We run all experiments on a single CPU machine locally or on compute cluster. Since all datasets are procedurally generated, training depends on both the model architecture and the complexity of sampling the data, but between 10 and 60 minutes for any single simulation run.
4.1 Validating Claim 3.2 with positive and negative predictions
In Fig. 5, we validate Claim 3.2 first via the single-neuron model (M2) with 30 initial conditions trained across range of excess kurtoses for the NLGP(g) and Kur(k) data models. We use the inverse participation ratio (IPR), defined in Appendix A.3. This measure, also used by Ingrosso and Goldt [IG22), is large when proportionally few weight dimensions "participate" (have large magnitude), and small when weight dimension magnitudes are more uniform. We see that when g and k: assume values that yield negative excess kurtosis, IPR is close to its maximum of 1.0, suggesting the weights are localized; if the excess kurtosis is positive, IPR is nearly zero, suggesting the weights are nor localized. The IPR is extremely consistent across random initializations, suggesting that localization is determined by data statistics and not initialization. The trend in IPR vs. excess kurtosis is very similar between the NLGP(g) (g) and Kur (k) data models, demonstrating that excess kurtosis is primary driver of localization and localization is largely independent from other properties of the data distribution.
1.00
Figure 3 further validates Claim 3.2 with specific ex- amples. We maintain constant initial conditions for our model and train on the Ising, NLGP = 0.01), and Kur(k 5) data models. The marginals of the Ising model have an excess kurtosis of -2, the smallest possible value for any distribution. As re- sult, we see that the amplifier for Ising (top left) is super-linear (the dark line exceeds the dashed light line for larger a), which drives localization via its role Eq. (5). Integrating Eq. (5) with yp expanded via a third-order Taylor approximation (red line) yields a similar localized receptive field to that from simula- tion (two right panels), validating this approximation.
NLGP Kur
0.75
g 0.50
0.25
0.00
Excess kurtosis
Figure 5: IPR excess kurtosis for NLGP and Kur data models, with mean and std. dev. across 30 re-initializations for the single-neuro neuron model (M2); error bars are small and may not be visible.

---

model: MI; data: Kur(In
model: MI; data: Kur(1)
model: ICA; data: Kur(3)
mageinik
dimenssoo of weight
dimetsion of weight
dimersion of weight w
Figure 6: (Left, Center) Receptive fields learned by many-neuron (M1) soft committee machines (second-layer weights fixed at X trained the Kur(10) and Kur (4) datasets, respectively. The models had N 40 input units, K 10 hidden units, and an initialization variance of 0.1. (Right) A random subset 10 components from the 40 learned by the FastCCA algorithm from scikit-learn [Hyy99;; Ped+ 11] on the Kur (3) dataset with length-scale correlation values of Eo 1 and & = 3. See Section 4.4, for exposition.
For the remaining distributions (middle and bottom rows) that elicit oscillatory (sinusoidal) weights, Claim 3.2 validated due to their positive excess kurtosis. The dynamical steady state (far right) assumes more negative value than in the simulation (to the left), difference that is the result of deviations of our early-time gradient flow in Eq. (5), but these deviations remain mild enough nevertheless recover the qualitative structure of the learned receptive field.
4.2 Validating Eq. (5) with localization position prediction
The simulated and integrated receptive fields in Fig. 3 demonstrate that our analytical model is able meaningfully reproduce localization in receptive fields from neural network training. For the Ising model, we see that the integration even has peak in the exact same position as the simulation (at index 6), suggesting precision in our approximation. Indeed, we simulated the condition in Fig. for the Ising model under 28 different initial conditions (weight initializations). and found that in 26 of them (93%), the peaks of the integrated and simulated receptive fields matched exactly. In the two cases where the peaks differed, they did so substantially (see Fig. 8 for an example).
4.3 Validating Proposition 3.3: Elliptical distributions fail to localize
Proposition 3.3 claims that the single-neuron model (M2) trained elliptical data will yield sinusoidal receptive fields, subject to a condition on the spectra of Eg and Σn We verify this claim Fig. with three distinct elliptical distributions. The first, 140 (v 3), gives preactivations (w, X) that have infinite kurtosis, yet our theory predicts the final receptive field will be sinusoidal. This confirmed in Fig. 4, where the learned receptive field indeed sinusoid with period 1 and intercept at zero.
We also consider data sampled from the surface of an ellipse, which done by fixing R, = in Definition 1. Here, we observe that the learned receptive field is menstconstant function at 00.04 (note that cos(2n 0 x) = 1 is a sinusoid, allowing nonzero intercepts and constant functions). Finally, consider an unconventional elliptical distribution where the density of R is given by Pn(()= (4c2++4)((e (e?" + before rapidly falling off, imposing minimum norm on X and pushing support near the surface of an ellipse. This distribution, too, yields oscillatory steady state, as shown in Fig. 4 (right). We confirm our visual observations by fitting sinusoids to the final receptive fields and see the relative
1(r 2 2). This particular density places most of its mass near
emrrr are quite low.
4.4 Extensions to many-neuron model and ICA
All of our analysis thus far has concerned single-neuro models with ReLU activation and without hidden-to-outpat bias terms, assumptions which were made make our analysis tractable. Here, we depart from that regime by considering the SCM and the full two-layer network (Model M1). In Fig. 6 (left) and (center), train SCM with 10 hidden units and sigmoid activation on the Kur(10) and Kur(4 (4) datasets, which have excess kurtoses of -0.93 and 3.28, respectively. So, based on our single-neuron analysis, do and do not expect see localization for these distributions. Indeed, this is precisely what we observe in Fig. 6, where the receptive fields are sharply localized for the former distribution, while they look like low-frequency oscillations for the latter.

---

In Fig. 7, we train many-neuron models with N 40 input units and K 10 hidden units, where all weights are learnable. In general, adding flexibility in the second layer leads to more varied structure in the first layer. We train on Kur(4 (4) (top), which has an excess kurtosis of 3.28, and Kur(30) (bottom), which has an excess kurtosis of -I.17. The receptive fields from the former are not localized, as in the single- neuron model: however, they appear more like high- frrqueecy oscillations than low- now-frequency sinusoids. For Kur (30). where we expect localization, see that the first three receptive fields exhibit localization, but less so than for a single neuron. Importantly, not all receptive fields are localized, a result of a variable second-laye weight effectively changing the variance a² in the third-derivative term in Lemma B.I.
magnindde
y
dimenviion of weight w
Figure 7: Receptive fields learned by the many- neuron model (MI) with learnable second- sccond-layer weights, N 40, K 10. (Top) A random subset of 4 receptive fields from a model with sig- moid activation, trained on Kur (4) (positive ex- cess kurtosis of 3.28). As predicted by Claim 3.2, the receptive fields not localized, and appear as high-frequency oscillations. (Bottom) A ran- dom subset of 4 receptive fields from a model with ReLU activation, trained on Kur (30) (negative excess kurtosis of 1.117) Receptive fields are localized (left three) or exhibit low-frequeney os- cillations (right). See Section 4.4 for exposition.
We further compare these predictions against ICA, another framework that has been used to model re- ceptive fields in visual cortex. We train on the Kur (3) dataset, which has marginals with excess kurtosis 7.66, fitting 10 components using the FastICA imple- mentation from scikit-leamn [sc; HO00]. We observe in Fig. 6 (right) that we learn localized receptive fields; this contrasts our neural network models, which require negative excess kurtosis. This stems from ICA's objective to maximize non-Gaussianity, regardless of how specifically it is done. The sign of the excess kurtosis is irrelevant, so long as it is nonzero. This deviation between our analytical model and ICA is interesting avenue for future study, perhaps by validation with natural images.
5 Conclusions
We derive effective learning dynamics for the minimal example of emergent localization in a neural receptive field given by Ingrosso and Goldt [IG22]. The analytical approach we take relies on the assumption that the conditional preactivation is Gaussian, refinement of previous work that assumes Gaussianity of the unconditioned preactivation as asserted by the Gaussian equivalence property targeted by Ingrosso and Goldt [IG22]. This approach may prove extensible beyond our specialized setting and may enable further analysis of how statistics of an input task drive emergent structure in neural network learning.
Emergence as an alternative mechanism to top-down constraints like sparsity is in line with recent work that reformulates cuta-distributional properties as driver for complex behavior [Cha+22]. Via these analytical effective dynamics. we observe that specific data properties- -te covariance structure and the marginals- localization in neural receptive fields. Though we cannot capture dynamical interactions between neurons that may shape receptive fields in other settings with the single-neuron analytical model, our empirical validations with many neurons suggest that these interactions do not, in fact, play a significant role in shaping localization [cf. Har+20].
The data model we consider is a simplified abstraction of the task faced by early sensory systems, and, as a consequence, we do not yet capture certain features of receptive fields that are observed in early sensory systems. In particular, we do not observe orientation nor phase selectivity, features of simple-cell receptive fields in early sensory cortices and in artificial neural networks that can be seen in subset of receptive fields in Fig. 1 (left and center, respectively). To capture orientation selectivity, may be fruitful to follow the approach of Karklin and Simoncelli [KS1]], who tie orientation selectivity in copulation-based efficient-coding framework to the presence of noise. Furthermore, input data, including the idealized data, gives receptive fields with subfields in our simulations, but is difficult to analyze. Lastly, we do not yet look at the distribution of receptive field shapes and do not validate against other models of receptive field learning beyond a brief comparison with ICA Icf. Sax+1 I], but these are exciting avenues for future work.
10

---

Acknowledgements
This work was supported by a Schmidt Science Polymath Award A.S., and the Sainsbury Well- come Centre Core Grant from Wellcome (219627/2/19/Z)) and the Gatsby Charitable Foundation (GAT3850). A.S. is CIFAR Azrieli Global Scholar the Learning Machines & Brains program.
References
[Bar+08]
J.-M. Bardet, P. Doukhan, G. Lang, and N. Ragache. "Dependent Lindeberg central limit theorem and some applications". In: ESAIM: Probability and Statistics 12 (2008), pp. 154-172 (cit. on p. 3).
[Bar93]
R. Barron, "Universal approximation bounds for superpositions of sigmoidal function". In: IEEE Transactions on Information theory 39.3 (1993), pp. 930-945 (cit. on o. p.
[Bra07]
R. C. Bradley. "Introduction to strong mixing conditions". In: (No Title) (2007) (cit. on
p. 17).
[BS97]
Bell and Sejnowski. "The "Independent Components" of Natural Scenes Are Edge Filters". In: Vision Research 37.23 (Dec. 1, 1997), pp. 3327-3338. URL: https: (cit.
//www. sciencedirect. com/ sessscieccceee httpa///arrraaee//////00/2200100 pp. 1, 4).
[Cha+22]
S. C. Y. Chan, A. Santoro, A. K. Lampinen, J. X. Wang, A. Singh, P. H. Richemond, J. McClelland, and F. Hill. "Data Distributional Properties Drive Emergent In-Context Learning in Transformers". In: Advances in Neural Information Processing Systems. Conference on Neural Information Processing Systems. Oct. 10, 2022. arXiv: 2205. 05055 [ca]. URL: http: h/tp//axvvvv.///// abs/2205. 05055 (cit. p. 10).
[Che+20)
T. Chen, Kornblith, M. Norouzi, and G. Hinton. Simple Framework for Con- trastive Learning of Visual Representations" In: P Proceedings of the 37th International Conference on Machine Learning. International Conference on Machine Learning. Pmlr, Nov. 21, 2020, pp. 1597-1607. URL: httpss:/proceediiiei chen20j .html (cit. on p. 4).
mlr press vv19/
[Cro+11]
S. Crochet, J. F. Poulet, Y. Kremer, and C. C. Petersen. "Synaptic Mechanisms Underly- ing Sparse Coding of Active Touch". In: Neuron 69.6 (Mar. 2011), pp. 1160-1175. URI.: on p. 1).
elsevier cow/retrieve.eeoeecoocommmmoeoee wttpiieeeeeeiieeeeeee http://eerreeeeppp////0/66610000 /0011//00062271000000000 (cit.
[Dac+00]
Dacey, O. Packer, L. Diller, D. Brainard, B. Peterson, and Lee. "Center Surround Receptive Field Structure of Cone Bipolar Cells in Primate Retina". In: Vision Research 40. 14 (June 1, 2000), pp. 1801-1811. URL: https: :tpps://www ///www... sciencedirectt. com/ sci ence/ carriiin clcccleee/ec /tt.c////.e///00999000000000
(cit. p. 1).
[dBM98]
R. C. deCharms, D. T. Blake, and M. M. Merzenich. "Optimizing Sound Features for Cortical Neurons". In: Science 280.5368 (1998), pp. 1439-1444. eprint: https: //www. science org/ doi / pdf 10 1126 science 280 5368 1439. URL: https: //www.acienceeee.m////////e///..
science. org/ doi abs / 10. 1126 science 280 5368 1439
(cit. p. 2).
[Doi+12]
E. Doi, J. L. Gauthier, G. D. Field, J. Shlens, A. Sher, M. Greschner, T. Machado, L. H. Jepson, K. Mathieson, D. E. Gunning, A. M. Litke, L. Paninski, E. J. Chichilnisky, and E. P. Simoncelli. "Efficient Coding of Spatial Information the Primate Retina". In: The Journal of Neuroscience 32.46 (Nov. 14, 2012), pp. 16256-16264 URL: https: httg//oooup//// doi / 10 1523 JNEUROSCI 4036 12. 2012
//www jjnurosci.
(cit. p. 1).
[DWZ03]
M. R. DeWeese, M. Wehr, and A. M. Zador. "Binary Spiking in Auditory Cortex".
In: The Journal of Neuroscience 23.21 (Aug. 27, 2003), pp. 7940-7949, URL: https: //www. jneurosci. wwwpoookg/////aaeooooooooo http://oooo////////// 1523/ BER////NERRSCCC 23-221-77940- 2003
(cit. on p. 1).
[EC24]
O. Elkabetz and N. Cohen. "Continuous vs. discrete optimization of deep neural networks". In: Proceedings of the 35th International Conference on Neurul Information Processing Systems. Curran Associates Inc., 2024 (cit. on 17).

---

[Fie99]
J. Field. "Wavelets, Vision and the Statistics of Natural Scenes". In: Philosophical Transactions of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences 557.1760 (Sept. 1999). Ed. by B. W. Silverman and J. C. Vas- silicos, pp. 2527-2542. URL: https: bttp:://reyalssccieeppull..ssss 1098/rsta. 1999. 0446 (cit. on p. 1).
org/ org/doi/100
[Fra04]
G. Frahm. "Generalized elliptical distributions: theory and applications" PhD thesis. Universität zu Köln, 2004 (cit. on pp. 7, 19).
[Ger+20]
F. Gerace, B. Loureiro, F. Krzakala, M. Mézard, and L. Zdeborová. "Generalisation error in learning with random features and the hidden manifold model"" In: Proceedings of the 37th International Conference on Machine Learning. ICML 2020, 13-18 July 2020. Virtual Event. Vol. 119. Proceedings of Machine Learning Research. PMLR, 2020, pp. 3452-3462. URL: http: btctt::/prreee pttp//eeoccceeesss///eeee/. mlr www.ppressvvvieeee gerace20a..... htmmll (cit. on pp. 5, 16).
[GG84]
S. Geman and D. Geman. "Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images". In: IEEE Transactions Pattern Analysis and Machine Intelligence Pami-6.6 (Nov. 1984), pp. 721-741. URL: org/ wwwcccuent/4767770
ieee.
(cit. on p. 4).
[God+23] C. Godfrey, D. Brown, T. Emerson, and H. Kvinge. On the Symmetries of Deep Learning Models and Their Internal Representations. Mar. 24, 2023. arXiv: 2205 14258 [cs].
URL: http: : / //arxivv...////////////////aavv org/ abs 2205/ 14258. preprint (cit. on p. 8).
[Gol+19] S. Goldt, M. Advani, A. M. Saxe, F. Krzakala, and L. Zdeborová. "Dynamics of stochastic gradient descent for two-layer neural networks in the teacher-student setup"..
In: Advances in neural information processing systems 32 (2019) (cit. on p. 2).
[Gol+20] S. Goldt, M. Mézard, F. Krzakala, and L. Zdeborová. "Modelling the Influence of Data Structure on Learning in Neural Networks: The Hidden Manifold Model". In: Physical Review X 10.4 (Dec. 3, 2020), p. 041044 arXiv: 1909. 11500 [cond-mat, stat]. URL: http: : // //trxivvvvv//////////eee//aaavl/ org/ abs 1909. 11500 (cit. on pp. 2. 5, 16).
[Gol+22]
S. Goldt, B. Loureiro, G. Reeves, F. Krzakala, M. Mézard, and L. Zdeborová. "The gaussian equivalence of generative models for learning with shallow neural networks". In: Mathematical and Scientific Machine Learning. Pmlr. 2022, pp. 426-471 (cit. on
pp. 5, 16).
[Har+20]
M. Harsh, Tubiana, Cocco, and R. Monasson. " "Place-cell' Emergence and Learn- ing of Invariant Data with Restricted Boltzmann Machines: Breaking and Dynamical Restoration of Continuous Symmetries in the Weight Space". In: Journal of Physics A: Mathematical and Theoretical 53.17 (Apr. 2020), 174002. URL: https: / //xx/ doi.
org/10. 10. 1 11//177111112 (101--812//ab///00
(cit. on pp. 7, 10).
[HDZ08]
T. Hromádka, M. R. DeWeese, and A. M. Zador. "Sparse Representation of Sounds in the Unanesthetized Auditory Cortex". In: PLOS Biology 6.1 (Jan. 29, 2008), c16. URL: pwwpsoorg//oosiolll.ooooo//// plos org/ httppploooooooooo/o/oo/oo http://artcccieiddaaaaee
1 journal..
gbio.0060016 (cit. p- 1).
[НHН09]
A. Hyvärinen, Hurri, and P. O. Hoyer. Natural Image Statistics. Red. by M. Viergever, G. Borgefors, R. Deriche. T. S. Huang, K. Ikeuchi, T. Jiang, R. Klette, A. Leonardis, H.-O. Peitgen, and K. Tsotsos. Vol. 39. Computational Imaging and Vision. London: Springer, 2009. URL: http: httpt:://innssssingeerr.r com/10. 97701007-----8888888 1-18-----8882-991111-- (cit. on p. 3).
[HO00]
A. Hyvärinen and E. Oja. "Independent Component Analysis: Algorithms and Ap- plications". In: Neural Networks 13.4 (June 1, 2000), pp. 411-430. URL: https
//www. wcwencedirect.....mmmc.......... wtw.sccoomieeeeeeeeeeeeoeeeeooee http//ccenceeeeeeeeeeeeeee httpa////aa.cceeeeee/////ooooo// pp. 5, 10).
(cit.
[HW59]
D. H. Hubel and T. N. Wiesel. "Receptive Fields of Single Neurones in the Cat's Striate Cortex". In: The Journal of Physiology 148.3 (Oct. 1959), pp. 574-591, pmid:
14403679 (cit. on p. 1).
[HW68]
D. H. Hubel and T. N. Wiesel. "Receptive Fields and Functional Architecture of Monkey Striate Cortex", In: The Journal of Physiology 195.1 (Mar. 1968), pp. 215-243, pmid: 4966457 (cit. on p. 1).
12

---

[Hyv99]
A. Hyvarinen. "Fast and Robust Fixed-Point Algorithms for Independent Component Analysis". In: IEEE Transactions on Neural Networks 10.3 (May 1999), pp. 626-634. URT: httt::://ieeepplooo
leee. org/ document/761722) (cit. on p. 9).
[IG22]
A. Ingrosso and S. Goldt. "Data-driven emergence of convolutional structure in neu- ral networks". In: Proceedings of the National Academy of Sciences 119.40 (2022),
e2201854119 (cit. on pp. 2-8, 10, 16, 20).
[KK78]
E. Knudsen and M. Konishi. "Space and frequency are represented separately in auditory midbrain of the owl". In: Journal of Neurophysiology 41.4 (1978), pp. 870-884 (cit. on p. 1).
[KS1]]
Y. Karklin and E. Simoncelli. "Efficient Coding of Natural Images with a Population of Noisy Linear-Nonlinen
Neurons". In: Advances in Neural Information Processing Sys-
tems. Vol. 24. Curran Associates, Inc., 2011. URL: https: / /ttp://////oo/ogpp//e neurips cc/ cc/paper/201 ht///2111/hhaahss hathhhh/2e59933333 www.aae000000004e6e611636666 www.adeeeeea---aaaataeaeaa htmll (cit. on p. 10).
[KSH12]
A. Krizhevsky, I. Sutskever, and G. E. Hinton. "ImageNet Classification with Deep Convolutional Neural Networks". In: Advances in Neural Information Pro- cessing Systems. Vol. 25. Curran Associates, Inc., 2012. URL: https / proceedings neurips cc paper 5C % 5Ffiles / paper / 2012 hash Abstract html (cit. on p. 2).
[KZB19]
A. Kolesnikov, X. Zhai, and L. Beyer. Revisiting Self- Supervise Visual Representation Learning. Jan. 25, 2019. arXiv: 1901.09005 09005 [cs]. URL: http: / arxiv. org/abs/ 1901. 0w0.990050. preprint (cit. on p. 4).
[MMN18]
S. Mei, Montanari, and P.M. Nguyen. "A mean field view of the landscape of two- layer neural networks". In: Proceedings of the National Academy of Sciences 115.33
(2018), e7671 (cit. on 2)
[NS08]
C. M. Niell and M. Stryker. "Highly Selective Receptive Fields in Mouse Visual
Cortex". In: The Journal of Neuroscience 28.30 (July 23, 2008), pp. 7520-7536. URL: 1ookup / doi 10. 1523 JNEUROSCI 0623
https: : / www jneur
08. 2008 (cit. on p. 1).
[OF96]
B. A. Olshausen and D. Field. "Emergence of Simple-Cell Receptive Field Properties
by Learning Sparse Code for Natural Images". In: Nature 38 381.6583 (June I, 1996),
pp. 607-609. URL: https: / //w.i.d/////o/////////.. org/10. 0008333180088000 a0 (cit. on pp. 1,4).
[OF97]
A. Olshausen and D. J. Field. "Sparse Coding with an Overcomplete Ba- sis Set: A Strategy Employed by VI?" In: Vision Research 37.23 (Dec. 1997), pp. 3311-3325. URL: https: //linkinghub. elsevier com / retrieve / 50042698997001697 (cit. on p. 1).
[Ped+11]
F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. "Scikit-learn: Machine Learning in Python". In: Journal of Machine Learning Research 12 (201 1), pp. 2825-2830 (cit. on 9).
[Pin99]
A. Pinkus. "Approximation theory of the MLP model in neural networks". In: Acta numerica (1999), pp. 143-195 (cit. on p. 2).
[Rin02]
D. L. Ringach. "Spatial Structure and Symmetry of Simple-Cell Receptive Fields in
Macaque Primary Visual Cortex". In: Journal of Neurophysiology 88.1 (July 2002), URI.: https: / htappp:://juuraeeeeeeeee journals...... .hhsioloogyy org/ /oi htrg/doi/fulll fwll/llllll////// 1152/ jn. 20..002..2 88. 455 (cit. on pp. 1)
pp. 455-463.
[Ros56]
M. Rosenblatt. "A A central limit theorem and strong mixing condition". In: Proceedings of the national Academy of Sciences 42.1 1 (1956), pp. 43-47 (cit. on p. 3).
[RSH02]
D. L. Ringach, R. M. Shapley, and M. J. Hawken. "Orientation Selectivity in Macaque v1: Diversity and Laminar Dependence", In: The Journal of Neuroscience 22.13 (July 1, URL: https / www jneurosci org/ /atg//ooup//////// doi 10.
2002), pp.
22- 02-13-05533 2002 (cit. on p.p.11 1).
[RT95]
E. T. Rolls and M. J. Tovee. "Sparseness of the Neuronal Representation of Stimuli in the Primate Temporal Visual Cortex". In: Journal of Neurophysiology 73.2 (Feb. 1995), pp. 713-726. URL: https: www. physiology. org/ doi /00. 1152/ jn.
1995. 73. 713 (cit. on 1).
13

---

[Sax+11]
A. Saxe. M. Bhand, R. Mudur, B. Suresh, and A. Ng. "Unsupervised Learning Models of Primary Cortical Receptive Fields and Receptive Field Plasticity". In: Advances in Neural Information Processing Systems. Vol. 24. Curran Associates, Inc., 2011. URL: https proceedings neurips. cc paper / 2011 hash/
htt.://cc.3.aoobooeeeee666666666 Abstrscttbbttttttt..o.. html (cit. on pp. 1, 10).
[Sen+18]
A. Sengupta, C. Pehlevan, M. Tepper. A. Genkin, and D. Chklovskii. "Manifold- Tillng Localized Receptive Fields Are Optimal in Similarity- Neural Networks".
In: Advances in Neural Information Processing Systems. Vol. 31. Curran Associates, Inc., 2018. URL: https: proceedings neurips- cc / paper / 2018 / hash ee14c41 eew4c41e92ecceecccccc 9ecccceec225c997555999744444
Abstract html (cit. on p. p. Y. Singer. Y. Teramoto, B. Willmore, J. W. Schnupp. A. J. King, and N. S. Harper. "Sensory Cortex Is Optimized for Prediction of Future Input". In: eLife 7 (June 18, 2018). Ed. by J. L. Gallant and S. Kastner, e31557. 7554/eLife 31557 (cit. on p. 2).
[Sin+18]
URL: https : //t////o..... org/10.
[SS95]
D. Saad and S. A. Solla. "On-line learning in soft committee machines". In: Physical Review E 52.4 (1995), 4225 (cit. on p. p.
[Vei+22]
R. Veiga, L. Stephan, B. Loureiro, F. Krzakala, and L. Zdeborová. "Phase Di- agram of Stochastic Gradient Descent in High-Dimensional Two-Layer Neural Networks". In: Advances in Neural Information Processing Systems. Neural In- URL: formation Processing Systems. Vol. 35. Dec. 6, 2022, pp. 2022 / hash / https : // papers nips cc / paper % 50% 5Ffiles /
paper / Abstract Conference. html (cit. on
p. 2).
[vHvdS98]
J. H. van Hateren and A. van der Schaaf. "Independent Component Filters of Natural Images Compared with Simple Cells in Primary Visual Cortex". In: Proceedings of the Royal Society of London. Series B: Biological Sciences 265.1394 (Mar. 7, 1998). pp. 359-366. URL: https rapb. 1993 (cit. on p. 1).
/royaloccietypubiihhng org doi / 10 1098/
[WMG11] D. B. Willmore, J. A. Mazer, and J. L. Gallant. "Sparse Coding in Striate and Ex- trastriate Visual Cortex". . In: Journal of Neurophysiology 105.6 (June 2011), pp. 2907- 2919. pmid: 21471391, URL: https: www.wwwccbb.........oo nlm. nih waallggo/ppree gww./p//aatccclles..eeeee aarticeettees PMC3118756/ (cit. on p. 1).
[Woo+20]
B. Woodworth, S. Gunasekar, J. D. Lee, E. Moroshko, Savarese, I. Golan, D. Soudry, and N. Srebro. "Kernel and Rich Regimes in Overparametrized Models". In: Proceed- ings of Thirty Third Conference on Learning Theory. Conference Learning Theory. Pmlr, July 15, 2020, pp. 3635-3673, URL: https: http://ppooe...../////////////ee mlr press/ v125/wo 1125//wodwoorhh206 html (cit. on p. 2).
[Yos+15]
J. Yosinski, J. Clune, Nguyen, Fuchs, and H. Lipson. "Understanding Neural Networks through Deep Visualization". In: (June 22, 2015). arXiv: 1506. 06579 [cs].
URL: http:://t// //tpr//avvvv///////////e.//aavv. org/ aa/aabas/1000 06579, preprint (cit. on p. 2).
[ZF13]
M. D. Zeiler and R. Fergus. "Visualizing and Understanding Convolutional Networks".
In: (Nov. 28, 2013). arXiv: 1311. 2901 [cs]. URL: http:/ // /arxivv. org/ harg//bb/// //////bb/1111///
2901. preprint (cit. on 2).
14

---

A Definitions and Notation
A.1 Notation
We use [xi] to refer to the set {i (iEN:SiSm
A.2 Algebraic sigmoid
For h > 0, the generalized algebraic sigmoid function is defined as
alga (x)
+
2
(1 (1+ t
Following the main text, we drop the subscript when 1: = 2.
A.3 Inverse participation ratio (IPR)
The IPR is defined as:
D Σ
(P) &
Σ
w
where wi is the magnitude of dimension of weight w.
B Extensions beyond the scope of the main text
B.1 Analytical properties of the amplifier y
We present several properties of the amplifying function yp defined in Lemma B.1. Lemma B.1. The localization amplifier in Eq. (6) satisfies (p(a) -e(a), for all E (-1,1). Moreover. its derivatives satisfy, where o² and denote the variance and kurtosis of x,, respectively:
2'(0)
-
2
and
(0 - -2
π -
π
To gain some understanding of how the marginal distributions of impact localization, we use the derivatives in Lemma B.1 to construct third-order Taylor approximation of yp about The derivatives in Lemma B.1 reveal that every distribution for X, with constant variance will look like the same linear function near 0. to only looks nonlinear once move sufficiently far away from when the third-order term becomes relevant. For the case of o² (where the variance of X, is equal to the value of the larger target = 1) the third order term suggests that yo is super-linear when 3, i.e., the excess kurtosis positive, and sub-linear otherwise.
super-linear & will encourage entries where Mn w is large to grow at a faster rate than other entries, which are all subject to the same linear norm constraint through the second term in Eq. (5). The covariance Σ, circulant matrix, acts the convolution operator between some vector and o (the first row in E Since y corresponds to the class with larger length-scale correlation, di will decay relatively slowly and act like a weight local average. Thus, Σ w the weighted local average for each entry in So, entries where is large within some neighborhood will be encouraged to grow faster than those which are smaller, an effect that compounds as Eq. (5) is integrated. Thus, super-linearity encourages localization.
As we will see Theorem 3.3 for the setting of elliptical data, yp is linear w learns be sinusoidal, and thus not localized. In the case of sub-lincarity, than promotion, as in the super- setting. Thus, to a first approximation, the sign of the excess kurtosis, & 3 (for o² = 1), indicates whether w localizes.
expect suppression of larger values, rather
However, simply studying ""((0) is not sufficient to fully characterize how the marginals impact localization. A function can be sub-linear for small a and super-linear for larger a, making unclear whether it will yield localization. For marginal distributions where 2 that do not exhibit strict super- or sub-linearity, this condition no longer precise enough to determine whether we see localization.
15

---

B.2 PDE limit of Eq. (5)
By taking to be large and treating w as continuous function with respect to position, i.e., w = w(2, t), one can treat Eq. (5) as partial differential equation (PDE). Finding its steady state then amounts solving
o *
- 2タ2 1o'00 +0') x =0,
(8)
10" w. w)
where w: :0, 1] - IR is periodic and o" is the convolution corresponding to the limiting case of the matrix E, This equation does not appear to have an explicit solution for non-identity Σ,, and thus, may not be possible to solve the steady states of Eq. (5) exactly in this PDE limit or for finite N.
B.3 Assumptions A1 and A2 vs. Gaussian equivalence
Assumptions Al and A2 are equivalent to approximating (w.X) X; as Gaussian early in training. Similar ideas have been used derive gradient flow dynamics for neural networks, including in developing the Gaussian equivalence property of [Gol+20; Ger+20; Gol+22]. However, these works model the unconditional preactivation (w, X) as a Gaussian, rather than first conditioning on X;- How this arises is that these previous works assert the Gaussian approximation prior to differentiating the loss function for the gradient flow dynamics. However, an approximation that stage neglects the presence of multiplicative factor X that appears as result of the chain rule applied to (w, X). Abstractly, this approach assumes that Cexact + LGauss implies V Lexact - V L but, in general, this does not follow, and here in particular this assumption does not capture the interplay of learning and higher-order input statistics. This contributes to the failure of Gaussian equivalence in [IG22]. In contrast, by assuming that (w. X) X2 rather than (w, X) is Gaussian. This conditioning approach, along with the translation invariance of the data (Property S2), also motivates considering the marginal distributions X, as the object of study to obtain gradient flows for neural networks trained on non-Gaussian inputs.
account for the additional X term in the derivation of Lemma 3.1
C Proofs of theoretical results
C.1 Gradient flow for mean-squared error (MSE) loss
The loss is given by:
Ex.y ( RRLUU((,,X)))) x)
Ex.y [/2] -2xx,y Y ReLUU((, X Ex.y
(ew,
(9)
A(1
The assumption of sign symmetry (S3) gives that (w, X) is also sign-symmetric. First, this implies
that (P(ww X) 0) 2 so:
(IT)=
2 E ww. ReLUU((w x))
Ex (w. ((w, x))
Second, sign-symmetry of (w, X) implies that we can drop the conditioning on (w. X) 2 0, since
(w.X) X) _ (w, Thus,
(m = Ex.y [(w,
((w,X))² x()2
2
w
Ex,Y
Xx
K-1
= w
w
2
K
0=
K-1
Σ
w
Σiy w,
2K
y=
16

---

where K is the number of values (classes) of discrete Finally, we differentiate L with respect to w:
K-1
V L 2Exx (I((W, 2 0)
Σ
y=0
The gradient flow [EC24] is given by dw
de = - V L. where T the learning rate. Thus,
dw
2 Ex, [VI 20xx
Σ
S,w.
(10)
т di
K
y=
C.2 Proof of lemma 3.1
Setting K = 2 in equation (10), we have
1 dw
1 = E 2 0)X]
(Eo + Σ1) w.
dt
2
We wish to express the first term explicitly. Note that the first term is vector in R. We consider each its entries separately by using the law of total expectation to write the i-th entry as:
(1(,, X) 2 0)X.] = Ex [X; xx [ww, (w,X) X) 2 0j]
By Assumption A3, (w,X, Si S N} satisfies Lindeberg's 's condition as N also known as uniform integrability requirement. Before formally stating it, let us introduce two variables: SN - Σラ (x, H))æ,), the partial sums, and their variance, o & E[ where Hyjæ; A E X; xı| the conditional mean of the j-entry given that i-th entry has value I, Then, Lindeberg's ss condition is formally stated as
This is
sup E S X N
00
This condition effectively states that no term in the partial sum w,X, will dominate. Under this condition, along with weak dependence (Property S1), we conclude from [Bra07, Theorems 1.19, 10.2] that SN/ON 4 A/(00, 1). Note that
S = (w, X) Ming)
and
NN
W
22
w,
where w, are N-dimensional vectors, 11x, = E and 2ile, Cov|X X, I,,
X, = x;} is the vector of conditional means,
211 IT
Since O and Mlz, constant, we can write
(w, X (w, Pojae)
Mylld ON
PxIx,=R,Y-11 [(w, 0] -X
ON 8> (W,Hjlm
P2~AT1.))
+oN(1)
ON
1 + eef (- (Ws
]
(1
2
-22N V2ON
1 + erf ( (w, (W,H)jaa)
1
oNN(1),
2 2
22
where the second step, in which we acquire (1),,, follows from the definition of convergence in distribution. Under Assumptions Al and A2, we may express this
(w, x,o!
PXIX, X)
0 0
erf
+ON(1).
/2 w E, ((w,a,)) ),1))
17

---

Therefore,
(x] X) 2 0]x,]
+ tow(1)) |
1
X;
W. (w,o,)
=Exiy==1
X
erf
2
V/
Vw En ((w, o/,2
1
X;
(w, w/..o///v
Ex
Xr
+ +xx, 1Ex11XX,looN21
2
V2
((W.o?))/ TT [w
1
X;
(w.
Ex,
Xperf
alg
+ EEExIIX,Ioor
2
V2
/ Vi
Defining (.(a) & Ex, erf(Xx alg
(aa// [2)/v2)] we can write
1 (w, Vww Σn w
ExIy=I|(((w,X) 2 0)x, 2
+ +Ex,1XX||ON(1)..
Note that [xx]|[x11] < [xx by Cauchy- Cauchy-Schwarz. So, Ex, Ex,11X, [|_x11X,oNNN = ox(1).
Moreover, by translation-ivvarinnee, all X, have the same marginal, so ip; 91 - 4p. Thus,
1
(w.o!)
Ex|y=I[((ww X): 2 0)X,] 20
+ON(1).
This form holds for all entries i. Concatenating them, we obtain
Σ +
Ex ((w, X) 2 0)x
V 2i
which gives the desired result.
C.3 Proof of lemma B.1
Property 1 follows from the fact that alg and erf are odd functions. This implies Property since an odd function must have zero for even Taylor coefficients.
Properties 2 and 3 follow from differentiating Eq. (6). The first derivative is given by
V2 Vi(1 22)
x².²
G <'(a)
Ex,
X²e
2(1033)
Setting 0, we get
5'(0) - v 12 E [x2]
2 2
The third derivative given by
2 (=(1-a²) a²)² (a² - 1)2 Ex, X2(1 -9x? -11aa* + (x² 8x7 +6)a x? + 3)e 20-03
x7a²
!!! (a)
Again setting a 0 gives
2 60 (²((0) V Ex, xi?-x? x² +3] = =v π
2
3 (xf] -x,xxi)---V
- - -30
18

---

C.4 Proof of Proposition 3.3
The pdf of X EE Φ) is
Px (x(x) deet(>) (2) g((x " "
(x nt)
for some function R20 + R20 [Fra04]. A key property of elliptical distributions is that if X & n (n, 2, 6), then its affine transformation also elliptical: (w, X) Ei((w, p2), w 2w, 6) for any w E R... Thus,
(s (w. 11)
P(w. (8) V w Σw 9
Ew
for some other function ÿ: R20 + R20
From our assumption of sign-symmetry, we have = 0. For brevity, we define o² 2 w (w, X). We begin by computing (I) in Eq. (9). Recall that we have O, (i.e., K = 2). So,
T Eww and
2 x (T) = Rg ReLU(S) = / 9
8 ds.
At this point, we apply a u-substitution with u = ds/o. This yields
3 =
and thus du = 28 da/o² = odu/2- =
C 2 (1) 2 ((u) du
2 w E,w.
c
Recall that we assume the MSE loss C is finite. The first term in Eq. (9) is clearly finite for = 0, 1. The term (TT) is also easily seen to be finite, since evaluates to w (Eg + Σ, w// Thus, being finite implies (1) in Eq. (9) is finite, which implies C co. We computed (II) from Eq. (9) above as
1 (Eo + En)
4
for K = 2. Plugging in (I) and (II) and differentiating with respect w, get
1 dw
Σ, (N + En) w.
(11)
F dt
Vw E
The steady states Eq. (11) thus satisfy
Σ W
1 2 (E E-E, w.
C
W W
Recall that translation-invariance implies that Eg and Σn are circulant, and since they are covariance matrices, they are symmetric. Then, they diagonalize in the basis given by the real and imaginary parts of the first n/2 Fourier components in the discrete Fourier transform, which we denote by the n real matrix J. Note that F is orthogonal. Define v= = F w and Ag = = E. for 0, 1. Thus, the steady states satisfy
A
(Ao AA)
VV A
This holds iff for all e [n].
1 C(A)a(V AAv) 22, (Aoo)i + (Ai)a),
2
19

---

Thus, if o # O, we must have that
(Ao)a 20(v A1v) -1. (A1)as
That is, the ratio of the i-th eigenvalues of E. and 2 must be constant for all s.t. v; o O. The cigenvalues of these matrices always come in pairs because of how defined F using both the real and imaginary parts of the discrete Fourier transform. In general, we observe that each pair assumes unique value. So, since C' is finite, the condition above hold for at most two distinct values of i. Therefore, V; = 0 for all but at most two é E [nl. implying that the steady state w is of the form c20k22 bin (iin22rkz), i.e., it is oscillatory. As such, it not localized.
D Additional experiments
D.1 Visualizing breakdown of Assumption A3
Fig. 8 demonstrates that our analytical model holds for long enough during training to capture the emergence of localization in the single ReLU neuron (M2). In the first three columns, we visualize the IPR of the weights from our empirical and analytical models, as well as the &g difference between these two weights. In the first four rows, we visualize these metrics for four random initializations of the model, training each on NLGP = 100) with Eo 0.3 and & 0.7, where we expect from Theorem 3.2 to see localization. We see the eπror rapidly increase shortly after IPR increases, indicating the formation of localized receptive fields. The last three columns confirm this, as they show snapshots from before, during, and after the divergence between the empirical and analytical weights. We observe the weights are nearly identical before, differ only slightly at the most localized point during, and are both localized after, but possibly with different magnitudes and positions. The difference that emerges during due to breakdown of Assumption A3 used to create our analytical model, which is violated when the norm of w is dominated by just a few entries, i.e., it is localized. While [IG22] also observe breakdown in their analytical model as localization emerges, ours, crucially, holds for long enough to characterize the emergence of localization.
We discuss the individual subplots in more detail. In all but the third row of Fig. the analytical predictions are near-exact; in the third row, we predict localization, but at the wrong position. Focusing again on the first row, we see that at 20, the weights have not yet become localized (from IPR, left, first, and visually) and analytical and empirical weights match nearlt exactly, as confirmed by the small distance in left, center above. At f = 30, a localized peak around = 21 begins to emerge, violating Assumption A3 and weakening analytical precision. The analytical model then underestimates the degree to which the main peak at i = dominates, while it overestimates the size of competing peaks at i 30, 37, and 90. Despite this, at = 50, we see that predictions from the analytical model retain a match to the empirical model.
In the last row of Fig. 8, we use the same initialization and setting as in the first row, except that we train on NLGP (g = 0.01) data instead. From Theorem 3.2, we do not expect to see localization. The evolution of IPR confirms this, as it stays low in magnitude. We also see that, because localization never emerges, Assumption A3 is never violated, and so our analytical model accounts for the empirical model nearly perfectly.
20

---

Zoured
049
1.54
1.8
604
600
ETSE
Zoamed
whights UL
We btte
211
Auayiical
w
0.3-
1.0
FR
Erer
Zcored
Weg 1010
We wwggita Bii
Weigrt tHiLE
Weights 100
00028
Emprtar
3.90
MM
m
Onep
60230
0.00
2.00
200
200
100
1ime
Figure 8: (Top) Four initializations trained on NLCP(g = 100) with Ço = 0.3 and & = 0.7. As expected, weights always localize. In (Leff, First) we plot IPR for empirical and analytical receptive fields (RFs) across time (defined (# of gradient steps) т. the learning rate). (Left, Second)
plot the time-evolution of & distance between the empirical and analytical RFs. (Left, Third) we zoom on (Left, First), restricting range to [0, 0.1] to more closely see divergence in IPR early training. In (Right, First) and (Right, Second), we snapshot the empirical and analytical RFs at time before and just after, respectively, the analytical model breaks down (according IPR and & distance) due to localization. Finally, in (Right, Third), we snapshot the first row top, trained on NLGP(g 0.01) data, again with & = 0.3 & = 0.7. As expected, weights do not localize. plot the same quantities as above. but here the predictions of our analytical model hold throughout the entire training process as localization never emerges and so assumption (A3) is not violated above.
end the training period. (Bottom) Same initialization
21

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Yes, in the abstract and introduction we claim derive an analytical model for the effective dynamics of localization (Section 3) in the setting described in Section 2 and validate this model with experiments (Section 4).
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: Yes, in the conclusion we note that our exact analysis is retricted to the single- neuron architecture M2 in the data setting described in Section 2, and that validations beyond that scope (like in Section 4) remain empirical even if they show promise.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification: Yes, our assumptions are rigourously stated in Section 3 and the proofs are provided Appendix
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Our setup (Section 4) and experiments (Section 4) provides full details on the data, hyperparameters, and training procedures used to obtain the figures. We also provide code (referenced on the first page).
Justification:
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, described in supplemental material?
Answer: [Yes]
Justification: Yes, we provide a link to the code repository in the first page the paper, and the repository contains all the code and data needed reproduce the experiments.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Yes, we provide primary details in Section and all the details in the code.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: Yes, for our prediction experiment in Section 4.2 we report a measure of variation over multiple runs.
8. Experiments Compute Resources
22

---

For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: Yes, we provide this information at the beginning of Section 4.
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: http:///eeu...p.eee wwweeerps..cppuuliieeeeee cc/public. Ceaiii////ieeeeeeeeee csGuidelines?
Answer: [Yes]
Justification: Yes, we have reviewed the NeurIPS Code of Ethics and believe that our work conforms to it.
Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: Since our work primarily theoretical analysis of existing phenomenon, we do not consider it to have direct societal impacts.
Safeguards
Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
Justification: Our work does not involve the release of data models that have significant risk for misuse.
12. Licenses for existing assets
Question: Are creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We wrote our own code for all experiments to procedurally generate data and train and analyze models. Visualizations from prior work in Fig. are properly credited.
13. New Assets
Question: Are assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: We document our code (the only asset we release) the accompanying repository.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: Our paper does not involve crowdsourcing research with human subjects. 15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human
Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: Our paper does not involve crowdsourcing research with human subjects.
23


Reviews:
[{"review_id": "yrCo6FsMSh", "reviewer": "Reviewer_iKD4", "summary": "The authors analytically derive the learning dynamics of (extremely) simple neural network models and characterize conditions under which units learn localized receptive fields. This builds on celebrated work in computational neuroscience on sparse coding and independent components analysis, offering a new perspective for biological findings.", "questions": "- In addition to showing examples in Figures 4, 5, and 6, can you somehow quantify localization?", "limitations": "Limiting assumptions are clearly stated and discussed.", "rating": 7, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "The paper is beautifully written (but see comments about figures below). It is well-motivated and accessible to a broad audience. The target audience for this kind of work may be somewhat niche, but it fits within the scope of the NeurIPS conference. The authors summarize prior work concisely and clearly. The assumptions of the analysis are stated precisely. The proofs in the appendix are well written, and the main claims of the theory are tested experimentally (though see below for some areas I am a bit unclear on).", "weaknesses": "To enable analytic tractability, the paper relies on very strong and simplifying assumptions. Figures 4, 5, 6, and the right hand side of Figure 3 are hard to see. It could be helpful to add color and show fewer lines. Also, the description of these figures is the one part of the text I had trouble following. Quantifying the outcomes somehow would be helpful for me to understand exactly what I'm supposed to be looking at in these examples. For instance, in Figure 4A, the claim is that the model learns an oscillatory filter that resembles a sinusoid. But the left panel in Fig 4A, though oscillatory, doesn't look sinusoidal?\n\nUltimately I believe these weaknesses can be addressed during the rebuttal phase and the strengths outweigh the weaknesses.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their well-formulated concerns and\nsuggestions, especially with regard to improving the interpretability\nand readability of our work.\n\n> **Figures 4, 5, 6, and the right hand side of Figure 3 are hard to\nsee. It could be helpful to add color and show fewer lines.**\n\nWe thank the reviewer for the suggestion to improve legibility of the\nreceptive field evolution. In the Global Response, we have plotted\nreceptive fields with a non-grayscale (blue-red) colormap to improve\nlegibility, which another reviewer suggested as well.\n\n> **the description of these figures is the one part of the text I had\ntrouble following.**\n\n> **Quantifying the outcomes somehow would be helpful for me to\nunderstand exactly what I'm supposed to be looking at in these\nexamples.** \n> **in addition to ... Figures 4, 5, and 6, can you\nsomehow quantify localization?**\n\nWe thank the reviewer for the suggestion to quantify the localization\nphenomenon. In Fig. 1--3 of the Global Response, we quantify\nlocalization with the inverse participation ratio (IPR), defined as\n$\\operatorname{IPR}(\\mathbf{w})=\\left(\\sum_{i=1}^D w_i^4\\right)/\\left(\\sum_{i=1}^D w_i^2\\right)^2$,\nwhere $w_i$ is the magnitude of dimension $i$ of weight $\\mathbf{w}$.\nThis measure, also used by [IG22], is large when\nproportionally few weight dimensions \\\"participate\\\" (have large\nmagnitude), and small when weight dimension magnitudes are more uniform.\n\n> **For instance, in Figure 4A, the claim is that the model learns an\noscillatory filter that resembles a sinusoid. But the left panel in Fig\n4A, though oscillatory, doesn't look sinusoidal?**\n\nIn Fig. 4 of the Global Response, we quantify this claim by fitting a\nsinusoid to the resulting receptive field, and find that we can do so\nvery well. We also rescale the receptive field to make the sinusoidal\nstructure of the steady state more apparent. Please see the figure\ncaption for more details.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Efwwn7DU80", "reviewer": "Reviewer_TcXg", "summary": "SETTING: Olshausen & Field famously showed that requiring natural image patches to be constructed from a small number of independent elements from an overcomplete dictionary populates that dictionary with spatially localized feature detectors.  But localization also appears in DNNs trained on discriminative-learning tasks.  It is also ubiquitous in the cortex.  This raises the question of what fundamentally drives the formation of localized receptive fields.\n\nAPPROACH: The MS attempts to explain the emergence of localized RFs by reference to the statistics of the stimuli (inputs).  In particular, the authors examine the time evolution of network weights trained with gradient descent to discriminate \"natural-image-like\" stimuli--analytically for a single (ReLU) unit of neural network, and in simulation for a weighted sum of multiple such units (a two-layer NN with one output unit).\n\nMore precisely, the authors consider stimuli/inputs with circulant covariance matrices, a 1D idealization of natural images (the covariance matrices of natural images are block-circulant with circulant blocks).  Under some additional simplifying assumptions (see below), they derive an ODE for the time evolution of the weights when the network is trained to discriminate stimuli according to whether their spatial correlations are \"long\" or \"short.\"  They show that this ODE depends only on the marginal distribution of a single \"pixel\" rather than the joint (and is valid until localization begins).  Investigation of this ODE (Appendix C.1) reveals that localization will occur (w large for some entries, small for the rest) when the inputs have (sufficiently) negative excess kurtosis.  The authors also demonstrate this in simulation with data distributions with varying levels of excess kurtosis, both by examining the learned receptive fields and by numerically integrating the ODE.  Finally, they extend their simulations to two-layer NNs.", "questions": "--One of the chief motivations for the study is that even DNNs trained on supervised-learning tasks acquire localized RFs.  In support of this claim they cite the AlexNet paper and some papers on visualizations of convnets.  Obviously the convolutions in these networks enforce spatial localization.  Do the authors have in mind here localized (bandpass) *frequency* RFs?\n\n--Is *negative* excess kurtosis consistent with the statistics of natural scenes?\n\n--Can the authors provide more intuition about assumptions A1 and A2, and how restrictive they are?\n\n--In the proof, l. 634, shouldn't it be P(<w,X> > 0) = 1/2?  (There is no > 0 in the expression in the MS.  Am I misunderstanding this?)", "limitations": "N/A", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "The MS persuasively presents a new route to localized receptive fields.  The proofs look correct (but I did not verify every line) and the simulations confirm that they are not undermined by the various approximations employer along the way.  The problem is an important one and of long-standing interest to the field (computational neuroscience).", "weaknesses": "This reviewer struggles to see how these results can be generalized beyond this very simplified setting of, essentially, a \"network\" Y = ReLU(<w, X>) (or its cousin, the SCM).  Introducing just one more learnable layer breaks (or at least vitiates) the connection between stimulus kurtosis and localized receptive fields (as the authors show in Fig. 6).\n\nThis makes it hard to find this explanation of RF localization more compelling than the classical one in terms of efficient codes.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their excellent feedback. We especially\nappreciated your precise and astute questions with regard to the\nvalidity and generalizability of our model, and also for reading closely\nenough to identify a typo in our proof.\n\n> **how these results can be generalized beyond this very simplified\nsetting ... one more learnable layer breaks (or at least vitiates) the\nconnection between stimulus kurtosis and localized receptive fields**\n\nThe reviewer correctly points out a key limitation of this work:\nextensions to more complicated models quickly lead to difficulties in\nextending our analytical approach and its interpretation. Indeed, we\nobserve that experiments with two-layer networks are noisier (as the\nreviewer pointed out in referencing Figure 6), as did [IG22].\n\nHowever, we consider it worth mentioning that some aspects of our\nanalytical approach seem to extend to the two-learnable-layer regime,\nthough we reserve this as an avenue for future work. As an example,\nusing the approach we have developed, consider the steady-state equation\nfor a single-neuron model with two learnable layers, $\\mathbf{w}_1$ and\n$w_2$, after plugging in for $w_2$:\n$$\\varphi\\left( \\frac{\\Sigma_1 \\mathbf{w}_1}{\\sqrt{\\langle \\Sigma_1 \\mathbf{w}_1, \\mathbf{w}_1 \\rangle}}  \\right) = \\frac{1}{\\sqrt{2 \\pi}} \\frac{(\\Sigma_0 + \\Sigma_1) \\mathbf{w}_1}{\\sqrt{\\langle (\\Sigma_0 + \\Sigma_1) \\mathbf{w}_1, \\mathbf{w}_1 \\rangle}}$$\n\nThis steady state is very similar to the one implied by Eq. 5 in Lemma\n3.1, the only difference being an additional scaling term in the\ndenominator of the right side. We have found that manipulating this term\ndoes not substantially change whether a given $\\varphi$ yields localized\nreceptive fields, but this observation is still preliminary. However,\ndynamics play a very important role in the precise structure obtained at\nconvergence, something we are exploring for potential future work. As\nthe reviewer points out, adding another learnable layer obfuscates the\nrelation between stimulus kurtosis and localization because the\nsecond-layer weight can be pulled into ReLU term and viewed as a\nrescaling of the stimulus. A thorough analysis will likely not be as\nsimple as just studying $\\varphi$ and will likely require a careful\nanalysis of initializations, but we expect that general strategies and\nideas from this work will carry over into future ones.\n\n> **hard to find this explanation of RF localization more compelling\nthan the classical one in terms of efficient codes**\n\nWe would like to emphasize that we don't wish to claim that our approach\nis much more compelling than the sparse coding work of\n[OF96]. Instead, we hope to present it as an\nalternative bottom-up, learning model that should be investigated\nfurther because of its ability to reproduce key qualitative phenomena of\nvisual receptive fields with an alternative set of assumptions that some\nmay view as more minimal (due to lack of sparsity regularization).\n\n> **the convolutions in these networks \\[such as AlexNet\\] enforce\nspatial localization**\n\nThe kernel size of the first-layer convolutional\nkernels is indeed usually taken to be much smaller than than the size of\nthe input image, thus enforcing a degree of spatial localization. For\nAlexNet, for example, receives input image of size $224 \\times 224$ with\nan $11 \\times 11$ kernel in the first layer. Here, we mean to refer to\nthe further localization that emerges within these convolutional kernels\nwithin that kernel bandwidth; i.e., the oriented receptive fields\nvisible in Figure 1 have width in the edge direction much smaller than\n11.\n\n> **Is negative excess kurtosis consistent with the statistics of\nnatural scenes?**\n\nWhile we focus on analyzing the model of emergent localized receptive\nfields of [IG22], this is a great question about broader\nrelevance. We can provide some context in the case of simple cells in\nprimary visual cortex (V1), of which sparse coding is classically taken\nas a model [OF97] Retinal ganglion cells prior to V1\nare understood to perform edge detection. Edge detection naturally\ninduces concentrated marginals, corresponding to positive and negative\nedges, with a large amount mass at zero, corresponding to no edge. Such\ndistributions typically have negative excess kurtosis (unless a very\nlarge amount of mass is at zero, corresponding to a nearly uniform\ninput).\n\n> **Can the authors provide more intuition about assumptions A1 and A2,\nand how restrictive they are?**\n\nWe thank the reviewer for this question. We attempted to address in\nlines 160-3 in the submission, which we expand on here.\n\nA1 and A2 are motivated by the limiting cases of the $\\texttt{NLGP}(g)$\ndata model from [IG22]: $g \\to 0$ (no localization) and\n$g \\to \\infty$ (localization). A1 is implied by the weaker assumption\n$\\mathbb{E}[X_j \\mid X_i = x_i, Y = y] \\propto x_i$ after applying S3.\nA2 captures that when conditioning on $X_i$, nearby entries in\n$\\mathbf{X}$ are almost deterministic, while distant ones are\nunaffected. The $i$-th row and column of\n$\\operatorname{Cov}[X_j \\mid X_i = x_i, Y = y]$ are zero. Local\ndependence implies entries $(k,j)$ of the conditional covariance for\n$k,j$ near $i$ should be small, while distant ones remain unchanged.\nWeak dependence (S1) implies $\\sigma_i^y$ is largest near $i$ and zero\nelsewhere. Subtracting $\\sigma_i^y {\\sigma_i^y}^\\top$ from $\\Sigma_y$\nexpresses these intuitions, supported by $\\texttt{NLGP}$ limiting cases.\nAs such we consider these assumptions to be relatively loose. Fig. 1 of\nthe Global Response, which shows a sharp uptick in IPR as excess\nkurtosis becomes negative, supports this.\n\n> **... l. 634 shouldn't it be P(\\<w,X\\> \\> 0) = 1/2?**\n\nYes; thank you for the correction.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "tfrzY753Gh", "reviewer": "Reviewer_PEfQ", "summary": "This paper investigates when localized receptive fields arise in supervised neural networks. Extending a recent work of Ingrosso and Goldt, the authors propose that simple single-neuron models learn localized receptive fields when trained on data with sufficiently negative excess kurtosis, while if the excess kurtosis is sufficiently positive they learn delocalized receptive fields.", "questions": "- A neuroscientific quibble: in the first sentence of the Introduction (Line 19), one need not restrict attention to the \"mammalian nervous system.\" There are numerous examples of localized receptive fields in non-mammalian species; see for instance the beautiful works of Knudsen & Konishi on audition in owls. \n\n- The authors might consider citing Sengupta et al., \"Manifold-tiling Localized Receptive Fields are Optimal in Similarity-preserving Neural Networks\" (NeurIPS 2018) in their discussion of unsupervised learning algorithms that give localized receptive fields. \n\n- Another potentially relevant reference is Shinn, \"Phantom oscillations in principal component analysis\" (PNAS 2023). \n\n- The reference in Footnote 3 is wrong; it should be to Appendix C.2 not C.3. \n\n- In Line 187, there is a small typo: \"termdepends\" -> \"term depends\"\n\n- Using a perceptually uniform non-grayscale colormap to represent time might improve the legibility of the plots relative to the grayscale used in the submitted manuscript. \n\n- The final paragraph of the conclusion is not tied to anything that came before; if you care about orientation selectivity you should mention & measure it in earleir portions of the paper. Otherwise, this should be omitted.", "limitations": "The authors do a largely adequate job of discussing the limitations of their work, up to the technical weaknesses noted above.", "rating": 7, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "The topic of this paper is of broad interest in both machine learning and neuroscience, and on the whole I think this manuscript makes a worthy contribution on top of the work of Ingrosso and Goldt. There are some weaknesses which dampen my enthusiasm (see below), but on the whole I favor acceptance.", "weaknesses": "I have two primary concerns about the results presented:\n\n- First, Lemma 3.1's treatment of time is not sufficiently precise. Can you provide a more precise answer than simply \"early in training\" or \"before A.3 is violated\"? Indeed, the logic in the paragraph beginning on Line 174 is not clear. What you show is that, in some cases (see concern below), eq. (5) generates localized RFs in a similar location to those observed in actual training. This does necessarily mean that \"the gradient flow in Eq. (5) holds sufficiently long to detect the emergence of localization in the weights,\" as you write in Lines 177-178. Moreover, you have not in fact defined what you mean by \"detect the emergence of localization;\" this must be reified. Can you see a change in participation ratio, even if only numerically, before the approximation breaks down? \n\n- Second, the experiments are rather limited, and rely largely on exemplars rather than systematic statistical investigation. This is important given the gap in Lemma 3.1: the authors rely on experiments to justify their claim that this approximation provides meaningful information about when localization will occur, but all they actually show is that there is resemblance in a few cases. The paper would be much stronger if the authors could also show that their claims hold statistically over many realizations of the data generation process and training procedure.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their extremely helpful feedback, in\nparticular for making us aware of relevant work and clarifying minor\nmisconceptions. We also appreciate your insistence to more\ncomprehensively quantify and replicate our analyses.\n\n> **Lemma 3.1's treatment of time is not sufficiently precise.**\n\nWe thank the reviewer for this criticism. Indeed, lemma 3.1 does not\nprovide a precise criterion for *when* the approximation breaks down.\nThis is because the only assumption that is violated as a result of\nlocalization, A3, does not break down at any predictable $t$. To make\nthis claim clearer, we have included Fig. 2 & 3 in\nthe Global Response. We see across examples that the analytical model\nclosely tracks the empirical one until a strongly localized bump emerges\nin the weights; when no localization emerges, the analytical model\nmaintains its precision through training. Please see the figure and its\ncaption for more detailed information.\nWe would happily incorporate suggestions from the reviewer on how best to represent or quantify this notion of time.\n\n> **the experiments are rather limited, and rely largely on exemplars\nrather than systematic statistical investigation**\n\nWe thank the reviewer especially for stating this limitation. To address\nthis, as well as other reviewers' concerns about the precision of our\nmodel and analysis, we have included Fig. 1 in the Global Response. This figure shows\nthe IPR against the excess kurtosis for various values of $g$ and $k$,\nrespectively, for $\\texttt{NLGP}$ and $\\texttt{Kur}$.\n\n> **one need not restrict attention to the \"mammalian nervous\nsystem\"**\n\nThank you for the reference; Knudsen & Konishi (1978) is indeed\nrelevant. We will broaden our terminology from \\\"the mammalian nervous\nsystem\\\" to \\\"animal nervous systems\\\".\n\nKnudsen & Konishi (1978). Center-surround organization of auditory\nreceptive fields in the owl. DOI:10.1126/science.\n\n> **consider citing Sengupta et al.**\n\nThank you for the reference; we will add Sengupta et al. (2018) to the\nlist of unsupervised learning algorithms producing localized receptive\nfields. As far as we understand, this algorithm does not optimize an\nexplicit or implicit sparsity criterion as in sparse coding, so it is\nalso an example of an alternative model for the emergence of\nlocalization. \\[Har+20\\] cited in the submission is another such\nexample; we will incorporate both of these references in the submission\nvia discussion of their modelling assumptions wrt those we employ.\n\n> **The final paragraph of the conclusion \\[on orientation\nselectivity\\] is not tied to anything that came before \\...**\n\nWe are asked by NeurIPS to address the limitations of our work, and\nsince we display oriented receptive fields in Figure 1 (left, center) we\nwould like to call attention to the fact that we cannot yet capture this\nfeature in our analytical model, and neither do the simulated receptive\nfields appear oriented. To improve on this point, we will introduce the\nterminology of oriented receptive fields when referring to Figure 1\n(left, center) in the introduction so that this terminology does not\nspring out of nowhere in the conclusion, as the reviewer has pointed\nout. Our analytical and simulated receptive fields are not oriented to\nany degree, so we do not think it is necessary to explicitly measure\nthis property.\n\n> **typo, corrections**\n\n> **us\\[e\\] a perceptually uniform non-grayscale colormap**\n\nWe particularly appreciate the reviewer's thorough examination of our\nwork and their identification of typographical errors. A compiled list\nof the typographical corrections we plan to implement is provided in the\nglobal response. We also intend to reformat our plots to use a\nnon-grayscale colormap to improve legibility, which another reviewer\nencouraged as well. Please see Fig. 4 in the attached PDF for an example.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We thank the reviewers for their excellent feedback. In particular, we
thank each of the reviewers for providing specific, actionable questions
and concerns. We have done our best to address each of these in our
responses.

### Minor corrections to errata

We also thank the reviewers for reading our manuscript very closely and
identifying typos. Those corrections, along with additional we
identified upon our own re-reading, are listed below:

1.  Line 163: \"does and not does\" $\to$ \"does and does not\"

2.  Line 171: \"$o(1)$\" $\to$ \"$o_N(1)$\", to clarify dependence
    w.r.t. dimension $N$ and not time $t$

3.  Line 187: \"termdepends\" $\to$ \"term depends\"

4.  Line 238: \"steady\" $\to$ \"steady state\"

5.  Line 634:
    $\mathbb{P}(\langle \mathbf{w}, \mathbf{X} \rangle) = \frac{1}{2}$
    $\to$
    $\mathbb{P}(\langle \mathbf{w}, \mathbf{X} \rangle > 0) = \frac{1}{2}$.

6.  Line 649: \"blow up\" $\to$ \"dominate\"

### On extension to 2D

We would also to clarify a potential misconception about our work. Our
analysis is presented in terms of $N$-dimensional inputs (i.e.,
$\mathbf{X} \in \mathbb{R}^N$), which are most naturally interpreted as
1D images. However, our analysis extends naturally to 2D images as well
through (un)vectorization, just as is done experimentally in
[IG22]. To do this, we construct $N^2$-dimensional signals
with a special covariance given by
$\Sigma_y = \tilde{\Sigma}^{a}_y \otimes \tilde{\Sigma}^{b}_y \in \mathbb{R}^{N^2 \times N^2}$,
where
$\tilde{\Sigma}^{a}_y, \tilde{\Sigma}^{b}_y \in \mathbb{R}^{N \times N}$
are covariances along the two axes of the 2D image. $N^2$-dimensional
data sampled from such a distribution can be turned into a 2D image by
inverting the vectorization operation:
$\tilde{\mathbf{X}} \equiv \operatorname{vec}^{-1}(\mathbf{X}) \in \mathbb{R}^{N \times N}$,
which puts the first $N$ entries of $\mathbf{X}$ in the first column of
$\tilde{\mathbf{X}}$, the next $N$ entries of $\mathbf{X}$ in the second
column of $\tilde{\mathbf{X}}$, and so on. This is the procedure we used
to generate 2D receptive fields in Figure 1 (Right) in the manuscript for the SCM.

We focused on the 1D case because it is, following the argument above,
the most general case, and also the most natural to consider with a
feedforward network that does not make any assumptions on intra-layer
connectivity.

Meta-review: This paper describes novel analysis of the emergence of localized receptive fields using top-down efficiency constraints. All three reviewers felt that it was above the threshold for acceptance, and I'm pleased to report that it has been accepted to NeurIPS.  Congratulations!  Please revise the manuscript according to the reviewer comments and discussion points.
Decision: Accept (spotlight)