Not all solutions are created equal: An analytical dissociation of functional and representational similarity in deep linear neural networks
Authors: Lukas Braun
Erin Grant
Andrew M Saxe
Venue: ICML 2025 spotlightposter (2025)
Keywords: deep learning
representational similarity
representation learning
deep linear networks
neuroscience
Primary area: applications->neuroscience_cognitive_science
TL;DR: Neural networks performing identical tasks can organize information completely differently, except for particularly robust ones that employ canonical internal patterns.

Abstract: A foundational principle of connectionism is that perception, action, and cognition emerge from parallel computations among simple, interconnected units that generate and rely on neural representations. Accordingly, researchers employ multivariate pattern analysis to decode and compare the neural codes of artificial and biological networks, aiming to uncover their functions. However, there is limited analytical understanding of how a network’s representation and function relate, despite this being essential to any quantitative notion of underlying function or functional similarity. We address this question using analysable two-layer linear networks and numerical simulations in nonlinear networks. We find that function and representation are dissociated, allowing representational similarity without functional similarity and vice versa. Further, we show that neither robustness to input noise nor the level of generalisation error constrain representations to the task. In contrast, networks robust to parameter noise have limited representational flexibility and must employ task-specific representations. Our findings suggest that representational alignment reflects computational advantages beyond functional alignment alone, with significant implications for interpreting and comparing the representations of connectionist systems

Not all solutions are created equal: An analytical dissociation of functional and representational similarity in deep linear neural networks
Lukas Braun Erin Grant
Andrew M. Saxe
Abstract
W,W,
C
Wix
D
9. WiW
Weight velue
Actrvdtion
A foundational principle of connectionism is that perception, action, and cognition emerge from par- allel computations among simple, interconnected units that generate and rely on neural representa- tions. Accordingly, researchers employ multivari- ate pattern analysis to decode and compare the neural codes of artificial and biological networks, aiming to uncover their functions. However, there is limited analytical understanding of how a net- work's 's representation and function relate, despite this being essential to any quantitative notion of underlying function or functional similarity. We address this question using analysable two-layer linear networks and numerical simulations in non- linear networks. We find that function and rep- resentation are dissociated, allowing representa- tional similarity without functional similarity and vice versa. Further, we show that neither robust- ness to input noise nor the level of generalization error constrain representations the task. In con- trast, networks robust to parameter noise have lim- ited representational flexibility and must employ task-specific representations. Our findings sug- gest that representational alignment reflects com- putational advantages beyond functional align- ment alone, with significant implications for in- terpreting and comparing the representations of connectionist systems.
0
Step
Step
500
Step
Step 500
Step
500
Figure Random walk. (A) A random walk on the solution manifold of a two-layer linear network reveals that input and read- out weights can change continuously. inducing changes in the (B) network parametrisation and thus the (C) hidden-layer repre- sentations, while preserving the (D) network output.
pute with distributed representations (Rumelhart et al. 1986). Accordingly, one might aim to identify function from net- works observables such as connectivity weights and neu- ral activity patterns; however, this is often complicated by the inherent complexity and partial observability of these systems. In particular, the structure of artificial and bio- logical networks is often non-identifiable in the sense that networks can be structurally distinct, yet implement the same input-output mapping. For example, biophysical neuron models can exhibit nearly identical functions at both the neuron (Goldman et al. 2001) and network levels (Prinz et 2004) despite considerable variation in their architec- ture (reviewed in Marder and Goaillard 2006; Albantakis et al. 2024). Similarly, artificial neural networks (ANNs) are almost always non-identifiable due to simple symmetries, such as -nvvaiaionnnvaaiianee of neurons (Sussmann 1992; Albertini and Sontag 1993), scale-invariance of activation functions (Neyshabur et al. 2015a), alongside more complex symmetries arising from feature composition across layers and from finite training data (Refinetti et al. 2021; Arous et al. 2022). As modern networks are deep and heavily over- parametrised (Zhang et al. 2021), they are inherently non- identifiable, with many parametrisations yielding the same input-output behaviour. Determining when parametrisations become identifiable and understanding the consequences of non-identifiability remain open problems (Roeder et al. 2021; Entezari et al. 2022: Vlačić and Böleskei 2022; Ghosh et al. 2022; Wang and Jordan 2021; Godfrey et al. 2022; Martinelli et al. 2023; Bona-Pellissic et al. 2023; Lampinen et al. 2024; Kori et al. 2024; Marconato et al. 2024).
1. Introduction
The parallel distributed processing hypothesis posits that function in artificial and biological networks emerges from interactions among simple interconnected units that com-
'Department of Experimental Psychology, University of Ox- ford, Oxford, UK "aatsby Unit & Sainsbury Wellcome Centre, University College London, London, UK. Correspondence to: Lukas Braun <lukas.bbann @ @ww..ox.as.xx..
Even deep linear networks exhibit both trivial and non- non-trivial symmetries, making their parametrisation non oon-identifiable. While any deep linear network can be re-parametrised as
Proceedings of the 4nnd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 20255 by the author(s).

---

An analytical dissociation of functional and representational similarity in deep linear neural networks
single linear transformation (Laurent and Brecht 2018), it does so through multistage computations that give rise to hidden-layer representations. Moreover, the optimisa- tion landscape of a deep linear network is non-convex and contains a high-dimensional solution manifold (Figure 1) whose shape is determined by the statistics of training data and the network architecture (Baldi and Hornik 1989 Saxe et al. 2014; Arora et al. 2019), making it a useful surro- gate for studying representation learning (Saxe et al. 2019; Braun et al. 2022; Dominé et al. 2025). Here, we leverage the analytical tractability of deep linear networks to study functionally equivalent parametrisations at global minimum error. Crucially, these solutions employ different internal representations, which has significant computational con- sequences, most notably in their affordances for linear de- coding, representational similarity analysis (Section 4), and their sensitivity to noise (Section 5).
Wi E R.N..N, project inputs hidden-layer neural repre- sentation h, Wx RN,,, which are projected to out- puts via the readout weights W2 E RNN,.., We denote by X = xx...... xpl, Y = [V...... YP., and H = [hi..... hp] the matrices that contain all inputs, targets and hidden-layer representations, respectively. The network's representa- tional similarity matrix (RSM) is then defined by
RSM X x = H.
(3)
capturing pairwise similarities between inputs in the hidden representational space. Laurent and Brecht (2018) showed that, under the following assumptions:
Assumption 2.1. The loss function is convex and differen- tiable, e.g., the mean-squared error loss.
Assumption 2.2. The network not bottle-necked, i.e., min (N₂, No) S No.
all local minima of a deep linear network are global and equivalent to the solution of the corresponding single-layer linear regression problem. Notably, this result holds without assumptions on the structure of the training data. In our set- ting, this permits the following definition (see Appendix C): Definition 2.3. Under Assumptions 2.1 and 2.2, any pair of network weights satisfying
We now detail our contributions:
We derive exact parametric equations characterising the complete and distinct subregions of the solution manifold in two-layer linear networks.
We demonstrate that, although all subregions allow flexi- ble neural representations, some inherently lead to iden- tifiable task-specific representational similarities, while others result in non-identifiable, task-agnostic representa- tional similarities.
W₂ W,W,E = Σye
(4)
is globally optimal general linear solution (GLS), where
We establish that in contrast to task-specific solutions, aank-agnostic solutions are non-identifiable and non- comparable.
P
P (5)
ΣnS Σ
X and Eye E
yu
T
1==1
We analytically show that input noise and generalisation error do not constrain representations to task-specific re- gions, whereas parameter noise does.
denote the input and input-output covariance matrices.
Note that the absence of suboptimal minima does not pre- clude the existence of other critical points, nor does it guar- antee convergence of gradient-based algorithms. To distin- guish general weight matrices W1 and W₂ from those that satisfy Equation (4), we denote the latter as optimal weights Z and f2 Further, we denote the compact singular value decomposition (cSVD), as defined in Appendix 1. of the inputs and least-squares solution as
We validate our analytical findings through numerical simulations, demonstrating that these computational prin- ciples persist in non- non-linear neural networks.
All simulations are detailed in Appendix A, and a code repository reproducing all figures is available on GitHub at mukas-braun/ddisociiaiiia
miiitryy..
cSV((X) - ABC",
(6)
2. Setting and preliminaries
and In the following, we analytically study the full set of global solutions, the so called solution manifold, and partition it into subregions with distinct representational and compu- tational properties. Crucially, our analysis holds without assumptions on the structure of the training data and irre- spective of how solution is obtained, and thus does not depend on any particular learning or optimisation algorithm. Figure 2A visualises the entire solution manifold and its subregions for a simple example, providing some intuition for the formal definitions and theorems developed next.
cSVD(E yz (Exx USVT.
(7)
We consider a two-layer linear network (Figure IA),
8,, W₂ W Wxx:
trained minimise the mean-squared error
MASE
1 2P
Σ
ynl
over a dataset D = ((xn, with inputs x, e R N. and corresponding targets yn E N.N. The input weights
2

---

An analytical dissociation of functional and representational similarity in deep linear neural networks
3. Partitioning the solution manifold of two-layer linear networks
(a is arbitrary matrix.
and F3 E R.N.N.
Two-layer linear networks architectures are typically highly overparametrised, admitting many combinations of input and output weights that achieve the global optimum for a given task. Formally, the set of all such network weights defines the solution manifold,
The first terms of S2 and 92 implement the core input- output mapping; I₁ and T2 project from task-irrelevant and unobserved input directions; l3 projects from the unoceu- pied hidden space: cancels interference from irrelevant inputs that are projected into the core; and the rank con- straint ensures that such correction exists. See Appendix C for a detailed proof and Figure 2B for a visualisation.
M { 92 = Ey
We note that useful intuition about the manifold's ' structure can be gained by viewing it as the set of weight configura- tions related by invertible linear transformations. For any invertible matrix Q R NNN,, the weight pair
Next, we partition the solution manifold into distinct regions and subsequently analyse their respective representational and computational properties. For proofs of Theorems 3.3, 3.5 and 3.7 refer Appendix D, and to Figure 2C-E for visualisations.
22 + 22Q-1 and Ω - QΩ
(9)
Definition 3.2. Any GLS that minimises the norm of the network function,
implements the same input-output map and thus lies on the same manifold (Baldi and Hornik 1989; Saxe et al. 2014). In the context of neural network architecture, these Q- transformations redistribute how information is processed across layers, for example, by rotating and scaling interme- diate representations. However, since they preserve rank, they only fully characterise the solution manifold when the input and output dimensions are identical and the task has full rank, conditions that may not be met in real-world sce- narios. To refine this view, partition the input space into three subspaces: Observed and relevant, observed but irrelevant, and unobserved null directions, with correspond- ing projections P, = VV"", Pi AAT VV", and P. II AAT. The distinction between relevant and irrel- evant directions arises because the input space can exceed the intrinsic dimensionality of the solution manifold (but not vice versa). Specifically,
argmin W12 s.t. W W,E,, Eye W..W2
(12)
is least-squares solution (LSS)..
Theorem 3.3. All LSS satisfy 2zg22 are exactly parametrised by
Eye yo
E
and
Ω QVSVT + r.P, + T2P. and = UVSQ + v + $
(13)
+ + 13(1 -
subject the definitions and constraints in Theorem 3.1, and the additional constraint that rank(HH 52 P.) S
rank((I - HHHHHHHH P.), and where +- = -UV/QQ* + L) P. [II HH
Here, 4 cancels interference from unobserved inputs pro- jected into the occupied hidden space, with the rank con- straint ensuring a correction exists.
r= (rank(Egr) S min(rank(X), rank
(10)
Definition 3.4. Any GL.S for which the norm of the hidden- layer representations and readout weights is minimised
so when the input rank exceeds the target rank, some in- put directions are irrelevant to solving the task. Likewise, the hidden space can be partitioned into subspaces corre- sponding to the hidden-layer representations of relevant, and irrelevant inputs, and all remaining unoccupied null directions. Importantly, the hidden representations of rele- vant and irrelevant inputs may overlap, which necessitates compensation and introduces structural constraints the form of valid solutions. The following parametrized equa- tion fully encapsulates this intricate structure of the solution manifold:
argmin W.W. W.
W.XX + l| 1
(14)
s.t. W2 W Egg Ege
is minimum representation--omn solution (MRNS). Theorem 3.5. All MRNS are parametrised by S = MVNR" and Ω RRNNOO x (15)
where R NNx an arbitrary Jothoooorhhn matrix, and
Theorem 3.1. Any GLS satisfies
CSDDYCCC)) = MNOT
(16)
+ and Ω Q +r,P,+Dp
(11)
Definition 3.6. Any GLS for which the sum of the norm of the weight matrices is minimised
Ω2 UVSQ + + HH*),
where Q E R NE is an arbitrary full-column- matrix, T. T2 E R N..N. are arbitrary matrices subject to the con- straint rank (QQQQQQ-T)) QQ+T,P:) & rank((I QQ- P.). PP,)),P)) w
argmin || w.ll + 2 s.t. W₂ W,W.E. E. Eyn (17)
W,WW
is minimum weight- solution (MWNS).
3

---

An analytical dissociation of functional and representational similarity in deep linear neural networks
B
General Linear
Squares
D Minimum Remrruuttnppeeentatttonnnn Norm E
Minimum-Waghh Nom
W 1
X
Cosupii
Lnn IF Pabear
Unsooupes Cosupto
Rotevon
WIE
2
Uv'Q
H Unpcon Pouuees
qu/yv
Uv$Q
Qv/5v
My
MVNR"
RvND'X
UVSR""
RvSS
T.P.
T.P.
Uioccappr
Wi2
0
G
F.P.
TyP.
Wi2
Wat
ng
Figure 2: Solution manifold. (A) Schematic of solution manifold (left) for two-layer linear network trained on a single training pair (right). The GLS (blue plane) reflects that weight W1 lies in the input null space and unconstrained. while W11 and W21 are coupled, increase in one requires a decrease in the other. Constrained L.S (pink), MRNS (green), MWNS (orange) are highlighted subregions of the manifold. (B) Schematic of the parametrisation of the GLS, showing how components of S2 map relevant, irrelevant, and unobserved input directions to the hidden space, and how components of f22 map from unoccupied and occupied hidden directions to the ouiput. Projections from irrelevant inputs can interfere with the core (blue). creating overlap (black) between the relevant (dark grey) and irrelevant (light grey) hidden space, which cancelled by V. (C) As in (B), but for LSS. Projections from unobserved input directions into the occupied hidden space are cancelled by Q. (D) and (E) are as in (B). but show MRNS and MWNS, respectively. The additional constraints remove projections and further restrict the core.
Theorem 3.7. All MWNS are parametrised by
GLS and LSS differ only in how they handle projections from unobserved input and unoccupied hidden directions and thus implement the same input-output map on the train- ing data. As result, they exhibit identical degrees of free- dom in their hidden- representations
f2 = S and Ω2 = =RVSVT.
(18)
We note, that the relation between MWNS and the singular value decomposition of the least-squares solution has been previously derived under strong assumptions, namely that Σxs- I, N₁ No and that En has full rank (Saxe al. 2019, see appendix $14-S15). Further, we note that Corollary 3.8. MRNS and MWNS are identical if inputs are whitened, i.e., Egz= I.
H QVSV' + +IPX
(19)
Since Q can be any full-column-rant -rank matrix and T1 is free up rank constraint, GLS and LSS support nearly arbitrary hidden-layer representations. To illustrate this, we consider a semantic learning task linking items to positions within a hierarchical structure (Figure 3A,B). For example, we can select a point on the solution manifold where the hidden-layer representations of the items form the shape of elephant (Figure 3C).
key difference between the four solution types lies in how they constrain the image and kernel of the weight matrices, which map between input, hidden, and output spaces. GLS impose minimal constraints. LSS restrict projections to and from null spaces in the input and hidden layers. MRNS fur- ther reduce freedom in the null space, eliminate irrelevant projections, and constrain the core solution itself. MWNS, the most restrictive class, eliminate all irrelevant and null- space projections, and enforce balance in the core by requir- ing equal contributions from the input and output weights. While this perspective clarifies how different subregions of the solution manifold constrain the structure of the weight matrices, it does not address a key question: how these solutions differ in their hidden-layer representations.
Corollary 3.9. GLS and LSS permit any RSM of the form RSM XT (VVSQTQv35 + VVVSTT,,B (20) + +P/TTQvSV" + PP))
In our example, this yields highly structured RSM, yet does not reflect the task structure, iee, the hierarchical re- lationships between items (Figure 3C). Accordingly, we make
Definition 3.10. Neural representations whose RSM de- pends on the specific choice of input weights are lask- agnostic representations.
3.1. Hidden-layer representations
In contrast, hidden-layer representations of MRNS
Understanding hidden-layer representations begins with identifying the degrees of freedom in the input weights, which govern how inputs are mapped into hidden space. Intuitively, input weights are constrained only by the need to preserve sufficient task-relevant information for the out- put weights to solve the task. In this section, we go be- yond this intuition by leveraging the exact parametrisations of 2 to precisely characterise the degrees of freedom in hidden-laaee representations. Proofs for Corollaries 3.9, 3.11 1111lm and 3.1 are in Appendix E. We begin by noting that
H = X
(21)
and MWNS
- RVSVTXX
(22)
are unique up to an orthogonal transformation R, which in- cludes rotations and reflections. Thus, in both cases, hidden- layer representations are not unique. In the semantic hierar- chy task, this results in representations that appear arbitrary and unstructured (Figure 3D, E). However,
4

---

An analytical dissociation of functional and representational similarity in deep linear neural networks
B
y
Hidden Reprosertation
RSM
MDS
200
Repreeentiiion
RSM
Taiget Varue
NDS
Hilssin Nouton Patvvy
fari
50
A4
Dmm
Figure 3: Hidden-layer representations. (A) Schematic of the semantic hierarchy task. (B) Inputs encoded random vectors (left) and corresponding target vectors encode for the position the hierarchy (right). one (zero) indicates that item (not) a child a node. (C) Example hidden-layer representations (left), RSM (centre) and corresponding 2D multidimensional scaling plot (right) for a GLS, (D) MRNS, and (E) MWNS.
Corollary 3.11. The RSM of MRNS unique and given by
is linking changes connectivity and representations to changes function. However, if representation and func- tion are dissociable, two key observations follow. First, changes in network function need not alter neural represen- tations, changes in one layer can be offset by compen- satory changes in the next. Second, changes in network function can proceed without altering representations, as it can, in principle, occur entirely in downstream layers. We now examine the implications of these findings for represen- tational comparisons, representational drift, and the synaptic stability-plasticity trade-off through series of illustrative simulations. While exact outcomes depend task specifies and hyperparameter choices, these are not our focus here; systematic analytical and numerical exploration is left for future work.
RSM ONO".
(23)
Since O and N are fully determined by the training data, the RSM is invariant to the specific choice input weights. Similarly, Corollary 3.12. The RSM of MWNS is unique and given by
RSM X
(24)
Again, the RSM invariant to the specific choice of input weights, as and S are fully determined by the training data. Accordingly, we make
Definition 3.13. Neural representations whose RSM if fully determined by the training data are lask-specific representa- tions.
4.1. Linear predictivity
A common method for comparing neural representations is to assess how well activation patterns from a source model or recording can predict those of a target via linear regres- sion (e.g., Yamins et 2014; Yamins and DiCarlo 2016). High linear predictivity often interpreted evidence that two systems process information similarly. However, since function and hidden-layer representations dissociable, strong linear predictivity does not necessarily imply func- tional alignment. We illustrate this by comparing hidden- layer representations from independent random walks on the solution manifold LSS, MWNS, and MRNS (Figure 4A). In each case, we compare either representations from two different network functions (across function) from the same network function (within function). While R² scores are on average slightly higher for within-function compar- isons, the main determinant of predictivity is whether the representations are task-agnostic or task-specific. Specifi- cally, across-function comparisons, R²² scores are highest when the source representation comes from LSS (which shares representational degrees of freedom with GLS), fol-
In the semantic hierarchy task, this yields an RSM that reflects the hierarchical structure, where representational similarity increases with proximity in the hierarchy, for MRNS; and an RSM that reflects combination of input statistics and target hierarchy for MWNS (Figure 3D.E).
In summary, in two-layer linear networks, neural representa- tions and the underlying function are dissociable: the same function can arise from different hidden-layee layer representa- tions, and the same representations can support different functions. For GLS and LSS this flexibility supports al- most arbitrary representations and task-agnostic RSMs. In contrast. MRNS and MWNS impose constraints which de- termine representations up orthonormal transformations, and are guaranteed to have unique and task-specific RSMs.
4. Implications for neural data analysis
fundamental challenge in understanding computations and learning in artificial and biological neural networks
5

---

An analytical dissociation of functional and representational similarity in deep linear neural networks
Across Functions
Within Function
B
Across Functions
Within Function
Source
TTagge
Drift
D
Input Noise
Parameter Noise
Decoding Error
1.0
Expected Loss
3.5
LS5 -MVNN
Poargon
LS8 MRNS
LSS
MRNS -155
MRNS -MNN
0.5
6
3.0
MM/S
MRNS
Theory
MVNS :LSS
0-
MVNE - -MWNS
0.0
2.5
Step 100
Step
100
Step
100
Step
100
MVNNS - -RRN
Stop
500
Step
100
Step
100
Figure 4: Implications for neural data analysis. All panels show results during random walks on the solution manifolds of LSS, MWNS, and MRNS. (A) Mean and standard deviation of At² scores for linear predictivity across n = 10 random walks, All sauree-target combinations are shown for across-function (left) and within-function (right) comparisons. (B) Example trajectories of RSA correlation scores, shown for across-function (left) and within-function (right) comparisons. (C) MSE of linear decoder trained on the hidden-layer representation at the initial time step. (D) Mean and expected MSE under input noise (left) and parameter noise (right).
lowed by MRNS, and lowest for MWNS. Within-function comparisons yield high R2 when the source is task-agnostic or when both source and target are of the same type; in contrast, predicting a lask-agnostic representation from a task-specific one lead to the lowest R² scores. These pat- terns arise because task-agnostic solutions process both rel- evant and irrelevant input directions, producing higher-rank representations that cannot be linearly predicted from the lower-rank task-specific ones. These results indicate that linear predictivity is predominantly driven by solution type rather than functional alignment, and may yield misleading conclusions if underlying representational constraints are not explicitly taken into account.
sentations should likewise remain stable (Rule al. 2019; Driscoll et al. 2022). However, this assumptions is chal- lenged by converging evidence of representation drift across species, brain regions and modalities (e.g.. Ziv et al. 2013; Driscoll et al. 2017; Schoonover et al. 2021; Marks and Goard 2021; Deitch et al. 2021; Ahmed et al. 2024). Our analysis shows that the existence of solution manifold al- lows hidden- layer representations to vary without changing the implemented function. This dissociation implies that sta- ble perception and behaviour do not require stable represen- tations. Indeed, an optimal linear decoder trained on an ini- tial representation rapidly degrades in performance during a random walk on the solution manifold (Figure 4C). Thus, representational drift need not signal functional change, but may instead reflect a reparametrisation within a functionally equivalent subspace.
4.2. Representational similarity analysis
Representational similarity analysis (RSA) compares neu- ral activation patterns by evaluation the similarity of RSMs across conditions, stimuli, models, or partici- pants (Kriegeskorte et al. 2008; Haxby et al. 2014). As with linear predictivity, our analytical results show that the interpretability of RSA in terms of functional alignment critically depends on the solution type of the comparanda. We illustrate this in Figure 4B using example trajectories from the previous section. Since task-agnostic solutions (i.e., GLS and LSS) exhibit highly flexible RSMs, correla- tion coefficients r involving such representations fluctuate unpredictably throughout the random walk, both within and across functions. By contrast, comparing within and across task-specific solutions (i.e., MWNS and MRNS) results in static and consistent r, as they exhibit unique RSMs. How- ever, because MWNS and MRNS induce different unique RSMs, comparisons across types yield imperfect similarity even within the same function. In summary, RSA reliably reflects functional similarity only when representational con- straints enforce unique RSMs, underscoring the importance of accounting for solution type in representational compar- isons.
4.4. Synaptic stability and plasticity
The so-called ttaility-plasticiit dilemma posits that neural systems must remain plastic enough acquire new knowl- edge while remaining stable enough to retain previously learned information (Grossberg 1987; Abraham and Robins 2005). This view, grounded in single-neuron and synapse- level intuitions, has motivated continual learning algorithms that explicitly regulate synaptic changes to preserve past knowledge (e.g., Kirkpatrick al. 2017: Zenke et al. 2017: Aljundi et al. 2018). However, our analysis shows that this dilemma need not apply at the network level, because, independent of the solution type, many distinct configura- tions of synapses and representations implement the same function (see e.g., Figure 1). This raises important method- ological concerns: observing or inducing isolated synaptic or representational changes may not suffice to infer function, learning, or the underlying learning mechanisms in artificial or biological neural networks.
5. Ad Advantages of task-specific representations
4.3. Drifting neural representations
A natural question arises from the observation that function and representation are dissociable: why do biological and artificial systems often converge to non-arbitrary represen- tations that obey the structure of the task? One possible
Intuitively, one might expect that if a stimulus elicits sta- ble perception and behaviour, the associated neural repre-
6

---

An analytical dissociation of functional and representational similarity in deep linear neural networks
explanation is that task-specific representations confer com- putational advantages, creating selective pressure in both biological and artificial systems. Consequently, such repre- sentations may emerge as preferred functional implementa- tions, leading to representational alignment across systems.
Thus, while robustness to input noise selects for solutions with minimal functional norm, LSS employ task-agnostic representations, so robustness alone does not ensure repre- sentational alignment.
Theorem 5.4. The expected loss under additive, i.i.d,, zero- centred noise ES and E2 in the parameters, with vari- ances of x 1/1 ||| and o x 1/N, is
5.1. Secondary error
One hypothesised advantage of task-specific representations is improved performance on secondary datasets, such as in- or out-of-distribution generalisation. To test this, we identify solutions on the primary-task solution manifold that minimise the eтror on an unseen secondary dataset
1
( + +2) (Ω) + B1) Xn vnll
2P
(28)
1
2 + 1221 + +c) (
|
2P
D = [(x,: yn)] In Appendix F we derive,
Theorem 5.1. The secondary error is minimised by all solutions of the form
where e
is again noise-independent constant.
Corollary 5.5. The expected loss under parameter noise exclusively minimised by MRNS (see Figure 4D).
Ω25 = Eys 2x + Z.P,,
(25)
where
We have scaled noise variances simplify the analytical expression; without this scaling, the results hold up to mul- tiplicative constants. Robustness to parameter noise thus selects for solutions with task-specific representations, en- suring representational alignment.
? = (8- - Σ E
PaX)
+ +TP
(26)
with I E N.NNN arbitrary and P., = I- (IPPP
Since the solution is a LSS with a perturbation in the unob- served null directions of the primary-task inputs, minimis- ing secondary error permits task-agnostic solutions. Thus, observing H gives no information about secondary task per- formance and secondary error cannot explain the emergence of task-specific representations in two-layer linear networks.
Theorem 5.6. Under the assumption that the input data is whitened, i.e.. Dez = I, the expected loss under additive, i..d., zero-centred parameter noise E₁ and E2 with vari- ance of x 1/N, and o x I/N, and input noise Ex.. with variance o² is
P
|1 (12 + 52) (Ω) + +E)) (x. + Ex..) -vall2
5.2. Sensitivity to noise
2P
n=1
Neural systems are subject multitude of internal and external sources of noise, which range from variability in
22
[ | 1 1| + || (1:)
(29)
2
incoming sensory signals to fluctuations in synaptic effi- cacy and spontaneous neural activity (Faisal et al. 2008). Therefore, solutions that exhibit robustness to such noise are advantageous, as they enable the neural circuitry to main- tain reliable function (Johnston al. 2020). The following theorems are derived in Appendix G.
+ 2P
(1
12
+ 11 X / +
with noise- iniisseeddeendentt constant
Corollary 5.7. Under the stated assumptions and con- straints, the expected loss under input and parameter noise is minimised exclusively by MRNS and MWNS.
Theorem 5.2. The expected loss under additive, indepen- dent and identically distributed (i.i.d.), zero-centred input noise Ex,, with variance o², is
We note, that if the input data is not whitened, interaction terms render the optimal subspace depends on input statis- tics, complicating its explicit analytical characterisation.
12222 (xn, + Ex..) -xx))- vnl13
2P
In summary, neither minimising secondary error nor input noise sensitivity promotes task-specific representations. In contrast, robustness to parameter noise selectively favours solutions with task-specific structure, thereby supporting representational alignment. This suggests that shared rep- resentations across biological and artificial systems may arise from implicit or explicit optimisation for parameter robustness, such as regularisation strategies that favour low- norm solutions. However, this reflects an inductive bias
(27)
12221111 +c,
where is indeeeeendeenddntt constant that only depends on the training data.
Corollary 5.3. The expected loss under input noise is ex- clusively minimised by LSS, which include MWNS as a subspace (see Figure 4D).
7

---

An analytical dissociation of functional and representational similarity in deep linear neural networks
Neural Activity
RSM
B
Neural Activity
RSM
RSM
Input
Parameter Nois
Hoddn Nouron
MNIST Input
Hidden Neuroon
MNIST Input
MNGe Inpui
test error (log)
10
duplicate scaleed
500
10
iigu-nuul trained
1000
30
500
1000
500
1000
500
1000
500
1000
500
1000
10
10"
10'
10-1 20- 10"
MNIIT Imput
MNIST Input
MNIST Input
MNIST Input
MNIST Input
noise scale (log)
noise scale (log)
Figure Function and representation are dissociable in non-linear networks. (A) Hidden-layer activations for 1024 MNIST inputs, grouped by class (left) and the corresponding spsc-fpe RSM (right) after training a ReLU network from small initial weights. (B) Same as (A), but for a network reparametrised via augmented Lagrangian optimisation to reshape hidden-layer representation while preserving (C) RSM of the network from (A) after reparametrisation using exact invariant transformations (Section 6. (D) Test error under input noise for all exact invariant transformations. Networks with input maputtnulll expansion sensitive. (E) As in (D) but for parameter noise; networks with scaled, nuisance, and duplic saaticate expansions are sensitive to varying degrees.
training set classifications.
rather than a general principle and without explicit justifi- cation, functional and representational alignment cannot be assumed to coincide.
not capture all functionally equivalent parametrisations out- side of the teacher-student setting, they provide means to construct network reparametrisations that exactly pre- serve network function. We realise two of their invariances by inserting hidden-layer neurons with arbitrary incoming weights and zero outgoing weights (nuisance-neuron in- variance) and by duplicating hidden-layer neurons while halving the outgoing weights of both the original and the duplicated neuron (duplication invariance). Lastly, one can add any perturbation to the input-layer weights that lies in the unobserved nullspace of the input data while preserving the network's function (inpu--ullspace invariance).
6. Nonlinear networks
The results presented thus far apply to two-layer linear net- works. We now extend our study to emphnonlinear net- works (networks with nonlinear activation functions), and show that they exhibit analogous degrees of freedom in representation and associated computational trade-offs as their linear counterparts. Here, we face a challenge: A full characterisation of the solution manifold of general non- linear network remains analytically intractable, even for two-layer networks (Misiakiewicz and Montanari 2024). However, substantial progress has been made in deriving transformations that allow reparametri- sation of nonlinear networks while leaving their onput-output map invariant (Simsek et al. 2021; Martinelli et al. 2023). Here, we exploit these function-preserving transformations to construct functionally equivalent reparametrisations of nonlinear networks, which allows us to probe computational differences between networks with minimal and expanded representations, in analogy to the task-specific and task- agnostic representations of Section 3.
6.2. Manipulating representations of ReLU networks
We train a two-layer ReLU network with 1024 hidden neu- rons on the MNIST dataset (LeCun et al. 1998) from small norm random weights, resulting in task-specific represen- tations (Figure 5A, "rich" learning from Jacot et al. 201 8; Chizat al. 2019; Woodworth et al. 2020). Starting from this trained model, we use augmented Lagrangian optimisa- tion to modify the hidden-layer activations of 1024 training inputs such that their representations collectively resemble an image of two elephants, while enforcing that the net- work's ' predicted class labels remain unchanged across the entire training set (Figure 5B). This transformation illus- trates the representational freedom among nonlinear ReLU networks that make the same class predictions. In addi- tion, even when limited to manipulations that exactly pre- serve a network's 's input-output map (a stricter condition than preserving classifications) one can induce a task-agnostic RSM (Figure 5C).
6.1. Functional invariances in deep ReLU networks
Feedforward networks with any activation function are output-invariant to permuting neurons within a layer, as permuting the rows of one weight matrix and the corre- sponding columns of the next leaves the network function unchanged (permutation invariance; Sussmann 1992). Feed- forward networks with rectified linear unit (ReLU) activa- tion further invariant rescaling a neuron's 's incoming weights by factor 0 and its outgoing weights by 1 1/a, due to the non-negative homogeneity of ReLU (scale invari- ance; Neyshabur et al. 2015a). Simsek et al. (2021) and Martinelli et al. (2023) identify additional invariances that fully characterise the manifold of global minima in teacher- student settings, where one network trained to replicate the function of another. Although these invariances may
6.3. Computational advantages in ReLU networks
To complement the analytical results in Section 5.2 on the ro- bustness of task-specific representations in linear networks, we empirically evaluate secondary (test) error and robust- ness to input and parameter noise across different nonlinear solution types. We train two-layer networks with 1024 hid- den dimensions and ReLU activation on the training set of the MNIST digit classification task (LeCun et al. 2010) from
8

---

An analytical dissociation of functional and representational similarity in deep linear neural networks
8 random initialisations. These models trained from small initial weights serve our task-trained (minimal) solutions. We next apply four dunction-preserving transforms defined by the four invariances of Section 6.1 to these task-trained (minimal) networks to construct expanded (non-minimal) parametrisations that exactly preserve the network function. These minimal and non-minimal parametrisations serve as our solution types in the nonlinear setting.
Comparing the solutions of artificial and biological net- works. Neuroscientists have identified parallels between artificial and biological neural computation (Richards et al. 2019; Saxe et al. 2021; Doerig et al. 2023) from hierarchi- cal feature extraction in visual processing (DiCarlo et al. 2012; Eickenberg et al. 2017; Lindsay 2021) to analogous population dynamics during decision-making tasks (Mante et al. 2013; Chaisangmongkoon developed various methods to quantify this shared repre- sentational structure, including representational similarity analysis (Kriegeskorte et al. 2008), linear predictivity (e-g-. Yamins et al. 2014; Yamins and DiCarlo 2016), and metric- based methods (Williams et al. 2021); see Klabunde et al. (2025) for an overview of methods. However, recent work has identified significant methodological challenges in com- paring representations between artificial neural networks, including confounding effects from stimulus correlations (Cai et al. 2019; Hermann and Lampinen 2020; Dujmović et al. 2023), metric-dependent results (Ding et al. 2021; Bo et al. 2025), and difficulties in matching representations even between identical networks trained with different ran- dom initialisations (Han et al. 2023); these negative results suggest further problems when comparing artificial and bio- logical neural networks, where little is known in advance of the computation the biological networks performs. These challenges may apply to existing neural predictivity bench- marks such as Brain-score (Schrimpf et al. 201 8; Schrimpf et al. 2020) and the Natural Scenes Dataset (NSD; Allen et al. 2022).
al. 2017). The field has
In Figure 5D, we observe the effect of input noise the task-trained (minimal) and expanded (non-minimal) net- works. In accordance with the linear result, only transforma- tions in the unobserved input space have a deleterious effect (input-null). In Figure SE, we observe the effect of pa- rameter noise on the initial and transformed networks. Non- minimal models (s cacclle ed, nuisance, dup i ciatte e) de- grade in test error more quickly than minimal ones, similarly to what is derived in Section 5.2, though duplicated expan- sions (duplic i cate)) are more robust due to noise averaging. In contrast, input-nullspace perturbations (input -null) have no effect because transformations are in unobserved input directions, and input noise is absent. Lastly, at near- zero noise levels, we observe that no manipulations inflate the secondary (test) error, consistent with the result of Sec- tion 5.1 that generalisation performance does not constrain network representations to be minimal.
7. Related work
The solution manifold of artificial networks. The solu- tion manifold of two-layer neural networks was first de- scribed by Baldi and Hornik (1989). Subsequent work showed under some assumptions that all minima in deep linear networks are global and equivalent to those in linear regression (Laurent and Brecht 20.8). The dissociation be- tween general linear solutions and minimum-norm solutions has been previously studied under a set of strong assump- tions (Saxe al. 2014). Sensitivity to noise for task-specific and task-agnostic rich and lazy solutions has been previously studied numerically in nonlinear neural networks (Flesch et al. 2022) and generalisation and transfer performance of deep linear networks have been previously investigated (Lampinen and Ganguli 2019; Advani et al. 2020; Tahir et al. 2024; Ingrosso et al. 2025) using a teacher-student paradigm (Gardner and Derrida 1989; Riegler and Bichl 1995; Saad and Solla 1995). The relation between representational drift and drift on the solution manifold that results from stochas- ticity during gradient descent (Chaudhari and Soatto 2018 8) has been studied on a subpart of the solution manifold in linear networks (Pashakhanloo and Koulakov 2023) and in nonlinear networks, again, relying on the teacher-student setting (Avidan et al. 2023; Li et al. 2024).
8. Discussion
In this work, we give a complete analytical characterisation of the global minima manifold for deep linear networks, and demonstrate that different subregions of this manifold afford different interpretability and computational properties due to their representational structure. We conclude that the use of deep, overparametrised networks poses fundamental challenges for representational analysis, interpretation, and comparison, as the impact of variability in the parametrisa- tion of functionally equivalent representations on these use cases is significant.
Our analysis does not assume specific learning algorithm and ignores the question of how a specific solution could be attained in practice. However, the computational advantages of task-specific representations detailed in Section 5 are compatible with the view that gradient descent, in particular in overparametrised models, is subject to implicit regularisa- tion that prefers solutions with certain optimality properties for both linear and nonlinear networks (Neyshabur et al. 2015b; Zhang et al. 2017: Neyshabur 2017: Du al. 2019; Arora al. 2018; Chizat and Bach 2020; Yun et al. 2021; Vardi and Shamir 2021).
9

---

An analytical dissociation of functional and representational similarity in deep linear neural networks
Acknowledgements
Arous, Gérard Ben, Reza Gheissari, and Aukosh Jagannath (2022). "High-dimensional limit theorems for SGD: Ef- fective dynamics and critical scaling". . In: Advances in Neural Information Processing Systems 35 L
L.B. was supported by the Woodward Scholarship awarded by Wadham College. Oxford and the Medical Research Council (MR/N013468)) E.G. and A.S. were supported by Schmidt Science Polymath Award to A.S., and the Sainsbury Wellcome Centre Core Grant from Wellcome (219627/2/1//ZZ) and the Gatsby Charitable Foundation (GAT3850).
Avidan, Yehonatan, Qianyi Li, and Haim Sompolinsky (2023), Connecting NTK and NNGP: A Unified Theo- retical Framework for Wide Neural Network Learning Dynamies. E
Baldi, Pierre and Kurt Homik (Jan. 1, 1989). "Neural Net- works and Principal Component Analysis: Learning from Examples without Local Minima". In: Neural Networks
Impact statement
2.1, pp. 53-58 E
This paper presents work whose goal is advance the field of machine learning. There are many potential societal consequences of our work, none of which we feel must be
Bo, Yiqing, Ansh Soni, Sudhanshu Srivastava, and Meenakshi Khosla (2025). "Evaluating Representational Similarity Measures from the Lens of Functional Cor- respondence". In: 8th Annual Conference on Cognitive Computationall Neuroscience L
specifically highlighted here.
References
Ponaa-ellissseer, Joachim, François Bachoc, and François Malgouyres (Nov. 1, 2023). "Parameter Identifiability of a Deep Feedforward ReLU Neural Network". In: Machine Learning 112.11, pp. 4431-4493 C
Abraham, Wickliffe C and Anthony Robins (2005). "Mem- ory retention-the synaptic stability versus plasticity dilemma".. In: Trends in neurosciences 28.2, pp. 73-78 Advani, Madhu S., Andrew M. Saxe, and Haim Sompolin- sky (2020). "Higg--imeensinall dynamics of generaliza- tion error in neural networks". In: Neural Networks 132,
Braun, Lukas, Clémentine Dominé, James Fitzgerald, and Andrew Saxe (2022). "Exact learning dynamies of deep linear networks with prior knowledge". In: Advances in Neural Information Processing Systems 35 L
pp. 428-446 [
Cai, Ming Bo, Nicolas W. Schuck, Jonathan W. Pillow, and Yael Niv (May 2019). "Representational Structure or Task Structure? Bias in Neural Representational Similarity Analysis and Bayesian Method for Reducing Bias".
Ahmed, Alisha, Bettina Voelcker, and Simon Peron (Dec. 16, 2024). "Representational Drift in Barrel Cortex Is Recep- tive Field Dependent". In: Current Biology 34.24, 5623- 5634.e4 L
In: PLOS Computational Biology 15.5, pp. 1-30 L Chaisangmongkonn Warasinee, Sruthi K. Swaminathan, David J. Freedman, and Xiao-Jing Wang (Mar. 22, 2017). "Computing by Robust Transience: How the Fronto- Parietal Network Performs Sequential, Category- Decisions". In: Newron 93.6, 1504-1517.e4 L Chaudhari, Pratik and Stefano Soatto (2018). "Stochas- tic gradient descent performs variational inference, con- verges to limit cycles for deep networks". In: Proceedings of the 6th International Conference on Learning Repre- sentations. Open L
Albantakis, Larissa et al. (2024). "The brain's s best kept secret is its degenerate structure". In: Journal of Neuro- science 44.40 E
Albertini, Francesca and Eduardo D. Sontag (1993). "For neural networks, function determines form". In: Neural Networks 6.7, pp. 975-990 L
Aljundi, Rahaf et al. (2018). "Memory aware synapses: Learning what (not) to forget". In: Proceedings of the Eu- ropean conference on computer vision (ECCV), pp. 139- 154
Allen, Emily et al. (2022). "A massive 7T fMRI dataset to bridge cognitive neuroscience and artificial intelligence". In: Nature Neuroscience 25.1, pp. 116 126 E
Chizat, Lénaïc and Francis R. Bach (2020). "Implicit Bias of Gradient Descent for Wide Two-layer Neural Networks Trained with the Logistic Loss". In: Conference on Learn- ing Theory. Vol. 125. Proceedings of Machine Learning Research. Pmlr, pp. 1305-1338 E Chizat, Lénaïc, Edouard Oyallon, and Francis R. Bach (2019). "On Lazy Training in Differentiable Program- ming". In: Advances in Neural Information Processing Systems, pp. 9933-2943 E Deitch, Daniel, Alon Rubin, and Yaniv Ziv (2021). "Repre- sentational drift in the mouse visual cortex". In: Current biology 39. pp. 4327-4339
Arora, Sanjeev, Nadav Cohen, Noah Golowich, and Wei Hu (2019). "A Convergence Analysis of Gradient Descent for Deep Linear Neural Networks". In: 7th International Con- ference on Learning Representations. OpenReview.net 2
Arora, Sanjeev, Nadav Cohen, and Elad Hazan (2018). "On the optimization of deep networks: Implicit acceleration by overparameeterizationn" In: Proceedings of the 35th International Conference on Machine Learning. Vol. 80. PMLR, pp. 244-253 E
10

---

An analytical dissociation of functional and representational similarity in deep linear neural networks
DiCarlo, James J., Davide Zoccolan, and Nicole C. Rust (Feb. 9, 2012). "How Does the Brain Solve Visual Object Recognition?" In:
Godfrey, Charles, Davis Brown, Tegan Emerson, and Henry Kvinge (2022). "On the symmetries of deep learning models and their internal representations". In: Advances in Neural Information Processing Systems 35, pp. 11893- 11905 E
Ding, Frances, Neuron 73.3, pp. 415-434 L Denain, and Jacob Steinhardt (2021). "Grounding Representation Similarity Through Statistical Testing". In: Advances in Neural Information Processing Systems 34, pp. 1556-1568 &2
Goldman, Mark S, Jorge Golowasch, Eve Marder, and LF Abbott (2001). "Global structure, robustness, and modu- lation of neuronal models". In: Journal of Neuroscience
Doerig, Adrien et al. (2023). "The neurocomnectionist re-
search programme". In: Nature Reviews Neuroscience
21.14, pp. 5229-5238 L
24.7 (7), pp. 431-450 L
Greville, Thomas Nall Eden (1966). "Note on the general- ized inverse of a matrix product". In: Siam Review 8.4, pp. 518-521
Dominé, Clémentine CJ al. (2025). "From lazy to rich: Ex- act learning dynamics in deep linear networks". In: 13th International Conference on Learning Representations
Grossberg, Stephen (1987). "Competitive learning: From in- teractive activation to adaptive resonance". In: Cognitive science 11.1. pp. 23-63
Driscoll, Laura N, Lea Duncker, and Christopher D Harvey (2022). "Representational drift: Emerging theories for continual learning and experimental future directions". In: Current Opinion in Neurobiology 76, p. 102609
Han, Yena, Tomaso A. Poggio, and Brian Cheung (2023). "System identification of neural systems: If we got right, would we know?" In: International Conference on Ma- chine Learning. Vol. 202. Proceedings of Machine Learn- ing Research. PMLR, pp. 12430- 12444 L
Driscoll, Laura N et al. (2017). "Dynamic reorganization of neuronal activity patterns in parietal cortex". In: Cell 170.5. pp. 986-999
Haxby, James V, Andrew C Connolly, and J Swaroop Gun- tupalli (2014). "Decoding neural representational spaces using multivariate pattern analysis". In: Annual review of neuroscience 37.1, pp. 435-456 L2'
Du, Simon S., Xiyu Zhai, Barnabás Póczos, and Aarti Singh (2019). "Gradient Descent Provably Optimizes Over preemettamteri Neural Networks". In: 7th Interna- tional Conference on Learning Representations. Openke-
Hermann, Katherine L.. and Andrew K. Lampinen (2020). "What shapes feature representations? Exploring datasets, architectures, and training". In: Advances in Neural Infor- mation Processing Systems 33 L
view.net L
Dujmović, Marin, Jeffrey S. Bowers, Federico Adolfi, and Gaurav Malhotra (2023), Obstacles to inferring mecha- nistic similarity using representational similarity analysis
Ingrosso. Alessandro, Rosalba Pacelli, Pietro Rotondo, and Federica Gerace (Apr. 30, 2025). "Statistical Mechanies of Transfer Learning in Fully Connected Networks in the Proportional Limit". In: Physical Review Letters 134.17,
Eickenberg, Michael, Alexandre Gramfort, Gaäl Varoquaux, and Bertrand Thirion (May 15, 2017). "Seeing It All: Convolutional Network Layers Map the Function of the Human Visual System". In: Neurolmage 152, pp. 184-
p. 177301 C
Jacot, Arthur, Franck Gabriel, and Clément Hongler (2018).
194 E
"Neural tangent kernel: Convergence and generalization in neural networks". In: Advances in neural information
Entezari, Rahim, Hanie Sedghi, Olga Saukh, and Behnam Neyshabur (2022). "The Role of Permutation Invariance in Linear Mode Connectivity of Neural Networks". In: The 10th International Conference on Learning Repre- sentations. OpenReview.nel L
processing systems 31 L
Johnston, W Jeffrey, Stephanie E Palmer, and David J Freed- man (2020). "Nonlinear mixed selectivity supports reli- able neural computation". In: PLOS computational biol- ogy 16.2, e1007544 L2
Faisal, A Aldo, Luc PJ Selen, and Daniel M Wolpert (2008).
"Noise in the nervous system". In: Nature reviews neuro-
Kirkpatrick, James et al. (2017). "Overcoming catastrophic forgetting in neural networks". In: Proceedings of the national academy of sciences 114.13, pp. 3521-3526 Klabunde, Max, Tobias Schumacher, Markus Strohmaier, and Florian Lemmerich (May 2025). "Similarity of Neu- ral Network Models: A Survey of Functional and Rep- resentational Measures". In: ACM Comput. Surv. 57.9 E
science 9.4, pp. 292-303 E
Flesch, Timo et al. (Apr. 6. 2022). "Orthogonal Representa- tions for Robust Context- Context-Dependent Task Performance in Brains and Neural Networks". In: Neuron 110.7 (7), 1258-1270.e11 E
Gardner, E. and B. Derrida (1989). "Three unfinished works on the optimal storage capacity of networks". In: Journal of Physics A: Mathematical and General 22.12, p. 1983 L
Kori, Avinash et al. (2024). "Identifiable Object-Centric Representation Learning via Probabilistic Slot Attention".. In: The Tiihth Annual Conference on Neural Infor- mation Processing Systems Z
Ghosh, Shubhangi et al. (2022). On Pitfalls of identifiability in Unsupervised Learning. A Note on: "Desiderata for Representation Learning: A Causal Perspective". E

---

An analytical dissociation of functional and representational similarity in deep linear neural networks
Kriegeskorte, Nikolaus, Marieke Mur, and Peter A Bandet- tini (2008). "Representational similarity analysis con- necting the branches of systems neuroscience". In: Fron- tiers in systems neuroscience 2. p. 249 E
nal of Statistical Mechanics: Theory and Experiment
2024.10, 10, p. 104006 L
Neyshabur, Behnam (2017). "Implicit Regularization in Deep Learning" f
Lampinen, Andrew K. and Surya Ganguli (2019). "An ana- lytic theory of generalization dynamics and transfer learn- ing in deep linear networks". In: 7th International Con- ference on Learning Representations. OpenReview.net 2
Neyshabur, Behnam, Ruslan Salakhutdinov, and Nathan Srebro (2015a). "Path-SGD: Path-Normalized Optimiza- tion in Deep Neural Networks". In: Advances in Neural Information Processing Systems, pp. 2422-2430 L Neyshabur, Behnam, Ryota Tomioka, and Nathan Srebro (Apr. 16, 2015b). "In Search of the Real Inductive Bias: On the Role of Implicit Regularization in Deep Learning". In: ICLR 2015 Workshop Track. International Conference on Learning Representations C
Lampinen, Andrew Kyle, Stephanie CY Chan, and Kather- ine Hermann (2024). "Learned feature representations are biased by complexity, learning order, position, and more". In: Transactions on Machine Learning Research.
ISSN: 2835-8856 C23
Pashakhanloo, Farhad and Alexei Koulakov (2023). "Stochastic gradient descent-induced drift of represen- tation in two-layer neural network". In: International
Laurent, Thomas and James von Brecht (2018). "Deep linear networks with arbitrary loss: All local minima are global". In: Proceedings of the 35th International Conference on Machine Learning- Vol. 80. Proceedings of Machine Learning Research. PMLR, pp. 2908-2913 L LeCun, Yann, Léon Bottou, Yoshua Bengio, and Patrick Haffner (1998). "Gradient-based learning applied to doc- ument recognition". In: Proceedings of the IEEE 86.11,
Conference on Machine Learning. Vol. 202. Proceedings of Machine Learning Research. PMLR, pp. 27401-27419
Prinz, Astrid A, Dirk Bucher, and Eve Marder (2004). "Sim- ilar network activity from disparate circuit parameters". In: Nature neuroscience 7.12. pp. 1345-1352 C Refinetti, Maria, Sebastian Goldt, Florent Krzakala, and Lenka Zdeborová (2021). "Classifying high dimensionni Gaussian mixtures: Where kernel methods fail and neu- ral networks succeed". In: Proceedings of the 38th In- ternational Conference on Machine Learning. Vol. 139, Proceedings of Machine Learning Research. PMLR, pp. 8936-8948 L Richards, Blake A. et al. (2019). "A deep learning frame- work for neuroscience". In: Nature Neuroscience 22.11 1 (11), pp. 1761-1770 L2 Riegler, P. and M. Biehl (Oct. 1995). "On-Line Backpropa- gation in Two-Layered Neural Networks". In: Journal of Physics A: Mathematical and General 28.20 (20), p. L5077
pp. 2278-2324 L
LeCun, Yann, Corinna Cortes, and CJ Burges (2010).
"MNIST handwritten digit database". In: ATT Labs [On- line]. Available: http:://anm.. http://yann.lecmm..om www.w.cww/eeateeee. cocm..com/xxd//miist 2 Li, Qianyi, Ben Sorscher, and Haim Sompolinsky (2024). "Representations and generalization in artificial and brain neural networks". In: Proceedings of the National Academy of Sciences 121.27, e2311805122
Lindsay, Grace W. (Sept. 1, 2021). "Convolutional Neural Networks as a Model of the Visual System: Past, Present, and Future". In: Journal of Cognitive Neuroscience 33.10, pp. 2017-2031 C
Mante, Valerio, David Sussillo, Krishna V. Shenoy, and William T. Newsome (Nov. 2013). "Context- Dependeet Computation by Recurrent Dynamics in Prefrontal Cor- tex". In: Nature 503.7474 (7474), pp. 78-84 L Marconato, Emanuele, Sébastien Lachapelle, Sebastian We- ichwald, and Luigi Gresele (2024). All or None: Identifi- able Linear Properties of -exe-Token Predictors in Lan- guage Modeling. L
Roeder, Geoffrey, Luke Metz, and Durk Kingma (2021).
"On Linear Identifiability of Learndd Representations" In: Proceedings of the 38th International Conference on Machine Learning. Vol. 139. Proceedings of Machine Learning Research. PMLR, pp. 9030-9039 L Rule, Michael E, Timothy O'l O'Leary, and Christopher D Har- vey (2019). "Causes and consequences of representational
Marder, Eve and Jean-Mare Goaillard (2006). "Variability, compensation and homeostasis in neuron and network function".. In: Nature Reviews Neuroscience 7.7, pp. 563-
drift". In: Current opinion in neurobiology 58. pp. 141- 147
574 L
Rumelhart, David E, James L McClelland, and the PDP Research Group (1986). Parallel Distributed Processing: Explorations in the Microstructure of Cognition, Volume I: Foundations. MIT Press E
Marks, Tyler D and Michael J Goard (2021). "Stimulus- dependent representational drift in primary visual cortex".
In: Nature communications 12.1, p. 5169
Martinelli, Flavio, Berfin Şimşek, Johanni Brea, and Wul- fram Gerstner (2023). Expand-and and-uuttrrr Exact Param- eter Recovery of Neural Networks. E
Saad, David and Sara A. Solla (Oct. 1, 1995). "On-Line Learning in Soft Committee Machines". In: Physical Re- view E 52.4 (4), pp. 4225-4243 E
Misiakiewicz, Theodor and Andrea Montanari (Oct. 2024). "Six Lectures on Lincarized Neural Networks". In: Jour-
Saxe, Andrew, Stephanie Nelli, and Christopher Summer- field (Jan. 2021). "If deep learning is the answer, what is
12

---

An analytical dissociation of functional and representational similarity in deep linear neural networks
the question?" In: Nature Reviews Neuroscience 22.1 (I), pp. 55-67 E
stand Sensory Cortex". In: Nature Neuroscience 19.3 (3), pp. 356-365 E
Saxe, Andrew M, James L McClelland, and Surya Ganguli (2019). "A mathematical theory of semantic development in deep neural networks". In: Proceedings of the National Academy of Sciences 116.23, pp. 11537-11546 E Saxe, Andrew M., James L. McClelland, and Surya Ganguli (2014). "Exact solutions the nonlinear dynamics of learning in deep linear neural networks". In: 2nd Interna- tional Conference on Learning Representations E Schoonover, Carl E, Sarah N Ohashi, Richard Axel, and An- drew JP Fink (2021). "Representational drift in primary olfactory cortex". In: Nature 594.7864, pp. 541-546 Schrimpf, Martin et al. (2018). "Brain-Score: Which Ar-
Yamins, Daniel L. K. et al. (2014). "Performance-Opiimizzd
Hierarchical Models Predict Neural Responses in Higher Visual Cortex". In: Proceedings of the National Academy
of Sciences 111.23 (23), pp. 8619-8624 L Yun, Chulhee, Shankar Krishnan, and Hossein Mobahi (2021). "A unifying view on implicit bias in training linear neural networks". In: 9th International Conference on Learning Representations, OpenReview.net L Zenke, Friedemann, Ben Poole, and Surya Ganguli (2017). "Continual learning through synaptic intelligence". In: International conference on machine learning. PMLR, pp. 3987-3995
tificial Neural Network for Object Recognition is most
Zhang, Chiyuan et al. (2017). "Understanding deep learning requires rethinking generalization". In: Proceedings of the 5th International Conference on Learning Represen- tations. OpenReview.net L
Brain-Like?' In: bioRxiv E
Schrimpf. Martin et al. (2020). "Integrative Benchmark- ing to Advance Neurally Mechanistic Models of Human Intelligence". In: Neuron L
Zhang, Chiyuan et al. (2021). "Understanding deep learn- ing (still) requires rethinking generalization". In: 64.3,
Simsek, Berfin et al. (2021). "Geometry of the Loss Land- scape in Overparameterized Neural Networks: Symme- tries and Invariances". In: Proceedings of the 38th In- ternational Conference on Machine Learning. Vol. 139. Proceedings of Machine Learning Research. PMLR,
pp. 107-115 L
Ziv, Yaniv et al. (2013). "Long-term dynamics of CA1 hip- pocampal place codes". In: Nature neuroscience 16.3, pp. 264-266
pp. 002999229999 9732 L
Sussmann, Héctor J. (July 1, 1992). "Uniqueness of the Weights for Minimal Feedforward Nets with a given Input-Output Map". In: Neural Networks 5.4, pp. 589-
593 L
Tahir, Javan, Surya Ganguli, and Grant M. Rotskoff (2024). Features Are Fate: A Theory Transfer Learning in High- Dimensional Regression. E
Vardi, Gal and Ohad Shamir (2021). "Implicit Regular- ization in ReL.U Networks with the Square Loss". In: Proceedings of 34th Conference on Learning Theory. Vol. 134. Proceedings of Machine Learning Research. Pmlr, pp. 4224-4255 L
Vlačić, Verner and Helmut Böleskei (2022). "Neural net- work identifiability for family of sigmoidal nonlineari- ties". In: Constructive Approximation 55.1, pp. 173-224 E
Wang, Yixin and Michael I. Jordan (2021). Desiderata for Representation Learning: Causal Perspective. E Williams, Alex H., Erin Kunz. Simon Kornblith, and Scott
W. Linderman (2021). "Generalized Shape Metrics on Neural Representations". In: Advances in Neural Infor- mation Processing Systems 34, pp. 4738-4750 E Woodworth, Blake E. et al. (2020). "Kernel and Rich Regimes in Overparametrizee Models". In: Conference on Learning Theory. Vol. 125. Proceedings of Machine Learning Research. Pmlr, pp. 3635-361 E Yamins, Daniel L K and James J DiCarlo (Mar. 2016). "Using Roal-Driven Deep Leanning Models to Under-
13

---

An analytical dissociation of functional and representational similarity in deep linear neural networks
A. Simulation details
code repository reproducing all figures is available on GitHub at Lukas btaamm/dsssccciaannnnniiilllll
A.1. Random walk
Given a two-layer linear network with weight matrices W and 22, we implemented random walks on the GL.S as follows. First, we randomly initialised weight matrices using a random normal initialisation with zero mean and standard deviation 1/vN; and 1/vN, respectively. Then, for each step. we first diffused both weight matrices by
W(t 1) (1-a)W(t)) + +
(30)
with
{~ 0, 0² 25), and 0.00625
(31)
and then performed gradient descent on both W, and W2 with learning rate - 0.25 to return back to the solution manifold. Similarly, for the random walk on the LSS, we sampled initial W₁ and W randomly, diffused them, performed gradient descent on both W, and W2, and subsequently enforced the two rank constraints to return the solution manifold. The random walk on the MRNS was implemented by first computing
x MNO
(32)
sampling random
T T~ N( = 0, 1/N,), and R ~ Ni = 0, = 1). Then, in every step, we diffused and R according to Equation (30), and then calculated
(33)
R R=R,R with cSVD(R) = R. DR..
(34)
We then set
W RVNO +r and W₂ - MVNR".
(35)
Similarly, for MWNS we first computed the cSVD of the LSS
cSVD ( yz (E)z)+)= = USV
(36)
and sampled a random matrix R N(p= 0, 1). Then, for each step, we defused R according to Equation (30) and then calculated R for MWNS. Network weights were then set to
W, RVSVT
(37)
and
W₂ + UVSR".
(38)
A.2. Tasks
Random regression task Random regression tasks are made up of a randomly sampled input
X ~~ N~ 0, 0² = 1/N,) E N.P
(39)
and target matrix
N~ O= E N.P.
(40)
Semantic hierarchy Each of the P 16 items in the semantic hierarchy task was encoded as a random normal distributed vector
X X~N(u= 1/N,) R N.P.
(41)
Corresponding target vectors were then generated according to the position of the item the hierarchy. If an item is a child of a particular node is encoded as a and otherwise. For example, a pea is alive, not an object, not an animal, a plant, not handy, not earth, not a bird, not furry, but veg, not a tree... resulting in the target vector [1, 0, 0, 1, O, O, O, 0, O.....)
14

---

An analytical dissociation of functional and representational similarity in deep linear neural networks
A.3. Figure!
Random walk on MRNS manifold of random regression task with N; N₁ 7, No 3, and / 3 for 500 steps with diffusion parameter 0.005.
A.4. Figure
The neural network has N₂ 16, N₂ = 16, N, = 31. The point on the LSS solution manifold which depicts an elephant of E RNP.P xP, was found by first initialising the weight matrices as
W = Xx" + +5x 10+ and W₂ - (W/(W(X()))) +1x 10-
(42)
and subsequently applying gradient descent on both weight matrices to find a point on the solution manifold. Randomly sampled points on the solution manifold of MRNS and MWNS were generated according to the definitions of 2 in Theorem 3.5, and Theorem 3.7.
A.5. Figure
Random walk on solution manifold for GLS, LSS, MRNS, and MWNS on a random regression task with N; - 6, N₁ = 16, No = 2, and P = 4 for 100 steps.
(A) We fit linear model to predict the hidden-layer activations of one model by the hidden-layer activations of another for each time step of the random walk, using ridge regression and subsequently calculated the R2 score of that fit.
(B) We performed representational similarity analysis on pairwise random walk trajectories of (B) by caleulating the euclidean distance matrix of their respective hidden-layer representations
RSM.,
|lb; hyll2
(43)
and subsequently compared their off-diagonal values using Pearson correlation coefficients.
(C) analyse drift, we fit linear decoder on the hidden-layer representation of a model at the beginning of the random walk
W = Wf W??)
(44)
and subsequently calculated the mean squared error of that classifier, given the hidden-layer representation at each time step.
(D) We numerically calculated the average loss across n = 500000 randomly sampled input noise vectors with of = at each time step. Accordingly, we numerically calculated the average loss across n 500000 randomly sampled parameter noise matrices with of x/|| and o² 1/No.
A.6. Simulating nonlinear networks
As precursor to all numerical experiments with nonlinear networks in Section 6, we train feed-forward neural networks on non- non--onniiie con-synthetic dataset produce task-specific parametrisation. All neural networks in this section are trained with back-propagation and (mini)-batch gradient descent from small initial weights, a regime that induces task-specific feature learning in nonlinear networks similarly to linear networks (Chizat et al. 2019).
A.7. Expanding nonlinear networks
The first expansion (scaled in figures) rescales the input weights to each neuron by a constant factor a and the output weights the neuron by 1/a, which preserves the output of each neuron due to the sale-invariance of ReLUs. However, the magnitude of the representations that this model employs increases. The second expansion (nuisance sance in figures) adds nuisance neurons with random incoming weights and zero outgoing weights ("'errotyppe" neurons per Martinelli et al. 2023); similarly, these neurons do not affect the output the model, but introduce noise in hidden-layer representations. Lastly, we include a parameter-expan expanded bascline (duplic (itte)) that duplicates each neuron and correspondingly rescales outgoing weights from the duplicated neuron and its copy by factor of one half.
15

---

An analytical dissociation of functional and representational similarity in deep linear neural networks
A.8. Figure
The neural networks start with 784, N,. 1024, N. 10. Manipulations adding parameters in panel (E) add twice the number of neurons. Models trained for 30 epochs the full MNIST training set of 50,000 images with exponential learning rate decay.
B. Notation and preliminaries
Throughout this paper we adhere to the following notation: Scalars are denoted by letters (e.g., a, T, L), matrices by bold uppercase letters (e.g., X, Π), column vectors are denoted by bold lowercase letters (e.g., x, A), and row vectors by the transpose of column vector (e.g., x" )²). The vector dot product and vector outer product denoted by a b and ab? respectively. The zero matrix and identity matrix of size are denoted by and I, respectively. A and A denote the inverse and Moore-Pennee pseudo-inverse of matrix. The f--norm of vector is expressed by ||x||2 and the Frobenius norm of matrix is expressed by ||. Finally, the expected value and trace operator are denoted by (-) and Tr(-).
B.1. Compact singular value decomposition
We extensively use the cSVD analyse matrix structures. For any matrix A E R with rank the cSVD decomposes it into a product of three:
cSVD(A) = USVT,
(45)
where U E Rmxx and V E R are 'emmi-jorthonorms matrices containing the left and right singular vectors, and E Rexr is a diagonal matrix with corresponding non-zero singular values in descending order. In contrast to the full singular value decomposition, the singular vectors of the cSVD are not generally square orthogonal matrices but may be semi-orthogonal dependent on the relationship m, n, and r (Table 1). It further follows, that S 1 is well defined. Using that (BC)+ + C+B c+B+ has orthonormal columns or has orthonormal rows (Greville 1966) it further follows that
A (USV?)*
= V (US)*
(46)
VS* U
VS-
the Phooreeeenrose inverse of any matrix A.
Table 1: Orthonormality of singular vectors of the cSVD
m
>>
r
m
r m
p m
p=
n
n
U
Im
Im
= I,
I
I,
UU
Ln
* Im
= Im
* Im
= Im
* Im
V
T
Im
I,
In
VV
I
* I
* I
1 I
I
In
B.2. Linearity of expected value and trace
Both, the expected value and trace operator are linear and therefore additive
(A + B) (A) + (B),
(47)
Tr Tr + B) = Tr(A) Tr(B),
(48)
and homogeneous
(aA) a(A),
(49)
Tr(aA) aTr(A),
(50)
from which it further follows that
(Tr(A)) T+((A))-
(51)
16

---

An analytical dissociation of functional and representational similarity in deep linear neural networks
B.3. Expected values of random vectors and matrices
Let E R be vector whose entries are i.idd and drawn from zero-centred random distribution with variance of. Then,
( - R"
(52)
and the expected value of the inner and outer product is
< not and (51E)) of
(53)
respectively. Let E2 RM be second random i.i.d. vector sampled from zero-centred distribution with variance 02. then
< E2) = 0 and (E E R Rm
(54)
We continue by deriving general forms for the expected value of products of random matrices. Let E E Rm be a matrix whose entries are i..dd, drawn from zero-centred random distribution with variance of. Then,
(1) E R Rm*
(55)
Further, let En and E denote the i-th and j-th column of the matrix. Then it follows from Equation (53) that the expected value the inner product for each column of E is
= -0, 0,
if f otherwise
BTE,,)
(56)
mof
and therefore that
=
= mo
(57)
Similarly, from Equation (53) it follow that
0, ofI,
i
[,,
(58)
otherwise
and therefore that
[ -ot
(59)
Let Eg R xn be second random i.i.d. matrix, sampled from a zero-centred distribution with variance 03, then follows
from Appendix B.3 that
[ =0
(60)
We proceed by deriving equalities for the expected values of random matrices and their interactions with arbitrary constant matrices. For arbitrary constant matrix B E Rmx and i + j we get
E EE
) = Tr B ((
(61)
= TTr(BO)
=0
and for i
BE
Tr B (=,
ij
= Tr (Bof1)
(62)
or (B)
and therefore
< of Tr I.
(63)
17

---

An analytical dissociation of functional and representational similarity in deep linear neural networks
C. The general linear solution
From Theorem in Laurent and Brecht (2018) it follows that any minimum the convex and differentiable mean-squared eπror
1
MSE
Ix,, -n1l2
(64)
2P
corresponds to the global optimum the convex single-layer optimisation problem
1
LMSF.
Wx Xn. y112.
(65)
2P
as long as the underlying network
= = 12211
(66)
has no bottle-nece necks. The global optima of the network then correspond to
aLMse UW
Wx,,
yn) X
P
(67)
P
Σ
X
x
Σ
Yn
n=1
WE WEE = Ege.
Resubstitution then gives the general linear solution
22 Ege = Eny..
(68)
Finally, note that the GLS can also be written
S2222 Egg
Eyz
Ω252
Eyz x 2x
+ Z(I - Ene Ete)
(69)
Eyz Dt
+ +Z(I -
12s USV Z(I AA"),
where Z N.NNN is an arbitrary matrix.
Proof of Theorem 3.1. In the following, we derive complete parametrisation of the GLS
Ω25 USVT + Z(I -AA").
(70)
We begin by rewriting 22 in the basis of relevant, irrelevant and unobserved null directions
Ω = S.P, + 2.P, +.
(71)
with corresponding projectors P. VV P. AAT - VVT. and P. I AAT. Substitution into the GLS (Equation (69)) than reveals that directions that lie in the relevant input space must obey
Ss (2, + S.P. = (USVT + PPP,),.
(72)
P. USV
18

---

An analytical dissociation of functional and representational similarity in deep linear neural networks
from which it follows that S.,P. s2 VVT can be parametrised as QVSV?, where Q R.Nx xr is any full-column-rank matrix, projecting relevant inputs into hidden space. It then follows that can further separate P2pPn into two subspaces, one projecting relevant hidden representations and one projecting all other dimensions
Pr = USV
(2 + 222I QQ
QV (
USV
(73)
Q2QQ
USVT
Ω2Q UvS,
and thus Q2* can be parametrised as UVSQt. We call these two parts of the network function the core. We continue by analysing the subspace covered by irrelevant inputs
(2²QQ* + 52 QQ (2, + +
(USV* + PP (02QQ* + + 1 Q000-Q0)) 0
(74)
l2 QQ*
2220
This implies that task-irrelevant inputs can be mapped into the core readout space by input weights (right-hand side of the equation), provided that other iask-irreevvant components are simultaneously projected in such a way that they cancel the effect (left-hand side of the equation). This ensure that the network output remains unchanged despite the first-layer weights processing task irrelevant components. For the compensation the two-layer network to be successful, we have have
2 [ QQ + P.PP = 22=Q0
- - QQ (T QQ P. = Q2QQ
Q2 = -Ω2QQ 2 [1-QQ*)),]]]]
+ [ (I- 10--(IIQQ0)))) 100000000PP(00000P.._p (I- QQ+) (75)
-2- QQ+) = - Q2QQ 0, [I-QQ" [
+ 7 00 (I-HH QQ*)
22 - QQ Q2 SeQ++1.F P. [[I [I- QQP Z(I - HH
where Ż N.NN. is an arbitrary matrix. Crucially, the pseudo-inverse only exists if
im (QQQQ* P.P S im - QQ QQ+QQQ+)200 P,P (,P)).
(76)
As both sides live in the same subspace P,, this equivalent
rank(QQ P P;)
(77)
Or in words, there must be at least as many dimensions the irrelevant input space that are projected outside the core as there are dimensions that are projected into the core. Substitution then gives
2g - QQ*) = UU/SQ S,P., [ QQ [.P. + -(I-HHHI
(78)
Finally, we analyse the unoccupied null directions
2 2 = (USVT + ZZP,).
(79)
Ω2 P. ZP..
from which it follows that 22 Pu can be chosen arbitrarily. In summary, we then have
Ω S.P. S2,, P
(80)
QVSVT + +r, T2P.
where Fi, Dg E NNM, can be chose freely up to the constraint that rank QQ+T,P.) < rank((I -nnk(I-QQQ" r,)),, and
Ω2 Ω2QQ + + 22 - QQ*)
(81)
UVSQ** + v + +T HH+),
where T3 No is an arbitrary matrix and V -UVV=SUUQ S Q
19

---

An analytical dissociation of functional and representational similarity in deep linear neural networks
D. Partitioning of the solution manifold
Proof of Theorem 3.3. We use the method of Lagrange multipliers to minimise
argmin W2
[2 1>Will
(82)
under the constraint that
W W Eg = Eyz.
(83)
We begin by substituting Ω W, Then the Lagrangian is
L =
S2 P
Tr
A (NEEn. Eyz))
(84)
with gradients
0C
20 + AE 0
(85)
4>
Ω
A EEgry
and
OL
ΩEgS
Eye = 0.
(86)
0A
Then starting from Equation (85) we have
Ω
AΣ
2
ΩΣ
A AEE Da
2 1
Σ
A
AB A
(87)
2
P
P
Σ PAB 2 AT
1
A
T
yur
2 P
yz - AE gar
where in the third step we substituted Equation (86). Then, resubstitution into Equation (85) yields
Ω Rgn y (ETx)* ) USVT.
(88)
Next, we derive a complete parametrisation of the LSS
Ω252 = USV".
(89)
Again, as in the parametrisation for the GLS (Theorem 3.1), rewrite Zt in the basis of relevant, irrelevant and unobserved null directions
Ω S2P, S.P, P
(90)
with corresponding projectors P, VVT P; AAT VVT and P. = I AAT First, we note that the equations for the relevant and irrelevant input directions identical the ones presented in Theorem 3.1, that is
12 O.P. + P.P.P USV"P,
(91)
PM = USV
and (22QQ* + 22 - P, 2 USV P:
(92)
2g (o(I-Q0t)9, P2P -2:QQ++ S.P,
20

---

An analytical dissociation of functional and representational similarity in deep linear neural networks
From which it follows that
R,P, QVSV
(93)
Q,P; = T,P;
(94)
Q₂QQ + UvSQ* 1 - QQ*) )+=)== v + Z(I -H*))
(95)
(96)
where Q E N.xxr is arbitrary full-column-rank matrix, Tı, is an arbitrary matrix subject to the constraint Pi)). rank( (ank(QQ*tu (Q* PP) & rank((I (Qt)T,P) L = -UVSQ* P - arbitrary matrix. However, LSS differ in how they constrain the solution manifold in the case unobserved null directions. In particular, Z(I
and Z E RN., is an
HH+) is constrained by
02 P - (0₂QQ* + + =
US
(UVSQ* + HH P)
-1- HH, = (UVSQ++W)ON +
ž(I Z(I-HHI = (UVSQ*+9)01)) U) (
(97)
+ Z[I (I HI - J2 P HH+)
ž(I -
+ P. (H-
+ 2|HH (I (1-
Z(I 7 - P. [[I HHHHHHHHH 2(I 1 1,st).
where 2 E No is an arbitrary matrix. Again, for the pseudo-invenn inverse to exist we must have
(mH S2PPa ([ (HHt),,
(98)
As both sides live in the same subspace P.. this is equivalent to
rank 22 < ran HH* H P.,
(99)
In other words, at least as many unobserved dimensions that are projected into the occupied hidden-layer dimensions have to be projected into the unoccupied hidden-layer dimensions in order for correction to exist. In summary, we then have
Ω S.P. Q.P. + +.P.
(100)
QVSVT + +
where Fı, I2 E RN,,,,, XN, can be chose freely up to the constraints that rank (QQ < rank((I- (P)) and rank(HH's HHHHHHH*PP. & rank((I aH P.), and
22 Q2QQ 2 QQ+)
(101)
UVVSS**** + + 2,
where T3 E N.X,, is an arbitrary matrix, v = -U P and $ -UU/SQ + I L. HH HHH)),PP
..
Proof of Theorem 3.5. We use the method of Lagrange multipliers to minimise
argmin
| 2 + W2
(102)
W.
under the constraint that
W,W. Eyz.
(103)
21

---

An analytical dissociation of functional and representational similarity in deep linear neural networks
Then the Lagrangian is
C = || P W2/1 + Tr (A² (W2W, Ent Enzx)
(104)
with gradients
Oc
OW1 2W XXX" A -0
(105)
W
W A 2Z1
2
0C
2W, 2 + AΣ W =0
OW2
(106)
+
W2 = AΣ TH WT.
and
2C
W, W
Σge =0 0.
(107)
0A
Starting from Equation (106), we then have
1
W2
AΣ W
2
W W2
W AE gm
W
(108)
W W2 w, WT.
where in the last line we substituted Equation (105). Let
cSVD(W2) = ABC and cSVD(W,X)) = DEF
(109)
be the cSVD of the network weights, then
W = W
CBA CBATABC" ABC DEF" FEF" FED CB.C DE
(110)
Since C and D are (semi-)orthonorraal matrices and B and E are diagonal matrices with strictly positive entries it follows
that
C - D and B E + B
(111)
In the following we denote C and D as R, and B and E as G and write
W = RGF and W₂ AGRT.
(112)
where R is an arbitrary (semi-)onne Jothhoootthognnl matrix. Finally, let
<SVD(X)= = JKL"",
(113)
and
cSVD(YX' CSVD(YLKJ JK- =CSVD(YLL") = MNOT.
(114)
Then starting from Equation (107) we get
W E Ezz
WX YXX
W. W,W,JK2J YLK.JT
(115)
W JKL YLLT
AGR
RGFT AGR
T
T - T
22

---

An analytical dissociation of functional and representational similarity in deep linear neural networks
from which it follows that
A M, F = =0, and N
(116)
and therefore that
W2 = MvNR and
(117)
Finally, we rewrite
WiX = RVÑO"
WIJKL' = RVÑO"
W1 RV +Z (I- JJ")
(118)
Wi = R (I
where Z is an arbitrary matrix.
Proof of Theorem 3.7. We use the method of Lagrange multipliers to minimise
argmin
W Will + 2
(119)
W1.W2
under the constraint that
W2W,E Dzz.
(120)
Then the Lagrangian is
C=
W 1 + W21 + (1 (W2W,E,, Eyz))
(121)
with gradients
Oc
2W + WT AΣ FF =0
OW
(122)
1
4 W
W A.grs
2
OC
2W, + AΣ W =0
OW2
(123)
1 W₂
AΣ nm W
2
and
OC
W. W,E,N Egx
Eys =0
(124)
0A
Starting from Equation (123) we then have
W2 1
AE W
2
W W2 = W AE WT
1
(125)
2
W W. W, WT.
where in the last line we substituted Equation (122). Let
cSVD(W,) ABC" and cSVD(W,) DEF"
(126)
be the cSVD of the network weights, then
W. W.WT
CBAT CBATABC" DEF FED
(127)
CB.C. - DT
23

---

An analytical dissociation of functional and representational similarity in deep linear neural networks
Since C and D are (semi-Jorthonorraal matrices and B and E diagonal matrices with strictly positive entries it follows that
C = D and B E² + B E.
(128)
It further follows that W1 and W2 must be of identical rank. In the following denote C and D R, and B and E G
and write
W = RGF and W2 = AGR". where R is an arbitrary Jotthooorthogonal matrix. Finally, let
(129)
cSVD(E JKJ
JKJ
(130)
and
cSVD E = USVT
(131)
denote the cSVD of the input covariance matrix and least-squares solution. Then, starting from Equation (124) and using Equation (122), we get
W2W1 Exz = Eyz
W W2 AE Exe Enz
6 1
W W. AJKKJT E
2
W. WT WTAJJJ = -yzJK-1J JT
4
2
(132)
W.W A Eyz Et 2z
W.W, = USVT
AGR RGRT USV?
AG'F" USV"
4
from which it follows that and therefore that
A U, V and G = S + G=
(133)
W₂ SR and W1 RVSVT.
(134)
E. Hidden-layer representations
Proposition E.1. Let cSVD(X) = ABCT, then GLS are identical to LSS when operating on inputs X as any GLS (Equation (69)) can be written as
2352 = Eyz E
X + Z(1 A
0 = Eyz (EE))
X Z(I [[I-AA") ABC AAT)ABCT
(135)
aus X Enz (E,))
X + Z(ABC" ABC")
020, = Eyz (Enz) *X.
As a consequence, hidden-layer representations of the training data in GLS and LSS can be analysed by means of studying the LSS only.
Proof of Corollary 3.9. Following Proposition E.1 and given the parametrisation of Ω derived in Theorems 3.1 and 3.3,
we derive
RSM x
=X"(VV5Q' + P + (.TT' + +I,,,
(136)
=X"(VVSQ"Q VVSQ'T P.T.Q/SVV TPTTTTPPXX
24

---

An analytical dissociation of functional and representational similarity in deep linear neural networks
Proof of Corollary 3. Let
cSVD(X) = ABC
(137)
then given the subset of 12 as defined in Theorem 3.5 derive
x (' OVNR"' r²) (R) + r)x - OVR'RR/NO
CBAAA AB CC CB A ABC
(138)
-CC* CC
=ONO
where in the last step we used that
"X'X+"=XCO"
(139)
Proof of Corollary 3.12. Given the subset of 12 as defined in Theorem 3.7 we derive
XVV'SR'VVV''XXXX x
(140)
F. Secondary error
Proof of Theorem 5.1. Let the secondary error be denoted by
Q Σ
GMSE 2Q ==1
W2W, - -2
(141)
where input vectors & and corresponding target values yn, come from a secondary dataset D = Fn) with total of Q input-output pairs. We want to find the point on the solution manifold of the primary task such that GMSE is minimised. We therefore substitute the GLS (see Equation (69)) into the secondary error
Q
1
GMSE
Σ
yz E + ZP.) Rn 8n1
(142)
2Q
where Z E NoN.. is arbitrary matrix and P. -I AAT. Thus ZP, u. which describes all possible transformations of the network function that lie in the unoccupied input space of the primary task is the only degree of freedom. Thus, we have to find Z, which minimises the error. To this end, we define
cSVD(P,X) DEFT.
(143)
and continue with
Q Σ
0GMSE
((S It
ZP.) Xn -ÿn) x P.
0Z
Q
( Syz Σイ + iP..)
Ezz
S yz
P.
ŻP S m0
P
S 1/
Σ
I S
P.
yy
(144)
ŻP TP
Y
Egg yu
Σ) Fm
X P.
Y Σ ym
+
FEDT
$
2 Y Σ
FE
'DT + (DI)
İ Y Σ
(PxX) + + I(I-DD")
where I' E N.XN is an arbitrary matrix. Note that - Pu describes all possible transformations that lie the null spaces of both X and X, and therefore have no effect on the processing either the primary or secondary data.
25

---

An analytical dissociation of functional and representational similarity in deep linear neural networks
G. Input and parameter noise
Proof of Theorem 5.6. Let Ex,, denote vectors and E and E denote matrices whose entries are i.i.d. and drawn from a pero-centred random distribution with variance o on and oi respectively. Then, we can write the expected mean-squared error under additive input noise and parameter noise
(S2 + =2) (52) + =1) (xn + Ex.)) ynll
2P
1220ix yn (2g + W2 E, (W2E, + + 11 (145)
2P
1 (b"b).
a'aa
a
a
2a
(b)
2P
2P ==1
2P 1=1
In the following we solve each of the three summands independently. Using the definition of linear solution (Definition 2.3).
we derive
P
1
1
1
1
x
In x
yn
Σ vf
22
Vayn
2P
2P
2P
2P
2P
Tr(O 1252 E..) Tr(n 2
2 ym
+ Tr(
2
(146)
TW 22 Exz Exz)
+
Σ
Tr (yn))
um
2
1 Tr(E
Tr(E (iEEgi(Dnn ya
Σ
2
c,
which is noise-independent constant that only depends the statisties of the training data. Next, using the assumption that the noise is zero-centred, we derive that
P 2a
T (b)
=0
(147)
2P
n=1
using that
(b) - - + 02 + EzΩ, + +,3,, (5x.)) + (R23 +50, + =0.
(148)
Which leaves us with the third term, which we can write as
P
1 Σ (....) (..)),)) + (x.T(..) + ( ((..((.)))
(Tb))
(149)
2P 1=1
2P n=1
In the following, solve each of the four summands independently. We begin with
+ + 2 ( + 2/ [ (z)) + 22 (B,) )3:
(3, (E + ( nz 3 + (= ( ),)=,)
(=f) 2
(150)
+ 22 (=. 2ol 22 ( - + oof ( Ω + 27 [ (=,)
{ (=)) am + ( + (=}
(E
Ex..
where we used that noise is i.i.d.. Using that noise zero-centred, we note that summands 2-5, 7-10, and 12-15 resolve which leaves with summands 1, 6, 11, and 16 which we solve using Equations (53), (57), (59) and (63),
<x. o 2, Tr 2 ( (5x., 5x.))
(151)
= 1
26

---

An analytical dissociation of functional and representational similarity in deep linear neural networks
E 1 Ex.) =Tr Tr (( ) ( (xx. Exx)) of Tr (1 22 (S!ET)
(152)
of S22 12:
T .t (M ) 126xx
- Tr (nT ( < > an (5xaa
= o (R (S/
(153)
= =o,02Nol1 [ 11:
and
< =|
= Tr (<ST
( (5x, (k.)
- o ((=E() (
(154)
= =o Tr (IN.)
= N,3,NN
We continue by noting that
(5x, (...)x.)) =Tr (((..))x,,
(155)
and
(x.) = Tr (((.)))) )x,) = O.
(156)
Finally, we solve
x (= + (=() 12 + + (= (3a)=)) > + ( [ S2 (3) + (E si
(157)
Ω (,/ > (a.) + (5 (=)) n.S.) + ( (E/ st + (S X,.
where we again used that that noise is i.i.d.. Using that noise is zero-centred, we note that summands 2-4, and 6-8 are equal to O. Which leaves us with summands 1, 5, and 9, which we solve using Equations (57) and (63) as
x 2 x. of T+(2 12 x Ix, = =f 1/r xxxx,
(158)
x (=/ Z2 Noo2 Tr(s2 S2 22 x,xx),))
(159)
and
x
x - of ((E x
ofNoo Tr(1) Txxx,x))
(160)
No No Tr(x,x))
Resubstitution into Equation (149) then yields
(112222
11: + Of 22 1/ + 22 +
2P
2
(161)
1
22
2 X
+ 1 + NNo x P22),
2P
Substituting Equations (146), (147) and (161) back into Theorem 5.6 concludes the proof.
27

---

An analytical dissociation of functional and representational similarity in deep linear neural networks
Proof of Theorem 5.2. Let En E R.,, denote random vector with independent and identically distributed, zero-centred entries with variance o²- Then the expected mean squared error over the training data when adding such random perturbation to each input is
P
P
1
12222 (x. + En) - -nyl11112
X2 Xn -Yn + +22221 En
11
2P
2P
1=1
n=1
(162)
1
aaa
2a (b)
1
Σ (b²b)
2P
2P
2P
The first summand is identical to Equation (146), the second summand is
Σ 2a (b)
Σ
2 - 2222 (En)
2P
2P
(163)
A=1
and for the third summand, using Equations (53) and (57) get
P
P
(b'b) =
2P
2P
1=1
==1
22 22 1252 (En.5)
Tr
2P ==1
x Tr
(S2 2 2252.)
(164)
2P
==1
2
12
Substituting Equations (146), (163) and (164) back into Equation (162), then concludes the proof.
Proof of Theorem 5.4. The expected mean squared error the training data of linear solution (Definition 2.3) under additive, independent and identically distributed zero-centred parameter noise with variance of and o3 is
P
(O2 + 3²) (2) + + x.. - yv
X2 ya
+12 + E2s2xx + 3
13
2P
2P
n=1
==1
P
P
1
a
a
2a
(b) +
Σ (b*b)
(165)
2P
2P
2P
==1
==1
The first summand is again identical to Equation (146) and the second summand is
1
2a (b) 0
(166)
2P
since
(b) = (N,3, E,0, E,E,x,))
(167)
22 (3) x, 22 + (E23,) x, =
Which leaves us with the third term, which identical Equation (157) which equals
Σ (bTb)
1 Σ
1
x S2 I.I f + Noof
2P
2P
n=1
(168)
P2 1
Tr(E,z) + No Tr(22 2 E,n) + NooiNoo Tr (me))
=
Substituting Equations (146). (166) and (168) back into Equation (165), then concludes the proof.
28


Reviews:
[{"review_id": "r8RHdr6Urc", "reviewer": "Reviewer_tSLM", "summary": "This paper analytically studies the hidden representations of two-layer feedforward networks trained to minimize differentiable, convex loss functions. The only sets of weights in the networks studied are read-in and read-out weights, leading to simple expressions for both in terms of the input data. The paper shows that, even in this simple case, function can be dissociated from representation: two networks with different representations can achieve the same loss on the task. The paper then investigates the set of solutions obtained when optimizing the network under various regularization schemes, such as minimizing the sum of squares of the weight norms. These regularized networks, in fact, preserve similarity between one another when taking random walks through function space. Since these regularized networks are also more robust and generalize better, this suggests a mechanism for the empirically observed brain-model similarity \"in the wild.\"", "questions": "", "limitations": "", "rating": 5, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their thorough evaluation of the manuscript and supplementary material, and sincerely appreciate their effort and positive feedback! We would like to kindly ask the reviewer, if time permits, to elaborate on the specific strengths and significance of the work to help the Area Chair better understand the basis of their positive assessment.\n\nRegarding the $\\stackrel{!}{=}$ notation in the supplementary material, it was used to indicate that an equality is not derived but rather an assumed condition from which a separate conclusion is derived. However, since this notation is not widely used, we have replaced it with explicit language such as “such that”, “with the requirement that”, or “we would like to show” for clarity.\n\nPlease do not hesitate to let us know if there are any further points or suggestions to address.", "claims_and_evidence": "Yes.", "methods_and_evaluation_criteria": "Yes.", "theoretical_claims": "Yes, all of them.", "experimental_designs_or_analyses": "N/A.", "relation_to_broader_scientific_literature": "This paper will be of interest to many researchers working in both neuroscience and machine learning.", "essential_references_not_discussed": "No.", "other_strengths_and_weaknesses": "This is a very, very good paper. It is hard to find any weaknesses.", "other_comments_or_suggestions": "- In the supplementary, I think it would be useful to define the \"exclamation over equal sign\" in the notation and preliminaries section.", "questions_for_authors": "N/A.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "JdY8wFws8i", "reviewer": "Reviewer_rKc5", "summary": "The authors study a two-layer linear network. They characterize the space of solutions for such networks, with emphasis on several normalization schemes. The result is that there are many zero-loss solutions that differ in how minimal they are. Specifically, whether the transformation from input to hidden or from hidden to output is minimal with respect to the training data.\nThe authors discuss possible implications of these results to representational drift and to the comparison of biological and artificial networks.\n\n## After rebuttal\n\nAfter reading all the rebuttals and discussion with all reviewers, I am keeping my score.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their thorough evaluation of our work, and for providing explicit feedback!\n\nFirst, we address the reviewer’s concern about the novelty of our analysis. To clarify this, we have added a dedicated \"Contributions\" section to the revised manuscript and reproduce it here in detail. As stated in the manuscript, deep linear networks perform “…multistage computations that give rise to hidden-layer representations.” Their overparameterization through depth, rather than width as in linear regression, introduces nontrivial representational degeneracies not present in linear regression. Our work provides a complete analytical characterization of these degeneracies and their implications for decodability and comparability of neural representations. Further, we study the consequences of our analysis on use cases in neuroscience, where neural networks are a commonly employed model of learning, in Section 4.\n\nIn particular:\n- Definition 3.1 directly follows from Laurent and Brecht (2018).\n- Theorem 3.3 resembles least-squares linear regression, but its derivation in the context of two-layer linear networks requires extra care (e.g., does not hold if the network is bottlenecked).\n- The constraint satisfaction problem in Definition 3.4 has been studied in Saxe et al. (2019), but only under a set of strong assumptions ($\\Sigma_{xx} = I$, $N_i = N_o$, and that $\\Sigma_{yx}$ has full rank) as stated in the manuscript.\n- In contrast, Theorem 3.5 imposes no additional assumptions about the task's statistics or network structure (beyond the network not being bottlenecked).\n- Definition 3.6 stems from our analysis of parameter-noise robustness (Section 5), but can be related to notions from implicit regularisation (e.g., [Smith et al., 2021](https://arxiv.org/abs/2101.12176)) and neural collapse (e.g., [Zhu et al., 2021](https://papers.nips.cc/paper/2021/file/f92586a25bb3145facd64ab20fd554ff-Paper.pdf)).\n- Accordingly, Theorem 3.7, which shows the parameterization of minimum representation-norm solutions, is novel to the best of our knowledge.\n- Theorems 3.8 and Corollaries 3.9–3.11 are novel and central to our analysis, analytically revealing the degrees of freedom within neural representations and defining when representational similarities are fixed and can be used for functional comparison.\n- For theorems and corollaries regarding robustness to noise in Section 5, similar results exist for linear regression (e.g., minimal norm confers noise-robustness) but again require careful treatment in the multi-layer setting (e.g., cross-correlation terms in the case of parameter noise).\n\nIn summary, to our knowledge, this is the first work to make an analytical and exact connection between different regions of the solution manifold of deep linear networks and the identifiability and comparability of neural representations, as well as their correspondence to optimality in noise robustness. If the reviewer is aware of any further prior work that has similar results, we would be delighted to include it and provide appropriate reference.\n\nMinor:\n\nWe would like to thank the reviewer for catching the typo in Assumption 2.2. Indeed, the greater-than-or-equal sign should have been a less-than-or-equal sign!\n\nRegarding terminology, we made an effort to be precise while avoiding overly lengthy terms like \"minimum-readout-and-representation-norm\" . We recognize that \"minimum-representation-norm\" is only partially descriptive, as it omits the fact that we are also minimizing the norm of the readout weights. While minimising the norm of the readout-weights has no influence on the hidden-layer representation, we chose the constraint satisfaction problems in Definitions 3.2, 3.4 and 3.6, such that they align with the analytical results on noise robustness in Section 5. We believe our terminology is sufficiently descriptive but would be grateful for any suggestions the reviewer may have for a more concise naming convention.\n\nRegarding Corollary 5.4, the parameter noise is scaled by the norm of the inputs and the size of the output layer (see the first and second summands in Equation 22). By inversely scaling the variance of the noise, we ensure the equation is factored consistently, independent of these measures. Without this adjustment, the solution would be identical up to a fixed scaling factor. Thus, this scaling is simply a matter of convenience. We have added a corresponding comment to the manuscript.\n\nRegarding the scaling in Figure 5E, the initial phase of the sigmoidal curve corresponds to test error (near-zero noise), while the ceiling aligns with random guessing (high noise). We lack analytical insights into the shape of this curve due to the non-linear setting. However, we would be happy to include additional simulations if the reviewer has a specific question in mind.\n\nAgain, we would like to thank the reviewer for their time and effort, and we remain open to any further suggestions or points for clarification.", "claims_and_evidence": "Yes. There is degeneracy in solutions, and different regularization schemes choose different solutions. There is also a relation between noise robustness and regularization.\nIt is not clear which of these claims is novel.", "methods_and_evaluation_criteria": "Not relevant", "theoretical_claims": "I read the proofs, but did not verify their correctness in detail.", "experimental_designs_or_analyses": "Figure 5: the definitions of scaled, nuisance, etc only appear in the appendices. This seems like something that should be in the main text.", "relation_to_broader_scientific_literature": "The authors mention the main relevant papers (Baldi & Hornik 1989, Laurent & Brecht 2018). It is not clear which aspects of the current paper are missing from prior work.", "essential_references_not_discussed": "Not aware", "other_strengths_and_weaknesses": "Strengths:\nThis is a simple setting, in which the characterization of the solution space is possible. Qualitative insights from this setting can be useful in broader scenarios.\nThe explanation of the different degeneracies (Figure 2) was very clear and intuitive.\n\nWeaknesses:\nIt is hard to understand what exactly is new relative to existing work. For instance, Theorem 5.1 seems like a textbook result on linear regression. \nThe definition of minimum representation-norm is not very intuitive. The first term is the norm of the hidden representation. But the definition also has a sum with the readout weights. Why call this sum “representation-norm”?\nFigure 5E: It would be useful to discuss the scaling of the effect. Can the theory provide any insights on the actual values of noise in which the different models degrade?\nCorrolary 5.4: What is the intuition behind the suggested scaling?\nLine 92 – The way assumption 2.2 is written is quite confusing. It seems as though not having a bottleneck implies narrow hidden layer.", "other_comments_or_suggestions": "None", "questions_for_authors": "None", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "KMkYQCH3gE", "reviewer": "Reviewer_xx3g", "summary": "The paper presents a mathematical analysis of the nature of solutions (for a given problem) in an overparameterised two layer linear neural network.  This is done through a theoretical study of the manifold of generic solutions, i.e. different choices of weights values which give the same fit of the training data, and identifying several types of solutions grouped by certain mathematical properties.  This leads to a conclusion that analysis of overparametrised networks is challenging, because individual transformations/internal representation is arbitrary given the next transformation can correct for it, and thus the entirety of the mapping needs to be taken into account.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We would like to sincerely thank the reviewer for their time and detailed feedback. Below, we address the key points raised in the review.\n\n\nThe reviewer asks whether our analysis provides insights beyond the conclusion that internal representations cannot be interpreted in isolation. First, we emphasize that this is _not_ widely understood, as many subfields of neuroscience, machine learning and their intersections _do center on analysis of representations in isolation_. For example, as we outline in great detail in Section 4, a significant body of research in computational neuroscience assumes that internal representations between systems (models and/or brains) can be meaningfully decoded and compared. Our work analytically demonstrates that this assumption does not necessarily hold—in particular, we identify in which regimes representations are arbitrary with respect to the task, and instead dependent on other parameters like parameter initialization. \n\nHowever, this is only one part of our findings. We further identify specific regimes on the solution manifold where representational similarities are _not_ arbitrary. We derive precise analytical conditions under which internal representations in fact _are_ informative and comparable _without_ requiring knowledge of other parts of the network; in particular, when they are _task-specific_. Additionally, we show that these task-specific stable representations coincide with regions of the solution manifold that are robust to noise, linking identifiable representational structure to a desirable computational property. This goes beyond the claim that “understanding internal representation is a hard problem” because we establish precisely _when_ it is hard.\n\nIt is not clear to us why the reviewer dismisses these results as “intuitive”, “preliminary”, and “not novel”. To ensure a constructive discussion, we kindly ask the reviewer to provide references to prior work that establishes the same conclusions as we do in the manuscript.\n\nAdditionally, to clarify our contributions, we have revised the manuscript by adding an explicit contributions section that is detailed in the response to Reviewer rKc5.\n\nMinor:\n\nWe would like to thank the reviewer for catching the typo in Assumption 2.2. Indeed, the greater-than-or-equal sign should have been a less-than-or-equal sign!\n\nWe appreciate the reviewer’s explicit feedback on Figure 2. Our visualizations follow well-established standards found in mathematical visualizations (e.g., the [Wikipedia article on “Kernel (linear algebra)”](https://en.wikipedia.org/wiki/Kernel_(linear_algebra))). However, we are happy to improve and expand on the explanations of the image and kernel of a matrix in the main text. Further, we have revised the figure caption and expanded the accompanying explanations to explicitly clarify that: (1) the orange areas represent the general vector space in which representations reside, (2) open black circles denote the image of matrices, with the surrounding area corresponding to their kernel, and (3) filled black circles indicate the subspaces occupied by the input, hidden, and output representations of the training data, respectively. We welcome any further suggestions for improving the clarity of our visualizations.\n\nFurther, the reviewer notes that they did not carefully review the supplementary material due to its mathematical density, but still conclude that the results “might be sound” as they do not disagree with the paper’s results and conclusion. This aligns precisely with [ICML guidelines](https://icml.cc/Conferences/2025/ReviewerInstructions), which state that reviewers are encouraged (but not required) to consult supplementary material and that key claims should be understandable from the main text. Furthermore, rigorous mathematical derivations are essential for theoretical contributions in machine learning, just as extensive simulations are standard for empirical work. Appendices that extend over many pages are common in ICML theory papers, and theory of machine learning and applications to (neuro)science are subject areas in [ICML’s yearly call for papers](https://icml.cc/Conferences/2025/CallForPapers). Throughout our 8-page supplement, we have made substantial efforts to fully provide all our assumptions, proofs and derivations to ensure our results are fully reproducible, testable, and clearly presented.\n\nFinally, we sincerely thank the reviewer again for their time and effort. We hope that, in light of our clarifications, the significance of our contributions and the value of our thorough theoretical analysis (including the detailed appendix) become evident. We remain open to any further suggestions or points for clarification and are hopeful that the reviewer may reconsider their assessment accordingly.", "claims_and_evidence": "Yes.  The paper is mathematically dense, but the conclusions seem justified.  It's just that I think most would take it as a given, without the presented evidence, that overparamterised networks are non-trivial due to the fact that what individual components do is not informative, and instead one has to take into account the entirety of the mapping.", "methods_and_evaluation_criteria": "Linear network analysis is one of tools for anlysing neural networks, and while it has its limits, for obvious reason of missing the non-linearity aspects, it's still tells us something.  I do find the connection made in this paper to the non-linear networks by stating that they also have multiple solutions (through symmetries) somewhat weak - not in accuracy - sure, non-linear network offer multiple solutions - but in relevance of this work to that aspect - do the categories of solutions analysed here tell us anything about how non-linear solutions work?", "theoretical_claims": "I did not check the proofs carefully.  I think it might be sound, since I don't disagree with the ultimate conclusion of the paper.", "experimental_designs_or_analyses": "Experimental design and analysis seems fine.  I find the arbitrary 'elephant' internal representation a \"cute\" illustrative example.   However, though I applaud authors' attempt to give a high-level/intuitive explanation of what different solutions entail (in the paragraph after Theorem 3.7) I am struggling to understand those explanations and I don't find Figure 2, while again appreciated as an attempt at intuitive explanation, all that helpful.", "relation_to_broader_scientific_literature": "This work continues the thread of linear network analysis - an analysis of simplified problem (linear netowk) in hope of understanding a hard problem (the non-linear counterpart).  I think this work might be a great starting point of an interesting analysis, just at this moment seems preliminary, since it only that analysis leads to a trivial conclusions - that understanding internal representation is a hard problem.  We already know that.", "essential_references_not_discussed": "Not to my knowledge.", "other_strengths_and_weaknesses": "Seems like early work, analysis might be promising, but the current conclusion seems obvious and not new.", "other_comments_or_suggestions": "The equation in Assumption 2.2 seems inverted.  If the network is not bottle-necked then it seems that $N_h \\ge min(N_i,N_o)$, not  $N_h \\le \\min(N_i,N_o)$.  In Laurent and Brecht this condition is the \"thinnest layer is either input or outut\", which means $N_h$ is at least the same or greater than input/ouput, not smaller or the same.  Not sure if this is misunerstaning on the part of the authors, or just a typo.", "questions_for_authors": "Does the analysis you provide tell us anything else other than it is hard to analyse parts of internal representation in isolation from other parts of the network?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "9vx2VI5vUS", "reviewer": "Reviewer_1NRJ", "summary": "This paper studies the relation between functional similarity and representational similarity, finding that there is a disassociation: i.e., functional equivalent networks may have very different representations. To be concrete, they mainly study a two-layer linear networks, concerning their weights $W_1, W_2$ and hidden representations $h$. They study the manifold of least-square solutions, and minimum-weight-norm and minimum-repr-norm solutions. These norms show different hidden representations and representational similarity matrices, even though they all have the same overall function. Least square solutions are the most flexible -- in fact, representations can be arbitrarily manipulated (e.g., into an elephant). For the minimum-weight-norm or minimum-repr-norm solutions, the representations still have some degree of freedom, but RSM is uniquely determined from task data, which is desirable. They finally study why there is representation alignment between biological and artificial systems -- due to their robustness to noise.\n\n## Update after rebuttal\nThe authors have addressed my questions. Overall I think this is a great paper, but 5 is a bit of a stretch, so I keep my score as 4.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their constructive and detailed feedback on the manuscript.\n\nFirst, we address the reviewer's question regarding the optimization reasons behind an artificial network seeking robust solutions. Our analysis is general—we study the entire solution manifold and derive broad statements about neural computations and their representations without relying on any specific learning or optimization algorithm. An investigation into why and when an algorithm converges to a particular region of the solution manifold is beyond the scope of our study. However, as noted by the reviewer, extensive work in the machine learning literature has explored this question, albeit without focusing on their relation to neural representations. We have now expanded our Discussion to address this important aspect in more detail, including relevant references concerning implicit regularisation (e.g., Smith et al., 2021, as suggested) and neural collapse (e.g., [Zhu et al., 2021](https://papers.nips.cc/paper/2021/file/f92586a25bb3145facd64ab20fd554ff-Paper.pdf)).\n\nNext, we address the reviewer's second question regarding the visualizations in Figure 2. Our aim was to follow well-established standard representations found in the literature (e.g., the [Wikipedia article on “Kernel (linear algebra)”](https://en.wikipedia.org/wiki/Kernel_(linear_algebra))). Nevertheless, we recognize the need for additional clarity. We have revised the figure caption and expanded the accompanying explanations to explicitly clarify that: (1) the orange areas represent the general vector space in which representations reside, (2) open black circles denote the image of linear transforms, with the surrounding area corresponding to their kernel, and (3) filled black circles indicate the subspaces occupied by the input, hidden, and output representations of the training data, respectively. We welcome any further suggestions for improving the clarity of our visualizations.\n\nWe appreciate the reviewer's suggestion for additional experiments. Could the reviewer specify which experiments they believe would most effectively strengthen our paper’s message? We are open to incorporating further empirical evaluations, even if complex, to reinforce the practical implications of our theoretical findings.\n\nFurthermore, as suggested, we have tempered our claims regarding noise as the cause of representational alignment between artificial and biological systems, instead framing them as hypotheses that open up an interesting field of investigation. In particular, it remains unclear if, why, and when brains may operate in these regimes. However, we emphasise that our results suggest that assuming they do *without* empirical validation could lead to misleading comparisons and conclusions about decodability and representational similarity.\n\nWe have also made the following revisions based on the reviewer’s comments:\n- Corrected the typo in line 152.\n- Clarified in the caption of Figure 3c that the elephant is deliberately engineered.\n- Expanded the caption of Figure 5 to explain that \"duplicate\" refers to networks extended by duplicating hidden neurons, a transformation that preserves the input-output mapping.\n\nWe once again thank the reviewer for their time and valuable insights. We remain open to further suggestions, particularly regarding simulation studies that could further illustrate the scope and significance of our contributions.", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "Yes", "theoretical_claims": "I skimmed through the theorems (although I didn't derive by hand myself) and did not spot any error.", "experimental_designs_or_analyses": "This is mainly a theory paper. The experiments are minimal but are very informative and nicely designed.", "relation_to_broader_scientific_literature": "Although training of deep linear networks can be analytically characterized, the solution space seems trivial to characterize, this paper takes a unique point -- by constraining the norm of representations/weights, hidden representations can be more interpretable or more truthful to the underlying structure of data. This paper uses both mathematical derivations and neat illustration examples to support this idea.", "essential_references_not_discussed": "The paper mentioned implicit regularizations in deep learning. It would be nice to mention the implications regularization of SGD:\n* ON THE ORIGIN OF IMPLICIT REGULARIZATION IN STOCHASTIC GRADIENT DESCENT, ICLR 2021", "other_strengths_and_weaknesses": "**Strengths**\n* The paper is nicely written and neatly presented. I like the elephant plot to show that representations can be arbitrarily engineered given the same functionality.\n* The idea is rigorously supported by mathematical derivations.\n* The minimum-norm (weight or representation) solutions are interesting objects to identify and study. In particular, their representations reveal the truthful hidden structure of data\n\n**Weaknesses**\n* This is probably too much of an ask for a theory paper, but adding more experiments would make the paper more appealing to practitioners. \n* The representation alignment between artificial vs biological systems due to robustness to noise is interesting but might be over-stated.", "other_comments_or_suggestions": "* Line 152, a comma out of place\n* Figure 3c. It would be nice to point out (in the caption) that the elephant is deliberately engineered. I was a bit confused at first because there is no elephant in the 16 items. \n* Figure 5 caption did not explain what \"duplicate\" means.", "questions_for_authors": "* what's the dynamic (optimization) reason that an (artificial) network would seek robust solutions? Is it due to the implicit regularization of optimizers?\n* I'm a bit confused by the illustration in Figure 1. When a smaller circle is contained inside a larger circle, what does this mean? For example in subfigure D, the grey oval H is contained in the larger orange oval which is connected to W2.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: The paper presents an analytical treatment of how hidden representations in two-layer linear neural networks are related to the network's function. The authors show that even under very simple conditions (linear net, convex loss) function and representation can be dissociated. They also show that certain regularization schemes preserve similarity of networks, suggesting a mechanism for why empirical results find similarities between brain representations and those in neural network models trained on tasks.

Overall, the reviewers applaud the conceptual framework and praise the mathematical rigour of the paper. They find the paper well written and the results intuitive. There were no major criticisms raised. Hence I strongly recommend accepting the paper, which I believe will make a great contribution to ICML.
Decision: Accept (spotlight poster)