Title: Bridging Critical Gaps in Convergent Learning: How Representational Alignment Evolves Across Layers, Training, and Distribution Shifts
Venue: NeurIPS 2025
Accepted: yes
Keywords: Artificial Intelligence

Machine Learning

Representational Convergence
Abstract: Understanding convergent learning---the degree to which independently trained neural systems---whether multiple artificial networks or brains and models---arrive at similar internal representations---is crucial for both neuroscience and AI. Yet, the literature remains narrow in scope---typically examining just a handful of models with one dataset, relying on one alignment metric, and evaluating networks at a single post-training checkpoint. We present a large-scale audit of convergent learning, spanning dozens of vision models and thousands of layer-pair comparisons, to close these long-standing gaps.  First, we pit three alignment families against one another---linear regression (affine-invariant), orthogonal Procrustes (rotation-/reflection-invariant), and permutation/soft-matching (unit-order-invariant). We find that orthogonal transformations align representations nearly as effectively as more flexible linear ones, and although permutation scores are lower, they significantly exceed chance, indicating a privileged representational basis. Tracking convergence throughout training further shows that nearly all eventual alignment crystallizes within the first epoch---well before accuracy plateaus---indicating it is largely driven by shared input statistics and architectural biases, not by the final task solution. Finally, when models are challenged with a battery of out-of-distribution images, early layers remain tightly aligned, whereas deeper layers diverge in proportion to the distribution shift. These findings fill critical gaps in our understanding of representational convergence, with implications for neuroscience and AI.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper investigates convergent learning within neural networks. Here, inter-architectural similarity is evaluated across depth (layers) and convergence (different checkpoints across training), and metric (Procrustes, soft-matching, and linear regression).\n\nThey present a number of findings: 1) While the most flexible, linear regression shows only a minor increase in alignment in all situations (across depth / convergence) over orthogonal Procrustes, suggesting  2) Across most scenarios, representation similarity almost fully manifests within the first epoch of training 3) OOD behavior is observed mostly within the later layers of the network 4) Unlike CNN's, vision transformers do not seem to have a \"privileged axis\" where different instantiations of the same architecture converge to similar representations.\n\nA few side experiments are run, with results such as: a) representation similarity between humans in the visual cortex decrease as information propagate through the visual pathway b) the similarity reported by soft-matching is often decreased when one network's activations are put through an orthogonal transformation, suggesting that actual axes along which information is computed and stored is retained across architectures c) much of their observations for supervised models are mirrored in self-supervised models.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths\n- No complaints about clarity and quality of writing. The paper was easy to follow throughout. \n- In terms of experiments, the specific ones ran are reasonable ones to do to answer their questions of how representation similarity across architectures evolve with a) layer depth (Section 4.1) b) with training time (Section 4.2) c) distributions shifts (Section 4.3). \n- I liked seeing evaluation of across layers, training time, and dataset all within one paper.\n- Large-scale evaluation (across models and layers) provide broad empirical results.\n\n## Weaknesses\n- Choice of center activation: I think choosing the channels at the center of the networks representation might be a problem. I certainly understand the motivation, the activation maps of a CNN--especially early in the network--can be *extremely* large, computing similarity across thousands of layer-layer combinations might well be intractable without some way of reducing dimensionality. But in many cases it may be a massive reduction in the fidelity of the representation. As an example, I believe 224x224x64 is a reasonable early activation map for a ResNet50; if we take the center channel we retain about 0.002% of the data. From this tiny window into the network, strong, generalized claims about the convergent behavior of neural networks are made. The inductive biases of CNN's are all about translation equivariance, local feature detection, and hierarchical feature computation, but it seems like most of that would be discarded by taking the center channel. This may be reasonable choice early in the network layers, but does it hold higher up? [https://arxiv.org/abs/2110.14739](https://arxiv.org/abs/2110.14739) is cited as a possible justification for this, but my reading (e.g., Section 2.4) of this paper doesn't seem to support this conclusion. Further, this paper uses the entire activation map (though it is reshaped for computational convenience).\n- The inductive biases of ViTs seem to encourage entirely distinct modes of computation compared to CNNs, it feels like an oversight to not include analysis across CNNs and ViTs.\n- Novelty: Prior work has explored layer wise similarity and early training dynamics. Methodological novelty is limited, centering mostly on the three axis analysis.\n\nMy major concerns are the use of the center activation and novelty. If use of only the center activate is assumed to be an appropriate choice, I largely have no issues with the methodological approach and experimental results.\n\nQUESTIONS:\n- Can you provide justification for taking the center values of the activation maps outside of computational efficiency? I'm quite concerned about how strong of claims can be made with so much of the representation thrown away.\n- How does similarity change if an arbitrary spatial location is chosen for the activation, rather than the center?\n- Would it be possible to evaluate similarity in a limited set of models / layers using the full activation map, compared to just the center channel, to understand the effect of that decision?\n- I may be misrepresenting the novelty in my review, can you state what distinguishes this work from previous?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper provides a large-scale audit of convergence learning, focusing on three alignment metrics and disentangling the influence of layer position, training phase, and distribution shifts. The main paper provides results for convolutional networks, whereas more networks (self-supervised vision models, vision transformers, and language models) are deferred to the appendix.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths:**\n- The motivation for the paper is clear, and the problem it investigates is important. Thorough studies comparing multiple variables and not just one method evaluated on one dataset can provide deeper insights, objectively compare the methods, and isolate the key factors.\n- The methods are clearly described, and the number of experiments conducted (including the ones in the appendix) is sufficient. All claims and conclusions are supported by the provided figures and tables.\n- The results are put into the context of relevant literature, the authors comment on them well, and provide possible explanations.\n\n**Weaknesses:**\n- I think the biggest weakness is how the results are organized and presented.\n\t- For instance, the main paper contains results only for convolutional models, whereas self-supervised vision and language models are deferred to the appendix. I think the paper could benefit from reorganizing. The key results and conclusions holding across model types could be in the main paper, and the detailed results could be moved to the appendix.\n\t- It is a bit hard to identify the key findings (partially because there is no conclusion section). I found many interesting findings and claims in the results section, but it seems to me that they are not highlighted enough in the other sections, so a reader might easily overlook them unless they read the whole paper carefully.\n- Figures should contain all the information needed to understand them (for example, what each row and column means). It is described in the captions, but one has to read the whole paragraph to find all the information, for instance, the data it was evaluated on.\n-  I struggle to fully understand section 4.3. Some of my questions might just need a clearer explanation, and some are challenging the whole experimental design.\n\t- If I understand it correctly, you use only the Procrustes metric in this part. Why not all three of them?\n\t- In Figure 5, do you compare only the same layers (at the same depth)? Why not all of them? Could it be the case that the networks process the OOD data slightly differently from the WD data, so some of the processing might actually happen at different layers? Also, just from the figure, the difference does not seem to be that big. Did you test the significance of the results?\n\t- I do not understand Figure 6. In a), what is the task performance? Is it layer-specific, or (as I guess) one number for the whole network? It is slightly confusing to plot one information twice. How exactly is b) computed?\n\t- You claim that \"... one could plausibly improve OOD generalisation by fine-tuning only the later layers.\" and \"...OOD stimuli could be especially useful to distinguish and select between models whose representations closely mirror the brain.\" Could you provide more details about this? Have you (or some other work) investigated that?\n\nQUESTIONS:\n1) What are the key findings that are shared across all model types? Are there any big differences?\n2) Can you address some of the questions regarding section 4.3 (see the weaknesses).\n3) In Table 1, why do you report min and max alignment scores, and not, for instance, mean and standard deviation? Do you think it better summarizes the results? To me, it feels like I am missing some important information to fully understand what the results look like.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-30"}

comments: SUMMARY:
This paper aims to understand convergent learning and proposes a large-scale audit for it.  The authors conduct an extensive experiment, which demonstrates the usefulness of the proposed method.

STRENGTHS AND WEAKNESSES:
**Strengths**

1. This paper proposes a comparative method of characterization from three perspectives: layer, training, and distribution shift.

2. Real data analysis demonstrated the impact of the new methods.

**Weaknesses**

1. The main concern of the paper is a presentation. In many places, the meaning of sentences is vague. It is better to elaborate on it and make the sentence shorter.

2. The paper involves numerous symbols and matrix operations, but some symbols lack clear explanations. It is recommended to define each symbol's meaning the first time it appears to avoid ambiguity.

3. All formulas in the article lack numbers, which makes it very inconvenient to quote and explain.

4. The motivation for the proposed method is not well explained, and the novelty of the method seems to be underdeveloped.

QUESTIONS:
1. What is the specific expression of the $\texttt{corr}$ formula in Line 134?

2. The symbol $\mathcal{O}$ is used repeatedly in Line 163, with different meanings in different places (Line 129).

3. The method part seems to lack a lot of core formula expressions. Should you migrate them from the problem statement?

4. The legend font in all the figures is too small.

5. Is there enough theory to support the effectiveness of the proposed method? Specific analysis is needed.

LIMITATIONS:
yes
TITLE: Official Review
CLARITY: 2
IMPACT: 2
ORIGINALITY: 2
DATE: 2025-07-03

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work seeks to understand how representations in various neural networks converge according to various metrics across datasets, architectures, throughout training, and in out of distribution settings. Through a large scale study, authors reveal that 1) representational convergence occurs even if the mapping function is quite restricted 2) convergence occurs extremely early in training, 3) that OOD stimuli disrupt representational convergence in later layers and 4) convergence occurs even between different CNN architectures.\n\nSTRENGTHS AND WEAKNESSES:\nThis study is quite extensive and thoughtful, revealing several notable findings that challenge prevailing wisdom regarding convergence. Most notable among them is that representations tend to converge within the first epoch of training. That said, there are a few opportunities to improve the work, and some sanity checks that might be needed to ensure their validity.\n\nFirst, it seems plausible that the correlation between representations could be driven by the presence of rogue/outlier dimensions (https://arxiv.org/abs/2310.17715). This could especially be the case with the permutation-based alignment score. Outlier dimensions are thought to encode generic task information in extremely high variance dimensions, and so it is possible that such dimensions always encode approximately the same basic information across datasets/architectures/etc. As I understand it, the correlation score is presently simply Pearson correlation, which is susceptible to such outliers. A simple sanity check is to replace Pearson correlation with Spearman correlation, and ensure that all results still hold. \nAdditionally, the results from Figure 1 should include a baseline randomly initialized model to contextualize the observed trends.\n\nWriting: \nThe Difference (%) column in table 1 could be more explicitly described in text.\n\nThe neuroscience framing seems largely unnecessary in this paper, and could very well be distracting to readers unfamiliar with current paradigms in that field. While the compneuro community might benefit from this work, it is unclear that framing paper around the neuroscientific applications is more compelling than immediately (i.e., in the intro) putting this work in conversation with the debates surrounding the platonic representation hypothesis in AI.\n\nQUESTIONS:\nAdd Spearman correlation to sanity check results.\n\nAdd randomly initialized baseline models.\n\nWhat produces basis alignment in CNNs as opposed to transformers?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-03"}