Model–Behavior Alignment under Flexible Evaluation: When the Best-Fitting Model Isn’t the Right One
Authors: Itamar Avitan
Tal Golan
Venue: NeurIPS 2025 poster ()
Keywords: Representational Alignment; Model Recovery; Deep Neural Networks; Similarity Judgments;Representational Geometry;Cognitve Computational modeling
Primary area: neuroscience_and_cognitive_science

Abstract: Linearly transforming stimulus representations of deep neural networks yields
high-performing models of behavioral and neural responses to complex stimuli.
But does the test accuracy of such predictions identify genuine representational
alignment? We addressed this question through a large-scale model-recovery study.
Twenty diverse vision models were linearly aligned to 4.5 million behavioral judg-
ments from the THINGS odd-one-out dataset and calibrated to reproduce human
response variability. For each model in turn, we sampled synthetic responses
from its probabilistic predictions, fitted all candidate models to the synthetic data,
and tested whether the data-generating model would re-emerge as the best predictor of the simulated data. Model recovery accuracy improved with training-set
size but plateaued below 80%, even at millions of simulated trials. Regression
analyses linked misidentification primarily to shifts in representational geometry
induced by the linear transformation, as well as to the effective dimensionality
of the transformed features. These findings demonstrate that, even with massive
behavioral data, overly flexible alignment metrics may fail to guide us toward artificial representations that are genuinely more human-aligned. Model comparison
experiments must be designed to balance the trade-off between predictive accuracy
and identifiability—ensuring that the best-fitting model is also the right one.

Model-Behavior Alignment under Flexible Evaluation: When the Best-Fitting Model Isn't the Right One
Itamar Avitan 1.2.3 Tal Golan 1.233
Department of Industrial Engineering and Management Data Science Research Center
School Brain Sciences and Cognition Ben-Gurion University of the Negev avi avitanit&post brgppostt.uu..... bgu.bc...... ac. golan newroobgur..u....mm ic
Abstract
Linearly transforming stimulus representations of deep neural networks yields high-performing models of behavioral and neural responses to complex stimuli. But does the test accuracy of such predictions identify genuine representational alignment? We addressed this question through large-scale model-recovery study. Twenty diverse vision models were linearly aligned to 4.5 million behavioral judg- ments from the THINGS odd-one-out dataset and calibrated to reproduce human response variability. For each model in turn, we sampled synthetic responses from its probabilistic predictions, fitted all candidate models to the synthetic data, and tested whether the Kata-generating model would re-emerge as the best predic- tor of the simulated data. Model recovery accuracy improved with training- set size but plateaued below 80%, even at millions of simulated trials. Regression analyses linked misidentification primarily shifts in representational geometry induced by the linear transformation, as well as to the effective dimensionality of the transformed features. These findings demonstrate that, even with massive behavioral data, overly flexible alignment metries may fail to guide toward arti- ficial representations that are genuinely more human-aligned, Model comparison experiments must be designed to balance the trade-off between predictive accuracy and identifability-enuuii that the best-fitting model is also the right one.
Introduction
The search for mechanistic explanations of human cognition, in combination with rapid advances in deep learning, has motivated the use of stimulus representations in pretrained neural networks as models of the biological representation of complex stimuli. Even without modification, activation patterns in artificial neural networks (ANNs) trained visual tasks show surprising correspondence with cortical visual representations [1-4] and visual perceptual judgments [4-11]. When evaluation is made flexible by fitting linear weights to improve the alignment between ANN representations and brain [10, 12-14] or behavioral data [7, 15-17]. this approach often achieves predictive accuracy exceeding that of any other computational model. In some neuroscientific applications (e.g.. brain- computer interfaces), accurate prediction useful regardless of the underlying mechanism. In contrast, basic-science studics in computational neuroscience often rely the assumption that neural network whose representations are more predictive of brain or behavioral data is a better model of the mechanisms underlying the observed biological data. For models evaluated without further data-driven fitting, high predictive accuracy occurring by chance is unlikely. However, once flexible, data-driven fitting procedures employed, an important question arises: does predictive accuracy under flexible evaluation reflect genuinely shared representations? [18-21]. This question carries
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

A
recovery accuracy
103
10K
37W
0.4
17:00
25.80
1130
204.96
number training triplets
B
C
D
data-generatinn model
111 171
number dra.noee
2000
VTT
recovered model
Figure 1: (A) Model recovery accuracy under linear probing with a p x p transformation matrix across different dataset sizes. (B, C, D) Confusion matrices at three training set sizes (400, 25.6K, and 1.6M triplets). Each matrix row corresponds to the data- tetta-gneratingg model, and each matrix column to the recovered model. Diagonal entries represent correct model recovery. Model recovery accuracy does not reach 80% % even for millions triplets.
weight since there are good reasons to employ flexible evaluation. A complete yet fully emergent representational alignment is unlikely even for models whose processing is qualitatively similar to that of humans. Furthermore, inter-inding indrviindiii variability may also motivate flexible alignment metries [22, 23). And yet, flexible evaluation may incur hidden cost: when each model allowed to adjust to best fit brain or behavioral data, predictive accuracy-even when obtained with held-out data-may no longer serve as a meaningful index of representational alignment.
Here, we employ a model recovery approach to test whether predictive accuracy, as measured using current analytic methods, is indicative of the probed representations. Specifically, use the behavioral THINGS odd-one-out dataset. We fit linear transformation for each of a diverse set of ANN representations to predict the empirical human judgments (Fig. 2). Then, in each simulation, we sample a synthetic behavioral dataset from one of the fitted models, fit each model to the synthetic data as it were a real experiment, and compute the models' cross-validated prediction accuracy with respect to this synthetic data. We then ask: does predictive accuracy correctly recover the data-generating model?
Related Work. There is growing concern about ability of commonly used representational
alignment metrics to accurately identify underlying representational structures [10, 14, 18, 19, 24-31]. The gold-standard approach for evaluating such risk is model recovery simulations, where data is simulated using the predictions of each of the models in turn to test whether the evaluation procedure identifies the generating model [32-35]. However, applying model recovery analyses to compar- isons of ANNs models of biological representations has largely been limited to simulations using noiseless ANN activations as observed data.
Kornblith et al. [24] compared activations from ANNs trained with different random initializations and found that flexible metrics such as linear encoding models and canonical correlation analysis failed to distinguish among specific layers, whereas an inflexible metric (centered kernel alignment, CKA) succeeded. However, since the flexible metrics were not cross- valds-adld, their failure in the large regime could be attributed to overfitting.

---

Han et al. [18] compared the ability of CKA and linear regression to identify ANN architectures across initializations. When the ground-truth model was included among the candidates, CKA ranked it highest. CKA did not differentiate well between model families (convolutional networks and transformers) when the ground-truth model was held out. Cross-validated linear regression identified most-but not all-models when the ground-truth architecture was present. It was somewhat less likely than CKA to misidentify the model family when the ground-truth model was held out. However, as Han et al. [18] emphasize, these evaluations were conducted in an intentionally idealized setting: the observed data in each simulation were deterministic ANN activations, subject to no measurement noise trial-to-trial variability.
Schütt et al. [36] demonstrated successful recovery of different cortical areas with non non-flexible representational similarity analysis (RSA) using realistic, subsampled fMRI and calcium calcium-imaging imaging data, treating areas as alternative models. However, their validation of flexible RSA using ANN activations focused on calibration of statistical inference rather than evaluating model recovery performance in noise-calibrated settings.
Overall, the current literature does not alleviate the concern that model comparison may be inaccurate when deep neural networks are flexibly evaluated against real, noisy biological data. On real data, model comparisons are strongly constrained by signal-to-noise ratio. Specifically, without a calibrated noise model, one cannot obtain realistic estimate of model recovery accuracy- the probability of correctly identifying the ground ground-truth -ruth model among the alternatives. Model recovery accuracy is a key diagnostic for model comparative procedures [35], generalizing statistical power (1 () to comparisons among more than two competing alternatives.
Faithfully simulating neural data is difficult because it must capture multivariate noise and signal correlation structures. Here, we turn to simulating realistic behavioral experiments involving large datasets of discrete similarity judgments. Similarity judgments have long been used to infer latent spaces, most famously via multidimensional scaling (MDS) [37]. Online testing enables such experiments scale. Specifically, Hebart et al. [38] introduced the THINGS odd-one-out dataset [39, 40], consisting of 4.7 million responses. In each trial, participants viewed three distinct images and selected the one they considered to be the odd one out. The images were randomly drawn from a standardized set of 1,854 photographs of recognizable objects [39]. The odd-one-out task is robust to differences in how participants use rating scales [38] and is straightforward to model probabilistically. Embedding--aseed models such as SPoSe [38] and VICE [41] capture judgments in THINGS odd-one- out with high accuracy. However, they are non- non-image-cue monggecompptable: they lack a forward mapping from raw pixels and thus cannot generalize to novel stimuli, and more importantly, do not explain how the representations arise. Neural-networkk-bsee models circumvent these limitations by providing fully image-computable candidate representations [1, 7, 12, 42, 43J. In a large-scale study, Muttenthaler et al. [17] used the representations of pre-trained ANNs to predict the human judgments THINGS odd-one-out either without further fitting ("zero-shot") or under linear transformations ("linear probing"). They found that flexible evaluation substantially improved prediction performance; a CoAtNet [44] trained on image-text alignment [45] and supervised vision transformer classifier were the top performers. This tie between functionally distinct models raises concerns that mett-even with massive data prediccive gains from linear probing may come the cost of reduced identifiability.
Our contributions:
1. We formalize and conduct large-scale model recovery simulations for representational aligmment in discrete behavioral tasks, using neural networks reweighted to mimic human judgment patterns and calibrated to match the human noise ceiling.
2. We show that that-even even with millions of training reliably recover the data-gene model, with recovery plateauing below 80% accuracy (Fig. 1). This result challenges the interpretability of predictive accuracy under flexible evaluation.
linear probing fails to
3. We identify two sources of model misidentification: alignment- alignment-induced shifts in representa- tional geometry and elevated post-alignment effective dimensionality.
4. We demonstrate that even with substantial datasets of odd-one-out judgments, there is a sharp trade-off between predictive accuracy and model identifiability. While flexible evaluation increases the apparent alignment with human responses, can compromise model identifiability by diminishing inter-model distinctions.
3

---

2 Methods
2.1 Mapping neural network representations human odd-one-out judgments
We followed the data-analytic approach of Muttenthaler et al. [17], with one notable modification (see Regularization below). For each pre-trained ANN, we extracted the final representational layer (see Appendix A.8) activation matrix X E I=* where n = 1,854 denotes the number of images and the number of units. We then applied a model-specific learnable linear transformation W E 12xP to produce the transformed representation matrix XW.
We computed representational similarity matrix (RSM) S R"**, where the similarity between each pair of stimuli is defined as the inner product between their transformed representations:
S = (XW) X
When W is set to the identity matrix, the RSM directly reflects the model's representational geometry, enabling parameter-free ("zero-shot"; [17]) evaluation of the model against human judgments. When W is optimized to predict human judgments, the RSM flexibly adjusts to best fit the human representational gcometry ("linear pnnearrprobbng"" [17]).
For each trial in the human data, the similarities among the three presented images determine the model's s predicted choice: the odd-one-out the stimulus that is not part of the most similar pair. To obtain probabilistic predictions of odd-one-out judgments, a softmax is applied over the pairwise similarities within each triplet [17]. Given trial with image triplet fa, b, c} and representations of model M, the probability of choosing image the odd-one-out is defined by
exp(Sb.e.T)
p(odd-one-out = a triplet = {a, b, c), M) exp(Sa,t/T) + exp(S.. exp(S...)) + exp(S
Here, a temperature parameter held constant during fitting and adjusted later during calibration.
To fit W to choice data, we used full-batch L-BFGS [46] to minimize the negative log-likelihood of the probabilistic predictions plus a regularization term:
Nerials
1
W* arg min W
logplodd-once onc-out = rs {a, b,, M) +WR(W),
Neriala
model prediction trial
where a, by. and c₁ index the images presented in the i-th trial, r; the corresponding human odd-one- choice.
The Frobenius-nome coaittus-nomm--asseee based regularization of W, as used in [17], can degrade performance below zero-shot levels when strong penalties applied. We replaced the regularization term with one that shrinks W toward a scalar matrix (see also [47]).
Regularization.
(Lr(W))²
R(W) min W
I
W
An analytic derivation is provided in Appendix A.1, and an empirical comparison to Frobenius-norm regularization is shown in Figure S1.
Calibration. While probabilistic models are often calibrated to minimize their negative log proba- bility on held-out data [48], here we adjust the temperature parameter ensure that the variability of simulated responses matches that of human judgments. When different participants judge the same randomly sampled triplet, their responses agree only about two-thirds of the time [38]. To reproduce this variability in simulation, we used responses from the THINGS odd-one-out noise ceiling ex- periment, in which 30 participants judged the same 1,000 triplets. We estimated the noise ceiling using a saav--one-subect took the majority vote of the remaining 29, and recorded whether the held-out answer matched that vote. We then repeated this step for all participants and averaged the match rates across all triplets (see Appendix A.2.1 for mathematical formulation). For each fitted model, we tuned the temperature parameter T' so that, when sampling responses to these 1,000 triplets according to the model's
procedure: for each triplet, we removed one participant's 's response,

---

probabilistic predictions, the resulting prediction accuracy matched the human wwwee--nn-sssje-e-eeeeee noise ceiling estimate (67.8%). This estimate was computed by predicting each experimental trial using the most common response among the other participants who viewed the same triplet. See Appendix A.2.1 for implementation details. Note that, unlike standard calibration-whice predictive entropy in less accurate models-this procedure matches each model's 's response variability to the level observed in human judgments, independently of its predictive accuracy.
increases
2.2 Model recovery experimental setup
To evaluate the identifiability of alternative neural network models of human perceptual representation, we simulated model- comparrson experiments in which behavioral normmally collected from humans-we was replaced with simulated responses generated by one of the models. Within simulation, synthetic behavioral data were compared to the predictions of each of the candidate models using flexible model-behavior evaluation (i.e., linear probing). Suppose the widely employed analytic approach of linearly transforming neural network representations is valid. that case, the specific model that has generated the data in a simulation should achieve the highest predictive accuracy, thereby supporting correct model recovery (see Fig. S2 for a visual illustration of the process). This setup is analogous to data distillation: the candidate models act as "students" attempting to approximate the input-output function of a "teacher"-the data-generating model. Model recovery is successful when the best-performing student is the teacher itself, rather than alternative model.
Importantly, real human data (THINGS odd-one-out was used only for shaping the predictions of the gette-ennratiigg models; during model comparison, each model was fitted from scratch to the synthetic data, closely emulating the constraints of real data analysis, where neither the ground-truth model nor its model-to-hehavion mapping parameters are known.
Human aligned models. We assembled set of 20 ANNs of diverse architectures and training tasks (model details in Table S1). To generate synthetic data under the hypothesis that neural network representations can and should be linearly transformed match empirical behavioral data, we first aligned each model to the THINGS odd-one-out training dataset (4.5 million triplets). For each model M, used three-fold cross validatiio over disjoint image subsets [17] to select the optimal regularization hyperparameter AM then fitted a model-specific transformation matrix W - (M r denotes mapping model M THINGS odd-one-out, 7), and finally calibrated the model's temperature to reproduce the human noise ceiling (see Calibration above). This procedure was applied to each neural network model independently. The resulting aligned models represent each neural network's 's best approximation to human judgments under linear model-to toeel-tto-hhviioo mapping assumption, and serve exclusively as generaenng models in the simulations that follow.
Simulated model comparison. Given a set of random triplets (sampling details in Appendix A.3), designated one of the 20 aligned models as the mata-generating model and used its probabilistic predictions to synthesize human responses. Specifically, the data-generating model defined a cate- gorical distribution over the three items of each triplet, from which we sampled single response, emulating the experimental paradigm employed by Hebart et al. [38], in which each triplet was presented to single participant. For each simulation, this procedure resulted in sets of discrete responses for the training, validation, and test triplets.
Once the synthetic responses had been generated, all 20 models, including the data-generating model, were independently fitted to the synthetic training data from scratch, with W initialized to the identity matrix. For each model, an optimal regularization hyperparameter was selected using the validation triplets, W was optimized on the training triplcts, and prediction accuracy was evaluated on the test triplets. Because the gentereenn model was calibrated, the human noise ceiling bounded prediction accuracy from above. However, if the evaluated model's predictions diverged from those of the data-generating model, the prediction accuracy could be lower. Even the data- geteeeenerrinnggg model was not guaranteed to attain the noise ceiling, since it was evaluated on the test triplets after fitting W to limited synthetic responses rather than the full THINGS odd-one-out dataset.
In each simulation, this procedure was repeated across the three cross-validation folds (testing generalization to new images [17]). averaging the resulting prediction accuracy. For model recovery simulation to be successful, the mean test prediction accuracy of the model that generated the data must be the highest among all of the models.
5

---

Model recovery accuracy. For each stimulus set size, we repeated the simulation using 30 different random seeds sample the stimuli. For each seed, we treated each of the 20 models as the data- generating model in turn, yielding 600 simulations per stimulus set size. Model recovery accuracy was defined as the proportion of simulations which the data-generating model achieved the highest cross-validated test accuracy among all candidate models. We summarize our results in confusion matrix defined as follows (full formulation is provided Appendix A.4):
Let M = 1, .M...., Mv} }o be a set of N models, with indices i E N. J. Let Mi-rr mark model M, as the data- generating model after it has been fitted and calibrated on human responses т. For each simulated dataset d E (1,...,) D} we define M the candidate model M, fitted to the simulated predictions of M;-r on simulated dataset d. Let Acc( M,+r) be the predictive accuracy achieved by candidate model M, on the d-th test set responses generated by the ground- truth model M, (see definition in Eq. 10). The model confusion matrix is defined by:
Cj = [[Acc(MA M max Aco(M mE (E(...M) m-st Mi C Ce MENMIIIM
(5)
Entry Cij denotes the number of simulated datasets (out of D) for which candidate model M, was the best predictor of data generated by model M, Model recovery accuracy is the empirical probability of correctly identifying the ground-truth model (chance level M/|M perfect recovery 1):
IMI
|
Model Recovery Accuracy = Σ C#
ΣΣ Cij
(6)
1=1
==
Results
A
Scpenteed 2or N0geser
Opendi CLP auss
Opencal CLP ReSSO
Barlow
RA0 END2
Barlow
SHAD RNCO
CATS
Daffs
CoRtee
Denoe
0.50
0.40
B:45
0.50
2.59
0.80
0.57
0.35
0.40
0.45
0.50
0.55
8.00
8,87
test accuracy (W lgxcal
accuracy (W Ding(ReS)))
FObl celling
Openli CLP 0155
OpenAA
VGDiig RNSO
Barlow
5W0D 2453
Barlow/urme
BETN
BETI2
Danealuec07
0.53
0.00 0.80
0.87
0.40
0.45
0.90
0.87
test accuracy (W's 20 10)
test accuracy (W 230
Figure 2: Model test prediction accuracy on the THINGS odd-one-out dataset across varying levels of evaluation flexibility. (A) Zero-shot evaluation (using each model's original embedding). (B) Linear probing with a diagonal transformation matrix, fitting parameters.
(C) Linear probing with 10 rectangular transformation matrix. (D) Linear probing with a p x p full matrix.
Significance plots: filled dot connected to open dot indicates that the filled-dot model had signifi- cantly higher accuracy (p-value < 0.05, sign test, Bonferroni-corrected across 190 comparisons).

---

OpenAy CUP RNN0
Open CUP RN00
Mita SLIP
Mata SLP
Mets CUP
Mets Al CLIP
VicRte
Vicfag
BarlowTwins RNNG
BarlowTwins RNUG
Mads Ste C
Mads Sinti It
DINOY2
DINO12
BEITV2
BEIT22
inceptionn
Inceptionv
Inceptorithiv22
Inceptoonthi22
CORnes
CORnec
NASNee
NAShee
EfnieenNit B7
Efneenti B7
Comoheat
Combiveal
DenseNes201
DenseNee201
VCC19
RauNle50
RauNat50
renked Leat
charra
rankte Tiret
nankac Leat
chanca
ransed Trrs
Invel
Figure 3: (A) Mean rank of each model's predictive accuracy when it generated the data. A mean rank above indicates systematic misidentification other models more often achieved higher predictive accuracy data it produced. (B) Mean rank when the model did not generate the data. In the absence of bias, average ranks should be near chance level (dashed line). Model misidentification is -ystematic-biased toward some models and away from others. All results were computed on simulated datasets with 4.2M training triplets.
3.1 Comparing model predictions to empirical behavioral responses.
Before conducting the simulation study, we examined the prediction accuracy of the 20 candidate models on the THINGS odd-one-out dataset under both zero-shot and flexible evaluation settings (Fig. 2, Table $1; 3-fold cross-validation with disjoint image sets). As in Muttenthaler et al. [17], linear probing yielded higher accuracy than zero-shot evaluation. Furthermore, greater flexibiiit-mooing moving from diagonal to rectangular full W consistently yielded additional gains.
Under the most flexible evaluation (full W; Fig. 2D), several models achieved near-noise-ceiling predictive accuracy of human responses, with single model performing significantly better than the others. Note that if the analytic strategy were guided solely by prediction accuracy, the most flexible evaluation would appear be the obvious choice.
3.2 Model recovery simulations
Recovery accuracy improves with data size but plateaus below 80%. We ran simulations across 18 training set sizes (i.c., the number of synthetic triplets used to fit W in each fold). logarithmically spaced between 50 and 4.2 million training triplets. As described in Section 2.2, in each simulation, synthetic data were generated from a model aligned to the full THINGS odd-one-out dataset via a fitted p matrix, and the candidate models competed to predict the synthetic responses, each fitting a p x p matrix to the synthetic training set, and then tested on held-out synthetic responses. Model recovery accuracy
function of training set size is shown in Figure 1A. For small datasets (i.e., those with thousands of triplets), model recovery accuracy remained below 20%. Recovery accuracy increased with dataset size; however, with 4.2 million training triplets, it did not reach 80%.
Controlling transtormation dimensionality does not mitigate model misidentification. One plausible cause for the limited model recovery differences the number of adjustable parameters: some models have more units in their final representational layer, resulting in greater number of adjustable parameters However, when reran the simulations using only the top 500 principal components of each model's representation as features (thus fixing the parameter count in to 500 x 500), model recovery accuracy did not improve (Fig. $3).
W. These models might better fit the data regardless of its source.
Model recovery performance plateaus despite objective-driven representational divergence Recent work by Lampinen et al. [49] showed that different training objectives induce distinct representational biases. To test how these biases affect model recovery, we expanded the model set to include 10 additional models, primarily mmage-text-allgneer (Table S5). We then evaluated model recovery accuracy using the expanded model set (see Fig. S4). As expected, model recovery
7

---

accuracy declined with the expanded model set. It plateaued near 70%, even with 4.2 million training triplets. Next, we categorized each model as supervised, unsupervised, or image- text-aligned and performed a between-objective model recovery analysis (see Fig. S5). Even with 4.2 million training triplets, objective- recovery reached only 73.7% (Fig. S5D), despite being significantly easier than model-based recovery. These results indicate that, although initial internal representations differ in objective-specific biases [49], linear probing can obscure objective-specifie differences in representational geometry. Grouping models by architecture type (convolutional vs. vision transformers) yielded similar results (70.3% accuracy, Fig. S5).
Certain models dominate recovery-even when incorrect. Inspection of confusion matrices (Fig. IB-D) indicates that the error is systematic: one model, OpenAl CLIP ResNet-50, was consistently misattributed as the ground-truth model. Would model recovery reach 100% accuracy if this model were excluded? To test this, we measured the mean rank of each model's predictive accuracy when served as the data-generating model (Fig. 3A). Four of the 20 models had a mean rank above 2 (i.e., worse than second place), indicating that, on average, when these models generated the data, more than one competitor achieved higher predictive accuracy. We also computed the mean rank of each model's predictive accuracy when it was not the data-generating model (Fig. 3B). This revealed considerable variation in the models' propensity to be falsely identified as the data-generating model.
3.3 Representational geometry-based causes of model misidentification
The limited model-recovery accuracy prompted us examine factors that might cause or modulate model misidentification. Specifically, we assessed how the representa- tional geometry of each model, defined by the set of pairwise distances among its stimulus representations [36], was altered by linear probing. For each model, we computed a representational dissimilarity matrix (RDM) consisting of squared Euclidean distances among its final representational layer ac- tivation patterns in response to the 1,854 THINGS object images. This measurement was conducted both before and after aligning these representations to THINGS odd-one- out with a full W. As a surrogate of human representations, we also included the RDM of VICE [41], an embedding model directly fitted to THINGS odd-one-out, We quantified within- and between-model RDM similarity using whitened Pearson correlation [50, 51] (Fig. S6) and employed multidimensional scaling (MDS) to summarize and visualize these results (Fig. 4). As expected, all models' representational geometries shifted toward that of VICE following the aligmment. Models that best predicted human judgments (e.g., VIT-L/16 and OpenAl CLIP-ResNNttO)) exhibited representational geometries more similar to VICE from the outset. By contrast, models that often failed to be recovered (e.g., EfficientNet B7, NASNet Large, or Inception-ResNer. V2) had initial representational geometries more distant from those of VICE and underwent substantial shifts in representational geometry as a result of the linear transformation.
Orighal Repr cspressttations
"aanifereed Rop sesrrtttae
5WW
Coplear
Hesad CUI'
Pewaing Spn
EBOGL
Figure 4: Shifts in model representations after linear probing, visualized using multidimensional scaling (MDS). Dots mark each model's original fi- nal representations. Arrowheads mark the aligned representations. VICE [41], an embedding model fitted to THINGS odd-one-out, serves as a proxy for human-like representation in this visualization.
representational shifts are related to poor model recovery outcomes To test whether alignment-induced representational shifts predict model-specific recovery outcomes, we used linear regression analysis with shift magnitude and other geometric and architectural features (Table S2) as predictors. The dependent variable was defined as the difference in predictive accuracy between the data-generating model and one alternative candidate model, computed separately within each simulation. This accuracy difference served as a continuous measure of the separability of each model pair.
Substantial alignment-induee
8

---

The analysis revealed three significant predictors (Bonferroni-coreeet) corrected over 10,000 bootstrap tests): First, the shift magnitude of the candidate model how much its geometry changed under alignment- positively predicted accuracy differences ( 0.495, p-value = 0.02), suggesting that models more altered by linear probing were less predictive of responses generated by other models. Conversely, the shift magnitude of the data-generating model negatively predicted accuracy differences (B -0,2510, p-value = 0.01), indicating that substantially adapted models yielded synthetic responses more easily predicted by other models than by themselves.
The third significant predictor was the effective dimensionality (ED) of the data- getta-eneatiing model representations after alignment to THINGS odd odd-one-out. ED quantifies how many feature space dimensions account for meaningful variance [52]; we used standard estimator detailed in Appendix A.5. Recent work has linked higher ED of neural network representations to improved prediction of visual cortical responses [53], though see [14, 54] for contrasting views. Higher post-transformation ED in the data-generating model negatively predicted accuracy differences (3 = -0.455, p-value 0.01), suggesting that models whose aligned representations are high- dimensional are less likely to be correctly recovered.
3.4 The prediccie--cccrre--odddddddiddid trade-off'
0.56
A seemingly straightforward solution to the problem of model misidentification is to re- strict evaluation flexibility- by using zero-shot predictions or by applying linear probing with fewer free parameters in the transformation matrix. However, the mean predictive accuracy across the twenty models drops markedly when they cannot reweight or linearly remix their features (Fig. 2A).
0.64
mean predictve Rccuney
0.47
To characterize the trade-off between pre- dictive accuracy and model identifiability, we repeated the model recovery experiments while varying the level of evaluation flexibil- ity: from zero-shot evaluation, through di- agonal and thin rectangular W matrices, to unrestricted linear probing. We matched flex- ibility constraints across the data- generation and evaluation stages: for example, the data- generating model was fitted using diagonal W, candidate models (including the generat- ing model) were also evaluated using diago- nal matrices. This was done across multiple simulated dataset sizes. Similarly, we re-estimated empirical predictive accuracy using randomly subsampled subsets of THINGS odd-one-out.
0.44
0.42
0.1
0.3
0.4
0.6
0.7
0.9
1.0
model recovery accuracy
Paadbility
reguiartratinn W-7
vet
Wo Deaul0
(WW7
-
20,033
20,000
Figure 5: Model recovery accuracy vs. predictive accuracy (averaged across models), as a function of flexibility level and dataset size.
As shown in Figure 5, there is clear trade-off between predictive accuracy and model recov- cry accuracy. As evaluation flexibility increases, we gain predictive accuracy at the expense of discriminability.
Discussion
Our results show that, even with millions of trials, linear probing can fail to identify the model that generated the data. For our of candidate models, model recovery accuracy plateaus below 80%. Holding the number of features constant across models-ane thus the parameter count of the linear transformation matrix-does not mitigate the problem. Furthermore, typical small-scale experiments (e.g.. 100,000 trials), model recovery accuracy can be far worse-for our model set, it remains below 50%. These findings call into question the use of predictive accuracy under linear probing alignment metric for comparing models of biological representation.

---

Limitations. The scope of our simulations is limited to behavioral data, and specifically, to the THINGS odd-one-out task. We chose this task as a test case because it is supported by a large empirical dataset [40] and allows straightforward simulation of synthetic responses by sampling from model-specified multinomial distributions. The noise-calibrated simulation approach can be readily extended to other behavioral paradigms, such as classification [15] or nulti-arrangement (Kricgeskorte & Mur, 2012). Model identification using neural data operates in a markedly different regime: responses are multivariate and continuous, rather than univariate and discrete as in the behavioral case. Therefore, while our results demonstrate a pronounced predictivty-idenifabllyy trade-off when comparing models to behavior in large dataset, the severity of this trade-off for neural data cannot be inferred from our findings. Recent reports of qualitatively distinct neural network models achieving indistinguishable performance under flexible comparisons to neural data [10. 14] make this question especially pertinent. Addressing will require future work using noise-calibrated, modality-specifff neural simulations. It is important to note that the quantitative model recovery accuracy levels reported are specific to the candidate model set used. Still, we expect the qualitative finding of prevalent model misidentification generalize to other model sets of similar size and to become more pronounced with larger sets.
A more fundamental limitation is that, in real-world comparisons between model predictions and empirical responses, the true model- the biological representations- is absent from the candidate set. Thus, model recovery within closed set is necessary but insufficient criterion for reliable model-comparison experiments.
Navigating the -iintfiiidy trade-off. The empirical findings highlight a tension between predictive performance and model identifiability; increasing the flexibility of the alignment metric improves predictive accuracy, but it also reduces the ability to discriminate among competing models. Experimental and analytical decisions guided solely by the goal of maximizing predictive accuracy risk overlooking this trade-off, thereby landing at its far end, where predictive performance is high but mechanistic correspondence to the modeled system is limited. Therefore, the pursuit of predictive performance must be tempered by attention to the specificity of the predictions: for example, through boise-calibrated model recovery simulations, as explored here.
Progress beyond the limitations of the iccuracyideetfffaaliit trade-off may require rethinking evaluation practices along three key directions.
1. Change the stimuli: As in many model comparison studies, we evaluated models out-of-sample- that is, on new stimuli drawn from the training distribution. -uittf--distiibution generalization, which more strongly probes the models' inductive biases, may offer greater model comparative power [55, 56]. Stimuli designed to elicit model disagreement may yield even greater gains [57-61].
The recovery gains we obtained from larger and more diagnostic triplet sets suggest that smarter sampling matters at least much as sheer volume (Fig. 1). Adaptive, model-driven stimulus crpettion-constuctiin trials that maximize the expected divergence in network responses-coa sharpen our ability to treat predictive accuracy as indicator for human-model alignment, enhancing current flexible alignment methods without compromising identifiability [56-60].
2. Change the metrics: Constraining data-driven model alignment by biologically motivated and/or inter-individual variability-infonna priors [23, 62-65] may improve upon the overly flexible family of linear transformations. Furthermore, imposing greater constraints on the readout may enhance its interpretability. For example, constraining the learned stimulus embeddings to be non-negative prevents features from canceling each other (e.g., [66]). Finally, Bayesian readout models, which estimate a distribution of feature weights rather than a point estimate, may improve robustness to sampling noise.
3. Change the models: The considerable geometric shifts required to align the networks suggest that linear probes can obscure important representational mismatches. Embedding richer priors directly into the models-through task design, objective functions, or biologically inspired architectures [5, 67- cll-could allow aligned representations to emerge natively, reducing the need for substantial post hoc transformations. More broadly, further progress in neural network-based modeling of brain and behavior may depend less ever-larger data-driven fits of pre-trained models and more on deliberate model refinement to embody explicit computational hypotheses.
Code and data are available on http////wtiiaaacccccce//ooooooo.
10

---

Acknowledgments
This work was supported by the Israel Science Foundation (grant number 534/24 to T.G.).
References
[1] Seyed-Mahdi Shaligh-Razane and Nikolaus Kriegeskorte. Deep Supervised, but Not Unsuper- vised, Models May Explain IT Cortical Representation. PLoS Computational Biology, 10(11); e1003915, November 2014. doi:
1003915.
[2] Radoslaw Martin Cichy, Aditya Khosla, Dimitrios Pantazis, Antonio Torralba, and Aude Oliva. Comparison of deep neural networks to spatio-temporal cortical dynamics of human visual object recognition reveals hierarchical correspondence. Scientific Reports, 6(1):27755, June 2016. doi: 1o:::0..000........ 10388//epp27755
[3] Charles F. Cadieu, Ha Hong, Daniel I.. K. Yamins, Nicolas Pinto, Diego Ardila, Ethan A. Solomon, Najib J. Majaj, and James DiCarlo. Deep Neural Networks Rival the Representation of Primate IT Cortex for Core Visual Object Recognition. PLOS Computational Biology, 10 102):e1003963, December 2014. doi:10.1311 pttpo//al....i.10030333...
[4] Marcie L. King, Iris LA. Groen, Adam Steel, Dwight J. Kravitz, and Chris I. Baker. Similarity judgments and cortical visual responses reflect different properties of ob- ject and scene categories in naturalistic images. Neurolmage, 197:368-382, 2019.
doi.htts://doi.org// 101...116/..nnee. www.monnnuriaage2219..e.
[5] Joshua C. Peterson, Joshua T. Abbott, and Thomas L. Griffiths. Adapting Deep Network Features to Capture Psychological Representations: An Abridged Report. In Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, pages Melbourne, Australia, August 2017. International Joint Conferences on Artificial Intelligence Organization. doi: 160:100229933iicc2221777666
[6] Kamila M. Jozwik, Nikolaus Kriegeskorte, Katherine R. Storrs, and Marieke Mur. Deep Convolutional Neural Networks Outperform Feature-Based But Not Categorical Models in Explaining Object Similarity Judgments. Frontiers in Psychology, 8:1726, October 2017. doi: (1333////ssg22217..011
[7] Joshua Peterson, Joshua T. Abbott, and Thomas Griffiths. Evaluating (and Im- proving) the Correspondence Between Deep Neural Networks and Human Representa- tions. Cognitive Science, 42(8):2648-266) November 2018. ISSN 0364-0213, 1551-6709. 111.1111//oog..267777
[8] Richard Zhang, Phillip Isola, Alexei A. Efros, Eli Shechtman, and Oliver Wang. The unrea- sonable effectiveness of deep features as a perceptual metric. In 2018 IEEF/CVF conference on computer vision and pattern recognition (CVPR), pages 586-595, Los Alamitos, CA, USA, June 2018. IEEE Computer Society. 100.1001109/CV..201.8.8000
[9] Elissa M. Aminoff, Shira Baror, Eric W. Roginek, and Daniel D. Leeds. Contextual associations represented both in neural networks and human behavior. Scientific Reports, 12(1):5570, April 2022. doi: 10. -59-34444119-022994555511
[10] Katherine R. Storrs, Tim C. Kietzmann, Alexander Walther, Johannes Mehrer, and Nikolaus Kriegeskorte. Diverse Deep Neural Networks All Predict Human Inferior Temporal Cortex Well, After Training and Fitting. Journal of Cognitive Neuroscience, pages 1-21, August 2021. 1oi: 1tt:001112//oen___01100
[11] Vijay Veerabadran, Josh Goldman, Shreya Shankar, Brian Cheung, Nicolas Papernot, Alexey Kurakin, Ian Goodfellow, Jonathon Shlens, Jascha Sohl-Dickstein, Michael C. Mozer, and Gamaleldin F. Elsayed. Subtle adversarial image manipulations influence both human and machine perception. Nature Communications, 14(1):4,33, August 2023, doi: 10. 1..1003 1038/<41467- 023-40499-0.

---

[12] Daniel L. K. Yamins, Ha Hong, Charles F. Cadieu, Ethan A. Solomon, Darren Seibert, and James DiCarlo. Performance-optimine hierarchical models predict neural responses in higher visual cortex. Proceedings of the National Academy of Sciences, 111(23):619-8622 June
2014. doi: 1073/pnas. 14031 12111.
[13] U. Guclu and M. A. J. Van Gerven. Deep Neural Networks Reveal Gradient in the Complexity of Neural Representations across the Ventral Stream. Journal of Neuroscience, 35(27): 10014. July 2015. 1o:: 10..5/////NEEURSS..12---2-2200
[14] Colin Conwell, Jacob S. Prince, Kendrick N. Kay, George A. Alvarez, and Talia Konkle. A large-scale examination of inductive biases shaping high-level visual representation in brains and machines. Nature Communications, 15(1):9383, October 2024. 53147-y.
[15] Ruairidh M. Battleday, Joshua C. Peterson, and Thomas L. Griffiths. Capturing human cat- egorization of natural images by combining deep networks and cognitive models. Nature Communications, 1(1):5448, October 2020. doi:10. Noi:10.10330 1038///44467-0000- 18946-2.
[16] Christoph Daube, Tian Xu, Jiayu Zhan, Andrew Webb, Robin A.A. Ince, Oliver G.B. Garrod, and Philippe G. Schyns. Grounding deep neural network predictions of human categorization behavior in understandable functional features: The case of face identity. Patterns, 2(10): 100348, October 2021. ISSN 26663899, doi: 10.16 p0tt::....tte..1...11..
[17] Lukas Muttenthaler, Jonas Dippel, Lorenz Linhardt, Robert A. Vandermeulen, and Simon Kornblith. Human alignment of neural network representations. In The eleventh international conference on learning representations, 2023, URL https: / wtwpemrrrvioo..n//ooor.. ReDQ10UQROX.
[18] Yena Han, Tomaso A. Poggio, and Brian Cheung. System Identification of Neural Systems: If We Got It Right, Would We Know? In Proceedings of the 40th International Conference on Machine Learning, pages 12430-12444, PMLR, July 2023. URL https:: http:///ooccee.n..//.mm mlr.preae wwwppppww//////eeepe hanpd//aandd..... html.
[19] Ansh Soni, Sudhanshu Srivastava, Konrad Kording, and Meenakshi Khosla. Conclusions about Neural Network to Brain Alignment are Profoundly Impacted by the Similarity Measure. bioRxiv, page
January 2024. do 101/2022408..0.6600070 07.00355..
[20] Nikolaus Kriegeskorte. Representational similarity analysis connecting the branches of sys- tems neuroscience. Frontiers in Systems Neuroscience, 2008. doi: (00.:338//euuo0000000.0.000
[21] Robert Geirhos, Kristof Meding, and Felix A. Wichmann. Beyond accuracy: quantifying triai-by-trial behaviour of CNNs and humans by measuring error consistency. Advances in neural information processing systems, volume 33, pages 1 13890-13908 Curran Associates, Inc., 2020. URL https: httpe//eeoeeeee.eeeeeeee/esssssp www.gngn.....ureeeeeeeeeeee.eewe peurips.cc/papee...e wwcpppperraaeeeeeeeeeeeeeeee files/ wwwes/ppper//00//0// pttpp/pppp/////iiii///p
[22] Rosa Cao and Daniel Yamins. Explanatory models in neuroscience, Part I: Taking mech- anistic abstraction seriously. Cognitive Systems Research, 87:101244, September 2024.
doi: 0o0:10.1016...oossy.00244 2014:/..coooss.22200001244.. 101244.
[23] Imran Thobani, Javier Sagastuy- BBeea, Aran Nayebi, Jacob S. Prince, Rosa Cao, and Daniel LK Yamins. Model-brain comparison using inter-animal transforms. In 8th annual conference cognitive computational neuroscience, 2025. URL https: h/tpp:/ss///peeeiieeee http://pppprrrreeeeeefotoooomm// bra7292CMm.
[24] Simon Komblith, Mohammad Norouzi, Honglak Lee, and Geoffrey Hinton. Similarity of Neural Network Representations Revisited. In Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pages 3519-3529, PMLR, June 2019. URL https: / /ppoceeddigg mlr press/v9) /tteee//////ooootttth111 /koo/ooobbitttiaaa html.
[25] MohammadReza Davari, Stefan Horoi, Amine Natik, Guillaume Lajoie, Guy Wolf, and Eugene Belilovsky. Reliability of CKA as similarity measure in deep learning. In The eleventh international conference on learning representations, 2023. URL. https: /ttp:///oe/e/....///// htt/forummidd-SHByyyaa
12

---

[26] Jacob S. Prince, George A. Alvarez, and Talia Konkle. Representation with a capital 'R'. In UniReps: 2nd Edition of the Workshop on Unifying Representations in Neural Models, 2024. URI. htttps:://peereieeeeeee http://wwww...eetooooooooe/////a
[27] Rylan Schaeffer, Mikail Khona, Sarthak Chandra, Mitchell Ostrow, Brando Miranda, and Sanmi Koyejo. Does Maximizing Neural Regression Scores Teach Us About The Brain? In NeurlPS 2024 Workshop on Symmetry and Geometry in Neural Representations 2025. URL
[28] Frances Ding, Jean-Stanislas Denain, and Jacob Steinhardt. Grounding Repre- sentation Similarity Through Statistical Testing. In Advances in Neural Informa- tion Processing Systems, volume 34, pages 1556-1568. Curran Associates, Inc.. 2021. URI. https::/prroceedingg meweeeedddinnnennieneeeeeee wwtppipp..pppppp..peeepeee wttp://e//pppee/22///1eeee (c0t5911T79925000011199999699
pdf.
[29] Yiqing Bo, Ansh Soni, Sudhanshu Srivastava, and Meenakshi Khosla. Evaluating repre- sentational similarity measures from the lens functional correspondence, 2025. URL
[30] Max Klabunde, Tobias Schumacher, Markus Strohmaier, and Florian Lemmerich. Similarity of Neural Network Models: A Survey of Functional and Representational Measures. ACM Comput- ing Surveys, 57(9):1-52, September 2025. ISSN 0360-0300, 1557-7341. doi:10. 1455337844/3
[31] Lukas Braun, Erin Grant, and Andrew M. Saxe. Not solutions are created equal: An analytical dissociation of functional and representational similarity in deep linear neural networks. In Proceedings of the 42nd International Conference on Machine Learning, pages 5355-5382. PMLR, October 2025. URL https: https: http:///pocceeen..///o mlr, wwwpppressrii1 http://////////a/// /brp///raunaa.... html. ISSN: 2640-3498.
[32] Mark A. Pitt and In Jae Myung. When a good fit can be bad. Trends in Cognitive Sciences, (10):421-425, October 2002. ISSN 13646613,
[33] Stefano Palminteri, Valentin Wyart, and Etienne Koechlin. The Importance of Falsification in Computational Cognitive Modeling. Trends in Cognitive Sciences, 21(6):425-43,, June 2017. ISSN 13646613. doi: t011/..
[34] Eric Jonas and Konrad Paul Kording. Could a Neuroscientist Understand a Microproces- sor? PLOS Computational Biology, 13())=e1052268, January 2017. ISSN 1553-7358.
107:10.377/jourmll 1005268.
[35] Robert C Wilson and Anne Ge Collins. Ten simple rules for the computational modeling of behavioral data. eLife, 8:e49547, November 2019. 1oi:10.7554/eLife..44 495477.
[36] Heiko Schütt, Alexander D Kipnis, Jöm Diedrichsen, and Nikolaus Kricgeskorte. Statistical inference representational geometries. eLife, 12:c82566, August 2023. Noi:110.7554/eLife..2555.
[37] Roger N. Shepard. The Analysis of Proximities: Multidimensional Scaling with an Unknown Distance Function. Psychometrika, 27(2):125-140 June 1962. doi: N00:100.007/BF02289663
[38] Martin N. Hebart, Charles Zheng, Francisco Pereira, and Chris I. Baker. Revealing the multi- dimensional mental representations of natural objects underlying human similarity judgements. Nature Human Behaviour, 4(11):1173-118) October 2020. 100::00..08///4152-00000
[39] Martin N. Hebart, Adam H. Dickter, Alexis Kidder, Wan Y. Kwok, Anna Corriveau, Caitlin Van Wicklin, and Chris I. Baker. THINGS: database of 1,854 object concepts and more than 26,000 naturalistic object images. PLOS ONE, 14(10):a0233792 October 2019. doi: 101:10..37//joummall pww..........22....m.
[40] Martin N Hebart, Oliver Contier, Lina Teichmann, Adam H Rockter, Charles Zheng, Alexis Kidder, Anna Corriveau, Maryam Vaziri-Pashkam, and Chris I Baker. THINGS-data, multi- modal collection of large-scale datasets investigating object representations in human brain and behavior. eLife, 12:882580. February 2023. doi: 700.10.7554/eiic.855555
13

---

[41] Lukas Muttenthaler, Charles Y. Zheng, Patrick McClure, Robert A. Vandermeulen, Martin N. Hebart, and Francisco Pereira. VICE: Variational Interpretable Concept Embeddings. In Advances in Neural Information Processing Systems, volume 35, pages 33661-33675, Decem- ber 2022. URL http:ss://prrrcceedinnsssss/ http:///diinseeeuusc.....e/eeeee files pttp:/pppp///aaaaaa//eee /2tp://aaaae.eeeeeepp//////////0 da
ConfoeeneeCeeeeeeeeeeoo html.
[42] Ilia Sucholutsky, Lukas Muttenthaler, Adrian Weller, Andi Peng, Andreea Bobu, Been Kim, Bradley C. Love, Christopher J. Cueva, Erin Grant, iris Groen, Jascha Achterberg. Joshua B. Tenenbaum, Katherine M. Collins, Katherine Hermann, Kerem Oktar, Klaus Greff, Martin N. Hebart, Nathan Cloos, Nikolaus Kriegeskorte, Nori Jacoby, Qiuyi Zhang, Raja Marjieh, Robert Geirhos, Sherol Chen, Simon Kornblith, Sunayana Rane, Talia Konkle, Thomas O' O'Connell, Thomas Unterthiner, Andrew Kyle Lampinen, Klaus Robert Muller, Mariya Toneva, and Thomas L. Griffiths. Getting aligned on representational alignment. Transactions on Machine Learning Research, 2025, ISSN 2835-8856. URL https: http:::///ppeerreee http///pppeeeeeieeeeeeeeeeeeeeee net/ ft/fforrm?cc id=
Hiq7lUh4Yn.
[43] Talia Konkle and George A. Alvarez. A self-supervised domain-general learning framework for human ventral stream representation. Nature Communications, 13(1):491, January 2022. doi: 10011000338//4446--222222
[44] Hieu Pham, Zihang Dai, Golnaz Ghiasi, Kenji Kawaguchi, Hanxiao Liu, Adams Wei Yu, Jiahui Yu, Yi-Ting Chen, Minh- Shan-Thang Luong. Yonghui Wu, Mingxing Tan, and Quoc Le. Combined scaling for zero-shot transfer learning. Neurocomputing, 355:126655 October 2023. doi: www.1/166..uuco...o
[45] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Πlya Sutskever. Learning transferable visual models from natural language supervi- sion. In Proceedings of the 38th international conference machine learning, volume 139 of Proceedings of machine learning research, pages 8748-8763. PMLR, July 2021. URL html.
attps://proceette
mlr
[46] Dong C. Liu and Jorge Nocedal. On the limited memory BFGS method for large scale optimiza- tion. Mathematical Programming, 45(1):503-522, August 1989. doi:10. 1007/BB01189911
[47] Lukas Muttenthaler, Klaus Greff, Frieda Born, Bernhard Spitzer, Simon Kornblith, Michael Mozer, Klaus-Robert Müller, Thomas Unterthiner, and Andrew K. Lampinen. Aligning ma- chine and human visual representations across abstraction levels. Nature, 647((889):349-335 0963116666966.. URL
November 2025. ISSN 0028-0836, 1476-4687. https://www. nature...
[48] Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q. Weinberger. On Calibration Modern Neural Networks. In Proceedings of the 34th International Conference on Machine Learn- ing, pages 1321-1330. PMLR, July 2017. URL /ttpps:/prooeeddnggs mlr wal..press/vv//// guo17a.htmr html.
[49] Andrew Kyle Lampinen, Stephanie C. Y. Chan, and Katherine Hermann. Learned feature representations are biased by complexity, learning order, position, and more. Transactions on Machine Learning Research, 2024. ISSN 2835-8856. URL https: http::///p///eeeeeeeeeee net/ forun?id-VV2ssE997
[50] Jöm Diedrichsen, Eva Berlot, Marieke Mur, Heiko H. Schütt, Mahdiyar Shahbazi, and Nikolaus Kriegeskorte. Comparing representational geometries using whitened unbiased- distance-matrix similarity, Neurons, Behavior, Data analysis, and Theory, 5(3), August 2021.
101:10051128/001cc.22
[51] Jasper Jf Van Den Bosch, Tal Golan, Benjamin Peters, JohnMark Taylor, Mahdiyar Shahbazi, Baihan Lin, Ian Charest, Jörn Diedrichsen, Nikolaus Kriegeskorte, Marieke Mur, and Heiko Schütt. A Python Toolbox for Representational Similarity Analysis, September 2025. URL https: httee///eeefffffeeeeeeeeeee
[52] Marco Del Giudice. Effective Dimensionality: A Tutorial. Multivariate Behavioral Research, 56(3):527-542, May 2021. 220.000000000021770000002222...0
14

---

[53] Eric Elmoznino and Michael F. Bonner. nigh-performing neural network models of visual cortex benefit from high latent dimensionality. PLOS Computational Biology, 20(1):e101172.. January 2024. 10t:10.377/jouraallpp 101 1792.
[54] Abdulkadir Canatar, Jenelle Feather, Albert Wakhloo, and SueYeon Chung. A spec- tral theory of neural prediction and alignment. In Advances in neural informa- tion processing systems, volume 36, pages 47052-47080. Curran Associates, Inc., 2023. URL https://proceeddii httprooeeeedinngneeeie..se...eee cc/ wwtppppprfiileeeeeeeeeeeeeeeeeee wol.fffleafaappeee /aaapprr_illeeppppe/222///////// /apprr/022///ille ConferrrC pdf.
[55] Robert Geirhos, Carlos R. M. Temme, Jonas Rauber, Heiko Schütt, Matthias Bethge, and Felix A. Wichmann. Generalisation in humans and deep neural networks. In Advances in neural information processing systems, volume 31. Curran Associates, Inc., 2018. URL https: httt:::://proceeedingg/ hew.eeeedddinnenenne..eee. wwturips.ccppppeeee wwtpccppppefiiieeeeee hillee/paperr2018///il 03375558440001155566655666777 Paper. pdf.
[56] Robert Geirhos, Kantharaju Narayanappa. Benjamin Mitzkus, Tizian Thieringer, Matthias Bethge, Felix A. Wichmann, and Wieland Brendel. Partial success in closing the gap between human and machine vision. In Advances in neural informa- tion processing systems, volume 34, pages 23885 23899. Curran Associates, Inc.,
2021. URL https://proceeddin www.reeeeeddinnnnnnnnneeeeeeeeee ct/ccc//ppp.o 0B87CF1720BB11139555777777773333
Paper. pdf.
[57] Z.. Wang and F. P. Simoncelli. Maximum differentiation (MAD) competition: A methodology for comparing computational models of perceptual quantities. Journal of Vision, 8(12):8-8,
September 2008. doi: 10.1 167/8.12.8.
[58] Tal Golan, Prashant C. Raju, and Nikolaus Kriegeskorte. Controversial stimuli: Pitting neural networks against each other models of human cognition. Proceedings of the National Academy of Sciences, (1747):29330-29333 November 2020. do1:10.1073/pnas.. 190:10..73//pas..11333111 19112341131 17.
[59] Tal Golan, Wenxuan Guo, Heiko H. Schütt, and Nikolaus Kriegeskorte. Distinguishing rep- resentational geometries with controversial stimuli: Bayesian experimental design and its application face dissimilarity judgments. In SVRHM 2022 workshop @ NeurlPS, 2022. URL
www.nne/ooummmdaaaae3mm https: httt:p///pppeeeeeeeeeeee///oc/cp Mf22M.
[60] Jingyang Zhou, Chanwoo Chun, Ajay Subramanian, and Eero P Simoncelli. Comparing neural models using their perceptual discriminability predictions. In Proceedings of UniReps: the first workshop on unifying representations in neural models, volume 243 of Proceedings of machine learning research, pages 170-181. PMLR, December 2024. URL https: /pppp/eeeceeee... mlr. www.ppressvvvee http//223//////// zhou24a. html.
[61] David Lipshutz, Jenelle Feather. Sarah E. Harvey, Alex H. Williams, and Eero P. Simoncelli. Comparing the local information geometry of image representations. In UniReps: 2nd Edition of the Workshop on Unifying Representations in Neural Models, October 2024. URL https: / /ppenreviee. htt/forrumidd-q/RRBe htw.a//wwwewe.aaaaaaaaaassssssae
[62] Baihan Lin and Nikolaus Kriegeskorte. The topology and geometry of neural representations. Proceedings of the National Academy of Sciences, 22244)::2317888122 1211 October 2024. ISSN 0027-8424, 1091-6490. doi: 10. www.117//.n..211111 2317888 231171111111 1211.
[63] Jenelle Feather, Meenakshi Khosla, N. Apurva Ratan Murty, and Aran Nayebi. Brain-Model Evaluations Need the NeuroAI Turing Test, February 2025. arXiv:2502.16238 [q-bio].
[64] Meenakshi Khosla and Alex H Williams. Soft Matching Distance: A metric on neural represen- tations that captures single-neuron tuning. In Proceedings of UniReps: the first workshop unifying representations in neural models, volume 243 of Proceedings of machine learning re- search, pages 326-341. PMLR, December 2024. URL https: / http///eeeeeee.eeeeee mll. .rrss// htm1.
[65] Alex H. Williams. Equivalence between representational similarity analysis, centered kernel alignment, and canonical correlations analysis. In UniReps: 2nd Edition of the Workshop on Unifying Representations Neural Models, October 2024. URL https:///////... httt:::///ppeeevveee net/ Berumiid-aMMmmFFaaCcddssssssss
15

---

[66] Florian P. Mahner, Lukas Muttenthaler, Umut Güçlü, and Martin N. Hebart. Dimensions underlying the representational alignment of deep neural networks with humans. Nature Machine Intelligence, 7(6):848-8555 June 2025. ISSN 2522-5839. 10i:10. 00.103//42225-00255 01041-7.
[67] Jonas Kubilius, Martin Schrimpf, Kohitij Kar, Rishi Rajalingham, Ha Hong, Najib Majaj, Elias Issa, Pouya Bashivan, Jonathan Prescott-Roy Roy, Kailyn Schmidt, Aran Nayebi, Daniel Bear, Daniel L Yamins, and James J DiCarlo. Brain- like object recognition with figh-performing shallow recurrent anns. In H. Wallach, H. Larochelle, A. Beygelzimer, F. dAlché Buc, E. Fox, and R. Garnett, editors, Advances in neural information processing systems, volume 32. Curran Associates, Inc., 2019. URL https: http::/::////preeeein///ee neur hctppppee.flllee/peeeeeeeeeeeeee fttps//aalssppe/////eeppep pdf..
[68] Kohitij Kar, Jonas Kubilius, Kailyn Schmidt, Elias B. Issa, and James J. DiCarlo. Evidence that recurrent circuits are critical the ventral stream's execution of core object recognition behavior. Nature Neuroscience, 22(6):974-995 983, June 2019. doi: 10. 0001008//41533-01--95510
[69] Daniel Yamins and James J DiCarlo. Using goal-driven deep learning models to understand sensory cortex. Nature Neuroscience, 19(3)356-3633 March 2016. 1oi:10. 1038/nn..2444
[70] Bhavin Choksi, Milad Mozafari, Callum Biggs OMay, B. ADOR, Andrea Alamia, and Rufin VanRullen. Predify: Augmenting deep neural networks with brain-inspired predictive coding dynamics. In Advances in neural information processing systems, volume 34, pages 14069- f
neurips. cc/paper_ 14083. Curran Associates, Inc., 2021. URL https: htttss:/pproceeddiiiii Paper .afer.
[71] Joel Dapello, Tiago Marques, Martin Schrimpf, Franziska Geiger, David Cox, and James J DiCarlo. Simulating primary visual cortex at the front of cnns improves robustness to image perturbations. In Advances in neural information processing systems, volume 33, pages 13073- 13087. Curran Associates, Inc., 2020. URL https: https://procceddngss neurips cc/paper_ Paretd--apprppppppp pdf..
[72] Francesco Denti, Diego Doimo, Alessandro Laio, and Antonietta Mira. The generalized ratios intrinsic dimension estimator. Scientific Reports, 12(1):20005, November 2022. ISSN 2045-2322. doi: 10. 1038/s4 11980/441598-002209999999
[73] Aldo Glielmo, Iuri Macocco, Diego Doimo, Matteo Carli, Claudio Zeni, Romina Wild, Maria d'Errico, Alex Rodriguez, and Alessandro Laio. DADApy: Distance-based analy- sis of data-manifolds in Python. Patterns, 3015),00588, October 2022. ISSN 26663899,
1oi:10. patter 100589.
[74] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang. Zachary Vito,, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. PyTorch: An imperative style, high-performance deep learning library. In Advances in neural information processing sys- tems, volume 32. Curran Associates, Inc., 2019. URL htttppp://pooceeeing..eeeeee paper_filess httiiilesspppr////e http:////0/..aaaaaaaaaeeeeeee/a0
cc/
pdf.
[75] Norman Mu, Alexander Kirillov, David Wagner, and Saining Xie. SLIP: Self-supervision Meets Language-Imas Pre-training. In Shai Avidan, Gabriel Brostow, Moustapha Cissé, Giovanni Maria Farinella, and Tal Hassner, editors, Computer Vision - ECCV 2022, pages 229-544, Cham, 2022. Springer Nature Switzerland. ISBN 978-3-031-19809-00
[76] Laura M. Stoinski, Jonas Perkuhn, and Martin N. Hebart. THINGSplus: New norms and meta- data for the THINGS database of 1854 object concepts and 26, natural object images. Behav- ior Research Methods, 56(3): April 2023. ISSN 1554-3528. doi:10..755/s13428. 023-02110-8.
[77] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. In International conference on learning representations, 2021. URL. https:// htt////rrrrrrr..eetooomoot////ie
16

---

[78] Hugo Touvron, Matthieu Cord, and Hervé Jégou. DeiT III: Revenge of the VIT. In Computer Vision - ECCV 2022, pages 516-533, Cham, 2022. Springer Nature Switzerland. ISBN 978-3- 031-20053-3.
[79] Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy V. Vo. Mare Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel HAZIZA, Francisco Massa, Alaaeldin El-Nouby, Mido Assran, Nicolas Ballas, Wojciech Galuba, Russell Howes, Po-Yao Huang, Shang-Wen Li, Ishan Misra, Michael Rabbat, Vasu Sharma, Gabriel Synnaeve, Hu Xu, Herve Jegou, Julien Mairal, Patrick Labatut, Armand Joulin, and Piotr Bojanowski. DINOv2: Learning robust visual features without supervision. Transactions on Machine Learning Research, 2024. ISSN 2835-8856. URL httttp::///ppeeevieeww net.eewwwweeeeeeeeee
[80] Mathilde Caron, Ishan Misra, Julien Mairal, Priya Goyal, Piotr Bojanowski, and Armand Joulin. Unsupervised learning of visual features by contrasting cluster assignments. In Advances in neural information processing systems, volume 33, pages 9912-9924. Curran Associates, Inc., 2020. URL 70feb62b69f 000.0000160000000111111122222880
neurips. cc/ caccppper_feee lameer_ffleee files/ wwwessppaeeee/eeeeeeeeeeeeaeeeee pttppppppp///////////// pdf..
[81] Zhiliang Peng, Li Dong, Hangbo Bao, Qixiang Ye, and Furu Wei. BEiT v2: Masked Image Modeling with Vector-Quantized Visual Tokenizers, October 2022. URL
org/
abs/2208.06388
arXiv:2208.06366 [cs].
[82] Adrien Bardes, Jean Ponce, and Yann LeCun. VICReg: Mariance-lvarriaee--Cvaaan Regularization for Self-Supervis Learning. In International Conference on Learning Repre- sentations, October 2021. URL pestpp:://ppereeeee
[83] Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. In Yoshua Bengio, editor, 3rd international conference on learning represen- tations, ICLR 2015, San Diego CA, USA, may 7-9, 2015, conference track proceedings, 2015. URL. http: http::///xxxvvvvie Baarrg/bbs/100/ wbtp///9.... 15566.
[84] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In 2016 IEEE conference on computer vision and pattern recognition (CVPR), pages 770-778, 2016. doi:10. 101:101110//CPR22206.90
[85] Jure Zbontar, Li Jing, Ishan Misra, Yann LeCun, and Stephane Deny. Barlow twins: Self- supervised learning via redundancy reduction. Marina Meila and Tong Zhang, edi- tors. Proceedings of the 38th international conference on machine learning, volume 139 of Proceedings of machine learning research, pages 12310-12320. PMLR, July 2021. URL https://proceedingg. mlr press/ press//v39) ht/p//111////// 3t333//bonttrr1111 html.
[86] Gao Huang, Zhuang Liu, Laurens van der Maaten, and Kilian Q. Weinberger. Densely connected convolutional networks. In 2017 IEEE conference on computer vision and pattern recognition, CVPR 2017, honolulu, HI, USA, july 21-26, 2017, pages 2261-2269. IEEE Computer Society, 2017. 10. 109/CVP..201..243..
[87] Zhuang Liu, Hanzi Mao, Chao Yuan Wu, Christoph Feichtenhofer, Trevor Darrell, and Saining Xic. A ConvNet for the 2020s. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recogmition (CVPR), pages 11966-1 1976, New Orleans, LA, USA, June 2022. IEEE. ISBN 978-1-6654-6946-33 10. 100.1199922R5228822222.00
[88] Mingxing Tan and Quoc Le. EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks. In Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pages 6105-61 14. PMLR, June 2019. URL attps://proceedingg mlr prtess/v97/tan/.e html.
[89] Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna. Re- thinking the inception architecture for computer vision. 2016 IEEE conference on computer vision and pattern recognition (CVPR), pages 2818-2828 2016. doi: 1091109P..0..0...00
[90] Barret Zoph, Vijay Vasudevan, Jonathon Shlens, and Quoe V. Le. Learning transferable architectures for scalable image recognition. In 2018 IEEF/CVF conference on computer vision and pattern recognition, pages 8697-8710, 2018. 1tt.10.119/C/PP...0. 2000000000..
17

---

[91] Mehdi Cherti, Romain Beaumont, Ross Wightman, Mitchell Wortsman, Gabriel Ilharco, Cade Gordon, Christoph Schuhmann, Ludwig Schmidt, and Jenia Jitsev. Reproducible Scaling Laws for Contrastive
Learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 2818 2829, 2023. URL https: //openaccess thecvf com/ hwwptteett/cvPR222///aaea Cter/hhhrrtteee Rttpphhhrii.eeeoouuuubbbbeeee
[92] Chao Jia, Yinfei Yang. Ye Xia. Yi-Ting Chen, Zarana Parekh, Hicu Pham, Quoc Le, Yun-Hsuar Sung. Zhen Li, and Tom Duerig. Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision. In Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pages 4904- 4916. PMLR, July 2021. URL https: httpss///ooocdddgeeee mlr wwr.mrrreeeeeeee.. wwwpp/eesseveeee jialbb..bbb... html..
[93] Amanpreet Singh, Ronghang Hu, Vedanuj Goswami, Guillaume Couairon, Wojciech Galuba, Marcus Rohrbach, and Douwe Kiela. FLAVA: Foundational Language And Vision Alignment Model. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 15617-15629, 2022. doi:10.
[94] Michael Tschannen, Alexey Gritsenko, Xiao Wang, Muhammad Ferjad Naeem, Ibrahim Alab- dulmohsin, Nikhil Parthasarathy, Talfan Evans, Lucas Beyer, Ye Xia, Basil Mustafa, Olivier Hénaff, Jeremiah Harmsen, Andreas Steiner, and Xiaohua Zhai. SigLIP 2: Multilingual Vision- Language Encoders with Improved Semantic Understanding, Localization, and Dense Features, February 2025. URL http:://aa/tttt orgoorgg///....ooooooo at////bb/////// 2502. 14786. arXiv:2502.14786 [cs].
[95] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton, ImageNet Classification with Deep Convolutional Neural Networks. In Advances in Neural Information Processing Systems, vol- ume 25. Curran Associates, Inc., 2012. URL Utt. nwwrips. cc/paper_ pdf.
[96] Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. BLIP-2: Bootstrapping Language- Image Pre-training with Frozen Image Encoders and Large Language Models. In Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pages 19730-19742. PMLR, July 2023. URL https: ptttps://proceet stttps://proceeddnggs mlr. press/v202/112225 html.
[97] Yuxin Fang, Quan Sun, Xinggang Wang, Tiejun Huang, Xinlong Wang, and Yue Cao. EVA- 02: A Visual Representation for Neon Genesis. Image and Vision Computing, 149:105171, September 2024. ISSN 02628856. doi: m04.....maaiss.222..1.0.
arXiv:2303. 11331 [cs].
[98] Daniel Bolya, Yao Huang, Peize Sun, Jang Hyun Cho, Andrea Madotto, Chen Wei, Tengyu Ma, Jiale Zhi, Jathushan Rajasegaran, Hanoona Abdul Rasheed, Junke Wang, Marco Monteiro, Hu Xu, Shiyu Dong, Nikhila Ravi, Shang- Wen Li, Piotr Dollar, and Christoph Feichtenhofer. Perception Encoder: The best visual embeddings are not at the output of the network. In The Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025. URL /ttps:///og /ttp://///peeeeiieeeeee net/ forummidd dd-InmdIIIINNNNII......
[99] Rohit Girdhar, Alaaeldin El-Nouby, Zhuang Liu, Mannat Singh, Kalyan Vasudev Al- wala, Armand Joulin, and Ishan Misra. ImageBind One Embedding Space to Bind Them All. In 2023 JEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 15180-15190. Los Alamitos, CA, USA, June 2023. IEEE Computer Society, 1o.: 119./C/ 090.72...02.01455777 0..1109/CVVPR522 2023.. 01457.
URL https://dot fttt::://dbi.ieeecoopptteee ieeecomputersociety org/
18

---

A Appendices
A.1 Scalar-matrix shrinkage regularizer
Lemma 1. Let W E IRXXP. Define
R(W) min TER
W
>
Then
tr (T))
R(W)
W }
p
Proof. Using |A At)
W 1 = tr((W II" - -1)] -W - 2y 2y tr(W) + = f(7).
Because
f(7) pr² - 2 (r(rr((W /
(()) 2p7 tr(W),
(" 2p 0,
the unique minimizer is
7" tr(W)
p
Substituting > into f(r) yields
min f(r) = || tr tr(
Notation. Throughout, let p N be the dimension of the square matrix W E TPPP; that is, W has rows and p columns. Because counts rows/columns it satisfies 2 1, hence > 0. This fact ensures the quadratic f(>) = pr² tr( (r(W)> |[ [| strictly convex: its second derivative is ("(n) 2p > 0, guaranteeing unique minimizer y in Lemma 1.
A.2 Calibration to human noise ceiling
To ensure that the simulated responses were realistically distributed, optimized each data- generating model's softmax temperature so that its simulated noise ceiling matches the empirical noise ceiling, estimated from subset of the 1,000 triplets, each presented to approximately 30 participants.
dataset that includes responses to
A.2.1 Noise eciling estimation
Let Co denote the number of participants who chose the stimulus in position a E {1,2,3} as the odd-one-out for triplet E NN).
(1) (cl :2 ,2))
())
T
i=1
For each i (1,2,3) define
c -00 c,
V.th {i E (1,2,3) 1 max (t) cik)
where e; the i-th standard basis vector 3 The leave-one-subjjettour (LOO) accuracy for triplet t is
3
Li w
(4)
T
1=1
19

---

and the overall LOO noise ceiling is
N N
NC
A.2.2 Temperature calibration
We calibrated the softmax temperature T of each data- generating model that the model's predicted noise ceiling matches the one estimated from human data.
Empirical noise ceiling. Let Meeil 0.678 denote the noise ceiling estimated from the THINGS odd-one-out triplet judgments using specific repeated set across participants Deal- This triplet set was used exclusively during the calibration phase.
Model-estimated noise ceiling. Let Deat be our calibration set of triplets drawn from the same pool. For each triplet {a, b, c/v E Deal, the model assigns the following probability:
exp(S,,=/T exp(S.,p/T) + ex (T) exp(Sr.c/T)
p(odd-one-out- triplet,)
where {v. = fa, b, c} {a}. We then define the model's noise ceiling as the average, over all calibration triplets, of the model's maximum (top-choice) probability:
Peall
Rooil (T)
max plodd-one-out (riplet.). =E(a,b.)
Deatl
Optimal temperature. We choose T to minimize the squared deviation between the empirical and model-estimated noise ceilings:
T arg min T
Meeil Recil (T))²
(9)
This procedure guarantees that, on average, a noise ceiling estimated from simulated responses to the calibration triplet set would match the empirical human noise ceiling.
A.3 Sampling random triplets
To test model recovery under general conditions, the simulation study used random triplets not included the THINGS odd-one-out dataset. each simulation, we randomly partitioned 1,854 images into three equally-sized disjoint subsets. One subset served as the test-image pool, and the remaining two subsets were concatenated and then randomly split into training (80%) and validation (20%) image pools. Using these pools, randomly sampled 50 to 5.25 million triplets (spanning logarithmically spaced range of stimulus set sizes), such that 80%, 10%, and 10% of the triplets were drawn from the training, validation, and test pools, respectively. No triplet included images from more than one split or overlapped with THINGS odd-one-out. Candidate model predictive accuracy (see Eq. 10) was evaluated and averaged across the three validation folds, each using different test-image pool assignment. Thus, in each simulation, each image appeared in the test pool in exactly one of the three crossvvaliatiinn folds.
A.4 Model-recovery formulation
This section describes the model recovery simulations in detail.
20

---

Notation:
Let W E RPXP denote linear transformation matrix that maps neural network represen- tations into target representational space (either behavioral model-generated)) This transformation is used to align models to human judgments or to other models' simulated responses.
Let M, denote the i-th neural network model.
Let M = [....... Mx) be the set of N pretrained encoders.
Let т denote the behavioral dataset consisting of human odd-one-out judgments.
Let W E IR*xP denote the linear transformation W which maps the p-dimensional features M, into a target response space.
Throughout, the notation W Wonrce- starget emphasizes that features from the model on the left are being aligned to judgments (or predictions) associated with the target on the right.
Model-to-Behavior Alignment. W M is the transformation matrix learned to map representa- tions from model M, to best predict responses from the THINGS odd-one-out dataset. We use the shorthands M,-,r and g:
M fM, W WM,++);
g (M MN
Model-to-Model Alignment (Simulated Data). model Mj M,, we fit
Given generator Mi E G and a candidate
W (
RPxp, Mi
M [M,, W(d) My-MM...)
separately for every simulated dataset d E f1...., D} so that M best predicts the synthetic responses of Mi-s7.
Model Recovery Experiments
Prediction vectors. Let d(k) be the k-th triplet in d. Let (d()) {1,2,3} be the true label of the (k) triplet from generator M Similarly, let ūز (d(K)) be the predicted labels of M For a full dataset d:
{)-((d) {ūj (d)-)(d(1)), ................................ 0J (d(k)) (d) (o .... + (d (K))
where K = [d].
Candidate model predictive accuracy For each (M,, My), let {dirains drest) be partition of dataset d. Candidate model predictive accuracy defined by:
1 deel
Acc(M
(dran) ((aa)) [
(cdhe) (c(ett) 3i-er (due))
(10)
|deest
1=1
where [-] equals if the condition is true and otherwise.
Model-recovery accuracy
The model recovery accuracy can be defined as:
Σ Σ Mcc(MM/4e
(dran) [M+7) max Acc( (dem) JE(1....[IM)) j-s
(dmm)))
M
(11)
M
This the fraction of simulations in which the candidate model with the highest predictive accuracy matches the true generator.
21

---

A.5 Estimation of effective dimensionality
For each model, we compute the D D covariance matrix of the activations in its deepest repre- sentational layer (where D is the number of units in that layer, details in Appendix A.8), across the 1,854 images from the THINGS dataset, obtain the eigenvalues (Aş) of this matrix using Principal Component Analysis (PCA), and then calculate:
ED EE 2₂) EP1 2
(12)
This quantity (labeled 222 in [52]) estimates the participation ratio-the effective number of principal components contributing to the total variance [52, 53]. We repeated this procedure for model activations obtained after applying the human-aligne linear transformation (Section 2.2).
A.6 Estimation of Intrinsic Dimensionality
We estimated intrinsic dimensionality (ID) with GRIDE (Generalized Ratios Intrinsic Dimension Es- timator) [72], using its implementation in the dadapy Python package [73] with default settings. For each model, extracted activations for the 1,854 THINGS images from its deepest representational layer (details in Appendix A.8), yielding a 1,854 D activation matrix (where D is the number of units in that layer). We next applied the learned linear human-alignment transform W fit to predict the THINGS odd-one-out responses (Section 2.2) and computed ID on both the original and transformed activations. Given n data points { R we compute Euclidean nearest-neighbor
distances and, at multiple scales, form the ordered-neighhor distance ratios j4, = Pi.xa ma = r /r with n2 > n1 2 1 (the chosen neighbor orders that set the scale). GRIDE then estimates the intrinsic dimension d by maximum likelihood under local homogeneity (Poisson) assumption, maximizing the log-likelihood
= arg max [n log(d) + (12-m111) Σ log log(ū, (1) log(B(n2-m1, n1)) ((n2-1)d+1) Σ log(At))
(13)
where :) denotes the Beta function.
The final ID estimate, d, is determined by identifying a stable "plateau" in the ID values across the dif- ferent scales. This stable value represents the representation's intrinsic dimensionality, distinguishing from noise that typically appears at very small scales.
A.7 Resources
Simulations. We conducted large-scale simulations in which each of 20 models acted both as data generator and as a candidate model. For each data Sata-gennratiig model, we sampled 30 random datasets per experimental condition (triplet set size).
Our main analysis covered 18 triplet-set sizes, yielding 18 x 600 = 10,800 simulations for the results shown in Fig. 1. For every run we selected the optimal regularization coefficient from ten values (logarithmically spaced from 10-6 6 105) via valis-vvlii
Simulations were conducted on the BGU ISE-CS-DT cluster, mainly using a server with eight NVIDIA RTX 6000 Ada GPUs. All preprocessing and analysis were performed local workstation. Simulation runtime depended on transformation flexibility, dataset size, model count, and random seed initialization. The mean runtime was approximately 9.5 minutes per simulation, so reproducing the 10,800 simulations of Fig. I on eight RTX 6000 GPUs would require roughly nine to ten days.
A.8
Implementation details
Feature vectors extracted using in-house Python package that wraps torchvision, Hugging Face, and TorchHub [74]. Meta Al models were obtained from the SLIP repository [75].
Feature extraction.
For all models, the deepest representational layer was extracted. Specifically, if the ultimate layer encoded predictions (c.g., logits in classifiers), we extracted the penultimate layer activations. If the
22

---

ultimate layer encoded embeddings (e.g., as in self-supervised or image-tex, text-aligned models), we extracted the ultimate layer activations.
RSA and MDS computations. Representational similarity analyses (RSA) [20] and multidimen- sional scaling (MDS) were performed with the RSA Toolbox for Python [51].
23

---

B Supplemental Figures
0.60
0.35
test accuracy
0.50
0.45
0.40
regularization
R(W) = w x/
R(W) [W]]
(W dhentgerr
Reseray
Rerany
nw arreery
goveny
Revely
(w apeerey
vey
nw gonnng
Dw tos
(W Rexey
flexibiity
Figure S1: To verify that ecalar-matrix shrinkage regularization (Eq. 4) does not impair model predictive accuracy, we evaluated all models on the THINGS odd-one-out dataset and compared the results obtained using Frobenius norm-based regularization to those obtained using scalar-matrix shrinkage regularization. Each violin plot depicts the distribution odd-one-out predictive accuracy across models under the two regularization methods. The x-axis indicates levels of transformation matrix flexibility, from diagonal (left) to unconstrained (right). As evident from the overlapping distributions, observed meaningful differences in predictive accuracy between the methods at any regularization level. A two one-sided tests (TOST) procedure with A = 0.05 was used to assess the equivalence of the regularization methods' means: a Bonferroni correction for multiple comparisons was applied, indicating statistical equivalence across all levels of flexibility. Note, however, that this equivalence holds for optimal regularization. When the transformation is over- regularized, the scalar-matrix shrinkage regularization pulls the predictions toward the zero-shot solution, whereas the Frobenius norm-ban based regularization shrinks the transformation matrix toward the zero matrix, pulling the predictions toward a uniform distribution and thus impairing performance.

---

Fil and calirrtte to THINGS odd-one-out
Model2
designated cata-generating model in this simulation
Sample random triplets
synthetic responses
orplad
rriporase
Va lanion
Hpaa
Model
Model
Model
Model Mt
reppoieto
Fitting to synthetic data
Model
Model
Model
Model
M
Ma
M Mumidd
W 34.
Models predictive accuracy evaluation
56%
63%
165%
80%
Falled model recovery: Model 3 achieved the highest prediction accuracy although was not the data generating model
Figure S2: An illustration of one model recovery simulation. The designated data-generating model is fitted and calibrated to the THINGS odd-one-out dataset. Then, using three disjoint sets of images, we generate training, validation, and test of randomly sampled triplets with corresponding simulated responses, All candidate models, including the one that originally generated the data, are fitted with model-specific linear transformation matrix using regularization hyperparameter selected based on the validation set. After fitting, each model predicts the responses to the test set triplets, and its predictive accuracy is evaluated. This illustration demonstrates case of misidentification, where the model that most accurately predicts the synthetic data is not the model that originally generated it. The natural images used in this illustration were taken from CCO-licensed set of 1,854 images corresponding to the same concepts in THINGS [40], included in THINGS+ [76].
25

---

A
10K
37W
0.55
12:00
25.80
204
training triplets
B
C
D
data-gonerating model
number of mre.noee
2000
recovered model
Figure S3: Same model recovery analysis as in Figure I, while using the top 500 principal components (PCs) of each model's representation instead of its original features. For each model, conducted principal component analysis on its final representational layer activation patterns in response to the ImageNet-Iie 1K validation set images and retained only the scores of the top 500 PCs. This dimensionality reduction was applied to the models' representations both when the models generated the data and when served candidate models. Consequently, all fits used 500 x 500 transformation matrices.
(A) Model recovery accuracy under linear probing with 500 500 matrix across different dataset sizes. (B, C, D) Confusion matrices at three training set sizes (400, 25.6K, and 1.6M triplets). Each matrix row corresponds
data-generating model, and each matrix column to recovered model. Diagonal entries represent correct model recovery. Between-model differences in transformation dimensionality do not explain model misidentification.
26

---

recovery
1BM
number training triplets
B
Figure S4: Model recovery simulations with an extended model set. We reran the model recovery simulations with 10 additional models (see Table S5), most of which were image-text aligned. To limit the computational cost, we used 10 simulations per experimental condition compared to 30 used in the main analysis (Fig. 1), and 10 different training dataset sizes compared to 18. Other than these changes, the analysis the same as Figure 1. (A) Model recovery accuracy under linear probing across different training sizes. (B-D) Confusion matrices three training set sizes (400, 25,600, and 1.6 million). As expected, adding more models reduced model recovery accuracy: it plateaued around 70%, compared to 80% in the original 20-model experiment (Fig. 1).
27

---

A
0.9
family recoery
0.8
recovery accuracy
0.7
0.8
B
0.2
0.1
0.0
number of training triplets
B
D
data-generating objective
120
supervised
51
70
number of smulatioos
100
00
self supervised
23
1
36
0
2
58
0
31
29
60
Image toat alignment
47
7
66
0
120
0
120
20
image gignmenn
superviedd
image axperviedd
supervised
suparalsed
superdised
wignmert
recovered objective
F
G
data-generating architecture
140
number of simulations
120
71
89
106
100
80
61
79
16
124
140
recovered architecture
Figure S5: Model recovery accuracy analysis, grouped by objective architecture type. Using the extended-model set simulations described in Figure S4, we inspected how well coarser model characteristies-ojeeee
type or architecture type-can can be recovered. For objective type. we categorized each model's objective as supervised, self-supervised, or image-text alignment. For architecture type, we categorized each model's architecture as convolutional or vision transformer. We then evaluated model recovery accuracy at the category level, defining correct recovery as cases where the best-performing candidate model belonged the same group as the data- aatta-generating model. Panel A shows model recovery accuracy across different training set sizes. The green line indicates recovery accuracy between objective types, and the blue line indicates recovery accuracy between architecture types. The grouped recovery accuracy for only 100 training triplets (confusion matrices shown in B and E) exceeded that of the non-grouped analysis (Fig. S4), owing the higher chance level associated with those conditions. Even with 4.2 million training triplets (confusion matrices shown in and G). the accuracy reached only about 72% for objective and 70% for architecture type. This result indicates that linear probing may obscure not only individual model identity but also broader representational motifs.
28

---

0.0
0.2
0.4
0.6
0.8
10
Whitened Pearson correlation
Barlowlliins RM50
ВЕTТ2
ConvNext
OpenAi CUPP RN50
Original representations
CORnet
DINOV2
DenseNel201
EfficientNet B7
Meta All CLIP
SimCLR
VIT L/16
InceptionR
Inceptionna
ResNier50
VicRleg
VGG19
NASNet
VICE
Barlow RN50
Transformed representations
Convieett
OpenAl CLIP RN50
CORnet 5
Dert3
DIMOv2
EmcclentNet.
Meta All CLIP
Meta Al SimCUR
Meta SUJP
Inceptionna
ResNerS0 SWAV RN50
Vickeg RN50
NASNet
VICE
BarfowTwinn RM50
BETT22
OpenA) Convieit
RNS0
Dolr
DINO22
DentrcNee200
Erficlentier
SimICLR
Meta SUP
VIT L/26
InceptioonWW/
Incepttinns
Ress RN50
Vidlleg RN50
VGG15
NASHeR
VICE
BET-2
Convient
Openal CUP RA50
CORnett
PanseNetzio
Edficienhht Met 87
Meta CLIP
Meta Al SIMCER
Meta VIT SUP
Inceptioo Inceeptionei22 inceeptione Res Massee50 SWAY RNNSO
VOGSS 19
NASHet! VICE
Original representations
Transformed representations
Figure S6: Similarity of model representations before and after alignment THINGS odd-one- out. For each network, we computed a squared Euclidean representational dissimilarity matrix (RDM; 1,854 1,854) on the THINGS images and vectorized its upper-triangule entries. Each cell above shows the whitened Pearson correlation between two such RDM vectors that is, the ordinary linear correlation after whitening by the sample covariance of RDM entries, which discounts shared-variand artifacts [50, 51]. Rows and columns are arranged in pairs: the first entry represents the model's s original features, and second entry its transformed features obtained from alignment to THINGS odd-one-out, estimated as part of our recovery experiments. The embeddings of VICE [41] were used without any further fitting, since this model was trained to human odd-onc-out responses. sooveen-model dissimilarities: The upper-left quadrant shows that unaligned models occupy distinct representational geometries; the off-diagonal correlations span a broad range (typically 0.2-0.5), reflecting diverse unaligned model representations. The lower-right quadrant displays a bright band of high off diagonal similarity, indicating that alignment drives disparate models toward a shared geometry, eroding their "individuality." Within-model representational shift: The block-diagonal formed by each model's original vs. transformed RDMs reveals how far a model's ''s geometry moves under the linear transform. We quantify this shift as daan 1 Pwhitened R RDM RDMaligned) so larger values reflect greater internal reorganization. These within-model shift scores serve as the "alignment-induced representational shift" predictor in the regression analysis (see Table $2) and visualized as green arrows in Figure 4.
29

---

C
Supplemental tables
Model
Objective
Zero-shot Ipxp
Diagonal Diag(RPXs)
Rectangularg E: Rexa0
Full
Reference
W e Raxy
OpenAI CLIP RNS0
Image/Text contrastive
0.4784
0.5577
0.5941
0.5959
[45]
VIT L/16
Image classification
0,5370
0.5558
0.5937
0,5927
[77]
DeiT3
Image classification
0.4651
0.5201
0.5809
0.5786
Meta AI L
Image- Text contrastive
0.4386
0.4865
0.5751
0,5761
[75]
SimCLR
Self-superiised contrastive
0.3883
0.5095
[75]
DINOv2 L
Self-distillation
0.4127
0.5282
0.5721
0,5645
[79]
Meta C
Image Text contrastive
0,4406
0.4882
0.5628
0.5636
[75]
SwAV RN50
Self-supervised clustering
0.4118
0.5015
0.5635
0,5616
B
Masked image modeling
0.4176
0.5133
0.5634
0,5605
[81]
VicReg
regularization
0.4387
0.5186
0.5610
0.5584
[82]
VGG19
Image classification
0.4679
0.5214
0.5583
0,5567
[83]
ResNet50
Image classification
0,4568
0.5080
0.5584
0,5562
[84]
BarlowTwins RN50
Redundancy reduucioo SSL.
0.4360
0.5072
0.5603
0,5559
CORnet-S
Image classification
0.4308
0.4815
0.5530
0,5526
[67]
DenseNet201
Image classification
0.4323
0.4961
0.5523
0,5518
ConvNeXt
Image classification
0.4493
0.4749
0.5367
0,5347
[87]
EfficientNet
Image classification
0.4088
0.4601
0.5281
0.5249
[88]
Inceptionv3
Image classification
0.3817
0.4198
0.5206
0.5184
NASNet
Image classification
0.3799
0.4344
0.5158
0,5110
[90]
InceptionRNv2
Image classification
0.3689
0.3989
0.5124
0.5067
Table S1: Candidate models used in this study and their cross-validated prediction accuracy on the THINGS odd-one-out dataset under varying alignment flexibility: zero-shot (W -Ixp), diagonal, rank-30 rectangular, and full square transformation. Features were extracted from each model's s final representational layer (dimension p) and evaluated via 3-fold cross-validation over disjoint image sets. To ensure each model was evaluated under favorable conditions, we used its largest publicly available variant. The model set spans self-supervised, masgeccaasification, and image-text alignment objectives. Note that models with substantial architectural and functional differences can achieve similar predictive performance.
30

---

Model Nome
Paacueterrs
#Features
Original
Transformel
Alignment inderett-indeedddd representational shift
Original Reatures
Trauofurmed features
Thine
25557012
2048
240.64
156.77
29.7
BeiV2 Lange
303405565
1024
234.07
57.19
0.53
23.13
20.51
ConoNeXi Large
197767336
103.15
45.25
27.64
UpenAI CLIP
RNSO
100001137
1034
47.3 20
16.13
0.17
17:37
14.49
CORteet S
53415616
3.00
23.93
20.62
15.25
Dent Hage
030665680
1251
83
59.51
0.38
16.13
16.90
DINO22 Largs
104366600
1034
476.25
74.21
0.54
26
25.35
DenseNat201
20013038
154.78
45.20
28.006
EmiciensNNe B7
66547960
2550
135.40
218.36
0.55
23.43
36,44
Large
367354017
20.58
Large
MELAl SinCLR
125345560
1024
$2.40
35.06
0.48
18.9
23.71
MERAI SLIP
389298945
512
101.65
35.37
0.48
25
22.08
303301632
$8.30
50.01
22.49
22.09
Inoe)WooRXV
54305454
1536
68.30
53.23
0.62
20.11
19.51
Inception
RoNNaS0
25557032
2048
130.50
84.88
0.39
28.27
24.73
SWAY RN5O
25557032
140.06
133.86
26.17
VicReg RN50
23505032
2048
225.99 99
119.59
0.45
29.87
33.52
VOGIV
163667240
100.72
134099
25.43
NASNi Large
84720150
238.65
117.97
33.40
27.57
Table S2: Model characteristics used as predictors in the regression analyses. For each model, we recorded the following predictors:
#Parameters total number of trainable parameters across all model layers.
#Features number of units in the model's final representational layer.
Original features ED the effective dimensionality (ED; see Appendix A.5) of the model's final representational layer activation patterns in response to the 1,854 THINGS images, measured before any transformation.
We included these first three predictors to test whether there are inherent, non-alignment-relatee properties each model that play a role in model misidentification
Transformed features ED the effective dimensionality of the model's final representational layer, obtained after applying the fitted and calibrated linear transformation W.
Alignment-induced representational shift- the representational dissimilarity between original
and transformed feature spaces derived from the representational similarity analysis (RSA; Fig. S6), which captures the representational shift induced by the linear transformation.
We included the latter two predictors to test whether there are representational alignment-induced geometric changes that are associated with model misidentification.
As an alternative to effective dimensionality, we also considered the following two predictors: Original feettures ID- the intrinsic dimensionality (ID; see Appendix A.6) of the model's final representational layer, obtained before any transformation.
Transformed features ID- the intrinsic dimensionality of the model's final representational layer, obtained after applying the fitted and calibrated linear transformation W.
To avoid multicollinearit. the effective dimensionality measures were included in the analysis reported in Table S3, and the intrinsic dimensionality predictors were included in a separate regression analysis, reported in Table $4.
All model characteristics were extracted or estimated directly from the specific model implementations we used to ensure accuracy and reproducibility. Prior to conducting the regression analyses, the feature tables were expanded to account for all 380 pairwise model comparisons in each of the 30 simulations, yielding 11,400 observations and 10 predictors.
31

---

Predictor
Candidate/Generator
%
p-value
Transformed features ED
Data generating model
0.455
[-0.440, 0.1755
0.02
Transformed features ED
Candidate model
0.182
[-0.967, 0.079]
1
Original features ED
Data generating model
0.069
[-0.259, 0.181]
Original features ED
Candidate model
-0.008
[-0.371, 0.328)
#Features
Data generating model
0.208
[-0.419, 0.059]
#Features
Candidate model
0.264
[-0.072, 0.764]
0.924
Aligament-induese representational shift
Data-generating
0.228
[-0.442, -0.117]
0.02
Alignment-induced representational shift
Candidate model
0.495
[0.286, 0.841]
0.03
#Parameters
Data generating model
0.035
[-0.122, 0.165]
#Parameters
Candidate model
-0.145
[ [-0.45, 0.077]
1
Table S3: Regression analysis identifying drivers of misidentification.
To determine which model properties (Table S2) may cause model to be misidentified by its own simulated responses, we regressed the pairwise accuracy gap (retrained data-generating model candidate model; negative values indicate misidentification) on model features across 11,400 com- parisons from 600 simulations (20 models x 30 datasets x 20 candidates 20 x 30 same model comparisons). We used the largest simulated dataset condition, each including 4.2 million training triplets. Predictors for both data-generators and candidates comprised:
(1-2) effective dimensionality (ED) on THINGS stimuli before and after linear alignment.
(3) The number of units in the model's final representational layer.
Alignment-induced representational shifts between original and transformed feature spaces de- rived from the model similarity matrix (Fig. S6).
(5) Total number of trainable parameters.
As used these predictors for every given pair of data-generating and candidate models in each simulation, we obtained total of ten regression coefficients. An intercept term was included. To estimate uncertainty, performed 10,000 bootstrap replicates at the model level: each replicate resampled with replacement, the 20 model identities from the original set to define both the data- generator and candidate pools (allowing duplicates), retained all 30 random seed initializations, and refit the regression to obtain empirical coefficient distributions. Table entries report the standard- ized coefficient (3), its 95% percentile bootstrap confidence interval, and bootstrap- p-value, corrected for the ten coefficients. Candidate model dissimilarity (B = 0.495, p-value = 0.02) and data-generating model post-alignment ED -0.455, p-value 0.02) are the strongest significant drivers of misidentification. highlighting the critical role of transformationiinduce geometric shifts.
32

---

Predictor
Candidate/Generator
/
95 % Cl
p-value
Transformed features ID
Data generating model
00.40
[-0.78, 0.28)
0.2
Transformed features ID
Candidate model
0.37
[-0.76,-0.09)
0.17
Original features ID
Data generating model
0.08
[-0.11, 0.34]
Original features ID
Candidate model
0.01
[-0.21, 0.28)
#Features
Data generating model
-0.39
[-0.71, -0.20)
0.03
#Features
Candidate model
0.26
[0.04, 0.60]
0.3
Alignment-ind induuedd representational shift Alignment-inducee representational shift #Parameters
Data-generating
0.31
[-0.76, -0.15]
0.03
Candidate model
0.65
[0.41,111]]
0.004
Data generating model
0.20
[-1.02, 1.8]
#Parameters
Candidate model
-1.2
[-2.82,0.17] ]
0.78
Table $4: Regression Analysis Using Intrinsic Dimensionality Measures.
To further explore which model properties (Table S2) may cause model to be misidentified by its own simulated responses, we ran the same regression analysis in Table $3, replacing the ED measures with ID) measures. None of the II) measures- for either the candidate or data-generating model, and for both the original and transformed features-went were significant. Alignment-induced representational shift became significant for both the candidate model (3 0.65, p- p-value = 0.004) and the gata-generating model (8 -0.31, p-value = 0.03), highlighting the effect the alignment- induced shift model misidentification.
Model
Objective
Zero-shot W Ipxp
Diagonal Diag(Re*p)
Rectangulargp W R
Reference
OpenCLIP ComvNeXt XI.
Image-Tec Text contrastive
0.4267
0.5065
0.5684
0.5599
[91]
ALIGN
Image-1u Text contrastive
0.4252
0.5274
0.5957
0.59324
FLAVA Full
Image-Tese Text contrastive
0.4738
0.4738
0.5833
0.5834
[93]
SigLIP2
Image
0.4497
0.6114
0.6120
[94]
AlexNet
Image classification
0.4518
0.4900
0.5356
0.5300
[95]
Image-Texy contrastive
0.3758
0.4063
[96]
ENA02 CLIP Enormous
Image- Texpe-Teeteee contrastive
0,5030
0.5686
0.6128
0.6108
[97]
PE Huge
Image- Text Contrastive
0.4803
0.5723
0.6129
0.6100
[98]
Image Bind Huge
Multinodal Contrastive
0.4621
0.4621
0.5336
0.6069
[99]
OpenAI CLIP VIT
Image-Ter Text Contrastive
0.4191
0.5642
0.6057
0.60482
[45]
Table S5: Prediction accuracies of the 10 additional models used in the small-scale model recovery simulations for the grouped model recovery results presented in figure S5. The models were evaluated on the THINGS odd-one-out triplets using cross-validated prediction accuracies under varying alignment flexibilities: zero-shot (W = Ip). diagonal, rank-30 rectangular transform, and full square transform. Features from each model's final representational layer (dimension p) were evaluated using 3-fold cross-validee validataai over disjoint image sets. To ensure each model was evaluated under favorable conditions, we used its largest publicly available variant.
33

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The abstract clearly states, "Model comparison experiments must be designed balance the trade-off between predictive accuracy and specificity." The introduction directly refers to Fig. 2 as evidence supporting the main claim that motivated our research. The introduction ends with clearly stated bullet points summarizing our contributions.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as is clear that these goals attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
Our discussion section includes detailed Limitations section (see Section 4).
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "I "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and prool's
Question: For each theoretical result, does the paper provide the full of assumptions and complete (and correct) proof?
34

---

Answer: [Yes]
Justification: We included complete analytical development of the scalar-matrix shrinkage regularizer introduced (see Appendix A.1).
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Question:
Answer: [Yes]
Section presents a step-by-step explanation of our simulation procedure, and the appendix includes an illustration to clarify the simulation process (Fig. S2). Tables: $2,S1,S3 captions elaborate further our regression analysis process.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
35

---

5. Open access to data and code
Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification: A zip file containing the code and data, along with reproduction instructions, is provided with the paper submission. These resources will be shared with the general public upon publication through GitHub.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https: //iiss.c.. cccppblcc/aaaacccccccccccccc guides Meee/CCooSubbii issiiss ssisssiin ssiissolicyyyy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they chosen, type of optimizer, etc.) necessary understand the results?
Answer: [Yes]
Justification: Full experimental details are provided in Section 2, including a clear breakdown of our simulation methodology. Furthermore, appendix includes all complementary extensions of our methodological work, presented in Appendices A.2.1, A.3, A.4, and A.5.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can provided either with the code, appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: As can be seen in Figures: 2,1.3, we introduced error bars where applicable. All p-values were corrected for multiple comparisons using Bonferroni correction (see: Table S3 and Fig. 2).
Guidelines:
The answer NA means that the paper does not include experiments.
36

---

The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall
with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out of range (e.g. negative rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: description of the computing resources used can found in Appendix A.7 Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics atttp://neurrl: cwtuurpp.cc//ppee Deublicc/EthcsCuideee
Answer: [Yes]
Justification: We have reviewed the code of ethics and our research conforms in every respect with this code. Our work is simulation study using existing, previously published human data. All the utilized data and code resources are licensed as freely available to the general public. The images used for the illustration are from the CCO version of the THINGS image dataset.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. there special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative socictal impacts of the work performed?
37

---

Answer: [NA]
This study is a basic science research on the methodology of representational alignment. We do not any direct societal impact.
Justification:
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative socictal impacts include potential malicious unintended uses (e.g.. disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to partieular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: This work conducts simulation study using are-trained DNNs and previously acquired behavioral data.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model. for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer:[
Justification: THINGS images used in this work are under cion-commercial research purposes only, according to fair use (see http://www.dnipp.ogllegggh
Images for the illustration are from the THINGS+ CCO dataset, which contains 1,854 THINGS concept images with a public domain/CCO license. All of the previous work we have relied on has been cited and mentioned in the paper.
38

---

Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license ge.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswitheee theode. contattsetss has curated licenses for some datasets. Their licensing guide can help determine the license dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
No new assets are being released with this paper. Open data and code for reproducibilityy
Justification:
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moddl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [Yes]
Justification: We used existing data from previously conducted and published crowd- sourcing experiments. Instructions and compensation are detailed in the original publications [40],[38].
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Bthies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
Institutional review board (IRB) approvals or equivalent for research with human subjects
39

---

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: No new human behavioral data was acquired.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer:[NA]
Justification: LLMs used only for copyediting and proofreading.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our L.LM policy (https: / httpe///neeuisppeee http///uripps.cccoo/eeeeeeeeeeee fece/oonerreecee22255///// for what should or should not be described.
40


Reviews:
[{"review_id": "KLBaYLuR9u", "reviewer": "Reviewer_LzV6", "summary": "The paper questions the common assumption that high predictive accuracy of human behavior is a good proxy for internal representational alignment. To test this, the authors take 20 pre-trained vision models, freeze their weights, and learn a linear transformation of each model's features to best predict judgments from the THINGS dataset. These aligned models, which could be called \"teacher\" models, then generate synthetic datasets. In the recovery phase, the same 20 frozen models (\"student\" models) are fitted to the synthetic data by learning a new linear transformation. The authors then tests if the original data-generating model can be correctly identified based on predictive accuracy.\n\nThe authors find that while model recovery improves with dataset size, it plateaus below 80% accuracy. The errors are systematic, with certain models often being misidentified as the best predictor. They find that that greater flexibility in the alignment metric improves predictive accuracy but hurts model identifiability.", "questions": "Recent work (e.g., Lampinen et al., 2024) has focused on how training objectives create distinct representational biases. How do you see these biases factoring into your analysis? It seems they would determine the initial representational geometries that your linear probe then attempts to align.\n\nYour work shows that DNNs can be confused for other DNNs. How do you think these claims extend to the much harder problem of comparing DNNs to the brain? It seems your findings represent a best-case scenario -- the problem of misidentification may be even more severe in the neuroscience setting.", "limitations": "Yes.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n\nThe model recovery setup is novel, as it allows for evaluation against a known ground truth.\n\nThe approach is applied to a large set of 20 vision architectures and, calibrates the synthetic data to realistic human noise levels, making the setup more realistic. The authors go beyond simply identifying the problem and use regression analysis to find predictors of misidentification.\n\n**Weaknesses / Feedback**\n\nSome of the presentation was difficult to parse. Adopting the \"teacher/student\" model analogy, as the setup is similar to distillation, would make the simulation process much more intuitive for readers.\n\nThe conclusion could benefit from proposed avenues of research to address the issue the paper identifies.", "was_revised": true, "final_justification": "I have considered the discussion with the authors and other reviewers. My concerns have largely been addressed and I believe the paper is strong. I will maintain my positive rating.", "rebuttal": "We thank the reviewer for their interest in our manuscript and for the constructive review.\n\n**1. Clarifying the model-recovery simulation using a student–teacher analogy.** We appreciate the suggestion and have incorporated this analogy into our Methods section:\n\n> This setup is analogous to data distillation: the candidate models act as \"students\" attempting to approximate the input–output function of a \"teacher\"—the data-generating model. Model recovery is successful when the best-performing student is the teacher itself, rather than an alternative model.\n\n**2. Elaborating on proposed avenues to address the issue identified in the paper.** We outlined several possible research directions at the end of the Discussion section, under the heading \"Navigating the accuracy–identifiability tradeoff.\" In response to this and related comments, we have revised and expanded the section, further elaborating on possible approaches to mitigate the issue identified in our work.\n>*1. Change the stimuli:* As in many model comparison studies, we evaluated models out-of-sample, that is, on new stimuli drawn from the training distribution. Out-of-distribution generalization, which more strongly probes the models' inductive biases, may offer greater model comparative power. Stimuli designed to elicit model disagreement may yield even greater gains.\nThe recovery gains we obtained from larger and more diagnostic triplet sets suggest that smarter sampling matters at least as much as sheer volume (Fig. 1). Adaptive, model-driven stimulus selection constructing trials that maximize the expected divergence in network responses—can sharpen our ability to treat predictive accuracy as an indicator for human—model alignment, enhancing current flexible alignment methods without compromising identifiability.\n\n>*2. Change the metrics:* Constraining data-driven model alignment by biologically motivated and/or inter-individual variability–informed priors may improve upon the overly flexible family of linear transformations. Furthermore, imposing greater constraints on the readout may enhance its interpretability. For example, constraining the learned stimulus embeddings to be non-negative prevents features from canceling each other (e.g., Mahner et al., 2025). Finally, Bayesian readout models, which estimate a distribution of feature weights rather than a point estimate, may improve robustness to sampling noise.\n\n>*3. Change the models:* The considerable geometric shifts required to align the networks suggest that linear probes can obscure important representational mismatches. Embedding richer priors directly into the models—through task design, objective functions, or biologically inspired architectures—could allow aligned representations to emerge natively, reducing the need for substantial post hoc transformations. More broadly, further progress in neural network-based modeling of brain and behavior may depend less on ever-larger data-driven fits of pre-trained models and more on deliberate model refinement to embody explicit computational hypotheses.\n\n**3. The effect of training objectives on model recovery accuracy.** We thank the reviewer for highlighting this for our consideration. After the submission deadline, we introduced ten additional models, primarily image–text-aligned (e.g., ALIGN, OpenCLIP-ConvNeXt-L, OpenAI-CLIP ViT-L) and reran the model-recovery simulations. To address this specific comment, we estimated a between-objective confusion matrix, in which model recovery is considered incorrect only if it confuses models trained on different objectives. However, even under this permissive criterion, model recovery plateaued at 72%, compared to 70% under the standard criterion applied to the same model set. We have added a summary of this result to the Results section of the manuscript:\n\n>**Model-recovery performance plateaus despite objective-driven representational divergence**\nRecent work (Lampinen et al., 2024) has shown that different training objectives produce distinct representational biases. We categorized each model as supervised, unsupervised, or image–text-aligned and then performed a between-objective model-recovery analysis. Even with a 4.2-million-triplet training set, objective-based grouping yielded only a modest accuracy gain: from approximately 70% to 72%. These results indicate that, although initial internal representations reflect objective-specific biases (Lampinen et al., 2024), flexible evaluation largely obscures these differences.\n\n4.**Extension beyond behavioral settings.** We expect that the predictivity–identifiability trade-off will likewise manifest in neural data. However, the severity of this trade-off may change, because model comparison using neural data operates in a somewhat different regime: responses are multivariate and continuous, rather than univariate and discrete. We have revised the discussion section to reflect this uncertainty:\n> The scope of our simulations is limited to behavioral data, and specifically, to the THINGS odd-one-out task. We chose this task as a test case because it is supported by a large empirical dataset (Hebart et al., 2023) and allows straightforward simulation of synthetic responses by sampling from model-specified multinomial distributions. The noise-calibrated simulation approach can be readily extended to other behavioral paradigms, such as classification (Battleday et al., 2020) or multi-arrangement (Kriegeskorte \\& Mur, 2012). Model identification using neural data operates in a markedly different regime: responses are multivariate and continuous, rather than univariate and discrete as in the behavioral case. Therefore, while our results demonstrate a pronounced predictivity–identifiability trade-off when comparing models to behavior in a large dataset, the severity of this trade-off for neural data remains an open question.\n\nWe aim to resolve this uncertainty in follow-up work. To reduce the risk of misinterpreting the scope of our empirical results, we added the word “behavioral” to the concluding sentence of the abstract:\n> These findings demonstrate that, even with massive behavioral data, overly flexible alignment metrics may fail to guide us toward artificial representations that are genuinely more human-aligned."}, {"review_id": "xFiDfpnLbH", "reviewer": "Reviewer_jQvh", "summary": "The authors address the hypothesis that good brain data predictivity in neural network models is reflective of a corresponding architectural similarity with the entity that generated said brain data, i.e. that being good at predicting brain data reflects mechanistic alignment between models and human subjects. Standard assumption is that this is the case, but drawing conclusions from such approaches is made hard by the presence of noise in the measurements and the limited data available for model identification. The authors study whether model identification is possible even when data availability is not an issue and the data generating procedure is known. They do this by substituting human participants with a neural network, recording its responses and checking if model identification can pick out the data generating architecture from a set of candidates at evaluation time. They find that in general, even with millions of examples, such identification is not possible. These results cast doubt on the possibility of identifying the mechanisms used by humans to represent data using brain-predictivity methods alone.", "questions": "I have no interesting questions at this point. If some come up I will post them.", "limitations": "The authors clearly state the limitations of the study, but I think what they have done is more than sufficient.", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 4, "originality": 4, "strengths_and_weaknesses": "Strengths:\n- The paper is well motivated, addressing a key question that has generated significant debate recently.\n- The method is a clever but simple way to side-step the issue of comparing to humans, by just focusing on whether models can identify each other first, which is a much simpler task.\n- The methods are well detailed, explaining how the experimental setting is matched as much as possible to what we would encounter when comparing to humans.\n\n\nWeaknesses:\n- The study is restricted to behavioral data, though I agree with the authors that this is a good starting point.\n- The explanation on how the noise ceiling was determined could be a bit more clear.", "was_revised": true, "final_justification": "I believe this work provides a good way to think about how representation alignment as currently computed/formulated may not match our intuitions of how it should works. This is important as, in my opinion, this field is riddled with misconceptions, concept creep, and straight-up motte-and-baileying of claims. Works such as help illustrate with clear experiments how incorrect assumptions can lead to false conclusions. I am also happy with the authors replies and while my score won't increase, I definitely recommend it being accepted.", "rebuttal": "We thank the reviewer for their thoughtful and supportive evaluation of our work. Below we address each point in turn.\n\n**1. Limitation to behavioral data** \nWe appreciate the reviewer's endorsement of behavioral data as an appropriate scope for a first noise-calibrated simulation study of this problem, and we intend to extend this work to neural data in follow-up research. We refer the reviewer to our reply to Reviewer SoKD for more details on how we understand the relationship between the behavioral and neural domains in this context.\n\n**2. Noise ceiling estimation details.** We thank the reviewer for the suggestion to make this description clearer. To estimate the noise ceiling, the THINGS odd-one-out dataset includes a subset of 1,000 triplets—each presented to approximately 30 participants. To compute a lower-bound on the performance of the ideal model, we used a leave-one-subject-out procedure on this data, following Hebart et al., 2020 (Nature Human Behaviour). For each triplet and each participant, we withheld that participant’s response, determined the majority choice among the remaining responses, and recorded whether the held-out response matched the majority. We repeated this for all participants, averaged the match rates across triplets. This reproduced the noise-ceiling values reported by Hebart et al. We have added the following paragraph under “Calibration” in the Methods section:\n>We estimated the noise ceiling with a leave-one-subject-out procedure: for every triplet, we removed one participant’s response, took the majority vote of the remaining responses, and noted whether the held-out answer matched that vote. We repeated this step for all participants and averaged the match rates across all triplets."}, {"review_id": "hQCC7VmyeA", "reviewer": "Reviewer_Suzu", "summary": "This paper investigates whether linear probing can reliably identify which model generated behavioral data in human perception tasks. The authors conduct large-scale model recovery simulations using 20 diverse vision models on the THINGS odd-one-out dataset. They find that even with massive datasets, model recovery accuracy plateaus below 80%, meaning the best-fitting model often isn't the data-generating model.", "questions": "Your study only looks at odd-one-out similarity judgments, which is pretty narrow. Do you think these problems with model recovery would show up in other kinds of behavioral tasks or even neural data? It would be helpful to see some theoretical reasoning or even small pilot studies showing this isn't just a quirk of this particular dataset. I'd say this is my primary qualm with the manuscript at this moment.", "limitations": "yes", "rating": 5, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths\n- Very thorough evaluation in terms of testing 20 diverse models across multiple dataset sizes (50 to 4.2M trials), with noise calibration to match human response variability. \n- The regression analysis identifying specific factors (representational geometry shifts, effective dimensionality) that drive misidentification provides useful mechanistic insights.\n- I really enjoyed the discussion section “Navigating the accuracy–identifiability tradeoff” at the end of the paper.\n\nWeaknesses\n- The study is restricted to one behavioral task of odd-one-out judgments and one dataset. The quantitative results are specific to this particular model set and may not generalize broadly. The paper does draw attention to this in its limitation section, but still, for a venue like NeurIPS, the scope feels narrow. \n- The core insight that flexible metrics can overfit isn't entirely novel; this has been recognized in various forms across ML. The specific quantification in this behavioral context is useful but somewhat incremental.\n- I wonder if this work would be better suited for a more specialized venue focused on computational cognitive science or model evaluation methodologies rather than NeurIPS.", "was_revised": true, "final_justification": "The authors' response, especially the new set of results, address some of my concerns sufficiently so I raised my score.", "rebuttal": "We thank the reviewer for their positive evaluation and detailed review.\n\n**In response to:** *“The study is restricted to one behavioral task of odd-one-out judgments and one dataset. The quantitative results are specific to this particular model set and may not generalize broadly.”*\n\nWe selected the THINGS odd-one-out paradigm because its massive scale (4.7 million human judgments) provides favorable conditions for identifiability, and its simple three-way response paradigm allows precise probabilistic simulation. Therefore, we believe that model recovery accuracy plateauing below 80% even with millions of simulated trials indicates a fundamental limitation of flexible linear evaluation, rather than a quirk of this dataset. \n\nWe agree that the particular composition of the model set may influence recovery outcomes. To test this possibility, we conducted additional simulations after the submission deadline, using an extended model set. This set included the 20 models reported in the manuscript along with OpenCLIP ConvNext XL, ALIGN, FLAVA Full, SigLIP2 B, BLIP 2, EVA02 CLIP-enormous, PE Huge, ImageBind Huge, OpenAI CLIP ViT Large, and AlexNet. When curating the new models, we prioritized multi-modal (visual-language) models, motivated by the strong performance of CLIP in the original evaluation.\n\nBelow, we report the model recovery performance of the extended model set as a function of dataset size:\n\n| Number of training triplets | Model recovery accuracy | 95% CI      |\n| --------------------------- | ----------------------- | ----------- |\n| 100                         | 0.07                    | [0.04, 0.10] |\n| 400                         | 0.11                    | [0.08, 0.15] |\n| 1,600                       | 0.18                    | [0.13, 0.22] |\n| 6,400                       | 0.26                    | [0.21, 0.31] |\n| 25,600                      | 0.36                    | [0.31, 0.41] |\n| 51,200                      | 0.43                    | [0.37, 0.48] |\n| 204,800                     | 0.62                    | [0.56, 0.67] |\n| 819,200                     | 0.66                    | [0.61, 0.72] |\n| 1,638,400                   | 0.70                    | [0.64, 0.75] |\n| 4,200,000                   | 0.70                    | [0.65, 0.75] |\n\nNotably, OpenAI CLIP ConvNeXT XL was not consistently identified when it was the data-generating model. We will incorporate these simulations along with a detailed confusion matrix in the revised (camera-ready or resubmitted) version of the paper. Overall, the results indicate that model confusability is not limited to the original 20-model set reported in the current version of the manuscript.\n\n**In response to:** *The core insight that flexible metrics can overfit isn't entirely novel; this has been recognized in various forms across ML. The specific quantification in this behavioral context is useful but somewhat incremental. I wonder if this work would be better suited for a more specialized venue focused on computational cognitive science or model evaluation methodologies rather than NeurIPS.*\n\nWe agree with the reviewer that the insight that flexible evaluation can lead to overfitting is well established in statistics and machine learning. It is also increasingly acknowledged in neuroAI and cognitive computational neuroscience. However, most neuroscientific studies comparing DNN and neural/behavioral responses place strong emphasis on maximizing predictivity, and seldom evaluate how well their chosen alignment metrics support model identification. Here, we analyze model identification performance in a leading benchmark—the largest currently available behavioral dataset—using a realistic simulation and a continuum of relevant evaluation metrics. To our knowledge, no prior work has systematically quantified model identification performance under realistic, noise-calibrated conditions. While our analysis focuses on a specific benchmark, the methodology we introduce and the substantial identifiability–predictivity trade-off it uncovers are relevant to a broader class of model-evaluation problems. We hope that this work will encourage researchers to conduct such analyses as an integral part of their empirical studies.\n\nWe agree that this manuscript could also be suitable for specialized venues such as CCN or NBDT. However, many researchers active in neural network modeling of brain and behavior are more likely to be exposed to a NeurIPS paper. Several studies that rely on flexible metrics for model identification have themselves been published at NeurIPS/ICLR/ICML. Publishing this work at NeurIPS can help ensure that the identifiability–predictivity trade-off is recognized and addressed by the broader modeling community.\n\n**In response to:** *“Your study only looks at odd-one-out similarity judgments, which is pretty narrow. Do you think these problems with model recovery would show up in other kinds of behavioral tasks or even neural data?\"*\n\nAs discussed in our response to the reviewer's first point, we do not see a reason that other behavioral tasks would yield better model identification performance. Moreover, existing real behavioral datasets are at least an order of magnitude smaller than THINGS-odd-one-out, and therefore even more prone to model misidentification.\n\nAs for neural datasets, the statistical regime is different—the responses are multivariate and continuous and rather than univariate and discrete. We are therefore more cautious about making strong predictions about these settings without additional data. Please see our response to Reviewer SoKD below for a more detailed discussion of this point, as well as a relevant modification of the manuscript."}, {"review_id": "izJDMWMMof", "reviewer": "Reviewer_SokD", "summary": "The authors conduct large-scale model-recovery simulations on the THINGS odd-one-out behavioral dataset to show that linear probing of DNN representations (essentially flexible evaluations that allow linear reparamerizations of the feature space) fails to reliably recover the true data-generating model. Things improve with more samples but the identifiability of the true model is still below 80% with millions of trials. They show a trade-off between predictivity of human behavior and identifiability of the true model as the flexibility of the probe increases (going from simple scaling to a rectangular to a full linear transformation matrix). The authors also try to identify factors causing misidentifiability and find that representations with higher effective dimensionality and those that incur higher geometry shifts after alignment with behavioral responses are the ones that are more misidentified.", "questions": "My main concern is about the generality of findings (that more flexible mappings reduce identifiiability) beyond behavioral tasks. All simulations use the THINGS odd-one-out behavioral paradigm. Yet the paper’s rhetoric (“flexible evaluation may incur a hidden cost”) implies broad applicability to any linear alignment metric—most pressingly those applied to neural data . It is hard to know if the finding obtained with one dataset would generalize to broader settings, particularly neural data. Without at least a toy simulation using a realistic neural noise model, it’s unclear whether the same identifiability collapse holds, improves, or worsens in neural-alignment settings. \n\nThe authors should also link their finding that flexible evaluations reduce identifiability with other findings in neuroscience. for eg that of conwell et al which show that when flexible evaluations are employed (eg linear predictivity), many different candidate models achieve similar brain predictivity. So in some sense, this issue has already been flagged in the field. \n\nThe paper could also benefit from discussion of other potential criteria that should be used for evaluating methods of identifying candidate models of brain/behavior. Identifiability is an important (almost necessary one) but are there other criteria that we want our evaluation methods to satisfy? e.g., interpretability of weights, robustness to noise etc\n\nThe experiments sweep from zero-shot through diagonal, low-rank, and full linear transforms—but do not examine any class of nonlinear mapping models. It would be valuable to see whether moderately nonlinear mappingsfurther amplify or mitigate identifiability failures. Beyond diagonal/low-rank/full, it would also have been useful to introduce other structured‐linear probes (e.g. sparse at different sparsity levels) and chart their place on the accuracy–identifiability curve. The current probe evaluation (diagonal vs low-rank vs full seems too limited)\n\nAlso the paper flags the main issue of flexible models as their misidentification rate. But which studies use model-behavior comparisons for predictivity vs using the most human behavior like model  to make conclusions about what shapes human behavior.  It would help to cite a few examples of such articles whose inference pipelines depend on the reliability and identifiability of flexible linear probes.\n\nOverall, this paper issues an important warning about over-flexible linear probes in behavioral model comparison, and suggests that similar pitfalls likely afflict neural-alignment studies. However, to substantiate its broader implications and to offer concrete remedies, it needs (1) at least a small neural-data simulation and (2) a thorough exploration of structured and mildly nonlinear probes.", "limitations": "yes", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "The results are intuitive and the paper has a nice take-home that also seems actionable: that whenever you use highly flexible, data-driven alignment (linear or otherwise) to compare models to brain or behavior, you risk mistaking best fit for true fit (predictability vs identifiability trade-off). \nThe paper is well written, the experiments are sound and the conclusions are straightforward.", "was_revised": true, "final_justification": "I have increased my score to borderline accept. I still don't know to what extent the conclusions from this study apply to model-brain comparisons (which was cited as a key motivation behind this paper).", "rebuttal": "We thank the reviewer for their thoughtful response and deep engagement with the manuscript.\n\n**Generalization to neural data.** We agree that generalization to neural data is non-trivial. Model identification using neural data operates in a markedly different regime: responses are multivariate and continuous rather than univariate and discrete.\n\nFollowing the comments of SokD and other reviewers, we carefully considered the possibility of conducting a simulation of model recovery using neural data with a *realistic neural noise model*. We concluded that this would be a substantial undertaking. A realistic simulation would entail accurately capturing both single-trial variability and inter-channel noise correlations. For the NSD fMRI dataset, a complete reanalysis would be required because GLMSingle does not provide statistically independent single-trial response estimates (across-trial reliability is used to guide HRF choice and signal denoising). For MUA or spike-rate data, access to single-trial responses is easier, but modeling must account for the multivariate, non-Gaussian distribution of the noise (i.e., faithfully capturing response variability and noise correlations). Furthermore, a realistic simulation may require modeling modality-specific noise sources, such as head motion in fMRI, or slow drift in response amplitudes in electrophysiology.\n\nHence, we believe that such a study would justify a separate paper devoted to this problem. Combining Things-Odd-One-Out and one or more neural datasets into a single paper would considerably broaden the scope. However, it would not allow for sufficient detail and depth in describing the various analytic choices required for each simulation study. Here, we chose to focus on a single but important test case and to pursue it in as much depth as possible. We hope we can gain the reviewer's support for publishing the present work in NeurIPS, despite our deliberate focus on a behavioral test case at this stage.\n\nWe have revised our draft to reflect the uncertainty about the generalization to neural data. Revised limitations paragraph:\n> The scope of our simulations is limited to behavioral data, and specifically, to the THINGS odd-one-out task. We chose this task as a test case because it is supported by a large empirical dataset (Hebart et al., 2023) and allows straightforward simulation of synthetic responses by sampling from model-specified multinomial distributions. The noise-calibrated simulation approach can be readily extended to other behavioral paradigms, such as classification (Battleday et al., 2020) or multi-arrangement (Kriegeskorte \\& Mur, 2012). Model identification using neural data operates in a markedly different regime: responses are multivariate and continuous, rather than univariate and discrete as in the behavioral case. Therefore, while our results demonstrate a pronounced predictivity–identifiability trade-off when comparing models to behavior in a large dataset, the severity of this trade-off for neural data cannot be inferred from our findings.\n\nWe also added the word \"behavioral\" to the abstract's conclusion:\n> These findings demonstrate that, even with massive behavioral data, overly flexible alignment metrics may fail to guide us toward artificial representations that are genuinely more human-aligned.\n\nWe wish to keep the sentence \"flexible evaluation may incur a hidden cost\". It presents our working hypothesis rather than the paper's empirical conclusion.\n\n**Link to recent neuroscientific reports on flexible evaluation.** We agree that these links are important, and in part, they have motivated our work. The remainder of our new limitations paragraph spells this out.\n\n> Recent reports of qualitatively distinct neural network models achieving indistinguishable performance under flexible comparisons to neural data (Conwell et al., 2024; Storrs et al., 2021) make this question especially pertinent. Addressing it will require future work using noise-calibrated, modality-specific neural simulations.\n\nNote that the equivalent model performance reported in Storrs et al. and Conwell et al. does not prove that the compared models make equivalent predictions. Similar accuracy is a necessary but insufficient condition for prediction equivalence. For example, models with similar aggregate performance may produce different error patterns (see Rajalingham et al., 2018, J. Neurosci.) or explain different subsets of the data. Therefore, comparing each model to neural or behavioral data does not eliminate the need to study how distinguishable the predictions of different models are, as we do here.\n\n**Additional criteria for evaluation measures.**\n\nThis is a good point. Given the limited space, we incorporated it by expanding the second future direction in the discussion:\n> Constraining data-driven model alignment by biologically motivated and/or inter-individual variability–informed priors (...) may improve upon the overly flexible family of linear transformations. Furthermore, imposing greater constraints on the readout may enhance its interpretability. For example, constraining the learned stimulus embeddings to be non-negative prevents features from canceling each other (e.g., Mahner et al., 2025). Finally, Bayesian readout models, which estimate a distribution of feature weights rather than a point estimate, may improve robustness to sampling noise.\n\n**Nonlinear and structured linear probes.** We would be happy to introduce additional probes into our analysis. We will update the next version of the manuscript regardless of whether it is a camera-ready version or a resubmission. We have already introduced a more traditional Frobenius-norm penalty on the weights and found that it is slightly less effective for model identification than the penalty introduced in the paper. \n\nHowever, we would be grateful for clarification on the suggested probes. Could the reviewer clarify what kind of nonlinear probes they envision for this task? The vast majority of the literature purposely relies on linear probes to ensure interpretability. Regarding structured linear probes, since we are modeling higher-order perceptual judgments, penultimate layers are used (following Muttenthaler et al., ICLR 2023). This rules out separable readout (i.e., distinct weights for feature channels and spatial locations). The remaining probes seem to be sparse. Does the reviewer have in mind probes that are sparse in the linear transformation matrix, or probes that are sparse in the resulting embedding matrix (as in the SPoSE and VICE embedding models)? If it is the former, are there any examples in the literature where sparse readout is motivated and implemented?\n\n**Distinguishing Predictive Performance from Model Identification.** All of the human–DNN comparison studies cited in the manuscript are aimed at understanding the brain, rather than predicting responses as an end in itself. However, we agree that—especially in the context of NeurIPS—it is worthwhile to explicitly distinguish between prediction for its own sake and prediction used as a means for model identification. We have edited the introduction to make this distinction and to cite several studies that use flexible evaluation of behavioral model predictions, with the aim of understanding the underlying biological computations:\n\n> When evaluation is made flexible by fitting linear weights to improve the alignment of neural network representations with brain [...] or behavioral data (e.g., Peterson et al., 2018, Cognitive Science; Battleday et al., 2020, Nature Communications; Daube et al., 2021, Patterns; Muttenthaler et al., 2022, ICLR), this approach achieves predictive accuracy that exceeds that of any other computational model.\nIn some neuroscientific applications (e.g., brain--computer interfaces), accurate prediction is useful regardless of the underlying mechanism. In contrast, basic-science studies in computational neuroscience often rely on the assumption that a neural network whose representations are more predictive of brain or behavioral data is a better model of the mechanisms underlying the biological data."}, {"review_id": "z2wgPdXSFl", "reviewer": "Reviewer_GT1J", "summary": "The authors show that when pretrained neural network representations are fitted to predict human similarity judgements, these representations lose their identifiability and recoverability. While the situation is improved by an increase in dataset size, the ceiling caps at 80%.\n\nFurther, the authors conduct a set of interesting analyses trying to understand the factors that underlie the problem. They find that a high intrinsic dimensionality and a large change in representational geometry are linked to lower recoverability. They also show that one successful model (CLIP) is recovered from the synthetic data generated by several other models.", "questions": "- I see that you fit $W$ several times in model recovery across seeds, but it seems to be fitted just once in the actual model fitting. Could some of the unrecoverability be attributed to stochastic optimisation? How similar would the learned representations be if you fitted the same linear probe several times?\n- Is $W$ initalised as an identity matrix in the model fitting as well?\n- 80% recoverability is deemed not good in the paper. Intuitively, I agree with this. However, I wonder if you gave the same triplets to human participants several times, what the test-retest reliability would look like. I know it's not the same as recovery, and that there are other factors here, but I think it could be a good and interesting proxy. I know it's not in the scope of this paper, but it might be interesting work for the future.", "limitations": "The authors have adequately discussed the limitations of their work.", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 4, "strengths_and_weaknesses": "# Strengths\n\nI think this is an excellent paper. The problem addressed is of high relevance to the representational alignment community, and the results are very interesting (also concerning). While the main result is already exciting and accessible, I think the authors do a great job of digging into the causes behind the main findings. The writing is clear, and the fitting and evaluation methods are sound.\n\nAs an additional plus, the code provided by the authors is very well documented. I haven't had a chance to run it, but it definitely increases my belief in the accuracy of the reported results.\n\n# Weaknesses\n\n## Interpretation of Results in High-Data Regime\n\nWhile both recoverability and identifiability are poor in low-data regimes, they are actually very good given a lot of data (ie, Figure 1D). The models that are not recovered well also seem to be outdated models (eg VGG19, EfficientNet, NasNet, Inception, BarlowTwins). If these models are not included, you would have almost a diagonal matrix in 1D. I think it's good you included these models, and I am not asking you to remove them. However, this needs to be carefully interpreted when thinking about how much of a practical problem unrecoverability is. e.g., if I compare a CLIP model, Imagenet ViT-L, and a DINO v2, the risk does not appear to be high. Nevertheless, I agree with the authors that such a high data regime is often not achieved in most studies. I just wanted to point this out for discussion.\n\n## Measure of Effective Dimensionality\n\nI know the PC based effective dimensionality metric is commonly used. However, I have my doubts about it. This paper [1] has shown that DNNs' representations lie on curved manifolds, which makes the PC based method inadequate due to its linear nature. I would recommend the authors use the Two-NN or the Gride method [2, 3], which does not make the linearity assumption. Either method can be easily used [with this package](https://github.com/sissa-data-science/DADApy), and would provide more accurate estimates.\n\n\n---\n\nAlthough I have outlined two weaknesses above, I think these are minor. The reason I'm giving a 5 and not a 6 is because I think the results are highly relevant to the representational alignment community, but I'm not sure if it has an impact in other areas. **I'd be willing to increase my score if the authors address the minor concerns above (the first one simply in writing, and the second one through simple analyses) and can make a convincing argument for why this work is relevant to a wider community.**\n\n\n\n# References\n\n[1] - Ansuini, A., Laio, A., Macke, J. H., & Zoccolan, D. (2019). Intrinsic dimension of data representations in deep neural networks. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d\\textquotesingle Alché-Buc, E. Fox, & R. Garnett (Eds.), Advances in Neural Information Processing Systems (Vol. 32).\n\n[2] - Facco, E., d’Errico, M., Rodriguez, A., & Laio, A. (2017). Estimating the intrinsic dimension of datasets by a minimal neighborhood information. Scientific Reports, 7(1), 12140. doi:10.1038/s41598-017-11873-y\n\n[3] - Denti, F., Doimo, D., Laio, A., & Mira, A. (2022). The generalized ratios intrinsic dimension estimator. Scientific Reports, 12(1), 20005. doi:10.1038/s41598-022-20991-1", "was_revised": true, "final_justification": "I’m convinced the results are of great importance to the comp cog sci and neuro AI communities.", "rebuttal": "We are grateful for the reviewer's constructive and insightful review of our work. \n\n**1. Interpretation of Results in High-Data Regime.** We agree that in the high-data regime, misidentification affected smaller convolutional models, which tend to have lower capacity and were trained on smaller datasets, potentially yielding less distinctive representations. However, in a follow-up analysis including ten additional models (see our response to Reviewer Suzu), we found that even OpenCLIP-ConvNeXt-XL was often misidentified. This finding suggests that misidentification in the high-data regime may not be limited to older or lower-capacity models. Importantly, model confusability is context dependent: it is a property of model pairs rather than of individual models.\n\nThis also raises a second concern. Suppose the read-out human representation is more similar to that of a simpler or smaller model. If flexible evaluation procedures tend to favor higher-capacity models because they can better approximate arbitrary targets, these procedures may systematically mislead us. In such cases, richer models could outscore simpler but more human-like ones—not because they are better aligned in their native geometry, but because they are more easily aligned post hoc. Therefore, we do not find the fact that it is mostly smaller CNNs being misidentified to be entirely reassuring.\n\nDue to space constraints in the current draft (even with the additional page), we cannot commit to a specific revision of the discussion section at this stage. However, we will incorporate this more nuanced interpretation, including the reviewer’s observation, into the revised version.\n\n**2. Measure of Effective Dimensionality.** We thank the reviewer for the helpful suggestion to explore non-linear dimensionality estimates, which we were glad to incorporate.\n\nFollowing this suggestion, we estimated intrinsic dimensionality (ID) using Generalized Ratios Intrinsic Dimensionality Estimator (GRIDE) (Denti et al., 2022) through the publicly available DADApy implementation (with GRIDE computed across neighborhood sizes k = 1–64).\n\nWe computed ID for each model’s representation before and after alignment, and reran the regression analyses described in Table S3 in the manuscript, replacing PCA-based effective dimensionality with the GRIDE measure.\n \nBelow are the updated results:\n| Predictor                                | Candidate/Generator                | β     | 95 % CI        | _p_-value |\n| ---------------------------------------- | --------------------- | ----- | -------------- | --------- |\n| # Parameters                             | Data‑generating model | 0.03  | [‑0.28, 0.28]  | 1.00      |\n| # Parameters                             | Candidate model       | ‑0.21 | [‑0.18, 0.3]   | 1.00      |\n| # Features                               | Data‑generating model | ‑0.37 | [‑0.44, 0.035] | 0.81      |\n| # Features                               | Candidate model       | 0.25  | [‑0.68, 0.2]   | 0.06      |\n| Original features ID                  | Data‑generating model | 0.09  | [‑0.14, 0.37]  | 1.00      |\n| Original features ID                  | Candidate model       | 0.01  | [-0.23, 0.31]  | 1.00      |\n| Transformed features ID               | Data‑generating model | -0.31 | [-0.61, ‑0.05] | 0.2       |\n| Transformed features ID               | Candidate model       | ‑0.29 | [‑0.6, ‑0.06]  | 0.16      |\n| Alignment‑induced representational shift | Data‑generating model | ‑0.25 | [‑0.61, ‑0.12] | 0.004     |\n| Alignment‑induced representational shift | Candidate model       | 0.52  | [0.33, 0.88]   | 0.0001    |\n\n*(ID: GRIDE intrinsic dimensionality estimate)*\n\nIntrinsic dimensionality—whether measured on original or transformed features—was not a significant predictor of the model recovery outcome (p > 0.15), whereas the alignment-induced representational shift remains the dominant, statistically significant predictor. Note that our significance threshold accounts for multiple comparisons, making the test relatively stringent.\n\nWe will include the updated GRIDE-based results as a supplemental figure in the revised manuscript.\n\n**3. Broader impact.** To avoid redundancy, we kindly refer the reviewer to our response to the second point of reviewer Suzu. In brief, we believe that while identifying models that mechanistically explain human brain and behavior is a central ambition in neuroAI and cognitive computational neuroscience, current benchmarking practices tend to reward predictive performance alone—often without asking whether the evaluation metrics can reliably identify the correct model. Our contributions are threefold: (1) we characterize the severity of this issue in an important benchmark; (2) we introduce a general approach for evaluating model recovery accuracy using noise-calibrated simulations; and (3) we aim to raise awareness of the tradeoff between predictivity and identifiability, advocating for model recovery simulation as a standard component of the analytic pipeline when comparing computational models.\n\n\n**4. Misidentification due to stochastic optimization and $W$ matrix initialization.** In all experiments, we optimized the matrix $W$ using full-batch L-BFGS and initialized $W$ to the identity matrix for both the data-generating and candidate models. Given fixed data, initialization, and optimizer hyper-parameters, full-batch L-BFGS is deterministic; consequently, the optimizer does not introduce stochasticity. The only stochastic elements in our pipeline are the triplet-sampling procedure described in Appendix A.3, and the sampling of the simulated responses from the discrete probability distribution defined by the data-generating model. The random seed was varied across repeats of the same experimental condition.\n\n**6. Human test-retest reliability.** We agree with the reviewer that studying inter- and intra-subject response variability would be an insightful direction for future work, and we are actively looking into this question. The current noise ceiling estimate reflects both sources of variability.\n\nWe believe the reviewer would agree that model recovery accuracy and inter- and intra-subject response reliability are related yet operate on different scales: even if the behavioral responses are unreliable (e.g., 50%), given many such responses, model recovery accuracy can still approach 100%. For example, consider comparing two biased coins, one with $p=0.5$ and the other with $p=0.55$. A single coin toss is not very informative, but many independent tosses can reliably reveal which of the two coins we are holding."}]

Author rebuttal:

Meta-review: This submission evaluates representational similarity measures using an odd-one-out task paradigm. While reviewers were generally positive about the thoroughness of the work and the novelty of applying the odd-one-out validation approach, I do have some concerns about the positioning of the contributions relative to existing literature.

The idea of validating representational similarity through model recovery has become standard practice across neuroscience and machine learning communities. Notable prior work includes:

* CKA fails to recover same-architecture inits ([Han et al., 2023](https://proceedings.mlr.press/v202/han23d))
* RSA recovers ground-truth geometries ([Schütt et al., 2023](https://elifesciences.org/articles/82566))
* RSA etc. fail to achieve high sensitivity/specificity in recovery of functional similarity i.a. ([Klabunde et al., 2023](https://arxiv.org/abs/2305.06329); [Ding et al., 2021](https://arxiv.org/abs/2108.01661); [Bo et al., 2024](https://arxiv.org/abs/2411.14633))
* And the classic [Jonas & Kording (2017)](https://doi.org/10.1371/journal.pcbi.1005268)

While I acknowledge that the odd-one-out task paradigm has not previously been used to directly validate RSA, etc. for model recovery, and this represents an interesting methodological contribution, the broader findings about failures of representational similarity align with established knowledge in the field.

Nevertheless, given the thorough execution and the new methodological contribution of using odd-one-out specifically, I recommend acceptance contingent on a minor revision that:

1. More clearly articulates the specific contributions relative to the prior work cited above;
2. Narrows the scope of claims in the paper (including potentially revising the title) to better reflect what is genuinely new; and 
3. Highlights the specific insights that the odd-one-out validation paradigm provides beyond existing model recovery approaches.

These revisions would help readers understand precisely how this work advances our understanding of representational similarity metrics beyond documenting failures that have been previously established.
Decision: Accept (poster)