Partial Soft-Matching Distance For Neural Representational Comparison With Partial Unit Correspondence
Authors: Chaitanya Kapoor
Alex H Williams
Meenakshi Khosla
Venue: ICLR 2026 Poster (2026)
Keywords: Optimal Transport
Neural Tuning
Representational Similarity
Deep Neural Networks
Primary area: applications to neuroscience & cognitive science

Abstract: Representational similarity metrics typically force all units to be matched, making them susceptible to noise and outliers common in neural representations. We extend the soft-matching distance to a partial optimal transport setting that allows some neurons to remain unmatched, yielding rotation-sensitive but robust correspondences. This partial soft-matching distance provides theoretical advantages---relaxing strict mass conservation while maintaining interpretable transport costs---and practical benefits through efficient neuron ranking in terms of cross-network alignment without costly iterative recomputation. In simulations, it preserves correct matches under outliers and reliably selects the correct model in noise-corrupted identification tasks. On fMRI data, it automatically excludes low-reliability voxels and produces voxel rankings by alignment quality that closely match computationally expensive brute-force approaches. It achieves higher alignment precision across homologous brain areas than standard soft-matching, which is forced to match all units regardless of quality. In deep networks, highly matched units exhibit similar maximally exciting images, while unmatched units show divergent patterns. This ability to partition by match quality enables focused analyses, \emph{e.g.,} testing whether networks have privileged axes even within their most aligned subpopulations. Overall, partial soft-matching provides a principled and practical method for representational comparison under partial correspondence.

Published as a conference paper at ICLR 2026
PARTIAL SOFT-MATCHING DISTANCE FOR NEURAL REPRESENTATIONAL COMPARISON WITH PARTIAL UNIT CORRESPONDENCE
Chaitanya Kapoor
Alex H. Williams
Department of Cognitive Science University of California, San Diego La Jolla, CA, 92093
Center for Neural Science, New York University
Center for Computational Neuroscience, Flatiron Institute
New York City, NY, 10003
chkapoor wuu.dduccddee edu
a alex. h. wi Willlamss nyu.edu
Meenakshi Khosla
Department of Cognitive Science
Department of Computer Science and Engineering University of California, San Diego La Jolla, CA, 92093
mkhosla@ucsea edu
ABSTRACT
Representational similarity metrics typically force all units to be matched, mak- ing them susceptible to noise and outliers common in neural representations. We extend the soft-matching distance partial optimal transport setting that allows some neurons to remain unmatched, yielding rotation-sensitive but ro- bust correspondences. This partial soft-matching distance provides theoretical strict mass conservation while maintaining interpretable transport costs-and practical benefits through efficient neuron ranking terms of cross-network alignment without costly iterative recomputation. In simula- tions, preserves correct matches under outliers and reliably selects the cor- rect model in noise-corrupted identification tasks. On fMRI data, it automati- cally excludes low-reliability voxels and produces voxel rankings by alignment quality that closely match computationally expensive brute-force approaches. It achieves higher alignment precision across homologous brain areas than standard soft-matching, which is forced match all units regardless of quality. In deep networks, highly matched units exhibit similar maximally exciting images, while unmatched units show divergent patterns. This ability to partition by match quality enables focused analyses, e.g., testing whether networks have privileged axes even within their most aligned subpopulations. Overall, partial soft-matching provides a principled and practical method for representational comparison under partial correspondence.
INTRODUCTION
Understanding how design choices (e.g.. training objectives, architecture) shape neural representa- tions requires comparing how different systems encode information. A fundamental challenge in this comparison is determining which computational units correspond across systems: do specific neurons implement similar functions across networks? This is central to understanding whether different systems converge similar computational solutions. Most existing representational sim- ilarity metrics, such as CKA (Kornblith et al., 2019), RSA (Kricgeskorte et al., 2008), Procrustes distance (Williams al., 2021; Ding al., 2021), and CCA variants (Raghu et al., 2017), are measure geometric similarity while ignoring the specific axes along which
All code is publicly available http::/ggthahh.oomeemMMMll.llaa hatp://aooooooaooaoioaaaaaaaaane

---

Published as a conference paper at ICLR 2026
information is encoded. This limitation prevents from understanding neuron-level correspon- dence and whether systems share axis-aligned representations.
A
B
Na
Rank order Tuning Similarity
Subject
A
Sporiaa
Pheeath ms)
P5
M
Subject
Na
Figure 1: Partial Soft-Matching Distance for Matching Tuning Curves. (A) Two toy networks NA and Ng: the layer of interest for alignment is shown in red. A partial matching recovers one- to-one correspondences between units with highly similar tuning curves. Line color encodes match strength (darker = stronger). By contrast, a purely soft-matching yields a spurious pair (hatched units, red dotted line). (B) The same metric be used to rank voxel/unit tuning-curve similarity between two subjects' responses (r1, }2), when exposed to the same visual stimulus.
To address these challenges, Khosla & Williams (2024) recently proposed a metric hased on discrete optimal transport (OT; Peyré al., 2019) called the soft-matching distance which finds rotation- sensitive correspondences between neurons while remaining invariant to their ordering. However, this approach requires that all neural units are matched across networks. This requirement may be crucial limitation in certain settings, since neural populations often contain noisy, inactive, or task- irrelevant units units-partinn particularry in biological recordings from fMRI electrophysiology. Moreover, even task-relevant units may be model-specific, implementing computations unique particular architecture or training regime in deep neural networks (DNNs). When comparing networks, we should not expect complete overlap in their functional units. Forcing all units into correspondence may therefore inflate distances and produce misleading alignments.
Here, we introduce the partial soft-matching distance (Fig. 1), which allows fraction of neurons to remain unmatched while preserving robust correspondences among the remainder. Our method provides several key advantages:
Relaxing mass conservation allows the metric to handle populations with different numbers of units, where some may lack correspondence (e.g., s,, due noise). Computational efficiency: Achieves comparable rankings with a single O(n log n) computa- tion, unlike brute-force methods requiring O(n" a log r2) operations.
Theoretical robustness:
Interpretable partitioning: Separates well-matched from unmatched units, enabling focused analysis of aligned subpopulations.
We demonstrate these advantages through controlled simulations showing correspondence despite spurious neurons, and accurate model identification noise-corrupted scenarios. In fMRI data from the Natural Scenes Dataset (Allen et al., 2022), our method discards low-quality voxels and outperforms standard soft-matching aligning homologous brain regions across subjects. When applied to DNNs, we find that highly-matched units produce similar maximally exciting images (MEIs) across models, while unmatched units show divergent MEIs, suggesting distinct computa- tional roles. Crucially, filtering unmatched units using partial soft-matching improves alignment over heuristics based on soft-matching correlations, matching the performance of a computationally intensive brute-force method that iteratively removes units in greedy fashion. This framework pro- vides a principled approach for comparing neural representations under partial correspondence common scenario in neuroscience and AI.
2

---

Published as a conference paper at ICLR 2026
2
METHODS
The optimal transport (OT) problem finds the minimum-cost mapping between probability distribu- tions, yielding metries like the soft-matching distance (Khosla & Williams, 2024). However, clas- sical OT requires equal total mass between distributions-4 constraint violated in neural recordings where units may be noisy, inactive, genuinely non-corresponding. We extend the soft-matching distance to handle these realistic scenarios through partial optimal transport.
2.1 SOFT-MATCHING DISTANCE
Consider two neural populations with N₂ and Ny units respectively, each with "tuning curves", Ny { and {y, ر j= taking values in R Each unit's tuning curve, respectively denoted and y; for the two neural populations, represents a neuron's ' response over a set of . probe stimuli. Stack- ing these tuning curve vectors column- wise produces matrices X ER MxN and Y MMN,
The soft- soft-matching distance treats each population as a uniform empirical measure and quantifies the optimal transport cost between them:
(x, (X
min TETIN..N.
VCC,T)P
where C₁₁ = ||,,, yj || is the squared Euclidean transport cost, F the Frobenius inner product, and T(N,, Ny,) is the transportation polytope (De Loera & Kim, 2013), i.e. the set of all N₂ x N₁ nonnegative matrices whose rows each sum to 1/N₂ and whose columns each sum to I/Ng-
This formulation is permutation permutation-invariant invariant yet rotation-sensitive, revealing single-neuron tuning alignment. Furthermore, dr is symmetric and satisfies the triangular inequality, which has been argued to be important for certain analyses of neural representations (Williams et al., 2021; Lange et al., 2023). The key limitation (which we document Sections 3 and 4) is that the marginal con- straints (i.e. that the transport plan T lie within the transportation polytope) forces all units to be matched, producing spurious correspondences when populations contain non-corresponding units.
2.2 PARTIAL SOFT-MATCHING DISTANCE
The soft-matching formulation requires the two empirical distributions have identical total mass and further enforces that all mass must be transported. The partial OT problem extends this by allowing only a pre-specified fraction S 8 S 1 of the total mass to be matched at minimal cost.
Formally, for empirical measures with unit total mass, natural set of admissible couplings is
N
N
T,NN,,N,) STERNNM
T S Σ j=1
Σ
T S N.
ΣT =8 i
Here, the inequalities the row/column marginals permit mass to remain unmatched in either population, and the scalar s controls the total matched mass. Since we normalize our populations to have unit total mass, directly represents the fraction of units that actually matched. The partial soft-matching distance is then the minimum transport cost over this feasible set,
dr(X, YX)
min TET*(N..N.))
(
with the usual cost matrix (e.g., C₁j- = a2 y 13). our formulation, we use pairwise cosine distance as the cost function. Several numerical approaches have been developed to solve partial OT problems (Benamou al., 2015; Chizat et al., 2018). More recently, Chapel et (2020) augmented the cost matrix with dummy (or virtual) points which assigned large transportation cost. All mass routed to these dummy nodes is effectively discarded, which yields an exact partial-matching solution in the augmented formulation.
Partial OT distances do not satisfy the triangle inequality and therefore are not proper metrics. How- ever, they still provide symmetric notion of dissimilarity in representation and, document in Sections 3 and 4, they provide robust and interpretable tool for tuning-level comparisons between neural populations with unequal or noisy measurements.
3

---

Published as a conference paper at ICLR 2026
2.3 CHOOSING OPTIMAL REGULARIZATION
To apply partial soft-matching in practice, must select the hyperparameter 0 < S which determines how much mass to transport between distributions. This key challenge when the abundance of outliers and magnitude of noise in the data unknown priori. To address this, we adopt an L-curve heuristic (Cultrera & Callegaro, 2020), inspired by classical regularization methods for ill-posed problems (e.g. Tikhonov regularization). The L-curve captures the trade- off between transport distance and regularization strength, with the "elbow" typically indicating balanced choice between these competing objectives. Concretely, we define the wwo-dimensional parametric curve: f(s) -(+) (C(s).
(C(s) (T(s), C)r (p(s) =
where C' is the cost matrix and (s) is the optimal transport plan for match fraction E [0, 1]. We interpret p(s) as the regularization strength-smalle s, or conversely larger P permits more mass to be left unmatched. The optimal regularization 8p is identified at the curve's point of maximal positive curvature (the elbow), which balances low transport cost against aggressive regularization.
In our discrete implementation, sample uniformly from sequence {s:}/ and compute the associated transportation costs Ç {(s,). We compute the elbow by approximating the second derivative of the cost curve with respect to the regularization strength p(s) by the centered second finite difference 52,
E = (i+1
2G Si-1 for i= = 2...., N
and select the index with maximal positive curvature
(o, 80 84-
arg max
2 2-iEN-1
allowing us to analytically select the optimal regularization so-
2.4 PARTIAL SOFT-MATCHING AS CORRELATION SCORE
Suppose that the tuning curves in two neuron populations X and Y have been centeree and scaled to unit-norm. Under this normalization, the inner product x! yj is identical to the Pearson correlation between neuron i in X and neuron j in Y. Using this, the optimization recast as a maximization of total matched correlation:
be
con X, Y.
max (N N.)
TTyor y₁
Intuitively, d measures the average correlation between paired neurons under the coupling T. We report door for the remainder the manuscript. We also report alignment obtained using a squared Euclidean cost function, Cıý || æzy1? in Appendix A1.8 and observe identical results.
2.5 INTERPRETATION AND OUTPUT
The optimal transport plan T* provides soft partial alignment where:
Row sums [0, 1/Na]: amount of mass transported from each source neuron
Column sums [o, 1/Ny]: amount of mass received by each target neuron
Total transported mass equals < (the fraction of total mass matched)
Near-zero row/column sums identify effectively unmatched units
This partitions populations based on participation the optimal matching, from completely un- matched to maximally participating units.
4

---

Published as a conference paper at ICLR 2026
3 SIMULATIONS: ROBUSTNESS TO NOISE AND SELECTING THE "CORRECT" MODEL
We designed controlled simulations to evaluate whether partial soft-matching (1) maintains accu- rate correspondences despite spurious neurons and (2) correctly identifies which model shares more signal with a reference population. Synthetic neural representation generation is detailed in Ap- pendix A1.3.
3.1 ROBUSTNESS AGAINST SPURIOUS NEURONS
We construct two neural populations X and Y, each containing K "signal" neurons matched pair- wise. We introduce noise by augmenting X with M, random neurons and Y with M, random neurons, where each random neuron is drawn from ~ (O, 1). The resultant populations are thus X E R. ) and Y E (KK M where N is the number of unique stimuli.
Soft statch Transpoct Plan
Partial Waseensten Transoort Plan
L-Curve
Legive
08
115
Infaction Print
04
0.2
0.0
0.5
0.1
0.5
08
Namora 10
Unmatched Masi (1
(a)
(b)
Figure 2: Comparison of Balanced and Partial Soft-Matching. (a) We simulate two neural repre- sentations, X and Y, with 120 and 190 neurons respectively. The first 100 neurons represent pure signal, while the rest are pure noise. Red denotes all pure signal neurons in the two representations. (b) The L-curve method selects the optimal mass regularization parameter (= 90/190 z 0.47), successfully discarding noisy units.
In the (fully balanced) soft-matching distance must match all K + MM neurons to K + My neu- rons. This forces spurious outlier-to-outlir assignments, which inflate the overall transport cost and, consequently, the distance. In contrast, the partial soft-matching distance only transports mass corresponding to K true matches, ignoring the random neurons. As result, the recovered transport cost is significantly smaller and reflects the true correspondence between the signal neurons. We vi- sualize the transport plans for both--sof--mathhing and partial soft-matching in Fig. 2, and observe that the L-curve heuristic is able to faithfully distinguish between noise and signal units.
3.2 CHOOSING BETWEEN TWO MODELS
Suppose we consider two models: Model A, where Y. shares exactly K correctly matched neurons with X, along with My additional noisy neurons; and Model B, where Y, (i) does not contain the same signal neurons as X, and (ii) shares fewer correctly matched neurons.
Model A is considered "correct" here because it preserves the maximum number of genuine signal correspondences with X X-the K matched neurons encode the same computational features as their counterparts in X X-plus additional noisy neurons. These extra neurons may reflect measurement noise, inactive recording channels, or recording artifacts that are common in real neural data. Model B, in contrast, shares only subset of X's signal neurons.
We compute the partial soft-matching scores Spanial X, Ya) and Spartial (X,, Yo). Because partial OT can ignore outliers and preserve only the true K matches, the distance, to the "correct" model Y. will be significantly smaller equivalently, the correlation satisfies Spartial X Y.) > Spartia (X, Yo), correctly identifying Model A as sharing more signal with X. By contrast, standard soft-match soft-matching forces matches for all units (including noise), obscuring signal differences and failing to discriminate Y, from Y,, shown in Fig. 3.
5

---

Published as a conference paper at ICLR 2026
L-Curve (X vs. Y
L-Curee (X vs. Yg
0.5
L-curve
L-surve Inflccionn Pojat
Inflcctonn Point
0.5
0L5
0.2)
0.3-
0.2
0.1
0.0
0.9
0.0
0.2
0.44
1001
0.5
1.0
0.0
0.2
0.4
0.6
0.5
1.01
Unmatched Mooe
Unmattced Max: (1
(a)
(b)
Figure 3: Model Selection Using Partial Soft-Matching. We simulate three synthetic representa- tions to test whether partial soft-matching correctly identifies which of two candidate models-Y or Y, shares more signal with a reference population X (100 units). Y, contains all 100 signal units from X plus 60 noise units; Y, contains 100 units, 80 of which match X. The true fraction of shared units known a priori, marked by a vertical gray line. With the L-curve-selected regularization, par- tial soft soft-matching yields correlation scores Spactial (X, Y.) 0.715 and Spartial (X, Yo) 0.645, correctly favoring Ya- Standard soft-matching fails, with Sem ( Yo) = 0.339 and 8am X, Y6) 0.415, incorrectly preferring Y, due to forced matching of noise.
APPLICATIONS IN NEUROSCIENCE AND AI
4.1 COMPARISONS OF NEURAL RECORDINGS ACROSS SUBJECTS
Vid
V2v
1.0
00
Alesu Natov Celling
115
Altgnment
0.7
0.0
0.5
105
-22
0.8
0.25
0.75
0.25
0.20
0.50
1.00
VI5
Vad
1.04
10.6
1.0
1.0
100
Moun Noine Cailian
0.20
08-
008
Aligattet
0.7-
0.4
0.8
0.5
+0.00
0.4
0.1
Figure 4: Aligning Voxel Responses Between Different Subjects in NSD. For each area, we plot the (i) partial soft-matching score different mass regularization values and (ii) the mean noise ceilings of the voxels that were kept at that regularization. The alignment criterion consistently identifies low noise-ceiling voxels for exclusion.
Perfect correspondence of neural populations across subjects mere-measurement noise, inactive voxels and anatomical variability implies imprecise region boundary definition. Voxels nominally assigned to the same brain area can sample neighboring regions implementing distinct computa- tions. Individual differences in functional organization further aggravate distinct computations across subjects. Partial soft-matching addresses these challenges by selectively excluding non- corresponding units from the alignment.

---

Published as a conference paper at ICLR 2026
We demonstrate this voxel responses from subject pair (IDs 1 and 2) across six visual eas (VIv, Vld, V2v, V2d, V3v, V3d) from the Natural Scenes Dataset (Allen et al., 2022). Fig. shows how voxel selection quality changes we vary the mass regularization parameter from (including all voxels) to O (excluding all voxels). As decreases and we exclude more voxels, the mean noise ceiling of the retained voxels steadily increases, while the alignment score between these retained voxels also improves. Since noise ceiling measures the reliability of a voxel's responses across repeated stimulus presentations, this demonstrates that our method successfully identifies and excludes voxels with poor response replicability. By progressively discarding these unreliable measurements, partial soft-matching automatically focuses the alignment on the subset of voxels that provide the most consistent and well-matched signal across subjects. We perform an identical experiment on different NSD subject pair (Appendix A A1.2) and observe identical results.
4.2 COMPARISON AGAINST BASELINE METHODS
In this section, we demonstrate the utility of our metric an efficient tool for rank-ordering neurons by their degree of cross-population alignment. We compare three approaches- frute-force match- ing, correlation-based ordering and our proposed partial soft-matching method. We test these methods in two distinct settings: (1) compar- ing convolutional kernels between two ResNet-18 models trained from different random initializa- tions on ImageNet (Deng et al., 2009), examining early, middle, and late layers (Fig. 5), and (2) aligning voxel responses between human subjects viewing natural images, across six visual areas (VIv, V1d, V2v, V2d, V3v, V3d) from the Natural Scenes Dataset (Fig. 6).
1ayar2.1.coms
1ayer4.1
Alignment Seater
1.0
6.9
0U6
066
0.5
Penii Miaeling
0.1
(U1
0.2
10
Figure 5: Evaluating Methods for Identifying (Un)matched Neurons in Deep Networks. We com- pare three methods for ranking convolutional kernels by alignment between two ResNet-18 models trained from different random initializations on ImageNet, across early, middle, and late layers. Removing low-alignment units identified by partial soft-matching yields alignment scores nearly identical ranked least important via brute-force ablations, while correlation- based rankings perform poorly.
those obtained by removing kernels
Brute-force matching provides the ground-trute ranking by exhaustively testing each neuron's 's contri- bution to alignment. We fit optimal soft-matching transformation to the complete representation, then iteratively remove each neuron and recompute the entire soft-matching optimization to measure the impact alignment score. This produces an exact ranking of neurons by their alignment qual- ity. However, each soft-matching optimization requires On log n) operations and for n neurons to test, the total complexity is On log n),, making this approach computationally prohibitive for realistic population sizes.
Correlation-basel ordering attempts computationally cheaper approximation by computing pair- wise Pearson correlations between neurons using the transport plan from a single soft-matching optimization. As shown in Figures and 6, this heuristic fails catastrophically-iit incorrectly iden- tifies and removes neurons that are actually crucial for alignment, resulting in dramatically degraded alignment scores. This failure occurs because individual correlation values don't 't capture the global optimization structure of the transport problem.
Partial soft-matching offers a nuanced tradeoff. To obtain complete ranking of all n neurons (matching the output of brute-force), we would still require separate optimizations at different reg- ularization values, maintaining Onn log r)) complexity. However, for the practically relevant task of identifying the top X% most-aligned or least-aligned neurons-which suffices for most analy- ses in neuroscience and deep learning which require identifying highly- or poorly-aligned subpopulations rather than complete rankings single optimization at the appropriate regulariza- tion value (O(n² log n)) provides near-identical results to brute-foree ranking. As Figures 5 and 6 demonstrate, when selecting subsets of neurons at various alignment thresholds, method's
7

---

Published as a conference paper at ICLR 2026
Viv
Vid
1.-
Partil Mlateling
100
100
0.9
Conrelatios Matching
6,9
Mcate Force Matching
0.8-
0,8
Alganenn
0.7-
6.5
0.1
0.4
0.4
0.2
0.4
0.5
0.8
1.0
0.2
0.4
0.5
0.8
1.0
0.2
0.4
0.6
0.8
1.0
V3v
1.0
1.0
0.8
0.8-
0.8
Alignment
0.6
0.8-
0,4
0.4
0.0
0.4
0.5
0.8
0.2
0.1
0.6
0.8
0.7 0.6
0.8
1.0
Figure 6: Evaluating Methods for Identifying (Un)matched Voxels in Brain Data. We evaluate three methods for ranking voxels by their degree of alignment between subject pair from NSD across six visual areas. Removing low-alignment voxels identified by partial soft-matching yields alignment scores nearly identical those obtained by removing voxels ranked least important brute-force ablations, while correlation- rankings perform poorly.
selections yield alignment scores nearly matching those from exhaustive brute-force ranking, while correlation-base selection performs poorly. Full algorithmic details are provided in Appendix A1.4.
4.3 MAPPING (DIs)SIMILAR BRAIN REGIONS
robust similarity metric for neural populations should exhibit specificity: it must identify when re- sponses come from the same brain area across subjects (true positives) while avoiding false matches between distinct areas (false positives) (Thobani et al., 2025). This specificity is crucial when anatomical boundaries are imprecise and individual variability high.
We evaluate this specificity by testing whether partial soft-matching correctly aligns homologous visual areas while maintaining separation between distinct areas. Concretely, we select two visual ROIs in a subject pair and then compute between -ubjeett-sjjett matching for all voxels of these regions. For each pair of visual regions within and across subjects from the NSD, we compute the precision of voxel assignments- the fraction of matched voxels that truly belong to corresponding regions. Table shows these precision scores comparing standard soft-matching (which must match all vox- els), thresholding using voxel noise ceilings, and partial soft-matching (which can exclude poor correspondences). The optimal regularization parameter for partial soft-matching is chosen via the L-curve heuristic described in Section 2.3.
Across most region pairs, partial soft-matching achieves higher precision than standard soft- matching and thresholding, with striking improvements for several cross-area comparisons (e.g.. V1d + V2v: 0.906 - 0.971). This improvement stems from the method's ability to exclude vox- els that lack clear correspondence-wheee
due boundary uncertainty or measurement noise. By not forcing these ambiguous voxels into the matching, partial soft-matching maintains cleaner separation between distinct regions while preserving strong alignment within homologous areas.
4.4 MAXIMALLY EXCITING IMAGES
Maximally Exciting Images (MEls)-synthett stimuli optimized maximize individual unit responses- provide an interpretable visualization of what each neuron "looks for" in its input (Erhan al., 2009; Pierzchlewicz et al., 2023; Walker et al., 2019; Bashivan et al., 2019). We synthesize
8

---

Published as a conference paper at ICLR 2026
Brain Region Pair
SM Precision (1)
ParSM Precision (1)
6 0.1 (t)
6 6 0.3 (t)
VIv + Vld
0.839
0.905 (0.76)
0.847 (0.98)
0.855 (0.94)
V1v V2v
0.677
0.680 (0.99)
0.680 (0.96)
0.695 (0.88)
VIv + V2d
0.880
0.884 (0.99)
0.884 (0.97)
0.894 (0.91)
VIv + V3v
0.798
0.853 (0.97)
0.803 (0.99)
0.815 (0.90)
Vlv + V3d
0.882
0.890 (0.98)
0.890 (0.97)
0.913 (0.91)
V1d + V2v
0.881
0.971 (0.71)
0.889 (0.96)
0.906 (0.89)
V1d + V2d
0.706
0.708 (0.99)
0.720 (0.97)
0.727 (0.92)
V1d + V3v
0.879
0.881 (0.99)
0.885 (0.97)
0.892 0.91
V1d + V3d
0.803
0.878 (0.76)
0.818 (0.97)
0.828 (0.92)
V2v + V2d
0.869
0.879 (0.98)
0.880 (0.95)
0.896 (0.87)
V2v + V3v
0,651
0.661 (0.95)
0.653 (0.95)
0.654 (0.86)
V2v + V3d
0.853
0,856 (0.99)
0.867 (0.95)
0.882 (0.87)
V2d V3v
0.833
0.971 (0.42)
0.845 (0.96)
0.863 (0.88)
V2d + V3d
0.638
0.643 (0.99)
0.642 (0.96)
0.643 (0.89)
V3v V3d
0.814
0.822 (0.99)
0.828 (0.96)
0.852 (0.88)
Table 1: Precision of Cross-Subject Voxel Alignment Within and Across Visual Areas. Compar- ison of soft-matching (SM), partial soft-matching (ParSM) and noise ceiling thresholding align voxels between visual regions in an NSD subject pair. Precision measures the fraction of matched voxels belonging corresponding anatomical regions (higher better specificity). We include the fraction of total voxels that contribute towards computing alignment in parenthesis. The values denote the noise ceiling threshold below which voxels are excluded. ParSM almost always yields higher precision by excluding voxels that lack clear correspondence.
MEIs for unit pairs from two ResNet-18 models trained with different random seeds, sampling from neurons ranked as highly-matched (top 10% of transport mass) versus poorly-matched (bot- tom 10%) by our metric. Fig. 7 shows striking differences: highly-matched pairs produce nearly identical MEIs, revealing that these units have converged on similar feature detectors despite inde- pendent training.
contrast, unmatched pairs yield divergent MEIs with distinct visual patterns, confirming they likely implement different computations. We demonstrate this with some additional results in Appendix A1.7.
Matched
Unmatched
Crand
Chassci
Chasze
Chaurl
Chaual
Chaszel
Churci
Chareel
Chaced in
Gaid
Crsned
Clanr
Choure
Figure 7: Visualization of (Un)matched Units Using Maximally Exciting Images. We show MEIs for two layers of R8s,Net-18, comparing units classified as matched or unmatched by the partial soft- matching metric. Matched examples sampled from the top 10% of partial soft-matching scores; unmatched examples are sampled from the bottom 10%.
4.5 TESTING FOR PRIVILEGED AXES IN ALIGNED NEURAL SUBPOPULATIONS
Neural networks could in principle encode information in arbitrarily rotated coordinate systems, yet recent evidence suggests they converge on specific "privileged" axes. Networks trained from different initializations not only share representational geometry but actually align their coordinate
Appendix A1.5

---

Published as a conference paper at ICLR 2026
systems- wwit individual neurons implementing similar computations across networks (Khosla & Williams, 2024; Khosla et al., 2024; Kapoor et al., 2025). This privileged basis hypothesis suggests that certain directions in activation space preferred, potentially due architectural constraints, e.g. axis-aligned nonlinearities (Rel.U). We ask whether this alignment holds when restricted to the best-matched neurons. Using partial soft-matching, we partition increasingly well-matched neurons and test for privileged axes from the full population down to the strongest matched pairs.
For two ImageNet- trained ResNet-18 models initialized with different random seeds, we extract representations {2, at each convolutional layer. To test for privileged apply
Aligunen Score
1ayer2. 1ayer2.1.corv
1ayor4.
Native Basis
Rotated Basis
0L5
Figure 8: Alignment between ResNet-18 models under original and ran- domly rotated coordinate systems, across early, middle and later layers and matched-mass thresholds. Rotation reduces alignment all thresholds- including among the best-matched units supportig convergence to shared coordinate system even among the most aligned subpopulations.
axes, random orthogonal transformation Q (sampled uniformly from the Haar mea- sure) to one network's representation and measure how this rotation affects alignment Spartial (X, neurons were arbitrarily oriented, this rotation would not affect the alignment. However, if a privileged basis exists, rotating away from should decrease alignment.
We test multiple regularization values (s) to sample subpopulations of varying aligmment quality. Fig. 8 reveals that alignment consistently decreases under rotation across for all and depths, with an identical pattern across all convolutional layers (Appendix A1.6). This demonstrates that privileged coordinate systems persist even among the most aligned neural subpopulations -te subset we might expect to be most robustly matched. The persistence of this effect suggests that coordinate alignment not merely a statistical artifact of analyzing many neurons together, but reflects true convergence on similar computational solutions at the single-unit unit level.
5 DISCUSSION
We introduced partial soft-matching, extending OT-based representational comparisons to account for partial correspondence between neural populations. This addresses key limitation of methods that force all units into alignment, which can obscure genuine matches in the presence of noise. Simulations show that the method preserves true correspondences under noise and selects the correct model in system identification tasks. In fMRI data, it excludes low-reliability voxels and improves alignment precision across homologous brain regions. In deep networks, matched units exhibit shared MEls, while unmatched units differ qualitatively. In both domains, partial soft-matching provides a more efficient way to order units by alignment quality, closely matching brute-force ablations, requiring only single optimization at each chosen mass regularization value.
Some limitations remain. The L-curve heuristic for selecting matched mass performs well empir- ically, but its generality is unclear. We list some good practices that practitioner should keep in mind while using the L-curve method in Appendix 11.1. Alternate strategies (e.g., area under the alignment-regularizztioo curve)- may offer more robust summarization across multiple regulariza- tion values. Because partial OT relaxes mass conservation and violates the triangle inequality, it should be understood as a comparative tool rather than strict metric. However, we note that re- cent theoretical work has developed partial Wasserstein variants that preserve full metric properties, including the triangle inequality (Raghvendra et al., 2024). Future extensions could integrate these formulations for applications requiring strict metric axioms, such clustering analyses. Although significantly faster than brute-force baselines, the O log n) cost can limit scalability to very large datasets. These considerations aside, this work highlights that meaningful comparison does not quire complete unit overlap: partial mott-nat enables principled analysis of convergent and divergent representational structure across neural systems.
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Emily J Allen, Ghislain Yee,,es, Yihan Wu, Jesse I. Breedlove, Jacob Prince, Logan Dowdle, Matthias Nau, Brad Caron, Franco Pestilli, Ian Charest, et massive 7t fmri dataset to bridge cognitive neuroscience and artificial intelligence. Nature neuroscience, 25(1): 16-126, 2022.
Pouya Bashivan, Kohitij Kar, and James J DiCarlo. Neural population control via deep image synthesis. Science, 36+1(6439)eaavv4436 2019.
Jean-David Benamou, Guillaume Carlier, Marco Cuturi, Luca Nenna, and Gabriel Peyré. Iterative bregman projections for regularized transportation problems. SIAM Journal on Scientific Com- puting, 372))A1111-11138 2015.
Laetitia Chapel, Mokhtar Z. Alaya, and Gilles Gasso, Partial optimal tranport with applications on positive-unlabeled learning. Advances in Neural Information Processing Systems, 33:2903-299 2913, 2020.
Lenaic Chizat, Gabriel Peyré, Bernhard Schmitzer, and François-Xavica Vialard. Scaling algorithms for unbalanced optimal transport problems. Mathematics of computation, 87(314):2533-2669 2018.
Alessandro Cultrera and Luca Callegaro. A simple algorithm to find the l-curve corner in the regu- larisation of ill- posee inverse problems. IOP SciNotes, 1(2):025004, 2020.
Jesús Loera and Edward D Kim. Combinatorics and geometry of transportation polytopes: An update. Discrete geometry and algebraic combinatorics, 625:37-76, 2013.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hier- archical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp.
2009. doi: 10.1 109/CVPR.2009 5206848.
Frances Ding, Jean-Staniste Stanislas Denain, and Jacob Steinhardt. Grounding representation similarity through statistical testing. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and Wortman Vaughan (eds.), Advances in Neural Information Processing Systems, volume 34, pp. 1556-1568. Curran Associates, Inc., 2021. URL /ttpt::::/pppccceeeeeeee neur cc pcppppperrrpppppp.pppppppppp ccppaeer__ppp paf............ppp......mm.....p
Dumitru Erhan, Yoshua Bengio, Aaron Courville, and Pascal Vincent. Visualizing higher-layer
features of a deep network. University of Montreal, 1341(3):1, 2009.
Chaitanya Kapoor, Sudhanshu Srivastava, and Meenakshi Khosla. Bridging critical gaps in conver- gent learning: How representational alignment evolves across layers, training, and distribution shifts. arXiv preprini 2rXiv:2502.187111 2025.
Meenakshi Khosla and Alex H Williams. Soft matching distance: A metric on neural representations that captures single-neuron tuning. In Proceedings of UniReps: the First Workshop on Unifying Representations in Neural Models, pp. 326-341. PMLR, 2024.
Meenakshi Khosla, Alex H Williams, Josh McDermott, and Nancy Kanwisher. Privileged represen- tational axes in biological and artificial neural networks. bioRxiv, pp. 2024-06, 2024.
Simon Kornblith, Mohammad Norouzi, Honglak Lee, and Geoffrey Hinton. Similarity of neural network representations revisited. In International conference on machine learning, pp. 3519- 3529. PMIR, 2019.
Nikolaus Kriegeskorte, Marieke Mur, and Peter A Bandettini. Representational similarity analysis- connecting the branches systems neuroscience. Frontiers in systems neuroxcience, 2:249, 2008.
Richard D Lange, Devin Kwok, Jordan Kyle Matelsky, Xinyue Wang, David Rolnick, and Konrad Kording. Deep networks as paths on the manifold of neural representations. In Timothy Doster, Tegan Emerson, Henry Kvinge, Nina Miolane, Mathilde Papillon, Bastian Rieck, and Sophia Sanborn (eds.), Proceedings of 2nd Annual Workshop Topology, Algebra, and Geometry in volume 221 of Proceedings of Machine Learning Research, pp- 102-133. PMLR, 28 Jul 2023.
Machine Learning
11

---

Published as a conference paper at ICLR 2026
Gabriel Peyré, Marco Cuturi, al. Computational optimal transport: With applications data science. Foundations and Trends@ in Machine Learning, 11(5-6):355-677, 201.
Pawel Pierzchlewiez, Konstantin Willeke, Ame Nix, Pavithra Elumalai, Kelli Restivo, Tori Shinn, Cate Nealley, Gabrielle Rodriguez, Saumil Patel, Katrin Franke, et al. Energy guided diffusion for generating neurally exciting images. Advances in Neural Information Processing Systems, 36: 32574-3220 32601, 2023.
Maithra Raghu, Justin Gilmer, Jason Yosinski, and Jascha Sohl-Dickstein. Sveca: Singular vector canonical correlation analysis for deep learning dynamics and interpretability. Advances in neural information processing systems, 30, 2017.
Sharath Raghvendra, Pouyan Shirzadian, and Kaiyi Zhang. A new robust partial p-wasserstein-basel metric for comparing distributions. arXiv preprint arXiv:2405. 405.03664, 2024.
Imran Thobani. Javier Sagastuy-Brent Aran Nayebi, Jacob Prince, Rosa Cao, and Daniel LK Yamins. Model-brain comparison using inter-animal transforms. In 8th Anmual Conference on Cognitive Computational Neuroscience, 2025.
Edgar Y Walker, Fabian Sinz, Erick Cobos, Taliah Muhammad, Emmanouil Froudarakis, Paul G Fahey, Alexander S Ecker, Jacob Reimer, Xaq Pitkow, and Andreas S Tolias. Inception loops discover what excites neurons most using deep predictive models. Nature neuroscience, 22(12): 2060-2063 2019.
Alex H Williams, Erin Kunz, Simon Kornblith, and Scott Linderman. Generalized shape metrics neural representations. Advances in neural information processing systems, 34:4738-4750, 2021.
12


Reviews:
[{"review_id": "0QRXcLA2ke", "reviewer": "Reviewer_Ny7g", "summary": "Existing representational similarity metrics frequently fall into two categories: rotation-invariant methods (e.g., CKA, RSA) that measure overall geometric similarity but obscure neuron-level correspondence, or rotation-sensitive methods (e.g., standard soft-matching) that find such correspondences but are brittle, as they inherit the \"critical limitation\" of classical optimal transport (OT) by forcing a complete, one-to-one match between all units. This paper identifies the latter's inadequacy in the common scenario of comparing noisy biological data (fMRI) or functionally distinct deep neural networks (DNNs), where correspondence is inherently partial. It introduces the \"unbalanced soft-matching distance,\" a principled extension of soft-matching to a partial OT framework. This method permits a fraction of units to remain unmatched, governed by a data-driven regularization parameter selected via an L-curve heuristic. The authors demonstrate through simulations that the method is robust to spurious \"outlier\" neurons and correctly identifies models with greater signal overlap in a task where standard soft-matching fails.", "questions": "1. The method relies on a squared Euclidean distance cost (Section 2.2). How sensitive are the resulting unit partitions and alignment rankings to this choice, and have alternative cost functions (e.g., cosine distance) been investigated?\n2. How does the L-curve heuristic's second-derivative approximation perform on smooth cost-regularization curves that lack a distinct \"elbow,\" and what is the method's failure mode in such cases?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. The method is theoretically well-grounded, providing a principled extension of optimal transport to solve the well-documented problem of spurious alignments in noisy, partially corresponding neural data.\n2. The paper provides compelling evidence of practical utility by showing that a single $\\mathcal{O}(n^3 \\log n)$ optimization can rank unit alignment as accurately as a computationally prohibitive $\\mathcal{O}(n^4 \\log n)$ brute-force ablation.\n3. The method provides highly interpretable outputs by partitioning units into \"matched\" and \"unmatched\" subsets.", "weaknesses": "1. The method's autonomy relies entirely on an L-curve heuristic to select the optimal matched mass, a technique the authors concede has unclear generality and is known to be unstable in non-ideal (e.g., smooth) cost-regularization landscapes.\n2. By relaxing mass conservation, the proposed method sacrifices formal metric properties, specifically the triangle inequality, limiting its use in downstream algorithms or theoretical proofs that require a true metric.\n3. The \"correlation-based ordering\" heuristic used as a fast baseline appears to be a strawman, as its catastrophic failure is predictable; a more competitive baseline, such as a greedy matching algorithm, was not included for comparison.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "nrskwKoI3E", "reviewer": "Reviewer_SBuW", "summary": "The paper extends soft matching for neuron to neuron alignment to a partial optimal transport setting that can leave units unmatched. It is rotation sensitive (in some cases, a desired quality) yet robust to noise and unequal population sizes. They use simulations, fMRI across NSD subject pairs, and DNN experiments to show that the method filters out low-quality units, improves cross-area precision, and closely tracks brute force ablations at far lower cost.", "questions": "1. The variables p and q are used in line 108 without being explained or introduced. \n\n2. For choosing optimal regularization, the authors first introduce a 2D-valued function f(s)=[\\zeta(s), \\rho(s)], with a justification that looking at both quantities is necessary for balancing the transport distance and reg. strength. But then the authors proceed to ignore \\rho(s) when computing the maximal positive curvature. Why is that?\n\n3. I don’t think I understand the Figure 2 left image. What is the red box and the yellow line? If both X and Y have 100 signal neurons, then shouldn’t the matching combinations form a square submatrix of size 100x100?", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 2, "contribution": 4, "strengths": "In summary, I think the originality and significance of this paper are good.\n\n- I believe the paper addresses an important and timely problem. \n\n- Recasting as average matched correlation gives an interpretable score in [−1,1], which helps. \n\n- Experiments seem thorough. Simulations show recovery with outliers and correct model selection. NSD shows exclusion of low noise ceiling voxels and higher within-area precision than balanced soft matching. DNNs show matched units have similar MEIs and that alignment drops after random rotations, consistent with privileged axes.", "weaknesses": "- Novelty relative to existing unbalanced OT and partial matching methods could be clearer. The authors cite Chapel et al. and related work, but I am not sure what is gained over the existing partial OT with dummy nodes plus a simple threshold.\n\n- My understanding is that the tuning curves are centered as unit-normalized before being used for the unbalanced soft-matching. However, there are many cases where the neuronal units have very small responses (dead neurons). This normalization scheme would blow up the activities of these dead neurons, and I think that would affect the matching result. I think the authors should address the sensitivity of UnSM to the preprocessing methods.\n\n- The clarity of the paper can be improved. Readers would appreciate a more detailed explanation of the original soft-matching algorithm at the beginning. The authors can use that opportunity to define variables more carefully. Currently, variables such as p, and q, and terminologies like \"mass\" are not defined clearly before being used. The mathematical part of the paper is currently only really readable to readers who are already familiar with the OT literature or the softmatching paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "s0r66Q7baT", "reviewer": "Reviewer_QEdB", "summary": "This paper proposes an extension of the soft matching distance for measuring representational alignment. The idea is to account for noisy, low-reliability units by relaxing the strict mass conservation of soft matching, thus allowing for unmatched units. The authors also provide a method for selecting the hyperparameter of the method, the optimal amount of unmatched mass. In simulation experiments, they show that their metric is able to select the better model of simulated data, while standard soft matching fails. They also test their method on real fMRI data (NSD), finding that it achieves higher ROI-matching accuracy than standard soft matching.", "questions": "- To address the main criticism of the metric being too flexible: It might be necessary in practice to combine unbalanced soft matching with train-test split evaluations, where you choose s (and the units to be discarded) on the train split, to then compute the similarity score on the test split.\n- Will you release a reference implementation to compute unbalanced soft matching? \n- Section 4.3: How exactly does the matching of similar brain regions work? I currently understand that you select two ROIs in two subjects each, then compute a between-subject matching for all voxels of these regions, to then measure the fraction of units that were correctly mapped to the corresponding region in the other subject. Is this correct? The paper could be a bit more detailed here.\n- For voxels of fMRI readings, there sometimes are uncertainty measures of individual voxels based on estimates of SNR. If you threshold units by this uncertainty measure, i.e. exclude them if they are too unreliable, does that match against the units your algorithm excludes? How do results in table change 1 if you compute soft matching but exclude unreliable units?\n- How does $m_{reg}$ relate to $s$? Is it simply $m_{reg} = s$?\n- I was initially confused about the notation $\\mathcal{T}^{u}(N_x, N_y)$, because the $u$ is never mentioned. I now think this simply stands for \"unbalanced\", is that correct? Should this maybe be $\\mathcal{T}^{s}(N_x, N_y)$? I would have found it logical, notation-wise, to include s in the reference to the set of natural solutions, because the set is parameterized in s.\n- Figure 7 states 3 layers but I think it should say 2.\n- Grammar in lines 192, 216.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "- The paper addresses a relevant question that is of great interest to the community.\n- It adds a sensible extension to soft matching, creating a promising candidate for a good neural similarity measure. The proposed unbalanced soft matching is well-motivated and initial sanity checks are promising.\n- The writing is clear and logical.\n- I see no issues with quality or correctness.", "weaknesses": "- Arguably, the fact that there are unmatched units between two systems is a difference that we might not want to ignore. Suppose there were two essentially identical candidate models to be evaluated, but one has (many) random noise units added. Unbalanced soft matching with optimally chosen s (for each model) would not find a difference between these models, correct? Standard soft matching thus has an implicit bias for simpler models, which might be desirable, but is somewhat lost by your method (unless one uses the s to decide).\n- On a similar note, the metric suffers from the fact that I can trivially achieve optimal similarity by just inflating my model: Because the metric is free to drop what it doesn't need, any sufficiently large model should eventually achieve perfect similarity (at potentially large values of s). This won't generalize to a test set, though. \n\nOverall, while there is a potential weakness of the proposed method, I do think that the paper adds a valuable proposition to the research field of representational similarity.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)