Aligning individual brains with fused unbalanced Gromov Wasserstein
Authors: Alexis Thual
Quang Huy TRAN
Tatiana Zemskova
Nicolas Courty
Rémi Flamary
Stanislas Dehaene
Bertrand Thirion
Venue: NeurIPS 2022 Accept (2022)
Keywords: Brain imaging
Optimal transport
fMRI
registration

Abstract: Individual brains vary in both anatomy and functional organization, even within a given species. Inter-individual variability is a major impediment when trying to draw generalizable conclusions from neuroimaging data collected on groups of subjects. Current co-registration procedures rely on limited data, and thus lead to very coarse inter-subject alignments. 
In this work, we present a novel method for inter-subject alignment based on Optimal Transport, denoted as Fused Unbalanced Gromov Wasserstein (FUGW). The method aligns two cortical surfaces based on the similarity of their functional signatures in response to a variety of stimuli, while penalizing large deformations of individual topographic organization.
We demonstrate that FUGW is suited for whole-brain landmark-free alignment. The unbalanced feature allows to deal with the fact that functional areas vary in size across subjects. Results show that FUGW alignment significantly increases between-subject correlation of activity during new independent fMRI tasks and runs, and leads to more precise maps of fMRI results at the group level.

Aligning individual brains with Fused Unbalanced Gromov- Wasserstein
Alexis Thual
Cognitive Neuroimaging Unit, INSERM, CEA, CNRS, NeuroSpin center, Gif sur Yvette, France Mind, Inria Paris-Saclay, Palaiseau, France Inserm, Collège de France, Paris, France alexis. thual@cea.fr
Huy Tran
CMAP, Ecole Polytechnique. Palaiseau, France Université Bretagne-Sut Sud, CNRS, IRISA, Vannes, France quang huy tran@univ-ubs.c frr
Tatiana Zemskova
Mind, Inria Paris-Saclay, Palaiseau, France tatiana remsskorraapllllttecniiuuue edu
Nicolas Courty
Université Bretagne-Sud, CNRS, IRISA, Vannes, France ncourtydirisa.fr
Rémi Flamary
CMAP, Ecole Polytechnique, Palaiseau, France
remi, mlmillllllarry@ooeeemmmiuuu edu
Stanislas Dehaene
Cognitive Neuroimaging Unit, INSERM, CEA, CNRS, NeuroSpin center, Gif sur Yvette, France Inserm, Collège de France, Paris, France
stanislas. dehaene@cea.ffr
Bertrand Thirion
Mind, Inria Paris-Saclay, Palaiseau, France bertrand. thirion@inri
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

Abstract
Individual brains vary in both anatomy and functional organization, within given species. Inter-individual variability is a major impediment when trying to draw generalizable conclusions from neuroimaging data collected on groups of subjects. Current co-registration procedures rely on limited data, and thus lead to very coarse inter-subject alignments. In this work, we present novel method for inter-subject alignment based on Optimal Transport, denoted as Fused Unbalanced Gromov Wasserstein (FUGW). The method aligns cortical surfaces based on the similarity of their functional signatures in response to variety of stimulation settings, while penalizing large deformations of individual topogruphic organization. We demonstrate that FUGW is well-suited for whole-brain landmark-free alignment. The unbalanced feature allows to deal with the fact that functional areas vary in size across subjects. Our results show that FUGW alignment significantly increases between-subject correlation of activity for independent functional data, and leads to more precise mapping at the group level.
Introduction
The availability of millimeter or sub-millimeter anatomical or functional brain images has opened new horizons to neuroscience, namely that of mapping cognition in the human brain and detecting markers of diseases. Yet this endeavour has stumbled on the roadblock of inter-individual variability: while the overall organization of the human brain largely invariant, two different brains (even from monozygotic twins (33 may differ at the scale of centimeters in shape, folding pattern, and functional responses. The problem is further complicated by the fact that functional images are noisy, due to imaging limitations and behavioral differences across individuals that cannot be easily overcome. The status quo of the field thus to rely on anatomy- based indiviaaviaall alignment that approximately matches the outline of the brain as well as its large-scale cortical folding Existing algorithms thus coarsely match anatomical features with diffeomorphic transformations, by warping individual data to simplified template brain. Such methods lose much of the original individual detail and blur the functional information that can be measured brain regions (see Figure[].
patterns 12 15)
sub-04
sub-07
sub-09
average all subjects
mesh fsaverage hlividua
Z-score
6.0
4.0
2.0
0.0
-2.0
-4.0
-6.0
Figure 1: High variability in human anatomies and functional MRI responses across subjects In this experiment contrasting areas of the brain which respond to mathematical tasks against other that don't. i, we observe great variability in locations and strength of brain activations across subjects (row 1). The classical approach consists in wrapping this data common surface template (row 2). where they can be averaged, often resulting in loss of individual details and detection power. These images were generated using Nilearn software T
In order to improve upon the current situation, a number of challenges have to be addressed: (i) There exists no template brain with functional information, which by construction renders any cortical matching method blind to function. This is unfortunate, since functional information is arguably the most accessible marker to identify cortical regions and their boundaries [T8). (ū) When comparing two brains coming from individuals from template- it is unclear what regularity should be

---

imposed on the matching [42]. While it is traditional in medical imaging to impose diffeomorphicity such a constrain does not match the frequent observation that brain regions vary across individuals in their fine-grained functional organization [18] 38 (iii) Beyond the problem of aligning human brains, it an even greater challenge to systematically compare functional brain organization two
different species, such as humans and macaques [26] 23 46 14). Such inter-species comparisons
introduce a more extreme form of variability in the correspondence model.
Related work Several attempts have been made constrain the brain alignment process by using functional information. The first one consists in introducing functional maps into the diffeomorphic framework and search for a smooth transformation that matches functional information [37
47 35
the most popular framework being arguably Multimodal Surface Matching (MSM) [35
18
A second family of less constrained functional alignment approaches have been proposed. based
on heuristics, by matching information in small, possibly overlapping, cortical patches [2T] 40 6
This popular framework has been called hyperalignment [21] 20]. shared response models [10).
Yet these approaches lack a principled framework and cannot be considered to solve the matching problem at scale. Neither do they allow to estimate group-level template properly (45)
An alternative functional alignment framework has followed another path (99, considering functional signal as three dinensinnni distribution, and minimizing the transport cost. However, this framework imposes unnatural constraints of nog-tviat of the signal and only works for one-dimensional contrasts, that it cannot be used to learn multi-dimensional matomo-functione structures. An important limitation of the latter two families of methods is that they operate on a fixed spatial context (mesh voxel grid), and thus cannot be used on heterogeneous meshes such as between two individual human anatomies or, worse, between a monkey brain and human brain.
Contributions Following [5]. we use the Wasserstein distance between source and target functional signals consisting of contrast maps acquired with fMRI- to compute brain alignments. We contribute two notable extensions of this framework: (i) a Gromov-Wasserster (GW) term to preserve global anatomical structure this term introduces an anatomical penalization against improbably distant anatomical matches, yet without imposing diffeomorphic regularity as well as (ii) an unbalanced correspondence that allows mappings from one brain to another to be incomplete, for instance because some functional areas are larger in some individuals than in others, or may simply be absent. We show that this approach successfully addresses the challenging case of different cortical meshes, and that derived brain activity templates are sharper than those obtained with standard anatomical alignment approaches.
2 Methods
Optimal Transport yields natural framework to address the alignment problem, as it seeks to derive a plan - a coupling that can be seen as soft assignment matrix between cortical areas of source and target individual. As discussed previously, there is a need for functional alignment method that respects the rich geometric structure of the anatomical features, hence the Wasserstein distance alone is not sufficient. By construction, the GW distance [24 25 can help preserve the global geometry underlying the signal. The more recent fused GW distance 44 goes one step further by making possible to integrate functional data simultaneously with anatomical information.
2.1 Fused Unbalanced Ctrmov--assersee
We leverage 44 39 to present a new objective function which interpolates between a loss preserving the global geometry the underlying mesh structure and a loss aligning source and target features, while simultancously allowing not to transport some parts of the source and target distributions. We provide an open-source solver that minimizes this los!"!
Formulation We denote F R the matrix of features per vertex for the source subject. In the proposed application, they correspond to e functional activation maps, sampled on a mesh with n vertices representing the source subject's cortical surface. Let D* e R be the matrix of pairwise
httgss//gghuubbccommmmssssss///s provides PoTor
provids
28 solver with scikit-learn
29 compatible API
3

---

geodesic distances between vertices of the source mesh. Moreover, we assign the distribution w" E Ry on the source vertices. Comparably, we define F RP... D' R and wt R for the target subject, whose individual anatomy is represented by mesh comprising p vertices. Eventually, wh and w² set the transportable mass per vertex, which, without prior knowledge, we choose to be uniform for the source and target vertices respectively: w a (n.... n) wt (p....p)
Given a tuple of hyper-parameters 0 A (p, E), where p, E R4 and [0, 1], for any coupling P E Rm we define the fused unbalanced Gromov- Wasserstein loss as
Wasserstein loss (P()
Gromov-Wasserster Wasserstein loss Low (P)
(P(P) A ar)
Σ
F
+ t
Σ oSi,en 0SJJEc
D - D
oSi<n 0S/<p
(1)
p (L((#) Pw wh) + (L #2 & P#2lwo @ w E(P)
+ s
Marginal constraints Lu(P)
Entropy
where (P) matches vertices with similar features, Low penalizes changes in geometry and Lu (P)) fosters matching all parts of the source and target distributions. Throughout this paper, we refer to relaxing the hard marginal constraints of the underlying OT problem into soft ones as unbalancing. Here, P#1 A (2, Pro)osien denotes the first marginal distribution of P. and
P#2 (E; P.,loss<p the second marginal distribution of P. The notation represents the Kronecker product between two vectors or two matrices. KL(-I-) denotes the Kullback Leibler divergence, which is a typical choice measure the discrepancy between two measures in the context of unbalanced optimal transport (22). The last term E(P) A KL (w w² ) (w* & w²)) is mainly introduced for computational purposes, as it helps accelerate the approximation scheme of the optimisation problem. Typically, is used in combination with small value of E, so that the impact of other terms is not diluted. On the other hand, the parameters a and offer control over two other aspects of the problem: while a realizes a trade-off between the impact of different features and different geometries in the resulting alignment, P controls the amount of mass transported by penalizing configurations such that the marginal distributions of the transportation plan P are far from the prior weights w" and w². This potentially helps adapting the size of areas where either the signal or the geometry differs too much between source and target.
Eventually, we define X A (F*, Do, w²) and X A (Fr, Dt. w²), and seek to derive an optimal coupling P E RR- minimizing
FUGW x²) A inf LP(P)
(2)
P20
This can be seen as a natural combination of the fused GW [44 and the unbalanced GW (39 distances. To the best of our knowledge, it has never been considered in the literature. Following 39 we approximate FUGW via a lower bound. It involves solving a minimization problem with respect to two independent couplings: Using minimize with respect to the other. This allows us to always be dealing with linear problems instead of a quadratic one. Eventually, each BCD iteration consists in alternatively solving two entropic unbalanced OT problems, whose solutions can be approximated using the scaling algorithm (TT). Details concerning the lower bound well the corresponding BCD iteration can be found in the Appendix (see Alg. S1).
Descent (BCD) scheme, we fix coupling and
Toy example illustrating the unbalancing property As exemplified in Figure brain responses elicited by the same stimulus vary greatly between individuals. Figure 2 illustrates a similar yet simplified version of this problem, where the goal is to align two different signals supported on the same spherical meshes. In this example, for each of the n p 3200 vertices, the feature is simply a scalar. On the source mesh. the signal is constituted of two von Mises density functions that differ by their concentration (large and small), while on the target mesh, only the large one is present, but different location. We use the optimal coupling matrix P obtained from Eq. to transport the source signal on the target mesh. As shown in Figure 2 B, the parameter p allows to control the mass
We compute geodesic distances using https://github..oom/// virtual- crantual-bbain//b-gddii..

---

Balanced (p 100)
%
15
10
Unbalanced (p = 1)
-15
Source
Target features
features
Source features transported target Mass change (%)
Source
Target
Figure 2: Unbalancing helps accounting for idiosyncrasies of the source and target signals When trying align the source and target signals (Panel A), the classical balanced setup (Panel B, top row) transports all parts of the source signal even if they have no counterpart in the target signal. In the unbalanced setup (Panel B, bottom row), less source-only signal is transported: particular, less mass is transported from the source's 's small blob onto the target (Pancl B, middle column).
transferred from source to target. When p 100. we approach the solution of the fused GW problem. Consequently, we observe the second mode on the target when transporting the source signal. When the mass control weaker (p 1), the smaller blob is partly removed because it has no counterpart in the target configuration, making the transport ill-posed.
Barycenters Barycenters represent common patterns across samples. Their role is instrumental in identifying a unique target for aligning given group of individuals. As seen in Fig. [ the vertex-wise group average does not usually provide well-contrasted maps. Inspired by the success of the GW distance when estimating the barycenter of structured objects [30] 44), we use FUGW to find the barycenter (FB, D IR&, IR&.k of all subjects pe, from each subject to the barycenter. More precisely, we solve
well the corresponding couplings
x
(FB DB BBBBBBBBBBB
E arg min Σ FFUGW(X", X)),
X
where we set the weights wy to be the uniform distribution. By construction, the resulting barycenter benefits from the advantages of FUGW, i.e. equilibrium between ceometry-preserviie and feature- matching properties, while not forcing hard marginal constraints. The FUGW barycenter is estimated using a Block-Coordinat Descent (BCD) algorithm that consists in alternatively (i) minimizing the OT plans pe.B for each FUGW computation in (3) with fixed B and (ūi) updating the barycenter B through closed form with fixed pe.B See Alg. $4 for more details.
3 Numerical experiments
We design three experiments to assess the performance FUGW. In Experiments and 2, we are interested in assessing if aligning pairs of individuals with FUGW increases correlation between subjects compared to a baseline correlation. We also compare the ensuing gains with those obtained when using the competing method MSM 335 36 to align subjects. In Experiment 3, we derive barycenter of individuals and assess its ability to capture fine-grained details compared to classical methods.
Dataset In all three experiments, leverage data from the Individual Brain Charting dataset [3] It is a longitudinal study on 12 human subjects, comprising 400 fMRI maps per subject collected on a wide variety of stimuli (motor, visual, auditory, theory of mind, language, mathematics, emotions, and more). movie- wertcin data, TI T1-weighted maps, as well as other features such as retinotopy which we don't use in this work. We leverage these 400 fMRI maps. The training, validation and test sets respectively comprise 326, 43 and 30 contrast maps acquired for each individual of the dataset. Tasks and MRI sessions differ between each of the sets. More details, including preprocessing. are provided in Supplementary Materials.
5

---

Baseline alignment correlation For each pair of individuals (s, under study, and for each fMRI contrast in the test set, we compute the Pearson correlation co .et F.e) after these maps have been projected onto a common surface anatomy (in this case, fsaverage5 mesh). Throughout this work, such computations are made for each hemisphere separately.
Aligning pairs of humans with the same anatomy For each pair (8, 1) under study, derive an alignment pe. Raxp using FUGW on set of training features. In this experiment, source and target data lie on the same anatomical mesh (fsaverage5), and n. = p = 10 240 for each hemisphere. Since each hemisphere's 's mesh is connected, we align one hemisphere at time.
Experiment 1
Computed couplings are used to align contrast maps of the validation set from the source subject
onto the target subject. Indeed, one can define Φ X E R"*4 + (( X) @ p&t E [Rxx where 2 represents the element- wise division. d. transports any matrix of features from the source mesh to the target mesh. We measure the Pearson correlation corr ( ( (F),, Ft) between each aligned source and target maps.
#2
We run similar experiment for MSM and compute the correlation gain induced on test set by FUGW and MSM respectively. For both models, we selected the syper-parameters maximizing correlation gain on a validation set. In the case of FUGW, in addition to gains in correlation, hyper-parameter selection was influenced by three other metries that help us assess the relevance of computed couplings:
Transported mass For each vertex i of the source subject, we compute
Σ OS/<P
P
Vertex displacement Taking advantage of the faet that the source and target anatomies are the same, we define D D* D' and compute for each vertex of the source subject the quantity Σj P.j. Da.s/ E, pss.. which measures the average geodesic distance on the cortical 1.j sheet between vertex and the vertices of the target it has been matched with
Vertex spread Large values of increase the entropy of derived couplings. To quantify this effect, and because we don't 't't want the matching be too blurry, we assess how much vertex was spread. Considering P. p.t E, P.a E IR probability measure on target vertices, estimate the anatomical variance of this measure by sampling 9 pairs (ig. kig) of P. and computing their average geodesic distance
Σ DDi Diank
fg.kg
Experiment 2 Aligning pairs of humans with individual anatomies We perform second align- ment experiment, this time using individual meshes instead of an anatomical template. Importantly, in this case, there is no possibility to compare FUGW with baseline methods, since those cannot handle this case.
However, individual meshes are significantly larger than the common anatomical template used in Experiment (n R m Rs 160k vs. 10k previously), resulting couplings too large to fit on GPUs for reference, coupling of size 10k x 10k already weights 400Mo on disk. We thus reduce the size of the source and target data by clustering them into 10k small connected clusters using Ward's algorithm [1 More details are given in supplementary section A.4.
Experiment 3 Comparing FUGW barycenters with usual group analysis Since it is very difficult to estimate the barycentric mesh, we force it to be equal to the fsaverage5 template. Empiri- cally, this we force the distance matrix DB to be equal to that of fsaverage5, and only estimate the functional barycenter FB We initialize it with the mean of (F*).es and derive F and (Ps,B) ))Es from problem
)aes
Then, for a given stimulus c, we compute its projection onto the barycenter for each subject. We use these projections to compute across subjects and (ii) TR.c. the t-statistic (for each vertex) of projected maps. We compare these two maps with their unaligned counterparts Mo,c and To,e respectively.
maps of interest: (i) MB,e the mean of projected contrast maps
The first map helps us to qualitatively evaluate the precision of FUGW alignments and barycenter. The second one is classically used to infer the existence of areas of the brain that respond to specific

---

MBC
P
Σ
FF
TBCc i-statistic
(( cEs))
|S|
SES
Mo.e
F
To.c - t-statistic (F. (F.C))Es)
1
stimuli. We assess whether FUGW helps find the same clusters of vertices. Eventually, we quantify the number of vertices significantly activated or deactivated with and without alignment respectively,
4 Results
4.1 Experiment 1 Template anatomy
Aligning subjects fixed mesh We set 0.5, p and 10-3, Pearson correlation between source and target contrast maps is systematically and significantly increased when aligned using FUGW, as illustrated in Figure where correlation grows by almost 40% from 0.258 0.356.
0.258
0,258
baseline
baseline
+0,000
0.043
0.
098 = 0:035
msmm
fugw
0.20 0.22 0.24 0,26
0.32
0.34
0.20
0.25
0.30
0.35
0.40
0.45
Pearson correlation
Pearson correlation
Figure 3: Comparison of gains correlation after inter-subject alignment For each pair of source and target subjects of the dataset, compute the average Pearson correlation between 30 test contrasts, leading to the (baseline) correspondence score, and compare with that of the same contrast maps aligned with either MSM (left) FUGW (right). Correlation gains much better for FUGW.
We also varied training sets by selecting subsets of training contrasts and find that similar performance on the test set can be achieved regardless of the training data (see Supplementary section A.5 and in particular Supplementary Table S1).
used to obtain these results were chosen after running grid search o, E and p and evaluating it on the validation dataset. Computation took about 100 hours using 4 Tesla V100-DGXS -0--DGXX--2GBBB GPUs. More precisely, it takes about minutes to compute one coupling between a source and target 10k-vertex hemisphere on a single GPU, when the solver was set to run 10 BCD and 400 Sinkhorn iterations. In comparison, MSM takes about the same time on Intel(R) Xeon(R) CPU E5-2698 v4 @ 2.20GHz CPUs. Results are reported in Figure and provide multiple insights concerning FUGW.
hyper-parameteer selection
Firstly, without anatomical constraint (a 0), source vertices can be matched with target vertices that are arbitrarily far on the cortical sheet. Even though this can significantly increase correlation, it also results in very high vertex displacement values (up to 100mm)). Such couplings plausible. Secondly, without functional information (a = 1), couplings recover a nearly flawless matching between source and target meshes, so that, when 10-5 (ic when we force couplings to-single-vertee matches), vertex displacement and spread are close to 0 and correlation is unchanged. Fusing both constraints (0
anatomically
to find single-vertex- hingle-vertex-to-sinnlle---e
1) yields the largest gains in correlation while allowing to compute anatomically plausible reorganizations the cortical sheet between subjects.
The impact of p (controlling marginal penalizations) correlation seems modest, with slight tendency of increased correlation in unbalanced problems (low p).
Finally, it is worth noting that a relatively wide range of a and p yield comparable gains. The fact that FUGW performance weakly sensitive to hyper-parameters makes a good off-the-shelf tool for neuroscientists who wish to derive inter- inter-individual alignments. However, is of dramatic importance
7

---

in computed results and should be chosen carefully. Vertex spread is a useful metric to choose sensible values of E; for human data one might consider that should not exceed 20mm.
Correlation gain (pearson)
Transported mass (%)
Vertex dement (mm)
spread (mm)
p =
0.1
0-0.002 0000
0.080
9.263
6:15
75
70.3
65.7
64.7
100
70.4
70.
11.5
25.7
100
15.8
15.7
300
0.033
0.40
0:00
0.250
0.25
22.8
23.1
3.44
7.22
5.05
50
6.5 6505
0:066
0.121
0.254
05
87.
85
96.2
77.1
0.5
13.9
14
16.2
64.9
9.5 2.48
5.73
15.1
111
0.015
0:210
0.35
55.0
2.04
-0 05
20
20
1.0000
0.024
0.054
0.047
99.8
46.5
90.2
1-9453
3.15
66.1
3-5.802
3.72
12.5
69.7
10:25
16:44
16:05
16:02
10:25
1o:04
10:03
92
1c:05
16:44
16:03
10:22
26:05
16:04
16-23
16-22
p
100
100
100
45
0.25
0.094
0.049
0.000
0.067
9.25
98
97.3
97.4
0.25
21
23.1
22.6
96.1
0.25
2.86
7.35
19.9
73.2
0.5
0.008
0.044
0.100
13.3
2.25
5,054
0.077
0:16
1.20
3.75
9
96.7
40.4
0.75
8.22
0.35
13.7
0.75
2.05
4.63
12.2
55.
05
73
20
3-0.002
0.024
0.254
0.033
99.
92.5
99.1
1.49
3.15
43.7
1.52
3.73
16-35
16:04
Te-da
1e-02
16-55
1e-01
Te-01
1e-02
1e-05
1e00
Te00
1e-32
16-05
1e-06
3e-33
1e-02
6:30
80
025 0.005 0.040
0:577
3.372
3.25
92
95
93.2
93
90
0.25
21
21
22.5
15.0
022 2.92
7.26
10.9
737
6.00
0.75
0.004 0.021
0:035
3.230
3.75
:02
102
93.2
93.8
0.75
8.21
8.34
11.9
52.9
0.72
1.25
4.52
12.
549
75
16-25
26:46
16:05
1c:02
16:25
1::04
16:33
16:02
16:05
16:04
26:33
16-22
20:03 16:04
20-23
18-22
Figure 4: Exploring hyper-parameter space to find relevant couplings Given transport plan aligning source and target subject, we evaluate how much this coupling (left) improves correlation between unseen contrast maps of the two subjects, (center left) actually transports data, (center right) moves vertices far from their original location on the cortical surface and (right) spreads vertices on the cortical sheet. We seek plans that maximize correlation gain, while keeping spread and displacement low enough.
Mass redistribution in unbalanced couplings Unbalanced couplings provide additional informa- tion about how functional areas might differ in size between pairs of individuals. This illustrated in Figure where we observe variation in size of the auditory area between given pair of individuals. This feature is indeed captured by the difference mass between subjects (although the displayed contrast was not part of the training set).
A
2-score
B
9
80
-A
44
130
sub-07 (source)
sub-09 (target)
transported mass (%)
fMRI contrast maps
source target
Figure Transported mass indicates areas which have be resized between subjects (Panel A) We show a contrast map from the test set which displays areas showing stronger activation during auditory tasks versus equivalent visual tasks. shows much more anterior activations on the target subject compared to the source subject. This is consistent with the observation that more mass is present in anterior auditory areas of the source subject than the target subject (Panel B).

---

0,258
baseline
+8.089
0.050
fugw
0.20
0.25
0.30
0.35
0.40
0.45
Pearson correlation
Figure 6: Correlation between pairs of subjects is significantly better after alignment individual anatomies than after projecting subjects onto a common anatomical template
z-score
t-statistic
2.0
1.5
1.0
group analysis on all subjects
group analysis on all subjects aligned barycenter
of all subjects of all subjects aligned to barycenter
0.5
Figure 7: FUGW barycenter yields mueh finer-grained maps than group averages We study the same statistical map as in Figure which contrasts areas of the brain involved in mathematical reasoning. A. These complex maps projected onto the barycenter and averaged show more specific activation patterns than simple group averages, especially in cortical areas exhibiting more variability, such as the prefrontal cortex. B. Deriving t-test on aligned maps captures the same clusters as the classical approach (plain green circles), but also new clusters areas where inter-subject variability is high (dotted black circles). Peak t-statistics are also higher with FUGW. C. Ratio of number of activated vertices (Itsstatistic] 4) with versus without alignment for each map of the test set. Our method finds significantly more of such vertices (p- value = 3- 1)_4).
4.2 Experiment 2 Individual anatomies
As shown in Figure 6 we obtain correlation gains which are comparable to that of Experiment (about 35% gain) while working individual meshes. This tends to show that FUGW compute meaningful alignments between pairs of individuals without the use of an anatomical template, which helps bridge most conceptual impediments listed in Section]
Moreover, this opens the way for computation of simple statistics in cohorts of individuals in the absence of template. Indeed, can pick individual of the cohort and use it reference subject on which to transport all other individuals. We give an example in Figure S4, showing that FUGW correctly preserved idiosyncrasies of each subject while transporting their functional signal in an anatomically sound way.
4.3 Experiment 3 Barycenter
In the absence proper metric to quantify the correctness of barycenter, we first qualitatively compare the functional templates obtained with and without alignment. In Figure 7 A, we do so using brain maps taken from the test set. We can see that the barycenter obtained with FUGW yields sharper contrasts and more fine-grained details than the barycenter obtained by per-vertex averaging. We also display in Figure B the result of one-sample test for the same contrast, which can readily be used for inference. The onemsa test map obtained after alignment the FUGW template exhibits the same tupra-threshold clusters as the original approach, but also some additional spots which were likely lost due to inter-subject variability the fsuverage5 space. This approach is thus very useful to increase power in group inference. We quantify this result by counting the number of supra-threshold vertices with and without alignment for each contrast map of the test set. Our alignment method significantly finds more such vertices of interest, as shown in Figure

---

Discussion
FUGW can derive meaningful couplings between pairs of subjects without the need of a pre-existing anatomical template. It is well-suited to computing barycenters of individuals, even for small cohorts.
In addition, we have shown clear evidence that FUGW yields gains that cannot be achieved by traditional diffeomorphic registration methods. These methods impose very strong constraints to the displacement field, that may prevent reaching optimal configurations. More deeply, this finding suggests that brain comparison ultimately requires lifting hard regularity constraints on the alignment models, and that human brains differ by more than simple continuous surface deformation. However, current results have not shown strong correlation gain of unbalanced OT compared to balanced OT, likely because the cohort under study is too small. Leveraging datasets such as HCP 43] with a larger number of subjects will help lower the standard error on correlation gain estimates. In this work, we decided to rely on predefined anatomical template (fsaverage5) to derive functional barycenters. It would be interesting to investigate whether more representative anatomical templates can be learned during the process. This would in particular help to customize templates to different populations or species. Additionally, using an entropic solver introduces new hyper-parameter that has a strong effect, but is hard to interpret. Future work may replace the scaling algorithm [TT] used here by the majorization-minn miniiiiiaiiiiiiitiionn one (9), which does not require entropic smoothing. This solution can yield sparse couplings while being orders of magnitude faster, which will prove useful when computing barycenters on large cohorts.
Finally, we plan to make use of FUGW derive alignments between human and non-human primates without anatomical priors. Indeed, the understanding of given brain mechanisms will benefit from more detailed invasive measurements made on other species only if brains can be matched across species; moreover, this raises the question of features that make the human brain unique, by identifying patterns that have no counterpart in other species. By maximizing the functional alignment between areas, but also allowing for some regions to be massively shrunk or downright absent in one species relative to the other, the present tool could shed an objective light on the important issue of whether and how the language-related areas of the human cortical sheet map onto the architecture of non-human primate brains.
Acknowledgement
We thank Maëliss Jallais and Thomas Moreau for their help in implementing Python wrapper for Multimodal Surface Matching, Thibault Séjourné and Gabriel Peyré for the useful discussions. We also thank Emma Robinson and Logan Williams for helping understand better the ropes of MSM and how to tweak it's hyper parameters.
A.T, T. Z, S. D and B.T's research has received funding from the European Union's Horizon 2020 Framework Programme for Research and Innovation under the Specific Grant Agreement No. 945539 (Human Brain Project SGA3). It has also been supported by the the KARAIB AI chair (ANR-20- CHIA-0025-01] and the NeuroMind Inria associate team.
H. T, N. C and R. F's work is funded by the projects OTTOPIA ANR-20-CHIA-D030 31A Côte d' Azur Investments ANR19.P3IA-O0T -0002 of the French National Research Agency (ANR), the 3rd Programme d' Investissements d' Avenir ANR-1 1--EUR-0006 for Future Banking" sponsored by NATIXIS. This research is produced within the framework of Energy4Climate Interdisciplinary Center (E4C) of IP Paris and Ecole des Ponts ParisTech.
and the Chair "Business Analytics
10

---

References
[1]
Alexandre Abraham et al. "Machine learning for neuroimaging with scikit-learn". In: Frontiers in Neuroinformatics 8 (2014). ISSN: 1662-5196. URL: https ://w:////www www.fffontierin... org/ article/10. /03/1//e..339////// fninf. 2014. 00014 (visited on 05/19/2022).
[2] Alexandre Abraham et al. "Machine learning for neuroimaging with scikit-learn, In: Front Neuroinform (2014), p. 14. ISSN: 1662-5196. URL: httpps://tp..i..: org/ 10. 0389/fninm
2014.00014
[3] Jesper L.R. Andersson, Stefan Skare, and John Ashburner. "How to correct susceptibility distortions in spin-echo echo-planar images: application to diffusion tensor imaging". In: Neuroimage 20.2 (2003), pp. 870-888, ISSN: 1053-8119. URL: http://doi. org/ 10011.00/.111/////
S1053- 8119 (03) 00336-7
[4] B.B. Avants et al. "Symmetric diffeomorphic image registration with cross-correlation: Evalu- ating automated labeling of elderly and neurodegenerative brain". In: Medical Image Analysis
12.1 (2008), pp. 26-41. ISSN: 1361-8415. DOr: 10 j media 2007 06 004, URL:
http://wwer http://www.scceeeeddeeeee.
com www.scciineeeccccc scwsscciceceeeeeeeeeeoeooeeccece arti www.aattttpeieeeeeeeeeeeeetttttt http://pp..111111111110000000000
[5] Thomas BAZEILLE et al. "Local Optimal Transport for Functional Brain Template Estima- tion". In: IPMI 2019- 26th International Conference Information Processing in Medical Imaging. Hong Kong, China, June 2019. DOL: 10 1007/9788 030 20351-1_11 URL:
attps://hate archives owwertes-eeeeeee... wtt.ffff/alloriie 02278663
[6] Thomas Bazeille et al. " An empirical evaluation of functional alignment using inter-subject decoding". In: Neurolmage 245 (Dec. 15, 2021). 11668. ISSN: 1053-8119, Dor: 10. 1016/
j j. neeuroimage 2021 118683 URL: https : www wwweeceeieeet..c.o..oe....roea.e com ccm/ncience//////
[7] Yashar Behzadi et al. "A component based noise correction method (CompCor) for (BOLD) and perfusion based fMRI". In: Neuroimage 37.1 (2007), pp. 90-101. ISSN: 1053-8119, URL.:
https://doi. 10. 1016/j neuroimage 2007 04. 042
[8] Shohini Bhattasali al. "Localising memory retrieval and syntactie composition: fMRI study of naturalistic language comprehension" In: Language, Cognition and Neuroscience
34.4 (2019), pp. 491-510.
[9] Laetitia Chapel al. "Unbalanced Optimal Transport through Non-negative Penalized Linear Regression". In: arXiv: 041 [cs, math, stat] (June 8, 2021). arXiv: 2106. 04145 URL:
http://arxiv. www.aaiiv..gggbbbb.. harggabs//100n 2106 04145)
[10] Po-Hsuan (Cameron) Chen et al. "A Reduced-Dimenssion fMRI Shared Response Model". In: Advances in Neural Information Processing Systems. Ed. by C. Cortes al. Vol. 28. Curran Associates, Inc., 2015. URL.: /ttttpss:/pprcceeeiiiii
neur
[330770
ppff
[11] Lenaic Chizat al. "Unbalanced Optimal Transport: Dynamic and Kantorovich Formulation"
In: ar.iv: 100.::::00080021166 [math] (Feb. 9, 2019). arXiv: 1508. 05216
URL: Aattp://arxiv org/
abs/1508. 002166
[12] Anders M. Dale, Bruce Fischl, and Martin Sereno. "Cortical Surface-Based Analysis: I. Segmentation and Surface Reconstruction" In: 9.2 (1999), pp. 179-194, ISSN: 1053-8119, DO1: 10. 1006 /nigg 1998 0395 URL: http: / /wwp//www.. www.ecedirecc......ccmcmm com sccence//
articie/p1t/S10533 0at.:95381999993950
[13] Christophe Destrieux et al. "Automatic parcellation of human cortical gyri and sulci us- ing standard anatomical nomenclature". In: Neurolmage 53.1 (Oct. 15, 2010), pp. 1-15. ISSN: 1053-8119. DOI: 10 1016 neuroimage 2010 010 URL: hhtpss
www sciencedirect. com sscincc /atreaattteee/eeeeee paclee/pppii/ /151//110011111100004444 (visited on 08/02/2022).
[14] Nicole Eichert al. "Cross-species cortical alignment identifies different types of anatomical reorganization in the primate temporal lobe". In: eLife 9 (Mar. 23, 2020). Ed. by Timothy Verstynen et al. Publisher: eLife Sciences Publications, Ltd, e53232. ISSN: 2050-084X. DOI:
10. 20.7554/eLi. eLife. 53232 URL: https:///oi. org/ 10 7554/ eLife......e .53232
[15] Bruce Fischl. "FreeSurfer". In: Neurolmage. 20 YEARS OF fMRI 62.2 (Aug. 15, 2012).
pp. 774-781. ISSN: 1053-8119
DOl: 10 1016 neuroimage 2012 01 021
URL:

---

[16] Bruce Fischl et al. "High-resolution intersubject averaging and coordinate system for the cortical surface". In: Hum Brain Mapp 8.4 (1999), pp. 272 284. URL: https://doi. org/
10. 10.1000/(0 (SICI) 1097-0193 (1999) 8: 47,32772: :ID- HBM10%/33, O. C0;
[17] K.J. Friston et al. "Characterizing Dynamie Brain Responses with fMRI: Multivariate Approach". In: Neuroimage 2.2 (1995), pp. 166-172. URL: http::ttps//// htops://dos org/ooo./.....o.oooo. 1006/
1995. 1019
[18] Matthew F Glasser et al. "A multi-modal parcellation of human cerebral cortex". en. In: Nature 536.7615 (Aug. 2016), pp. 171-178.
[19] Alexandre Gramfort, Gabriel Peyré, and Marco Cuturi. Fast Optimal Transport Averaging of URL: https: / //axivv
Neuroimaging Data. 2015. DO1: 10. 48550/ ARXIV. 1503. 08596
org/ ht/aabb/150... 08596
[20] J. Swaroop Guntupalli et al. "A Model of Representational Spaces in Human Cortex". In: Cerebral Cortex 26.6 (June I, 2016). Publisher: Oxford Academic, pp. 2919-2934, ISSN;
1047-3211 1. DOL: 10. cercor//e/r////// cwrcor/bbcoocc bhw068
URL:
oup. /
article/26/6// /111///////999///939099
[21] James V. Haxby et al. "A common, high-dimensional model of the representational space in human ventral temporal cortex". In: Neuron 72.2 (Oct. 20, 2011), pp. 404-416. ISSN:
1097-4199, DOL: 10. 1016/j neuron. 2011. 18. .2.0026
[22] Matthias Liero, Alexander Mielke, and Giuseppe Savaré. "Optimal Catropy-Transspe prob- lems and a new Hellinger-Kantorooonn
distance between positive measures". In: Inventiones mathematicae 21 1.3 (Mar. 2018), pp. 969-1199971...1 ISSN: 1432- Dot: [0. 10010000000022
017-0759-8 URL: http:////.............0000000/22
[23] Rogier B Mars et al. "Whole brain comparative anatomy using connectivity blueprints". In: elife 7 (May 11, 2018). Fd. by Klaas Enno Stephan. Publisher: el Sciences Publications, Ltd, e35237. ISSN: 2050-084X. DOT: 1o.. 700.7555/eeiiee 35237
URL: httts:///oi.. org/10
7554/eLife, 35237
[24] Facundo Memoli. On the use of
Distances for Shape Comparison. Accepted: 2914-01-29T1::52:11 1Z ISSN: 1811-7813. The Eurographics Association, 2007. ISBN: 978-
3-905673-51-7, DOL: 10. 2312//PEEG/SPPGO///////
URL: htttttt/ss//dilbbbbb.. org
443 443/xmlui/handle/100 1ttpe//e.1.2222/PP111 SPBG07 081-0900
[25] Facundo Mémoli. "Gromov- Wasserstein
GGromoy-Wasserstei Distances and the Metric Approach Object Match- ing" In: Foundations of Computational Mathematics 1.. (Aug. 1, 2011), pp. 417-487. ISSN: 1615-3383. DOL: 10. 1007 st0:01100000-- 011 9093-5
URL: https: //doi. org 100 1007
010208-011-900 9093-5
[26] Franz-Xaver Neubert et al. "Comparison of human ventral frontal cortex areas for cognitive control and language with areas in monkey frontal cortex". In: Neuron 81.3 (Feb. 5, 2014),
pp. 700-700-700 713. ISSN: 1097-4199. DOI: 10 1016/j 1016/j.neuron. neuron. 2013. 11. 012
[27] Shinji Nishimoto et al. "Reconstructing visual experiences from brain activity evoked by natural movies". In: Current biology 21.19 (2011), pp. 1641-1646.
[28] Adam Paszke et al. "PyTorch: An Imperative Style. Hligh-Peffrrmnce Deep Learning Library". In: Advances in Neural Information P Processing Systems 32. Ed. by H. Wallach et al. Curran Associates, Inc., 2019, pp.
URL: http://paperrs neurips cc hccpppppr/9011 pytorch an imperative style- high performance deep learning library pdf
[29] F. Pedregosa al. "Seikit-lear: Machine Learning in Python"". In: Journal of Machine Learning Research 12 (201 1). pp. 2825-22252 2830.
[30] Gabriel Peyré. Marco Cuturi, and Justin Solomon. Cetomov-Wassersterr Averaging of Kernel and Distance Matrices". In: Proceedings of The 33rd International Conference on Machine Learning. International Conference Machine Learning. ISSN; 1938-7228. PMLR, June 11, 2016, pp. 2664-2672 URL: https: ptttt:pe///poooeedidnne mlr press / v48/peyre11eee html
[31] Ana Luísa Pinho et al. "Individual Brain Charting, a high-resolution fMRI dataset for cognitive mapping" In: Scientific Data (June 2018), p. 180105. Do1: 10. 1038/ satts//daaa... 2018. 105 URL: sstts://hall httpa///haa...aiiieeeeeeeee
fr/hal-0181775
[32] Ana Luísa Pinho et al. "Subject-speciffe segregation of functional territories based on deep phenotyping" In: Hum Brain Mapp 42.4 (2021), pp. 841-870. URL: hol::/ttps:: /ttp::://ooooo//0000/
1002/ hbm. 25189
12

---

[33] F. Pizzagalli et al. "The reliability and heritability of cortical folds and their genetic correlations across hemispheres". In: Commun Biol 3.1 (Sept. 2020), p. 510.
[34] Hugo Richard et al. "Modeling shared responses in neuroimaging studies through multiview
ica". In: Advances in Neural Information Processing Systems 33 (2020), pp. 19142-1966..
[35] Emma C. Robinson al. "MSM: new flexible framework for Multimodal Surface Matching".
In: Neurolmage 100 (Oct. 15, 2014), pp. 414-426. ISSN: 1095-9572. DOE: 10 1016/
neuroimage 2014 2014 05 069
[36] Emma C. Robinson et al. "Multimodal surface matching with higher-order smoothness con- straints". In: Neurolmage 167 (2018), pp. 453-465. ISSN: 1095-9572. DOr: 10. 1016/
neuroimage 2ww.iaaeeee........... 2017 037
[37] Mert R. Sabuncu et al. "Function-basse intersubject alignment of human cortical anatomy". In: Cerebral Cortex (New York, N.Y.: 1991) 20.1 (Jan. 2010), pp. 130-140. ISSN: 1460-2199.
10. https//ercor/bbbpoe
[38)
Marian Schneider et al. "Columnar clusters in the human motion complex reflect consciously perceived motion axis". In: Proceedings of the National Academy of Sciences 116. (2019),
pp. 509600066000 5101. DOI: 10. 1073 /pass//pa..... 18 1814504116 eprint: https::/wwwe pnas org/doi/
pdf / 10. 1073 / ppw.//nas.... 1814504116
URL: https /www. pnas org/ doi abs 10. 1073/
pnas 1814504116
[39] Thibault Séjourné, François-Xaviee Vialard, and Gabriel Peyré. "The Unbalanced Gromov
Wasserstein Distance: Conic Formulation and Relaxation". In: arXiv:2009. 200900 [math, stat] URL: htt.:/ttt://aaaee/ org/ abs/ 2009 04266
(June 8, 2021). arXiv: 2009. 04266)
[40]
Ido Tavor et al. "Task- MRI predicts individual differences in brain activity during task performance". In: Science 352.6282 (2016), pp. 216-220.
[41] Bertrand Thirion al. "Which IMRI clustering gives good brain parcellations?" In: Frontiers in Neuroscience 8.7 (May 2014), p. 13. DOI: 10. 3089/338fnnnnn 2014.00167
URL: https
7/hal. www.iiiiiii...
[42] D. C. Van Essen et "Parcellations and hemispheric asymmetries of human cerebral cortex analyzed on surface-based atlases". In: Cereb Cortex 22. (Oet. 2012), pp. 2241-2262.
[43] D. C. Van Essen al. "The WU-Minn Human Connectome Project: overview". In: Neu- roimage 80 (Oct. 2013), pp. 62-79.
[44] Titouan Vayer et al. "Fused Gromov- Wasserstein distance for structured objects: theoretical foundations and mathematical properties". In: /tx.v:/8/1.0028.. [cs, stat] (Nov. 7, 2018). arviv: 1811. 02834 URL: http:://rxix/ orgg/bs////0 1111/1111111 1atp:/1811..0811
[45] S. AI- Al-Wasity et al. "Hyperalignment of motor cortical areas based on motor imagery during action observation". In: Sci Rep 10.1 (Mar. 2020), p. 5362.
[46] Ting Xu et al. "Cross-species functional alignment reveals evolutionary hierarchy within the connectome". In: Neurolmage 223 (Dec. 1, 2020), p. 117346. ISSN: 1053-8119. Dor:
1016/ neuroimage 2020 117346 URL: http:// www sciencedirect. com/
[47] B.T. Thomas Yeo et al. "Spherical Demons: Fast Diffeomorphic Landmark-Free Surface Registration". In: IEEE transactions on medical imaging 29.3 (Mar. 2010). pp. 650-668. ISSN:
0278-0062. DOI: 10. 1109/TMI 2009. 2030797
URL: https: : // www ncbi nlm nih
http:p/caarticlee//MccSSS
13


Reviews:
[{"review_id": "rkuCAG3jw0S", "reviewer": "Reviewer_SBLC", "summary": "The authors propose an optimal transport based registration method for meshes with multi-variate features, a principle use-case of which is neuro-imaging (cortical mesh alignment). They construct a \"Gromov-Wasserstein\" loss which is the usual transport distance term to penalize large deformations w.r.t. the mesh geodesic, which they use alongside a \"Wasserstein\" loss, which is the transport matching term. Overall the transport method is defined for an \"unbalanced\" transport problem, where the domains need not match and where restrictions may be placed on transport to/from specific regions.\n\nExperiments are presented with both synthetic and real data, using subsets of the Individual Brain Charting dataset (composed of a variety of task fMRI and the standard anatomical scans). Comparisons are made with Multisurface Matching (MSM) (Robinson 2014), as well as a template analysis (barycenters).", "questions": "My primary concern is with evaluation. While evaluating non-linear inter-subject registrations is always difficult, here we have chosen to ignore landmarks and focus on functional correlation of a hold-out set. While this is not unreasonable on its own, it is in the optimization desiderata themselves, and also in the hyper-parameter tuning objective. How can we know that these registrations are reasonable? Hold out correlation is one solution, but surely these trials are not fully independent (they do come from the same heads and, importantly, the same arterial structures). Could these comparisons be made using, e.g., myelin patterning, or its proxy by T1/T2 ratio?  \n\nAnother issue is that the comparison is made against MSM, which has default parameters set for an HCP-like dataset. Were the same search-grid to be made for MSM, would we see a similar increase? It appears to do nothing to the correlations; perhaps this is actually the case, but it seems like a very weak baseline.  \n\nThe authors specifically avoid diffeomorphic constraints. Given this, and average displacement on the order of 1-2cm (Fig 4), do we find extreme topological changes? Extreme geometric changes? Sulci to gyri?\n\nOther comments:\n* Robinson 2014 is not cited in the actual text of the paper; it may be helpful to readers during the introduction of MSM to do so.  \n* Very little time is spent on the solution method, or the mesh-reduction method. While understandably space is constrained, I feel it would be helpful to have more explanation in the main text.  ", "limitations": "None.", "rating": 7, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:  \n* This paper is well written, and enjoyable to read.  \n* The treatment of the transport problem is well done, and the implementation of a standard solution method (scaling) to this particular problem seems sound.  \n\nWeaknesses:\n* The method requires a dense interaction matrix of $V^2$ size, where $V$ is the number of vertices on the mesh, limiting this method to something on the order of 10k vertices. (Though, to the authors credit, considerations were made here.)   \n* The baseline method is untuned.  \n\nComment:\n* The technical innovation is limited to a specific application; this is not disqualifying, but it is limiting in \"significance\" to the NeurIPS community relative to, say, a conference or journal from the application domain (e.g., IEEE TMI).  ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "eIQ8V93RCne", "reviewer": "Reviewer_J9Ws", "summary": "The manuscript proposed an unsupervised image registration algorithm based on the optimal transfer scheme, solving an unbalanced fused Gromov-Wasserstein optimization problem. The algorithm was evaluated on an fMRI data registration task to test whether the correspondence between fMRI signals can be improved after the registration. ", "questions": "The proposed FUGW framework was tested on a cortical surface registration task on the activation maps and evaluated by the correlation of fMRI signals before/after the alignment. It will be great if it can be evaluated by anatomical features of the cortical surface (e.g., correspondence between the major gyri/sulci locations), which can be done by a small amount of human annotation. This is specifically important as in the paper, the fMRI-derived activation value is used as the feature to perform the registration, then the registration was evaluated by the same fMRI signal’s correlation. Thus, the improved correlation after the registration is somehow expected, as the vertex with a similar response to stimuli (reflected in the activation value) would usually have a similar fMRI signal. \n\nAs the author mentioned in line 220, the selection of α is important for balancing anatomical validity and registration effectiveness. Is there a way to estimate this hyperparameter in a more intuitive way (e.g., by enforcing voxels in the same gyri/sulci would not move to another gyri/sulci)?\n\nMore details of the FUGW framework need to be clarified to understand its mechanism and apply it for other tasks:\n1)\t How w^s and w^t in Eq. 1 are defined? The author mentioned that “We also assign the distribution w^s∈R^n_+ on the source vertices, which we interpret as their relative importance in the mesh” but more details are needed.  \n2)\tFor the Barycenter estimation, how the mesh of the Barycenter is defined/updated (e.g., number of vertices) in each iteration? As each subject has a different number of vertices, it is curious that whether the Barycenter has a fixed number of vertices or varying vertices at each iteration. \n\nIn section 4.1, the first experiment, the author mentioned “Aligning subjects on a fixed mesh”, does the “fixed mesh” here indicate the target of registration which is the template? ", "limitations": "N/A, this works only developed an image registration algorithm.", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Effective registration from one individual image to a common space, as well as from one individual to another individual, is highly needed in medical image analysis. Although various registration algorithms have been developed, the proposed fused Gromov-Wasserstein optimal transfer algorithm with unbalanced matching is a novel and plausible way to solve this task in an unsupervised approach. The capability of considering both the image itself (i.e., the mesh) and features defined on the image (e.g., the activation map used in this work) is much needed especially for the multi-modal image fusion. The methodology of this work is clearly written, and the overall quality of this manuscript is good. \n\nThe major issue of this work is that its evaluation scheme needs to be improved, see questions below.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "4GeEmhC5Sbg", "reviewer": "Reviewer_gBHr", "summary": "A common problem in the use of brain imaging data is that individual brains differ in both geometry and function, requiring some form of alignment before comparison. The authors propose a novel method for aligning different brains, FUGW, which uses a combination of distance losses to respect functional, geometric, and even fundamental region differences between different brain images", "questions": "What would the impacts of these improvements be on downstream experiments/comparisons? There’s clearly a numerical advantage, but does that translate to application or understanding?\n\nDo you have any examples of actually aligning brain images where one lacks certain region(s)? The toy sphere example sort of shows this\n", "limitations": "The technique may not scale well computationally, though that could more be an issue of the size of the images themselves. ", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths: \n\nThe problem statement and motivations are clear. The problem of accurate brain alignment is interesting both for current human neuroscience research and potential future cross-species comparisons.\nThe paper is well organized and clearly written. Toy examples are given to demonstrate the differences of their method\nThe method itself seems technically sound, with appropriate references to past work and a clear intuition of what each term does. \nThe numerical experiments are well-designed and explained, and they demonstrate a marked advantage of the proposed method in both quantitative and qualitative terms.\n\nWeaknesses:\nI would have liked to see the impact of these brain alignments on downstream experiments/tasks. \nI would also be interested in computational comparisons; how much more difficult is it to do this vs MSM? In experiment 2, it was necessary to reduce the scale of the individual brain meshes to run at all (though admittedly, the other methods couldn’t even be applied)\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept