# Inter-individual alignment of multimodal brain networks with anatomical constraints

Yanis Aeschlimann, Anna Calissano, Theodore Papadopoulo, Samuel

Deslauriers-Gauthier

► To cite this version:

Yanis Aeschlimann, Anna Calissano, Theodore Papadopoulo, Samuel Deslauriers-Gauthier. Inter-individual alignment of multimodal brain networks with anatomical constraints. Network Neuroscience, 2025, 10.1162/NETN.a.514. hal-05322137

HAL Id: hal-05322137

https://inria.hal.science/hal-05322137v1

Submitted on 20 Oct 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
1 RESEARCH

INTER-INDIVIDUAL ALIGNMENT OF MULTIMODAL BRAIN NETWORKS WITH ANATOMICAL CONSTRAINTS

Yanis Aeschlimann<fnref n="1" />, Anna Calissano<fnref n="2" />, Theodore Papadopoulos<fnref n="1" />,
and Samuel Deslauriers-Gauthier<fnref n="1" />

6

7

8

**Keywords:** [graph alignment, inter-subject variability, brain network, structural connectivity, functional connectivity, cortical atlas]

9

ABSTRACT

10 When using a cortical parcellation, it is generally assumed that the regions correspond across subjects,
11 meaning regions with the same label are expected to have the same structural or functional roles from one
12 subject to any other. However, at a fine-grained scale, environmental and genetic factors shape both
13 cortex and white matter differently, making it challenging to consistently label all parcels for every
14 subject. Brain networks can be constructed with regions as nodes and their connectivity as edges. One
15 critical step while constructing those networks is the definition of the nodes due to the spatial
16 inter-individual variability, which can limit the reliability of group studies' results. In this work, we
17 propose alignment criteria to register the brain nodes across subjects by allowing the permutation of tiny
18 cortical regions. Those criteria account for multiple brain network perspectives built from different
19 imaging modalities. Our across-subject multi-modal alignment of brain networks includes constraints
20 that restrict possible permutations to anatomically plausible ones. The identified permutations are finally
21 applied to the brain networks not used for optimization, and also improve the alignment of these

<footnote n="1">¹Université Côte d'Azur, Inria, Sophia-Antipolis, France</footnote>

<footnote n="2">²University College London, Department of Statistical Science, London, United Kingdom</footnote>
22 networks. This work has been validated on real Magnetic Resonance Imaging data from the Human Connectome Project.
23

## AUTHOR SUMMARY

24 When using a cortical parcellation, it is generally assumed that a labeled region has the same roles from one subject to any other for a structural or functional brain network's perspective. However, at a 25 fine-grained scale, inter-individual spatial variability due to environmental and genetic factors makes it 26 challenging to consistently define the network nodes across subjects. A matching of the regions across 27 subjects is proposed. It maximizes the similarity of structural and functional connectivity across subjects 28 by permuting regions. This multimodal alignment criterion includes constraints to forbid anatomically 29 unrealistic permutations. The identified permutations show robustness from one brain network modality 30 to another if the proper optimization criterion with correct spatial constraints is selected.
31

## INTRODUCTION

32 Due to the large number of neurons in the brain (Herculano-Houzel, 2009), detecting whole-brain 33 activation patterns at the neuronal scale is technically impossible with the current technologies. 34 Therefore, an approximate but reasonable solution is to evaluate the activation of cortical regions 35 assumed to have a homogeneous activity from the structural or functional point of view (Cabezas, Oliver, 36 Lladó, Freixenet, & Cuadra, 2011). It is assumed that the cortical regions correspond across subjects, 37 meaning regions with the same label are expected to have the same structural or functional roles from one 38 subject to another. At a macroscale, the brain is organized similarly across human beings (Pessoa, 2014). 39 Nonetheless, at a finer level, environmental and genetic factors shape differently both cortex and white 40 matter (Thompson et al., 2001), leading to a relative spatial inter-subject variability from a structural and 41 functional point of view (Hagmann et al., 2008; Mueller et al., 2013). As a result, it is challenging to 42 ensure a consistent labeling of all parcels to every subject (Bohland, Bokil, Allen, & Mitra, 2009; de 43 Reus & Van den Heuvel, 2013; Dickie et al., 2017; Salehi et al., 2020). Inter-subject variability of brain 44 activity is the topic of many studies (Mueller et al., 2013; Seghier & Price, 2009, 2018; Van Horn, 45 Grafton, & Miller, 2008). It is notably noticed in neurosurgery, particularly for brain tumor ablations,
where patients undergo highly invasive awake surgery with cortical electrical stimulation to precisely localize given functional areas (Tate, Herbet, Moritz-Gasser, Tate, & Duffau, 2014) because relying solely on atlas-predefined regions for tumor resections is not sufficient. The problem of producing a single brain atlas is exacerbated for parcellations with tiny regions.

Vigneau et al. (2006) have shown that activation peaks for language areas have a standard deviation reaching 20 mm, while the cortical atlas of 1000 regions proposed by Schaefer et al. (2018) has a region size down to 6.0 mm. It is essential to consider this variability for studies that compare the properties of brain regions across different subjects, as we may compare mismatched regions which not represent the same structure or function, thus limiting the reliability of the results based on group analysis.

In this work, we focus on the concept of brain networks. Mathematically, a brain network is represented as a graph (Fornito, Zalesky, & Bullmore, 2016), with cortical regions and the connectivity between them being respectively the nodes and the edges of the graph (Cao et al., 2014; Sporns, Tononi, & Kötter, 2005; Yeh, Smith, Dhollander, Calamante, & Connelly, 2019). The connectivity of a region to the rest of the cortex is a unique identifier of a region within the cortex (Passingham, Stephan, & Kötter, 2002), and thus is a sufficient feature to distinguish and compare regions across different subjects (Fig. 1A). The mismatching regions translate to inconsistencies in the definition of the graph nodes between subjects (Fig. 1) (Eickhoff, Thirion, Varoquaux, & Bzdok, 2015; Hagmann et al., 2008; Mueller et al., 2013; Sporns, 2011). Comparing brain connectivities, for instance, allows us to identify the brain regions' homologs between species (Mars et al., 2016). A list of many methods and challenges for comparing graphs in the case of brain networks is described in Mheich, Wendling, and Hassan (2020).

In a previous work, Mheich et al. (2015) measured a graph similarity across subjects and showed a significant spatial inter-subject variability in the case of EEG recordings during a picture naming task. However, they have not proposed to correct for this variability. Frigo et al. (2021) proposed an alignment algorithm to choose a suitable parcellation for many subjects. To our knowledge, only one work (Calissano, Papadopoulo, Pennec, & Deslauriers-Gauthier, 2024) has proposed to correct the inter-subject variability using the weighted structural connectivity computed from diffusion Magnetic Resonance Imaging (MRI) by swapping the network nodes (See Fig. 1A). They propose to find a matching of the regions across subjects by maximizing the **Graph Similarity**, a process well-known as **Graph Alignment** (See Conte, Foggia, Sansone, and Vento (2004) for a review).
The work of Calissano, Papadopoulos, et al. (2024) only considers structural connectivity to align brain regions across subjects, which is a static view of the paths connecting brain regions with well-known limitations (Maier-Hein et al., 2017; Schilling et al., 2022; R. E. Smith, Tournier, Calamante, & Connelly, 2013; F. Zhang et al., 2022) and does not reveal the actual brain exchanges of information. A more dynamic perspective is provided by functional connectivity (Van Den Heuvel & Pol, 2010) computed from resting-state functional MRI (fMRI) (Biswal et al., 2010) as a correlation between the Blood Oxygen Level Dependent (BOLD) signal of brain regions. Functional connectivity is probably coupled with structural connectivity, but it is still challenging to make the correspondence between them (Deslauriers-Gauthier, Zucchelli, Frigo, & Deriche, 2020; Smolders, De Baene, Rutten, van der Hofstad, & Florack, 2024), suggesting they bring different information about the brain networks.

Furthermore, in the graph alignment proposed by Calissano, Papadopoulos, et al. (2024), the permutations of spatially close regions are initially favored in the alignment optimization, but anatomically implausible permutations of regions are not strictly forbidden.

Our work proposes a framework for a multimodal brain network alignment using simultaneously different connectivity modalities that includes a regularization term forbidding the swapping of too-distant regions to ensure anatomically plausible results. Indeed, a correct matching of regions should respect the macroscale structure of the human brain (Pessoa, 2014). The permutations identified in some brain networks are applied to brain networks from the same subjects, but not used for optimization to assess the validity of the resulting permutations. In particular, we use brain networks computed from diffusion MRI (dMRI), different fMRI runs, and paradigms. Permutations identified in one type of connectivity do not necessarily align the networks of another type of connectivity, but we show that it is possible to find permutations supported by several brain networks with our multimodal alignment and that the restriction to anatomically plausible permutations increases the robustness of the results.
<figure><img src="image_2.png" /><figcaption>**Figure 1.** A. Schematic example of misalignment of two predefined regions *i* and *j* on two subjects (1 and 2). Top: the ground-truth region for the same function (red outline) corresponds to regions *i* in subject 1 and region *j* in subject 2. Bottom: we see a close matching of the connectivity profile between regions *i* of subject 1 and region *j* of subject 2. B. Schematic representation of the alignment of cortical regions between two subjects' brains (top panel). The alignment of regions corresponds to permuting rows and columns of symmetric connectomes (bottom panel).</figcaption></figure>

# MATERIALS AND METHODS

## Connectome alignment

To align two brain networks, we look for the best matching of nodes across the two subjects. The best matching corresponds to the one that maximizes the similarity between the permuted graph of one subject and the original graph of the subject chosen as a reference. In this work, a brain network (graph) is represented by a connectome, a $N \times N$ symmetric matrix, with *N* being the number of regions. We only consider symmetric matrices, as it is still complex to robustly evaluate the direction of the connectivity between the regions with the current MRI techniques. The values of the connectomes (i.e., the edges of the graphs) represent the connectivity strengths between the cortical regions. Swapping nodes corresponds to permuting the rows and the columns of the symmetric connectomes (See Fig. 1B).
Permutations can be applied on a connectome C by a permutation matrix P belonging to the permutation group $\mathcal{P} = \{P \in \{0, 1\}^{n \times n} : P^T \mathbf{1} = \mathbf{P}\}$. The permuted connectome is then $\mathbf{PCP}^T$.

The best matching of nodes across two subjects is given by a specific $\hat{P}$. To identify this $\hat{P}$, we solve the problem

$$ \hat{P} = \arg \min_{P \in \mathcal{P}} \| \mathbf{PCP}^T - C_{\text{ref}} \|_{\mathcal{F}}^2. \quad (1) $$

In our case, with symmetric connectomes, the problem simplifies to

$$ \hat{P} = \arg \min_{P \in \mathcal{P}} - \text{Tr} (\mathbf{PCP}^T C_{\text{ref}}). \quad (2) $$

This optimization yields the permutation matrix that minimizes the distance between a corrected connectome $\mathbf{PCP}^T$ and a connectome of a subject of reference $C_{\text{ref}}$. The similarity between the permuted connectome C and $C_{\text{ref}}$ is based on the Frobenius norm $\|\cdot\|_{\mathcal{F}}$. Solving this Quadratic Assignment Problem (QAP) is NP-hard (Cela, 2013). Vogelstein et al. (2015) proposed to use the Fast Approximate QAP (FAQ) algorithm to tackle this difficulty by solving a relaxed version of the optimization problem in Eq.(1): P is allowed to be in the set of Bistochastic Matrices $\mathcal{D} = \{P \in [0, 1]^{n \times n} : P^T \mathbf{1} = \mathbf{P}\}$ instead of just the set of permutation matrices $\mathcal{P}$. This relaxed version enables optimization by the Frank-Wolfe algorithm, also known as Conditional Gradient Descent (Frank & Wolfe, 1956). The gradient descent should be initialized by a bistochastic matrix. At the end of the descent, the final bistochastic matrix is projected onto the set of permutation matrices (Crouse, 2016), yielding an approximate solution to Eq.(1). This alignment strategy implies choosing a given connectome modality. Calissano, Papadopoulos, et al. (2024) used this algorithm on structural connectomes to align brain networks.

### Combined alignment of structural and functional networks

To enrich the alignment of brain regions, we propose combining the information of brain networks of different modalities or types. We modified the FAQ algorithm such that it finds the permutation matrix P that minimizes the optimization criterion $L_1(P)$:

$$ L_1(P) = \sum_{i=1}^{I} \beta_i \|PC_i P^T - C_{i,\text{ref}}\|_{\mathcal{F}}^2, \quad \text{subject to} \quad \sum_{i=1}^{I} \beta_i = 1, \quad (3) $$
which is, in the case of symmetric connectomes, equivalent to minimizing $L'_1(P)$:

$$L'_1(P) = \sum_{i=1}^{I} -\beta_i \operatorname{Tr}(PC_i P^T C_{i,\operatorname{ref}}), \quad \text{subject to} \quad \sum_{i=1}^{I} \beta_i = 1, \qquad (4)$$

with $C_i$ and $C_{i,\operatorname{ref}}$ being the connectomes of a modality $i$ for a subject and a subject of reference respectively. $I$ is the number of connectome types taken into account. Because the different types of connectomes are built from the same atlas and considering that for a given subject, each region has the same role for all connectivity modalities, notably from a structural and functional point of view, the same Permutation Matrix $P$ is applied to all the connectomes of the subject to be aligned. The hyperparameters $\beta_i \in \mathbb{R}_{\ge 0}$ allow weighting the different modalities. Indeed, if one modality is less noisy or more insightful than the others, we may favor it.

## Constraint to specific regions

The second contribution to the brain region alignment is the addition of a regularization term to include spatial information in $L_1(P)$, leading to the new criterion

$$L_2(P) = L_1(P) + \lambda \operatorname{Tr}(P^T R), \quad \text{subject to} \quad \sum_{i=1}^{I} \beta_i = 1 \qquad (5)$$

$$= L_1(P) + \lambda \sum_{k,l=1}^{N} p_{k,l} \cdot r_{k,l}, \qquad (6)$$

where $\lambda$ is a scalar hyperparameter. $R$ is a $N \times N$ regularization matrix with binary entries 0 and 1, which penalizes specific region permutations. Indeed, if $p_{k,l} = 1$ (i.e. permutation of region $l$ to region $k$) and $r_{k,l} = 1$, then $L_2$ increases by $\lambda$.

Dropping terms independent of $P$ in $L_2(P)$, the problem simplifies for symmetric connectomes to minimizing $L'_2(P)$ defined as

$$L'_2(P) = -\sum_{i=1}^{I} \beta_i \operatorname{Tr}(PC_i P^T C_{i,\operatorname{ref}}) + \lambda \operatorname{Tr}(P^T R). \qquad (7)$$

In the FAQ algorithm, the gradient of the new $L'_2(P)$ optimization criterion is updated as:

$$\nabla L'_2(P) = -2 \sum_{i=1}^{I} \beta_i C_{i,\operatorname{ref}} PC_i + \lambda R, \qquad (8)$$
<figure><img src="image_3.png" /></figure>

This gradient is not a bistochastic matrix, so performing the usual gradient descent by subtracting the gradient from the initialization matrix will make the resulting matrix *P* out of the bistochastic matrix space. To stay in the bistochastic matrix space, the Frank-Wolfe algorithm first computes the search direction. That is given by

$$Q_m = \arg \min_{P \in \mathcal{D}} \langle \nabla L'_2(P_m), P \rangle, \quad (9)$$

at the iteration *m* of the gradient descent. *Q*<sub>*m*</sub> is the bistochastic matrix that minimizes the scalar product with the value of the gradient of *L*'<sub>2</sub>(*P*<sub>*m*</sub>). Birkoff's theorem (Chen, Qi, Caccetta, & Zhou, 2019) tells us that the minimum of this linear form in *D* is an extremum of *D*, thus *Q*<sub>*m*</sub> is a permutation matrix in *P*. With our updated gradient, the search direction *Q*<sub>*m*</sub> is:

$$ 
\begin{aligned}
Q_m &= \arg \min_{P \in \mathcal{D}} \langle \nabla L'_2(P_m), P \rangle \\
&= \arg \min_{P \in \mathcal{D}} \mathrm{Tr} (\nabla L'_2(P_m)^T P) \\
&= \arg \min_{P \in \mathcal{D}} \mathrm{Tr} \left( \left( -2 \sum_{i=1}^I \beta_i C_{i, \mathrm{ref}} P_m C_i + \lambda R \right)^T P \right) \\
&= \arg \min_{P \in \mathcal{D}} \mathrm{Tr} \left( (-A + \lambda R)^T P \right), \quad \text{with } A = 2 \sum_{i=1}^I (\beta_i C_{i, \mathrm{ref}} P_m C_i) \\
&= \arg \min_{P \in \mathcal{D}} \sum_{k,l=1}^N (-a_{k,l} + \lambda r_{k,l}) \cdot p_{k,l}.
\end{aligned} 
\quad (10) $$

The sum of Eq. (10) takes values only for the *N* entries equal to 1 in *P*, so when $p_{k,l} = 1$ ($N$ times).

When $p_{k,l} = 1$, if $r_{k,l} = 1$, then the sum increases by $\lambda$, but if $r_{k,l} = 0$ the term in the sum only depends on the connectomes' values $a_{k,l}$ and not on $\lambda$. If the matrix $P$ has its entries equal to 1 only where the matrix $R$ has entries equal to 0, then no penalty ($\lambda$ value) is added to the sum. For $\lambda \to \infty$, if any entry $r_{k,l} = 1$ and $p_{k,l} = 1$, the regularization term takes an infinite value, and the sum cannot be minimized. It therefore forces $p_{k,l} = 0$ for any pair of indices $k, l$ such that $r_{k,l} = 1$. The regularization term $\lambda \mathrm{Tr}(P^T R)$, with $\lambda \to \infty$ acts as a hard constraint, as $Q_m$ is a permutation matrix (Chen et al., 2019) with entries equal to ones only where the entries of $R$ are zeros, thus forbidding given permutations of regions. As long as it respects the constraints, the search direction $Q_m$ is data-driven by the values of $a_{k,l}$ in Eq. (10). In other words, $Q_m$ takes values that minimize the scalar product with the gradient of the optimization data criterion $L_1(P_m)$, while still respecting the constraints imposed by $R$. Also, no arbitrary scalar value has to be chosen for $\lambda$. Finding the matrix $Q_m$ is a linear sum assignment problem.
It is computed using a version of the Hungarian algorithm by the Scipy python package (Virtanen et al., 2020), the modified Jonker-Volgenant algorithm (Crouse, 2016). Numerically, this algorithm can handle infinite values to solve linear sum assignment problems.

The only condition in $R$ for the algorithm to yield a proper permutation matrix is that there is at least a combination of entries equal to 0 in $R$ that forms a permutation matrix. In our work, we furthermore use a symmetric regularization matrix $R$. If we prohibit one region from permuting to another, the latter is also forbidden from permuting to the former.

Once the search direction $Q_m$ is computed, the step size $\gamma_m$ is computed as

$$ \gamma_m = \arg \min_{\gamma \in [0,1]} L_2 (\gamma P_m + (1-\gamma)Q_m). \qquad (11) $$

Since the regularization term in $L_2$ is forced to be 0 (otherwise it takes an infinite value), it is the same to use $L_1$ or $L_2$ in Eq. (11). The bistochastic matrix is then updated at every iteration $m$ using the step size $\gamma_m$

$$ P_{m+1} = \gamma_m P_m + (1 - \gamma_m) Q_m. \qquad (12) $$

The bistochastic matrix at iteration $m$ takes values between $P_m$ and $Q_m$, which both respect the constraints imposed by the matrix $R$. As a consequence, at every iteration, the resulting matrix is a bistochastic matrix that respects the constraints imposed by $R$. At convergence, the last bistochastic matrix is projected onto the permutation group using the modified Jonker-Volgenant algorithm (Crouse, 2016).

In this work, we chose to use structural connectomes (SC) and functional connectomes (FC) simultaneously, corresponding to $I = 2$. The function $L_1$, simplifies to

$$ L_1^*(P) = \alpha \|PSP^T - S_{\text{ref}}\|_{\mathcal{F}}^2 + (1 - \alpha) \|PFP^T - F_{\text{ref}}\|_{\mathcal{F}}^2, \qquad (13) $$

while, $L_2$ simplifies to

$$ L_2^*(P) = L_1^*(P) + \lambda \operatorname{Tr}(P^T R) \qquad (14) $$

with $S$ and $F$ being the structural and functional connectomes of the subject to be aligned, and $S_{\text{ref}}$ and $F_{\text{ref}}$ the connectomes of the subject to align with. The values of the hyperparameter $\alpha$ can vary between 0 and 1.
165 **Permutation matrix assessment**

To evaluate how a permutation matrix *P* align a pair of subject (*a*, *b*), the following error criterion is computed:

$$
\eta_i(P, a, b) = \|C_i^a - C_i^b\|_{\mathcal{F}} - \|PC_i^a P^T - C_i^b\|_{\mathcal{F}} \quad (15)
$$

166 which is the difference between the connectomes' distances $C_i^a$ and $C_i^b$ of subjects *a* and *b* for modality *i*
167 before and after the permutations of the rows and columns using *P*. If the permutation improves the
168 similarity between the brain networks of modality *i* of subject *a* and *b*, then $\eta_i(P, a, b) > 0$. A
169 permutation matrix resulting from the optimization of $L_1(P)$ or $L_2(P)$ does not necessarily yield $\eta > 0$
170 for both connectome types used in the optimization because the optimization is only done on a
171 combination of the distances of the two brain network types to their respective reference connectomes.
172 Thus, $\eta$ is computed independently on both connectome types used in the optimization. The error
173 criterion is also computed on connectome types not used to optimize the values of $P$, to assess the
174 generalization of the resulting permutation matrix. We say that a permutation *generalizes* to another
175 connectome type when optimized on a first type, it also increases the connectome similarity of
176 connectomes built from the other type.

177 **MRI acquisition and preprocessing, HCP data**

178 For this study, we selected the first six hundred subjects from the HCP S1200 release who underwent
179 diffusion MRI (dMRI), resting-state fMRI (rs-fMRI), and task-fMRI (t-fMRI) for the seven paradigms:
180 motor, language, social, gambling, emotion, relational, and working memory. Data were acquired using a
181 3T Siemens Skyra scanner (Siemens AG, Erlanger, Germany). For further information, the imaging
182 protocols have been described in detail in Van Essen et al. (2012). The fine-grained atlas of 1000 regions
183 (17 networks) proposed by Schaefer et al. (2018) is used to produce the symmetric connectomes. The
184 first 500 rows and columns of the connectomes correspond to the labels of the regions of the left
185 hemisphere and the last 500 to the right hemisphere.

186 Functional The resting-state data were acquired using multiband gradient-echo echo-planar imaging
187 (EPI). Each session consisted of 2 runs (with left-to-right and right-to-left phase encoding) of 14 min and
188 33 s each (TR = 720 ms, TE = 33.1 ms, voxel size = 2 mm isotropic, number of volumes = 1200). The
189

190

191

task-state data were acquired using the same protocol, but the duration of acquisition per run is lower and depends on the task. Details on the protocols of task-fMRI can be found in Barch et al. (2013). Both rs-fMRI and t-fMRI are already registered in the MNI space.

192

193

194

195

196

197

198

199

Rs-fMRI data originate from the “FIX-Denoised (Extended)” dataset of HCP (S. M. Smith et al., 2013) in 4D volumetric format. This preprocessed dataset has been cleaned of structured noise through a process that pairs independent component analysis (MELODIC) with the FSL tool FIX, which automatically removes artifactual or “bad” components, along with rigid-body head motion correction and high-pass temporal filtering to remove slow drifts. For further details on this method see Griffanti et al. (2014); Salimi-Khorshidi et al. (2014). Task-functional images were preprocessed through the HCP Minimal Preprocessing (HCP-MP) pipeline (Glasser et al., 2013), including EPI distortion correction, FLIRT-based motion correction, TOPUP-based field map preprocessing, and intensity normalization.

200

201

202

203

204

205

206

207

208

209

210

211

A Butterworth bandpass filter of order 8 with cut-off frequencies of 0.06 and 0.125 Hz is used to filter out irrelevant parts of the BOLD signal (Romascano et al., 2015). The processed BOLD signals are averaged over the regions of the atlas. A volumetric version of the Schaefer parcellation in MNI space is used. Regarding the rs-fMRI, the time series of left-right and right-left phase encoding are concatenated along the time axis, leading to time series of 29 min and 6 s. The symmetric correlation matrix is computed on processed BOLD signals using Pearson's coefficient. This pipeline is processed on two runs of resting-state separately, leading to two resting-state-FC (rs-FC) per subject, which in the following will be called rs-FC₁ and rs-FC₂ respectively. The same pipeline is applied to t-fMRI. A single task Functional Connectome (t-FC) (Amico, Arenas, & Goñi, 2019; Cole, Ito, Cocuzza, & Sanchez-Romero, 2021; X. Zhang et al., 2013) is constructed from the seven t-fMRI time series concatenated along the time axis, leading to time-series of 46 min and 56 s. This connectome represents a combination of signal BOLD correlation of the seven tasks cited previously.

212

213

214

215

216

*Diffusion* The diffusion MRI was acquired using a multiband spin-echo EPI sequence with right-to-left and left-to-right phase encoding polarities (TR = 5520 ms, TE = 89.5 ms, voxel size = 1.25 mm isotropic). 18 volumes are acquired at b-value 0 s/mm², 90 volumes at b-values 1000 s/mm², 90 volumes at b-values 2000 s/mm², and 90 volumes at b-values 3000 s/mm². Each volume is composed of 145 × 174 × 145 voxels.
217

218

219

Diffusion images were preprocessed through the HCP-MP pipeline (Glasser et al., 2013), including a zero gradient intensity normalization, EPI distortion correction using reversed-phase encoding directions, eddy-current correction, and nonlinear registration to MNI space.

220

221

222

223

224

225

226

227

228

229

230

231

232

233

234

235

236

We used the TractoFlow pipeline (Avants, Tustison, & Song, 2009; Di Tommaso et al., 2017; Garyfallidis et al., 2014; Jenkinson, Beckmann, Behrens, Woolrich, & Smith, 2012; Kurtzer, Sochat, & Bauer, 2017; Theaud et al., 2020; Tournier et al., 2019) to reconstruct **Tractograms** from preprocessed dMRI. Spherical harmonics of order 10 maximum are fitted to the preprocessed dMRI data. Then, 120 seeds per voxel at the intersection of the gray matter and white matter are positioned as starting points of streamlines. The streamlines are reconstructed using a step size for Particle Filter Tracking (PFT) of 0.3125 mm. Only valid streamlines with a length ranging from 20 to 300 mm are kept. The other parameters of the TractoFlow script are set to their default values. A structural connectome $S'$ is constructed by counting the number of streamlines included in the tractogram that connects the two regions of the given matrix entry (Yeh et al., 2019). A local search is performed to find the closest cortical region to the endpoints of the streamline, with an upper bound distance of 5.0 mm. Each SC is symmetrized by adding its transpose and dividing by 2. As the SC and FC have different units and different value ranges, we need to rescale them. To make the values of $\alpha$ meaningful for quantifying the balancing between the two modalities, we decided to rescale the SC such that it has the same Frobenius norm as the FC of the same subject, i.e.: $S = \frac{\|F\|_F}{\|S'\|_F}S'$, with $S'$ and $F$ the original symmetric SC and FC, and $S$ the normalized SC. This normalization only modifies the values of SC, and thus, the FC matrices remain correlation matrices.

237

## Alignment Pipeline

238

239

240

241

242

243

244

245

An overview of the alignment pipeline is illustrated in Fig. 2A. The method proposed by Calissano, Papadopoulos, et al. (2024), performs the alignment on hemispheric connectomes because the proposed alignment, if performed on whole-brain connectomes, could unrealistically permute regions across the hemispheres. The alignment is carried out independently on the left and right hemispheric connectome. The whole-brain connectome is then permuted with the reconstructed $N \times N$ permutation matrix, with on its two diagonal blocks, the left and right hemispheric $N/2 \times N/2$ permutation matrices. As a consequence, the optimization misses the across-hemisphere connectivity information. Mathematically, only the two square diagonal blocks of the whole-brain connectomes are considered (See Fig. 2B).
To improve the alignment from Calissano, Papadopoulos, et al. (2024), we propose to align whole-brain connectomes while forbidding across-hemisphere permutations, so that the **Inter-hemispheric Connectivity** still informs the alignment. To forbid permutations of regions across the hemispheres, the optimization criterion $L_2^*$ of Eq.(14) with the regularization term is required. Because the connectome matrices are built such that the first $N/2$ and last $N/2$ rows and columns correspond to the regions of the left and right hemispheres, respectively, the regularization matrix $R$ is a $2 \times 2$ block matrix with equal-size blocks and values equal 0 and 1 at the diagonal and anti-diagonal blocks, respectively. We refer to this matrix as $R_{\text{hemi}}$:

$$R_{\text{hemi}} = \left( \begin{array}{c|c} \mathbf{0} & \mathbf{1} \\ \hline \mathbf{1} & \mathbf{0} \end{array} \right) \qquad (16)$$

where $\mathbf{0}$ and $\mathbf{1}$ are $N/2 \times N/2$ matrices of entries 0 and 1 respectively.

The alignment criterion with the previous constraints restricts permutations to regions within the same hemisphere. Still, it does not consider the spatial distance between the regions to perform the permutations. Regions with similar connectivity could be permuted even though they are distant, leading to anatomically implausible permutations. We propose another regularization matrix to restrict the permutations to spatially close regions. In this case, the regularization matrix refers to $R_{\text{neigh}}$ defined as:

$$R_{\text{neigh}} = \left( \begin{array}{c|c} \theta & \mathbf{1} \\ \hline \mathbf{1} & \theta \end{array} \right) \qquad (17)$$

where $\theta$ is an $N/2 \times N/2$ binary matrix with entries equal to 0 when the region labels corresponding to the row and column are spatially adjacent on the parcellation, and otherwise to 1. This regularization matrix aims to restrict the permutations to spatially adjacent regions. The alignment optimization using $R_{\text{neigh}}$ is a sub-problem of the alignment using $R_{\text{hemi}}$, because the permutations allowed by $R_{\text{neigh}}$ are a subset of those allowed by $R_{\text{hemi}}$.

Three strategies are explored:

* Hemispheric Alignment (HA), using $L_1^*$ criterion from Eq.(13). Two permutation matrices are independently identified for left and right hemispheric connectomes. It is the method of Calissano, Papadopoulos, et al. (2024).

* Whole-brain Alignment (WBA), using $L_2^*$ criterion from Eq.(14) with $R = R_{\text{hemi}}$ defined in Eq.(16).
* Whole-brain Alignment with Neighborhood Constraints (WBANC), using $L_2^*$ criterion from Eq.(14) with $R = R_{neigh}$ defined in Eq.(17).

Those strategies of alignment are summed up in (Fig.2B). The HA and the WBA will be compared to see the influence of the inter-hemispheric connectivities on the alignment results. It is interesting to check if it is still possible to find permutation matrices that are not the identity matrix when we restrict the permutations to a subset of all possible permutations, even though they are the most anatomically plausible.

The alignment requires a pair of connectomes for both the subject to be aligned and the subject to align with, leading to one permutation matrix identification. The value of $\alpha$ adjusts the identified permutation matrix as it balances the influence of the two types of connectomes in the alignment process. Therefore, for each value of $\alpha$, one permutation matrix is identified for the alignment of two subjects (Fig.2A2). For the particular cases $\alpha = 0$ and 1, the identified permutations do not ensure alignment of both brain network modalities, as only one modality is considered, either FC or SC. We compute the error criterion $\eta$ seen in Eq.(15) for each permutation matrix per $\alpha$ and both connectome modalities. For the specific cases, $\alpha = 0$ and 1, this would inform us whether the identified permutations on one type of connectomes also align the other type. The permutation matrices are identified using a training dataset consisting of pairs of SC and FC. The error criterion $\eta$ is computed on this dataset. Additionally, the permutation matrices identified for a given subject pair and a value of $\alpha$ are also applied to external connectomes from the same subjects. The error criterion is also computed on this external dataset to assess whether the alignment improves similarity on unseen data. (Fig.2A3).

In this work, we use two training datasets and a different validation dataset for each of those (Fig.2C). Permutation matrices are

* first, identified on the pair SC and rs-FC₁ (training dataset 1) and assessed on rs-FC₂ and t-FC,

* secondly, identified on the pair SC and t-FC (training dataset 2) and assessed on rs-FC₁ and rs-FC₂.

In practice, as with standard registration methods where a template brain or a specific brain subject is selected as the reference to which all others are registered, one can similarly choose a random subject as the reference to align with, using the connectivity of the regions of the chosen subject as the region connectivity template. In our study, each subject is iteratively selected as a reference to all other
connectomes. The gradient descent is initialized with the identity matrix, corresponding to performing no
permutations. The optimization is stopped after convergence (in our case 50 iterations). This number has
been determined experimentally, as the criterion function has almost always reached a minimum after
about 15 iterations. The matrices *P* are identified for $\alpha$ from 0 to 1 with a step size of 0.1 and $\lambda = \infty$.
The 600 subjects are aligned to each other, for 11 values of $\alpha$, the 3 strategies, and the two training
datasets, leading to 23, 720, 400 alignment of connectomes.

A Wilcoxon signed-rank test is used to assess the significance of the values of η. A statistically
significant result indicates that the differences of the connectome distances before and after the
permutations have stochastically greater values than a distribution symmetric about 0, which is equivalent
to testing whether the distances after alignment have values stochastically lower than the ones of the
distances before the alignment. We use the Bonferroni correction to compensate for the 264 hypotheses
tested, corresponding to the 11 values of α, the 3 strategies, the 4 connectome types, and the 2 training
datasets. A result is then considered significant if $p < \frac{0.001}{264} = 3.78 \times 10^{-6}$.

## RESULTS

### Validation of the algorithm on simulated data

To validate the alignment algorithm, we simulated 15 random permutation matrices. Those permutation
matrices are applied to real connectomes of 10 randomly selected subjects. The connectomes to align are
the permuted connectomes, and the respective references are the original unpermuted connectomes of the
same subjects. Since the FAQ algorithm is not optimal (Vogelstein et al., 2015), we cannot guarantee the
exact retrieval of the simulated permutation matrices (more precisely, we would retrieve the inverses of
the simulated permutation matrices). We ran these alignments with simulated data for the HA, WBA, and
WBANC algorithms and $\alpha = 0.5$. The simulated permutation matrices were constructed to satisfy the
constraints imposed by the WBA and WBANC algorithms, respectively. Using the HA strategy, in the
retrieved permutations, on average, only 3.56 labels are not matched to their original labels over the 1000
labels of the parcellation, and 58% of the resulting permutation matrices are exactly the original ones.
Using the WBA and WBANC algorithms, on average, 3.32 and 1.21 labels are not matched to their
original labels. 54% and 61% of the resulting permutation matrices are exactly the original ones for
<figure><img src="image_4.png" /></figure>

## B Comparison of 3 alignment strategies

<figure><img src="image_5.png" /></figure>

* Alignment on hemispheres independently

* Without spatial constraint

<figure><img src="image_6.png" /></figure>

* Alignment on whole-brain

* Within same hemispheres constraint

<figure><img src="image_7.png" /></figure>

* Alignment on whole-brain

* Neighbor regions constraint

## C Comparison of 2 pairs of "training data"

<figure><img src="image_8.png" /></figure>

* Optimization: SC and rs-FC₁

* Validation: rs-FC₂ and t-FC

<figure><img src="image_9.png" /></figure>

* Optimization: SC and t-FC

* Validation: rs-FC₁ and rs-FC₂

**Figure 2.** Flow chart of connectome construction from dMRI and fMRI (A1), alignment of pair of connectomes between subjects (A2), and validation of identified permutations on extra connectomes for the same group of subjects (A3). (B) Summary of the 3 alignment methods. (C) Summary of the connectome modalities used for the optimization (training dataset) and those used for assessment.

WBA and WBANC. The non-optimality characteristic of the FAQ algorithm is detailed in Vogelstein et al. (2015).
316 **Alignment of real connectomes**

317 As explained in the subsection *Alignment Pipeline*, the error criterion $\eta$ is computed on the connectomes
318 of the training datasets using the *P* identified for each discretized value of $\alpha$ between 0 and 1. The
319 criterion $\eta$ is also computed on connectomes not used by the optimizations to assess the validity of the
320 identified permutations. Results are shown in Figures 3 and 4 for each of the two training datasets,
321 respectively, and the three strategies HA (first row), WBA (second row), and WBANC (third row). Only
322 results for three values $\alpha$ are shown: $\alpha$ = 0, corresponding to an alignment on the FC only (either rs-FC₁
323 or t-FC); $\alpha$ = 1, corresponding to an alignment on SC only; and the value between 0 and 1 that gives the
324 most significant results on the data, depending on the training dataset. The results for every value of $\alpha$ are
325 shown in *Supplementary Material*. A distribution of $\eta$ for a particular connectome type that is shifted to
326 the positive values means that the similarities between the connectomes have globally increased when
327 applying the permutations identified by a given optimization strategy (HA, WBA, and WBANC) using a
328 given training dataset and for a given value of $\alpha$. Statistically significant increases in similarity are
329 indicated with a star above the distributions in Figures 3 and 4, while significant decreases are marked
330 with a negative star.

331 **Alignment using resting-state connectomes**

332 The permutation matrices are first identified on the primary training dataset consisting of SC and
333 rs-FC₁ (See Fig. 2C). The error criterion $\eta$ is computed on the training dataset, thus on SC, and rs-FC₁.
334 The criterion $\eta$ is also computed on rs-FC₂ and t-FC to assess the validity of the identified permutations.
335 The distributions of the criterion $\eta$ are plotted in Fig. 3 for $\alpha$ = 0 (first column, alignment on rs-FC₁
336 only), 0.3 (second column), and 1 (third column, alignment on SC only). For the specific cases: $\alpha$ = 0
337 and 1, the alignment criteria use only one type of connectome modality, for this training data, either
338 rs-FC₁ ($\alpha$ = 0) or SC ($\alpha$ = 1). The results for the 11 values of $\alpha$ are shown in *Supplementary Material*.

342 The $\eta$ distributions for connectomes used in the optimizations (training dataset) are shown in dark
343 colors (red for SC and dark green for rs-FC₁), and in light colors for connectomes not used in the
344 optimization criteria (light green for rs-FC₂ and light blue for t-FC). For the single-modality alignment
345 ($\alpha$ = 0 and 1), the permutations significantly align the connectome types on which they are optimized. In
346 particular, for $\alpha$ = 0 the rs-FC₁ are significantly aligned and for $\alpha$ = 1 the SC are significantly aligned,
<figure><img src="image_10.png" /><figcaption>**Figure 3.** Distributions of the difference between the connectomes' distances respectively after and before the alignment on structural and resting-state functional connectomes of the first run, for the three strategies: HA (top row), WBA (middle row), and WBANC (bottom row) and $\alpha = 0$ (left column), 0.3 (middle column), and 1 (right column).</figcaption></figure>

regardless of alignment strategies. Nonetheless, the alignment on SC only ($\alpha = 1$) does not align $rs-FC_1$ and conversely. Hence the need for a multimodal alignment with an $\alpha$ value between 0 and 1 excluded. For $\alpha = 0.3$, so for permutations identified on both SC and $rs-FC_1$, the distributions of $\eta$ for SC and $rs-FC_1$ are shifted to the positive direction, meaning significant alignment of SC and $rs-FC_1$ simultaneously for the three alignment strategies. In addition, for $\alpha = 0$ and 0.3, the distribution of the
352 rs-FC₂ (light green) is also shifted to the positive direction for the three strategies, meaning a
353 generalization of the identified permutations to rs-FC₂. We also notice that when the results are
354 significant, notably for rs-FC₁ at α = 0 (dark green in the left column) and for SC at α = 1 (red in the
355 right column), the WBA strategy (second row) shifts the distributions more towards positive directions
356 than the HA strategy (first row). This means that the interhemispheric information helps the alignment
357 process as expected. For the WBANC strategy, the permutations have a lower impact on the shifts of the
358 distributions, but the results are still significant, meaning that even with the constraints to spatially
359 adjacent regions, we can find permutations that decrease the distances between the connectomes, for SC
360 and rs-FC₁ simultaneously and that also improve the matching for rs-FC₂.

361 Alignment using task connectomes

362 Secondly, permutation matrices are identified on the second training dataset, so on SC and task t-FC. The
363 error criterion η is thus computed on this training dataset (SC and t-FC). The same criterion η is
364 computed on rs-FC of both runs to assess the validity of the identified permutations. The distributions of
365 the criterion η are plotted in Fig. 4 for the three strategies and α = 0 (left column, alignment on SC only),
366 0.2 (middle column), and 1 (right column, alignment on t-FC only). The results for the 11 values of α are
367 shown in *Supplementary Material*.

371 The η distributions for connectomes used in the optimizations are shown in red for SC and blue for
t-FC, and in light green for connectomes not used for the optimizations, in this case, rs-FC₁ and rs-FC₂.
372 For α = 0 and 1, the permutations significantly align the t-FC and SC, respectively, but do not generalize
373 to the other connectome modalities of the training data (SC or t-FC). In other words, permutations
374 identified on SC, when applied to t-FC do not decrease the distances between the connectomes, and the
converse. However, for α = 0.2, using the identified permutations on both SC and the t-FC, the
375 distributions of η for both SC and t-FC are shifted to the positive directions. We notice that for α = 0, so
376 for an alignment considering only t-FC, the distributions of the rs-FC of the two runs (light greens) are
also shifted to the positive directions, meaning a generalization of the alignment to resting-state
377 connectomes for the three strategies. As for the results on the previous training dataset, when results are
378 significant, the distances after permutations are always lower with the WBA strategies than with the HA
379 strategy, meaning interhemispheric connectivities still inform the alignment. For α = 0.2, the three
380 strategies allow alignment of simultaneously SC and t-FC. The more interesting result is that for any α
381
382
383
<figure><img src="image_11.png" /><figcaption>Figure 4. Distributions of the difference between the connectomes' distances respectively after and before the alignment on structural and task functional connectomes, for the three strategies: HA (top row), WBA (middle row), and WBANC (bottom row) and $\alpha = 0$ (left column), 0.2 (middle column), and 1 (right column).</figcaption></figure>

368 values, only the WBANC strategy enables alignment of the four connectome modalities at the same time. Here, in particular, for $\alpha = 0.2$ (third row and second column), the four distributions are shifted to the positive values. 368

387 Constraints' effects on distributions of permutations
The regularization term directly influences the distance between the permuted regions in the WBA and WBANC strategies. We compare the identified permutations of the two approaches in terms of spatial distances between the permuted regions. The spatial distance between two permuted regions is calculated as the minimum path length between the two regions on the graph of spatially adjacent regions. In particular, self-permutations have a distance of 0, and permutations of spatially adjacent regions have a distance of 1. In Fig. 5A we can observe the averaged distance with the permuted region for every region on a brain map, for the WBA (left) and the WBANC (right). HA and WBA have similar spatial distances. For that reason, we only compare the values between WBA and WBANC. In the WBANC, permutations of remote regions are forbidden, while in the WBA, permutations of regions are only limited to within hemispheres. In Fig. 5B, we show a pie chart of the distributions of the entire set of permutations. Permutations of regions are classified into three categories: self-permutation (region not permuted), local permutation (region permuted with a neighbor's region), and remote permutation (region permuted with a region within the same hemisphere that is not a neighbor). The pie chart indicates that there are significant amounts of permutations performed, notably remote ones when no spatial constraints are used (Fig. 5B, left). The use of the constraint to spatially adjacent regions has indeed restricted the permutations to local ones (Fig. 5B, right), but there are still an important number of permutations with the constraints (48.17%).

<figure><img src="image_12.png" /><figcaption>**Figure 5.** Distributions of the permutations for the optimization with constraints on intra-hemispheric permutations and neighbor's permutations.

A shows the averaged distance with the permuted regions for each region on brain maps for the alignment with constraints on intra-hemispheric regions (left) and constraints on neighbor regions (right). The distance is computed as the minimum path length between two regions on the graph of spatially adjacent regions. B shows the distributions of the permutations into 3 categories. The labels 'self', 'local', and 'remote' stand for self-permutations (no permutations), local permutations (permutations between neighbors' regions), and remote permutations (permutations between non-neighbor regions), respectively. The left pie chart corresponds to the optimizations with constraints on intra-hemispheric regions, and the right one on neighbors' regions. The values shown are averaged values over all α values and for the training dataset made of SC and rs-FC₁.</figcaption></figure>
DISCUSSION

The results on the alignment of different types of brain connectomes highlight the complexity of aligning brain regions. Viewing the brain networks from different perspectives changes the alignment results, and aligning the brain regions with only one modality network does not allow for alignment on other brain network modalities (structural and functional). However, using a multimodal alignment optimization permits identifying permutations supported by both structural and functional connectivities, with functional connectivity being from either task or resting-state. Furthermore, the permutations found for the alignment on the rs-FC₁, increase the similarities between the rs-FC₂. This suggests that the permutations of regions are not purely noise alignment because they are reliable on the rs-FC₂.

It seems that the generalization of the permutations identified on task connectomes to the resting-state ones is easier than the converse. Cole, Bassett, Power, Braver, and Petersen (2014) have shown that task functional networks are shaped primarily by an intrinsic network architecture that is also present in resting-state and secondarily by a limited set of task-general and task-specific evoked connectivity changes. The presence of the resting-state intrinsic network in task connectomes can explain the generalization to resting-state connectomes, while the permutations identified on resting-state miss the task-related networks and thus do not generalize easily to the task.

Overall, alignments do not generalize well across modalities (SC and FC). Alignment using the same modality (in our case different FC) generalizes better between them, and this is further improved when aligning between data acquired with the same protocol (in our case rs-FC).

Despite the quantitative noise removal of MRI data, the SC and FC probably still contain non-negligible noise, which could lead to overfitting of some permutations. For example, the permutations resulting from the alignment on rs-FC of run-1 (with alpha = 0) increase the similarity of the connectome when applied to the rs-FC of run-2, but the increase is less pronounced, meaning some subsets of permutations have possibly been overfitted. The lack of generalization between SC and FC is more complex. While spurious permutations resulting from the alignment on a single modality presumably exist, we believe that SC and FC reveal very different information about the brain networks.
In the article of Deslauriers-Gauthier et al. (2020), they show that SC and FC have a correlation of only 0.25% on average, which argues for the need for a multimodal alignment.

To reinforce the relevance of the permutations, we tested the multimodal alignment using either rs-FC₁ or rs-FC₂. This allows us to observe the difference between the resulting permutations from the two sets of connectomes (with the conditions that give the best results of generalization, i.e., α = 0.3). We align the connectomes of 50 subjects with the mean connectomes using either rs-FC₁ or rs-FC₂. On average, 84% of the labels are exactly at the same location for the two alignments using either rs-FC₁ or rs-FC₂ (including permuted and non-permuted labels). When restricting to permuted labels only, 52% of the labels were identically placed by both permutations. The different permutations across the two runs can be accounted for partly by the non-optimality characteristic of the FAQ algorithm, partly by the noise of BOLD signals, and partly by the significant inter-session variability of rs-fMRI widely studied (S. M. Smith et al., 2005; Wens et al., 2014; Zandbelt et al., 2008).

The widths of the distributions of η in Figures 3 and 4 show the great variability in the alignment results, meaning some subject connectomes align better to certain subjects. The results of Rasero, Diez, Cortes, Marinazzo, and Stramaglia (2019) indeed suggest that there are population subgroups of functional networks as clustering algorithms find natural clusters in a functional connectome set. Interestingly, the WBANC strategy has a lower variability of η values than other strategies (more peaked distributions), suggesting that restricting the set of permutations leads to a more homogeneous group of connectomes. To study this phenomenon, we calculated the mean η value per subject of reference, instead of on all the pairs of subjects. The standard deviation of these means across subjects of reference is strictly lower for the WBANC than for the WBA (1.58 versus 6.89 on SC for α = 0.3, using the first training dataset). This means that some subjects are better references, in the sense that they decrease more significantly the connectome distance when chosen as a reference, but it also suggests that the restriction to the spatially adjacent regions decreases this variability across the references. The selection of a reference is a common problem in neuroscience, encountered, for example, in registration. The typical choices are a subject of the cohort or a template such as the MNI152. In our case, generating a template would require constructing group-averaged structural and functional connectomes. However, we cannot simply average the group connectomes, as we do not assume a perfect matching of the labels across subjects. We could have computed the Frechet mean of all the connectomes, which consists of the iterative process of computing the Euclidean mean of the subject's connectomes and aligning every
connectome to this intermediate mean, until convergence (Calissano, Feragen, & Vantini, 2024). This
would have given a graph-unlabeled mean, where each region does not correspond to a single label
anymore, therefore losing the interpretability of the labels. Using a given subject as reference allows us to
get a direct mapping between real subjects, where we preserve the specific “meaning” of each region in
the graph.

Regarding the frequency of the permutations, the pie chart illustrated in Fig. 5B shows the great
number of permutations that are performed on a parcellation with such tiny regions (1000 regions within
the cortex). These results strengthen the need to consider the spatial variability across subjects when
using fine-grained parcellation. The need for an alignment procedure depends on the granularity of the
parcellation. Indeed, for broad regions, there is no need for alignment. In the article of Calissano,
Papadopoulos, et al. (2024), they show that permutations start to arise for parcellation granularities of
more than 400 regions over the cortex.

Our alignment criteria currently use two types of connectomes, and we have shown that this is not
enough to align perfectly with every other connectome modality. Including many connectome modalities
directly in the alignment process, such as SC, rs-FC, and t-FC, might be judicious to ensure the alignment
of all brain networks, as we have shown it is mathematically possible with our improved version of the
FAQ algorithm. This would imply having other hyperparameters to tune, and this would require external
data not seen in the optimization to validate the identified permutations, which are not available in the
HCP dataset.

We investigated how the permutations resulting from the alignment of SC and rs-FC₁ affect the
activation maps computed from t-fMRI for different motor stimuli. We observed that these permutations
do not significantly increase or decrease the similarity across subjects of these activation maps. This
result reflects the complexity of the relationship between a brain region's network role and its
task-specific activation, which is not straightforward. Under specific hypotheses, we can show that
aligning the connectome computed from a task-fMRI is more similar to aligning the *correlation* matrices
of the activation estimates of several stimuli, which does not imply an increase in the similarity of the
parameter estimates' maps directly. The demonstration is in the Supplementary materials. One way to
bridge this gap is to add another term to the objective function of the alignment that includes direct
alignment of activation maps. Future work could include modifying the alignment framework to account
for the increased similarity of different maps, such as activation maps or other properties, like cortical thickness maps.

The main argument for the use of the spatial constraint lies in the limitations of the functional view of the brain networks. Functional connectivity computed using Pearson's correlation coefficient is a statistical approximation of the effective connectivity between gray matter regions and represents direct and indirect connections using intermediary regions (Sporns, 2011). As a consequence, more regions are considered connected than in structural networks, and remote regions with similar functional connectivity profiles might be exchanged. Performing brain graph alignment on the sole basis of functional connectivity could lead to spurious alignment with meaningless permutations. As the brain cortex organization is globally similar across subjects (Pessoa, 2014), we avoid absurd permutations of remote regions by using a regularization term in the optimization process of alignment, which acts as a constraint to restrict the set of permutations.

Using the WBANC method, we restrict permutations to neighboring regions that depend on the parcellation size. We could imagine using the spatial distance between regions to construct a regularization matrix, to favor permutations of spatially close regions rather than strongly restricting permutations as our optimization criterion does.

Our alignment framework has been used for individuals from the same species but could be applied across species, similarly to the work of Mars et al. (2016), with the limitations that we need to have the same number of regions across the cortex for all the species. One possibility would be to combine our network alignment with the clustering of regions to map several small regions to a different number of regions.

Analysis tools for groups of subjects generally do not account for spatial variability. To consider it, the usual tools should be revisited, as was the case in fMRI analyses, where some methods account for spatial variation between subjects to extract common fMRI patterns (Haxby et al., 2011; Michael, Anderson, Miller, Adalı, & Calhoun, 2014; Takerkart, Auzias, Thirion, & Ralaivola, 2014). Our alignment approach is a step toward correcting for inter-subject variability before using the classical tools of connectome analysis at the group-level. For between-group comparisons (e.g., control vs. patient groups), the selection of the reference is a critical concern, as we would not want to decrease the between-group differences we are looking for. Thus, it remains to be determined whether a single reference should be used across all groups or whether separate references should be selected for each group.
Future work should include finding a way to identify the subjects who are the best references for the alignments, as we can only identify them post hoc. We also plan to use our alignment framework as a preprocessing step before group studies, to observe to what extent inter-subject variability impacts the results of the state-of-the-art group analysis tools. As an example, we will align the brain regions before attempting to predict FC from SC. Indeed, it remains challenging to produce a good estimator of functional networks from structural ones (Deslauriers-Gauthier et al., 2020; Smolders et al., 2024), possibly because current methods do not capture inter-subject variability when constructing structure-function mapping of subject groups.

525

526

527

528

529

530

531

532

## CONCLUSION

In this work, we propose a multimodal alignment algorithm that simultaneously increases the similarity of structural and functional connectomes across subjects. In comparison to previous work, our approach enforces anatomical constraints when permuting regions, leading to more robust permutations across connectome modalities. Beyond the novelty of the alignment framework, the results reveal substantial variability in connectivity at a fine-grained scale. Therefore, we recommend that researchers take spatial inter-subject variability into account when studying fine-grained connectivity.

533

534

535

536

537

538

## ACKNOWLEDGEMENTS

The authors are grateful to the OPAL infrastructure from Université Côte d'Azur for providing resources and support.

539

540

Data were provided by the Human Connectome Project, WU-Minn Consortium (Principal Investigators: David Van Essen and Kamil Ugurbil; 1U54MH091657) funded by the 16 NIH Institutes and Centers that support the NIH Blueprint for Neuroscience Research; and by the McDonnell Center for Systems Neuroscience at Washington University.

541

542

543

544

545

546

547

## REFERENCES
548

Amico, E., Arenas, A., & Goñi, J. (2019). Centralized and distributed cognitive task processing in the human connectome.

549

*Network Neuroscience*, 3(2), 455-474.

550

Avants, B. B., Tustison, N., & Song, G. (2009). Advanced normalization tools (ants). *Insight j*, 2, 1-35.

551

Barch, D. M., Burgess, G. C., Harms, M. P., Petersen, S. E., Schlaggar, B. L., Corbetta, M., ... others (2013). Function in the human connectome: task-fmri and individual differences in behavior. *Neuroimage*, 80, 169-189.

552

553

Biswal, B. B., Mennes, M., Zuo, X.-N., Gohel, S., Kelly, C., Smith, S. M., ... Milham, M. P. (2010). Toward discovery science of human brain function. *Proceedings of the National Academy of Sciences*, 107(10), 4734-4739. Retrieved from

554

555

https://www.pnas.org/doi/abs/10.1073/pnas.0911855107 doi: 10.1073/pnas.0911855107

556

Bohland, J. W., Bokil, H., Allen, C. B., & Mitra, P. P. (2009). The brain atlas concordance problem: quantitative comparison of anatomical parcellations. *PloS one*, 4(9), e7200.

557

558

Cabezas, M., Oliver, A., Lladó, X., Freixenet, J., & Cuadra, M. B. (2011). A review of atlas-based segmentation for magnetic resonance brain images. *Computer methods and programs in biomedicine*, 104(3), e158-e177.

559

560

Calissano, A., Feragen, A., & Vantini, S. (2024). Populations of unlabelled networks: Graph space geometry and generalized geodesic principal components. *Biometrika*, 111(1), 147-170.

561

562

Calissano, A., Papadopoulos, T., Pennec, X., & Deslauriers-Gauthier, S. (2024). Graph alignment exploiting the spatial organization improves the similarity of brain networks. *Human Brain Mapping*, 45(1), e26554.

563

564

Cao, M., Wang, J., Dai, Z., Cao, X., Jiang, L., Fan, F., ... others (2014). Topological organization of the human brain functional connectome across the lifespan. *Developmental cognitive neuroscience*, 7, 76-93.

565

566

Cela, E. (2013). *The quadratic assignment problem: theory and algorithms* (Vol. 1). Springer Science & Business Media.

567

Chen, H., Qi, L., Caccetta, L., & Zhou, G. (2019). Birkhoff-von Neumann theorem and decomposition for doubly stochastic tensors. *Linear Algebra and its Applications*, 583, 119-133. doi: 10.1016/j.laa.2019.08.027

568

569

Cole, M. W., Bassett, D. S., Power, J. D., Braver, T. S., & Petersen, S. E. (2014). Intrinsic and task-evoked network architectures of the human brain. *Neuron*, 83(1), 238-251.

570

571

Cole, M. W., Ito, T., Cocuzza, C., & Sanchez-Romero, R. (2021). The Functional Relevance of Task-State Functional Connectivity. *The Journal of Neuroscience*, 41(12), 2684-2702. doi: 10.1523/JNEUROSCI.1713-20.2021

572
573

Conte, D., Foggia, P., Sansone, C., & Vento, M. (2004). Thirty years of graph matching in pattern recognition. *International journal of pattern recognition and artificial intelligence*, 18(03), 265–298.

574

575

Crouse, D. (2016). On implementing 2d rectangular assignment algorithms. *IEEE Transactions on Aerospace and Electronic Systems*, 52(4), 1679–1696.

576

577

de Reus, M. A., & Van den Heuvel, M. P. (2013). The parcellation-based connectome: limitations and extensions. *Neuroimage*, 80, 397–404.

578

579

Deslauriers-Gauthier, S., Zucchelli, M., Frigo, M., & Deriche, R. (2020). A unified framework for multimodal structure–function mapping based on eigenmodes. *Medical Image Analysis*, 66, 101799.

580

581

Dickie, D. A., Shenkin, S. D., Anblagan, D., Lee, J., Blesa Cabez, M., Rodriguez, D., ... Wardlaw, J. M. (2017). Whole brain magnetic resonance image atlases: a systematic review of existing atlases and caveats for use in population imaging. *Frontiers in neuroinformatics*, 11, 1.

582

583

584

Di Tommaso, P., Chatzou, M., Floden, E. W., Barja, P. P., Palumbo, E., & Notredame, C. (2017). Nextflow enables reproducible computational workflows. *Nature biotechnology*, 35(4), 316.

585

586

Eickhoff, S. B., Thirion, B., Varoquaux, G., & Bzdok, D. (2015). Connectivity-based parcellation: Critique and implications. *Human brain mapping*, 36(12), 4771–4792.

587

588

Fornito, A., Zalesky, A., & Bullmore, E. (2016). *Fundamentals of brain network analysis* (1st ed.). United States of America: Academic Press. doi: 10.1016/B978-0-12-407908-3.09999-4

589

590

Frank, M., & Wolfe, P. (1956). An algorithm for quadratic programming. *Naval Research Logistics Quarterly*, 3(1-2), 95-110. Retrieved from https://onlinelibrary.wiley.com/doi/abs/10.1002/nav.3800030109 doi: https://doi.org/10.1002/nav.3800030109

591

592

593

Frigo, M., Cruciani, E., Coudert, D., Deriche, R., Natale, E., & Deslauriers-Gauthier, S. (2021). Network alignment and similarity reveal atlas-based topological differences in structural connectomes. *Network Neuroscience*, 5(3), 711–733.

594

595

Garyfallidis, E., Brett, M., Amirbekian, B., Rokem, A., Van Der Walt, S., Descoteaux, M., & Nimmo-Smith, I. (2014). Dipy, a library for the analysis of diffusion mri data. *Frontiers in neuroinformatics*, 8, 8.

596
597

Glasser, M., Sotiropoulos, S., Wilson, J., Coalson, T., Fischl, B., Andersson, J., ... others (2013). The minimal preprocessing pipelines for the human connectome project. *Neuroimage*, 80, 105-124.

598

599

Griffanti, L., Salimi-Khorshidi, G., Beckmann, C. F., Auerbach, E. J., Douaud, G., Sexton, C. E., ... others (2014).

600

Ica-based artefact removal and accelerated fmri acquisition for improved resting state network imaging. *Neuroimage*, 95, 232-247.

601

602

Hagmann, P., Cammoun, L., Gigandet, X., Meuli, R., Honey, C. J., Wedeen, V. J., & Sporns, O. (2008). Mapping the structural core of human cerebral cortex. *PLoS biology*, 6(7), e159.

603

604

Haxby, J. V., Guntupalli, J. S., Connolly, A. C., Halchenko, Y. O., Conroy, B. R., Gobbini, M. I., ... Ramadge, P. J. (2011).

605

A common, high-dimensional model of the representational space in human ventral temporal cortex. *Neuron*, 72(2), 404-416.

606

607

Herculano-Houzel, S. (2009). The human brain in numbers: a linearly scaled-up primate brain. *Frontiers in human neuroscience*, 31.

608

609

Jenkinson, M., Beckmann, C. F., Behrens, T. E., Woolrich, M. W., & Smith, S. M. (2012). Fsl. *Neuroimage*, 62(2), 782-790.

610

611

Kurtzer, G. M., Sochat, V., & Bauer, M. W. (2017). Singularity: Scientific containers for mobility of compute. *PloS one*, 12(5), e0177459.

612

613

Maier-Hein, K., Neher, P., Houde, J.-C., Côté, M.-A., Garyfallidis, E., Zhong, J., ... others (2017). The challenge of mapping the human connectome based on diffusion tractography. *Nature communications*, 8(1), 1349.

614

615

Mars, R., Verhagen, L., Gladwin, T., Neubert, F., Sallet, J., & Rushworth, M. (2016). Comparing brains by matching connectivity profiles. *Neuroscience & Biobehavioral Reviews*, 60, 90-97.

616

617

Mheich, A., Hassan, M., Gripon, V., Khalil, M., Berrou, C., Dufor, O., & Wendling, F. (2015). A novel algorithm for measuring graph similarity: application to brain networks. In *2015 7th international ieee/embs conference on neural engineering (ner)* (pp. 1068-1071).

618

619

620

Mheich, A., Wendling, F., & Hassan, M. (2020). Brain network similarity: methods and applications. *Network Neuroscience*, 4(3), 507-527.

621
622

Michael, A. M., Anderson, M., Miller, R. L., Adalı, T., & Calhoun, V. D. (2014). Preserving subject variability in group fmri analysis: performance evaluation of gica vs. iva. *Frontiers in systems neuroscience, 8*, 106.

623

624

Mueller, S., Wang, D., Fox, M., Yeo, B., Sepulcre, J., Sabuncu, M., ... Liu, H. (2013). Individual variability in functional connectivity architecture of the human brain. *Neuron, 77*(3), 586-595.

625

626

Passingham, R. E., Stephan, K. E., & Kötter, R. (2002). The anatomical basis of functional localization in the cortex. *Nature Reviews Neuroscience, 3*(8), 606-616.

627

628

Pessoa, L. (2014). Understanding brain networks and brain organization. *Physics of life reviews, 11*(3), 400-435.

629

Rasero, J., Diez, I., Cortes, J. M., Marinazzo, D., & Stramaglia, S. (2019). Connectome sorting by consensus clustering increases separability in group neuroimaging studies. *Network Neuroscience, 3*(2), 325-343.

630

631

Romascano, D., Meskaldji, D., Bonnier, G., Simioni, S., Rotzinger, D., Lin, Y., ... others (2015). Multicontrast connectometry: A new tool to assess cerebellum alterations in early relapsing-remitting multiple sclerosis. *Human brain mapping, 36*(4), 1609-1619.

632

633

634

Salehi, M., Greene, A., Karbasi, A., Shen, X., Scheinost, D., & Constable, R. (2020). There is no single functional atlas even for a single individual: Functional parcel definitions change with task. *NeuroImage, 208*, 116366.

635

636

Salimi-Khorshidi, G., Douaud, G., Beckmann, C. F., Glasser, M. F., Griffanti, L., & Smith, S. M. (2014). Automatic denoising of functional mri data: combining independent component analysis and hierarchical fusion of classifiers. *Neuroimage, 90*, 449-468.

637

638

639

Schaefer, A., Kong, R., Gordon, E. M., Laumann, T. O., Zuo, X.-N., Holmes, A. J., ... Yeo, B. T. (2018). Local-global parcellation of the human cerebral cortex from intrinsic functional connectivity mri. *Cerebral cortex, 28*(9), 3095-3114.

640

641

Schilling, K. G., Tax, C. M., Rheault, F., Landman, B. A., Anderson, A. W., Descoteaux, M., & Petit, L. (2022). Prevalence of white matter pathways coming into a single white matter voxel orientation: The bottleneck issue in tractography. *Human brain mapping, 43*(4), 1196-1213.

642

643

644

Seghier, M. L., & Price, C. J. (2009). Dissociating functional brain networks by decoding the between-subject variability. *Neuroimage, 45*(2), 349-359.

645
646

647

Seghier, M. L., & Price, C. J. (2018). Interpreting and utilising intersubject variability in brain function. *Trends in cognitive sciences*, 22(6), 517–530.

648

649

Smith, R. E., Tournier, J.-D., Calamante, F., & Connelly, A. (2013). Sift: Spherical-deconvolution informed filtering of tractograms. *NeuroImage*, 67, 298-312. Retrieved from

650

https://www.sciencedirect.com/science/article/pii/S1053811912011615 doi:

651

https://doi.org/10.1016/j.neuroimage.2012.11.049

652

653

Smith, S. M., Beckmann, C. F., Andersson, J., Auerbach, E. J., Bijsterbosch, J., Douaud, G., ... others (2013). Resting-state fmri in the human connectome project. *Neuroimage*, 80, 144–168.

654

655

Smith, S. M., Beckmann, C. F., Ramnani, N., Woolrich, M. W., Bannister, P. R., Jenkinson, M., ... McGonigle, D. J. (2005). Variability in fmri: A re-examination of inter-session differences. *Human brain mapping*, 24(3), 248–257.

656

657

Smolders, L., De Baene, W., Rutten, G.-J., van der Hofstad, R., & Florack, L. (2024). Can structure predict function at individual level in the human connectome? *Brain Structure and Function*, 1–15.

658

659

Sporns, O. (2011). The human connectome: a complex network. *Annals of the new York Academy of Sciences*, 1224(1), 109–125.

660

661

Sporns, O., Tononi, G., & Kötter, R. (2005). The human connectome: a structural description of the human brain. *PLoS computational biology*, 1(4), e42.

662

663

Takerkart, S., Auzias, G., Thirion, B., & Ralaivola, L. (2014). Graph-based inter-subject pattern analysis of fmri data. *PloS one*, 9(8), e104586.

664

665

Tate, M., Herbet, G., Moritz-Gasser, S., Tate, J., & Duffau, H. (2014). Probabilistic map of critical functional regions of the human cerebral cortex: Broca's area revisited. *Brain*, 137(10), 2773–2782.

666

667

Theaud, G., Houde, J.-C., Boré, A., Rheault, F., Morency, F., & Descoteaux, M. (2020). Tractoflow: A robust, efficient and reproducible diffusion mri pipeline leveraging nextflow & singularity. *NeuroImage*. Retrieved from

668

https://www.sciencedirect.com/science/article/pii/S105381192030375X doi:

669

https://doi.org/10.1016/j.neuroimage.2020.116889

670

671

Thompson, P., Cannon, T., Narr, K., Van Erp, T., Poutanen, V., Huttunen, M., ... others (2001). Genetic influences on brain structure. *Nature neuroscience*, 4(12), 1253–1258.
672

Tournier, J.-D., Smith, R. E., Raffelt, D. A., Tabbara, R., Dhollander, T., Pietsch, M., ... Connelly, A. (2019). Mrtrix3: A fast, flexible and open software framework for medical image processing and visualisation. *bioRxiv*, 551739.

673

674

Van Essen, D., Ugurbil, K., Auerbach, E., Barch, D., Behrens, T., Bucholz, R., ... Yacoub, E. (2012). The human connectome project: A data acquisition perspective. *NeuroImage*, 62(4), 2222-2231. Retrieved from
https://www.sciencedirect.com/science/article/pii/S1053811912001954 (Connectivity) doi:
https://doi.org/10.1016/j.neuroimage.2012.02.018

675

676

677

678

Van Den Heuvel, M. P., & Pol, H. E. H. (2010). Exploring the brain network: a review on resting-state fmri functional connectivity. *European neuropsychopharmacology*, 20(8), 519-534.

679

680

Van Horn, J. D., Grafton, S. T., & Miller, M. B. (2008). Individual variability in brain activity: a nuisance or an opportunity?
*Brain imaging and behavior*, 2, 327-334.

681

682

Vigneau, M., Beaucousin, V., Hervé, P., Duffau, H., Crivello, F., Houde, O., ... Tzourio-Mazoyer, N. (2006).

683

Meta-analyzing left hemisphere language areas: phonology, semantics, and sentence processing. *Neuroimage*, 30(4), 1414-1432.

684

685

Virtanen, P., Gommers, R., Oliphant, T. E., Haberland, M., Reddy, T., Cournapeau, D., ... SciPy 1.0 Contributors (2020).

686

SciPy 1.0: Fundamental Algorithms for Scientific Computing in Python. *Nature Methods*, 17, 261-272. doi:
10.1038/s41592-019-0686-2

687

688

Vogelstein, J., Conroy, J., Lyzinski, V., Podrazik, L., Kratzer, S., Harley, E., ... Priebe, C. (2015). Fast approximate quadratic programming for graph matching. *PLOS one*, 10(4), e0121002.

689

690

Wens, V., Bourguignon, M., Goldman, S., Marty, B., Op de Beeck, M., Clumeck, C., ... others (2014). Inter-and intra-subject variability of neuromagnetic resting state networks. *Brain topography*, 27(5), 620-634.

691

692

Yeh, C.-H., Smith, R., Dhollander, T., Calamante, F., & Connelly, A. (2019). Connectomes from streamlines tractography: Assigning streamlines to brain parcellations is not trivial but highly consequential. *Neuroimage*, 199, 160-171.

693

694

Zandbelt, B. B., Gladwin, T. E., Raemaekers, M., van Buuren, M., Neggers, S. F., Kahn, R. S., ... Vink, M. (2008).
Within-subject variation in bold-fmri signal changes across repeated measurements: quantification and implications for
sample size. *Neuroimage*, 42(1), 196-206.

695
697

Zhang, F., Daducci, A., He, Y., Schiavi, S., Seguin, C., Smith, R. E., ... O'Donnell, L. J. (2022). Quantitative mapping of the brain's structural connectivity using diffusion mri tractography: A review. *Neuroimage, 249*, 118870.

698

699

Zhang, X., Guo, L., Li, X., Zhang, T., Zhu, D., Li, K., ... Liu, T. (2013). Characterization of task-free and task-performance brain states via functional connectome patterns. *Medical Image Analysis, 17*(8), 1106-1122. doi:

700

10.1016/j.media.2013.07.003

701
1 RESEARCH

SUPPLEMENTARY MATERIAL: INTER-INDIVIDUAL ALIGNMENT
OF MULTIMODAL BRAIN NETWORKS WITH ANATOMICAL
CONSTRAINTS

Yanis Aeschlimann¹, Anna Calissano², Theodore Papadopoulo¹,
and Samuel Deslauriers-Gauthier¹

7
¹Université Côte d'Azur, Inria, Sophia-Antipolis, France

8
²Department of Mathematics, Imperial College London, London, United Kingdom
<figure><img src="image_13.png" /><figcaption>Figure 1: Distributions of the difference between the connectomes' distances respectively before and after the alignment on structural and resting-state functional connectomes of the first run, for the three strategies: HA (top row), WBA (middle row), and WBANC (bottom row) and every value of $\alpha$ between 0 and 1 with a step size of 0.1.</figcaption></figure>
<figure><img src="image_14.png" /><figcaption>Figure 2: Distributions of the difference between the connectomes' distances respectively before and after the alignment on structural and task functional connectomes, for the three strategies: HA (top row), WBA (middle row), and WBANC (bottom row) and every value of $\alpha$ between 0 and 1 with a step size of 0.1.</figcaption></figure>
<figure><img src="image_15.png" /><figcaption>Figure 3: Four different examples of alignment of subjects using the WBANC strategy. The permutations of regions occur in cycles of spatially adjacent areas. Each color corresponds to a cycle of permutations.</figcaption></figure>
9 Alignment of connectome versus alignment of activation maps

10 A functional connectome is computed as

$$
C = AA^{T} \tag{1}
$$

11 where A is an $N_{\text{regions}} \times t_{\text{samples}}$ matrix containing the averaged normalized BOLD signals for each brain region. Assuming a very simple general linear model (GLM), we can compute the activation maps. We have
13

$$
A^T = G\hat{\beta}, \tag{2}
$$

14 neglecting the noise and the BOLD normalization. G is the design matrix that contains the regressors
15 (i.e., stimuli convolved with the hemodynamic response function and noise regressors), and $\hat{\beta}$ are the
16 estimated strengths of the regressors for all regions.

17 Aligning the functional connectomes minimizes

$$
\| PAA^T P^T - A_r A_r^T \| = \| P \hat{\beta}^T G^T G \hat{\beta} P^T \hat{\beta}_r^T G^T G \hat{\beta}_r \|. \quad (3)
$$

18 With the strong assumption that $G$ is semi-orthogonal (which is not in practice, at least due to the overlap
19 of the regressors), i.e. $G^T G = I$, this simplifies to minimizing $\|P\hat{\beta}^T\hat{\beta}P^T - \hat{\beta}_r^T\hat{\beta}_r\|$, which is the
20 alignment of the *correlation* matrices of the estimators $\hat{\beta}$.