<figure><img src="image_1.png" /></figure>

HAL
open science

# Mitigating analytical variability in fMRI results with
style transfer

Elodie Germani, Camille Maumet, Elisa Fromont

<figure><img src="image_2.png" /></figure>

## To cite this version:

Elodie Germani, Camille Maumet, Elisa Fromont. Mitigating analytical variability in fMRI results with style transfer. 2024. inserm-04531405

HAL Id: inserm-04531405

https://inserm.hal.science/inserm-04531405

Preprint submitted on 3 Apr 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_3.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_1.png" /></figure>

HAL
open science

# Mitigating analytical variability in fMRI results with
style transfer

Elodie Germani, Camille Maumet, Elisa Fromont

► To cite this version:

Elodie Germani, Camille Maumet, Elisa Fromont. Mitigating analytical variability in fMRI results with style transfer. Medical Imaging with Deep Learning (MIDL 2025), Jul 2025, Salt Lake City (US), United States. inserm-04531405v3

HAL Id: inserm-04531405

https://inserm.hal.science/inserm-04531405v3

Submitted on 16 Jan 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Mitigating analytical variability in fMRI results with style transfer

Elodie Germani<fnref n="1" />, Camille Maumet<fnref n="2" />, and Elisa Fromont<fnref n="1" />

**Abstract.** We propose a novel approach to improve the reproducibility of neuroimaging results by converting statistic maps across different functional MRI pipelines. We make the assumption that pipelines can be considered as a style component of data and propose to use different generative models, among which, Diffusion Models (DM) to convert data between pipelines. We design a new DM-based unsupervised multi-domain image-to-image transition framework and constrain the generation of 3D fMRI statistic maps using the latent space of an auxiliary classifier that distinguishes statistic maps from different pipelines. We extend traditional sampling techniques used in DM to improve the transition performance. Our experiments demonstrate that our proposed methods are successful: pipelines can indeed be transferred, providing an important source of data augmentation for future medical studies.

**Keywords:** Neuroimaging · Analytical variability · Deep learning · Style transfer · Diffusion models · Generative models.

## 1 Introduction

For the past ten years, scientific research faced a “reproducibility crisis”, prompting the adoption of new research practices, in particular to increase sample sizes. Data sharing platforms [18,8] were developed to facilitate the re-use of raw data but also of derived data through meta- and mega-analyses [5]. In functional Magnetic Resonance Imaging (fMRI), due to the high flexibility of the analytical pipelines [2], derived data shared on public databases often come from different workflows, also known as ‘pipelines’. However, different pipelines can lead to different results [1] and combining results from different pipelines in mega-analyses can lead to a higher risk of false positive findings [26]. To benefit from these large amount of derived data available, it is necessary to find a way to mitigate the effect of analytical variability.

In computer vision, recent advances gave rise to performing deep generative models such as Generative Adversarial Networks (GANs) [7] and Denoising Diffusion Probabilistic Models (DDPMs) [9]. These models produce high quality results for generating new images from a known distribution but also in the task of image-to-image transition [11,28], i.e. converting data from a source domain to

<footnote n="1">¹ Univ Rennes, Inria, CNRS, Inserm, Rennes, France</footnote>

<footnote n="2">² Univ Rennes, IUF, Inria, CNRS, Rennes, France
{elodie.germani},{elisa.fromont}@irisa.fr</footnote>
# Mitigating analytical variability in fMRI with style transfer

Elodie Germani<fnref n="1" />

Camille Maumet*<fnref n="1" />

Elisa Fromont*<fnref n="2" />

ELODIE.GERMANI@UKBONN.DE

CAMILLE.MAUMET@INRIA.FR

ELISA.FROMONT@IRISA.FR

**Editors:** Under Review for MIDL 2025

## Abstract

We propose a novel approach to facilitate the re-use of neuroimaging results by converting statistic maps across different functional MRI pipelines. We make the assumption that pipelines used to compute fMRI statistic maps can be considered as a style component and we propose to use different generative models, among which, Generative Adversarial Networks (GAN) and Diffusion Models (DM) to harmonize statistic maps across different pipelines. We explore the performance of multiple GAN and DM frameworks for unsupervised multi-domain style transfer. We developed an auxiliary classifier that distinguishes statistic maps from different pipelines, allowing us to validate pipeline transfer, but also to extend traditional sampling techniques used in DM to improve the transition performance. Our experiments demonstrate that our proposed methods are successful: pipelines can indeed be transferred as a style component, providing an important source of data augmentation for future studies.

**Keywords:** style transfer, generative models, analytical variability, fMRI, data re-use

## Introduction

Over the last decades, the question of understanding brain functions has taken an important place in many research fields. With the development of brain imaging techniques such as task-based functional MRI (task-fMRI), researchers can now explore the brain activity of individuals while they perform predefined tasks, and get a better understanding of the neural correlates of different cognitive processes.

However, the “reproducibility crisis” raised concerns about the reliability of published findings, including in neuroimaging (Button et al., 2013; Poldrack et al., 2017; Botvinik-Nezer and Wager, 2023), and prompted the adoption of new research practices. Efforts have been made to increase sample sizes by acquiring data from a larger number of participants for a few numbers of cognitive tasks (e.g. UK Biobank (Sudlow et al., 2015)) or for a small number of participants on a larger number of cognitive tasks (e.g. Individual Brain Charting (Pinho et al., 2018)). However, the number of research questions that can be explored is always limited by the characteristics of each dataset.

With the increased adoption of data sharing (Poline et al., 2012), more and more neuroimaging data are made available on dedicated platforms (e.g. OpenNeuro (Markiewicz et al., 2021) or NeuroVault (Gorgolewski et al., 2015)). Re-using shared data in new studies would allow researchers to explore new research questions, with larger and more diverse

<footnote n="1">¹ Univ Rennes, Inria, CNRS, Inserm, Rennes, France</footnote>

<footnote n="2">² Univ Rennes, IUF, Inria, CNRS, Rennes, France</footnote>

<footnote>* Joint senior authorship</footnote>
a target domain. Unsupervised frameworks [32,30,15] do not necessitate pairs of data in different domains for their training as they use constraints like cycle consistency [32] and shared latent space assumption [15,30]. They provide a good opportunity to benefit from large unlabeled databases to learn complex mapping without any ground-truth target data. By conditioning on domain-specific features (e.g. class vector, latent space of auxiliary classifiers, etc.) instead of full target images, unsupervised frameworks also extend to multi-domain transitions [4,3,10] to learn transfer between multiple domains in a single model.

Recently, DDPMs have achieved state-of-the-art performance in synthesizing natural images, overpassing GANs by producing complex and diverse images [20], while reducing the risk of modality collapse [14]. In medical imaging, image-to-image transition frameworks are used for multiple tasks, including modality transition in which DDPMs and GANs showed promising performance [21,16,22,6,13]. Lyu et al [16] showed the superiority of diffusion models compared to GANs in this task for the conversion between MRI and Computed Tomography (CT) using a supervised framework (i.e., with pairs of data from both modalities). In unsupervised settings, Pan et al [22] developed a cycle-guided DDPM composed of two DDPMs that condition each other to generate synthetic images from two different MRI pulse sequences. Considering the achievements of these models in modality transition, which involves transitioning between distinct acquisition modalities, there is reason to anticipate their success in transitioning between other image types, such as analysis pipelines.

However, DDPMs are challenging to control when the objective is to generate images that maintain the intrinsic properties of the source images while transferring the extrinsic properties to the target domain. DDPMs are iterative generative models, i.e. they learn to model the transition from a Gaussian distribution to a target data distribution. Thus, data generated by the DDPM depend on the initial samples drawn from the Gaussian distribution, usually at random. Moreover, conditioning is often performed using a single target image or a one-hot encoded vector which might not represent the whole variability of the target domain.

In this work, we design a new unsupervised multi-domain DDPM framework in order to convert fMRI results (i.e. statistic maps) across pipelines, i.e. to convert a map from a pipeline (source domain) to the corresponding maps from another pipelines (target domains). Our contributions are as follows:

- We are the first to make the assumption that pipelines can be considered as extrinsic properties of statistic maps and can be transferred between maps.

- We extend the methods from [10,25] to build a classifier-conditional DDPM, called CCDDPM, in an unsupervised multi-domain transition framework to convert between multiple pipelines using a single model.

- We condition our CCDDPM using the latent space of a classifier trained to distinguish statistic maps between pipelines, a task previously unexplored.

- We propose a novel sampling strategy by selecting multiple target samples using a guided process based on clustering, to improve both transfer of target domain features while maintaining the source image properties.
datasets, while bypassing the difficulties associated with acquiring new data. Derived data could also be combined instead of raw data through meta- and mega-analyses (Costafreda, 2009). This process also reduces privacy constraints and avoids costly re-computations.

In task-fMRI, due to the high flexibility of analytical pipelines (Carp, 2012), derived data shared on public databases often come from different pipelines. However, different pipelines can lead to different results (Botvinik-Nezer et al., 2020), and combining results from different pipelines in mega-analyses can lead to a higher risk of false positive findings (Rolland et al., 2022). To benefit from this large amount of derived data available, it is thus necessary to find a way to mitigate the effect of pipeline differences on derived data.

To mitigate the effect of different sources of variability, researchers usually perform data harmonization with techniques such as style transfer (Gatys et al., 2016) that allow learning mappings between different domains. This technique makes use of generative models such as Generative Adversarial Networks (GANs) (Goodfellow et al., 2014) and Diffusion Models (DMs) (Ho et al., 2020). Supervised frameworks (Isola et al., 2017; Saharia et al., 2022) can be trained to learn a mapping between data pairs from different domains. In contrast, unsupervised frameworks (Zhu et al., 2017; Sasaki et al., 2021; Liu et al., 2017) do not necessitate pairs of data in different domains for their training as they use constraints like cycle consistency (Zhu et al., 2017) or shared latent space assumption (Liu et al., 2017; Sasaki et al., 2021). These provide a good opportunity to benefit from large unlabeled databases and learn complex mapping without ground truth. By conditioning on domain-specific features, unsupervised frameworks also extend to multi-domain (Choi et al., 2018, 2021; Ho and Salimans, 2021) to learn transfer between multiple domains in a single model.

In this work, we explore the ability of style transfer frameworks to convert task-fMRI statistic maps between pipelines. Our goal is to propose a solution to mitigate the effect of pipeline differences for fMRI mega-analyses in order to benefit from the large amount of derived data shared on public databases. To be useful in real practice, the proposed method should rely on unpaired data and perform multi-domain transitions. To our knowledge, this application of style transfer to data conversion between different analysis pipelines is new, and off-the-shelf methods do not directly apply as these were not designed on the same type of data.

To tackle these challenges, we made the following contributions:

* We are the first to make the assumption that pipelines can be considered as a style property of statistic maps which can be transferred between maps.

* We re-implement three state-of-the-art style transfer frameworks based on GAN, namely Pix2Pix (Isola et al., 2017), CycleGAN (Zhu et al., 2017) and StarGAN (Choi et al., 2018), and adapt them to our 3-dimensional statistic maps.

* We re-implement a state-of-the-art conditional DM (Ho and Salimans, 2021) and adapt it for style transfer by conditioning the sampling on the source image.

* We explore different types of conditioning for DM frameworks: in particular using the latent space of an auxiliary classifier trained to distinguish statistic maps between pipelines, a task previously unexplored.
<figure><img src="image_4.png" /><figcaption>Fig. 1. Diagram of the workflow. During the forward diffusion (A), original maps $X_0$ are turned into $X_t$ after $t$ steps of noise addition $\epsilon$. (B) Class conditioning uses latent vectors extracted from a classifier. These are averaged across $N$ images, which are the centroids of $N$ clusters identified using a K-Means algorithm. (C) Time and class are embedded using two Multi-Layers Perceptrons (MLP). A mask is applied to the class conditioning vector to jointly train an unconditional model with a pre-defined probability. (D) During the reverse diffusion, the neural network $\epsilon_\theta(X_t, t, c)$ learns to predict the noise added to the image and reconstructs $X_{t-1}$ iteratively until $t = 0$.</figcaption></figure>

- We compare CCDDPM to simpler DDPM models and also to a state-of-the-art multi-domain image-to-image transition model, starGAN [4], specifically implemented in 3D. We evaluate and compare the performance of each competitors on a broad set of metrics.

## 2 Classifier-Conditional Diffusion Model (CCDDPM)

Fig. 1 illustrates the model design. CCDDPM uses traditional diffusion processes [9] and the conditioning principles of [10]. We improve the sampling strategies by fixing the initial state of the DDPM with noisy source images to keep its intrinsic properties and by conditioning on the latent space of a classifier using multiple target images to represent the diversity of the target domain.

### 2.1 Diffusion processes (forward and reverse)

Diffusion models work by successively adding noise to the training data, and then learn to reverse the process to construct desired data samples from the noise. In the forward diffusion process (Fig. 1(A)), the source image $X_0$ is subjected to $t$ steps of gradual noise $\epsilon$ addition to generate intermediate noisy versions of the image $\{X_0, X_1, ..., X_t\}$. The $t$-th version of the image is expressed as:

$$X_t = \sqrt{\bar{\alpha}_t} * X_0 + \sqrt{1 - \bar{\alpha}_t} * \epsilon \quad \text{with} \quad \epsilon \sim N(0, I) \quad (1)$$
# Materials and Methods

## Dataset

We use group-level statistic maps from the *HCP multi-pipeline dataset*. More details can be found in (Germani et al., 2023). Briefly, this dataset is composed of subject-level (1,080 participants) and group-level (1,000 groups) statistic and contrast maps derived from raw data of the *Human Connectome Project Young Adult S1200 release* (Van Essen et al., 2013b). In this dataset, raw fMRI data for the motor task were analyzed with 24 different pipelines for the 5 contrasts: *right-hand*, *right-foot*, *left-hand*, *left-foot* and *tongue*. The pipelines used in this dataset vary in terms of software package, smoothing kernel Full-Width at Half-Maximum (FWHM), number of motion regressors, and derivatives of the Haemodynamic Response Function (HRF) included in the first-level analysis.

We explore in particular the statistic maps obtained with four different pipelines that differ in terms of software packages (SPM (Penny et al., 2011) or FSL (Jenkinson et al., 2012)) and the presence or absence of the derivatives of the HRF for the first-level analysis. We use all the available group-level statistic maps (N = 1,000) for each pipeline for the contrast *right-hand*. In the following, these pipelines are labeled with "<software>-<derivatives>", for instance "fsl-1" means the use of FSL software package with HRF derivatives.

The selected group-level statistic maps are resampled to a size of 48 x 56 x 48 and masked using the intersection mask of all groups. The voxel values are normalized between -1 and 1 for each statistic map using a min-max operation. The 1,000 groups are split into train and test with an 80/20 ratio and all models are trained and evaluated on the same sets. Further investigation about possible data leakage across groups is provided in Supplementary Figure 1 (Germani et al., 2024b).

## GAN frameworks

First, we assess the potential of GAN frameworks to convert statistic maps between pipelines. In particular, we evaluate the performance of *Pix2Pix* (Isola et al., 2017), *CycleGAN* (Zhu et al., 2017) and *StarGAN* (Choi et al., 2018). A detailed description of each framework is available in the corresponding papers. We provide a quick description of the main properties of these models in Table 1.

<table><caption>Table 1: Description of GAN frameworks</caption><thead><tr><td>Framework</td><td>Learning</td><td>Transition</td><td>Loss</td></tr></thead><tbody><tr><td>Pix2Pix (Isola et al., 2017)</td><td>Supervised</td><td>One-to-one</td><td>Adversarial Reconstruction</td></tr><tr><td>CycleGAN (Zhu et al., 2017)</td><td>Unsupervised</td><td>One-to-one</td><td>Adversarial Cyclic</td></tr><tr><td>StarGAN (Choi et al., 2018)</td><td>Unsupervised</td><td>Multi-domain</td><td>Adversarial Cyclic Classification</td></tr></tbody></table>

**Architecture and training** We use the default architecture of these models, as described in their respective papers, and we only modify the 2-dimensional convolutions and batch
where $\alpha_t$ corresponds to fixed hyper-parameters between 0 and 1 related to the variance and $\bar{\alpha}_t = \prod_{i=1}^t \alpha_t$, extracted from [9].

The reverse diffusion process (Fig. 1(D)) uses a neural network trained to predict the noise added to the image $\hat{\epsilon}_t = \epsilon_\theta(X_t, t, C)$ at each time step $t$ given $X_t$ the noisy version of $X_0$, $t$ the corresponding time step and $C$ a conditioning vector. During training, the conditioning vector correspond to features of the true domain of the image, see 2.2). This way, the model learns to predict the noise added to the image while taking into account the domain of the image. This creates a multi-domain framework in which the model can predict the noise from any source image and to convert this image to each domain seen during training. Starting from $X_t$ and using the predicted noise, the image $X_{t-1}$ from previous step can be reconstructed using the following equation and we can reconstruct $X_0$ by repeating this process for $t$ times:

$$ \hat{X}_{t-1} = \frac{1}{\sqrt{\alpha_t}} \cdot (X_t - \frac{1-\alpha_t}{\sqrt{1-\bar{\alpha}_t}} \cdot \hat{\epsilon}_t) + \sqrt{1-\bar{\alpha}_{t-1}} \cdot z \quad \text{where} \quad z \sim N(0, I) \quad (2) $$

The equation is extracted from [9]. The network $\epsilon_\theta(X_t, t, C)$ is trained using a Mean Squared Error loss, $L_{MSE} := \mathbb{E}_{X,C,t,\epsilon \sim N(0,1)} [\| \epsilon_t - \hat{\epsilon}_t \|_2^2]$.

## 2.2 Conditional diffusion

The general method of conditional diffusion follows the principles described in [10]. Timestep and conditioning are embedded using 2 MLP and infused with the neural network activations at a certain layer via $a_{L+1} = c_{emb} \cdot a_L + t_{emb}$ (Fig. 1(C)). An unconditional DDPM is trained along with the conditional one by setting a contrast mask $m$. This mask changes the conditioning vector to a null token $\emptyset$ with some probability $p_{uncond.}$, set as hyper-parameter. During sampling, we compute both conditional and unconditional noise prediction and we perform a linear combination of the two with a weight $w$ to represent the strength of the conditional guidance using the equation from [10]:

$$ \hat{\epsilon}_{\theta}(X_t, t, C) = (1+w) \cdot \hat{\epsilon}_{\theta}(X_t, t, C) - w \cdot \hat{\epsilon}_{\theta}(X_t, t) \quad (3) $$

**Classifier conditioning.** In [10], the diffusion is conditioned using a one-hot encoding of the domain, which decreases the diversity of samples. In [25], a semantic encoder is used to guide sampling. In CCDDPM, we extend this idea by using a pre-trained Convolutional Neural Network (CNN) classifier that identifies the pipeline used in statistic maps to condition the model (Fig. 1(B)). The features are extracted just before the fully connected layer, to get a latent vector with the most important features that distinguish images across pipelines.

## 2.3 Sampling strategy for image-to-image transition

**Multi-target images.** To condition on the latent space of the classifier during sampling, target images must be selected. In [3], authors showed that conditioning on multiple images generates images that share coarse or fine features with
normalization layers to cope with our 3-dimensional statistic maps. These were implemented using PyTorch (Paszke et al., 2019) and each framework was trained for 200 epochs on 1 GPU NVIDIA Tesla V100.

## DM frameworks

Due to the promising performance of DM on natural images and medical imaging (Dhariwal and Nichol, 2021), we also assess the potential of DM frameworks. However, only a few DM frameworks have been developed for style transfer, and to our knowledge, they rely on paired data (Saharia et al., 2022) or learn only one-to-one transitions (Pan et al., 2023). Thus, to perform multi-domain transitions, we adapt an existing conditional DM to style transfer tasks. We use the framework from (Ho and Salimans, 2021), which generates images conditioned using a one-hot encoding of the class (i.e. class vector). We extend this model to conditioning based on the latent space of an auxiliary classifier, inspired from (Preechakul et al., 2022). Both are unsupervised frameworks, learning multi-domain transitions. A more detailed description of the original framework is available in (Ho and Salimans, 2021).

**Source content preservation** To adapt our framework to style transfer, we first needed to find a solution to generate images that maintain the intrinsic properties of the source image. In (Saharia et al., 2022), authors concatenate the source image with random Gaussian noise to initialize the diffusion. Here, we propose to fix the initial state of the DM by directly using the forward diffusion process to generate a noisy version of the source image $X_t$. Then, the noisy source image is iteratively denoised using the predicted noise conditioned on the target domain and the reverse diffusion process.

**Classifier conditioning** We also extend the model from (Ho and Salimans, 2021) with conditioning on the latent space of an auxiliary classifier. In (Ho and Salimans, 2021), the diffusion is conditioned using a one-hot encoding of the domain, limiting the diversity of generated samples. In (Preechakul et al., 2022), a semantic encoder is used to guide sampling. We extend this idea by conditioning the model on a latent feature vector extracted from a pre-trained CNN. This CNN was pre-trained to distinguish pipelines from the statistic maps. The features are extracted just before the classification layer, to get a good representation to distinguish images across pipelines.

**Multi-target images** (Choi et al., 2021) showed that conditioning on multiple images generates images that share more coarse or fine features with the target ones depending on the number of selected images. Here, we aim to better represent the heterogeneity of the target domain. In practice, the whole set of images available in the target domain could be used. This is impractical for large datasets and might lead the model to focus on specific patterns of the target domain if these are over-represented in the dataset.

**Architecture and training** The neural network used to predict the noise follows a simple U-Net architecture (Ronneberger et al., 2015) with two downsampling and upsampling blocks with 3D convolution layers and skip connections. Hyperparameters are the following: $t = 500$ diffusion steps; linear noise schedule with variances in the range of $\beta_1 = 10^4$ and $\beta_t = 0.02$; batch size of 8 and learning rate of 1e-4. The model is implemented using PyTorch (Paszke et al., 2019) and trained for 200 epochs on 1 GPU NVIDIA Tesla V100.
the target ones depending on the number of selected images. Selecting multiple
target images to convert images between domains can help to generate images
that represent the diversity of the target domain. In practice, the whole set of im-
ages available in the target domain could be used. This is impractical for large
datasets and might lead the model to focus on specific patterns of the target
domain if these are over-represented in the dataset. Here, we propose to use a
K-Means algorithm [17] to identify N clusters of images in the target domain
(Fig. 1(B)). Then, we extract the centroïd of these clusters and average their
latent vector for conditioning. The number of cluster N must be chosen depend-
ing on the dataset and its diversity.

**Source content preservation.** To keep the intrinsic properties of the source image [28] concatenated the source image along with random Gaussian noise to initialize the diffusion. Here, we propose to fix the initial state of the DDPM by directly using the forward diffusion process to generate a noisy version of the source image $X_t$ (Equation 1). Then, the noisy source image is iteratively denoised using the predicted noise and the reverse diffusion process (Equation 2) with an additional conditioning on the target domain.

# 3 Experiments

## 3.1 Dataset

This work was performed using data from the Human Connectome Project [31]. Written informed consent was obtained from participants and the original study was approved by the Washington University Institutional Review Board. We agreed to the Open Access Data Use Terms. We used fMRI data from the motor task of the HCP Young Adult dataset [31] release S-1200 and computed the fMRI maps for the 1,080 participants using 4 analysis pipelines that differed in terms of software package (SPM [24] or FSL [12]) and presence or absence of the derivatives of the Hemodynamic Response Function (HRF) for the first-level analysis. We built 1,000 groups of 50 randomly-sampled participants, leading to 1,000 group-level statistic maps for each pipeline. In the following, we will refer to each pipeline as ‘software-presence of hrf derivatives’. For instance, the pipeline using FSL and no HRF derivatives will be denoted by ‘fsl-0’.

The selected group-level statistic maps were resampled to a size of 48 x 56 x
48 and masked using the intersection mask of all groups. The voxel values were
normalized between -1 and 1 for each statistic maps using a min-max operation.
The 1,000 groups were split into train, valid and test with a 90/8/2 ratio and all
models were trained and evaluated on the same sets. Further investigation about
possible data leakage across groups is provided in Supplementary Materials.

## 3.2 Implementation settings

Our code is available at: https://anonymous.4open.science/r/style-transfer_diffusion-12B2. Trained models are shared as Supplementary Materials.
The auxiliary classifier used to extract class conditional features contains five 3-dimensional convolution layers with 3-dimensional batch normalization and leaky rectified linear units (ReLU) activation functions, followed by a fully connected layer. The latent features correspond to 4,096 flattened vectors injected as conditioning to the U-Net. It is trained for 150 epochs using a learning rate of 1e-4 and a batch size of 64 on 1 GPU NVIDIA Tesla V100.

## Evaluation metrics

We used two types of metrics: Pearson's correlation coefficient and Mean Squared Error (MSE) to study the adequacy of generated images to the ground truth target, and Inception Score (Salimans et al., 2016) (IS) to explore the quality and diversity of the generated images. IS combines the confidence of the class predictions (i.e. each image's label distribution $p(Y|X)$) with the variety in the output of the model (i.e. the marginal label distribution for the whole set of images $P(Y)$). As an additional evaluation criterion, we used the auxiliary classifier to classify the generated images and verify if these images were correctly classified in the target pipeline class.

## Results

### GAN frameworks

<table><thead><tr><th></th><th>fsl-1 → spm-0</th><th>spm-0 → fsl-1</th><th>fsl-1 → spm-1</th><th>fsl-1 → fsl-0</th></tr><tr><th></th><th>IS</th><th colspan="3">Mean correlations (Std. errors)</th></tr></thead><tbody><tr><th>Initial</th><td>3.69</td><td>78.2 (0.5)</td><td>78.2 (0.5)</td><td>82.8 (0.3)</td></tr><tr><th>Pix2Pix</th><td>-</td><td><strong>91.4 (0.1)</strong></td><td><strong>89.1 (0.2)</strong></td><td><strong>90.1 (0.2)</strong></td></tr><tr><th>CycleGAN</th><td>-</td><td>85.5 (0.3)</td><td>67.1 (0.4)</td><td>70.0 (0.5)</td></tr><tr><th>StarGAN</th><td>3.63</td><td>90.5 (0.4)</td><td>86.8 (0.5)</td><td>87.6 (0.5)</td></tr></tbody></table>

<table><caption>Table 2: Performance of GAN frameworks. IS means “Inception Score”. Pearson's correlation (%) and Mean Squared Error (MSE) are computed between generated and ground truth images and averaged across 20 images. Initial represents the metrics between the source image (before transfer) and the ground-truth target image. **Boldface marks the top model.**</caption><thead><tr><th></th><th>fsl-1 → spm-0</th><th>spm-0 → fsl-1</th><th>fsl-1 → spm-1</th><th>fsl-1 → fsl-0</th></tr><tr><th></th><th colspan="4">Mean MSE (Std. errors)</th></tr></thead><tbody><tr><th>Initial</th><td>0.0076 (0.0003)</td><td>0.0076 (0.0003)</td><td>0.0041 (0.0002)</td><td>0.0022 (0.0001)</td></tr><tr><th>Pix2Pix</th><td><strong>0.0027 (0.0001)</strong></td><td><strong>0.0014 (0.0001)</strong></td><td><strong>0.0025 (0.0001)</strong></td><td><strong>0.0005 (0.0)</strong></td></tr><tr><th>CycleGAN</th><td>0.0049 (0.0002)</td><td>0.0049 (0.0002)</td><td>0.0072 (0.0002)</td><td>0.0048 (0.0001)</td></tr><tr><th>StarGAN</th><td>0.0035 (0.0002)</td><td>0.002 (0.0001)</td><td>0.0035 (0.0001)</td><td>0.0017 (0.0001)</td></tr></tbody></table>

In Table 2, we show the performance of GAN frameworks for transfers between pipelines with: a different software and a different HRF (columns 1-4), a different software and the same HRF (columns 4-6) and, the same software and a different HRF (columns 6-8). Overall, using Pix2Pix (Isola et al., 2017) and StarGAN (Choi et al., 2018), the conversion of statistic maps between pipelines is successful, with increased correlations between target and generated maps compared to correlations between source and target (similar observations
The neural network used to predict the noise follows a simple U-Net architecture [27] with two downsampling and upsampling blocks with 3D convolutions layers and skip connections. The hyperparameters of the DDPMs are the following: $t = 500$ diffusion steps; linear noise schedule with variances in the range of $\beta_1 = 10^4$ and $\beta_t = 0.02$; batch size of 8 and learning rate of 1e-4. The weight $w$ used to control the conditional guidance is optimized on the validation set by comparing $w = 0, w = 0.5$ and $w = 2$ and a value of 0.5 was found to give the best results in terms of Pearson's correlation coefficient between the target ground-truth and the generated image on this set. The model is implemented using PyTorch [23] and trained for 200 epochs on 1 GPU NVIDIA Tesla V100.

The CNN used to extract class conditional features is composed of five 3D convolution layers with 3D batch normalization and leaky rectified linear units (ReLU) activation functions, followed by a fully connected layer. The latent space corresponds to a 4,096 flatten vector which is injected as conditioning to the U-Net. It is trained for 150 epochs using a learning rate of 1e-4 and a batch size of 64 on 1 GPU NVIDIA Tesla V100.

## 3.3 Evaluation metrics & competitors

We evaluate the performance using different metrics: Pearson's correlation (Corr.) in percent, Peak Signal to Noise Ratio (PSNR) and Inception Score (IS) [29], computed using the pipeline classifier described above. The first two metrics study the adequacy of generated images to the ground truth target, whereas IS explore the confidence of the conditional class predictions (quality) and the integral of the marginal probability of the predicted classes (diversity).

We compare CCDDPM to different variations of conditional DDPMs: conditioning using a one-hot encoding of the class, as in [10] and conditioning using the latent space of the classifier but only one target image selected randomly, inspired from [25]. We also implement a 3D version of the state-of-the-art starGAN [4] to compare the performance of DDPMs with GANs.

In default settings, CCDDPM uses $N = 10$ target images. We also assess the impact of the number of target images for $N = 5, 10$ and $20$. We also compare the selection process with a random sampling of target images and with a sampling based on the identification of images that are close to the source image using a K-Nearest Neighbors algorithm [19].

## 3.4 Results

In Table 1, we show the performance for 4 transfers, between two pipelines with: different HRF and different software (columns 1-4), same HRF and different software (columns 4-6) and, different HRF and same software (columns 6-8). Performance of CCDDPM are compared to the one in [10] and one inspired from [25] with classifier-conditioning and $N = 1$ target image selected randomly. Regarding the quality of the transfer to the target domain, all models succeed in changing the class identified by a pipeline classifier to the target domain. CCD-DPM outperforms the other models (including starGAN) in terms of Inception
are made with MSE, in Supplementary Tables), e.g. 91.4% for target-generated compared to 76.2% for source-target with Pix2Pix for conversion “fsl-1” to “spm-0”.

We can point out the large superiority of the supervised Pix2Pix framework compared to unsupervised alternatives. By benefiting from paired data, Pix2Pix generates images closer to the target image than the source image for all transfers. Correlations between target and generated images are close to 0.9, which is nearly perfect. On the other hand, the CycleGAN (Zhu et al., 2017) framework gives surprising results, relatively low compared to other frameworks. While it makes use of a cyclic loss, similarly to StarGAN (Choi et al., 2018), it only learns transfers between two domains. We can suppose that StarGAN leverages the data from the multiple source domains and benefits from the additional classification loss, leading to higher performance in similar settings.

<figure><img src="image_3.png" /><figcaption>Figure 1: Generated images for two transfers and different competitors: Pix2Pix (Isola et al., 2017), CycleGAN (Zhu et al., 2017) and StarGAN (Choi et al., 2018). Correlation with ground truth are indicated above each image (in percent).</figcaption></figure>

In Figure 1, we illustrate two transfers: (first row) between pipelines with different software packages and different HRF (spm-0 to fsl-1) and (second row) between pipelines with the same software package and different HRF (fsl-1 to fsl-0). We randomly selected a statistic map of the source pipeline and generated the corresponding converted map in the target pipeline. We also display the ground-truth map in the target pipeline. Maps generated using Pix2Pix are closer to the ground truth, with more similar patterns, as seen with the similarity metrics.

## DM frameworks

In Table 3, we show the performance of the DM frameworks for the same four transfers. Different frameworks are compared: one-hot conditioning from (Ho and Salimans, 2021), auxiliary classifier-conditioning with *N* = 1 target image selected randomly, inspired from (Preechakul et al., 2022), and auxiliary classifier-conditioning with *N* = 10 target images selected randomly (named *N* = 10, ∞ in the Table).
<table><caption>**Table 1.** Performance associated with four transfers. IS means "Inception Score" across all transfers. Pearson's correlation (%) and Peak Signal to Noise Ratio (PSNR) computed between generated and ground-truth target image for 20 images per transfer. Initial represents the metrics between the source image (before transfer) and the ground-truth target image. Default CCDDPM uses N=10. **Boldface marks the top model.**</caption><thead><tr><th rowspan="2"></th><th rowspan="2"></th><th colspan="2">fsl-1 → spm-0</th><th colspan="2">spm-0 → fsl-1</th><th colspan="2">fsl-1 → spm-1</th><th colspan="2">fsl-1 → fsl-0</th></tr><tr><th>IS</th><th>Corr.</th><th>PSNR</th><th>Corr.</th><th>PSNR</th><th>Corr.</th><th>PSNR</th><th>Corr.</th><th>PSNR</th></tr></thead><tbody><tr><td>Initial</td><td>3.69</td><td>76.2</td><td>78.2</td><td>76.2</td><td>78.2</td><td>82.6</td><td>81.3</td><td>91.0</td><td>83.9</td></tr><tr><td>One-hot [10]</td><td>3.66</td><td>83.8</td><td>77.2</td><td>75.0</td><td>79.4</td><td>78.7</td><td>77.7</td><td>81.1</td><td>79.5</td></tr><tr><td>N=1 [25]</td><td>3.70</td><td>85.5</td><td>79.0</td><td>77.8</td><td>80.0</td><td>79.9</td><td>78.0</td><td>82.8</td><td>80.2</td></tr><tr><td>StarGAN [4]</td><td>3.63</td><td><b>90.5</b></td><td><b>81.9</b></td><td><b>87.5</b></td><td><b>84.2</b></td><td><b>87.6</b></td><td><b>81.8</b></td><td><b>91.5</b></td><td><b>85.0</b></td></tr><tr><td>CCDDPM</td><td><b>3.86</b></td><td>86.1</td><td>79.4</td><td>79.0</td><td>80.7</td><td>81.2</td><td>78.9</td><td>84.1</td><td>80.6</td></tr><tr><td>N=5</td><td>3.86</td><td>86.4</td><td>79.8</td><td>78.7</td><td>80.6</td><td>81.2</td><td>79.4</td><td>84.5</td><td>80.9</td></tr><tr><td>N=20</td><td>3.87</td><td>86.1</td><td>79.5</td><td>79.2</td><td>80.7</td><td>81.3</td><td>79.2</td><td>83.9</td><td>80.9</td></tr><tr><td>N=5, random</td><td>3.89</td><td>86.5</td><td>79.4</td><td>79.1</td><td>80.4</td><td>82.0</td><td>79.2</td><td>84.2</td><td>80.2</td></tr><tr><td>N=10, random</td><td>3.86</td><td>86.5</td><td>79.2</td><td>79.0</td><td>80.2</td><td>81.8</td><td>79.4</td><td>84.3</td><td>80.8</td></tr><tr><td>N=20, random</td><td>3.85</td><td>86.7</td><td>79.1</td><td>79.3</td><td>80.6</td><td>81.5</td><td>79.4</td><td>84.4</td><td>80.7</td></tr><tr><td>N=10, KNN</td><td>3.75</td><td>84.9</td><td>79.3</td><td>78.7</td><td>80.0</td><td>81.6</td><td>79.1</td><td>83.6</td><td>80.7</td></tr></tbody></table>

Score. The use of a DDPM with classifier-conditioning and multiple target images seems to improve both quality and diversity of images, which might be explained by the non-determinism of sampling in DDPMs.

In terms of similarity to the ground-truth target image, CCDDPM outperforms the other DDPM models by up to 4% in correlations between target ground-truth and generated image compared to [10] for transfer spm-0 to fsl-1 and up to 3% for fsl-1 to spm-0. Even compared to the other best DDPM models in each task, CCDDPM is at least 0.6% better in terms of correlations. For the last two transfers in Table 1, our performance does not overpass the initial values between the source image and the ground-truth target image. These low performance could be explained by the difficulty of the models to learn differences between close pipelines. In Supplementary Figures, we show the performance of the pipeline classifier and compare the similarity of features.

However, the performance of starGAN [4] remain highly superior to the ones obtained with any DDPM, even for the last two transfers for which it overpasses the initial metrics. This superiority can be explained by the differences between frameworks: GAN-based methods use adversarial training and starGAN improves this by using a classifier loss and a cyclic-reconstruction loss. Moreover, GANs sampling rely on the source image directly and do not require to set an initial state, which might facilitate the source content preservation.

At the bottom of Table 1, we show the influence of the number of target images and of the selection methods. The number of images does not seem to impact the performance, correlations are very similar between N = 5, N = 10 and N = 20. Performing selection using K-Means algorithm does not seem to improve performance compared to a random selection, for any N values,
<table><thead><tr><th></th><th>fsl-1 → spm-0</th><th>spm-0 → fsl-1</th><th>fsl-1 → spm-1</th><th>fsl-1 → fsl-0</th></tr></thead><tbody><tr><th>IS</th><td colspan="4">Mean correlations (Std. errors)</td></tr><tr><th>Initial</th><td>3.69</td><td>78.2 (0.5)</td><td>78.2 (0.5)</td><td>82.8 (0.3)</td><td>92.3 (0.5)</td></tr><tr><th>One-hot</th><td>3.66</td><td>83.9 (0.7)</td><td>75.0 (0.9)</td><td>78.8 (0.8)</td><td>81.1 (0.6)</td></tr><tr><th>N=1</th><td>3.70</td><td>85.4 (0.6)</td><td>77.4 (0.8)</td><td>80.1 (0.8)</td><td>82.8 (0.8)</td></tr><tr><th>N=10, ∞</th><td><strong>3.86</strong></td><td><strong>86.1 (0.4)</strong></td><td><strong>78.9 (0.6)</strong></td><td><strong>81.5 (0.4)</strong></td><td><strong>84.1 (0.6)</strong></td></tr></tbody></table>

<table><caption>Table 3: Performance of DM frameworks. IS means “Inception Score”. Pearson’s correlation (%) and Mean Squared Error (MSE) are computed between generated and ground truth images and averaged across 20 images per transfer. Initial represents the metrics between the source image (before transfer) and the target image. **Boldface marks the top model.**</caption><thead><tr><th></th><th>fsl-1 → spm-0</th><th>spm-0 → fsl-1</th><th>fsl-1 → spm-1</th><th>fsl-1 → fsl-0</th></tr></thead><tbody><tr><th>IS</th><td colspan="4">Mean MSE (Std. errors)</td></tr><tr><th>Initial</th><td>0.0076 (0.0003)</td><td>0.0076 (0.0003)</td><td>0.0041 (0.0002)</td><td>0.0022 (0.0001)</td></tr><tr><th>One-hot</th><td>0.0097 (0.0014)</td><td>0.0048 (0.0007)</td><td>0.0088 (0.0014)</td><td>0.0043 (0.0003)</td></tr><tr><th>N=1</th><td>0.0053 (0.0003)</td><td>0.0037 (0.0003)</td><td>0.0073 (0.0009)</td><td>0.0037 (0.0003)</td></tr><tr><th>N=10, ∞</th><td><strong>0.0043 (0.0003)</strong></td><td><strong>0.0028 (0.0002)</strong></td><td><strong>0.0049 (0.0003)</strong></td><td><strong>0.0029 (0.0002)</strong></td></tr></tbody></table>

The conversion between pipelines seems more difficult than with the GAN frameworks. While all models succeed in changing the class identified by a pipeline classifier to the target domain, the success of the conversion in terms of similarity to the ground truth is variable across transfers. For instance, all DM frameworks succeed for the transfer “fsl-1” to “spm-0”, while none is successful for the transfer “fsl-1” to “fsl-0”. These low performance could be explained by the difficulty of the models to learn differences between pipelines that provide similar results (i.e. whose results display very similar activation patterns).

Using a DM with auxiliary classifier conditioning and multiple target images ($N = 10, \infty$) improves the performance compared to the alternative frameworks. Both the quality and diversity of images are increased ($IS = 3.86$). In terms of similarity to the ground truth, this framework also outperforms other DM models by up to 4% in correlations between ground-truth and generated image compared to (Ho and Salimans, 2021) for the transfer “spm-0” to “fsl-1” and up to 3% for “fsl-1” to “spm-0”.

The first row of Figure 2 illustrates a transfer between pipelines with different software packages and different HRF (“spm-0” to “fsl-1”). The second row shows a transfer between pipelines with the same software package and different HRF (“fsl-1” to “fsl-0”). DM with multiple target images generates statistic maps close to the ground truth for both transfers, representing the intrinsic properties of the map while modifying its extrinsic properties to the target domain. Using the one-hot encoding, generated statistic maps seem far from the ground truth, failing to represent the full characteristics of the target domain.

The performance of DM frameworks remains notably inferior to the ones obtained with Pix2Pix (Isola et al., 2017) or StarGAN (Choi et al., 2018). This superiority can be explained by the differences between frameworks: GAN methods use adversarial training and StarGAN improves this by using a classifier loss and a cyclic-reconstruction loss. Moreover, the sampling process of GAN relies on the source image directly and does not require setting an initial state, which might facilitate the source content preservation. However,
<figure><img src="image_5.png" /><figcaption>**Fig. 2.** Generated images for two transfer and different competitors: conditioning with one-hot encoding [10], with a classifier and N=1 [25], starGAN [4] and CCDPM. Correlation with target ground-truth are indicated below generated and source images.</figcaption></figure>

probably due to the low diversity in our dataset. However, selection using a K-Nearest Neighbors (KNN) algorithm decreases the performance from 1.6%, meaning that the diversity of target images is beneficial for a good transfer.

The first row of Figure 2 illustrates a transfer between pipelines with different HRF and different software (spm-0 to fsl-1). The second row shows a transfer between pipelines with different HRF (fsl-1 to fsl-0). CCDPM generates statistic maps close to the ground-truth for both transfer, representing the intrinsic properties of the map while modifying its extrinsic properties to the target domain. Maps generated using starGAN [4] remain closer to the target ground-truth, with more similar patterns, as stated by the similarity metrics.

# 4 Conclusion

We explore the ability to convert fMRI maps between pipelines using generative models. In particular, we provide a new DDPM unsupervised multi-domain image-to-image transition framework by conditioning on multiple target images in the latent space of a classifier. Our results show that images can be converted successfully, but with lower adequacy with the ground-truth target compared to GANs. In future work, we would like to focus on latent diffusion models to improve image quality and conservation of intrinsic properties of the image.

# 5 Acknowledgements

This work was funded by Region Bretagne (ARED MAPIS) and ANR project ANR-20-THIA-0018). Data were provided by the Human Connectome Project, WU-Minn Consortium (Principal Investigators: David Van Essen and Kamil Ugurbil; 1U54MH091657) funded by the 16 NIH Institutes and Centers that support the NIH Blueprint for Neuroscience Research; and by the McDonnell Center for Systems Neuroscience at Washington University.
<figure><img src="image_4.png" /><figcaption>Figure 2: Generated images for two transfer and different competitors: conditioning with one-hot encoding (Ho and Salimans, 2021), with a classifier-conditioning N=1 and N=10 target images randomly selected. Correlations with ground truth are indicated above generated and source images (in percent).</figcaption></figure>

we can note that Inception Scores (IS) obtained with DM frameworks are better than the ones obtained with StarGAN, which indicates that images generated by DM frameworks are more diverse. This observation is consistent with the literature (Dhariwal and Nichol, 2021) and the sampling process of DM frameworks which includes randomness.

Conclusion

In this study, we explored the potential of style transfer frameworks on the task of converting fMRI statistic maps between different pipelines. We showed that the StarGAN framework, trained on unsupervised and multi-domain data, could be easily trained and applied to generate statistic maps that maintain the intrinsic properties of brain activity while changing the style of the image. These newly generated data could be used to build valid mega-analyses on heterogeneous datasets and hence increase sample sizes in fMRI data analysis. Future works will focus on validating these assumptions by evaluating the false positives and false negatives rates of mega-analyses built with converted statistic maps.
References

1. Botvinik-Nezer, R., Holzmeister, F., Camerer, C.F., Dreber, A., Huber, J., Johannesson, M., Kirchler, M., Iwanir, R., Mumford, J.A., Adcock, R.A., Avesani, P., Baczkowski, B.M., Bajracharya, A., Bakst, L., Ball, S., Barilari, M.: Variability in the analysis of a single neuroimaging dataset by many teams. Nature **582**(7810), 84–88 (2020)

2. Carp, J.: On the plurality of (methodological) worlds: estimating the analytic flexibility of fMRI experiments. Front. Neurosci. p. 13 (2012)

3. Choi, J., Kim, S., Jeong, Y., Gwon, Y., Yoon, S.: ILVR: conditioning method for denoising diffusion probabilistic models. In: 2021 IEEE/CVF International Conference on Computer Vision, ICCV. pp. 14347–14356. IEEE (2021)

4. Choi, Y., Choi, M., Kim, M., Ha, J.W., Kim, S., Choo, J.: StarGAN: Unified Generative Adversarial Networks for Multi-domain Image-to-Image Translation. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 8789–8797. IEEE (2018)

5. Costafreda, S.: Pooling fMRI data: meta-analysis, mega-analysis and multi-center studies. Front. Neuroinform. **3** (2009)

6. Dai, X., Lei, Y., Fu, Y., Curran, W.J., Liu, T., Mao, H., Yang, X.: Multimodal MRI Synthesis Using Unified Generative Adversarial Networks. Medical physics **47**(12), 6343–6354 (2020)

7. Goodfellow, I.J., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y.: Generative adversarial nets. In: 27th International Conference on Neural Information Processing Systems - Volume 2. p. 2672–2680. NIPS’14 (2014)

8. Gorgolewski, K.J., Varoquaux, G., Rivera, G., Schwarz, Y., Ghosh, S.S., Maumet, C., Sochat, V.V., Nichols, T.E., Poldrack, R.A., Poline, J.B., Yarkoni, T., Mar-gulies, D.S.: NeuroVault.org: a web-based repository for collecting and sharing unthresholded statistical maps of the human brain. Front. Neuroinform. (2015)

9. Ho, J., Jain, A., Abbeel, P.: Denoising diffusion probabilistic models. In: 34th International Conference on Neural Information Processing Systems. NIPS’20 (2020)

10. Ho, J., Salimans, T.: Classifier-Free Diffusion Guidance. In: "Deep Generative Models and Downstream Applications" Workshop@NeurIPS’21 (2021)

11. Isola, P., Zhu, J.Y., Zhou, T., Efros, A.A.: Image-to-image translation with conditional adversarial networks. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). pp. 5967–5976 (2017)

12. Jenkinson, M., Beckmann, C.F., Behrens, T.E.J., Woolrich, M.W., Smith, S.M.: FSL. NeuroImage **62**(2), 782–790 (2012)

13. Jin, C.B., Kim, H., Liu, M., Jung, W., Joo, S., Park, E., Ahn, Y.S., Han, I.H., Lee, J.I., Cui, X.: Deep CT to MR Synthesis Using Paired and Unpaired Data. Sensors **19**(10) (2019)

14. Li, H., Yang, Y., Chang, M., Chen, S., Feng, H., Xu, Z., Li, Q., Chen, Y.: Srdiff: Single image super-resolution with diffusion probabilistic models. Neurocomputing **479**, 47–59 (2022)

15. Liu, M., Breuel, T.M., Kautz, J.: Unsupervised image-to-image translation networks. In: Advances in Neural Information Processing Systems 30 (NIPS). pp. 700–708 (2017)

16. Lyu, Q., Wang, G.: Conversion Between CT and MRI Images Using Diffusion and Score-Matching Models. arXiv (2022)
## Acknowledgments

This work was supported by Region Bretagne (ARED MAPIS) and ANR project ANR-20-THIA-0018). Data were provided by the Human Connectome Project, WU-Minn Consortium (Principal Investigators: David Van Essen and Kamil Ugurbil; 1U54MH091657) funded by the 16 NIH Institutes and Centers that support the NIH Blueprint for Neuroscience Research; and by the McDonnell Center for Systems Neuroscience at Washington University.

## Data statement

This study was performed using derived data from the HCP Young Adult (Van Essen et al., 2013b), publicly available at ConnectomeDB. Data usage requires registration and agreement to the HCP Young Adult Open Access Data Use Terms available at: (Van Essen et al., 2013a).

The HCP multi-pipeline dataset (Germani et al., 2023) is publicly available on Public-nEUro (at Rigshopitalet, 2023): https://publicneuro-catalogue.netlify.app/dataset/PN000003%20HCP%20multipipelines/V1.

## Ethics

This study was performed using derived data from the HCP Young Adult (Van Essen et al., 2013b). No experimental activity involving the human participants was made by the authors. Only shared data were used.

Written informed consent was obtained from participants and the original study was approved by the Washington University Institutional Review Board.

We agreed to the HCP Young Adult Open Access Data Use Terms available at: (Van Essen et al., 2013a).

## Code and data availability

All the scripts used to perform the study (models training, testing and performance evaluation) are available on Software Heritage: swh:1:snp:b0b52aa88bef8f4411bdd7e00a2d71715d7830bb (Germani et al., 2024a).

Derived data such as pre-trained models and computed metrics are available on Zenodo (Germani et al., 2024b).

## References

* Neurobiology Research Unit at Rigshopitalet. Public nEUro, 2023. URL https://public-neuro.github.io/index.html.

* Rotem Botvinik-Nezer and Tor D. Wager. Reproducibility in Neuroimaging Analysis: Challenges and Solutions. *Biological Psychiatry: Cognitive Neuroscience and Neuroimaging*, 8(8):780-788, 2023. ISSN 2451-9022. doi: 10.1016/j.bpsc.2022.12.006.
17. MacQueen, J.: Some methods for classification and analysis of multivariate observations. In: Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability, Volume 1: Statistics, vol. 5.1, pp. 281–298 (1967)

18. Markiewicz, C.J., Gorgolewski, K.J., Feingold, F., Blair, R., Halchenko, Y.O., Miller, E., Hardcastle, N., Wexler, J., Esteban, O., Goncavles, M., Jwa, A., Poldrack, R.: The OpenNeuro resource for sharing of neuroscience data. eLife 10, e71774 (2021)

19. Mucherino, A., Papajorgji, P.J., Pardalos, P.M.: k-nearest neighbor classification. In: Data Mining in Agriculture, pp. 83–106. Springer (2009)

20. Nichol, A.Q., Dhariwal, P.: Improved Denoising Diffusion Probabilistic Models. In: 38th International Conference on Machine Learning, pp. 8162–8171. PMLR (2021)

21. Özbey, M., Dalmaz, O., Dar, S.U.H., Bedel, H.A., Oztürk, S., Güngör, A., Çukur, T.: Unsupervised medical image translation with adversarial diffusion models. IEEE Trans. Medical Imaging 42(12), 3524–3539 (2023)

22. Pan, S., Chang, C.W., Peng, J., Zhang, J., Qiu, R.L.J., Wang, T., Roper, J., Liu, T., Mao, H., Yang, X.: Cycle-guided Denoising Diffusion Probability Model for 3D Cross-modality MRI Synthesis. arXiv (2023)

23. Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., Desmaison, A., Kopf, A., Yang, E., DeVito, Z., Raison, M., Tejani, A., Chilamkurthy, S., Steiner, B., Fang, L., Bai, J., Chintala, S.: Pytorch: An imperative style, high-performance deep learning library. In: Advances in Neural Information Processing Systems 32, pp. 8024–8035. Curran Associates, Inc. (2019)

24. Penny, W., Friston, K., Ashburner, J., Kiebel, S., Nichols, T.E.: Statistical Parametric Mapping: The Analysis of Functional Brain Images. Elsevier edn. (2011)

25. Preechakul, K., Chatthee, N., Wizadwongsa, S., Suwajanakorn, S.: Diffusion autoencoders: Toward a meaningful and decodable representation. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR. pp. 10609–10619. IEEE (2022)

26. Rolland, X., Maurel, P., Maumet, C.: Towards efficient fmri data re-use: can we run between-group analyses with datasets processed differently with spm ? In: ISBI 2022 - IEEE International Symposium on Biomedical Imaging. pp. 1–4 (2022)

27. Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention (MICCAI). pp. 234–241 (2015)

28. Saharia, C., Chan, W., Chang, H., Lee, C.A., Ho, J., Salimans, T., Fleet, D.J., Norouzi, M.: Palette: Image-to-image diffusion models. In: SIGGRAPH '22: Special Interest Group on Computer Graphics and Interactive Techniques Conference. pp. 15:1–15:10. ACM (2022)

29. Salimans, T., Goodfellow, I., Zaremba, W., Cheung, V., Radford, A., Chen, X., Chen, X.: Improved Techniques for Training GANs. In: Lee, D., Sugiyama, M., Luxburg, U., Guyon, I., Garnett, R. (eds.) Advances in Neural Information Processing Systems. vol. 29. Curran Associates, Inc. (2016)

30. Sasaki, H., Willcocks, C.G., Breckon, T.P.: UNIT-DDPM: UNpaired Image Translation with Denoising Diffusion Probabilistic Models. arXiv (2021)

31. Van Essen, D.C., Ugurbil, K., Auerbach, E., Barch, D., Behrens, T.E.J., WU-Minn HCP Consortium: The Human Connectome Project: a data acquisition perspective. NeuroImage 62(4), 2222–2231 (2012)

32. Zhu, J., Park, T., Isola, P., Efros, A.A.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: IEEE International Conference on Computer Vision, ICCV. pp. 2242–2251. IEEE Computer Society (2017)
Rotem Botvinik-Nezer, Felix Holzmeister, Colin F. Camerer, Anna Dreber, Juergen Huber, Magnus Johannesson, Michael Kirchler, Roni Iwanir, Jeanette A. Mumford, and al. Variability in the analysis of a single neuroimaging dataset by many teams. *Nature*, 582 (7810):84–88, 2020. ISSN 1476-4687. doi: 10.1038/s41586-020-2314-9.

Katherine S. Button, John P. A. Ioannidis, Claire Mokrysz, Brian A. Nosek, Jonathan Flint, Emma S. J. Robinson, and Marcus R. Munafò. Power failure: why small sample size undermines the reliability of neuroscience. *Nature Reviews Neuroscience*, 14(5):365–376, 2013. doi: 10.1038/nrn3475.

Joshua Carp. On the Plurality of (Methodological) Worlds: Estimating the Analytic Flexibility of fMRI Experiments. *Frontiers in Neuroscience*, 6, 2012. ISSN 1662-453X. doi: 10.3389/fnins.2012.00149.

Jooyoung Choi, Sungwon Kim, Yonghyun Jeong, Youngjune Gwon, and Sungroh Yoon. ILVR: conditioning method for denoising diffusion probabilistic models. In *2021 IEEE/CVF International Conference on Computer Vision, ICCV*, pages 14347–14356. IEEE, 2021.

Yunjey Choi, Minje Choi, Munyoung Kim, Jung-Woo Ha, Sunghun Kim, and Jaegul Choo. StarGAN: Unified Generative Adversarial Networks for Multi-domain Image-to-Image Translation. In *2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)*, pages 8789–8797. IEEE, 2018.

Sergi G. Costafreda. Pooling fMRI Data: Meta-Analysis, Mega-Analysis and Multi-Center Studies. *Frontiers in Neuroinformatics*, 3:33, 2009. ISSN 1662-5196. doi: 10.3389/neuro. 11.033.2009.

Prafulla Dhariwal and Alexander Quinn Nichol. Diffusion models beat GANs on image synthesis. In A. Beygelzimer, Y. Dauphin, P. Liang, and J. Wortman Vaughan, editors, *Advances in Neural Information Processing Systems*, 2021.

Leon A. Gatys, Alexander S. Ecker, and Matthias Bethge. Image Style Transfer Using Convolutional Neural Networks. In *2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)*, pages 2414–2423, 2016. doi: 10.1109/CVPR.2016.265.

Elodie Germani, Elisa Fromont, Pierre Maurel, and Camille Maumet. The hcp multi-pipeline dataset: an opportunity to investigate analytical variability in fmri data analysis, 2023. URL https://arxiv.org/abs/2312.14493.

Elodie Germani, Camille Maumet, and Elisa Fromont. Software heritage archive for the github repository "style_transfer_diffusion", 2024a. URL https://archive.softwareheritage.org/swh:1:snp:b0b52aa88bef8f4411bdd7e00a2d71715d7830bb; origin=https://github.com/elodiegermani/style-transfer_diffusion.

Elodie Germani, Camille Maumet, and Elisa Fromont. Mitigating analytical variability in fMRI with style transfer - Supplementary Materials, September 2024b. URL https://doi.org/10.5281/zenodo.13748563.
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative Adversarial Nets. In Z. Ghahramani, M. Welling, C. Cortes, N. Lawrence, and K. Q. Weinberger, editors, *Advances in Neural Information Processing Systems*, volume 27. Curran Associates, Inc., 2014.

Krzysztof J. Gorgolewski, Gael Varoquaux, Gabriel Rivera, Yannick Schwarz, Satrajit S. Ghosh, Camille Maumet, Vanessa V. Sochat, Thomas E. Nichols, Russell A. Poldrack, Jean-Baptiste Poline, Tal Yarkoni, and Daniel S. Margulies. NeuroVault.org: a web-based repository for collecting and sharing unthresholded statistical maps of the human brain. *Front. Neuroinform.*, 2015.

Jonathan Ho and Tim Salimans. Classifier-Free Diffusion Guidance. In *"Deep Generative Models and Downstream Applications" Workshop@NeurIPS'21*, 2021.

Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising Diffusion Probabilistic Models. In H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, and H. Lin, editors, *Advances in Neural Information Processing Systems*, volume 33, pages 6840–6851. Curran Associates, Inc., 2020.

Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A. Efros. Image-to-image translation with conditional adversarial networks. In *2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)*, pages 5967–5976, 2017.

Mark Jenkinson, Christian F. Beckmann, Timothy E. J. Behrens, Mark W. Woolrich, and Stephen M. Smith. FSL. *NeuroImage*, 62(2):782–790, 2012. ISSN 1095-9572(Electronic),1053-8119(Print).

Ming-Yu Liu, Thomas M. Breuel, and Jan Kautz. Unsupervised image-to-image translation networks. In *Advances in Neural Information Processing Systems 30 (NIPS)*, pages 700–708, 2017.

Christopher J Markiewicz, Krzysztof J Gorgolewski, Franklin Feingold, Ross Blair, Yaroslav O Halchenko, Eric Miller, Nell Hardcastle, Joe Wexler, Oscar Esteban, Mathias Goncavles, Anita Jwa, and Russell Poldrack. The OpenNeuro resource for sharing of neuroscience data. *eLife*, 10:e71774, 2021. ISSN 2050-084X.

Shaoyan Pan, Chih-Wei Chang, Junbo Peng, Jiahan Zhang, Richard L. J. Qiu, Tonghe Wang, Justin Roper, Tian Liu, Hui Mao, and Xiaofeng Yang. Cycle-guided Denoising Diffusion Probability Model for 3D Cross-modality MRI Synthesis. *arXiv*, 2023.

Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high-performance deep learning library. *Advances in Neural Information Processing Systems 32*, pages 8024–8035, 2019. doi: 10.48550/arXiv.1912.01703.

W.D. Penny, K. Friston, J. Ashburner, S.J. Kiebel, and Thomas E. Nichols. *Statistical Parametric Mapping: The Analysis of Functional Brain Images*. Elsevier edition, 2011.
Ana Luísa Pinho, Alexis Amadon, Torsten Ruest, Murielle Fabre, Elvis Dohmatob, Isabelle Denghien, Chantal Ginisty, Séverine Becuwe-Desmidt, Séverine Roger, Laurence Laurier, Véronique Joly-Testault, Gaëlle Médiouni-Cloarec, Christine Doublé, Bernadette Martins, Philippe Pinel, Evelyn Eger, Gaël Varoquaux, Christophe Pallier, Stanislas Dehaene, Lucie Hertz-Pannier, and Bertrand Thirion. Individual Brain Charting, a high-resolution fMRI dataset for cognitive mapping. *Scientific Data*, 5(1):180105, 2018. ISSN 2052-4463. doi: 10.1038/sdata.2018.105.

Russell A. Poldrack, Chris I. Baker, Joke Durnez, Krzysztof J. Gorgolewski, Paul M. Matthews, Marcus R. Munafò, Thomas E. Nichols, Jean-Baptiste Poline, Edward Vul, and Tal Yarkoni. Scanning the horizon: towards transparent and reproducible neuroimaging research. *Nature Reviews Neuroscience*, 2017. doi: 10.1038/nrn.2016.167.

Jean-Baptiste Poline, Janis L. Breeze, Satrajit S. Ghosh, Krzysztof Gorgolewski, Yaroslav O. Halchenko, Michael Hanke, Karl G. Helmer, Daniel S. Marcus, Russell A. Poldrack, Yannick Schwartz, John Ashburner, and David N. Kennedy. Data sharing in neuroimaging research. *Frontiers in Neuroinformatics*, 6, 2012. ISSN 1662-5196. doi: 10.3389/fninf.2012.00009.

Konpat Preechakul, Nattanat Chatthee, Suttisak Wizadwongsa, and Supasorn Suwanjanakorn. Diffusion autoencoders: Toward a meaningful and decodable representation. In *IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR*, pages 10609–10619. IEEE, 2022.

Xavier Rolland, Pierre Maurel, and Camille Maumet. Towards efficient fmri data re-use: can we run between-group analyses with datasets processed differently with spm ? In *ISBI 2022 - IEEE International Symposium on Biomedical Imaging*, pages 1–4, 2022.

Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-Net: Convolutional Networks for Biomedical Image Segmentation. In Nassir Navab, Joachim Hornegger, William M. Wells, and Alejandro F. Frangi, editors, *Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015*, pages 234–241, Cham, 2015. Springer International Publishing. ISBN 978-3-319-24574-4. doi: 10.1007/978-3-319-24574-4_28.

Chitwan Saharia, William Chan, Huiwen Chang, Chris A. Lee, Jonathan Ho, Tim Salimans, David J. Fleet, and Mohammad Norouzi. Palette: Image-to-image diffusion models. In *SIGGRAPH '22: Special Interest Group on Computer Graphics and Interactive Techniques Conference*, pages 15:1–15:10. ACM, 2022.

Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, Xi Chen, and Xi Chen. Improved Techniques for Training GANs. In D. Lee, M. Sugiyama, U. Luxburg, I. Guyon, and R. Garnett, editors, *Advances in Neural Information Processing Systems*, volume 29. Curran Associates, Inc., 2016.

Hiroshi Sasaki, Chris G. Willcocks, and Toby P. Breckon. UNIT-DDPM: UNpaired Image Translation with Denoising Diffusion Probabilistic Models. *arXiv*, 2021.

Cathie Sudlow, John Gallacher, Naomi Allen, Valerie Beral, Paul Burton, John Danesh, Paul Downey, Paul Elliott, Jane Green, Martin Landray, Bette Liu, Paul Matthews,
Giok Ong, Jill Pell, Alan Silman, Alan Young, Tim Sprosen, Tim Peakman, and Rory
Collins. UK biobank: An open access resource for identifying the causes of a wide range
of complex diseases of middle and old age. *PLOS Medicine*, 12, 2015.

David C. Van Essen, Stephen M. Smith, Deanna M. Barch, Timothy E.J. Behrens, Essa Yacoub, and Kamil Ugurbil. Human connectome project: Data usage agreement. https://www.humanconnectome.org/study/hcp-young-adult/document/wu-minn-hcp-consortium-open-access-data-use-terms, 2013a.

David C. Van Essen, Stephen M. Smith, Deanna M. Barch, Timothy E.J. Behrens, Essa Yacoub, and Kamil Ugurbil. The WU-Minn Human Connectome Project: An overview. *NeuroImage*, 80:62-79, 2013b. ISSN 1053-8119. doi: 10.1016/j.neuroimage.2013.05.041.

Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A. Efros. Unpaired image-to-image translation using cycle-consistent adversarial networks. In *IEEE International Conference on Computer Vision, ICCV*, pages 2242-2251. IEEE Computer Society, 2017.
# Supplementary materials

## Generalizability study

Ideally, researchers should be able to re-use a model trained to convert statistic maps of a task for another one. Thus, we test the generalizability of the frameworks trained to convert statistic maps from a particular task to statistic maps of other tasks. Here, we explore the generalizability of StarGAN trained on maps of *right-hand* to maps of *right-foot* and *left-hand*.

<table><thead><tr><th></th><th>fsl-1 → spm-0</th><th>spm-0 → fsl-1</th><th>fsl-1 → spm-1</th><th>fsl-1 → fsl-0</th></tr></thead><tbody><tr><td colspan="5"><strong>Converting right-foot statistic maps with:</strong></td></tr><tr><td><em>Initial</em></td><td>86.2</td><td>86.3</td><td>85.8</td><td>95.9</td></tr><tr><td>Trained on <em>right-foot</em></td><td>90.5</td><td>88.8</td><td>88.9</td><td>93.1</td></tr><tr><td>Trained on <em>right-hand</em></td><td>71.2</td><td>71.0</td><td>63.0</td><td>82.2</td></tr><tr><td colspan="5"><strong>Converting left-hand statistic maps with:</strong></td></tr><tr><td><em>Initial</em></td><td>82.3</td><td>82.3</td><td>85.9</td><td>92.9</td></tr><tr><td>Trained on <em>left-hand</em></td><td>88.4</td><td>85.4</td><td>86.8</td><td>85.8</td></tr><tr><td>Trained on <em>right-hand</em></td><td>73.2</td><td>74.5</td><td>69.5</td><td>75.2</td></tr></tbody></table>

Supplementary Table 1 - Robustness to distribution shifts (i.e. trained and evaluated with statistic maps from different tasks) of StarGAN. Pearson's correlation (%) is computed between generated and ground truth and averaged across 20 images per transfer. Initial represents the metrics between the source image (before transfer) and the target image.

Supplementary Table 1 shows the performance of the StarGAN framework trained on *right-hand* statistic maps when applying it to statistic maps of *right-foot* and *left-hand*. We also show the performance of StarGAN trained directly on *right-foot* and *left-hand* for comparison. Our results show that the StarGAN framework does not behave positively when applied to a task different from the task used in the training data: the generated images seem further from the target images (e.g. 71.2% for the first transfer between “fsl-1” to “spm-0”) than the source image was from the target (e.g. 86.2% in this case). We can see a large performance drop between the framework trained on statistic maps from the task compared to the one trained on maps from another task. Similar observations can be made for generalizability to closer tasks (here, frameworks trained on *right-hand* evaluated on *left-hand*). These results make us suppose that the mapping from one pipeline to another is different between tasks and thus, if the same model can be used for different subjects on the same task, different models should be trained to transfer statistic maps from tasks unseen during training.
Supplementary figures

<table><caption>**Supplementary Table S1.** Mean correlations between feature maps learned at each layer for each pair of pipelines. Features are close for pipelines sharing the same software at Layer 4, which might explain the difficulty to rely on these features to perform transfer.</caption><thead><tr><th>Pipelines</th><th>Layer 1</th><th>Layer 2</th><th>Layer 3</th><th>Layer 4</th></tr></thead><tbody><tr><td colspan="5">Same software, different parameters</td></tr><tr><td>fsl-5-0-0 / fsl-5-0-1</td><td>86.5</td><td>91.4</td><td>95.4</td><td>99.2</td></tr><tr><td>spm-5-0-0 / spm-5-0-1</td><td>86.5</td><td>90.9</td><td>94.2</td><td>98.4</td></tr><tr><td colspan="5">Same parameters, different software</td></tr><tr><td>fsl-5-0-0 spm-5-0-0</td><td>88.8</td><td>88.2</td><td>93.6</td><td>98.2</td></tr><tr><td>fsl-5-0-1 spm-5-0-1</td><td>84.8</td><td>85.8</td><td>92.4</td><td>98.0</td></tr><tr><td colspan="5">Different software, different parameters</td></tr><tr><td>fsl-5-0-0 spm-5-0-1</td><td>74.5</td><td>81.0</td><td>88.7</td><td>97.1</td></tr><tr><td>fsl-5-0-1 spm-5-0-0</td><td>74.8</td><td>77.7</td><td>88.2</td><td>97.3</td></tr></tbody></table>

<figure><img src="image_5.png" /><figcaption>**Supplementary figure S1.** Original mean statistic maps (column 1) and mean feature maps across groups learned by the pipeline classifier for the first 4 convolutional layers for the different classes. Pipelines with the same software show similar feature maps at Layer 2 and 3.</figcaption></figure>
<figure><img src="image_6.png" /><figcaption>**Supplementary Figure S2.** Histogram of the number of shared participants between two groups for each pair of groups across the whole dataset. While shared participants between groups can impact our results by slightly over-estimating our performance, the impact is likely to be low due to the small number of shared participants (2.3 on average). In addition, this has no impact the conclusions of our study on the comparison of performance between different models as all models are trained and evaluated on the same sets of groups.</figcaption></figure>