<figure><img src="image_1.png" /></figure>

HAL
open science

# Towards reproducible neuroimaging across different analysis pipelines

Camille Maumet

<figure><img src="image_2.png" /></figure>

## To cite this version:

Camille Maumet. Towards reproducible neuroimaging across different analysis pipelines. OHBM 2025 - Annual Meeting of the Organization of Human Brain Mapping, Jun 2025, Brisbane (Online), Australia. 2025. [https://doi.org/10.51220/ha-05122041](https://doi.org/10.51220/ha-05122041)

HAL Id: hal-05122041

https://inria.hal.science/hal-05122041v1

Submitted on 20 Jun 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_3.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_4.png" /></figure>

# Towards reproducible neuroimaging across different analysis pipelines

Camille Maumet

Inria, Univ Rennes, Inserm, CNRS, IRISA
# Reproducible research

<figure><img src="image_5.png" /></figure>

Data

<figure><img src="image_6.png" /></figure>

Analysis

<figure><img src="image_7.png" /></figure>

<figure><img src="image_8.png" /></figure>

Results
# Reproducible research

## Original study

<figure><img src="image_9.png" /></figure>
# Reproducible research

## Original study

<figure><img src="image_10.png" /></figure>

<figure><img src="image_11.png" /></figure>
# Reproducible research

## Original study

<figure><img src="image_12.png" /></figure>

<figure><img src="image_13.png" /></figure>
# Reproducible research

## Original study

<figure><img src="image_14.png" /></figure>

<figure><img src="image_15.png" /></figure>
# Reproducible research

## Original study

<figure><img src="image_16.png" /></figure>

Hum... I can't reproduce
the original results...

<figure><img src="image_17.png" /></figure>
# Reproducible research

## Original study

<figure><img src="image_18.png" /></figure>

Data

Results

Did we
understand the
method correctly?

... a different
computing
platform?

... or is this due to
my use of a
different dataset?

... something
else!?

Which release of
SPM was used?

Which registration
algorithm was
selected?

How was the brain
mask computed?

Hum... I can't reproduce
the original results...

<figure><img src="image_19.png" /></figure>

≠ Data

≠ Analysis

<figure><img src="image_20.png" /></figure>

Results
# Reproducible research

<figure><img src="image_21.png" /><figcaption>Reproductions</figcaption></figure>

<figure><img src="image_22.png" /></figure>

<figure><img src="image_23.png" /></figure>
# Reproducible research

<figure><img src="image_24.png" /><figcaption>Reproductions</figcaption></figure>

<figure><img src="image_25.png" /></figure>

<figure><img src="image_26.png" /></figure>
# Reproducible research

<figure><img src="image_27.png" /></figure>

<figure><img src="image_28.png" /></figure>
# Reproducible research

<figure><img src="image_29.png" /></figure>

<figure><img src="image_30.png" /></figure>

<figure><img src="image_31.png" /></figure>

<figure><img src="image_32.png" /></figure>

<figure><img src="image_33.png" /></figure>

<figure><img src="image_34.png" /></figure>

<figure><img src="image_35.png" /></figure>
<figure><img src="image_36.png" /></figure>

Different analysis...
different results?
# A multiplicity of analysis pipelines

<figure><img src="image_37.png" /></figure>
# A multiplicity of analysis pipelines

<figure><img src="image_38.png" /></figure>

Input data

Data preparation

<figure><img src="image_39.png" /></figure>

Derived
data

Statistics

<figure><img src="image_40.png" /></figure>

Results
# A multiplicity of analysis pipelines

<figure><img src="image_41.png" /></figure>

Input data

<figure><img src="image_42.png" /></figure>

<figure><img src="image_43.png" /></figure>

Results
# A multiplicity of analysis pipelines

<figure><img src="image_44.png" /></figure>

Input data

<figure><img src="image_45.png" /></figure>

Motion
correction

Segmentation

<figure><img src="image_46.png" /></figure>

Results
# A multiplicity of analysis pipelines

<figure><img src="image_47.png" /></figure>

Input data

<figure><img src="image_48.png" /></figure>

<figure><img src="image_49.png" /></figure>

Results
# A multiplicity of analysis pipelines

<figure><img src="image_50.png" /></figure>

Input data

<figure><img src="image_51.png" /></figure>

<figure><img src="image_52.png" /></figure>

Results
# A multiplicity of analysis pipelines

<figure><img src="image_53.png" /></figure>

Input data

<figure><img src="image_54.png" /></figure>

<figure><img src="image_55.png" /></figure>
# A multiplicity of analysis pipelines

<figure><img src="image_56.png" /></figure>

Input data

<figure><img src="image_57.png" /></figure>

<figure><img src="image_58.png" /></figure>

Results

<figure><img src="image_59.png" /></figure>

≠ algorithm

<figure><img src="image_60.png" /></figure>

≠ software
# A multiplicity of analysis pipelines

<figure><img src="image_61.png" /></figure>

Input data

<figure><img src="image_62.png" /></figure>

<figure><img src="image_63.png" /></figure>

Results

<figure><img src="image_64.png" /></figure>

<figure><img src="image_65.png" /></figure>

<figure><img src="image_66.png" /></figure>
# A multiplicity of analysis pipelines

<figure><img src="image_67.png" /></figure>

Input data

<figure><img src="image_68.png" /></figure>

<figure><img src="image_69.png" /></figure>
# A multiplicity of analysis pipelines

<figure><img src="image_70.png" /></figure>

Input data

<figure><img src="image_71.png" /></figure>

Results

<figure><img src="image_72.png" /></figure>

≠ algorithm

<figure><img src="image_73.png" /></figure>

≠ parameters

<figure><img src="image_74.png" /></figure>

≠ software

<figure><img src="image_75.png" /></figure>

≠ environment

<figure><img src="image_76.png" /></figure>

≠ software version
# A multiplicity of analysis pipelines

<figure><img src="image_77.png" /></figure>

Input data

<figure><img src="image_78.png" /></figure>

Results

<figure><img src="image_79.png" /></figure>

... and many more...

<figure><img src="image_80.png" /></figure>

≠ algorithm

<figure><img src="image_81.png" /></figure>

≠ software

<figure><img src="image_82.png" /></figure>

≠ software version

<figure><img src="image_83.png" /></figure>

≠ parameters

<figure><img src="image_84.png" /></figure>

≠ environment

A family of acceptable pipelines
over $10^{30}$ combinations...
# On the impact of analytical variability

## Studied across:

* End-to-end pipelines in task-fMRI [Carp, 2012] [Botvinik-Nezer, 2020] in PET [Nørgaard 2020] in EEG [Trübutschek 2024]

* Different operating systems [Glatard 2015]

* Different neuroimaging software [Bowring 2019]

* Different versions [Gronenschild 2012]

Why ?

Numerical
uncertainty
(e.g. ≠ OS)

Different
implem.

Unreliable
results

Assumption
violations

Better fit
to the data
<figure><img src="image_85.png" /></figure>

# Understanding analytical variability
# Exploring the analytical space

<figure><img src="image_86.png" /><figcaption>Elisa
Fromont</figcaption></figure>

<figure><img src="image_87.png" /><figcaption>fMRI statistic
maps from
different
analyses</figcaption></figure>

<figure><img src="image_88.png" /></figure>

<figure><img src="image_89.png" /></figure>

24 analyses

* **1 dataset** : HCP Young Adult [Van Essen et al., 2013]

* **5 motor fMRI tasks**

<figure><img src="image_90.png" /></figure>

* **1000 groups** of 50 participants (among 1080)

**Objective:** Study the variability observed in the results obtained with different analyses.
# Exploring the analytical space

<figure><img src="image_91.png" /><figcaption>Elodie
Germani</figcaption></figure>

<figure><img src="image_92.png" /><figcaption>Elisa
Fromont</figcaption></figure>

<figure><img src="image_93.png" /></figure>

[Germani et al, ICIP 2024]

✓ Patterns reproduced across participants

? Patterns variable across fMRI tasks

Shared data on
PublicnEUro (under GDPR)

<figure><img src="image_94.png" /></figure>

HCP multi pipeline

DOI: 10.70883/GTKK1541
<figure><img src="image_95.png" /></figure>

# Towards large-scale exploration of the pipeline space
# Deriving a consensus across multiple pipelines

<figure><img src="image_96.png" /><figcaption>Jérémy Lefort-Besnard</figcaption></figure>

<figure><img src="image_97.png" /><figcaption>Tom Nichols
Oxford Uni. (UK)</figcaption></figure>

## Meta-analysis

<figure><img src="image_98.png" /></figure>

[Lefort-Besnard et al, Imaging Neuroscience 2025]
# Deriving a consensus across multiple pipelines

<figure><img src="image_99.png" /><figcaption>Jérémy Lefort-Besnard</figcaption></figure>

<figure><img src="image_100.png" /><figcaption>Tom Nichols
Oxford Uni. (UK)</figcaption></figure>

<figure><img src="image_101.png" /><figcaption>## Multiverse analysis</figcaption></figure>

<figure><img src="image_102.png" /></figure>

[Lefort-Besnard et al, Imaging Neuroscience 2025]
# Deriving a consensus across multiple pipelines

<figure><img src="image_103.png" /><figcaption>Jérémy Lefort-Besnard</figcaption></figure>

<figure><img src="image_104.png" /><figcaption>Tom Nichols
Oxford Uni. (UK)</figcaption></figure>

Meta-analysis

<figure><img src="image_105.png" /></figure>

Multiverse analysis

<figure><img src="image_106.png" /></figure>

**Objective:** Propose a meta-analysis adapted for multiverse studies

[Lefort-Besnard et al, Imaging Neuroscience 2025]
# Deriving a consensus across multiple pipelines

<figure><img src="image_107.png" /><figcaption>Jérémy Lefort-Besnard</figcaption></figure>

<figure><img src="image_108.png" /><figcaption>Tom Nichols
Oxford Uni. (UK)</figcaption></figure>

## Simulations

### Meta-Analyse

Stouffer

$$Z_j^S = \frac{\bar{Y}_j}{\sqrt{1/K}}$$

Standardized mean

### Multiverse

SDMA Stouffer

$$Z_j^{SS} = \frac{\bar{Y}_j}{\sqrt{\frac{1}{K} \frac{Q1}{Q} K_k / K^2}}$$

Standardized mean w/ dependance

SDMA GLS

$$Z_j^{SG} = \frac{1^T Q^{-1} Y_j}{\sqrt{1^T Q^{-1} 1}}$$

Optimal average, standardized

### Independence

<figure><img src="image_109.png" /></figure>

Valid 4.7%

Valid 4.7%

Valid 4.4%

### Dependance

<figure><img src="image_110.png" /></figure>

Invalid 30%

Valid 4.9%

Valid 4.9%

P<0.05 (uncorrected)

[Lefort-Besnard et al, Imaging Neuroscience 2025]
# Deriving a consensus across multiple pipelines

<figure><img src="image_111.png" /><figcaption>Jérémy Lefort-Besnard</figcaption></figure>

<figure><img src="image_112.png" /><figcaption>Tom Nichols
Oxford Uni. (UK)</figcaption></figure>

## Real data

## Multiverse

SDMA
Stouffer

SDMA GLS

$$Z_j^{\text{SG}} = \frac{1^{\top} Q^{-1} Y_j}{\sqrt{1^{\top} Q^{-1} 1}}$$

Standardized
mean w/
dependance

Optimal average,
standardized

<figure><img src="image_113.png" /></figure>

P<0.05 (uncorrected)

[Lefort-Besnard Imaging Neuroscience 2025]
<figure><img src="image_114.png" /></figure>

Towards reproducible
neuroimaging across
different analysis pipelines

Camille Maumet

Thank you!

Acknowledgments

<figure><img src="image_115.png" /></figure>

<figure><img src="image_116.png" /></figure>

<figure><img src="image_117.png" /></figure>

PROGRAMME
DE RECHERCHE

SANTE
NUMERIQUE

Data were provided in part by the Human Connectome Project, WU-Minn Consortium (Principal Investigators: David Van Essen and Kamil Ugurbil; 1U54MH091657) funded by the 16 NIH Institutes and Centers that support the NIH Blueprint for Neuroscience Research; and by the McDonnell Center for Systems Neuroscience at Washington University.

<figure><img src="image_118.png" /></figure>

<figure><img src="image_119.png" /></figure>