# Towards reproducible neuroimaging across different analysis pipelines

Camille Maumet

<figure><img src="image_1.png" /></figure>

## To cite this version:

Camille Maumet. Towards reproducible neuroimaging across different analysis pipelines. INT 2024 - Seminar at Institut de Neurosciences de la Timone, Oct 2024, Marseille, France. pp.1-72, 2024. [hal-04736842](https://hal-INRA.hal.sciences.inra-04736842v1)

HAL Id: hal-04736842

https://inria.hal.science/hal-04736842v1

Submitted on 15 Oct 2024

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_3.png" /></figure>

<figure><img src="image_4.png" /></figure>

# Towards reproducible neuroimaging

across different analysis pipelines

Camille Maumet

Inria, Univ Rennes, Inserm, CNRS, IRISA
# A crisis in experimental research

The reproducibility crisis has led to reduced
confidence in research findings

Low reproduction rates in many fields

Cancer research: <11%

Psychology: 36%

Medicine: 44%

(Begley & Ellis 2012) (Open Science Collab 2016) (Ioannidis 2005)
# A crisis in experimental research

The reproducibility crisis has led to reduced confidence in research findings

## Low reproduction rates in many fields

- Cancer research: <11%

- Psychology: 36%

- Medicine: 44%

(Begley & Ellis 2012) (Open Science Collab 2016) (Ioannidis 2005)

## Wasted money & effort for research

<figure><img src="image_5.png" /></figure>

## Delayed translation into clinical practice

<figure><img src="image_6.png" /></figure>

## Reduced trust in science

<figure><img src="image_7.png" /></figure>
# Reproducible research

<figure><img src="image_8.png" /></figure>

Data

<figure><img src="image_9.png" /></figure>

Analysis

<figure><img src="image_10.png" /></figure>

<figure><img src="image_11.png" /></figure>

Results
# Reproducible research

## Original study

<figure><img src="image_12.png" /></figure>
# Reproducible research

## Original study

<figure><img src="image_13.png" /></figure>

<figure><img src="image_14.png" /></figure>
# Reproducible research

## Original study

<figure><img src="image_15.png" /></figure>

<figure><img src="image_16.png" /></figure>
# Reproducible research

## Original study

<figure><img src="image_17.png" /></figure>

<figure><img src="image_18.png" /></figure>
# Reproducible research

## Original study

<figure><img src="image_19.png" /></figure>

Data

Analysis

Results

Which release of SPM was used?

Which registration algorithm was selected?

... or is this due to my use of a different dataset?

How was the brain mask computed?

Did we understand the method correctly?

... a different computing platform?

... something else!?

Hum... **we can't reproduce** the original results...

<figure><img src="image_20.png" /></figure>

≠ Data

≠ Analysis

<figure><img src="image_21.png" /></figure>

Results
# Reproducible research

<figure><img src="image_22.png" /><figcaption>Original study</figcaption></figure>

<figure><img src="image_23.png" /></figure>
# Reproducible research

## Original study

<figure><img src="image_24.png" /></figure>

<figure><img src="image_25.png" /></figure>

<figure><img src="image_26.png" /></figure>
# Reproducible research

<figure><img src="image_27.png" /></figure>
# Reproducible research

Original study

<figure><img src="image_28.png" /></figure>

same data

<figure><img src="image_29.png" /></figure>

different data

<figure><img src="image_30.png" /></figure>

<figure><img src="image_31.png" /></figure>

<figure><img src="image_32.png" /></figure>

<figure><img src="image_33.png" /></figure>

same analysis different analysis

<figure><img src="image_34.png" /></figure>

<figure><img src="image_35.png" /></figure>
# Reproducible research

## Original study

<figure><img src="image_36.png" /></figure>

## Reproductions

<figure><img src="image_37.png" /></figure>

<figure><img src="image_38.png" /></figure>

<figure><img src="image_39.png" /></figure>

<figure><img src="image_40.png" /></figure>

<figure><img src="image_41.png" /></figure>
<figure><img src="image_42.png" /></figure>

<figure><img src="image_43.png" /></figure>

On our way to fix
irreproducibility...
# Reproducible research

## Original study

<figure><img src="image_44.png" /></figure>

## Reproducibility

<figure><img src="image_45.png" /></figure>

<figure><img src="image_46.png" /></figure>

<figure><img src="image_47.png" /></figure>

<figure><img src="image_48.png" /></figure>

<figure><img src="image_49.png" /></figure>
# Reproducible research

## Original study

<figure><img src="image_50.png" /></figure>

## Reproducibility

<figure><img src="image_51.png" /></figure>
# Reproducible research

## Original study

<figure><img src="image_52.png" /></figure>

## Reproducibility

<figure><img src="image_53.png" /></figure>

<figure><img src="image_54.png" /></figure>

<figure><img src="image_55.png" /></figure>

<figure><img src="image_56.png" /></figure>

<figure><img src="image_57.png" /></figure>
# Reproducible research

<figure><img src="image_58.png" /></figure>
<figure><img src="image_59.png" /></figure>

# Different pipelines... different results?
# A multiplicity of analysis pipelines

<figure><img src="image_60.png" /></figure>
# A multiplicity of analysis pipelines

<figure><img src="image_61.png" /></figure>
# A multiplicity of analysis pipelines

<figure><img src="image_62.png" /></figure>

Input data

<figure><img src="image_63.png" /></figure>

<figure><img src="image_64.png" /></figure>

Results
# A multiplicity of analysis pipelines

<figure><img src="image_65.png" /></figure>

Input data

<figure><img src="image_66.png" /></figure>

<figure><img src="image_67.png" /></figure>

Results
# A multiplicity of analysis pipelines

<figure><img src="image_68.png" /></figure>

Input data

<figure><img src="image_69.png" /></figure>

<figure><img src="image_70.png" /></figure>

Results

<figure><img src="image_71.png" /></figure>

≠ algorithm
# A multiplicity of analysis pipelines

<figure><img src="image_72.png" /></figure>

Input data

<figure><img src="image_73.png" /></figure>

<figure><img src="image_74.png" /></figure>

Results

<figure><img src="image_75.png" /></figure>

<figure><img src="image_76.png" /></figure>
# A multiplicity of analysis pipelines

<figure><img src="image_77.png" /></figure>

Input data

<figure><img src="image_78.png" /></figure>

<figure><img src="image_79.png" /></figure>

Results

<figure><img src="image_80.png" /></figure>

<figure><img src="image_81.png" /></figure>

<figure><img src="image_82.png" /></figure>
# A multiplicity of analysis pipelines

<figure><img src="image_83.png" /></figure>

Input data

<figure><img src="image_84.png" /></figure>

<figure><img src="image_85.png" /></figure>

Results

<figure><img src="image_86.png" /></figure>

≠ algorithm

<figure><img src="image_87.png" /></figure>

<figure><img src="image_88.png" /></figure>

≠ parameters

<figure><img src="image_89.png" /></figure>

<figure><img src="image_90.png" /></figure>

≠ software

<figure><img src="image_91.png" /></figure>

<figure><img src="image_92.png" /></figure>

≠ software version
# A multiplicity of analysis pipelines

<figure><img src="image_93.png" /></figure>

Input data

<figure><img src="image_94.png" /></figure>

<figure><img src="image_95.png" /></figure>

Results

<figure><img src="image_96.png" /></figure>

≠ algorithm

<figure><img src="image_97.png" /></figure>

<figure><img src="image_98.png" /></figure>

≠ parameters

<figure><img src="image_99.png" /></figure>

<figure><img src="image_100.png" /></figure>

≠ software

<figure><img src="image_101.png" /></figure>

<figure><img src="image_102.png" /></figure>

≠ environment

<figure><img src="image_103.png" /></figure>

<figure><img src="image_104.png" /></figure>

≠ software version
# A multiplicity of analysis pipelines

<figure><img src="image_105.png" /></figure>

Input data

<figure><img src="image_106.png" /></figure>

<figure><img src="image_107.png" /></figure>

Results

... and many more...

<figure><img src="image_108.png" /></figure>

≠ algorithm

<figure><img src="image_109.png" /></figure>

≠ software

<figure><img src="image_110.png" /></figure>

≠ software version

<figure><img src="image_111.png" /></figure>

≠ parameters

<figure><img src="image_112.png" /></figure>

≠ environment
# A multiplicity of analysis pipelines

<figure><img src="image_113.png" /></figure>

Input data

<figure><img src="image_114.png" /></figure>

Results

... and many more...

<figure><img src="image_115.png" /></figure>

≠ algorithm

<figure><img src="image_116.png" /></figure>

≠ software

<figure><img src="image_117.png" /></figure>

≠ software version

<figure><img src="image_118.png" /></figure>

≠ parameters

<figure><img src="image_119.png" /></figure>

≠ environment

**A family of acceptable pipelines**
over $10^{30}$ combinations...
# Different pipelines... different results?

## Variability across neuroimaging software

<figure><img src="image_120.png" /></figure>

## Variability across neuroimaging experts

<figure><img src="image_121.png" /></figure>
# Different pipelines... different results?

## Variability across neuroimaging software

<figure><img src="image_122.png" /></figure>

## Variability across neuroimaging experts

<figure><img src="image_123.png" /></figure>
# Variability across neuroimaging software

## Principles of functional MRI

<figure><img src="image_124.png" /><figcaption>Alex Bowring</figcaption></figure>

<figure><img src="image_125.png" /><figcaption>Tom Nichols</figcaption></figure>

Oxford Uni. (UK)

<figure><img src="image_126.png" /></figure>
# Variability across neuroimaging software

## Principles of functional MRI

<figure><img src="image_127.png" /><figcaption>Alex Bowring</figcaption></figure>

<figure><img src="image_128.png" /><figcaption>Tom Nichols</figcaption></figure>

Oxford Uni. (UK)

<figure><img src="image_129.png" /></figure>
# Variability across neuroimaging software

<figure><img src="image_130.png" /><figcaption>Alex Bowring
Tom Nichols
Oxford Uni. (UK)</figcaption></figure>

<figure><img src="image_131.png" /></figure>

## Principles of functional MRI

<figure><img src="image_132.png" /></figure>

Data preparation

<figure><img src="image_133.png" /></figure>

Derived
data

Statistical analysis

<figure><img src="image_134.png" /></figure>

Results

## fMRI results: Activations maps

<figure><img src="image_135.png" /></figure>

<figure><img src="image_136.png" /><figcaption>Visual</figcaption></figure>

<figure><img src="image_137.png" /></figure>

<figure><img src="image_138.png" /><figcaption>Hand motion</figcaption></figure>

<figure><img src="image_139.png" /></figure>

<figure><img src="image_140.png" /><figcaption>Music</figcaption></figure>
# Variability across neuroimaging software

## Principles of functional MRI

<figure><img src="image_141.png" /><figcaption>Alex Bowring</figcaption></figure>

<figure><img src="image_142.png" /><figcaption>Tom Nichols</figcaption></figure>

Oxford Uni. (UK)

<figure><img src="image_143.png" /></figure>

**Goal 1:** Investigate the variability observed in fMRI results when using different software packages
# Variability across neuroimaging software

## Principles of functional MRI

<figure><img src="image_144.png" /><figcaption>Alex Bowring
Tom Nichols
Oxford Uni. (UK)</figcaption></figure>

<figure><img src="image_145.png" /></figure>

**Goal 1:** Investigate the variability observed in fMRI results when using different software packages

* Reproduced 3 studies from the literature

* Using 3 fMRI software: AFNI, FSL, SPM
  (used in 80% literature (Carp, 2012))

* Pipelines aligned using available configurations.
# Variability across neuroimaging software

* Variability in the activation maps

<figure><img src="image_146.png" /><figcaption>Alex Bowring Tom Nichols
Oxford Uni. (UK)</figcaption></figure>

<table><thead><tr><th></th><th>study ds001</th><th>study ds109</th><th>study ds120</th></tr></thead><tbody><tr><td>AFNI software</td><td></td><td></td><td></td></tr><tr><td>FSL software</td><td></td><td></td><td></td></tr><tr><td>SPM software</td><td></td><td></td><td></td></tr><tr><td>Original paper</td><td></td><td></td><td></td></tr></tbody></table>

[Bowring Maumet* Nichols*, HBM 2019]
# Variability across neuroimaging software

* Variability in the activation maps

<figure><img src="image_147.png" /><figcaption>Alex Bowring Tom Nichols
Oxford Uni. (UK)</figcaption></figure>

<table><thead><tr><th></th><th>study ds001</th><th>study ds109</th><th>study ds120</th></tr></thead><tbody><tr><td>AFNI software</td><td></td><td></td><td></td></tr><tr><td>FSL software</td><td></td><td></td><td></td></tr><tr><td>SPM software</td><td></td><td></td><td></td></tr><tr><td>Original paper</td><td></td><td></td><td></td></tr></tbody></table>

[Bowring Maumet* Nichols*, HBM 2019]
# Variability across neuroimaging software

* Variability in the activation maps

<figure><img src="image_148.png" /><figcaption>Alex Bowring Tom Nichols
Oxford Uni. (UK)</figcaption></figure>

<figure><img src="image_149.png" /></figure>

<figure><img src="image_150.png" /></figure>

<figure><img src="image_151.png" /></figure>

[Bowring Maumet* Nichols*, HBM 2019]
# Variability across neuroimaging software

* Variability in the activation maps

<figure><img src="image_152.png" /><figcaption>Alex Bowring Tom Nichols
Oxford Uni. (UK)</figcaption></figure>

<figure><img src="image_153.png" /></figure>

study ds001

study ds001

<figure><img src="image_154.png" /></figure>

z=4

study ds109

<figure><img src="image_155.png" /></figure>

z=0

study ds120

<figure><img src="image_156.png" /></figure>

x=0

AFNI
software

<figure><img src="image_157.png" /></figure>

z=4

z=0

<figure><img src="image_158.png" /></figure>

x=0

FSL
software

<figure><img src="image_159.png" /></figure>

z=4

<figure><img src="image_160.png" /></figure>

z=0

<figure><img src="image_161.png" /></figure>

x=0

SPM
software

<figure><img src="image_162.png" /></figure>

Original

<figure><img src="image_163.png" /></figure>

<figure><img src="image_164.png" /></figure>

AFNI software

<figure><img src="image_165.png" /></figure>

z=-32

<figure><img src="image_166.png" /></figure>

z=-18

<figure><img src="image_167.png" /></figure>

z=0

<figure><img src="image_168.png" /></figure>

z=12

<figure><img src="image_169.png" /></figure>

z=24

FSL software

<figure><img src="image_170.png" /></figure>

z=-32

<figure><img src="image_171.png" /></figure>

z=-18

<figure><img src="image_172.png" /></figure>

z=0

<figure><img src="image_173.png" /></figure>

z=12

<figure><img src="image_174.png" /></figure>

z=24

SPM software

<figure><img src="image_175.png" /></figure>

z=-32

<figure><img src="image_176.png" /></figure>

z=-18

<figure><img src="image_177.png" /></figure>

z=0

<figure><img src="image_178.png" /></figure>

z=12

<figure><img src="image_179.png" /></figure>

z=24

Dice coefficient

<figure><img src="image_180.png" /></figure>
# Variability across neuroimaging software

* Variability in the unthresholded statistic maps

<figure><img src="image_181.png" /><figcaption>Alex Bowring Tom Nichols
Oxford Uni. (UK)</figcaption></figure>

<figure><img src="image_182.png" /></figure>

## Correlations

AFNI vs. FSL 0.62

AFNI vs. SPM 0.61

FSL vs. SPM 0.68

**Conclusion:** The choice of software impacted both the final activation and statistic maps.
# Variability across neuroimaging software

<figure><img src="image_183.png" /><figcaption>Alex Bowring Tom Nichols
Oxford Uni. (UK)</figcaption></figure>

<figure><img src="image_184.png" /></figure>

**Goal 2:** Investigate how remaining differences in software implementations can explain the observed differences

study ds001

1. Preprocessing

2. Subject-level Signal

3. Subject-level Drift

4. Subject-level Noise

5. Group-level Stat

<figure><img src="image_185.png" /></figure>

<figure><img src="image_186.png" /></figure>

<figure><img src="image_187.png" /></figure>

<figure><img src="image_188.png" /></figure>

<figure><img src="image_189.png" /></figure>

AFNI
software

<figure><img src="image_190.png" /></figure>

<figure><img src="image_191.png" /></figure>

<figure><img src="image_192.png" /></figure>

FSL
software
# Variability across neuroimaging software

<figure><img src="image_193.png" /><figcaption>Alex Bowring Tom Nichols
Oxford Uni. (UK)</figcaption></figure>

<figure><img src="image_194.png" /></figure>

**Goal 2:** Investigate how remaining differences in software implementations can explain the observed differences

**study ds001**

1. Preprocessing

2. Subject-level Signal

3. Subject-level Drift

4. Subject-level Noise

5. Group-level Stat

<figure><img src="image_195.png" /></figure>

<figure><img src="image_196.png" /></figure>

<figure><img src="image_197.png" /></figure>

<figure><img src="image_198.png" /></figure>

AFNI
software

<figure><img src="image_199.png" /></figure>

<figure><img src="image_200.png" /></figure>

<figure><img src="image_201.png" /></figure>

<figure><img src="image_202.png" /></figure>

FSL
software

**Conclusion:** 1 main contributing factor was found but different for each study

[Bowring Nichols\* Maumet\*, HBM 2021]
# Different pipelines... different results?

## Variability across neuroimaging software

<figure><img src="image_203.png" /></figure>

## Variability across neuroimaging experts

<figure><img src="image_204.png" /></figure>
**Many-analyst** project : NARPS

<figure><img src="image_205.png" /></figure>

1 dataset

9 'yes/no'
questions

[Botvinik-Nezer et. al, Nature 2020]
**Many-analyst** project : NARPS

<figure><img src="image_206.png" /></figure>

1 dataset

<figure><img src="image_207.png" /></figure>

9 'yes/no'
questions

<figure><img src="image_208.png" /></figure>

70 analysis pipelines
by different teams

[Botvinik-Nezer et. al, Nature 2020]
**Many-analyst** project : NARPS

Results over the **70 post-processing pipelines**:

<figure><img src="image_209.png" /></figure>

9 'yes/no' questions

[Botvinik-Nezer et. al, Nature 2020]
**Many-analyst** project : NARPS

Results over the **70 post-processing pipelines**:

<figure><img src="image_210.png" /></figure>

**Significant finding**
(>80% of teams agree)

All 'yes'

Contradictory results
(Teams disagree)

**Non-significant finding**
(>80% of teams agree)

All 'no'

**9 'yes/no' questions**

[Botvinik-Nezer et. al, Nature 2020]
<figure><img src="image_211.png" /></figure>

# Understanding
analytical variability
# Understanding analytical variability...

?

<figure><img src="image_212.png" /></figure>
# Understanding analytical variability...

?

<figure><img src="image_213.png" /></figure>

Numerical
uncertainty
(e.g. ≠ OS)

Different
implen.

<figure><img src="image_214.png" /></figure>

Unwanted
variability

<figure><img src="image_215.png" /></figure>
# Understanding analytical variability...

?

<figure><img src="image_216.png" /></figure>

Numerical
uncertainty
(e.g. ≠ OS)

Assumption
violations

Unwanted
variability

Different
implen.

Unreliable
results

Data
induced
variability
# Understanding analytical variability...

?

Better fit to the data

Variability of interest

?

Assumption violations

<figure><img src="image_217.png" /></figure>

Numerical uncertainty (e.g. ≠ OS)

Data induced variability

Unreliable results

Different implem.

Unwanted variability
<figure><img src="image_218.png" /></figure>

# Describing the neuroimaging pipeline space
# Open code and data to investigate analytical variability

<figure><img src="image_219.png" /><figcaption>## HCP many pipeline</figcaption></figure>

<figure><img src="image_220.png" /></figure>
**Open code and data to investigate
analytical variability**

**NARPS open pipelines**

<figure><img src="image_221.png" /></figure>

**HCP many pipeline**

<figure><img src="image_222.png" /></figure>
# NARPS open pipelines

<figure><img src="image_223.png" /><figcaption>Boris
Clenet</figcaption></figure>

<figure><img src="image_224.png" /><figcaption>Elodie
Germani</figcaption></figure>

<figure><img src="image_225.png" /><figcaption>Jérémy
Lefort-Besnard</figcaption></figure>

<figure><img src="image_226.png" /></figure>

1 dataset

<figure><img src="image_227.png" /></figure>

9 'yes/no'
questions

<figure><img src="image_228.png" /></figure>

70 analysis pipelines
by different teams
# NARPS open pipelines

<figure><img src="image_229.png" /><figcaption>Boris
Clenet</figcaption></figure>

<figure><img src="image_230.png" /><figcaption>Elodie
Germani</figcaption></figure>

<figure><img src="image_231.png" /><figcaption>Jérémy
Lefort-Besnard</figcaption></figure>

<figure><img src="image_232.png" /></figure>

1 dataset

<figure><img src="image_233.png" /></figure>

9 'yes/no'
questions

<figure><img src="image_234.png" /></figure>

70 analysis pipelines
by different teams

1- Q1: “yes”

2- Activation +
Statistic brain maps

3- Textual
COBIDAS-compliant
description of the
methods

<figure><img src="image_235.png" /></figure>
# NARPS open pipelines

<figure><img src="image_236.png" /><figcaption>Boris
Clenet</figcaption></figure>

<figure><img src="image_237.png" /><figcaption>Elodie
Germani</figcaption></figure>

<figure><img src="image_238.png" /><figcaption>Jérémy
Lefort-Besnard</figcaption></figure>

<figure><img src="image_239.png" /></figure>

1 dataset

<figure><img src="image_240.png" /></figure>

9 'yes/no'
questions

<figure><img src="image_241.png" /></figure>

70 analysis pipelines
by different teams

1- Q1: “yes”

2- Activation +
Statistic brain maps

3- Textual
COBIDAS-compliant
description of the
methods

<figure><img src="image_242.png" /></figure>

+ Code

<figure><img src="image_243.png" /></figure>
# NARPS open pipelines

<figure><img src="image_244.png" /><figcaption>Boris
Clenet</figcaption></figure>

<figure><img src="image_245.png" /><figcaption>Elodie
Germani</figcaption></figure>

<figure><img src="image_246.png" /><figcaption>Jérémy
Lefort-Besnard</figcaption></figure>

<figure><img src="image_247.png" /></figure>

brainhack.org

**Credits** - This project was presented and received contributions during the following events:

* Empenn hackathon (February 2024)

* Brainhack Marseille 2023 (December 2023)

* ORIGAMI lab hackathon (September 2023)

* OHBM Brainhack 2023 (July 2023): Arshitha Basavaraj, Boris Clénet, Rémi Gau, Élodie Germani, Yaroslav Halchenko, Camille Maumet, Paul Taylor.

* e-ReproNim FENS NENS Cluster Brainhack (June 2023) : Liz Bushby, Boris Clénet, Michael Dayan, Aimee Westbrook.

* OHBM Brainhack 2022 (June 2022): Elodie Germani, Arshitha Basavaraj, Trang Cao, Rémi Gau, Anna Menacher, Camille Maumet.

**Open code**

<figure><img src="image_248.png" /></figure>

https://github.com/Inria-Empenn/narps_open_pipelines/
**Open code and data to investigate
analytical variability**

<figure><img src="image_249.png" /><figcaption>HCP many pipeline</figcaption></figure>

<figure><img src="image_250.png" /></figure>
# HCP many pipeline

* **1 dataset**

* HCP Young Adult

* **24 pipelines**

* FSL, SPM / 2 HRF / 2 smoothing / 3 motion regressors

* **1080 participants and 1000 groups** (50 subjects)

<figure><img src="image_251.png" /><figcaption>Elodie
Germani</figcaption></figure>

<figure><img src="image_252.png" /><figcaption>Pierre
Maurel</figcaption></figure>

<figure><img src="image_253.png" /><figcaption>Elisa
Fromont</figcaption></figure>

<figure><img src="image_254.png" /></figure>

<figure><img src="image_255.png" /></figure>

<figure><img src="image_256.png" /></figure>

<figure><img src="image_257.png" /></figure>

<figure><img src="image_258.png" /></figure>

<figure><img src="image_259.png" /></figure>

<figure><img src="image_260.png" /></figure>

<figure><img src="image_261.png" /></figure>

<figure><img src="image_262.png" /></figure>

<figure><img src="image_263.png" /></figure>

<figure><img src="image_264.png" /></figure>

<figure><img src="image_265.png" /></figure>

<figure><img src="image_266.png" /></figure>

<figure><img src="image_267.png" /></figure>

<figure><img src="image_268.png" /></figure>

<figure><img src="image_269.png" /></figure>

<figure><img src="image_270.png" /></figure>

<figure><img src="image_271.png" /></figure>

<figure><img src="image_272.png" /></figure>

<figure><img src="image_273.png" /></figure>

<figure><img src="image_274.png" /></figure>

<figure><img src="image_275.png" /></figure>

<figure><img src="image_276.png" /></figure>

<figure><img src="image_277.png" /></figure>

## Shared data (in compliance with GDPR)

<figure><img src="image_278.png" /></figure>

https://publicneuro-catalogue.netlify.app
<figure><img src="image_279.png" /></figure>

# Which solutions for practitioners ?
# Same data meta-analysis

<figure><img src="image_280.png" /><figcaption>Jérémy
Lefort-Besnard</figcaption></figure>

<figure><img src="image_281.png" /><figcaption>Tom Nichols
Oxford Uni. (UK)</figcaption></figure>

<figure><img src="image_282.png" /></figure>

[Lefort-Besnard Nichols* Maumet*, OHBM 2024]
# Same data meta-analysis

<figure><img src="image_283.png" /><figcaption>Jérémy
Lefort-Besnard</figcaption></figure>

<figure><img src="image_284.png" /><figcaption>Tom Nichols
Oxford Uni. (UK)</figcaption></figure>

<figure><img src="image_285.png" /></figure>

[Lefort-Besnard Nichols* Maumet*, OHBM 2024]
# Same data meta-analysis

<figure><img src="image_286.png" /><figcaption>Jérémy
Lefort-Besnard</figcaption></figure>

<figure><img src="image_287.png" /><figcaption>Tom Nichols
Oxford Uni. (UK)</figcaption></figure>

<figure><img src="image_288.png" /></figure>

**Goal:** Investigate the validity of same-data meta-analysis methods

[Lefort-Besnard Nichols* Maumet*, OHBM 2024]
# Same data meta-analysis

<figure><img src="image_289.png" /><figcaption>Jérémy
Lefort-Besnard</figcaption></figure>

<figure><img src="image_290.png" /><figcaption>Tom Nichols
Oxford Uni. (UK)</figcaption></figure>

## Meta-Analysis

Stouffer

$$Z_j^S = \frac{\bar{Y}_j}{\sqrt{1/K}}$$

Average
standardized

## Same-Data Meta-Analysis

SDMA
Stouffer

$$Z_j^{SS} = \frac{\bar{Y}_j}{\sqrt{\frac{1}{K} \sum \mathbf{Q}_k \sum \mathbf{Q}_k / K^2}}$$

Average
standardized
accounting for
dependence

Consensus
SDMA
Stouffer

$$Z_j^{CSS} = Z_j^{SS} - \langle Z^{SS} \rangle + \mu_C$$

Average standardized
accounting for
dependence, and
shifted empirically

Consensus
Average

$$Z_j^{CA} = \frac{\bar{Y}_j - \langle \bar{Y} \rangle}{\sqrt{\langle \langle \bar{Y} \rangle \rangle}} \sigma_C + \mu_C$$

Average, scaled
and shifted
empirically

SDMA GLS

$$Z_j^{SG} = \frac{1^\top \mathbf{Q}^{-1} \mathbf{Y}_j}{\sqrt{1^\top \mathbf{Q}^{-1}}}$$

Optimal average,
standardized

Valid
4.7%

Valid
4.7%

Valid
4.8%

Valid
4.8%

Valid
4.4%

<figure><img src="image_291.png" /></figure>

Invalid
30%

Valid
4.9%

Valid
5.0%

Valid
5.0%

Valid
4.9%

<figure><img src="image_292.png" /></figure>

P<0.05 (uncorrected)

[Lefort-Besnard Nichols\*, Maumet\*, OHBM 2024]
# Same data meta-analysis

<figure><img src="image_293.png" /><figcaption>Jérémy
Lefort-Besnard</figcaption></figure>

<figure><img src="image_294.png" /><figcaption>Tom Nichols
Oxford Uni. (UK)</figcaption></figure>

<figure><img src="image_295.png" /></figure>

[Lefort-Besnard Nichols* Maumet*, OHBM 2024]
<figure><img src="image_296.png" /></figure>

<figure><img src="image_297.png" /></figure>

# Towards reproducible neuroimaging
across different analysis
pipelines

Camille Maumet

Thank you!

Credit: Presentation template and icons by
[SlidesCarnival](https:// slidescarnival.com ), adapted

<figure><img src="image_298.png" /></figure>

@cmaumet.bsky.social

<figure><img src="image_299.png" /></figure>

@cmaumet@fediscience.org

<figure><img src="image_300.png" /></figure>

@cmaumet