# Towards large-scale brain imaging studies

Camille Maumet

► To cite this version:

Camille Maumet. Towards large-scale brain imaging studies. workshop Virtual 2020 - Inteligencia Articial en imagenes medicas, Dec 2020, Santiago (online), Chile. inserm-03058348

HAL Id: inserm-03058348

https://inria.hal.science/inserm-03058348

Submitted on 11 Dec 2020

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Towards large-scale brain imaging studies

<figure><img src="image_2.png" /><figcaption>Camille Maumet</figcaption></figure>

Univ Rennes, Inria, CNRS, Inserm

<figure><img src="image_3.png" /><figcaption>@cmaumet</figcaption></figure>

<figure><img src="image_4.png" /></figure>
# A brain imaging study

<figure><img src="image_5.png" /></figure>

Data preparation

<figure><img src="image_6.png" /></figure>

Derived data

Analysis

<figure><img src="image_7.png" /></figure>
# A brain imaging study

<figure><img src="image_8.png" /></figure>

about 30 participants
per study

<figure><img src="image_9.png" /></figure>

[Poldrack et. al, Nature Neuroscience 2017]
SCIENCE

# A Waste of 1,000 Research Papers

Decades of early research on the genetics of depression were built on nonexistent foundations. How did that happen?

ED YONG MAY 17, 2019

<figure><img src="image_10.png" /></figure>

SEAN NEL / SHUTTERSTOCK

In 1996, a group of European researchers found that a certain gene, called *SLC6A4*, might influence a person's risk of depression.

It was a blockbuster discovery at the time. The team found that a less active version of the gene was more common among 454 people who had mood disorders than in 570 who did not. In theory, anyone who had this particular gene variant could be at higher risk for depression, and that finding, they said, might help in diagnosing such disorders, assessing suicidal behavior, or even

(Ir)-Reproducibility
Why are middle-aged marathon
runners faster than
twentysomethings?

According to new data from the running app Strava, runners in their 40s are streets ahead of younger rivals

<figure><img src="image_11.png" /><figcaption>▲ 'Middle-aged runners outperform runners in the 20s ...' Photograph: FatCamera/Getty</figcaption></figure>

According to data released by the running app Strava, middle-aged runners consistently average faster marathon times than their younger rivals, apparently defying the usual rules of athletic performance. Men in the 40-49 age bracket clock an average time of four hours and 17 minutes for a marathon, according to the recent figures. Women in the same age range typically come in at just under the five-hour mark.

Faster in their forties
than twenties?

Selection bias
# A brain imaging study

<figure><img src="image_12.png" /></figure>

Data preparation

<figure><img src="image_13.png" /></figure>

Derived data

Analysis

<figure><img src="image_14.png" /></figure>

We need **bigger & more representative samples**
# Open data

<figure><img src="image_15.png" /></figure>

Unique study

30 participants

<figure><img src="image_16.png" /></figure>

OpenNEURO

studyforrest.org

<figure><img src="image_17.png" /></figure>

NEUROVAULT

L I E L N L F T Q K T Q R V
S M Y C O N N E C T O M E Q
G S P K K W A R R G K E H R

<figure><img src="image_18.png" /></figure>

NITRC

<figure><img src="image_19.png" /></figure>

OSF

+ Images

+ Homogenous

- Datasets
# Open data

<figure><img src="image_20.png" /></figure>

Unique study

30 participants

<figure><img src="image_21.png" /></figure>

OpenNEURO

<figure><img src="image_22.png" /></figure>

NEUROVAULT

studyforrest.org

L I E N L F T Q K T Q R V
S M Y C O N N E C T O M E Q
G S P K K W A R R G K E H R

Consortium

1000 participants

<figure><img src="image_23.png" /></figure>

ABIDE

Autism Brain Imaging
Data Exchange

1000 Functional Connectomes Project

<figure><img src="image_24.png" /></figure>

CORR

CONSORTIUM FOR
RELIABILITY AND
REPRODUCIBILITY

<figure><img src="image_25.png" /></figure>

<figure><img src="image_26.png" /></figure>

NITRC

<figure><img src="image_27.png" /></figure>

OSF

+ Images

+ Homogenous

- Datasets
# Open data

<figure><img src="image_28.png" /></figure>

Unique study
30 participants

<figure><img src="image_29.png" /></figure>

OpenNEURO

<figure><img src="image_30.png" /></figure>

NEUROVAULT

studyforrest.org

L I E L N L F T Q K T Q R V
S M Y C O N N E C T O M E Q
G S P K K W A R R G K E H R

Consortium
1000 participants

<figure><img src="image_31.png" /></figure>

ABIDE

Autism Brain Imaging
Data Exchange

1000 Functional Connectomes Project

Cohort
100 000 participants

<figure><img src="image_32.png" /></figure>

HUMAN
Connectome
PROJECT

<figure><img src="image_33.png" /></figure>

NITRC

<figure><img src="image_34.png" /></figure>

OSF

<figure><img src="image_35.png" /></figure>

CONSORTIUM FOR
RELIABILITY AND
REPRODUCIBILITY

<figure><img src="image_36.png" /></figure>

+ Images

+ Homogenous

- Datasets

biobank™
Improving the health of future generations
**Many Facial-Recognition Systems Are**
**Biased, Says U.S. Study**

Algorithms falsely identified African-American and Asian faces 10 to 100 times more than Caucasian faces, researchers for the National Institute of Standards and Technology found.

<figure><img src="image_37.png" /><figcaption>Morning at Grand Central Terminal. Technology for facial recognition is frequently biased, a new study confirmed. Timothy A. Clary/Agence France-Presse — Getty Images</figcaption></figure>

Lack of diversity of the
input data

**Algorithmic bias**
OPT-OUT CULTURE

# Using a fitness app taught me the scary truth about why privacy settings are a feminist issue

<figure><img src="image_38.png" /></figure>

The social fitness app Strava is meant to help people compare data on their workouts.

REUTERS/EDDIE KEOGH

FROM OUR SERIES

Ideas

Our home for bold arguments and big thinkers.

<figure><img src="image_39.png" /><figcaption>By Rosie Spinks</figcaption></figure>

August 1, 2017 • This article is more than 2 years old.

Lack of diversity of the
team

Data privacy
# A brain imaging study

<figure><img src="image_40.png" /></figure>

Data preparation

<figure><img src="image_41.png" /></figure>

Derived data

Analysis

<figure><img src="image_42.png" /></figure>

We need **bigger** and
**representative** samples

We need **diversity in datasets** *and* in **leadership teams**
# Data integration
# Data integration

<figure><img src="image_43.png" /></figure>

Raw data

<figure><img src="image_44.png" /></figure>

Data preparation

<figure><img src="image_45.png" /></figure>

<figure><img src="image_46.png" /></figure>

Derived data

<figure><img src="image_47.png" /></figure>

Analysis

<figure><img src="image_48.png" /></figure>

<figure><img src="image_49.png" /></figure>

<figure><img src="image_50.png" /></figure>

Results
# Data integration

<figure><img src="image_51.png" /></figure>

Raw data

<figure><img src="image_52.png" /></figure>

Raw data

Data preparation

<figure><img src="image_53.png" /></figure>

Derived data

Analysis

<figure><img src="image_54.png" /></figure>

<figure><img src="image_55.png" /></figure>

Results
# Data integration

<figure><img src="image_56.png" /></figure>

Raw data

<figure><img src="image_57.png" /></figure>

Raw data

Data preparation

Data preparation

<figure><img src="image_58.png" /></figure>

Derived data

<figure><img src="image_59.png" /></figure>

Derived data

Analysis

<figure><img src="image_60.png" /></figure>

<figure><img src="image_61.png" /></figure>

Results
# Data integration

<figure><img src="image_62.png" /></figure>
# Data integration

<figure><img src="image_63.png" /></figure>
A new level of
variability
# Levels of variability

* Variability in **single-lab** studies

* Across subjects

* Across scans

<figure><img src="image_64.png" /></figure>
# Levels of variability

* Variability in **single-lab** studies

* Across subjects

* Across scans

* Variability in **multi-lab** studies

* All of the above plus

* Across machines

* Across sites

* Across acquisition protocols

<figure><img src="image_65.png" /></figure>
# Levels of variability

* Variability in **single-lab** studies

* Across subjects

* Across scans

* Variability in **multi-lab** studies

* All of the above plus

* Across machines

* Across sites

* Across acquisition protocols

* Variability in **data integration** studies

* All of the above plus

* Across preparation protocols

* Across analysis protocols

<figure><img src="image_66.png" /></figure>
# Levels of variability

* Variability in **single-lab** studies

* Across subjects

* Across scans

* Variability in **multi-lab** studies

* All of the above plus

* Across machines

* Across sites

* Across acquisition protocols

* Variability in **data integration** studies

* All of the above plus

* Across preparation protocols

* Across analysis protocols

<figure><img src="image_67.png" /></figure>

Analytic Variability

<figure><img src="image_68.png" /></figure>
# Analytic variability

“Different acceptable analysis methods”

Carp et al. (2012)

<figure><img src="image_69.png" /></figure>

<figure><img src="image_70.png" /></figure>

Data preparation

<figure><img src="image_71.png" /></figure>

<figure><img src="image_72.png" /></figure>

Derived data

<figure><img src="image_73.png" /></figure>

Analysis

<figure><img src="image_74.png" /></figure>

<figure><img src="image_75.png" /></figure>
Analytic variability

<figure><img src="image_76.png" /></figure>
Analytic variability

<figure><img src="image_77.png" /></figure>
Analytic variability

<figure><img src="image_78.png" /></figure>
Analytic variability

<figure><img src="image_79.png" /></figure>

<figure><img src="image_80.png" /></figure>
Analytic variability

<figure><img src="image_81.png" /></figure>

≠ software

[Paul et al. 2016,
Bowring et al. 2019]

<figure><img src="image_82.png" /></figure>

≠ algorithm

<figure><img src="image_83.png" /></figure>

≠ software

<figure><img src="image_84.png" /></figure>
<figure><img src="image_85.png" /></figure>

[Groenenschild et al. 2012]

<figure><img src="image_86.png" /></figure>

≠ algorithm

<figure><img src="image_87.png" /></figure>

≠ software

<figure><img src="image_88.png" /></figure>

≠ software version
# Analytic variability

<figure><img src="image_89.png" /></figure>

<figure><img src="image_90.png" /></figure>

≠ algorithm

<figure><img src="image_91.png" /></figure>

≠ software

<figure><img src="image_92.png" /></figure>

≠ software version

<figure><img src="image_93.png" /></figure>

<figure><img src="image_94.png" /></figure>

<figure><img src="image_95.png" /></figure>

≠ parameters

<figure><img src="image_96.png" /></figure>
# Analytic variability

<figure><img src="image_97.png" /></figure>

≠ environment

[Glatard et al. 2015]

<figure><img src="image_98.png" /></figure>

≠ algorithm

<figure><img src="image_99.png" /></figure>

≠ software

<figure><img src="image_100.png" /></figure>

≠ software version

<figure><img src="image_101.png" /></figure>

≠ parameters

<figure><img src="image_102.png" /></figure>

≠ environment
# Analytic variability

<figure><img src="image_103.png" /></figure>

<figure><img src="image_104.png" /></figure>

≠ algorithm

<figure><img src="image_105.png" /></figure>

≠ software

<figure><img src="image_106.png" /></figure>

≠ software version

<figure><img src="image_107.png" /></figure>

≠ parameters

<figure><img src="image_108.png" /></figure>

≠ environment
# Towards data integration in brain imaging
<figure><img src="image_109.png" /></figure>

Towards data integration
in brain imaging studies

<figure><img src="image_110.png" /></figure>

Standards for
data sharing

Statistical modelling
of analytic variability
<figure><img src="image_111.png" /></figure>

Towards **data integration**
in brain imaging studies

<figure><img src="image_112.png" /></figure>

Standards for
data sharing

Statistical modelling
of analytic variability
# Variations across software

* Reproduced 3 published functional MRI studies

* Using 3 different software

<figure><img src="image_113.png" /><figcaption>Alex Bowring Tom Nichols</figcaption></figure>

Software 1

<figure><img src="image_114.png" /></figure>

Pipeline

<figure><img src="image_115.png" /></figure>

Software 2

<figure><img src="image_116.png" /></figure>

Pipeline

<figure><img src="image_117.png" /></figure>

<figure><img src="image_118.png" /></figure>

Software 3

<figure><img src="image_119.png" /></figure>

Pipeline

<figure><img src="image_120.png" /></figure>

<figure><img src="image_121.png" /></figure>
# Variations across software

<figure><img src="image_122.png" /></figure>

Comparison of the final results

<figure><img src="image_123.png" /></figure>

Comparison of the statistic maps
<figure><img src="image_124.png" /></figure>

Towards data integration
in brain imaging studies

<figure><img src="image_125.png" /></figure>

Standards for
data sharing

Statistical modelling
of analytic variability
# Sharing more research outputs

<figure><img src="image_126.png" /></figure>

Data preparation

<figure><img src="image_127.png" /></figure>

Derived data

<figure><img src="image_128.png" /></figure>

Analysis

<figure><img src="image_129.png" /></figure>
# Sharing more research outputs

<figure><img src="image_130.png" /></figure>
# Sharing more research outputs

<figure><img src="image_131.png" /></figure>
# Sharing more research outputs

<figure><img src="image_132.png" /></figure>
# Brain Imaging Data Structure
(BIDS)

Traffic to bids.neuroimaging.io

* Used in over 60 labs around the world

* Adopted by: FCP-INDI, Developing Human Connectome, SchizConnect and Donders Data repository.

* Extensions: MEG, iEEG, EEG

<figure><img src="image_133.png" /><figcaption>Krys
Gorgolewski</figcaption></figure>

[Gorgolewski et al., Sci. Data 2016]

<figure><img src="image_134.png" /></figure>
# Sharing more research outputs

<figure><img src="image_135.png" /></figure>

Data preparation

<figure><img src="image_136.png" /></figure>

Derived data

<figure><img src="image_137.png" /></figure>

Analysis

<figure><img src="image_138.png" /></figure>

<figure><img src="image_139.png" /></figure>

<figure><img src="image_140.png" /></figure>

<figure><img src="image_141.png" /></figure>

BIDS

BRAIN IMAGING DATA STRUCTURE

<figure><img src="image_142.png" /></figure>

<figure><img src="image_143.png" /></figure>

<figure><img src="image_144.png" /></figure>

HAL, BiorXiv
Pubmed
# Sharing more research outputs

<figure><img src="image_145.png" /></figure>
# Sharing more research outputs

<figure><img src="image_146.png" /></figure>

Data preparation

<figure><img src="image_147.png" /></figure>

Derived data

<figure><img src="image_148.png" /></figure>

Analysis

<figure><img src="image_149.png" /></figure>

<figure><img src="image_150.png" /></figure>

BIDS
BRAIN IMAGING DATA STRUCTURE

<figure><img src="image_151.png" /></figure>

<figure><img src="image_152.png" /></figure>

OpenfMRI

<figure><img src="image_153.png" /></figure>

<figure><img src="image_154.png" /></figure>

SHANOIR

<figure><img src="image_155.png" /></figure>

LORIS

<figure><img src="image_156.png" /></figure>

OpenNEURO

<figure><img src="image_157.png" /></figure>

HAL, BiorXiv
Pubmed

<figure><img src="image_158.png" /></figure>

brainmap.org

neurosynth.org

brainspell

<figure><img src="image_159.png" /></figure>

Image credits: Parcels 1 2 & 4 (CCO), Parcel 3 (CCO), Parcel 5 (CCO).
fMRI Results

Publication

<figure><img src="image_160.png" /></figure>

<figure><img src="image_161.png" /><figcaption>Figure
(selected slices)</figcaption></figure>

<figure><img src="image_162.png" /><figcaption>Thresholded
statistics</figcaption></figure>

<figure><img src="image_163.png" /><figcaption>Peak locations</figcaption></figure>

<figure><img src="image_164.png" /></figure>

Incomplete statistical results

Ambiguous/incomplete methods

Metadata is not searchable
# NIDM-Results pack

<figure><img src="image_165.png" /><figcaption>NIDM-Results
.nidm.zip</figcaption></figure>

<figure><img src="image_166.png" /></figure>

ClusterLabels.nii.gz

<figure><img src="image_167.png" /></figure>

Contrast.nii.gz

<figure><img src="image_168.png" /></figure>

ContrastStandardError.nii.gz

<figure><img src="image_169.png" /></figure>

DesignMatrix.csv

<figure><img src="image_170.png" /></figure>

DesignMatrix.png

<figure><img src="image_171.png" /></figure>

ExcursionSet.nii.gz

<figure><img src="image_172.png" /></figure>

ExcursionSet.png

<figure><img src="image_173.png" /></figure>

GrandMean.nii.gz

<figure><img src="image_174.png" /></figure>

Mask.nii.gz

<figure><img src="image_175.png" /></figure>

nidm.json

<figure><img src="image_176.png" /></figure>

nidm.ttl

<figure><img src="image_177.png" /></figure>

ParameterEstimate_001.nii.gz

<figure><img src="image_178.png" /></figure>

ParameterEstimate_002.nii.gz

<figure><img src="image_179.png" /></figure>

ResidualMeanSquares.nii.gz

<figure><img src="image_180.png" /></figure>

SearchSpaceMask.nii.gz

<figure><img src="image_181.png" /></figure>

TStatistic.nii.gz

<figure><img src="image_182.png" /></figure>

ZStatistic.nii.gz

[Maumet et al.,
Sci. Data 2016]
# NIDM-Results pack

<figure><img src="image_183.png" /></figure>

<figure><img src="image_184.png" /><figcaption>NIDM-Results</figcaption></figure>

<figure><img src="image_185.png" /><figcaption>.nidm.zip</figcaption></figure>

ClusterLabels.nii.gz

<figure><img src="image_186.png" /></figure>

Contrast.nii.gz

<figure><img src="image_187.png" /></figure>

ContrastStandardError.nii.gz

<figure><img src="image_188.png" /></figure>

DesignMatrix.csv

<figure><img src="image_189.png" /></figure>

DesignMatrix.png

<figure><img src="image_190.png" /></figure>

ExcursionSet.nii.gz

<figure><img src="image_191.png" /></figure>

ExcursionSet.png

<figure><img src="image_192.png" /></figure>

GrandMean.nii.gz

<figure><img src="image_193.png" /></figure>

Mask.nii.gz

<figure><img src="image_194.png" /></figure>

nidm.json

<figure><img src="image_195.png" /></figure>

nidm.ttl

<figure><img src="image_196.png" /></figure>

ParameterEstimate_001.nii.gz

<figure><img src="image_197.png" /></figure>

ParameterEstimate_002.nii.gz

<figure><img src="image_198.png" /></figure>

ResidualMeanSquares.nii.gz

<figure><img src="image_199.png" /></figure>

SearchSpaceMask.nii.gz

<figure><img src="image_200.png" /></figure>

TStatistic.nii.gz

ZStatistic.nii.gz

<figure><img src="image_201.png" /></figure>

<figure><img src="image_202.png" /></figure>

Design of the
experiment, pre-
processed data...

<figure><img src="image_203.png" /></figure>

Statistical
estimation

<figure><img src="image_204.png" /></figure>

Effects, standard error
and statistical maps...

<figure><img src="image_205.png" /></figure>

Inference

<figure><img src="image_206.png" /></figure>

Thresholded maps, list
of peaks and clusters...

<figure><img src="image_207.png" /></figure>
# Sharing more research outputs

<figure><img src="image_208.png" /></figure>

Image credits: Parcels 1 2 & 4 (CCo), Parcel 3 (CCo), Parcel 5 (CCo).
# Sharing more research outputs

<figure><img src="image_209.png" /></figure>

Image credits: Parcels 1 2 & 4 (CCo), Parcel 3 (CCo), Parcel 5 (CCo).
# BIDS Prov **Bridging BIDS & NIDM**

<figure><img src="image_210.png" /></figure>

my_dataset/

<figure><img src="image_211.png" /></figure>

participants.tsv

<figure><img src="image_212.png" /></figure>

sub-01/

<figure><img src="image_213.png" /></figure>

anat/

<figure><img src="image_214.png" /></figure>

sub-01_T1w.nii.gz

<figure><img src="image_215.png" /></figure>

func/

File naming convention + JSON files

60 labs worldwide

<figure><img src="image_216.png" /></figure>

## BIDS

<figure><img src="image_217.png" /></figure>

<figure><img src="image_218.png" /></figure>

## NIDM

<figure><img src="image_219.png" /></figure>

<figure><img src="image_220.png" /></figure>

Semantic web (RDF)

<figure><img src="image_221.png" /></figure>
# BIDS Prov Bridging BIDS & NIDM

<figure><img src="image_222.png" /></figure>

my_dataset/

<figure><img src="image_223.png" /></figure>

participants.tsv

<figure><img src="image_224.png" /></figure>

sub-01/

<figure><img src="image_225.png" /></figure>

anat/

<figure><img src="image_226.png" /></figure>

sub-01_T1w.nii.gz

<figure><img src="image_227.png" /></figure>

func/

File naming convention + JSON files

— 60 labs worldwide

<figure><img src="image_228.png" /></figure>

BIDS Prov

<figure><img src="image_229.png" /></figure>

<figure><img src="image_230.png" /></figure>

NIDM

<figure><img src="image_231.png" /></figure>

<figure><img src="image_232.png" /></figure>

Semantic web (RDF)
# BIDS Provenance

<figure><img src="image_233.png" /><figcaption>Satra
Ghosh</figcaption></figure>

<figure><img src="image_234.png" /><figcaption>Rémi
Adon</figcaption></figure>

<figure><img src="image_235.png" /></figure>

Looking for
contributors and
community input

https://github.com/bids-standard/BEP028_BIDSprov
# References

**Bowring, A., Maumet*, C., & Nichols*, T. E. (2019). Exploring the impact of analysis software on task fMRI results. Human Brain Mapping, 0(0). https://doi.org/10.1002/hbm.24603**

**Carp, J. (2012). On the Plurality of (Methodological) Worlds: Estimating the Analytic Flexibility of fMRI Experiments. Frontiers in Neuroscience, 6.**
https://doi.org/10.3389/fnins.2012.00149

**Gorgolewski, K. J., Auer, T., Calhoun, V. D., Craddock, R. C., Das, S., Duff, E. P., Flandin, G., Ghosh, S. S., Glatard, T., Halchenko, Y. O., Handwerker, D. A., Hanke, M., Keator, D., Li, X., Michael, Z., Maumet, C., Nichols, B. N., Nichols, T. E., Pellman, J., ... Poldrack, R. A. (2016). The brain imaging data structure, a format for organizing and describing outputs of neuroimaging experiments. Scientific Data, 3(1), 160044.**
https://doi.org/10.1038/sdata.2016.44

**Maumet, C., Auer, T., Bowring, A., Chen, G., Das, S., Flandin, G., Ghosh, S., Glatard, T., Gorgolewski, K. J., Helmer, K. G., Jenkinson, M., Keator, D. B., Nichols, B. N., Poline, J.-B., Reynolds, R., Sochat, V., Turner, J., & Nichols, T. E. (2016). Sharing brain mapping statistical results with the neuroimaging data model. Scientific Data, 3, 160102.**
https://doi.org/10.1038/sdata.2016.102

**Poldrack, R. A., Baker, C. I., Durnez, J., Gorgolewski, K. J., Matthews, P. M., Munafò, M. R., Nichols, T. E., Poline, J.-B., Vul, E., & Yarkoni, T. (2017). Scanning the horizon: Towards transparent and reproducible neuroimaging research. Nature Reviews Neuroscience, 18(2), 115-126. https://doi.org/10.1038/nrn.2016.167
Towards large-scale brain imaging studies

Camille Maumet

Thank you!

<figure><img src="image_236.png" /><figcaption>Sculpture by Malin Bjornsdotter "Cerebia", OHBM Brain Art SIG</figcaption></figure>