# Données ouvertes : comment permettre leur réutilisation?

Camille Maumet

## ► To cite this version:

Camille Maumet. Données ouvertes : comment permettre leur réutilisation?. REMI 2021 - Rencontres du Réseau d'Entraide Multicentrique en IRM, Mar 2021, Online, France. pp.1-51. inserm-03174616

HAL Id: inserm-03174616

https://inria.hal.science/inserm-03174616

Submitted on 19 Mar 2021

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Données ouvertes : comment permettre leur réutilisation?

<figure><img src="image_2.png" /></figure>

Camille Maumet

Univ Rennes, Inria, CNRS, Inserm

<figure><img src="image_3.png" /></figure>

@cmaumet
# A brain imaging study

<figure><img src="image_4.png" /></figure>

Data preparation

<figure><img src="image_5.png" /></figure>

Derived data

Analysis

<figure><img src="image_6.png" /></figure>
# A brain imaging study

<figure><img src="image_7.png" /></figure>

about 30 participants
per study

<figure><img src="image_8.png" /></figure>

(Poldrack et. al, Nat. Neurosci. 2017)
SCIENCE

# A Waste of 1,000 Research Papers

Decades of early research on the genetics of depression were built on nonexistent foundations. How did that happen?

ED YONG MAY 17, 2019

<figure><img src="image_9.png" /></figure>

SEAN NEL / SHUTTERSTOCK

In 1996, a group of European researchers found that a certain gene, called *SLC6A4*, might influence a person's risk of depression.

It was a blockbuster discovery at the time. The team found that a less active version of the gene was more common among 454 people who had mood disorders than in 570 who did not. In theory, anyone who had this particular gene variant could be at higher risk for depression, and that finding, they said, might help in diagnosing such disorders, assessing suicidal behavior, or even

(Ir)-Reproducibility
Une étude démontre les biais de la reconnaissance faciale, plus efficace sur les hommes blancs

Lorsqu'il s'agit de reconnaître le genre d'un homme blanc, des logiciels affichent un taux de réussite de 99 %. La tâche se complique lorsque la peau d'une personne est plus foncée, ou s'il s'agit d'une femme.

LE MONDE I 12.02.2018 à 14h52 • Mis à jour le 13.02.2018 à 18h18 |

Par Perrine Signoret

Représentativité de l'espace d'entraînement
# A brain imaging study

<figure><img src="image_10.png" /></figure>

Data preparation

<figure><img src="image_11.png" /></figure>

Derived data

Analysis

<figure><img src="image_12.png" /></figure>

We need **bigger & more representative samples**
# Data sharing in Europe
# Neuroimaging data sharing

<figure><img src="image_13.png" /></figure>

Data preparation

<figure><img src="image_14.png" /></figure>

Derived data

<figure><img src="image_15.png" /></figure>

Analysis

<figure><img src="image_16.png" /></figure>
# Neuroimaging data sharing

<figure><img src="image_17.png" /></figure>

<figure><img src="image_18.png" /></figure>

Data preparation

<figure><img src="image_19.png" /></figure>

<figure><img src="image_20.png" /></figure>

Derived data

<figure><img src="image_21.png" /></figure>

Analysis

<figure><img src="image_22.png" /></figure>

<figure><img src="image_23.png" /></figure>

<figure><img src="image_24.png" /></figure>

<figure><img src="image_25.png" /></figure>
# Neuroimaging data sharing

<figure><img src="image_26.png" /></figure>
# Neuroimaging data sharing

<figure><img src="image_27.png" /></figure>

Data preparation

<figure><img src="image_28.png" /></figure>

Derived data

<figure><img src="image_29.png" /></figure>

Analysis

<figure><img src="image_30.png" /></figure>

<figure><img src="image_31.png" /></figure>

<figure><img src="image_32.png" /></figure>

<figure><img src="image_33.png" /></figure>

<figure><img src="image_34.png" /></figure>

<figure><img src="image_35.png" /></figure>

HAL, BiorXiv
Pubmed
# Brain Imaging Data Structure
(BIDS)

Traffic to bids.neuroimaging.io

* Used in over 60 labs around the world

* Adopted by: FCP-INDI, Developing Human Connectome, SchizConnect and Donders Data repository.

* Extensions: MEG, iEEG, EEG

<figure><img src="image_36.png" /><figcaption>Krys
Gorgolewski</figcaption></figure>

(Gorgolewski et al., Sci. Data 2016)

<figure><img src="image_37.png" /></figure>

<figure><img src="image_38.png" /></figure>

my_dataset/

<figure><img src="image_39.png" /></figure>

participants.csv

<figure><img src="image_40.png" /></figure>

sub-01/

<figure><img src="image_41.png" /></figure>

anat/

<figure><img src="image_42.png" /></figure>

sub-01_T1w.nii.gz

<figure><img src="image_43.png" /></figure>

func/

<figure><img src="image_44.png" /></figure>

sub-01_task-rest bold.nii.gz

<figure><img src="image_45.png" /></figure>

sub-01_task-rest bold.json
# Neuroimaging data sharing

<figure><img src="image_46.png" /></figure>

Data preparation

<figure><img src="image_47.png" /></figure>

Derived data

<figure><img src="image_48.png" /></figure>

Analysis

<figure><img src="image_49.png" /></figure>

<figure><img src="image_50.png" /></figure>

<figure><img src="image_51.png" /></figure>

<figure><img src="image_52.png" /></figure>

BIDS

BRAIN IMAGING DATA STRUCTURE

<figure><img src="image_53.png" /></figure>

<figure><img src="image_54.png" /></figure>

HAL, BiorXiv
Pubmed
# Neuroimaging data sharing

<figure><img src="image_55.png" /></figure>

Data preparation

<figure><img src="image_56.png" /></figure>

Derived data

Analysis

<figure><img src="image_57.png" /></figure>

<figure><img src="image_58.png" /></figure>

<figure><img src="image_59.png" /></figure>

<figure><img src="image_60.png" /></figure>

OpenfMRI

<figure><img src="image_61.png" /></figure>

<figure><img src="image_62.png" /></figure>

<figure><img src="image_63.png" /></figure>

<figure><img src="image_64.png" /></figure>

SHANOIR

<figure><img src="image_65.png" /></figure>

OpenNEURO

<figure><img src="image_66.png" /></figure>

HAL, BiorXiv
Pubmed
# Data sharing *and* privacy

<figure><img src="image_67.png" /></figure>

**Open Brain Consent:** Consent forms & Data User Agreement that enable data sharing of neuroimaging data in compliance with GDPR.

(Bannier et al., HBM 2020)

<figure><img src="image_68.png" /></figure>
# Data sharing *and* privacy

<figure><img src="image_69.png" /></figure>

## Formulaire de consentement

Lien direct

À signer par les
participant.e.s.

"[...] Il vous est proposé de participer à une étude. Dans ce cadre, des données à caractère personnel vous concernant seront collectées, utilisées et stockées. Si vous l'acceptez, vos données peuvent également être utilisées pour d'autres projets de recherche futurs dans le domaine des neurosciences médicales et cognitives dans une mission d'intérêt public. [...]"

Version française

<figure><img src="image_70.png" /><figcaption>Anne
Hespel</figcaption></figure>

<figure><img src="image_71.png" /><figcaption>Elise
Bannier</figcaption></figure>

## License d'utilisation

Lien direct

À signer par les
scientifiques souhaitant
accéder aux données

"[...] 2. Je ne poursuivrai jamais, par quelque moyen que ce soit, la finalité visant à établir ou à retrouver l'identité des personnes concernées par le traitement des données à caractère personnel, ci-devant les participants à l'étude [...]"

3. Je ne diffuserai pas ces données.
[...]"
# Shared (open) data

<figure><img src="image_72.png" /></figure>

Unique study

30 participants

<figure><img src="image_73.png" /></figure>

OpenNEURO

studyforrest.org

<figure><img src="image_74.png" /></figure>

NEUROVAULT

L I E L N L F T Q K T Q R V
S M Y C O N N E C T O M E Q
G S P K K W A R R G K E H R

<figure><img src="image_75.png" /></figure>

NITRC

<figure><img src="image_76.png" /></figure>

OSF

+ Images

+ Homogenous

- Datasets
# Shared (open) data

<figure><img src="image_77.png" /></figure>

Unique study
30 participants

<figure><img src="image_78.png" /></figure>

OpenNEURO

<figure><img src="image_79.png" /></figure>

NEUROVAULT

studyforrest.org

L I E N L F T Q K T Q R V
S M Y C O N N E C T O M E Q
G S P K K W A R R G K E H R

Consortium
1000 participants

<figure><img src="image_80.png" /></figure>

ABIDE

Autism Brain Imaging
Data Exchange

1000 Functional Connectomes Project

<figure><img src="image_81.png" /></figure>

CORR

CONSORTIUM FOR
RELIABILITY AND
REPRODUCIBILITY

<figure><img src="image_82.png" /></figure>

<figure><img src="image_83.png" /></figure>

NITRC

<figure><img src="image_84.png" /></figure>

OSF

+ Images

+ Homogenous

- Datasets
# Shared (open) data

<figure><img src="image_85.png" /></figure>

Unique study
30 participants

<figure><img src="image_86.png" /></figure>

OpenNEURO

<figure><img src="image_87.png" /></figure>

NEUROVAULT

studyforrest.org

L I E L N L F T Q K T Q R V
S M Y C O N N E C T O M E Q
G S P K K W A R R G K E H R

Consortium
1000 participants

<figure><img src="image_88.png" /></figure>

ABIDE

Autism Brain Imaging
Data Exchange

1000 Functional Connectomes Project

Cohort
100 000 participants

<figure><img src="image_89.png" /></figure>

HUMAN
Connectome
PROJECT

<figure><img src="image_90.png" /></figure>

NITRC

<figure><img src="image_91.png" /></figure>

OSF

<figure><img src="image_92.png" /></figure>

CONSORTIUM FOR
RELIABILITY AND
REPRODUCIBILITY

<figure><img src="image_93.png" /></figure>

+ Images

+ Homogenous

- Datasets

biobank™
Improving the health of future generations
# Data integration
# Data integration

<figure><img src="image_94.png" /></figure>

Raw data

<figure><img src="image_95.png" /></figure>

Data preparation

<figure><img src="image_96.png" /></figure>

<figure><img src="image_97.png" /></figure>

Derived data

<figure><img src="image_98.png" /></figure>

Analysis

<figure><img src="image_99.png" /></figure>

<figure><img src="image_100.png" /></figure>

<figure><img src="image_101.png" /></figure>

Results
# Data integration

<figure><img src="image_102.png" /></figure>

Raw data

<figure><img src="image_103.png" /></figure>

Raw data

Data preparation

<figure><img src="image_104.png" /></figure>

Derived data

Analysis

<figure><img src="image_105.png" /></figure>

<figure><img src="image_106.png" /></figure>

Results
# Data integration

<figure><img src="image_107.png" /></figure>

Raw data

<figure><img src="image_108.png" /></figure>

Raw data

Data preparation

Data preparation

<figure><img src="image_109.png" /></figure>

Derived data

<figure><img src="image_110.png" /></figure>

Derived data

Analysis

<figure><img src="image_111.png" /></figure>

<figure><img src="image_112.png" /></figure>

Results
# Data integration

<figure><img src="image_113.png" /></figure>
# Data integration

<figure><img src="image_114.png" /></figure>
A new level of
variability
# Levels of variability

* Variability in **single-lab** studies

* Across subjects

* Across scans

<figure><img src="image_115.png" /></figure>
# Levels of variability

* Variability in **single-lab** studies

* Across subjects

* Across scans

* Variability in **multi-lab** studies

* All of the above plus

* Across machines

* Across sites

* Across acquisition protocols

<figure><img src="image_116.png" /></figure>
# Levels of variability

* Variability in **single-lab** studies

* Across subjects

* Across scans

* Variability in **multi-lab** studies

* All of the above plus

* Across machines

* Across sites

* Across acquisition protocols

* Variability in **data integration** studies

* All of the above plus

* Across preparation protocols

* Across analysis protocols

<figure><img src="image_117.png" /></figure>
# Levels of variability

* Variability in **single-lab** studies

* Across subjects

* Across scans

* Variability in **multi-lab** studies

* All of the above plus

* Across machines

* Across sites

* Across acquisition protocols

* Variability in **data integration** studies

* All of the above plus

* Across preparation protocols

* Across analysis protocols

<figure><img src="image_118.png" /></figure>

**Analytical Variability**

<figure><img src="image_119.png" /></figure>
# Analytic variability

<figure><img src="image_120.png" /></figure>

<figure><img src="image_121.png" /></figure>

Data preparation

<figure><img src="image_122.png" /></figure>

<figure><img src="image_123.png" /></figure>

Derived data

<figure><img src="image_124.png" /></figure>

Analysis

<figure><img src="image_125.png" /></figure>

<figure><img src="image_126.png" /></figure>

“Different acceptable analysis methods”

(Carp, Front. Neurosci. 2012)
Analytic variability

<figure><img src="image_127.png" /></figure>
# Analytic variability

<figure><img src="image_128.png" /></figure>
Analytic variability

<figure><img src="image_129.png" /></figure>
# Analytic variability

<figure><img src="image_130.png" /></figure>

<figure><img src="image_131.png" /></figure>
# Analytic variability

<figure><img src="image_132.png" /></figure>

(Pauli et al.,
Front. Neuroinform. 2016)
(Bowring et al. HBM 2019)
# Analytic variability

<figure><img src="image_133.png" /></figure>

(Gronenschild et al.,
Plos One 2012)

<figure><img src="image_134.png" /></figure>

≠ algorithm

<figure><img src="image_135.png" /></figure>

≠ software

<figure><img src="image_136.png" /></figure>

<figure><img src="image_137.png" /></figure>

≠ software version

<figure><img src="image_138.png" /></figure>
# Analytic variability

<figure><img src="image_139.png" /></figure>

<figure><img src="image_140.png" /></figure>

≠ algorithm

<figure><img src="image_141.png" /></figure>

≠ software

<figure><img src="image_142.png" /></figure>

≠ software version

<figure><img src="image_143.png" /></figure>

<figure><img src="image_144.png" /></figure>

<figure><img src="image_145.png" /></figure>

≠ parameters

<figure><img src="image_146.png" /></figure>
# Analytic variability

<figure><img src="image_147.png" /></figure>

(Glatard et al., Front. Neuroinform. 2015)

<figure><img src="image_148.png" /></figure>

≠ algorithm

<figure><img src="image_149.png" /></figure>

≠ software

<figure><img src="image_150.png" /></figure>

≠ software version

<figure><img src="image_151.png" /></figure>

≠ parameters

<figure><img src="image_152.png" /></figure>

≠ environment
Analytic variability

<figure><img src="image_153.png" /></figure>

<figure><img src="image_154.png" /></figure>

<figure><img src="image_155.png" /></figure>

<figure><img src="image_156.png" /></figure>

<figure><img src="image_157.png" /></figure>

<figure><img src="image_158.png" /></figure>
# Variations across software

* Reproduced 3 published functional MRI studies

* Using 3 different software

(Bowring et. al, HBM 2019)

<figure><img src="image_159.png" /></figure>

Alex Bowring Tom Nichols

Software 1 - AFNI

<figure><img src="image_160.png" /></figure>

Pipeline

<figure><img src="image_161.png" /></figure>

Software 2 - FSL

<figure><img src="image_162.png" /></figure>

Pipeline

<figure><img src="image_163.png" /></figure>

<figure><img src="image_164.png" /></figure>

Software 3

<figure><img src="image_165.png" /></figure>

Pipeline

<figure><img src="image_166.png" /></figure>

<figure><img src="image_167.png" /></figure>
# Variations across software

<figure><img src="image_168.png" /></figure>

Comparison of the final results

<figure><img src="image_169.png" /></figure>

Comparison of the statistic maps

* 1 dataset, 70 teams, 9 pre-defined research questions (yes/no)
(Botvinik-Nezer et. al, Nature 2020)
Sharing more

**research outputs**
# Sharing more research outputs

<figure><img src="image_170.png" /></figure>

Data preparation

<figure><img src="image_171.png" /></figure>

Derived data

Analysis

<figure><img src="image_172.png" /></figure>

<figure><img src="image_173.png" /></figure>

<figure><img src="image_174.png" /></figure>

BIDS
BRAIN IMAGING DATA STRUCTURE

<figure><img src="image_175.png" /></figure>

<figure><img src="image_176.png" /></figure>

<figure><img src="image_177.png" /></figure>

OpenfMRI

XNAT

<figure><img src="image_178.png" /></figure>

SHANOIR

<figure><img src="image_179.png" /></figure>

LORIS

<figure><img src="image_180.png" /></figure>

OpenNEURO

<figure><img src="image_181.png" /></figure>

HAL, BiorXiv
Pubmed

<figure><img src="image_182.png" /></figure>

brainmap.org

neurosynth.org

brainspell
# Sharing more research outputs

<figure><img src="image_183.png" /></figure>

Data preparation

<figure><img src="image_184.png" /></figure>

Derived data

<figure><img src="image_185.png" /></figure>

Analysis

<figure><img src="image_186.png" /></figure>

<figure><img src="image_187.png" /></figure>

BIDS
BRAIN IMAGING DATA STRUCTURE

<figure><img src="image_188.png" /></figure>

<figure><img src="image_189.png" /></figure>

OpenfMRI

<figure><img src="image_190.png" /></figure>

<figure><img src="image_191.png" /></figure>

<figure><img src="image_192.png" /></figure>

<figure><img src="image_193.png" /></figure>

SHANOIR

<figure><img src="image_194.png" /></figure>

OpenNEURO

NIDM

<figure><img src="image_195.png" /></figure>

<figure><img src="image_196.png" /></figure>

<figure><img src="image_197.png" /></figure>

brainmap.org

HAL, BiorXiv
Pubmed

neurosynth.org

brainspell

<figure><img src="image_198.png" /></figure>

NEUROVAULT

Image credits: Parcels 1 2 & 4 (CCO), Parcel 3 (CCO), Parcel 5 (CCO).
# Sharing more research outputs

<figure><img src="image_199.png" /></figure>

Image credits: Parcels 1 2 & 4 (CCO), Parcel 3 (CCO), Parcel 5 (CCO).
# BIDS Prov **Bridging BIDS & NIDM**

- my_dataset/

- participants.tsv

- sub-01/

- anat/

- sub-01_T1w.nii.gz

- func/

## BIDS

<figure><img src="image_200.png" /></figure>

File naming convention + JSON files
—
60 labs worldwide

<figure><img src="image_201.png" /></figure>

<figure><img src="image_202.png" /></figure>

## NIDM

<figure><img src="image_203.png" /></figure>

<figure><img src="image_204.png" /></figure>

Semantic web (RDF)
# BIDS Prov **Bridging BIDS & NIDM**

- my_dataset/

<figure><img src="image_205.png" /></figure>

- participants.tsv

- sub-01/

- anat/

<figure><img src="image_206.png" /></figure>

- sub-01_T1w.nii.gz

- func/

File naming convention + JSON files

60 labs worldwide

<figure><img src="image_207.png" /></figure>

BIDS Prov

<figure><img src="image_208.png" /></figure>

NIDM

<figure><img src="image_209.png" /></figure>

<figure><img src="image_210.png" /></figure>

Semantic web (RDF)
# BIDS Provenance

<figure><img src="image_211.png" /><figcaption>Satra Ghosh</figcaption></figure>

<figure><img src="image_212.png" /><figcaption>Rémi Adon</figcaption></figure>

<figure><img src="image_213.png" /></figure>

Looking for
contributors and
community input

https://github.com/bids-standard/BEP028_BIDSprov
References

**Botvinik-Nezer**, R., Holzmeister, F., Camerer, C. F., Dreber, A., Huber, J., Johannesson, ... Schonberg, T. (2020). Variability in the analysis of a single neuroimaging dataset by many teams. Nature, 582(7810), 84–88.
https://doi.org/10.1038/s41586-020-2314-9

**Bowring**, A., Maumet*, C., & Nichols*, T. E. (2019). Exploring the impact of analysis software on task fMRI results. Human Brain Mapping, 40(11), 3362–3384. https://doi.org/10.1002/hbm.24603

**Carp**, J. (2012). On the Plurality of (Methodological) Worlds: Estimating the Analytic Flexibility of fMRI Experiments. Frontiers in Neuroscience, 6. https://doi.org/10.3389/fnins.2012.00149

**Glatard**, T., Lewis, L. B., Ferreira da Silva, R., Adalat, R., Beck, ... Evans, A. C. (2015). Reproducibility of neuroimaging analyses across operating systems. Frontiers in Neuroinformatics, 9.
https://doi.org/10.3389/fninf.2015.00012

**Gorgolewski**, K. J., Auer, T., Calhoun, V. D., Craddock, R. C., Das, S., Duff, ... Poldrack, R. A. (2016). The brain imaging data structure, a format for organizing and describing outputs of neuroimaging experiments. Scientific Data, 3(1), 160044. https://doi.org/10.1038/sdata.2016.44

**Gronenschild**, E. H. B. M., Habets, P., Jacobs, H. I. L., Mengelers, R., Rozendaal, ... Marcelis, M. (2012). The Effects of FreeSurfer Version, Workstation Type, and Macintosh Operating System Version on Anatomical Volume and Cortical Thickness Measurements. https://doi.org/10.1371/journal.pone.0038234

**Nørgaard**, M., Ganz, M., Svarer, C., Frokjaer, V. G., Greve, ... Knudsen, G. M. (2020). Different preprocessing strategies lead to different conclusions: A [11C]DASB-PET reproducibility study. Journal of Cerebral Blood Flow & Metabolism, 40(9), 1902–1911. https://doi.org/10.1177/0271678X19880450

**Pauli**, R., Bowring, A., Reynolds, R., Chen, G., Nichols, T. E., & Maumet, C. (2016). Exploring fMRI Results Space: 31 Variants of an fMRI Analysis in AFNI, FSL, and SPM. Frontiers in Neuroinformatics, 10.
https://doi.org/10.3389/fninf.2016.00024

**Poldrack**, R. A., Baker, C. I., Durnez, J., Gorgolewski, K. J., Matthews, P. M., Munafò, M. R., Nichols, T. E., Poline, J.-B., Vul, E., & Yarkoni, T. (2017). Scanning the horizon: Towards transparent and reproducible neuroimaging research. Nature Reviews Neuroscience, 18(2), 115–126. https://doi.org/10.1038/nrn.2016.167
# Données ouvertes : comment permettre leur réutilisation?

Camille Maumet

Merci

<figure><img src="image_214.png" /><figcaption>Sculpture by Malin Bjornsdotter "Cerebia", OHBM Brain Art SIG</figcaption></figure>