# From NIDM to BIDS-Prov: A personal perspective on 10 years of metadata standardization towards neuroimaging data sharing

Camille Maumet

► To cite this version:

Camille Maumet. From NIDM to BIDS-Prov: A personal perspective on 10 years of metadata standardization towards neuroimaging data sharing. Workshop "The future of FAIR data management and engineering for open and reproducible neuroimaging", Jul 2024, Oldenburg, Germany. hal-04651624

HAL Id: hal-04651624

https://inria.hal.science/hal-04651624v1

Submitted on 17 Jul 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
Session 2: Metadata standardization

<figure><img src="image_2.png" /></figure>

# From NIDM to BIDS-Prov

A personal perspective on 10 years of
metadata standardization towards
neuroimaging data sharing

Camille Maumet

Inria, Univ Rennes, Inserm, CNRS, IRISA
# 10 years of standards for data sharing...

<figure><img src="image_3.png" /></figure>

## Why sharing neuroimaging data?

Power of neuroscience studies

<figure><img src="image_4.png" /></figure>

(Button et. al, 2013)

## Small samples and low statistical power

Sample sizes in neuroimaging

<figure><img src="image_5.png" /></figure>

(Poldrack et. al, 2013)

High rate of **false positives** in the literature

<figure><img src="image_6.png" /></figure>

**Solution:** Data sharing to get to **larger samples**
<figure><img src="image_7.png" /></figure>

**incf** NeuroImaging Data Sharing (NIDASH) Task Force

<figure><img src="image_8.png" /></figure>

<figure><img src="image_9.png" /></figure>

<figure><img src="image_10.png" /></figure>

Members of NIDASH (2014)

**David Kennedy** (USA), **Jean-Baptiste Poline**
(France), **Cameron Craddock** (USA), **Guillaume Flandin**
(UK), **Stephan Gerhard** (Switzerland), **Chris Gorgolewski**
(UK), **Satrajit Ghosh** (USA), **Yaroslav Halchenko** (USA)
**Michael Hanke** (Germany), **Christian Haselgrove** (USA)

**David Keator** (USA) Arno Klein (USA) Daniel Marcus
(USA) Camille Maumet (UK) Franck Michel (France)
Nolan Nichols (USA) Thomas Nichols (UK) Simon
Milton (Australia) Russ Poldrack (USA) Rich Stoner
(USA) Jessica Turner (USA).
<table><thead><tr><th>2014</th><th>2015</th><th>2016</th><th>2017</th><th>2018</th><th>2019</th><th>2020</th><th>2021</th><th>2022</th><th>2023</th><th>2024</th></tr></thead><tbody><tr><td>★</td><td>★★★★</td><td>★★★</td><td>★★</td><td></td><td>★★★</td><td></td><td></td><td></td><td></td><td></td></tr></tbody></table>

incf NeuroImaging Data Sharing (NIDASH) Task Force

<figure><img src="image_11.png" /></figure>

<figure><img src="image_12.png" /></figure>

## Members of NIDASH (2014)

**David Kennedy** (USA), **Jean-Baptiste Poline** (France), **Cameron Craddock** (USA), **Guillaume Flandin** (UK), **Stephan Gerhard** (Switzerland), **Chris Gorgolewski** (UK), **Satrajit Ghosh** (USA), **Yaroslav Halchenko** (USA), **Michael Hanke** (Germany), **Christian Haselgrove** (USA)

**David Keator** (USA), **Arno Klein** (USA), **Daniel Marcus** (USA), **Camille Maumet** (UK), **Franck Michel** (France), **Nolan Nichols** (USA), **Thomas Nichols** (UK), **Simon Milton** (Australia), **Russ Poldrack** (USA), **Rich Stoner** (USA), **Jessica Turner** (USA).

<figure><img src="image_13.png" /></figure>
<table><thead><tr><th>2014</th><th>2015</th><th>2016</th><th>2017</th><th>2018</th><th>2019</th><th>2020</th><th>2021</th><th>2022</th><th>2023</th><th>2024</th></tr></thead><tbody><tr><td>★</td><td>★★★</td><td>★★★</td><td>★★★</td><td>★★★</td><td>★★★</td><td>■</td><td>■</td><td>■</td><td>■</td><td></td></tr></tbody></table>

incf NeuroImaging Data Sharing (NIDASH) Task Force

<figure><img src="image_14.png" /></figure>

Members of NIDASH (2014)

**David Kennedy** (USA), **Jean-Baptiste Poline** (France), **Cameron Craddock** (USA), **Guillaume Flandin** (UK), **Stephan Gerhard** (Switzerland), **Chris Gorgolewski** (UK), **Satrajit Ghosh** (USA), **Yaroslav Halchenko** (USA), **Michael Hanke** (Germany), **Christian Haselgrove** (USA)

**David Keator** (USA) Arno Klein (USA) Daniel Marcus (USA) Camille Maumet (UK) Franck Michel (France) Nolan Nichols (USA) Thomas Nichols (UK) Simon Milton (Australia) Russ Poldrack (USA) Rich Stoner (USA) Jessica Turner (USA).

<figure><img src="image_15.png" /></figure>

<figure><img src="image_16.png" /></figure>

W3C PROV

<figure><img src="image_17.png" /></figure>
<table><thead><tr><th>2014</th><th>2015</th><th>2016</th><th>2017</th><th>2018</th><th>2019</th><th>2020</th><th>2021</th><th>2022</th><th>2023</th><th>2024</th></tr></thead><tbody><tr><td>★</td><td>★★★★</td><td>★★★</td><td>★★★</td><td>★★★</td><td>★★★</td><td>□</td><td>□</td><td>□</td><td>□</td><td>□</td></tr></tbody></table>

<figure><img src="image_18.png" /></figure>

**incf** NeuroImaging Data Sharing (NIDASH) Task Force

<figure><img src="image_19.png" /></figure>

## Members of NIDASH (2014)

**David Kennedy** (USA), **Jean-Baptiste Poline** (France), **Cameron Craddock** (USA), **Guillaume Flandin** (UK), **Stephan Gerhard** (Switzerland), **Chris Gorgolewski** (UK), **Satrajit Ghosh** (USA), **Yaroslav Halchenko** (USA), **Michael Hanke** (Germany), **Christian Haselgrove** (USA)

**David Keator** (USA), **Arno Klein** (USA), **Daniel Marcus** (USA), **Camille Maumet** (UK), **Franck Michel** (France), **Nolan Nichols** (USA), **Thomas Nichols** (UK), **Simon Milton** (Australia), **Russ Poldrack** (USA), **Rich Stoner** (USA), **Jessica Turner** (USA).

<figure><img src="image_20.png" /></figure>

## NIDM Core Vocabulary

### PROV Family of Specifications

<figure><img src="image_21.png" /></figure>

<figure><img src="image_22.png" /></figure>

<figure><img src="image_23.png" /></figure>

[Keator et al, NeuroImage 2013]
<table><thead><tr><th>2014</th><th>2015</th><th>2016</th><th>2017</th><th>2018</th><th>2019</th><th>2020</th><th>2021</th><th>2022</th><th>2023</th><th>2024</th></tr></thead><tbody><tr><td>★</td><td>★★★★</td><td>★★★</td><td>★★★</td><td>★★★</td><td>★★★</td><td>★★</td><td>★★</td><td>★★</td><td>★★</td><td>★★</td></tr></tbody></table>

incf NeuroImaging Data Sharing (NIDASH) Task Force

<figure><img src="image_24.png" /></figure>

Members of NIDASH (2014)

**David Kennedy** (USA), **Jean-Baptiste Poline** (France), **Cameron Craddock** (USA), **Guillaume Flandin** (UK), **Stephan Gerhard** (Switzerland), **Chris Gorgolewski** (UK), **Satrajit Ghosh** (USA), **Yaroslav Halchenko** (USA), **Michael Hanke** (Germany), **Christian Haselgrove** (USA)

**David Keator** (USA) **Arno Klein** (USA) **Daniel Marcus** (USA) **Camille Maumet** (UK) **Franck Michel** (France) **Nolan Nichols** (USA) **Thomas Nichols** (UK) **Simon Milton** (Australia) **Russ Poldrack** (USA) **Rich Stoner** (USA) **Jessica Turner** (USA).

<figure><img src="image_25.png" /></figure>

NIDM Core Vocabulary

PROV Family of Specifications

<figure><img src="image_26.png" /></figure>

Semantic Web Technologies

<figure><img src="image_27.png" /></figure>

<figure><img src="image_28.png" /></figure>

<figure><img src="image_29.png" /></figure>

<figure><img src="image_30.png" /></figure>

<figure><img src="image_31.png" /></figure>

[Maumet et al, Sci Data 2016]

[Keator et al, NeuroImage 2013]
<table><thead><tr><th>2014</th><th>2015</th><th>2016</th><th>2017</th><th>2018</th><th>2019</th><th>2020</th><th>2021</th><th>2022</th><th>2023</th><th>2024</th></tr></thead><tbody><tr><td>★</td><td>★★★★</td><td>★★★</td><td>★★★</td><td>★★★</td><td>★★★</td><td>□</td><td>□</td><td>□</td><td>□</td><td>□</td></tr></tbody></table>

<figure><img src="image_32.png" /></figure>

NIDASH NeuroImaging Data Sharing (NIDASH) Task Force

<figure><img src="image_33.png" /></figure>

Members of NIDASH (2014)

**David Kennedy** (USA), **Jean-Baptiste Poline** (France), **Cameron Craddock** (USA), **Guillaume Flandin** (UK), **Stephan Gerhard** (Switzerland), **Chris Gorgolewski** (UK), **Satrajit Ghosh** (USA), **Yaroslav Halchenko** (USA), **Michael Hanke** (Germany), **Christian Haselgrove** (USA)

**David Keator** (USA) Arno Klein (USA) Daniel Marcus (USA) Camille Maumet (UK) Franck Michel (France) Nolan Nichols (USA) Thomas Nichols (UK) Simon Milton (Australia) Russ Poldrack (USA) Rich Stoner (USA) Jessica Turner (USA).

<figure><img src="image_34.png" /></figure>

Data Acquisition

<figure><img src="image_35.png" /></figure>

Image Processing

<figure><img src="image_36.png" /></figure>

Statistical Model

NIDM Core Vocabulary

PROV Family of Specifications

<figure><img src="image_37.png" /></figure>

SPARQL

Semantic Web Technologies

<figure><img src="image_38.png" /></figure>

<figure><img src="image_39.png" /></figure>

<figure><img src="image_40.png" /></figure>

<figure><img src="image_41.png" /></figure>

<figure><img src="image_42.png" /></figure>

[Maumet et al, Sci Data 2016]

[Keator et al, NeuroImage 2013]
10 years of standards for data sharing...

<table><thead><tr><th>2014</th><th>2015</th><th>2016</th><th>2017</th><th>2018</th><th>2019</th><th>2020</th><th>2021</th><th>2022</th><th>2023</th><th>2024</th></tr></thead><tbody><tr><td>★</td><td>★★★</td><td>★★★</td><td>★★</td><td>★</td><td>★★</td><td>□</td><td>□</td><td>□</td><td>□</td><td></td></tr></tbody></table>

<figure><img src="image_43.png" /></figure>
<table><thead><tr><th>2014</th><th>2015</th><th>2016</th><th>2017</th><th>2018</th><th>2019</th><th>2020</th><th>2021</th><th>2022</th><th>2023</th><th>2024</th></tr></thead><tbody><tr><td>★</td><td>★★★</td><td>★★★</td><td>★★★</td><td>★★★</td><td>★★★</td><td>■</td><td>■</td><td>■</td><td>■</td><td>■</td></tr></tbody></table>

<figure><img src="image_44.png" /></figure>

BIDS
BRAIN IMAGING DATA STRUCTURE

<figure><img src="image_45.png" /></figure>

## Brain Imaging Data Structure

<figure><img src="image_46.png" /></figure>

MRI-BIDS

[Gorgolewski et al, Sci Data 2016]

<figure><img src="image_47.png" /></figure>
<figure><img src="image_48.png" /></figure>

2014

2015

2016

2017

2018

2019

2020

2021

2022

2023

2024

<figure><img src="image_49.png" /></figure>

BIDS

BRAIN IMAGING DATA STRUCTURE

<figure><img src="image_50.png" /></figure>

<figure><img src="image_51.png" /></figure>

<figure><img src="image_52.png" /></figure>

<figure><img src="image_53.png" /></figure>

<figure><img src="image_54.png" /></figure>

## Brain Imaging Data Structure

First
Governance
For BIDS

<figure><img src="image_55.png" /></figure>

[Gorgolewski et al, Sci Data 2016]

iEEG-BIDS

(Holdgraf et al., 2019)

MEG-BIDS

(Niso et al., 2018)

EEG-BIDS

(Pernet et al., 2019)

Genetics
-BIDS

(Moreau et al., 2020)

qMRI-BIDS

(Karakuzu et al., 2022)

<figure><img src="image_56.png" /></figure>

ASL-BIDS

(Clement et al., 2022)

Microscopy
BIDS

(Bourget et al., 2022)

<figure><img src="image_57.png" /></figure>

PET-BIDS

(Norgaard et al., 2022)

<figure><img src="image_58.png" /></figure>

my_dataset/

<figure><img src="image_59.png" /></figure>

participants.tsv

<figure><img src="image_60.png" /></figure>

sub-01/

<figure><img src="image_61.png" /></figure>

anat/

<figure><img src="image_62.png" /></figure>

sub-01_T1w.nii.gz

<figure><img src="image_63.png" /></figure>

func/

<figure><img src="image_64.png" /></figure>

sub-01_task-rest bold.nii.gz

<figure><img src="image_65.png" /></figure>

sub-01_task-rest bold.json

<figure><img src="image_66.png" /></figure>

dwi/

<figure><img src="image_67.png" /></figure>

sub-01_dwi.nii.gz

<figure><img src="image_68.png" /></figure>

sub-01_dwi.json

<figure><img src="image_69.png" /></figure>

sub-01_dwi.bval

<figure><img src="image_70.png" /></figure>

sub-01_dwi.bvec

<figure><img src="image_71.png" /></figure>

sub-02/

>1 million BIDS-compliant images
in 2024

Source: [MRIQC Api](https://api.mriqc.org)
# 10 years of standards for data sharing...

<figure><img src="image_72.png" /></figure>

Why sharing neuroimaging data?

<figure><img src="image_73.png" /></figure>
# 10 years of standards for data sharing...

<figure><img src="image_74.png" /></figure>

2014

2015

2016

2017

2018

2019

2020

2021

2022

2023

2024

<figure><img src="image_75.png" /></figure>

## Why sharing neuroimaging data?

### Generalizability and fairness

**X-ray: Lung opacity detection**

Model trained on male images, tested on female images

<figure><img src="image_76.png" /></figure>

(Larrazabal et. al, 2020)

### Biased outcomes

<figure><img src="image_77.png" /></figure>

**Solution: Decentralized data sharing to get more diverse samples**
# 10 years of standards for data sharing...

<figure><img src="image_78.png" /></figure>

## Generalizability and fairness

X-ray: Lung opacity detection

Model trained on male images, tested on female images

<figure><img src="image_79.png" /></figure>

(Larrazabal et. al, 2020)

Biased outcomes

<figure><img src="image_80.png" /></figure>

**Solution:** Decentralized data sharing to get **more diverse samples**

## Why sharing neuroimaging data?

<figure><img src="image_81.png" /></figure>

## Different pipelines, different results

<figure><img src="image_82.png" /></figure>

(Botvinik-Nezer et. al, 2020)

Analytical variability

<figure><img src="image_83.png" /></figure>

**(Partial) Solution:** Transparency on the pipeline used
# Understanding analytical variability

?

<figure><img src="image_84.png" /></figure>
# Understanding analytical variability

?

<figure><img src="image_85.png" /></figure>

Numerical
uncertainty
(e.g. ≠ OS)

Unwanted
variability

Different
implen.
# Understanding analytical variability

?

<figure><img src="image_86.png" /></figure>

Numerical
uncertainty
(e.g. ≠ OS)

Assumption
violations

Unwanted
variability

Different
implem.

Unreliable
results

Data
induced
variability
# Understanding analytical variability

?

Better fit to the data

Variability of interest

Assumption violations

<figure><img src="image_87.png" /></figure>

Numerical uncertainty (e.g. ≠ OS)

Data induced variability

Unreliable results

Different implem.

Unwanted variability
# What is BIDS-Prov

<figure><img src="image_88.png" /></figure>
# What is BIDS-Prov

<figure><img src="image_89.png" /></figure>

my_dataset/

<figure><img src="image_90.png" /></figure>

participants.tsv

<figure><img src="image_91.png" /></figure>

sub-01/

<figure><img src="image_92.png" /></figure>

anat/

<figure><img src="image_93.png" /></figure>

sub-01_T1w.nii.gz

<figure><img src="image_94.png" /></figure>

func/

BIDS
"raw"

<figure><img src="image_95.png" /></figure>

<figure><img src="image_96.png" /></figure>

<figure><img src="image_97.png" /></figure>

my_dataset/

<figure><img src="image_98.png" /></figure>

participants.tsv

<figure><img src="image_99.png" /></figure>

sub-01/

<figure><img src="image_100.png" /></figure>

BIDS-derivatives
(NIDM-Results)

<figure><img src="image_101.png" /></figure>
# What is BIDS-Prov

<figure><img src="image_102.png" /></figure>

my_dataset/

<figure><img src="image_103.png" /></figure>

participants tsv

<figure><img src="image_104.png" /></figure>

sub-01/

<figure><img src="image_105.png" /></figure>

anat/

<figure><img src="image_106.png" /></figure>

sub-01_T1w.nii.gz

<figure><img src="image_107.png" /></figure>

func/

BIDS
"raw"

<figure><img src="image_108.png" /></figure>

<figure><img src="image_109.png" /></figure>

BIDS-Prov

* BIDS focus on metadata that **maximize (re)usability**

* Relies on expert knowledge

* For **derived/intermediary data**, we sometimes lack expert knowledge to identify the metadata that “matter” in neuroimaging pipelines and that will impact reusability.

<figure><img src="image_110.png" /></figure>

BIDS-derivatives
(NIDM-Results)

<figure><img src="image_111.png" /></figure>

my_dataset/

<figure><img src="image_112.png" /></figure>

participants tsv

<figure><img src="image_113.png" /></figure>

sub-01/

<figure><img src="image_114.png" /></figure>
# What is BIDS-Prov

<figure><img src="image_115.png" /></figure>

my_dataset/

<figure><img src="image_116.png" /></figure>

participants.tsv

<figure><img src="image_117.png" /></figure>

sub-01/

<figure><img src="image_118.png" /></figure>

anat/

<figure><img src="image_119.png" /></figure>

sub-01_T1w.nii.gz

<figure><img src="image_120.png" /></figure>

func/

BIDS
"raw"

<figure><img src="image_121.png" /></figure>

<figure><img src="image_122.png" /></figure>

JSON-LD

RDF +

<figure><img src="image_123.png" /></figure>

BIDS
BRAIN IMAGING DATA STRUCTURE

<figure><img src="image_124.png" /></figure>

* BIDS focus on metadata that **maximize (re)usability**

* Relies on expert knowledge

* For **derived/intermediary data**, we sometimes lack expert knowledge to identify the metadata that "matter" in neuroimaging pipelines and that will impact reusability.

<figure><img src="image_125.png" /></figure>

BIDS-derivatives
(NIDM-Results)

<figure><img src="image_126.png" /></figure>

my_dataset/

<figure><img src="image_127.png" /></figure>

participants.tsv

<figure><img src="image_128.png" /></figure>

sub-01/

<figure><img src="image_129.png" /></figure>
# Open discussion

* Transparency of data processing

* How to we keep the ease of use of BIDS while having large expressivity (many parameters)?

* Different levels of granularity...

* Or should we better understand analytical variability first to know which parameters are most impactful?

* Standards and large-scale data sharing efforts (ABCD, HCP, UK Biobank) not always aligned

* Decoupling metadata and data sharing is particularly important for the EU (GDPR)
Session 2: Metadata standardization

<figure><img src="image_130.png" /></figure>

From NIDM to BIDS-Prov

A personal perspective on 10 years of
metadata standardization towards
neuroimaging data sharing

Camille Maumet

Supported by: ANR JCJC VICUNA

AGENCE NATIONALE DE LA RECHERCHE

ANR

<figure><img src="image_131.png" /></figure>

@cmaumet.bsky.social

<figure><img src="image_132.png" /></figure>

@cmaumet@fediscience.org

<figure><img src="image_133.png" /></figure>

@cmaumet