# A systematic and large-scale exploration of analytical variability in task-fMRI

Youenn Merel Jourdan, Mathieu Acher, Camille Maumet

► To cite this version:

Youenn Merel Jourdan, Mathieu Acher, Camille Maumet. A systematic and large-scale exploration of analytical variability in task-fMRI. OHBM 2026, Organization for Human Brain Mapping (OHBM), Jun 2026, Bordeaux, France. hal-05458481

HAL Id: hal-05458481

https://inria.hal.science/hal-05458481v1

Submitted on 14 Jan 2026

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
# A systematic and large-scale exploration of analytical variability in task-fMRI

Youenn Merel Jourdan¹, Mathieu Acher², Camille Maumet¹

1. Univ Rennes, Inria, CNRS, Inserm, Rennes, France

2. CNRS, INSA Rennes, IUF, IRISA, Inria, Rennes, France

## Introduction

Configurability of neuroimaging pipelines is essential for tailoring the analyses to different experimental conditions but also gives rise to *analytical variability*: i.e. distinct pipelines, equally valid from a scientific point of view, may produce different findings [1]. Previous works have explored the sources of this variability typically by studying the variability induced (in the results) by a small set of pre-defined pipeline variations [2] – such as the choice of software, operating systems or preprocessing methods.

In this work, we leverage methods from software engineering [3] to systematically explore analytical variability from preprocessing up to group analysis in a multi-task fMRI dataset. We investigate 20 parameters – including interpolation algorithm, coregistration cost function, number of motion regressors – expressing a total of over 27,000 distinct pipelines. We propose a method to study variability in this large space without any assumptions made on the expected results or target downstream task. We separate variability of interest (differences observed across valid results) from unwanted variability (differences linked to pipelines producing unacceptable or low quality results) while minimizing the required amount of visual quality control. Finally, among the valid pipelines, we identify the parameters that have the strongest impact on the final results.

## Methods

**Pipeline modeling** We create a *feature model*, a formal representation of all the possible pipelines with their associated *parameters* and (range of) *values*. Each fMRI pipeline is composed of: motion correction, slice timing correction, coregistration, segmentation and normalisation followed by a subject-level analysis and a one-sample t-test group-level analysis. The configuration space expressed by our model encompass 27,648 distinct pipelines. We focus on pipelines of the SPM software package [4].

**Pipelines execution** We select 30 subjects in the Human Connectome Project Young Adult dataset [5] and study the "motor" (right hand vs rest) and "emotion" (fear vs neutral) tasks. First, we process the data using a pipeline with: motion correction to mean image, coregistering structural on functional with normalised mutual correlation, non linear normalisation with b-spline interpolation, slice timing with middle slice, smoothing at 6mm, HRF with 1 derivatives, 24 motion regressors and modeling of autocorrelation using FAST. This pipeline was crafted in order to serve as a reference in the remainder of the article. Then, we run 504 randomly sampled pipelines from the feature model separately on the two task-fMRI tasks. To control the variation in the degree of signal mixing, we lock spatial smoothing FWHM at 6 mm for all pipelines. We perform a quality control of the subject-level results by visually inspecting the statistic maps – of the reference pipeline and of 10 randomly chosen pipelines – and comparing to Neurosynth meta-analyses ('fear' and 'motor') – seen here as a ground-truth. This lead to the exclusion of 1 subject.

**Filtering out invalid pipelines** We compute Pearson's r against the statistic map of the reference pipeline – our reference – and between each pairs of maps. We apply hierarchical clustering using scipy complete linkage on pairwise correlation distance (1 - r) matrices and extract clusters by applying a threshold t that represent the maximum distance allowed between statistic maps in the same cluster. We chose t = 0.20, meaning that distance between two statistic maps in the same given cluster cannot exceed 0.20. For each cluster, we visually inspected a representative statistic map – the medoid, i.e. the most centrally located statistic map in each cluster – and classify each cluster as valid or
invalid. Finally, we fit a classifier decision tree model to predict validity class (i.e. valid/invalid) of a statistic map from the pipeline that produced it.

**Analytical variability investigation** We apply hierarchical clustering a second time on the pairwise correlation matrices, keeping only the valid statistic maps identified in the previous step. We set a $t = 0.10$ threshold, separating the statistics maps into clusters based on this distance criterion. We apply a classifier decision tree to predict the cluster membership of any statistic maps based on the values of its pipeline parameters. Finally, we compute a weight that measures the level of contribution of each parameter to the final results, a process known as *feature importance* – that provides a measure of the impact of each parameter on the classifier predictions. Each parameter that has a feature importance greater than zero is considered having a significant, visible role in the pipeline statistic map.

## Results

**Filtering out invalid pipelines** We get 4 clusters for motor and 7 for emotion. After visual inspection of each cluster, we identify 2 invalid clusters for the motor task and 5 for emotion. In all cases, it is visible misalignment – probable registration failure – that lead us to classify a cluster as invalid (see Fig.2.a.). The pipeline that are identified as invalid, are the same across task-fMRI tasks. Indeed, we observe that pipeline classification between valid and invalid can be explained by one coregistration parameter: use of the normalised cross correlation cost function which induces a misalignment. This is in-line with SPM manual that specifies that this parameter value is indicated for within modality co-registration – while we coregister functional and structural images.

We examine the distribution of pipelines in the 2 classes (valid/invalid) with respect to their correlation to the reference pipeline – the medoid is used as a representant for each cluster (Fig.2.b.). As expected, invalid pipelines have the smallest correlation to the reference. We further observe a strict separation between valid versus invalid pipelines for emotion, while for motor there is some overlap. While correlation to the reference gives us a good indication of the quality of a cluster of maps, it is not sufficient by itself to discriminate between valid and invalid pipelines.

**Analytical variability investigation** We get 4 clusters for each fMRI tasks and identify 4 parameters in total that drive cluster memberships: 1/ the number of motion regressors in signal modeling (both tasks), 2/ the HRF model (motor), 3/ the use of slice timing correction (emotion) and 4/ the choice of reference slice (emotion). Pipeline with no motion regressor are grouped in one cluster (Fig. 2.c., Cluster 1) and pipeline with 6 motion regressors in a separate cluster (Cluster 2) – with the exact same clustering of pipelines for both fMRI tasks. Then, when using 18 or 24 motion regressors, an additional – task-dependent – parameter determines membership to the remaining clusters. For motor, pipelines using the canonical HRF are in one cluster (Cluster 3 [motor]) while those using 1 or 2 derivatives are grouped separately (Cluster 4 [motor]). For emotion, pipelines using the first slice as slice timing correction reference are in one cluster (Cluster 3 [emotion]), while pipelines using no slice timing correction or the middle slice as reference are grouped together in a separate cluster (Cluster 4 [emotion]). In addition, we looked for each cluster at the correlation of their representative map to result of the reference pipeline. We observe that for both tasks the farthest cluster from the reference uses no motion regressors, the obtained correlations are very close across tasks (0.79 for motor and 0.81 for emotion).

## Conclusions

We propose a large-scale exploration of task-fMRI analytical variability across two fMRI tasks of the HCP Young Adult dataset. We consistently excluded invalid pipelines in both tasks by visually inspecting only a few representative images instead of the whole sample, and identified the cause for invalid results. Finally, we found 3 parameters of interest that explain results variability among valid pipelines.

For future contributions, we intend to refine our model, progressing towards a more representative pipeline space. In particular, we plan to extend the model with new tools (e.g. AFNI, FSL, fMRIPrep), algorithms and parameters. We will strengthen our study of inter-task variability by incorporating new paradigms from the HCP (e.g. gambling) and
by exploring new dataset (e.g. NARPS [1]). Furthermore, introducing variations in the size and composition of subject groups will allow us to integrate inter-subject variability.

## Acknowledgments

YMJ, MA and CM were supported by ANR JCJC VICUNA (ANR-23-CE45-0022). YMJ was also supported by Region Bretagne (ARED GUANACO).

Data were provided by the Human Connectome Project, WU-Minn Consortium (Principal Investigators: David Van Essen and Kamil Ugurbil; 1U54MH091657) funded by the 16 NIH Institutes and Centers that support the NIH Blueprint for Neuroscience Research; and by the McDonnell Center for Systems Neuroscience at Washington University.

Experiments presented in this paper were carried out using the Abaca testbed, supported by Inria (see https://abaca.inria.fr).

## References

[1] Botvinik-Nezer Rotem, Holzmeister Felix, Camerer Colin F., et al. Variability in the analysis of a single neuroimaging dataset by many teams *Nature*. 2020;582:84–88.

[2] Dafflon Jessica, F. Da Costa Pedro, Váša František, et al. A guided multiverse study of neuroimaging analyses 2022;13:3758. Number: 1 Publisher: Nature Publishing Group.

[3] Pereira Juliana Alves, Acher Mathieu, Martin Hugo, Jézéquel Jean-Marc, Botterweck Goetz, Ventresque Anthony. Learning software configuration spaces: A systematic literature review ;182:111044.

[4] Penny William D, Friston Karl J, Ashburner John T, Kiebel Stefan J, Nichols Thomas E. *Statistical parametric mapping: the analysis of functional brain images*. Elsevier 2006.

[5] Van Essen David C, Smith Stephen M, Barch Deanna M, et al. The WU-Minn Human Connectome Project: An overview *NeuroImage*. 2013;80:62–79.
<figure><img src="image_2.png" /></figure>

## 1. Pipeline modeling

* Model SPM pipelines

* 20 params / 27 000+ pipelines

* Sample 504 pipelines

* Build 1 reference pipeline

<table><thead><tr><th>slice_timmingCorrection/ ref_slice/middle</th><th>signal_modeling/hrf/ temporal Dispersión derivs</th><th>spatial_smothing/ hwhm</th><th>signal_modeling/hrf/ canonical</th></tr></thead><tbody><tr><td>False</td><td>False</td><td>0</td><td>True</td></tr><tr><td>False</td><td>False</td><td>12</td><td>False</td></tr><tr><td>...</td><td>...</td><td>...</td><td>...</td></tr><tr><td>True</td><td>False</td><td>2</td><td>False</td></tr><tr><td>True</td><td>False</td><td>6</td><td>False</td></tr></tbody></table>

## 2. Pipelines execution

* Run pipelines on HCP t-fMRI data

* 30 subjects

* 2 task-fMRI tasks (emotion, motor)

* Compute statistic maps Pearson correlations

* pipeline-to-pipeline

* to reference pipeline

<figure><img src="image_3.png" /></figure>

<figure><img src="image_4.png" /></figure>

## 3. Filtering out invalid pipelines

* Cluster statistic maps

* Classify clusters as valid or invalid

<figure><img src="image_5.png" /></figure>

<figure><img src="image_6.png" /></figure>

## 4. Analytic variability investigation

* Cluster valid statistic maps

* Compute

* feature importances

* decision tree

<figure><img src="image_7.png" /></figure>

<figure><img src="image_8.png" /></figure>

**Fig.1.** Method. **1.** We sample 504 SPM pipeline out of 27,648 pipelines, and build one pipeline as reference. **2.** We run these pipelines on 2 HCP task-fMRI dataset and 30 subjects; we compute Pearson correlations of statistic maps **3.** Using hierarchical clustering we cluster the statistic maps by their distance; we visually check clusters to classify them as valid or invalid. **4.** We reapply clustering on valid statistic maps; we use a classifier tree to predict cluster membership of any statistic maps based on the values of pipeline parameters; we measure the level of contribution of each pipeline parameter with feature importances.
<figure><img src="image_9.png" /><figcaption>**Fig. 2.** Results. **a.** Examples of valid and invalid statistic maps for both tasks. **b.** Distribution of correlations to the reference pipeline by clusters; correlation of medoids is given for each cluster. **c.** Decision tree presenting how parameters of interest interacts with each other to explain variability for both tasks; clusters 1 & 2 are common to both tasks, while cluster 3 & 4 are different between tasks; correlation of medoids to the reference pipeline is given under each cluster; selected parameters in reference pipeline are labeled with 'ref.'.</figcaption></figure>