dataesr/scientific-paragraphs-categorization huggingface dataset:
, 2006). Also,     (Zimmermann et al. , 2021). This study also has some limitations. First, the current study used validation statistics and assessment of the developmental course in the same dataset. However, Virtual Class differs from School-Cam mainly in terms of classroom design and avatars graphics. Second, despite a large sample size (N = 83), we identified a transition threshold of approximately 9 years; in addition, the large variance in the second cluster suggests that a larger sample size is needed to determine how age, sex, or cognitive functions may influence the development of PT. For example,.
This was confirmed by power spectral densities, which showed a clear peak at 4 Hz (corresponding to the theta drive) and increases in gamma band activity between 40 and 80 Hz (Figure  2B ). To quantify PAC between theta and gamma oscillations, we used the modulation index (MI)  (Tort et al. , 2008 (Tort et al. , , 2010)) , which has been shown to outperform other similar measures  (Hülsemann et al. , 2019). However, we discovered that this metric would give erroneous results in our simulated datasets due to the absence of certain frequency components. To overcome this limitation and avoid artifacts, uniform noise was added to the firing rates prior to computing the MI (Figure  S3 ). We first visualized the quantification of PAC using the comodulogram, which indicates the MI as a function of two frequencies corresponding to the modulating phase signal and the modulated amplitude signal.
Numerical Implementation The model was implemented with the Brian2 libraries for Python  (Stimberg et al. , 2019)  (see Data availability section for access to the code). Simulations were performed using a timestep of 0. 1 ms and a total simulation duration ranging between 3-10 seconds (depending on the experiment, see results section). The average time for simulating 1 second for the complete model locally was 10 minutes.
Data Analysis During each simulation, we monitored and exported the following data for subsequent analysis: (i) spike timings per neuron, (ii) time series of ionic currents, (iii) input theta rhythm and phase, and (iv) time series of electrical stimulation. Theta phase was wrapped between [-π, π] with a phase of 0 radians corresponding to the peak of theta rhythm. All PAC analyses were performed using the TensorPAC toolbox for Python  (Combrisson et al. , 2020).
Funding This work was supported by the  Bordeaux Neurocampus junior chair program  co-funded by the  Région Nouvelle-Aquitaine  and the  University of Bordeaux Initiative of Excellence (IdEx)  (F. B. W. , Neurocampus junior chair), and the  European Union 's  Horizon Europe research and innovation program  under  European Research Council (ERC)   Starting Grant  agreement N°1  01040391  (F. B. W. , MEMOPROSTHETICS).
Reducing contributions from non-neuronal sources is a crucial step in functional magnetic resonance imaging (fMRI) connectivity analyses. Many viable strategies for denoising fMRI are used in the literature, and practitioners rely on denoising benchmarks for guidance in the selection of an appropriate choice for their study. However, fMRI denoising software is an ever-evolving field, and the benchmarks can quickly become obsolete as the techniques or implementations change. In this work, we present a denoising benchmark featuring a range of denoising strategies, datasets and evaluation metrics for connectivity analyses, based on the popular fMRIprep software. The benchmark is implemented in a fully reproducible framework, where the provided research objects enable readers to reproduce or modify core computations, as well as the figures of the article using the Jupyter Book project and the Neurolibre reproducible preprint server (https://neurolibre.
org/). We demonstrate how such a reproducible benchmark can be used for continuous evaluation of research software, by comparing two versions of the fMRIprep software package. The majority of benchmark results were consistent with prior literature. Scrubbing, a technique which excludes time points with excessive motion, combined with global signal regression, is generally effective at noise removal. Scrubbing however disrupts the continuous sampling of brain images and is incompatible with some statistical analyses, e. g. auto-regressive modeling. In this case, a simple strategy using motion parameters, average activity in select brain compartments, and global signal regression should be preferred. Importantly, we found that certain denoising strategies behave inconsistently across datasets and/or versions of fMRIPrep, or had a different behavior than in previously published benchmarks.
Denoising benchmarks are however at a constant risk of becoming obsolete, with new strategies being regularly developed or revised, as well as an ever-expanding scope of populations being enrolled in research studies. The main objective of the present work is to develop a fully reproducible denoising benchmark for fMRI functional connectivity, and demonstrate how the proposed infrastructure enables the continuous evaluation of denoising strategies across multiple software versions and datasets. Reproducible and robust results have become a recurring interest in the neuroimaging community  (Botvinik-Nezer et al. , 2020; Niso et al. , 2022). The popular package fMRIPrep  (Esteban et al. , 2019)  is a prominent solution for fMRI preprocessing designed with reproducibility in mind, and we decided to build upon that software for our benchmark.
The main references for denoising benchmarks  (Ciric et al. 2017 , Parker et al. , 2018)  did not use the then-novel fMRIPrep. Whether the results of these benchmarks remain consistent with fMRIPrep outputs is an open question. Different fMRI preprocessing softwares provide largely similar results, but noticeable differences are still present  (Bowring et al. , 2019; Li et al. , 2021). Other computational factors can possibly impact the conclusion of a benchmark, such as the version of software and operating system  (Gronenschild et al. , 2012). Recent research has also demonstrated that, given one fMRI dataset and similar research goals, different researchers will select a wide variety of possible analytical paths  (Botvinik-Nezer et al. , 2020). The lack of standard integration between fMRIPrep and Nilearn could lead to differences (and errors) in the implementation of the same denoising strategies by researchers, which can in turn lead to marked differences in the impact of denoising methods.
9. 0, can be used to flexibly retrieve a subset of fMRIPrep confounds for denoising and precisely replicate nuisance regressors based on denoising strategies proposed in the literature. • Our second objective was to implement a denoising benchmark to provide recommendations on the choice of functional connectivity denoising strategies for fMRIPrep users. We used easily fetchable open access data, specifically two datasets on OpenNeuro  (Markiewicz et al. , 2021)  with diverse participant profiles: ds000228  (Richardson et al. , 2019)  and ds000030  (Bilder et al. , 2020). ds000228 contains adult and child samples, and ds000030 includes psychiatric conditions. The benchmark systematically evaluates the impact of denoising choices using a series of metrics based on past research  (Ciric et al.
Denoising workflow The denoising workflow is implemented through Nilearn. Figure  1  presents the graphic summary of the workflow. An fMRI dataset in the Brain Imaging Data Structure (BIDS) standard was first passed to fMRIPrep. Brain parcellation atlases were retrieved through the TemplateFlow  (Ciric et al. , 2022)  Python client (see https://www. templateflow. org/usage/client/). In cases where an atlas was absent from TemplateFlow, it was converted into TemplateFlow naming convention to enable use of the Python client. Each atlas was passed to the NiftiLabelsMasker or NiftiMapsMasker for time series extraction. fMRIPrep outputs were input to a Nilearn-based connectome generating workflow using load_confounds_strategy. The filtered confounds and the corresponding preprocessed NIFTI images were then passed to the Nilearn masker generated with the atlas.
Benchmark workflow OpenNeuro datasets were retrieved through DataLad  (Halchenko et al. , 2021)  and fMRIPrep images were pulled from DockerHub. SLURM job submission scripts to process the fMRI data were generated with the Python tool fMRIPrep-SLURM (https://github. com/SIMEXP/fmriprep-slurm). The fMRIPrep derivatives and atlas retrieved from the TemplateFlow archive were passed to the connectome workflow described in Figure  1. We extracted the signals using a range of atlases at various resolutions (see Materials and Methods for details). For each parcellation scheme and each fMRI dataset, 10 sets of time series were generated, including one baseline and 9 different denoising strategies (see Table  2 ). We report the quality metrics and break down the effect on each dataset, preprocessed with fMRIPrep 20.
2. 1 long-term support branch (LTS). Motion characteristics were also generated per dataset and used to exclude fMRI runs with excessive motion from entering the benchmark. Trends in each atlas were similar, so we combined all atlases for the following report. The detailed breakdown by parcellation scheme can be found in the associated Jupyter Book  (Wang et al. , 2023). Figure  2  presents a graphical summary of the benchmark workflow. Benchmark results from fMRIPrep 20. 2. 1 LTS We reported the demographic information and the gross mean framewise displacement before and after excluding subjects with high motion. We then aimed to assess the overall similarity between connectomes generated from each denoising strategy, and evaluated the denoising strategies using four metrics from Ciric and colleagues' benchmark (2017): 1.
Loss of degrees of freedom: sum of number of regressors used and number of volumes censored. 2. Quality control / functional connectivity (QC-FC;  Power et al. , 2015) : partial correlation between motion and connectivity with age and sex as covariates. 3. Distance-dependent effects of motion on connectivity (DM-FC  Power et al. , 2012) : correlation between node-wise Euclidean distance and QC-FC. 4. Network modularity  (Satterthwaite et al. , 2012) : graph community detection based on Louvain method, implemented in the Brain Connectome Toolbox. Significant differences in motion levels existed both between datasets, and within-dataset, across clinical and demographic subgroups We applied a motion threshold to exclude subjects with marked motion in the two OpenNeuro datasets: dataset ds000228 (N=155)  (Richardson et al.
, 2019)  and dataset ds000030 (N=212)  (Bilder et al. , 2020). Table  3  shows the demographic information of subjects in each dataset after the automatic motion quality control. Following this, we checked the difference in the mean framewise displacement of each sample and the sub-groups (Figure  3 ). In ds000228, there was still a significant difference (t(73) = -2. 17, p = 0. 033) in motion during the scan captured by mean framewise displacement between the child (M = 0. 17, SD = 0. 05, n = 51) and adult samples (M = 0. 15, SD = 0. 04, n = 24). In ds000030, the only patient group that showed a difference compared to control subjects (M = 0. 12, SD = 0. 04, n = 88) was the schizophrenia group (M = 0. 16, SD = 0. 05, n = 19; t(105) = -3. 49, p = 0. 033). There was no difference between the control and ADHD group (M = 0.
12, SD = 0. 05, n = 32; t(118) = 0. 04, p = 0. 966), or the bipolar group (M = 0. 13, SD = 0. 05, n = 29; t(115) = -1. 24, p = 0. 216). In summary, children moved more than adults , and subjects with schizophrenia moved more than controls. We also examined the differences between male and female in the control groups of the two datasets: the adult sample for ds000228 and healthy control for ds000030. In ds000228, we found no significant differences (male: M = 0. 16 , SD = 0. 04; female: M = 0. 14, SD = 0. 05; t(22) = 1. 19, p = 0. 249). In ds000030 we found the male sample (M = 0. 13, SD = 0. 04) showed higher mean framewise displacement than the female sample (M = 0. 11, SD = 0. 04; t(86) = 2. 17, p = 0. 033). Due to the imbalanced samples per group and low number of subjects in certain groups after the automatic motion quality control, we collapsed all groups within each dataset to avoid speculation on underpowered samples in the results.
For a breakdown of each metric by atlas, please see the supplemental Jupyter Book 3. To evaluate the metrics in a practical analytic scenario, we excluded subjects with high motion while preserving 1 minute of data for functional connectivity calculation: gross mean framewise displacement > 0. 25 mm, above 80. 0% of volumes removed while scrubbing with a 0. 2 mm threshold. In ds000228, the child group still had higher motion compared to the adult groups. In ds000030, where all subjects were adults, the control group only showed significant differences in motion with the schizophrenia group. In both datasets, the sample sizes from each group were highly imbalanced (see Table  3 ), hence no between group differences were assessed in quality metrics analysis. Most denoising strategies converged on a consistent average connectome structure With the benchmark workflow in place, we first aimed to assess the overall similarity between connectomes generated from each denoising strategy.
We calculated Pearson's correlations between connectomes generated from all strategies presented in the benchmark (Figure  4 ). The connectome correlation pattern across denoising strategies was similar in both datasets. Overall, the strategies displayed at least moderate similarity with each other, with Pearson's correlations above 0. 6. There were two large clusters of highly-related strategies, driven by the presence (or lack) of global signal regression. Within each cluster of strategies, the correlations amongst the strategies were strong, with values above 0. 9. baseline and aroma did not fit well in either of the two clusters, indicating that denoising generally impacts the connectome structure, and that the ICA-AROMA might be sensitive to different sources of noise, compared to those captured by other strategies in the benchmark.
Loss in temporal degrees of freedom varied markedly across strategies and datasets In previous research, the loss of temporal degrees of freedom has shown an impact on the subsequent data analysis. Higher loss in temporal degrees of freedom can spuriously increase functional connectivity  (Yan et al. , 2013). Volume censoring-based and data-driven strategies (ICA-AROMA and some variations of CompCor) introduce variability to degrees of freedom and can bias group level comparisons  (Ciric et al. , 2017). The loss of temporal degrees of freedom is the sum of the number of regressors used and censored volume lost. Depending on the length of the scan, the number of discrete cosine-basis regressors can differ given the same repetition time (TR). 5. The loss of degrees of freedom per strategy varied across the two datasets shown in the benchmark.
The two datasets showed different loss of degrees of freedom in scrubbing-based strategies, while using the same gross motion-based exclusion criterias. This was expected, as the amount of motion between time points was higher in ds000228, a dataset consisting mostly of children. compcor did not always have a lower loss of degrees of freedom in ds000030. aroma had the lowest loss of temporal degrees of freedom in ds000030. Quality control / functional connectivity (QC-FC) showed a heterogeneous impact of denoising strategies based on data profile The denoising methods should aim to reduce the impact of motion on the data. To quantify the remaining impact of motion in connectomes, we adopted a metric proposed by  Power and colleagues (2015)  named quality control / functional connectivity (QC-FC).
QC-FC is a partial correlation between mean framewise displacement and functional connectivity, with age and sex as covariates. Significance tests associated with the partial correlations were performed. P-values above the threshold of were deemed significant. α = 0. 05 Scrubbing-based strategies consistently performed better than the baseline in both datasets. In ds000228, the most effective method according to QC-FC was scrubbing. 5 (scrubbing at a liberal threshold), followed by scrubbing. 2 and simple. All the GSR counterparts of the methods had slightly higher residual motion. Amongst all the data-driven methods, compcor performed the best. compcor6 and aroma performed close to baseline. In ds000030, the best performing method was compcor, followed by scrubbing. 2 (aggressive scrubbing).
The simple and scrubbing. 5 methods performed similarly as very few volumes were censored with a liberal threshold, and the GSR variations (simple+gsr and scrubbing. 5+gsr) performed better than baseline (see Figure  6 ). simple performed close to the baseline in terms of the number of edges correlated with motion. The aroma and compcor6 strategies were better than baseline. The average percentage of significant QC-FC and the average median of absolute value of QC-FC are presented in Figure  6  and Figure  7. In summary, based on a QC-FC evaluation, diverse strategies performed quite differently based on the dataset used for evaluation.
Scrubbing-based strategies decreased distance-dependent effects of motion The impact of motion on functional connectivity has been reported to be higher for brain parcels closer to each other in space  (Power et al. , 2012). To determine the residual distance-dependent effects of subject motion on functional connectivity (DM-FC), we calculated a correlation between the Euclidean distance between the centers of mass of each pair of parcels  (Power et al. , 2012)  and the corresponding QC-FC correlations. We reported the absolute DM-FC correlation values and expected to see a general trend toward zero correlation after denoising. All strategies performed better than the baseline in both datasets (Figure  8 ). We observed a trend consistent across both datasets, whereby strategies scrubbing. 2 and scrubbing.
2+gsr were the most effective in reducing the correlation. aroma also performed consistently well in both datasets, ranked after scrubbing. 2. In ds000228, simple was the least effective strategy for reducing distance dependency. Data-driven methods showed similar results to each other. scrubbing. 5 and simple greatly benefited from adding GSR in the regressors. In ds000030, the difference between scrubbing. 2 and other strategies was bigger than in ds000228, with the remainder performed similarly with each other. The impact of GSR was small with the exception of scrubbing. 2+gsr. In summary, we observed similar trends across strategies between the two datasets, yet with differences in the magnitude of correlations. All strategies reduced the correlation lower than the baseline. Consistent with the literature, scrubbing strategies were the best at reducing distance dependency.
Global signal regression increases network modularity Confound regressors have the harmful potential to remove real signals of interest as well as motion-related noise. To evaluate this possibility, we examined the impact of denoising strategies on a common graph feature, network modularity, generally regarded as a key feature of biological network organization  (Satterthwaite et al. , 2012). Network modularity was quantified using the Louvain method for graph community detection  (Rubinov & Sporns, 2010). We computed the partial correlation between subjects' modularity values and mean framewise displacement, using age and sex as covariates, following the implementation of  Power and colleagues (2015). The inclusion of global signal regressors increased average Louvain network modularity in both datasets (Figure  9 , top panel).
The remaining strategies performed as follows in both datasets, from best to worst: compcor, scrubbing. 2, scrubbing. 5, simple, compcor6, and aroma. In both datasets, aroma performed almost at the similar level as the baseline. We found fixed results in the ability of denoising in reducing the impact of motion on modularity (Figure  9  lower panels). In ds000228, we see simple and scrubbing. 5 reducing the impact of motion. In ds000030, only scrubbing. 2 performed better than baseline. In both datasets, the data-driven strategies and strategies with GSR performed consistently worse than baseline. The overall trend across strategies is similar to QC-FC with the exception of the baseline strategy (see Figure  6  and 7 ). The reason behind this observation could be a reduction of variance in the Louvain network modularity metric for GSR-based denoising strategies.
4. scrubbing. 2 and scrubbing. 2+gsr were the best strategies to reduce DM-FC. 5. GSR-enabled strategies showed higher network modularity. Observations 1, 4, and 5 from 20. 2. 5 LTS were consistent with results from 20. 2. 1 LTS. The results of QC-FC demonstrated similar overall trends in 20. 2. 5 LTS, but with aroma performing worse than baseline for ds000228 (observation 2) and simple performing better than baseline for ds000030 (observation 3) (see Figure  11 ). Inconsistency in outcomes across the two fMRIPrep versions were found in strategies with data-driven noise components. In version 20. 2. 5 LTS, and unlike 20. 2. 1 LTS, comcpor6 performed worse than the baseline in metric QC-FC for both datasets. In ds000228, aroma was the second worst performing strategy. For ds000030, the strategies with no data-driven noise components showed better performance in 20.
Denoising strategy In order to summarize our results, we created a table ranking strategies from best to worst, based on four benchmark metrics, across datasets and fMRIPrep versions (see Figure  12 ). The ranking of the loss of temporal degrees of freedom is an important consideration accompanying the remaining metrics, as any denoising strategy aims at a particular trade-off between the amount of noise removed and the preservation of degrees of freedom for signals. Aside from the loss of temporal degrees of freedom, the baseline strategy consistently performs the worst, as expected, with the notable exception of aroma performing worst on QC-FC. Figure  12. Ranking of all denoising strategies. We ranked four metrics from best to worst. Larger circles with brighter color represent higher ranking.
Metric "correlation between network modularity and motion" has been excluded from the summary as it is potentially a poor measure. Loss of temporal degrees of freedom is a crucial measure that should be taken into account alongside the metric rankings. The simple+gsr strategy is not the best for any particular individual evaluation metric, but it performed consistently well across metrics, datasets and software versions. The loss in degrees of freedom simple (26 + number of cosine terms) and simple+gsr (27+number of cosine terms) used slightly more regressors than aroma, and had markedly lesser loss than scrubbing methods. simple+gsr is consistently better than other data-driven strategies, which makes it the best choice for analysis that requires low loss of degrees of freedom and also preserve continuous sampling time series (which is broken by scrubbing).
Scrubbing based strategies are the best when it comes to minimizing the impact of motion, with a cost of higher loss in degrees of freedom. We found that scrubbing with an aggressive 0. 2 mm threshold (scrubbing. 2) mitigates distance dependency well consistently, regardless of the group of subjects. Despite excluding data with the same standard on both datasets, the child-dominant sample (ds000228) showed more volumes censored with the scrubbing strategy, and a liberal framewise displacement threshold showed sufficient ability to reduce the distance dependency of motion as observed in the original study of the strategy  (Power et al. , 2012). In a sample with higher motion, such as ds000228, a liberal scrubbing threshold reduced the impact of motion and performed similarly with a higher threshold.