FAIRly big: A framework for reproducible processing of large-scale data
Creators: name: Wagner, Adina S
name: Waite, Laura K
name: Małgorzata   Wierzba
name: Hoffstaedter, Felix
name: Waite, Alexander Q
name: Poldrack, Benjamin
name: Eickhoff, Simon B
name: Hanke, Michael
Publisher: name: F1000 Research Limited
Year: 2022

description: This poster was presented at the Organization of Human Brain Mapping conference 2022 in Glasgow.<br> <br> The original conference abstract was:<br> "Datasets, such as the Human Connectome Project, the Adolescent Brain Cognitive Development Study, or the UK Biobank project contain heterogenous data that scales to millions of files and hundreds of terabytes. This large-scale data presents unique research opportunities, but also challenges. Storage and computational demands strain hardware capabilities. As large-scale computations are more difficult to reproduce, comprehend, and verify, the trustworthiness of derivative data decreases. Moreover, restrictive software licenses as well as analytical flexibility can impede transparent analysis provenance in the first place, and privacy considerations prohibit the open distribution of sensitive data. Yet, with large scale datasets, sharing and reusing derivatives emerges as the most viable way to extend previous work. It minimizes duplicate efforts to perform resource-heavy computations with considerable environmental impact, and it can open up research on large data to scholars who do not have access to adequate computational resources. In such contexts, data should thus not only be as findable, accessible, interoperable, and reusable (FAIR) as possible, but also handled in a sustainable manner that places data sharing and reuse as a priority. Here, we present a DataLad-based framework to reproducibly process large-scale datasets, and share results and provenance as openly as the responsible use of sensitive data or legal requirements permit.<br> We built a scalable processing framework using open source software for version control, containerization, and workload management, and workflows from distributed software development. To test its scalability and portability, we performed voxel-based morphometry (VBM) on N = 42,715 participants of the UK Biobank Imaging dataset using the Computational Anatomy Toolbox (CAT) on a mid-sized computational cluster and a modular supercomputer. We used machine actionable provenance generated by the framework for automatic recomputations on infrastructure as small as a personal laptop, and performed structured investigations of variability between recomputations. To demonstrate the framework’s potential for open and transparent sharing,, we created a fully open showcase with open data and open source software.<br> The framework was portable across systems and their employed workload managers, and recomputation of results based on provenance records succeeded on different hardware. In an investigation of result variability between recomputations we found high congruence, with more than 50% of all output files being identical across computations. Remaining variability was largely attributable to minor numerical differences. We complemented the framework implementation with open tutorials and materials available at github.com/psychoinformatics-de/fairly-big-processing-workflow."<br> <br> Sources for the poster and a virtual version can be found at https://github.com/adswa/ohbm-2022