Using SED-ML for reproducible curation: Verifying BioModels across multiple simulation engines
Authors: Smith Lucian , Malik-Sheriff Rahuman S. , Nguyen Tung V. N. , Hermjakob Henning , Karr Jonathan , Shaikh Bilal , Drescher Logan , Moraru Ion I. , Schaff James C. , Agmon Eran , Patrie Alexander A. , Blinov Michael L. , Hellerstein Joseph L. , May Elebeoba E. , Nickerson David P. , Gennari John H. , Sauro Herbert M. 
Published: January 2025

The BioModels Repository contains over 1000 manually curated mechanistic models drawn from published literature, most of which are encoded in the Systems Biology Markup Language (SBML). This community-based standard formally specifies each model, but does not describe the computational experimental conditions to run a simulation. Therefore, it can be challenging to reproduce any given figure or result from a publication with an SBML model alone. The Simulation Experiment Description Markup Language (SED-ML) provides a solution: a standard way to specify exactly how to run a specific experiment that corresponds to a specific figure or result. BioModels was established years before SED-ML, and both systems evolved over time, both in content and acceptance. Hence, only about half of the entries in BioModels contained SED-ML files, and these files reflected the version of SED-ML that was available at the time. Additionally, almost all of these SED-ML files had at least one minor mistake that made them invalid. To make these models and their results more reproducible, we report here on our work updating, correcting and providing new SED-ML files for 1055 curated mechanistic models in BioModels. In addition, because SED-ML is implementation-independent, it can be used forverification, demonstrating that results hold across multiple simulation engines. Here, we use a wrapper architecture for interpreting SED-ML, and report verification results across five different ODE-based biosimulation engines. Our work with SED-ML and the BioModels collection aims to improve the utility of these models by making them more reproducible and credible.

Author summary

Reproducing computationally-derived scientific results seems like it should be straightforward, but is often elusive. Code is lost, file formats change, and knowledge of what was done is only partially recorded and/or forgotten. Model repositories such as BioModels address this failing in the Systems Biology domain by encoding models in a standard format that can reproduce a figure from the paper from which it was drawn. Here, we delved into the BioModels repository to ensure that every curated model additionally contained instructions on what to do with that model, and then tested those instructions on a variety of simulation platforms. Not only did this improve the BioModels repository itself, but also improved the infrastructure necessary to run these validation comparisons in the future.

Author contributions

LS: Writing, Conceptualization, Data Curation, Investigation, Methodology, Project Administration, Software, Validation. RMS: Reading, Writing, Data Curation, Methodology TN: Reading, Data Curation, Methodology HH: Reading JK: Conceptualization, Data Curation, Investigation, Methodology, Software. BS: Software LD: Software IIM: Reading, Conceptualization, Funding JCS: Software, Methodology EA: Reading, Writing AAP: Software MLB: Reading, Writing JH: Writing, Methodology EM: Reading, Writing DPN: Reading, Writing, Methodology JG: Reading, Writing, Methodology HMS: Reading, Writing, Funding