FROG Analysis Ensures the Reproducibility of Genome Scale Metabolic Models
Authors: Raman Karthik , Kratochvíl Miroslav , Olivier Brett G. , König Matthias , Sengupta Pratyay , Kuppa Baskaran Dinesh Kumar , Nguyen Tung V N , Lobo Daniel , Wilken St Elmo , Tiwari Krishna Kumar , Raghu Aswathy K. , Palanikumar Indumathi , Raajaraam Lavanya , Ibrahim Maziya , Balakrishnan Sanjaay , Umale Shreyansh , Bergmann Frank , Malpani Tanisha , Satagopam Venkata P , Schneider Reinhard , Beber Moritz E. , Keating Sarah , Anton Mihail , Renz Alina , Lakshmanan Meiyappan , Lee Dong-Yup , Koduru Lokanand , Mostolizadeh Reihaneh , Dias Oscar , Cunha Emanuel , Oliveira Alexandre , Lee Yi Qing , Zengler Karsten , Santibáñez-Palominos Rodrigo , Kumar Manish , Barberis Matteo , Puniya Bhanwar Lal , Helikar Tomáš , Dinh Hoang V. , Suthers Patrick F. , Maranas Costas D. , Casini Isabella , Loghmani Seyed Babak , Veith Nadine , Leonidou Nantia , Li Feiran , Chen Yu , Nielsen Jens , Lee GaRyoung , Lee Sang Mi , Kim Gi Bae , Monteiro Pedro T. , Teixeira Miguel C. , Kim Hyun Uk , Lee Sang Yup , Liebal Ulf W. , Blank Lars M. , Lieven Christian , Tarzi Chaimaa , Angione Claudio , Blaise Manga Enuh , Aytar Çelik Pınar , Kulyashov Mikhail , Akberdin llya , Kim Dohyeon , Yoon Sung Ho , Xu Zhaohui , Gautam Jyotshana , Scott William T. , Schaap Peter J. , Koehorst Jasper J. , Zuñiga Cristal , Canto-Encalada Gabriela , Benito-Vaquerizo Sara , Olm Ivette Parera , Suarez-Diez Maria , Yuan Qianqian , Ma Hongwu , Islam Mohammad Mazharul , Papin Jason A. , Zorrilla Francisco , Patil Kiran Raosaheb , Basile Arianna , Nogales Juan , León Granado San , Castillo-Alfonso Freddy , Olivares-Hernández Roberto , Canto-Encalada Gabriela , Vigueras-Ramírez Gabriel , Hermjakob Henning , Dräger Andreas , Malik-Sheriff Rahuman S 
Published: September 2024

Genome-scale metabolic models (GEMs) and other constraint-based models (CBMs) play a pivotal role in understanding biological phenotypes and advancing research in areas like metabolic engineering, human disease modelling, drug discovery, and personalized medicine. Despite their growing application, a significant challenge remains in ensuring the reproducibility of GEMs, primarily due to inconsistent reporting and inadequate model documentation of model results. Addressing this gap, we introduce FROG analysis, a community-driven initiative aimed at standardizing reproducibility assessments of CBMs and GEMs. The FROG framework encompasses four key analyses—Flux variability, Reaction deletion, Objective function, and Gene deletion—to produce standardized, numerically reproducible FROG reports. These reports serve as reference datasets, enabling model evaluators, curators, and independent researchers to verify the reproducibility of GEMs systematically.

BioModels, a leading repository of systems biology models, has integrated FROG analysis into its curation workflow, enhancing the reproducibility and reusability of submitted GEMs. In our study evaluating 65 GEM submissions from the community, approximately 40% reproduced without intervention, 28% requiring minor adjustments, and 32% needing input from authors. The standardization introduced by FROG analysis facilitated the detection and resolution of issues, ultimately leading to the successful reproduction of all models. By establishing a standardized and comprehensive approach to evaluating GEM reproducibility, FROG analysis significantly contributes to making CBMs and GEMs more transparent, reusable, and reliable for the broader scientific community.