Critical assessment of variant prioritization methods for rare disease diagnosis within the Rare Genomes Project
Authors: Stenton Sarah L. , O’Leary Melanie , Lemire Gabrielle , VanNoy Grace E. , DiTroia Stephanie , Ganesh Vijay S. , Groopman Emily , O’Heir Emily , Mangilog Brian , Osei-Owusu Ikeoluwa , Pais Lynn S. , Serrano Jillian , Singer-Berk Moriel , Weisburd Ben , Wilson Michael , Austin-Tse Christina , Abdelhakim Marwa , Althagafi Azza , Babbi Giulia , Bellazzi Riccardo , Bovo Samuele , Carta Maria Giulia , Casadio Rita , Coenen Pieter-Jan , De Paoli Federica , Floris Matteo , Gajapathy Manavalan , Hoehndorf Robert , Jacobsen Julius O.B. , Joseph Thomas , Kamandula Akash , Katsonis Panagiotis , Kint Cyrielle , Lichtarge Olivier , Limongelli Ivan , Lu Yulan , Magni Paolo , Mamidi Tarun Karthik Kumar , Martelli Pier Luigi , Mulargia Marta , Nicora Giovanna , Nykamp Keith , Pejaver Vikas , Peng Yisu , Pham Thi Hong Cam , Podda Maurizio S. , Rao Aditya , Rizzo Ettore , Saipradeep Vangala G , Savojardo Castrense , Schols Peter , Shen Yang , Sivadasan Naveen , Smedley Damian , Soru Dorian , Srinivasan Rajgopal , Sun Yuanfei , Sunderam Uma , Tan Wuwei , Tiwari Naina , Wang Xiao , Wang Yaqiong , Williams Amanda , Worthey Elizabeth A. , Yin Rujie , You Yuning , Zeiberg Daniel , Zucca Susanna , Bakolitsa Constantina , Brenner Steven E. , Fullerton Stephanie M , Radivojac Predrag , Rehm Heidi L. , O’Donnell-Luria Anne 
Published: August 2023

ABSTRACTBackgroundA major obstacle faced by rare disease families is obtaining a genetic diagnosis. The average “diagnostic odyssey” lasts over five years, and causal variants are identified in under 50%. The Rare Genomes Project (RGP) is a direct-to-participant research study on the utility of genome sequencing (GS) for diagnosis and gene discovery. Families are consented for sharing of sequence and phenotype data with researchers, allowing development of a Critical Assessment of Genome Interpretation (CAGI) community challenge, placing variant prioritization models head-to-head in a real-life clinical diagnostic setting.MethodsPredictors were provided a dataset of phenotype terms and variant calls from GS of 175 RGP individuals (65 families), including 35 solved training set families, with causal variants specified, and 30 test set families (14 solved, 16 unsolved). The challenge tasked teams with identifying the causal variants in as many test set families as possible. Ranked variant predictions were submitted with estimated probability of causal relationship (EPCR) values. Model performance was determined by two metrics, a weighted score based on rank position of true positive causal variants and maximum F-measure, based on precision and recall of causal variants across EPCR thresholds.ResultsSixteen teams submitted predictions from 52 models, some with manual review incorporated. Top performing teams recalled the causal variants in up to 13 of 14 solved families by prioritizing high quality variant calls that were rare, predicted deleterious, segregating correctly, and consistent with reported phenotype. In unsolved families, newly discovered diagnostic variants were returned to two families following confirmatory RNA sequencing, and two prioritized novel disease gene candidates were entered into Matchmaker Exchange. In one example, RNA sequencing demonstrated aberrant splicing due to a deep intronic indel inASNS, identified intranswith a frameshift variant, in an unsolved proband with phenotype overlap with asparagine synthetase deficiency.ConclusionsBy objective assessment of variant predictions, we provide insights into current state-of-the-art algorithms and platforms for genome sequencing analysis for rare disease diagnosis and explore areas for future optimization. Identification of diagnostic variants in unsolved families promotes synergy between researchers with clinical and computational expertise as a means of advancing the field of clinical genome interpretation.