CAGI, the Critical Assessment of Genome Interpretation, establishes progress and prospects for computational genetic variant interpretation methods
Authors: Shantanu Jain, Constantina Bakolitsa, Steven E. Brenner, Predrag Radivojac, John Moult, Susanna Repo, Roger A. Hoskins, Gaia Andreoletti, Daniel Barsky, Ajithavalli Chellapan, Hoyin Chu, Navya Dabbiru, Naveen K. Kollipara, Melissa Ly, Andrew J. Neumann, Lipika R. Pal, Eric Odell, Gaurav Pandey, Robin C. Peters-Petrulewicz, Rajgopal Srinivasan, Stephen F. Yee, Sri Jyothsna Yeleswarapu, Maya Zuhl, Ogun Adebali, Ayoti Patra, Michael A. Beer, Raghavendra Hosur, Jian Peng, Brady M. Bernard, Michael Berry, Shengcheng Dong, Alan P. Boyle, Aashish Adhikari, Jingqi Chen, Zhiqiang Hu, Robert Wang, Yaqiong Wang, Maximilian Miller, Yanran Wang, Yana Bromberg, Paola Turina, Emidio Capriotti, James J. Han, Kivilcim Ozturk, Hannah Carter, Giulia Babbi, Samuele Bovo, Pietro Di Lena, Pier Luigi Martelli, Castrense Savojardo, Rita Casadio, Melissa S. Cline, Greet De Baets, Sandra Bonache, Orland Díez, Sara Gutiérrez-Enríquez, Alejandro Fernández, Gemma Montalban, Lars Ootes, Selen Özkan, Natàlia Padilla, Casandra Riera, Xavier De la Cruz, Mark Diekhans, Peter J. Huwe, Qiong Wei, Qifang Xu, Roland L. Dunbrack, Valer Gotea, Laura Elnitski, Gennady Margolin, Piero Fariselli, Ivan V. Kulakovskiy, Vsevolod J. Makeev, Dmitry D. Penzar, Ilya E. Vorontsov, Alexander V. Favorov, Julia R. Forman, Marcia Hasenahuer, Maria S. Fornasari, Gustavo Parisi, Ziga Avsec, Muhammed H. Çelik, Thi Yen Duong Nguyen, Julien Gagneur, Fang-Yuan Shi, Matthew D. Edwards, Yuchun Guo, Kevin Tian, Haoyang Zeng, David K. Gifford, Jonathan Göke, Jan Zaucha, Julian Gough, Graham R. S. Ritchie, Adam Frankish, Jonathan M. Mudge, Jennifer Harrow, Erin L. Young, Yao Yu, Chad D. Huff, Katsuhiko Murakami, Yoko Nagai, Tadashi Imanishi, Christopher J. Mungall, Julius O. B. Jacobsen, Dongsup Kim, Chan-Seok Jeong, David T. Jones, Mulin Jun Li, Violeta Beleva Guthrie, Rohit Bhattacharya, Yun-Ching Chen, Christopher Douville, Jean Fan, Dewey Kim, David Masica, Noushin Niknafs, Sohini Sengupta, Collin Tokheim, Tychele N. Turner, Hui Ting Grace Yeo, Rachel Karchin, Sunyoung Shin, Rene Welch, Sunduz Keles, Yue Li, Manolis Kellis, Carles Corbi-Verge, Alexey V. Strokach, Philip M. Kim, Teri E. Klein, Rahul Mohan, Nicholas A. Sinnott-Armstrong, Michael Wainberg, Anshul Kundaje, Nina Gonzaludo, Angel C. Y. Mak, Aparna Chhibber, Hugo Y. K. Lam, Dvir Dahary, Simon Fishilevich, Doron Lancet, Insuk Lee, Benjamin Bachman, Panagiotis Katsonis, Rhonald C. Lua, Stephen J. Wilson, Olivier Lichtarge, Rajendra R. Bhat, Laksshman Sundaram, Vivek Viswanath, Riccardo Bellazzi, Giovanna Nicora, Ettore Rizzo, Ivan Limongelli, Aziz M. Mezlini, Ray Chang, Serra Kim, Carmen Lai, Robert O’Connor, Scott Topper, Jeroen van den Akker, Alicia Y. Zhou, Anjali D. Zimmer, Gilad Mishne, Timothy R. Bergquist, Marcus R. Breese, Rafael F. Guerrero, Yuxiang Jiang, Nikki Kiga, Biao Li, Matthew Mort, Kymberleigh A. Pagel, Vikas Pejaver, Moses H. Stamboulian, Janita Thusberg, Sean D. Mooney, Nuttinee Teerakulkittipong, Chen Cao, Kunal Kundu, Yizhou Yin, Chen-Hsin Yu, Michael Kleyman, Chiao-Feng Lin, Mary Stackpole, Stephen M. Mount, Gökcen Eraslan, Nikola S. Mueller, Tatsuhiko Naito, Aliz R. Rao, Johnathan R. Azaria, Aharon Brodie, Yanay Ofran, Aditi Garg, Debnath Pal, Alex Hawkins-Hooker, Henry Kenlay, John Reid, Eliseos J. Mucaki, Peter K. Rogan, Jana M. Schwarz, David B. Searls, Gyu Rie Lee, Chaok Seok, Andreas Krämer, Sohela Shah, ChengLai V. Huang, Jack F. Kirsch, Maxim Shatsky, Yue Cao, Haoran Chen, Mostafa Karimi, Oluwaseyi Moronfoye, Yuanfei Sun, Yang Shen, Ron Shigeta, Colby T. Ford, Conor Nodzak, Aneeta Uppal, Xinghua Shi, Thomas Joseph, Sujatha Kotte, Sadhna Rana, Aditya Rao, V. G. Saipradeep, Naveen Sivadasan, Uma Sunderam, Mario Stanke, Andrew Su, Ivan Adzhubey, Daniel M. Jordan, Shamil Sunyaev, Frederic Rousseau, Joost Schymkowitz, Joost Van Durme, Sean V. Tavtigian, Marco Carraro, Manuel Giollo, Silvio C. E. Tosatto, Orit Adato, Liran Carmel, Noa E. Cohen, Tzila Fenesh, Tamar Holtzer, Tamar Juven-Gershon, Ron Unger, Abhishek Niroula, Ayodeji Olatubosun, Jouni Väliaho, Yang Yang, Mauno Vihinen, Mary E. Wahl, Billy Chang, Ka Chun Chong, Inchi Hu, Rui Sun, William Ka Kei Wu, Xiaoxuan Xia, Benny C. Zee, Maggie H. Wang, Meng Wang, Chunlei Wu, Yutong Lu, Ken Chen, Yuedong Yang, Christopher M. Yates, Anat Kreimer, Zhongxia Yan, Nir Yosef, Huying Zhao, Zhipeng Wei, Zhaomin Yao, Fengfeng Zhou, Lukas Folkman, Yaoqi Zhou, Roxana Daneshjou, Russ B. Altman, Fumitaka Inoue, Nadav Ahituv, Adam P. Arkin, Federica Lovisa, Paolo Bonvini, Sarah Bowdin, Stefano Gianni, Elide Mantuano, Velia Minicozzi, Leonore Novak, Alessandra Pasquo, Annalisa Pastore, Maria Petrosino, Rita Puglisi, Angelo Toto, Liana Veneziano, Roberta Chiaraluce, Mad P. Ball, Jason R. Bobe, George M. Church, Valerio Consalvi, David N. Cooper, Bethany A. Buckley, Molly B. Sheridan, Garry R. Cutting, Maria Chiara Scaini, Kamil J. Cygan, Alger M. Fredericks, David T. Glidden, Christopher Neil, Christy L. Rhine, William G. Fairbrother, Aileen Y. Alontaga, Aron W. Fenton, Kenneth A. Matreyek, Lea M. Starita, Douglas M. Fowler, Britt-Sabina Löscher, Andre Franke, Scott I. Adamson, Brenton R. Graveley, Joe W. Gray, Mary J. Malloy, John P. Kane, Maria Kousi, Nicholas Katsanis, Max Schubach, Martin Kircher, Angel C. Y. Mak, Paul L. F. Tang, Pui-Yan Kwok, Richard H. Lathrop, Wyatt T. Clark, Guoying K. Yu, Jonathan H. LeBowitz, Francesco Benedicenti, Elisa Bettella, Stefania Bigoni, Federica Cesca, Isabella Mammi, Cristina Marino-Buslje, Donatella Milani, Angela Peron, Roberta Polli, Stefano Sartori, Franco Stanzial, Irene Toldo, Licia Turolla, Maria C. Aspromonte, Mariagrazia Bellini, Emanuela Leonardi, Xiaoming Liu, Christian Marshall, W. Richard McCombie, Lisa Elefanti, Chiara Menin, M. Stephen Meyn, Alessandra Murgia, Kari C. Y. Nadeau, Susan L. Neuhausen, Robert L. Nussbaum, Mehdi Pirooznia, James B. Potash, Dago F. Dimster-Denk, Jasper D. Rine, Jeremy R. Sanford, Michael Snyder, Atina G. Cote, Song Sun, Marta W. Verby, Jochen Weile, Frederick P. Roth, Ryan Tewhey, Pardis C. Sabeti, Joan Campagna, Marwan M. Refaat, Julianne Wojciak, Soren Grubb, Nicole Schmitt, Jay Shendure, Amanda B. Spurdle, Dimitri J. Stavropoulos, Nephi A. Walton, Peter P. Zandi, Elad Ziv, Wylie Burke, Flavia Chen, Lawrence R. Carr, Selena Martinez, Jodi Paik, Julie Harris-Wai, Mark Yarborough, Stephanie M. Fullerton, Barbara A. Koenig, Gregory McInnes, Dustin Shigaki, John-Marc Chandonia, Mabel Furutsuki, Laura Kasak, Changhua Yu, Rui Chen, Iddo Friedberg, Gad A. Getz, Qian Cong, Lisa N. Kinch, Jing Zhang, Nick V. Grishin, Alin Voskanian, Maricel G. Kann, Elizabeth Tran, Nilah M. Ioannidis, Jesse M. Hunter, Rupa Udani, Binghuang Cai, Alexander A. Morgan, Artem Sokolov, Joshua M. Stuart, Giovanni Minervini, Alexander M. Monzon, Serafim Batzoglou, Atul J. Butte, Marc S. Greenblatt, Reece K. Hart, Ryan Hernandez, Tim J. P. Hubbard, Scott Kahn, Anne O’Donnell-Luria, Pauline C. Ng, John Shon, Joris Veltman, Justin M. Zook
Journal: Genome biology (2024)

Abstract Background The Critical Assessment of Genome Interpretation (CAGI) aims to advance the state-of-the-art for computational prediction of genetic variant impact, particularly where relevant to disease. The five complete editions of the CAGI community experiment comprised 50 challenges, in which participants made blind predictions of phenotypes from genetic data, and these were evaluated by independent assessors. Results Performance was particularly strong for clinical pathogenic variants, including some difficult-to-diagnose cases, and extends to interpretation of cancer-related variants. Missense variant interpretation methods were able to estimate biochemical effects with increasing accuracy. Assessment of methods for regulatory variants and complex trait disease risk was less definitive and indicates performance potentially suitable for auxiliary use in the clinic. Conclusions Results show that while current methods are imperfect, they have major utility for research and clinical applications. Emerging methods and increasingly large, robust datasets for training and assessment promise further progress ahead.