Tree-based classification model for Long-COVID infection prediction with age stratification using data from the National COVID Cohort Collaborative
Authors: Wang Will Ke , Jeong Hayoung , Hershkovich Leeor , Cho Peter , Singh Karnika , Lederer Lauren , Roghanizad Ali R , Shandhi Md Mobashir Hasan , Kibbe Warren , Dunn Jessilyn ,   National COVID Cohort Collaborative (N3C) Consortium, Wilcox Adam B , Lee Adam M , Graves Alexis , Anzalone Alfred (Jerrod) , Manna Amin , Saha Amit , Olex Amy , Zhou Andrea , Williams Andrew E , Southerland Andrew , Girvin Andrew T , Walden Anita , Sharathkumar Anjali A , Amor Benjamin , Bates Benjamin , Hendricks Brian , Patel Brijesh , Alexander Caleb , Bramante Carolyn , Ward-Caviness Cavin , Madlock-Brown Charisse , Suver Christine , Chute Christopher , Dillon Christopher , Wu Chunlei , Schmitt Clare , Takemoto Cliff , Housman Dan , Gabriel Davera , Eichmann David A , Mazzotti Diego , Brown Don , Boudreau Eilis , Hill Elaine , Zampino Elizabeth , Marti Emily Carlson , Pfaff Emily R , French Evan , Koraishy Farrukh M , Mariona Federico , Prior Fred , Sokos George , Martin Greg , Lehmann Harold , Spratt Heidi , Mehta Hemalkumar , Liu Hongfang , Sidky Hythem , Awori Hayanga J W , Pincavitch Jami , Clark Jaylyn , Harper Jeremy Richard , Islam Jessica , Ge Jin , Gagnier Joel , Saltz Joel H , Saltz Joel , Loomba Johanna , Buse John , Mathew Jomol , Rutter Joni L , McMurry Julie A , Guinney Justin , Starren Justin , Crowley Karen , Bradwell Katie Rebecca , Walters Kellie M , Wilkins Ken , Gersing Kenneth R , Cato Kenrick Dwain , Murray Kimberly , Kostka Kristin , Northington Lavance , Pyles Lee Allan , Misquitta Leonie , Cottrell Lesley , Portilla Lili , Deacy Mariam , Bissell Mark M , Clark Marshall , Emmett Mary , Saltz Mary Morrison , Palchuk Matvey B , Haendel Melissa A , Adams Meredith , Temple-O’Connor Meredith , Kurilla Michael G , Morris Michele , Qureshi Nabeel , Safdar Nasia , Garbarini Nicole , Sharafeldin Noha , Sadan Ofer , Francis Patricia A , Burgoon Penny Wung , Robinson Peter , Payne Philip R O , Fuentes Rafael , Jawa Randeep , Erwin-Cohen Rebecca , Patel Rena , Moffitt Richard A , Zhu Richard L , Kamaleswaran Rishi , Hurley Robert , Miller Robert T , Pyarajan Saiju , Michael Sam G , Bozzette Samuel , Mallipattu Sandeep , Vedula Satyanarayana , Chapman Scott , O’Neil Shawn T , Setoguchi Soko , Hong Stephanie S , Johnson Steve , Bennett Tellen D , Callahan Tiffany , Topaloglu Umit , Sheikh Usman , Gordon Valery , Subbian Vignesh , Kibbe Warren A , Hernandez Wenndy , Beasley Will , Cooper Will , Hillegass William , Zhang Xiaohan Tanner 
Published: October 2024

Objectives

We propose and validate a domain knowledge-driven classification model for diagnosing post-acute sequelae of SARS-CoV-2 infection (PASC), also known as Long COVID, using Electronic Health Records (EHRs) data.

Materials and Methods

We developed a robust model that incorporates features strongly indicative of PASC or associated with the severity of COVID-19 symptoms as identified in our literature review. The XGBoost tree-based architecture was chosen for its ability to handle class-imbalanced data and its potential for high interpretability. Using the training data provided by the Long COVID Computation Challenge (L3C), which was a sample of the National COVID Cohort Collaborative (N3C), our models were fine-tuned and calibrated to optimize Area Under the Receiver Operating characteristic curve (AUROC) and the F1 score, following best practices for the class-imbalanced N3C data.

Results

Our age-stratified classification model demonstrated strong performance with an average 5-fold cross-validated AUROC of 0.844 and F1 score of 0.539 across the young adult, mid-aged, and older-aged populations in the training data. In an independent testing dataset, which was made available after the challenge was over, we achieved an overall AUROC score of 0.814 and F1 score of 0.545.

Discussion

The results demonstrated the utility of knowledge-driven feature engineering in a sparse EHR data and demographic stratification in model development to diagnose a complex and heterogeneously presenting condition like PASC. The model’s architecture, mirroring natural clinician decision-making processes, contributed to its robustness and interpretability, which are crucial for clinical translatability. Further, the model’s generalizability was evaluated over a new cross-sectional data as provided in the later stages of the L3C challenge.

Conclusion

The study proposed and validated the effectiveness of age-stratified, tree-based classification models to diagnose PASC. Our approach highlights the potential of machine learning in addressing the diagnostic challenges posed by the heterogeneity of Long-COVID symptoms.