Effectiveness, Explainability and Reliability of Machine Meta-Learning Methods for Predicting Mortality in Patients with COVID-19: Results of the Brazilian COVID-19 Registry
Authors: Delfino-Pereira Polianna , De Andrade Cláudio Moisés Valiense , Gomes Virginia Mara Reis , Lima Maria Clara Pontello Barbosa , Souza-Silva Maira Viana Rego , Carneiro Marcelo , Martins Karina Paula Medeiros Prado , Sales Thaís Lorenna Souza , De Carvalho Rafael Lima Rodrigues , Pires Magda Carvalho , Ramos Lucas Emanuel Ferreira , Silva Rafael T , Bezerra Adriana Falangola Benjamin , Schwarzbold Alexandre Vargas , Nunes Aline Gabrielle Sousa , Maurilio Amanda de Oliveira , Scotton Ana Luiza Bahia Alves , Costa André Soares de Moura , Castro Andriele Abreu , Farace Bárbara Lopes , Cimini Christiane Corrêa Rodrigues , De Carvalho Cíntia Alcântara , Silveira Daniel Vitorio , Ponce Daniela , Pereira Elayne Crestani , Manenti Euler Roberto Fernandes , Cenci Evelin Paola de Almeida , Lucas Fernanda Barbosa , Rodrigues Fernanda d’Athayde , Anschau Fernando , Botoni Fernando Antônio , Aranha Fernando Graça , Bartolazzi Frederico , Bastos Gisele Alsina Nader , Vietta Giovanna Grunewald , Nascimento Guilherme Fagundes , Noal Helena Carolina , Duani Helena , Vianna Heloísa Reniers , Guimarães Henrique Cerqueira , Gomes Isabela Moraes , Costa Jamille Hemerito Salles Martins , Fonseca Jessica Rayane Corrêa Silva Da , Guimarães Júlia Di Sabatino Santos , De Morais Júlia Drumond Parreiras , Rugolo Juliana Machado , Batista Joanna d’Arc Lyra , De Alvarenga Joice Coutinho , Chatkin José Miguel , Ruschel Karen Brasil , Moreira Leila Beltrami , De Oliveira Leonardo Seixas , Zandona Liege Barella , Pinheiro Lilian Santos , Monteiro Luanna da Silva , Sousa Lucas de Deus , Kopittke Luciane , Viana Luciano de Souza , De Castro Luís César , Assis Luísa Argolo , Santos Luísa Elem Almeida , Cabral Maderson Álvares de Souza , Raposo Magda Cesar , Floriani Maiara Anschau , Ferreira Maria Angélica Pires , Bicalho Maria Aparecida Camargos , De Godoy Mariana Frizzo , Nogueira Matheus Carvalho Alves , De Figueiredo Meire Pereira , Júnior Milton Henriques Guimarães , De Sordi Monica Aparecida de Paula , Sampaio Natália da Cunha Severino , De Oliveira Neimy Ramos , Assaf Pedro Ledic , Lutkmeier Raquel , Valacio Reginaldo Aparecido , Finger Renan Goulart , Menezes Rochele Mosmann , Silva Rufino de Freitas , Francisco Saionara Cristina , Guimaraes Silvana Mangeon Meireles , Araujo Silvia Ferreira , Oliveira Talita Fischer , Kurtz Tatiana , Fereguetti Tatiana Oliveira , De Oliveira Thainara Conceição , Diniz Túlio Henrique Oliveira , Ribeiro Yara Cristina Neves Marques Barbosa , Ramires Yuri Carlotto , Gonçalves Marcos André , Marcolino Milena Soriano , de Paiva Bruno Barbosa Miranda 
Published: January 2022

Abstract
The majority prognostic scores proposed for early assessment of coronavirus disease 19 (COVID-19) patients are bounded by methodological flaws. Our group recently developed a new risk score - ABC2SPH - using traditional statistical methods (least absolute shrinkage and selection operator logistic regression - LASSO). In this article, we provide a thorough comparative study between modern machine learning (ML) methods and state-of-the-art statistical methods, represented by ABC2SPH, in the task of predicting in-hospital mortality in COVID-19 patients using data upon hospital admission. We overcome methodological and technological issues found in previous similar studies, while exploring a large sample (5,032 patients). Additionally, we take advantage of a large and diverse set of methods and investigate the effectiveness of applying meta-learning, more specifically Stacking, in order to combine the methods' strengths and overcome their limitations. In our experiments, our Stacking solutions improved over previous state-of-the-art by more than 26% in predicting death, achieving 87.1% of AUROC and MacroF1 of 73.9%. We also investigated issues related to the interpretability and reliability of the predictions produced by the most effective ML methods. Finally, we discuss the adequacy of AUROC as an evaluation metric for highly imbalanced and skewed datasets commonly found in health-related problems.