Potential and limitations of machine meta-learning (ensemble) methods for predicting COVID-19 mortality in a large inhospital Brazilian dataset
Authors: de Paiva Bruno Barbosa Miranda , Pereira Polianna Delfino , de Andrade Claudio Moisés Valiense , Gomes Virginia Mara Reis , Souza-Silva Maira Viana Rego , Martins Karina Paula Medeiros Prado , Sales Thaís Lorenna Souza , de Carvalho Rafael Lima Rodrigues , Pires Magda Carvalho , Ramos Lucas Emanuel Ferreira , Silva Rafael Tavares , de Freitas Martins Vieira Alessandra , Nunes Aline Gabrielle Sousa , de Oliveira Jorge Alzira , de Oliveira Maurílio Amanda , Scotton Ana Luiza Bahia Alves , da Silva Carla Thais Candida Alves , Cimini Christiane Corrêa Rodrigues , Ponce Daniela , Pereira Elayne Crestani , Manenti Euler Roberto Fernandes , Rodrigues Fernanda d’Athayde , Anschau Fernando , Botoni Fernando Antônio , Bartolazzi Frederico , Grizende Genna Maira Santos , Noal Helena Carolina , Duani Helena , Gomes Isabela Moraes , Costa Jamille Hemétrio Salles Martins , di Sabatino Santos Guimarães Júlia , Tupinambás Julia Teixeira , Rugolo Juliana Machado , Batista Joanna d’Arc Lyra , de Alvarenga Joice Coutinho , Chatkin José Miguel , Ruschel Karen Brasil , Zandoná Liege Barella , Pinheiro Lílian Santos , Menezes Luanna Silva Monteiro , de Oliveira Lucas Moyses Carvalho , Kopittke Luciane , Assis Luisa Argolo , Marques Luiza Margoto , Raposo Magda Cesar , Floriani Maiara Anschau , Bicalho Maria Aparecida Camargos , Nogueira Matheus Carvalho Alves , de Oliveira Neimy Ramos , Ziegelmann Patricia Klarmann , Paraiso Pedro Gibson , de Lima Martelli Petrônio José , Senger Roberta , Menezes Rochele Mosmann , Francisco Saionara Cristina , Araújo Silvia Ferreira , Kurtz Tatiana , Fereguetti Tatiani Oliveira , de Oliveira Thainara Conceição , Ribeiro Yara Cristina Neves Marques Barbosa , Ramires Yuri Carlotto , Lima Maria Clara Pontello Barbosa , Carneiro Marcelo , Bezerra Adriana Falangola Benjamin , Schwarzbold Alexandre Vargas , de Moura Costa André Soares , Farace Barbara Lopes , Silveira Daniel Vitorio , de Almeida Cenci Evelin Paola , Lucas Fernanda Barbosa , Aranha Fernando Graça , Bastos Gisele Alsina Nader , Vietta Giovanna Grunewald , Nascimento Guilherme Fagundes , Vianna Heloisa Reniers , Guimarães Henrique Cerqueira , de Morais Julia Drumond Parreiras , Moreira Leila Beltrami , de Oliveira Leonardo Seixas , de Deus Sousa Lucas , de Souza Viana Luciano , de Souza Cabral Máderson Alvares , Ferreira Maria Angélica Pires , de Godoy Mariana Frizzo , de Figueiredo Meire Pereira , Guimarães-Junior Milton Henriques , de Paula de Sordi Mônica Aparecida , da Cunha Severino Sampaio Natália , Assaf Pedro Ledic , Lutkmeier Raquel , Valacio Reginaldo Aparecido , Finger Renan Goulart , de Freitas Rufino , Guimarães Silvana Mangeon Meirelles , Oliveira Talita Fischer , Diniz Thulio Henrique Oliveira , Gonçalves Marcos André , Marcolino Milena Soriano 
Published: March 2023

The majority of early prediction scores and methods to predict COVID-19 mortality are bound by methodological flaws and technological limitations (e.g., the use of a single prediction model). Our aim is to provide a thorough comparative study that tackles those methodological issues, considering multiple techniques to build mortality prediction models, including modern machine learning (neural) algorithms and traditional statistical techniques, as well as meta-learning (ensemble) approaches. This study used a dataset from a multicenter cohort of 10,897 adult Brazilian COVID-19 patients, admitted from March/2020 to November/2021, including patients [median age 60 (interquartile range 48–71), 46% women]. We also proposed new original population-based meta-features that have not been devised in the literature. Stacking has shown to achieve the best results reported in the literature for the death prediction task, improving over previous state-of-the-art by more than 46% in Recall for predicting death, with AUROC 0.826 and MacroF1 of 65.4%. The newly proposed meta-features were highly discriminative of death, but fell short in producing large improvements in final prediction performance, demonstrating that we are possibly on the limits of the prediction capabilities that can be achieved with the current set of ML techniques and (meta-)features. Finally, we investigated how the trained models perform on different hospitals, showing that there are indeed large differences in classifier performance between different hospitals, further making the case that errors are produced by factors that cannot be modeled with the current predictors.