Precision oncology in AML: validation of the prognostic value of the knowledge bank approach and suggestions for improvement

<jats:title>Abstract</jats:title><jats:p>Recently, a novel knowledge bank (KB) approach to predict outcomes of individual patients with acute myeloid leukemia (AML) was developed using unbiased machine learning. To validate its prognostic value, we analyzed 1612 adults with de novo AML treated on Cancer and Leukemia Group B front-line trials who had pretreatment clinical, cytogenetics, and mutation data on 81 leukemia/cancer-associated genes available. We used receiver operating characteristic (ROC) curves and the area under the curve (AUC) to evaluate the predictive values of the KB algorithm and other risk classifications. The KB algorithm predicted 3-year overall survival (OS) probability in the entire patient cohort (AUC<jats:sub>KB</jats:sub> = 0.799), and both younger (&lt; 60 years) (AUC<jats:sub>KB</jats:sub> = 0.747) and older patients (AUC<jats:sub>KB</jats:sub> = 0.770). The KB algorithm predicted non-remission death (AUC<jats:sub>KB</jats:sub> = 0.860) well but was less accurate in predicting relapse death (AUC<jats:sub>KB</jats:sub> = 0.695) and death in first complete remission (AUC<jats:sub>KB</jats:sub> = 0.603). The KB algorithm’s 3-year OS predictive value was higher than that of the 2017 European LeukemiaNet (ELN) classification (AUC<jats:sub>2017ELN</jats:sub> = 0.707, <jats:italic>p</jats:italic> &lt; 0.001) and 2010 ELN classification (AUC<jats:sub>2010ELN</jats:sub> = 0.721, <jats:italic>p</jats:italic> &lt; 0.001) but did not differ significantly from that of the 17-gene stemness score (AUC<jats:sub>17-gene</jats:sub> = 0.732, <jats:italic>p</jats:italic> = 0.10). Analysis of additional cytogenetic and molecular markers not included in the KB algorithm revealed that taking into account atypical complex karyotype, infrequent recurrent balanced chromosome rearrangements and mutational status of the <jats:italic>SAMHD1</jats:italic>, <jats:italic>AXL</jats:italic> and <jats:italic>NOTCH1</jats:italic> genes may improve the KB algorithm. We conclude that the KB algorithm has a high predictive value that is higher than those of the 2017 and 2010 ELN classifications. Inclusion of additional genetic features might refine the KB algorithm.</jats:p>

section_content: treatment received, and outcomes from 1540 German AML patients [3]. Testing of several machine learning models revealed that inclusive, multistage statistical models scored best in predicting OS and probabilities of non-remission death, relapse death, and death in CR1. Although a relatively small study [4] confirmed prognostic usefulness of KB approach, to our knowledge, it has not been hitherto validated in a large, independent patient cohort. Therefore, we applied the KB algorithm to 1612 adults with de novo AML and investigated whether additional cytogenetic and molecular alterations might improve its accuracy. No patient receiving an allogeneic stem-cell transplantation in CR1 was included in the analyses (Additional file 1). 

We used ROC curves and the AUC to assess the ability of the KB approach to predict 3-year OS probability in comparison with the actual patient outcomes. The KB algorithm had a high AUC KB = 0. 799 (95% CI 0. 777-0. 821) for the entire patient cohort, for younger (< 60 years) patients AUC KB = 0. 747 (95% CI 0. 717-0. 776) and for older (≥ 60 years) patients AUC KB = 0. 770 (95% CI 0. 716-0. 824), for whom risk stratification is more difficult because they have generally poor prognosis (Fig. 1a-c ). 

Concerning other outcome endpoints, the KB algorithm was excellent for prediction of non-remission death (i. e., death within 3 years after diagnosis without CR1 achievement) with an AUC KB = 0. 860 (95% CI 0. 838-0. 882). For relapse death (i. e., death of patients achieving CR1 who relapsed and died within first 3 years), the predictive ability of the KB approach was worse (AUC KB = 0. 695, 95% CI 0. 662-0. 727). It was even worse for prediction of death in CR1, with a poor AUC KB of 0. 603 (95% CI 0. 537-0. 670; Fig. 1d-f ). 

Next, we compared the predictive values of the KB approach and of two well-established genetic-risk classifications, the 2010 [1, 5, 6] and 2017 ELN [2, 7, 8] classifications. Among all patients, the KB approach had the highest predictive value with AUC KB = 0. 799 (95% CI 0. 777-0. 821), followed by the 2010 ELN classification (AUC 2010ELN = 0. 721, 95% CI 0. 696-0. 746) and the 2017 ELN classification (AUC 2017ELN = 0. 707, 95% CI 0. 682-0. 732; Fig. 1g ). Compared directly, the KB approach was significantly better than both the 2017 (p < 0. 001) and 2010 (p < 0. 001) ELN classifications. 

When we performed the aforementioned comparisons after excluding early death patients, the KB approach still outperformed both the 2010 and 2017 ELN classifications, but the differences among classifications were smaller than in the entire patient cohort (Fig. 1h ; Additional file 1). 

We also compared the predictive value of the KB approach [3] with another AML risk classification, the 17-gene stemness score [9, 10], which is calculated as the weighted sum of the normalized expression values of 17 genes whose expression differs between leukemia stem cells and leukemic bulk blasts [9]. Among our 863 patients with RNA expression data available, the predictive values of the KB approach (AUC KB = 0. 764, 95% CI 0. 733-0. 800) and of the 17-gene stemness score (AUC 17-gene = 0. 732, 95% CI 0. 700-0. 765) did not differ significantly (p = 0. 10; Fig. 1i ). 

To determine whether genetic alterations not included in the KB algorithm might improve its performance, we compared the frequencies of 44 gene mutations and eight cytogenetic categories (listed in Additional file 1) between patients alive 3 years after diagnosis who were correctly predicted alive and patients falsely predicted to be dead. Three molecular and two cytogenetic markers were significantly different between the patient groups (Table 1 ). 

To cross-validate these findings, we compared these markers' frequencies between patients who died within first 3 years and were correctly predicted as dead and those falsely predicted to be alive. The frequencies of SAMHD1 mutations and atypical complex karyotype (i. e., without 5q, 7q and 17p abnormalities) [11] were significantly different in both comparisons. Frequencies of AXL and NOTCH1 mutations and of infrequent recurrent balanced chromosome rearrangements [12] were significantly different among patients alive and tended to be different among patients who died (Table 1 ). 

Summarizing, we show that the KB algorithm has a high predictive value, higher than the 2017 and 2010 ELN classifications, and identify additional genetic factors that might improve it.

section_name: Acknowledgements

section_content: We thank the patients who participated in clinical trials and their families supporting them; Donna Bucci and Christopher Manring and the CALGB/Alliance Leukemia Tissue Bank at The Ohio State University Comprehensive Cancer Center, Columbus, OH, for sample processing and storage services; and Lisa J.
section_name: Acknowledgements

section_name: Abbreviations

section_content: The online version contains supplementary material available at https:// doi. org/ 10. 1186/ s13045-021-01118-x.
section_name: Supplementary Information

section_content: Authors' contributions MB, KM, JK, DN, CDB, and CCO designed the study; MB, KM, BG, JK, DN, DP, A-KE, RG, and CCO analyzed the data; JK and DN performed the statistical analyses; MB, KM, and JK wrote the manuscript; MB, KM, JK, BG, JCB, CDB, and CCO edited the manuscript; JEK, BLP, AJC, RMS, JCB, and CDB provided study materials or patients; JCB, CDB, and CCO provided administrative support; JCB, and CDB provided financial support. All authors, except CDB, who died before the completion of the final draft, read and approved the final manuscript.
section_name: Additional file 1. Supplementary Material.

section_content: Research reported in this publication was supported in part by the National Cancer Institute of the National Institutes of Health under Award Numbers U10CA180821, U10CA180882, and U24CA196171, (to the Alliance for Clinical Trials in Oncology) UG1CA233180, UG1CA233331, UG1CA233338, U10CA101140, U10CA180861, P30CA016058, and RCA197734, the Leukemia Clinical Research Foundation, the Warren D. Brown Foundation, and by an allocation of computing resources from The Ohio Supercomputer Center. Support to Alliance for Clinical Trials in Oncology and Alliance Foundation Trials programs is listed at https:// ackno wledg ments. allia ncefo und. org.
section_name: Funding

section_content: The datasets used and/or analyzed during the current study are available from the corresponding authors on reasonable request.
section_name: Availability of data and materials

section_content: All study protocols were approved by the Institutional Review Boards at each participating center in accordance with the Declaration of Helsinki. Each patient provided written informed consent for the research use of their specimens before enrollment.
section_name: Declarations Ethics approval and consent to participate

section_content: Not applicable.
section_name: Consent for publication

section_content: The authors declare that they have no competing interests.
section_name: Competing interests

section_content: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
section_name: Publisher's Note