International testing and refinement of AI algorithms predicting acute leukemia subtypes from routine laboratory data

Domain: chemistry
Published: 2026-03-20
Editorial outcome: minor_revision

Abstract
Despite advances for patients with acute leukemia health disparities limit access to diagnosis and treatment. Artificial Intelligence (AI) approaches may address some disparities. We retrospectively assemble a diverse, international cohort of 6206 leukemia patients from 20 centers to test an AI tool designed to support leukemia diagnosis using standard laboratory results. Executing the pretrained algorithm results in varying accuracy metrics. With confidence cutoff predictions, 2000-fold bootstrapped area under the curve (AUROC) metrics are 0.94 for acute myeloid leukemia (AML), 0.98 for the promyelocytic subtype and 0.84 for acute lymphoblastic leukemia. However, this cutoff excludes 70.8–92.5% of patients from predictions. We improve accuracy and robustness, while maintaining generalizability via an ensemble of Isolation Forest and Local Outlier Factor increasing AUROC for AML from 0.72 to 0.84 (hold-out test set, patients below confidence threshold), while excluding only 12.1% of patients. Furthermore, we retrain the algorithm for pediatric patients.

Introduction
While the care for patients with acute myeloid leukemia (AML) and acute lymphoblastic leukemia (ALL) has been broadly improved over the past decades e.g., via novel and targeted treatment options1–3, refined genomic risk stratification4 and both reduced early mortality5–7, and reduced nonrelapse mortality after HCT8,9, barriers to equitable diagnosis and treatment prevail and limit the access to these benefits10. In particular, patients in low- and middle-income countries face significant barriers to accessing specialized diagnosis and care11,12, but systemic health disparities are also observed within high-income countries13. Several constraints, including access to flow cytometry and molecular genetics testing14, may challenge a timely diagnosis of acute leukemias and delay referral or specialized care. For patients with acute promyelocytic leukemia (APL), this diagnostic gap significantly increases early mortality15,16. APL has favorable long-term outcomes, with >90% long-term remission in patients who survive the critical first 4–6 weeks until hematological remission. Early diagnosis and treatment, which combines ATRA, cytoreduction, and efficient coagulation management, aims to reduce morbidity and mortality due to thromboembolic and bleeding events and differentiation syndrome. Early death rates still range from 5–10% in clinical trials and up to 30% in real-world data from middle-income countries15 and tragically often occur before complete APL diagnosis. In addition, for other subtypes of acute leukemia, facilitated diagnosis is warranted to allow early referral to specialized centers or initiation of targeted therapy under restricted resources as early as possible and manage acute leukemia patients appropriately to avoid early death.

The potential of Artificial Intelligence (AI) with deep learning approaches using marrow morphology images to support the diagnosis of acute leukemia has been previously demonstrated with high levels of accuracy17,18; however, at present, such approaches require complex infrastructures and substantial resources that limit access. Recently, routine laboratory features have been leveraged to develop and test machine learning (ML) classification algorithms for predicting three types of leukemia (AML, ALL, and APL) in multicenter French cohorts19, an approach that could make AI diagnostic support broadly available at low costs. Recognizing the importance of independent international validation and real-world evaluation of potentially relevant models, we here assemble a large, global cohort to systematically test this AI classifier’s accuracy, generalizability, speed, and robustness and to refine this model for practical clinical application.

Results
Testing on a diverse cohort with highly accurate predictions for AML, yet not applicable to all patients
This global cohort (n=6206) was assembled from 20 contributing centers in 16 countries across 5 continents (Fig. 1a). Its characteristics are detailed in Supplementary Table 3. The cohort was diverse (geographically, socially, for age (range 0.08–97 years)), including both sexes (male 50%), adults (≥18 years, n=4460), and pediatric patients (n=1746), which were analyzed separately. The ethnicity was not recorded, but partly associates with the site geography. According to the UN classifications, 43% (7/16) of the participating countries were of low- and middle-income (Fig. 1b and Supplementary Table 2). AML was the most common diagnosis (n=3510, Fig. 1c). In adults, the pretrained model’s 2000-fold bootstrap AUROC was 0.82 (95% CI, 0.82–0.82) for the prediction of AML, 0.92 (95% CI, 0.92–0.92) for APL and 0.79 (95% CI, 0.79–0.79) for ALL (Fig. 1d). Given that XGBoost models reflect the confidence of its predictions, we tested the previously published confidence cutoff19. These predictions reached in specific sites very high AUROCs of up to 1.0 for AML, 1.0 for APL, and 0.85 for ALL (Supplementary Data 1 and Fig. 1e). For clinical use, however, this cutoff was overly restrictive, excluding 70.8–92.5% of adult patients from predictions, which was why we focused on the performance of the general model. Here, high-accuracy metrics were obtained across geographic regions (Fig. 2 and Supplementary Fig. 3a), yet with substantial variance between centers and classes. Bootstrapped AUROC for APL ranged from 0.98 for Salamanca, Spain, 0.93 for Suzhou, China, and 0.63 for Antananarivo, Madagascar. AUROC for adult AML reached 0.93 (95% CI, 0.93–0.94) in Melbourne, Australia, and Maastricht, the Netherlands (Fig. 2). The highest F1 scores (range 0.56–0.95) were obtained with AML, whereas ALL (0.15–0.70) and APL F1 scores were lower due to a precision-recall tradeoff (Supplementary Fig. 3a). Interestingly, the probability distribution of true positives per country per class of the adult AML model was relatively high (i.e., approximately 0.9) across countries (Fig. 3a). Its variance was comparatively small, with some spread in the variance of smaller-sized cohorts. For adult ALL and APL, the variation between centers was substantial, yet the number of cases was also much smaller. The probability distribution of the false negative values was lowest in APL, supporting the favorable results regarding the AUROC (Supplementary Fig. 3b). Given the observed variations in performance across cohorts and classes, we also compared our cohorts’ feature distributions to those French cohorts, in which the pretrained model was developed and validated (Supplementary Data 2). In APL, there were differences in fibrinogen levels (mean 1.84 g/l vs. 2.51 g/l in French cohorts) and in ALL, monocyte and lymphocyte counts were substantially higher (mean 3.90/nl vs. 0.49/nl and 18.62/nl vs. 3.72/nl).

Differences in feature distributions between acute leukemia subtypes. Coagulation parameters and MCV are most relevant for the model
Explainable predictions may support the clinical implementation of AI models in addition to their accuracy, robustness, and generalizability. We comparatively analyzed the range of laboratory results of correctly classified and misclassified adult patients within the leukemia subtype algorithm. Among correctly classified patients with APL, those with MCHC values above 340 g/l, lymphocyte counts below 1 g/l, prothrombin times below 75%, and fibrinogen levels below 2 g/l were enriched (Fig. 3b). For patients with AML, an MCV above 90 fl and higher monocyte counts support this prediction. The majority of correctly predicted ALL patients had a PT above 85% and an MCV below 90 fl. However, the algorithm was susceptible to high monocyte counts, which are regularly classified as AML, when they occur in other classes (Supplementary Fig. 4). In adults, feature importance analysis revealed that fibrinogen (270.80), age (260.52), and MCV (170.04) were the most discriminative variables. For AML, SHAP analysis revealed that monocytes (0.284 for %; 0.165 for counts; Fig. 3c) were the strongest positive predictor, followed by MCV (0.133). In ALL patients, the monocyte percentage (−0.229) and PT percentage (−0.158) were the strongest negative predictors, whereas the MCHC (0.036) showed modest positive feature importance. Low fibrinogen levels (−0.361) and low MCHCs (−0.141) favor the prediction of APL. According to the subgroup analyses, the algorithm generalized well between men (AUROC AML 0.82, APL 0.92 and ALL 0.8) and women (AUROC AML 0.8, APL 0.93 and ALL 0.78). Patients with leukocytosis and peripheral blasts (WBC > 20,000/µl) had predictions comparable to those of the entire population (AUROC AML 0.8, APL 0.93 and ALL 0.76). When we analyzed AML patients with documented ELN risk status from high-income countries, we observed comparable performance between genetic risk profiles (AUROC 0.91 for ELN favorable risk AML, 0.9 for intermediate-risk AML, and 0.92 for adverse risk AML).

Refining the algorithm predictions for adults to achieve a good generalizability
Given the observed shortcomings, we applied several methods to refine the algorithm’s predictions for greater generalizability and clinical applicability. All adult data points are plotted in Supplementary Fig. 5. First, we applied dimensionality reduction with PCA to the features and successfully separated most patients with APL. The AML samples presented the widest distribution in the PCA space, which was consistent with their heterogeneous nature and had some overlap with the ALL samples (Fig. 4a). The first two principal components (PC1, 53.1%; PC2, 30.9%) collectively explained 84.0% of the total variance in the dataset; however, complete class separation was not achieved (Fig. 4b, c). Next, we applied an ensemble of Isolation Forest and Local Outlier Factor to reduce the proportion of distribution outliers and improve algorithm performance (Fig. 4d, e). Multicenter patients with confident predictions (cutoff > 0.9, n=2809) served as a training set (Fig. 4d) for this pipeline and were tested on a hold-out dataset (n=2692 with confidence < 0.9, Fig. 4e). Overall, this pipeline identified incorrectly predicted samples with twice as high a probability as correctly predicted samples (54.79% vs. 24.46%), improving the algorithm’s accuracy while including as many patients as possible for accurate predictions. Across diverse populations, the outlier detection approach consistently improved predictive performance (Fig. 4f, g). For AML, the accuracy improved in cohorts with high accuracy and linearly increased in cohorts with lower accuracy (Fig. 4h). The filtering preferentially removed incorrect predictions (AML 20.40% incorrect vs. 12.83% correct), and the isolated proportion was greater in the ALL class (65.98% vs. 27.56%). The sensitivity was increased for all classes, especially for APL (+26.85%) and ALL (+18.54%). Stratified per center, the accuracy metrics revealed substantial improvements for most feature constellations (Supplementary Fig. 6). The developed preprocessing pipeline did also allow to challenge the algorithm with differential diagnoses to AML, ALL and APL presenting with leukocytosis or cytopenia (Supplementary Table 4). Cases with leukocytes above the normal were excluded with high accuracy (87.9%), while cytopenic cases were more challenging (78.4% of leukopenia cases with other diagnoses excluded).

Model retraining on pediatric data substantially improves precision
To further address health disparities in leukemia, we tested the algorithm’s applicability among pediatric patients (n=1746). ALL was the most common type of pediatric leukemia (Fig. 5a). Here, the pretrained model, originally developed on adult cohorts, achieved low accuracy metrics. The median AUROC for ALL was 0.75 (95% CI 0.74–0.75, Fig. 5b, c), far below the highest reported AUROC in adult ALL (Maastricht, The Netherlands, AUROC 0.92). Confidence cutoff predictions increased the accuracy but excluded too many patients (Fig. 5D). The ALL SHAP values were comparable to those of adults, but for AML, there were substantial differences (Supplementary Fig. 7a–c). As expected, pediatric patients with ALL had distinct feature ranges from centrally-tested adults, including significantly lower LDH levels (mean 558.46 U/l vs. 969.24 U/l); APL patients had higher PT percentages (mean 90.52% vs. 71.68% in adults) and higher monocyte counts (mean 9.41/nl vs. 2.88/nl in adults, Supplementary Data 3). Fibrinogen levels were lower than in adults, with particularly marked differences in AML patients (mean 2.63 g/l vs. 3.94 g/l in adults). Given the generalizability issues and unreliable precision-recall balance (Supplementary Data 4), we retrained the XGBoost on our pediatric cohort data resulting in substantially improved accuracy metrics with a 2000-fold bootstrap AUROC for ALL reaching 0.95 (95% CI 0.95–0.95, Fig. 5e). Pediatric APL was rare (n=24), yet the pretrained predictions were accurate (e.g., AUROC Riyadh 0.90, Suzhou 0.97; Supplementary Fig. 7d). An age-decade stratified analysis of the entire cohort including adult and pediatric patients indicated that accuracy increased from childhood to adulthood and remains stable through older adulthood (Supplementary Fig. 8 and Supplementary Table 5). This result is encouraging for the algorithm’s potential in APL detection in adults and children.

Methods
Statement of ethics
This study was conducted according to the Declaration of Helsinki and approved by the Ethics Committee of the Medical Faculty of the University Duisburg-Essen, approval number 24-11882-BO, as well as by further local IRBs (Dallas, Hannover, Maastricht, Riyadh, Suzhou). Given the use of anonymized routine laboratory data as features, informed consent was waived by the IRB. No participant compensation was provided. This study has been registered in the German Registry for Clinical Trials (DRKS). https://drks.de/search/en/trial/DRKS00037360.

Study cohort and data
To account for data diversity during independent testing and model refinement, we retrospectively assembled international patient cohorts with acute leukemia, whose laboratory features included the complete blood cell (CBC) count (total white blood cell count (WBC; g/L or 10⁹ per L), monocyte and lymphocyte counts (g/L or 10⁹ per L), platelet count (g/L or 10⁹ per L), mean corpuscular volume (MCV; fL), mean corpuscular hemoglobin concentration (MCHC; g/L), lactate dehydrogenase (LDH; IU/L), fibrinogen (g/L) and prothrombin time (%) at the earliest possible point in leukemia diagnosis (i.e., at hospital admission). Data on prothrombin time reported in seconds were converted into percentages via calibration plasma. In addition, metadata such as age, sex, and, if available, European LeukemiaNet (ELN) 2022 risk4 and World Health Organization (WHO) 2022 classification25 information were included and considered for subset analyses. Sex was self-reported. The features are detailed in Supplementary Table 1. Pediatric (<18 years) and adult patients (≥18 years) were analyzed separately. Patients were included from high-, upper middle-, lower middle and low-income countries according to the World Bank and United Nations (UN) classification40 and categorized using the UN Human Development Index41 (accessed February 2025, Supplementary Table 2). Most countries have combined state and private health systems for cancer care42,43.

Data preparation
The methodology overview is illustrated in Supplementary Fig. 1. This study was conducted primarily in a centralized manner with transfer of anonymized data (n=19). In addition, the site in Riyadh, Saudi-Arabia, implemented and tested the algorithm locally. Cohorts were provided by contributing investigators either through a custom Fast Healthcare Interoperability Resources (FHIR) pipeline or as file data imports (e.g., .csv files). For FHIR-derived data, the relevant FHIR resources were merged to construct feature-label datasets for predictions. To ensure consistency across data sources, some features of centrally submitted laboratory data were transformed into SI units, which were required to run the algorithm via a configuration file that facilitated the reproducible processing of new cohorts. A laboratory medicine expert manually inspected all centrally collected data for potential errors and excluded technically implausible samples (n=10). The multicenter data underwent programmatic preprocessing, which included filtering for quality and removing samples with more than 20% missing features (n=439). These processes are visualized in a STROBE44 diagram (Supplementary Fig. 2). In the cohort from Antananarivo, Madagascar, coagulation markers (PT and fibrinogen) and LDH were systematically unavailable; exceptionally, these patient data were tested despite the greater proportion of missing data.

Model evaluation
For independent, international testing, we conducted several experiments. First, acute leukemia subtype predictions were generated by executing the pretrained eXtreme Gradient Boosting (XGBoost) model19 to the prepared datasets. The performance of the AI classifier was evaluated via various metrics, including the area under the receiver operating characteristic curve (AUROC), sensitivity, specificity, F1-score, and confusion matrices. Bootstrapping (2000-fold) was employed to estimate metric means and generate 95% confidence intervals (CIs). Bootstrapping required a minimum cohort size of 30 valid samples per center. The analysis was stratified by leukemia subtype to allow for granular performance evaluation. Subclasses containing fewer than ten samples were not reported to maintain statistical validity. We analyzed the performance separately for each center and adult and pediatric patients to account for data variance. The XGBoost algorithm provides information on the confidence of predictions, which may be leveraged to define clinically meaningful cutoff values for potential clinical implementation. For direct comparison of our results to those obtained by Alcazer et al., we next applied their confidence cutoff strategy using positive predictive value (PPV) thresholds (AML: 0.945, APL: 0.749, ALL: 0.959) as described previously19. We also tested the applicability in specific clinically relevant subgroups (e.g., ELN-defined). Third, we compared the feature importance via SHapley Additive exPlanations (SHAP) values. The feature distributions of correctly classified and misclassified patients were comparatively analyzed for each predicted class using boxplots. We also employed principal component analysis (PCA) on the features to check whether the classes were linearly separable.

Model refinement
Given that pediatric patients have distinct laboratory ranges and feature constellations, which the XGBoost algorithm was not previously exposed to, we retrained it on our pediatric dataset, split into training (80%) and test (20%) sets. The retraining was performed in R using stratified sampling to preserve the class distribution. Performance was evaluated on the hold-out test set as described above. We chosen not to retrain the model on adult patient data because we aimed preparing the algorithm for clinical testing, robust to future unknown feature constellations. The subsequent steps aimed to standardize the applicability of the pretrained model, to clarify when it offers trustable results, and to enhance both robustness and generalizability. The distribution outlier detection pipeline combined Isolation Forest and Local Outlier Factor models trained per class on preprocessed, normalized data obtained from adult patients (Supplementary Fig. 1). Centers contributing more than 30 samples were included to maintain statistical reliability. The dataset was partitioned into two sets: a high-confidence training set comprising samples with prediction confidence of 0.9 or higher and a lower-confidence test set. Before Isolation Forest training, the data underwent a series of preprocessing steps; this included imputing missing values via the median method and normalizing data via standard scaling, to achieve a zero mean and a unit variance. The training phase employed a class-specific approach, where separate models were trained for each disease class. Each class incorporated an Isolation Forest with automatic contamination detection and a Local Outlier Factor model configured with 20 neighbors and automatic contamination settings. The test data underwent these same preprocessing steps. The performance was evaluated using AUROC scores and classification reports, which compared the results before and after removing distribution outliers.

Software and implementation
The original algorithms had previously been implemented in R and made publicly available (https://github.com/VincentAlcazer/AIPAL). We selected the best performer on the development cohort, an extreme gradient boosting (XGBoost) algorithm, as the pre-trained model for this study (The workflow is illustrated in Supplementary Fig. 1). For independent testing, we developed a custom FHIR pipeline using Python (version ≥ 3.10) leveraging multiple open-source packages: FHIR-PYrate for FHIR data handling, Weights & Biases (wandb) for experiment tracking, SQLAlchemy and psycopg2-binary for database operations, PyYAML and python-dotenv for configuration management, and various data science libraries including matplotlib, scikit-learn, XGBoost, SHAP, and UMAP for analysis and visualization. Details and references to all libraries are provided in the Supplementary Methods.

Further, we created a standardized prediction workflow pipeline that can ensure reproducibility of methods across all cohorts and to ensure the validity of the predictions for a broader patient population. It included pre-processing the data, making predictions and evaluating the results. This validation pipeline employed a combination of the R programming language and Python. From electronic medical records, the pipeline identified the first occurrence of coded leukemia diagnoses and extracted the required laboratory results at diagnosis, as well as age and sex. The same Python pipeline was employed for importing data for leukemia subtype detection. This comprehensive and reproducible pipeline ensured the robust validation of the leukemia subtype classifier across diverse data sources and settings. Further, the pipeline was extended to allow new samples to be validated by the developed outlier detection algorithm. Probabilistic predictions for leukemia subtypes (AML, APL, ALL) and other evaluation results were logged to Weights & Biases (https://wandb.ai/), including cohort characteristics, performance metrics, and outlier statistics. Results were also tabulated and stored in .CSV files for further analysis. The code of the validation Python pipeline, including the outlier detection algorithm, is publicly available in the UMEssen GitHub repository (https://github.com/UMEssen/aipal-validation).

Inclusion and ethics statement
All local contributors to this research were included as authors or acknowledged when not fulfilling authorship criteria. We confirm that this research has local, regional and global impact and has been designed and conducted in collaboration with local communities.

Reporting summary
Further information on research design is available in the Nature Portfolio Reporting Summary linked to this article.

Data availability
Anonymized acute leukemia laboratory data is deposited to the HARMONY Alliance data repository due to IRB restrictions. Data access can be requested by researchers in written form via office@harmony-alliance.eu (expected response time 2 weeks) and is subject to HARMONY data access committee approval. HARMONY will make data available for up to 12 months. Source data are provided with this paper. A demo dataset is provided with the GitHub code. Source data are provided in this paper.

Code availability
The pipeline code is publicly available (https://github.com/UMEssen/aipal-validation) under the MIT License. The version of the code used in this study is citable via Engelke M and Turki AT, International testing and refinement of an AI algorithm to predict acute leukemia subtypes from routine laboratory data, 202645. https://doi.org/10.5281/zenodo.18436461.

Peer review rounds
[{"round": 1, "reviewer_comments": {"reviewer_1": "The study by Turki describes a AI algorithm for predicting acute leukemia subtypes (ALL, APL and AML) from routine laboratory data, such as LDH, PT, age, MCHC, fibrinogen, MCV, monocytes, lymphocytes and platelets. Impressively they test this on over 6000 cases from 20 worldwide centers across the age spectrum. They first tested a previously established XGBoost model across the cohort and within specific centers and then refined the model for better performance. Overall, this is a timely and interesting study. A few concerns need to be addressed prior to publication:\n1. The variability in performance across the different centers is significant yet there isn't enough discussion regarding the source of the variability.\n2. The monocyte count appears to lead to issues with predictions. Does removal of monocytes from the prediction have a positive or negative impact?\n3. Although it makes sense to have a separate predictor for kids and adults, the firm cutoff of 18 could lead to issues. The team should show the performance of each classifier across the age spectrum (perhaps broken down into decades). It is possible that the a strict cutoff is not ideal.\n4. The discussion is the same length as the results and is too long. Please shorten to focus on the take away messages rather than summarizing the results.", "reviewer_2": "Turki et al. have compiled an international multicentric dataset of routine laboratory parameters from a diverse group of over 6000 patients and used it to improve an approach introduced previously by Alcazer et al. in 2024 on a French cohort and develop a group of algorithms for subtyping acute leukemia. The topic responds to a relevant medical question, namely quickly and easily subtyping AML patients, which has implications on treatment and potentially survival of this patient population. They also address subtyping in pediatric populations. Overall, the paper is well written.\nWhile the machine learning approach is largely derived from previous work, the machine learning approach used is well documented and the code made available.\nApart from these strengths, there are a few points that the authors should address before the manuscript is ready for publication.\nMajor points:\n1) Large part of the scientific value of this work justifing publication in a journal like Nature Communications lies in compiling a unique and diverse global dataset of leukemia patients, especially as the algorithmic strategy has been described previously. Hence, data accessibility to the scientific community is key when considering the scientific impact of the manuscript. The authors address this point by intending to publicly share the dataset via the HARMONY Alliance repository. However, they are very vague on the amount of data or centers included in the final deposit (\"centers willing to share their data\"). For high relevance in the community, the anonymized dataset should be deposited in full or with minimal redactions in order to be accessible for independent testing and as a basis for future work. The authors should clearly state what they are willing to share in a revised version of the manuscript.\n2) The authors' outlier filtering method is at the heart of their refinement, but its description is unclear. In the relevant section, the authors state \"Overall, this pipeline identified incorrectly predicted samples with twice as high a probability as correctly predicted samples (54.79% vs. 24.46%), improving the algorithm’s accuracy while including as many patients as possible for accurate predictions.\" It is not clear what the prediction then is for these cases. Are they excluded from the analysis? Also, could there be any pattern shared by the outliers removed that is not captured by the algorithm (e.g. secondary diagnoses or lab errors)?\n3) Before model refinement, there is considerable inter-site difference for the dataset, down to models that do not work at all (AUROC 0.5) in some cases, as shown in Fig. 2. Can the authors provide this full analysis also for the post-refinement model? This would provide a more comprehensive overview of the site-specific impact of their refinement method.\n4) While the authors achieve a substancial improvement using their outlier removal strategy, classification performance based on lab parameters is still far from the diagnostic gold standard. What strategies to the authors forsee to improve from this point? Do they believe that further increasing case numbers or including other clinical values would be important?\nMinor points:\n1) When considering pediatric cases, the authors find that different relevant lab value constellations differ from the adult case. Do the authors believe that this reflects the distinct biology of pediatric disease, or is it due to the different case number distribution\n2) Presentation: Figures are overall quite busy, with small legends, especially in Fig. 2 and 3.\n3) In Fig.2, the same formatting should be used for all graphs, and multiple similar legends avoided.", "reviewer_3": "In this manuscript, the authors have validated and improved upon a published model for the prediction of acute leukemia subtype across several centres. They use CBC counts, blood LDH levels, and coagulation parameters to predict the leukemia subtypes. The concept, overall, is promising and tries to solve an important problem, i.e., lack of access to sophisticated investigations in developing countries. That being appreciated, these are the concerns that are noted:\n1. Patients with acute leukemia (AL) can have varied clinical presentations from frankly leukemic (elevated WBC counts) to cytopenic. Frank leukemic presentations are also noted in other myeloid malignancies such as chronic myeloid leukemia and other MPNs, as well as MDS-MPN overlap conditions, such as chronic myelomonocytic leukemia in adults and juvenile myelomonocytic leukemia in children, which have monocytosis. These presentations are also seen in lymphomas such as chronic lymphocytic leukemia or mantle cell lymphoma, to name a few. Similarly, AL also commonly presents with frank cytopenia mimicking other malignancies, for example, MDS, CMML (dysplastic type), and reactive conditions (eg, B12/Folate deficiency, sepsis, or chemotherapy-induced cytopenia). The training set that is used for the model uses features of frankly leukemic patients and, in all probability, excludes patients with other conditions or leukemia that have been partly treated with disease-modifying therapy (as stated in their Lancet Digital Health paper). This is a major shortcoming of the study, as these are common problems in the real world and are amplified in LMIC settings, which is the premise of this paper.\n2. Validation studies to refute the diagnosis of acute leukemia: This ML model should be validated on conditions that are not acute leukemia but present with leucocytosis and monocytosis (lymphoma/chronic lymphoproliferative disorders, MDS-MPN overlap syndromes), as well as cytopenia such as MDS, MDS-MPN, and acute leukemia with cytopenic counts. This will ensure that this model has true clinical value and is applicable to a resource constrained setting.\n3. It is unclear how patients were selected for training the model or for validation. Are these consecutively diagnosed patients, or is there a selection bias?\n4. There are minor errors in the supplementary data as total WBC, monocyte, platelet counts are in G/L"}, "author_response": "Point by point response to peer review for\n\"International testing and refinement of an AI algorithm to predict acute leukemia\nsubtypes from routine laboratory data\" by Turki et al.\nReviewer 1 report:\n(Remarks to the Author):\nThe study by Turki describes a AI algorithm for predicting acute leukemia subtypes (ALL,\nAPL, and AML) from routine laboratory data, such as LDH, PT, age, MCHC, fibrinogen,\nMCV, monocytes, lymphocytes, and platelets. Impressively they test this on over 6000\ncases from 20 worldwide centers across the age spectrum. They first tested a previously\nestablished XGBoost model across the cohort and within specific centers and then\nrefined the model for better performance. Overall, this is a timely and interesting study. A\nfew concerns need to be addressed prior to publication:\nResponse: Thank you very much for the positive evaluation of our work.\n1. The variability in performance across the different centers is significant yet there isn't\nenough discussion regarding the source of the variability.\nResponse: We have extended the discussion on page 13, lines 352-358, to include\npossible sources of variability between centers. This now includes population\ncharacteristics and feature distributions such as age, pre-analytics, and - to a minor\nextent - laboratory analyzers. Our analysis indicated that the further away the\ncollective data points (e.g. leukocytes) were from the laboratory results of the trained\ndistributions, the worse they performed. This explains also the lower performance in\nleukopenia than in leukocytosis. The systematically abnormal results in leukemia\npatients remain a challenge to defining a robust pattern, which, however, was\naddressed by the preprocessing pipeline.\n2. The monocyte count appears to lead to issues with predictions. Does removal of\nmonocytes from the prediction have a positive or negative impact?\nResponse: Following this suggestion, we tested an additional model without\nmonocytes, which did not improve the performance in adults but increased F1 scores\nin pediatric AML patients. The following performance metrics were recorded:\nAdult patients (n=3,173, monocyte excluded model)\n- ALL: AUC 0.79 (unchanged), F1 0.48 (+0.01)\n- AML: AUC 0.83 (+0.01), F1 0.83 (-0.02)\n1\n- APL: AUC 0.92 (unchanged), F1 0.65 (+0.05)\nPediatric patients (n=517, monocyte excluded model)\n- ALL: AUC 0.69 (-0.05), F1 0.74 (unchanged)\n- AML: AUC 0.62 (-0.02), F1 0.40 (+0.15)\n- APL: AUC 0.77 (-0.03), F1 0.16 (+0.01)\nHowever, excluding monocyte variables (both absolute counts and %) forced us to\nremove many patients with missing features, as we only allowed up to 20% missing\nfeatures. This removal reduced the adult cohort by 25% (to n=3,173, monocyte\nexcluded model) and the pediatric cohort by 66% (to n=517, monocyte excluded\nmodel). The AUC metrics remained largely unchanged for adults while the F1\nimproved for pediatric AML. We think that the altered dataset composition has\nimpacted the performance metrics by selecting those datasets without or with less\nmissing features. Improvements in pediatric AML may be the consequence of\nimproved data quality in the subset rather than the true effect of removing monocyte\nfeatures. Given that substantial changes were only observed for pediatric AML and\nto preserve sample size, generalizability, and robust population representation, we\ndecided to keep the monocyte features for the final model.\n3. Although it makes sense to have a separate predictor for kids and adults, the firm\ncutoff of 18 could lead to issues. The team should show the performance of each\nclassifier across the age spectrum (perhaps broken down into decades). It is possible that\nthe a strict cutoff is not ideal.\nResponse: Indeed, the biological separation between adult and childhood leukemia\nis not strictly drawn at the age of 18, however, there were several reasons why we\nadopted it. First, 18 years is a common transition age from pediatric to adult care.\nSecond, Alcazer et al. Lancet digit. Health 2024 trained their algorithm only on\npatients aged 18 and older, and one of the objectives of our study was to validate\nthat algorithm. Hence, it appeared reasonable to adopt the 18-year cutoff.\nYet, we followed the suggestion of Reviewer 1 and comparatively tested the entire\ncohort, including both adult and pediatric patients, with age cutoffs at each decade\non the model without confidence cutoffs. The AUC metrics indicate that accuracy\nincreases gradually from childhood to young adulthood and remains stable through\nolder adulthood (particularly for AML and APL). The best AUCs were observed for\nmiddle-aged patients around the median age of the patient population in our study.\nImportantly, the APL prediction remains at high accuracy (AUC 0,94) until the age\ncutoff of 80 years. Two specific subsets were not fully evaluable due to low case\nnumbers (APL in patients aged 0-9 years (n=2); ALL in patients aged 80-89 years\n2\n(n=6)). We added this decade-stratified analysis to the manuscript as Supplementary\nFigure 8 and Supplementary Table 9. We added on page 10, line 267ff: “An age-\ndecade stratified analysis of the entire cohort, including adults and pediatric\npatients, indicated that accuracy increased from childhood to adulthood and remains\nstable through older adulthood (Supplementary Figure 8 and Supplementary Table\n9).” and copied the results here for your convenience:\nAge (years) ALL AML APL\n0–9 0,65 0,56 n/a\n10–19 0,74 0,69 0,87\n20–29 0,74 0,78 0,89\n30–39 0,78 0,77 0,91\n40–49 0,80 0,81 0,93\n50–59 0,80 0,83 0,93\n60–69 0,75 0,76 0,89\n70–79 0,75 0,79 0,94\n80–89 n/a 0,78 0,85\nAbbreviations: 'n/a', no available AUC for that age/disease subgroup combination\n4. The discussion is the same length as the results and is too long. Please shorten to\nfocus on the take away messages rather than summarizing the results.\nResponse: Thank you. The manuscript has been changed as suggested. We\nshortened the discussion on page 12f. lines 331 ff., 338ff., 358ff. and 362 ff.\n(Remarks to the Author):\nTurki et al. have compiled an international multicentric dataset of routine laboratory\nparameters from a diverse group of over 6000 patients and used it to improve an\napproach introduced previously by Alcazer et al. in 2024 on a French cohort and develop\na group of algorithms for subtyping acute leukemia. The topic responds to a relevant\nmedical question, namely quickly and easily subtyping AML patients, which has\nimplications on treatment and potentially survival of this patient population. They also\naddress subtyping in pediatric populations. Overall, the paper is well written.\nWhile the machine learning approach is largely derived from previous work, the machine\nlearning approach used is well documented and the code made available.\nResponse: Thank you very much for the positive evaluation of our work.\nApart from these strengths, there are a few points that the authors should address before\nthe manuscript is ready for publication.\nMajor points:\n1) Large part of the scientific value of this work justifing publication in a journal like\nNature Communications lies in compiling a unique and diverse global dataset of\nleukemia patients, especially as the algorithmic strategy has been described previously.\nHence, data accessibility to the scientific community is key when considering the\nscientific impact of the manuscript. The authors address this point by intending to\npublicly share the dataset via the HARMONY Alliance repository. However, they are very\nvague on the amount of data or centers included in the final deposit (\"centers willing to\nshare their data\"). For high relevance in the community, the anonymized dataset should\nbe deposited in full or with minimal redactions in order to be accessible for independent\ntesting and as a basis for future work. The authors should clearly state what they are\nwilling to share in a revised version of the manuscript.\nResponse: Indeed, data availability for further research is an important contribution\nto the advancement of science and for the academic community, hence we included\nthis statement. With the revised manuscript, we have signed a data sharing\nagreement contract with the HARMONY Alliance Foundation, a copy of which has\nbeen provided to the handling editor. We confirm that all investigators from the 20\nstudy centers have agreed on this sharing policy (n=6,206 cases). Data is available\nfrom the HARMONY Alliance repository with the date of publication of this\nmanuscript, as documented in the data availability statement.\n2) The authors' outlier filtering method is at the heart of their refinement, but its\ndescription is unclear. In the relevant section, the authors state \"Overall, this pipeline\nidentified incorrectly predicted samples with twice as high a probability as correctly\npredicted samples (54.79% vs. 24.46%), improving the algorithm’s accuracy while\nincluding as many patients as possible for accurate predictions.\" It is not clear what the\nprediction then is for these cases. Are they excluded from the analysis? Also, could there\nbe any pattern shared by the outliers removed that is not captured by the algorithm (e.g.\nsecondary diagnoses or lab errors)?\nResponse: The outlier detector (i.e., cases identified as being out of distribution) is\npart of the pipeline improvement to ensure robust and generalizable predictions in\npatients with acute leukemia and to limit false-positive findings in patients\npresenting with leukocytosis or cytopenia, but without acute leukemia. Please note\nthat the advantage of the outlier prediction is a) that it removes comparatively few\nsamples and that b) it enables to use the algorithm safely on unknown samples\nwith a high level of confidence. We believe that this trade-off is favorable: we\nexclude predictions on atypical or low-quality data to improve the reliability and\ninterpretability of data within the learned manifold.\nWe followed the suggestion of Reviewer 2 and analyzed the removed outliers with a\ndistribution analysis. Two patterns are noted:\n- Feature completeness has a limited impact. Outliers have a slightly higher\nproportion of missing features for several key inputs compared to the non-\noutliers. For instance, lymphocytes (+5.94%), monocytes (+5.52%), PT\n(+2.79%), WBC (+2.13%). Interestingly, fibrinogen has slightly lower\nmissingness in outliers (−1.86%).\n- As for the accuracy metrics, the proportion of detected outliers varies\nacross centers. Low rates were detected across continents, e.g., Antananarivo\n(7.3%), Bochum (9.6%), Melbourne (11.3%), and Maastricht (12.2%). Very high\noutlier rates were more likely in centers with limited performance than in\nthose with very high accuracy metrics, e.g., Lagos (82.7%), Kolkata (57.1%),\nWroclaw (55.2%). Rome and Suzhou were around 33%. This pattern could be\nleveraged for benchmarking purposes.\nOverall, these differences indicate that data quality and completeness contribute to\nthe “outlier” signal and sample exclusion.\n3) Before model refinement, there is considerable inter-site difference for the dataset,\ndown to models that do not work at all (AUROC 0.5) in some cases, as shown in Fig. 2.\nCan the authors provide this full analysis also for the post-refinement model? This would\nprovide a more comprehensive overview of the site-specific impact of their refinement\nmethod.\nResponse: As requested, we added the per-center analysis of the centrally tested\ncohort after refinement as Supplementary Figure 6. Across centers, most accuracy\nmetrics improved after refinement (e.g., AUC AML in Barcelona 0.905 (+0.04);\nindicate that no evaluable samples remained post-refinement for that site–class.\nEven in sites with previous poor performance, e.g. Kolkata, high accuracy predictions\nwere obtained for ALL 0.91 and AML 0.9. The adequate performance of APL\nprediction in the Lagos cohort (AUC APL 0.92) is contrasted with its poor predictions\nin AML, which remained unchanged (AUC 0.58). However, the Lagos cohort had\nsubstantially distinct feature distribution and leukopenia, was numerically very small\nand had the highest proportion of missing features beyond the level that we\naccepted for all other centers, as coagulation features were systematically missing\non the African cohorts. Please note that the AUC curve of Dallas is technically skewed\nsince only 4 APL samples and no ALL samples were provided into the study. Its data\nhowever populated the overall study population and provided highly accurate\npredictions for AML as documented by the other metrics. We copied the results here\nfor your convenience:\n6\n4) While the authors achieve a substantial improvement using their outlier removal\nstrategy, classification performance based on lab parameters is still far from the\ndiagnostic gold standard. What strategies to the authors forsee to improve from this\npoint? Do they believe that further increasing case numbers or including other clinical\nvalues would be important?\nResponse: We think that this comment refers to the discussion of accessibility versus\nprecision. The main question is what the algorithm will be used for. Indeed, it remains\nbehind the current gold standard of leukemia care in high-income countries, but that\nstandard includes bone marrow examination with cytogenetics as well as molecular\ngenetics and flow cytometry to achieve its high diagnostic accuracy. However, this\nprecision comes with high costs and still requires time today. While most recent\nresearch (Steinicke T et al., Nature Genetics, 2025; Marchi F et al., Nature\nCommunications, 2025) is addressing this issue using long read sequencing for rapid\ndiagnosis, the infrastructure implementation remains costly. This is also true for\nprocessing, even though individual samples sequencing is reported to cost several\nhundred dollars (Furlan S, The Hematologist, 22 (6), 2025) instead of thousands, this\nremains above the affordable level for leukemia screening in LMIC. In contrast, the\nalgorithm and pipeline reported in this manuscript use standard laboratory which\nare accessible, cheap and provide an immediate answer, which in our view may\nsupport early triage and support referral, particularly in resource poor settings. The\nresults that we provided with the revised manuscript further support the validity of\nthe pipeline, including the out-of-distribution detector. With the over 6,000 cases\nthat we have readily assembled, we do not think that performance will be\nsubstantially increased by further adding patients for training. Yet, the variation in\nthe features and exploration of additional features appears promising. As leukemia\nis a disease of the marrow cells, its examination remains the ultimate diagnostic tool.\nApproaches that first foresee a peripheral blood examination for triage and a marrow\nexamination as second step have been followed in Hematology for decades and\ntoday may both be AI-supported, and, depending on the resources, can be combined\nwith (epi-)genetics data for full diagnosis.\nMinor points:\n1) When considering pediatric cases, the authors find that different relevant lab value\nconstellations differ from the adult case. Do the authors believe that this reflects the\ndistinct biology of pediatric disease, or is it due to the different case number distribution\nResponse: Indeed, several factors may explain the observed differences, including\ncase number distributions and distinct biology, which likely both contribute to the\n7\ndifference between groups. We followed the suggestion of Reviewer 2 and\nperformed a downsampling experiment with the adult cohort (n=3979) to match\ntheir case numbers and disease proportions of leukemia subgroups (AML, APL and\nALL) to those of the pediatric cohort (n=1414). The downsampling decreased the\nmodel’s AUC (e.g. adult AML from 0,82 to 0,77, but it remained above the range of\nthe pediatric cohorts (AML AUC 0,64). This finding supports the hypothesis that both\nfactors contribute and that the case number distribution accounts for about 2-6% of\nthis AUC difference, depending on the leukemia subtype, while the biology appears\nto contribute to the remaining 2-13%, with the greatest difference observed for AML.\nWith respect to biological differences there is an extensive repertoire of publications\n(e.g. Bolouri H et al. Nature Medicine 2018, Roberts KG Am Soc Hematol Educ\nProgram, 2018). The results of our experiments are shown below:\nAccuracy metrics:\nPediatric cohort metrics (n=1,414)\nMetric ALL AML APL\n------------------------------------------------------------\nAUC 0.7436 0.6381 0.8093\nAccuracy 0.6198 0.6438 0.9435\nPrecision 0.9409 0.1562 0.1026\nRecall 0.6093 0.5734 0.4444\nF1 Score 0.7396 0.2455 0.1667\nAdult cohort metrics (n= 3,979, original, centrally tested)\nMetric ALL AML APL\n------------------------------------------------------------\nAUC 0.7924 0.8169 0.9209\nAccuracy 0.8223 0.7851 0.9256\nPrecision 0.4551 0.8433 0.7116\nRecall 0.4783 0.8663 0.5267\nF1 Score 0.4664 0.8547 0.6053\nMetric ALL AML APL\n------------------------------------------------------------\nAUC 0.7721 0.7745 0.8649\nAccuracy 0.5237 0.5661 0.9406\nPrecision 0.9723 0.1715 0.1071\nRecall 0.4761 0.8601 0.5000\nF1 Score 0.6392 0.2860 0.1765\n2) Presentation: Figures are overall quite busy, with small legends, especially in Fig. 2 and\n3.\nResponse: The manuscript legends of Figure 2 have been revised as suggested. We\nalso have updated Figures 3 and 5 including minor adjustments.\n3) In Fig.2, the same formatting should be used for all graphs, and multiple similar\nlegends avoided.\nResponse: The manuscript Figure 2 has been changed accordingly.\nReviewer 3 report:\n(Remarks to the Author):\nIn this manuscript, the authors have validated and improved upon a published model for\nthe prediction of acute leukemia subtype across several centres. They use CBC counts,\nblood LDH levels, and coagulation parameters to predict the leukemia subtypes. The\nconcept, overall, is promising and tries to solve an important problem, i.e., lack of access\nto sophisticated investigations in developing countries.\nResponse: Thank you very much for the positive evaluation of our work.\nThat being appreciated, these are the concerns that are noted:\n1. Patients with acute leukemia (AL) can have varied clinical presentations from frankly\n9\nleukemic (elevated WBC counts) to cytopenic. Frank leukemic presentations are also\nnoted in other myeloid malignancies such as chronic myeloid leukemia and other MPNs,\nas well as MDS-MPN overlap conditions, such as chronic myelomonocytic leukemia in\nadults and juvenile myelomonocytic leukemia in children, which have monocytosis. These\npresentations are also seen in lymphomas such as chronic lymphocytic leukemia or\nmantle cell lymphoma, to name a few. Similarly, AL also commonly presents with frank\ncytopenia mimicking other malignancies, for example, MDS, CMML (dysplastic type), and\nreactive conditions (eg, B12/Folate deficiency, sepsis, or chemotherapy-induced\ncytopenia). The training set that is used for the model uses features of frankly leukemic\npatients and, in all probability, excludes patients with other conditions or leukemia that\nhave been partly treated with disease-modifying therapy (as stated in their Lancet Digital\nHealth paper). This is a major shortcoming of the study, as these are common problems\nin the real world and are amplified in LMIC settings, which is the premise of this paper.\nResponse: We agree with Reviewer 3 that differential diagnoses of acute leukemia\nare a challenge in clinical care and have acknowledged this potential limitation in\nthe previously submitted manuscript. In response to diagnostic challenges, the\nversion already included a preprocessing pipeline for the identification of cases that\nwere out of distribution.\nFollowing the suggestions of Reviewer 3, we further expanded the preprocessing\npipeline. We validated it with several cohorts of patients with possible differential\ndiagnoses to acute leukemia (n=341) presenting with leukocytes above, within and\nbelow the normal range. It included frank leukemic presentations (e.g. chronic\nlymphocytic leukemia, mantle cell lymphoma, chronic myeloid leukemia, or\nmyeloproliferative neoplasms) or cytopenic conditions (including a variety of\ndiseases, e.g., MDS, MPN, B12 deficiency. We also had a few patients with Juvenile\nmyelomonocytic leukemia. Laboratory results of patients with severe reactive\nconditions, in particular with SARS-CoV-2 and sepsis, were obtained from two public\ndatasets (Ordoñez-Avila R, et al. Data Brief. 2023, and Johnson, A. et al. Sci Data,\n2016). The updated preprocessing pipeline including filters (e.g. proportion of\nneutrophils in leukocytosis and cytopenia) assessment and the out of distribution\ndetector was tested with these patients. The proportion of misclassified patients\nvaried depending on the constellation: Patients with leukocytes above the normal\nwere well identified as either acute leukemia or having an out of distribution\nconstellation (87.9%, Supplementary Table 8) using only laboratory data. In\ncytopenic conditions, the rates were lower (78.4%), as expected, because the\ndifferentiation of some differential diagnoses to acute leukemia can be challenging\nwith the CBC alone. For instance, MDS and AML cases may have a broad overlap in\nperipheral blood phenotypes and mutated genes. In particular in formerly AML-MR,\naccording to the current ICC classification (Arber et al. Blood 2022). Dysplasia in\nmarrow smears and histology and the percentage of marrow blasts support the\ndiagnosis (Arber et al. Blood 2022, Khoury et al. Leukemia 2022). However, there is\nalso evidence that the blast count may be substantially overlapping between both\n10\nentities (Estey et al. Blood 2022). For high-risk MDS with adverse genetic settings,\nthis ambiguity may translate into similar immediate care measures in the emergency\nroom and to a therapeutic repertoire involving early consolidating HSCT in eligible\npatients.\nWe think that combining regular medical practice (including anamnesis, physical\nexamination, and initial emergency unit assessment) with the calculator's results\nshould remain the approach to these cases at the edge of secondary AML. We copied\nthe results for your convenience:\nCategory Out of Out of distribution, Total, n\ndistribution, n\n%\nLeukocytes below normal 78.4% 40 51\nLeukocytes above normal 87.9% 109 124\nLeukocytes within normal 90.4% 150 166\nDiagnosis subsets\nMalignancies\nChronic lymphocytic leukemia 81.3% 26 32\nChronic myeloid leukemia 94.7% 18 19\nJuvenile myelomonocytic\nleukemia 100.0% 2 2\nLymphoma 62.5% 5 8\nMyelodysplastic syndromes 60.0% 6 10\nMyeloproliferative neoplasms 87.5% 7 8\nNon-malignant disorders\nVitamin B12 deficiency 71.4% 10 14\nSARS-CoV2 93.7% 118 126\nSepsis 90.4% 103 114\nOthers: Individual cases (each n= 1) with folate deficiency, PNH, TTP, parvovirus B19\ninfection and EBV infection were not reported as separate entities but are included in the\noverall statistics.\n2. Validation studies to refute the diagnosis of acute leukemia: This ML model should be\nvalidated on conditions that are not acute leukemia but present with leucocytosis and\nmonocytosis (lymphoma/chronic lymphoproliferative disorders, MDS-MPN overlap\nsyndromes), as well as cytopenia such as MDS, MDS-MPN, and acute leukemia with\ncytopenic counts. This will ensure that this model has true clinical value and is applicable\nto a resource constrained setting.\nResponse: Although this was not our initial study design, we followed this suggestion\nto validate the preprocessing pipeline and the model in conditions presenting with\nleukocytosis and monocytosis, as well as in those with cytopenia. The results\n11\npresented above indicate that the preprocessing pipeline has a low rate of false\npositive findings in cases with leukocytosis, which in our view have the greatest need\nfor rapid action, and that the classifier using limited features has indeed clinical value\nin this constellation. Such a classifier may potentially accelerate the initiation of\ncytoreductive treatments in patients with acute leukemia depending of their myeloid\n(AML) or lymphoid (ALL) lineage in resource constrained settings. We added the\nfollowing to the results section on page 9, line 243: “The developed preprocessing\npipeline did also allow to challenge the algorithm with differential diagnoses to AML,\nALL and APL presenting with leukocytosis or cytopenia (Supplementary Table 8).\nCases with leukocytes above the normal were excluded with high accuracy (87.9%),\nwhile cytopenic cases were more challenging (78.4% of leukopenia cases with other\ndiagnoses excluded).”\nBeyond LMIC, we think that this tool may also be beneficial for triage purposes in\ncountries with high income or as a referral tool in resource-poor settings.\n3. It is unclear how patients were selected for training the model or for validation. Are\nthese consecutively diagnosed patients, or is there a selection bias?\nResponse: Participating centers were contacted in the second half of 2024 and invited\nto submit consecutive patients. No selection was applied except for data availability\nat diagnosis and completeness of data at diagnosis. The probability of selection bias\nis therefore low. Plausibility checks and manual data review was centrally performed\nbut individual centers were not externally monitored as in clinical trials.\n4. There are minor errors in the supplementary data as total WBC, monocyte, platelet\ncounts are in G/L\nResponse: Thank you, there was an inconsistency, which is now solved. While the\nrepresentation of CBC results in Germany and the UK is common in counts per\nnanoliter e.g. leukocytes of 4.0 - 11.0 x 10*9/L, representations in g/L are\ncomparable and have also been adopted by Alcazer et al. Lancet Digit. Health 2024.\nWe changed the supplementary information into g/L.\nReviewer #3 (Remarks on code availability):\nYes, the github repository does have info on how to run the pipeline and generate\nsynthetic data\nhttps://github.com/UMEssen/aipal-validation\n12"}, {"round": 2, "reviewer_comments": {"reviewer_1": "The authors have satisfactorily addressed my comments.", "reviewer_3": "The authors have answered all queries. no further comments", "reviewer_4": "The authors have adequately addressed all the points raised.\nHowever, several aspects of the abstract remain unclear and would benefit from clarification:\nL89: The authors first report the performance of the “confident” AI-PAL predictions (AUROC 0.94/0.98/0.84 for acute myeloid leukemia/acute promyelocytic leukemia/acute lymphoblastic leukemia). Are these results derived from the overall merged dataset, or from a specific cohort or data split (e.g., test set)?\nL93: The AUROC reported for acute myeloid leukemia is 0.72 here, whereas 0.94 is reported immediately before. Since the test set is not defined in the abstract, it is difficult to interpret what this comparison refers to (e.g., random split of the pooled cohort, external dataset, or a selected subset).\nI therefore suggest a minor revision of the abstract to clearly specify what is being referred to in terms of the baseline model and the evaluated cohorts."}, "author_response": "Point by point response to peer review for “International testing and refinement of AI\nalgorithms predicting acute leukemia subtypes from routine laboratory data” by Turki\net al.\nRemarks to the Author\nReviewer 1: The authors have satisfactorily addressed my comments.\nResponse: We thank the reviewer for the positive evaluation of our work.\nReviewer 3: The authors have answered all queries. no further comments\nResponse: We thank the reviewer for the positive evaluation of our work.\nReviewer 4: The authors have adequately addressed all the points raised. However, several\naspects of the abstract remain unclear and would benefit from clarification:\nL89: The authors first report the performance of the “confident” AI-PAL predictions (AUROC\n0.94/0.98/0.84 for acute myeloid leukemia/acute promyelocytic leukemia/acute lymphoblastic\nleukemia). Are these results derived from the overall merged dataset, or from a specific\ncohort or data split (e.g., test set)?\nL93: The AUROC reported for acute myeloid leukemia is 0.72 here, whereas 0.94 is reported\nimmediately before. Since the test set is not defined in the abstract, it is difficult to interpret\nwhat this comparison refers to (e.g., random split of the pooled cohort, external dataset, or a\nselected subset).\nI therefore suggest a minor revision of the abstract to clearly specify what is being referred to\nin terms of the baseline model and the evaluated cohorts.\nResponse: We thank the reviewer for the positive evaluation of our work.\nHereby, we clarify the points raised by reviewer 4 and have adjusted the abstract\naccordingly. The high AUROC values in the abstract (0.94 in AML, 0.98 in APL and 0.84\nin ALL) were obtained by applying the previously published confidence cutoff to the\npooled adult cohort, which retains only predictions with high model certainty.\nHowever, this cutoff excluded 70.8–92.5% of patients from predictions, limiting its\nclinical applicability on many patient cases or leaving uncertainty for those patients\nnot qualified for confident predictions. We therefore focused on the baseline model\nwithout cutoff. The baseline AUROC of 0.72 for AML shown in the abstract refers to the\nalgorithm performance on the hold-out test set (n=2,692) used for evaluating the\noutlier detection refinement pipeline that we present in the manuscript. This test set\ncomprised all patients whose predictions were below the 0.9 confidence threshold.\nOur refined approach improved the AUROC from 0.72 to 0.84 on this challenging\nsubset while excluding only 12.1% of patients from predictions. We have shortened\nthe abstract as requested by the editorial office and specified the evaluated cohorts\nand the rationale for comparing these metrics as requested by Reviewer 4 and copy it\nhere for your convenience. The tracked version is provided within the manuscript.\nDespite advances for patients with acute leukemia health disparities limit access to diagnosis\nand treatment. Artificial Intelligence (AI) approaches may address some disparities. We\nretrospectively assembled a diverse, international cohort of 6,206 leukemia patients from 20\ncenters to test an AI tool designed to support leukemia diagnosis using standard laboratory\nresults. Executing the pretrained algorithm resulted in varying accuracy metrics. With\nconfidence cutoff predictions, 2000-fold bootstrapped area under the curve (AUROC) metrics\nwere 0.94 for acute myeloid leukemia (AML), 0.98 for the promyelocytic subtype and 0.84 for\nacute lymphoblastic leukemia. However, this cutoff excluded 70.8–92.5% of patients from\npredictions. We improved accuracy and robustness, while maintaining generalizability via an\nensemble of Isolation Forest and Local Outlier Factor increasing AUROC for AML from 0.72\nto 0.84 (hold-out test set, patients below confidence threshold), while excluding only 12.1% of\npatients. Furthermore, we retrained the algorithm for pediatric patients."}]

Decision letter