P638: DECISION SUPPORT IN THE CLINICAL ROUTINE FOR RAPID AND RELIABLE DIAGNOSTIC WORKFLOWS WITH THE HEMA.TO PLATFORM
Authors: Karsten Miermans, Hannes Lueling, Franz Elsner, Felix Kunzweiler, Aleksandra Mezydlo, Felix Spöttel, Nanditha Mallesh, Artur Toloknieiev, Holger Hauspurg
Journal: HemaSphere (2023)

Background Flow cytometry is an integral part of routine diagnostics for hematologic malignancies and is most relevant in mature B-cell neoplasms. Almost all diagnostic laboratories completely rely on expert knowledge individually applied to each patient sample to perform data analysis. The skill of flow cytometry analysis isn’t just (increasingly) hard to find, it also results in inter- and intra-observer variability. To reduce the labor time per analysis, dependency on expert knowledge and variability of data interpretation, the implementation of automated processes using machine learning (ML) is desirable. Aims: We aimed to build and validate a CE-marked decision support software (“hema.to”) that can be used in clinical routine for B-NHL screening and classification by flow cytometry using ML algorithms. Methods: During the period January 2021–January 2022, we combined thousands of flow cytometry files from multiple centers into a single database. The files were taken from clinical routine, annotated by expert users that had access to multiple diagnostic methods – hence, we took these results as a ground truth. We trained ML algorithms on these data to classify the samples (B-NHL sub-type or non-malignant, eight classes in total), using data pooling approaches to combine data from samples recorded under different protocols. We built a web-application (“hema.to”) in which selected users could upload cytometry data and diagnose new samples. Both speed and accuracy of the diagnostic recommendations was tested in a four-center blinded clinical study, where annotations against the ground truth were tested in a ‘positive arm’ with the use of hema.to and compared to a ‘control arm’ with the conventional preferred software of each center. Each of the two arms contained 96 non-overlapping randomly selected samples. Results: The four-center clinical study revealed that diagnoses performed with the support of hema.to were faster and of a high diagnostic quality. Compared to the control arm, time-to-diagnosis was reduced by >2x, and sensitivity and specificity were increased by resp. 6.4 and 7.6 percentage points. Although the quality improvement was not statistically significant, we take this as a strong indication that ML can support the expert in routine workflows. Following this initial success, we fine-tuned the model for routine use at two labs, with one diagnostic center using two device models. We found that for all three set-ups the diagnostic recommendations were of a high quality. A worst-case performance was computed by testing the model against historic data of these labs without any expert supervision, and found an f1 score of >90% and diagnostic sensitivity of ca. 96%. These results demonstrated that the software can be used as a reliable tool for both sub-typing and screening. We therefore integrated hema.to into the diagnostic routine for B-NHL screening and sub-typing at the HpH (Hamburg) since January, including a deep integration into the laboratory database. The quality of the diagnostic recommendations is robust and aids the hemato-pathologists to process their flow cytometry caseload.Summary/Conclusion: We’ve shown that machine learning can be used as a decision support tool for screening and classifying mature B-cell neoplasms from flow cytometry in a routine setting. Data pooling was necessary to achieve a clinical-grade performance. These promising results represent major strides towards decision support software for any lab and all hematological neoplasms. Such a software will not only speed up and simplify diagnostic workflows, but also improve the quality of the analysis. Keywords: Machine learning, Diagnosis, Cytometry, Artificial intelligence