P617: FACING LIMITED DATASET IN HEMATOLOGICAL MALIGNANCIES: OPTIMIZATION OF MACHINE LEARNING MODELS WITH THE GENERATION OF NEW TRAINING VARIABLES AND HYPER-PARAMETERS SELECTION.
Authors: R. BIZOÏ, L. Mauvieux
Journal: HemaSphere (2022)

Background: There is a growing interest in the application of Machine-Learning (ML) techniques in medical research. However, the datasets in this field concern a limited number of individuals. Our goal was to implement technologies and measures for estimating learning performance in a small dataset of B-cell chronic lymphoid malignancies with extreme individuals (outliers) and a modality imbalance for the target variable. Aims: We postulated that a limited set of antigen expression could decipher the different diagnosis of chronic B-cell malignancies (CBCM). In this aim we retrospectively studied the flow cytometry results obtained at the diagnosis for 673 patients explored in our center: 212 CLL/lymphocytic lymphoma, 160 lymphoplasmacytic lymphoma (LPL, including Waldenstrom Macroglobulinemia), 76 mantle cell lymphoma (MCL), 169 marginal zone lymphoma (MZL), 35 Hairy cell lymphoma (HCL) and 27 follicular lymphoma. We focused on median fluorescence intensity (MFI) values of CD148, CD180 and CD200 and three categorical “positive or negative” markers (CD5, CD23, FMC7). Methods: We used the Scikit-Learn python library and LightGBM to test several ML models and develop techniques for generating new variables and selecting hyper-parameters. The treatments on the dataset were the following: 1. The three quantitative dimensions were normalized using Box-Cox transformation for LightGBM, LogisticRegression, RandomForest, KNN and GaussianNaiveBayes algorithms. 2. 70% of the data for training and 30% for testing were randomly separated. 3. Outlier removal and diagnostic dimension, modality balancing treatments were performed only on the learning sample using different unsupervised methods: localOutlierFactor, IsolationForest, OneClassSVM and EllipticEnvelope. 4. In order to avoid overfitting, the number of individuals of the least frequent modalities were balanced using (Synthetic Minority Over-sampling Technique) SMOTE algorithm. 5. For a better accuracy of classification algorithms, a discretization was set on the quantitative dimensions (CD148, CD180 and CD200 MFI). Thus all the values of the three dimensions were merged into one and this dimension was discretized into 96 quantiles. 6. New variables were created, that allowed a better learning for the LightGBM and RandomForest classification algorithms. Polynomial transformation and a principal component analysis were used for Logistic Regression. 7. Algorithms hyperparameters of ML were optimized in order to improve performance using GridSearchCV. The selection of the best ML models was done using cross-validation. Results: The models were used for the prediction of CLL, LPL, MCL and MZL diagnosis. The results for each of the classifiers are summarized in Table 1. The global accuracy of the prediction using only 6 antigenic markers was very stable, ranging from 80 to 85% for the different algorithms. The F1-score, that represents the harmonic mean of the precision and recall of prediction (TP/(TP+ ½* (FP + FN)) was the best for CLL (f1 = 91,4%) and the lowest values were obtained for MCL and LPL. Image:Summary/Conclusion: In conclusion, the methodology presented here allows an optimization of machine learning algorithms for limited set of data. Using only the results of six flow cytometry stainings, that can be analyzed in a single tube, high diagnosis prediction rates were obtained. These results, that need to be confirmed in different series, open the way to innovating diagnosis assistance tools in the field of hematological malignancies. These methods can also be applied to other diagnosis issues.