POS0470 A MACHINE LEARNING MODEL FOR IDENTIFYING IMPORTANT CLINICAL VARIABLES AND PREDICTING ONE-YEAR TREATMENT RETENTION IN PATIENTS WITH RHEUMATOID ARTHRITIS PERFORMING AN INFLIXIMAB BIOSIMILAR-TO-BIOSIMILAR SWITCH
Authors: N. Prenga, H. Nabi, N. Steen Krogh, A. Akkoc, M. L. Hetland, B. Glintborg
Journal: Annals of the Rheumatic Diseases (2023)

<h3>Background</h3> In Denmark, a nationwide mandatory infliximab biosimilar-to-biosimilar switch, from CT-P13 to GP1111, was recently conducted in patients with rheumatoid arthritis (RA) to save costs (ref). It is of interest to identify patients having favorable outcomes based on clinical characteristics at time of switch. <h3>Objectives</h3> We aimed to explore important baseline clinical factors by implementing a robust machine learning (ML) model to predict one-year GP1111 treatment retention. <h3>Methods</h3> Patients with RA who performed a biosimilar-to-biosimilar switch from CT-P13 to GP1111 between April 2019 -Feb 2020 (=switchers) were identified in the nationwide DANBIO registry. Clinical characteristics at GP1111 start (=baseline) were identified in DANBIO and through linkage to national patient registries (comorbidities). Missing baseline data were estimated using Multivariate Imputation by Chained Equations (20 imputations). A tree-based algorithm Extreme Gradient Boosting (XGBoost) was used to predict main outcome, defined as stop of GP1111 during one year of follow-up (stop/no stop). Contributing baseline characteristics were investigated as SHAP values. Due to imbalanced dataset (low stop rate), the ML model excellently predicted the majority outcome (no stop) but poorly the minority (stop). Thus, we incorporated rebalancing strategies (Random Oversampling, ROS, Random Undersampling, RUS, Synthetic Minority Oversampling Technique, SMOTE, Near Miss Algorithm, NearMiss) and the performance was inspected (confusion matrix). Optimal cut-offs (thresholds) were identified by Youden Index, Receiver Operating Characteristic Curve (ROC), Precision-Recall Curve to improve model-performance for the minority outcome. Analyses were performed in R and Python. <h3>Results</h3> We included 685 RA switchers whereof 17% stopped GP1111 during one year of follow-up (<b>Table 1</b>). SHAP values for 26 characteristics were investigated (<b>Figure 1</b>, see footnote). The ranges for accuracy and Area Under the ROC (AUC) were 79-87% and 0.78-0.85, with specificity and sensitivity ranging 0.79-0.99 and 0.01-0.12 respectively. This indicated a high true negative rate (specificity, predicting no stop of GP1111) and low true positive rate (sensitivity) i.e., poor prediction of GP1111 stop. With oversampling and undersampling methods, ranges for accuracy and AUC were 60-87% and 0.60-0.87 respectively, with specificity ranging 0.61-0.73 and sensitivity 0.50-0.67 corresponding to a higher true positive rate. This could be explained by RUS and NearMiss removing data from majority outcome and by ROS and SMOTE adding data to the minority outcome. Optimal threshold methods resulted in higher true positive rates and with a range for accuracy 70-85% (specificity 0.67-0.74, sensitivity 0.66-0.76). <h3>Conclusion</h3> We implemented a robust machine learning model for predicting one-year retention in infliximab switchers and identified important baseline clinical characteristics. However, the model was challenged by imbalanced data (low stop rate) which was overcome by oversampling methods and optimal cut-offs. Careful methodological considerations are needed when advanced statistical models are applied to imbalanced datasets. <h3>Reference</h3> [1]Nabi et al, doi: 10.1136/rmdopen-2022-002560 <h3>Acknowledgements</h3> This project was supported by Vinnova, Innovationsfonden and The Research Council of Norway, under the frame of Nordforsk (Grant agreement no. 90825, Project NORA). <h3>Disclosure of Interests</h3> Nikolin Prenga: None declared, Hafsah Nabi Grant/research support from: AbbVie og Sandoz, Niels Steen Krogh: None declared, Ahmet Akkoc: None declared, Merete Lund Hetland Grant/research support from: AbbVie, Biogen, BMS, Celltrion, Eli Lilly, Janssen Biologics B.V., Lundbeck Foundation, Medac, MSD, Pfizer, Roche, Samsung Biopies, Sandoz, Novartis, Bente Glintborg Grant/research support from: Pfizer, AbbVie, BMS, Sandoz.