Rolling window-based hepatitis toxicity prediction from routine bloodwork in patients undergoing immune checkpoint inhibitor therapy.
Authors: Csernai Eszter , Horváth Gergely , LeNoue-Newton Michele , Mittendorf Kathleen , Smith David , Park Ben Ho , Wolber Jan , Osterman Travis John 
Published: June 2022

e13565

Background: Hepatitis toxicity is one of the most important adverse effects of immune checkpoint inhibitor (ICI) therapy, occurring in approximately 10% of patients. However, when identified early, it can be managed clinically, potentially allowing continuation of ICI treatment. The goal of the study was to evaluate the feasibility and clinical usefulness of an artificial intelligence (AI) model to predict the risk of developing hepatitis toxicity during the course of ICI treatment from routine bloodwork values. Methods: Our model uses a clinical dataset of 2438 patients who received ICI treatment at the Vanderbilt University Medical Center prior to the end of 2020. Hepatitis toxicity was defined as one or more of ALT, AST, ALKPHOS, BILIRUBIN values exceeding 2.5-times the upper limit of normal value. The available feature set was limited to the routinely available blood test values. All features were normalized to the upper limit of normal and transformed to a discretized symbolic representation, a modified version of Symbolic Aggregate ApproXimation. Motifs were extracted as n-grams from the symbol series, and the counts were used as input features for the predictive model. The study uses standard data science model training and evaluation concepts: train, validation, and test splits were created randomly on the patient level; the reported evaluation metrics are median AUC, TPR, TNR, PPV, NPV over 10 sampling runs. The final, best-performing model architecture is a boosted decision tree model (XGBoost) trained on the last four blood tests to predict hepatitis at the next blood sampling timepoint (i.e., at the time of the next ICI treatment appointment). Results: The best model uses the following eight blood values as features: ALT, AST, ALKPHOS, BILIRUBIN, ALBUMIN, CO2, CALCIUM, and BUN, and achieves an AUC of 0.82 (std. 0.01), with TPR = 0.32 (0.03), TNR = 0.97 (0.005), PPV = 0.18 (0.03), and NPV = 0.99 (0.002). It finds 32% of the timepoints where the patient is going to develop hepatitis toxicity prior to their next treatment, and about 1 in 5 positive predictions are correct. It is important to note that only about 1% of all ‘sequences’ of four consecutive blood tests are followed by hepatitis at the next test. That is, while a relatively large proportion of patients are going to develop hepatitis toxicity during their ICI treatment, the timepoint at which this happens is very uncertain. Conclusions: We demonstrate that an AI model built using only already available patient laboratory data could provide clinically useful input for clinicians to support their ICI treatment decisions to reduce the occurrence of hepatitis toxicity. The dynamic nature and below-patient-level granularity of the model would allow a clinician / clinical trial investigator to make adjustments to the therapy based on individual patient reaction over time.