Clinical large language model to predict loss to follow up for oncology patients discharged to skilled nursing facilities.
Authors: Tianshi Liu, Ziyi Chen, Yonghui Wu, Yi Guo, Jonathan Alexander Chatzkel, Jiang Bian
Journal: Journal of Clinical Oncology (2024)

e13602 Background: The prognosis of cancer patients is heavily influenced by performance status which also informs an oncologist’s decision for initiating or continuing with cancer treatment. From SEER data, there is evidence that oncology patients who are discharged to a skilled nursing facility are less likely to receive chemotherapy and radiation. Patients with advanced cancer discharged from hospital also have a readmission rate as high as 34 percent. Those that are unlikely to follow up are a vulnerable population that should be identified to discuss how to best optimize post hospitalization care. Methods: University of Florida's Integrated Data Repository (IDR) was queried for established oncology patients at University of Florida Health from 2012 – 2023. The cohort was further narrowed down to patients who had exposures to antineoplastics within 6 months and loss to follow up was defined as not having a clinic visit within 6 months of hospital discharge. The study cohort included 194 patients and 97 patients were not lost to follow up and 97 patients were lost to follow up. Structured health data including ICD-9 and 10 codes, antineoplastic drug exposures, Area Deprivation Index (ADI), discharge vitals were extracted. Dataset was split into 70% training and 30% testing. Using logistic regression, random forest and XGBoost, loss to follow up was predicted with 5 fold cross validation and hyperparameter tuning with GridSearchCV for the best ROC AUC. Results: The median Charlson score for cohort was 11, median length of stay in the hospital was 7 days, median ED visits within 6 months was 8, median LACE score of 21. 86% of the cohort had Medicare, 6% with Medicaid, 6% Blue Cross, 2% Managed Care and 0.5% Other. There were 96 females and 98 males. 66% of the cohort was Caucasian and 31% Black, 3.9% Hispanic, 1% Asian, 0.5% multiracial. XGBoost and random forest performed better than logistic regression, both achieving a AUC ROC of 0.83 vs. 0.78. XGBoost had a higher F1 score of 0.75, sensitivity of 0.71, compared to 0.68 and 0.55 respectively for both Random Forest and logistic regression. The most important features for Random Forest were ED visits within 6 months, followed by diastolic blood pressure, heart rate, temperature then systolic blood pressure on day of discharge. The most important features for XGBoost were diagnoses of pulmonary congestion, pulmonary embolism, pleural effusion and type 2 diabetes with complications. Conclusions: Our XGBoost model achieved metrics which are in line with the higher performances seen with machine learning models predicting readmissions following hospital discharge. This serves as a good baseline model to compare against new state of the art models that incorporate unstructured data such as clinical notes and radiology image reports. These models are currently under development by the same authors using the clinical large language model GatorTron.