Predicting ICU Length of Stay for Patients using Latent Categorization of Health Conditions (2025)

Abstract: Predicting the duration of a patient’s stay in an Intensive Care Unit (ICU) is a critical challenge for healthcare administrators, as it impacts resource allocation, staffing, and patient care strategies. Traditional approaches often rely on structured clinical data, but recent developments in language models offer significant potential to utilize unstructured text data such as nursing notes, discharge summaries, and clinical reports for ICU length-of-stay (LoS) predictions. In this study, we introduce a method for analyzing nursing notes to predict the remaining ICU stay duration of patients. Our approach leverages a joint model of latent note categorization, which identifies key health-related patterns and disease severity factors from unstructured text data. This latent categorization enables the model to derive high-level insights that influence patient care planning. We evaluate our model on the widely used MIMIC-III dataset, and our preliminary findings show that it significantly outperforms existing baselines, suggesting promising industrial applications for resource optimization and operational efficiency in healthcare settings.

## introduction intensive care units (icus) deliver critical care for severely ill patients, but due to the high costs associated with their setup and operation, hospitals face limitations on the number of available icu beds. efficient resource management is essential to maximize icu capacity and avoid life-threatening shortages. predictive planning, powered by historical patient data-such as medical history, test results, treatments, nursing notes, and previous icu admissions-can significantly enhance the allocation of icu resources. by leveraging advanced analytics and machine learning models, healthcare providers can optimize bed usage, streamline staffing, and improve patient outcomes, ensuring that icu resources are deployed where they are needed most. this approach has wide industrial applications in healthcare operations, improving both efficiency and patient care while reducing operational costs. nursing notes contain vital information about a patient's physical and psychological condition, offering insights beyond physiological data or radiology reports. these notes also document a patient's response to treatment through behavioral descriptions, making them a rich source for predicting critical care needs. our model leverages unstructured nursing notes, which include linguistic expressions like "extensive cardiac hx" or "slightly tachypneic," providing human assessments that numerical data alone cannot capture. these details are crucial for distinguishing between similar patients with different treatment responses. figure 1 illustrates a sample nursing note with highlighted clinical details. earlier models typically process all nursing notes as input to predict a specific output, limiting their ability to predict outcomes during the icu stay @xcite @xcite . recent efforts have aimed at early prediction of icu length of stay (los), readmission, and interventions, but their performance remains sub-optimal due to the lack of domain knowledge and the nuances of text discourse @xcite @xcite @xcite . in this paper, we present a technique for predicting icu length-of-stay (los) by analyzing nursing notes, a rich source of unstructured data. by extracting health status information from these notes, our model identifies both common and unique features, leading to enhanced prediction accuracy. we introduce a joint model of latent note categorization, which recognizes critical health contexts that shape language patterns in nursing documentation. this model not only improves predictions but also offers insights that can be used for more efficient icu resource management. evaluated on the mimic-iii dataset, our approach outperforms competitive baselines, including large language models such as llama-3.1 and fine-tuned biomistral-7b. these results demonstrate the potential of integrating unstructured text data into industrial applications like predictive healthcare analytics, optimizing icu operations, and improving patient care strategies. ## the proposed los prediction model we define the problem as follows: let x be a set of n nursing note transcripts. each x i is a sequence of m i nursing notes for patient i, where the model takes a sequence of nursing notes p ij and predicts the remaining length of stay y i . its success is measured by prediction accuracy and the timestamp at which the correct prediction is made. the earlier the prediction, the more valuable it is to users. ## joint latent note categorization for icu-los prediction based on the work of @xcite , we have adopted a similar network architecture for predicting the icu length of stay(los) for an individual patient. we modified the above architecture by categorizing the daily nursing notes for a patient (p) for the day (t) along with the encoded clinical details (h t p ) of the patient. thus, we propose an nursing note categorization model that jointly learns to predict the icu los of a patient from the nursing note transcripts and encoded clinical details while grouping the information into their respective health condition (hc) classes. the rational behind the joint categorization is the fact that icu stay for a patient will largely depend upon patients' progressive health condition. a detailed overview of the model architecture is depicted in figure the details of each of the components are discussed in the following subsections. we represent every day nursing note of a patient as contextual embeddings n @xmath0. along with this we extract the specific clinical details of the patient h p (t) from the notes as discussed in section a.5. we concatenate these two representations together and form a patient-centric contextual embedding p @xmath1. where v is the dimension of the clinical detail vector. we hypothesize that each note can be grouped into k latent categories such that similar category of patient will exhibit unique, useful patterns. we have used the health condition (hc) of each patient per day, corresponding to each note as the latent categories. to perform a soft assignment of the notes to the hc classes, for each note, our model computes a category membership vector h here, h j represents the probability distribution for the j th note of the patient over each of k latent categories for the patient's health condition. h j is computed as a function ϕ of p j and trainable parameters θ ci . this is depicted as the category inference layer: based on these category memberships for each nursing note, the model then analyze the corresponding health categories so that unique patterns can be learned for each category. specifically, we form k category-aware note aggregations ( p k t ). each of these aggregations, ( p k t ) ∈ r e , is a category-aware representation of all the nursing notes till the t th timestamp with respect to the k th category. here, h k t is the k th scalar component of the latent category distribution vector h t . z k t is the normalizer added to prevent varying signal strength, which interferes with training. we then compute the output class probability vector y i as a function ψ of the note aggregations [ p 1 t , ..., p k t ] and trainable parameters θ d (illustrated as the decision layer in figure 2 ). the predicted label y i is selected as the class with the highest probability based on y i . ## evaluation experiments: we investigate the performance of the proposed model in terms of the following criteria: a) efficacy of the joint model with respect to the other base lines. b) prediction accuracy of the network architectures, and c) the timeliness of the prediction. accordingly, we propose baseline models that considers only the nursing notes as input (notesonly), clinical details (h p (t)) only (cd), and taking both the inputs into account but without considering the joint categorisation (notes+cd). in terms of the neural network architectures, we have used the clinicalbert and blue-bert models @xcite fine-tuned on our dataset as baselines. we also present our experimental results on fine-tuned open-source llms such as llama-3.1 @xcite and biomistral-7b @xcite . first, we have evaluated the los prediction ability of llama-3.1 using zeroshot @xcite and few-shot prompt techniques. here, we have used the few-shot technique demonstrated by @xcite and given examples of series of notes for two patients as prompt. we have also fine-tuned the pre-trained biomistral-7b model with the mimic-iii dataset to compare its ability to perform los prediction. details of the fine-tuning process is discussed in appendix a.1. evaluation metrics: prediction accuracy of the models are computed in terms of evaluation matrices such as r 2 score for accuracy, mean absolute error (mae), and root mean squared error (rmse). we have also performed evaluation with area under the roc curve (auc-roc) and cohen kappa scores. the roc curve shows the trade-off between true positive rate (tpr) and false positive rate (fpr) and provides the ability of a classifier in distinguishing between classes. the closer an auc-roc curve is to the upper left corner, the more efficient in distinguishing the classes. cohen's kappa score measures the agreement between model predictions and actual class values and it is defined by, κ = p 0 -pe 1-pe where p 0 is the observed agreement of the model and p e is the chance agreement. since the model aims to predict icu los, it is important to evaluate how early it provides predictions. early warnings enable hospital administrators to adjust strategies effectively. to measure this, we calculate the time between the model's initial warning and the end of the patient's icu stay. we introduce a time-coupled prediction score, which modifies the existing evaluation parameters by combining the prediction accuracy with the elapsed time from the model's warning to the patient's icu discharge. accordingly, we modify the mae, and rmse scores of the proposed model as follows: where, τ is the elapsed time from the model's warning to the patient's icu discharge and ϵ is a constant set to 0.0001. all the models have used sentence embeddings from either the pre-trained bluebert or the pretrained clinicalbert model. the models are trained using the adam optimizer. mean validation performance was used to select hyper-parameter values. we trained the models with 10 epochs, and the learning rate of 5 × 10 -4 . ## conclusion in this paper, we develop a neural network architecture that uses the nursing notes, prepared at the time of admission to icu, to predict icu los. the novelty of the model lies in the fact that it processes the the notes during the development of the patient's icu stay. we proposed a joint model of latent categorization of patient's health status for the task. we have demonstrated that the proposed approach allows the model to identify high-level health status that influence the prediction. results showed that the proposed joint model outperforms the baseline systems that uses individual clinical notes or health status representations. david t wong and william a knaus. 1991. predicting outcome in critical care: the current status of the apache prognostic scoring system. canadian journal of anaesthesia, 38(3):374-383. li yujian and liu bo. 2007. a normalized levenshtein distance metric. ieee transactions on pattern analysis and machine intelligence, 29(6):1091-1095.