Predicting Inpatient Length of Stay at Hospitals Using Python + Big Data | by Vishal Tien | Towards Data Science

Patient length of stay is a critical indicator of the efficiency of hospital management. Hospitals have limited resources, requiring efficient use of beds and clinician time. With the recent onset of the COVID-19 pandemic, this notion has been exemplified. Now more than ever, we can see that it is in the best interest of patients, hospitals, and public health to limit hospital stays to no longer than necessary and to have an idea of how long a given inpatient may need to stay.
The ability to predict how long a patient will stay, only with information available as soon as they enter the hospital and are diagnosed, can therefore have many positive effects for a hospital and its efficiency. A model that can predict patient length of stay could allow hospitals to better analyze the factors that influence length of stay the most. Such analysis could pave the path for reductions in the length of inpatient stay, which could, in turn, have the effect of decreased risk of infection and medication side effects, improvement in the quality of treatment, and increased hospital profit with more efficient bed management. Furthermore, predicting patient’s length of stay also greatly benefits the patients and patient’s families as they can have an idea of how long they can expect to stay upon being admitted. The goal of this project is to create a model that can predict length of stay for patients upon admission to a hospital, and the steps to accomplish such a task are detailed here.
The entire code used throughout this analysis can be found here.
To conduct this analysis, I used the publicly available “NY Hospital Inpatient Discharges in 2015” dataset, which can be found on the New York State Government health data website. This dataset contains 2.3+ million rows of patient data, including information such as patient demographics, diagnoses, treatments, services, costs, and charges. Patient data has been de-identified according to HIPAA regulations. To analyze a dataset of this size, I utilized various big data analytic tools within a Python interface such as Spark, AWS clusters, SQL query optimization, and dimensionality reduction techniques. However, the bulk of the code shown in this post uses Pandas and scikit learn. Let’s first take a look at all the features and associated datatypes in the dataset.
Next, let’s take a look at how our numerical features are correlated with each other. We can achieve this through the use of a correlation heat map.
From this initial look at a correlation map of our data, it is evident that a couple of features are strongly correlated with each other, and perhaps more importantly, with length of stay. Although the correlation matrix does not make sense to look at for all of our columns (such as the license numbers), we can see from it that APR Severity of Illness Code has a strong positive correlation with length of stay, along with total charges and total costs. CCS Diagnosis Code also seems to have a slight positive correlation with length of stay. We can see other positive correlations in the dataset between features such as CCS Diagnosis codes and APR DRG codes. This plot gives us an idea about which features might be especially important to look at to predict length of stay, in addition to letting us know if there is multicollinearity present in the data.
Next, let’s explore and visualize underlying relationships in the data. For the following analysis, length of stay is kept as the primary variable along the y-axis of the plots I create since it is the predictor variable for this project. First, the univariate distribution of length of stay is visualized.
As we can see, length of stay values range from 1 to 120+ days (120 has been aggregated to include all lengths of stay 120 days or longer). In addition, the distribution is very skewed, with most of the patients in the data having lengths of stay between ~1 to 5 days. This is an important aspect of the data to keep in mind. We’ll put a pin in this for now and revisit it later. Next, let’s take a look at how different age groups vary in their length of stay distributions.
Does length of stay vary with age of the patient?
From this plot, the trend of longer lengths of stay distributions on average as we go up in age groups is easily apparent.
Which diagnoses have the longest average length of stay?
On average, patients with diagnoses related to birth complications have the longest length of stay on average, followed by those with respiratory diseases. Let’s also look at how some features, which we might not expect to have much influence, vary with length of stay.
Does length of stay vary with race?
We can see that as one might suspects, a patient’s race does not result in largely different distributions of length of stay. However, it is interesting to note that Black/African Americans have the largest spread in length of stay values by ~2 days from the group with the next largest distribution (white).
How does length of stay vary with patient payment types?
From this plot, it is evident that different health insurance forms of payment tend to have different length of stay distributions. Specifically, we can see that Medicare patients, for example, tend to have some of the longest lengths of stay on average. However, health insurance programs are strongly correlated with other factors, such as income and age. Thus, it may be that Medicare patients tend to have longer lengths of stay because they also are in an older age bracket. Let’s dig deeper into this idea:
What is the age distribution of patients with Medicare as their primary payment typology?
This idea is supported by the plot above and gives us an idea of how different variables can interact to affect the length of stay.
Let’s move on to some of the most important features present in the dataset: the severity of illness and the type of patient admission into the hospital.
How much does length of stay vary with the severity of the illness?
We can see a large amount of variance within the APR Severity of Illness feature, indicating it will most likely be an important feature to include in machine learning (ML) models.
How does length of stay vary with type of admission?
How do diagnosis descriptions vary for each illness severity type?
Here, I perform text parsing analysis on the diagnosis descriptions column of the dataset, followed by the creation of wordclouds to visualize these results. Wordclouds were created using the diagnosis descriptions for each patient stratified by their illness severity category.
From this analysis, we can see some similarities but also some large differences in the diagnosis descriptions for each illness severity type.
In the brief discussion of health insurance programs above, I also mentioned patient income as potentially playing a role, and ultimately being a feature that might be correlated with length of stay. One example of how this might be the case is that for some low-income patients, the hospital provides better living conditions than their situation outside of the hospital. Thus, these patients are motivated to do everything in their ability to lengthen their stay. Unfortunately, our dataset does not include any information about patient income. However, it does include the 3 digit zipcode of the patient. As we prepare to begin the modeling section of this project, is important to consider the fact that zipcode is not a good feature for most ML models. Although its information is encoded by numbers, it is not a numerical feature, as its numbers do not mathematically mean anything. In turn, this makes zipcodes hard to understand and create patterns from for prediction by ML models. As a result, the next step in my analysis will be replacing this raw 3 digit zipcode data with a rough measure of patient income, while will prove a much more useful feature.
To achieve this, I webscraped average income data by zipcode from this website, which contains data from 2006–2010. Below is a plot of the median income for each 3 digit zipcode present in the dataset, achieved by inner joining (through SQL in python) the webscraped data onto our dataframe.
We can use this new income feature to visualize how other features vary with the new constructed feature.
We are now ready to begin preparing for the modeling portion of this analysis.
To begin this section, I first will drop all columns that will not be useful for predictive modeling. This includes zipcodes, diagnosis descriptions, operating certificate numbers, etc., in addition to features that would not be present at the time of patient admittance, such as total costs and total charges, thereby preventing data leakage.
Next, I performed feature encoding of all categorical columns containing strings. Categories that had corresponding numerical codes, such as APR Severity of Illness Code vs. APR Severity of Illness Description, were used as opposed to their string counterparts in order to prevent the need for category binarization and thus potentially large increases in dimensionality.
In addition, I will utilize string indexing for a particular subset of the features. For both Age Group and APR Risk of Mortality, string indexing is performed due to the fact that there is an inherent ordinality to the categories within these features. For example, age group ‘0 to 17’ is a younger age group than ’70 or Older’. As such, it makes sense to encode the youngest age group ‘0 to 17’ with a 1, and larger age groups with increasingly larger integers. Similarly, a 1 is given to the ‘Minor’ category in APR Risk of Mortality, whereas a 4 is given to ‘Extreme’. String indexing on particular features such as this can be extremely beneficial as it prevents increasing the dimensionality of the dataset and allows for models to learn the ordinality present in the feature categories. The code to manually accomplish this is shown below. However, it is important to be wary of using string indexing on features that do not have an inherent ordinality, such as ‘Health Service Area’. Thus, one-hot encoding of the remaining categorical features is performed.
mort_string_index = {'Minor': 1, 'Moderate': 2, 'Major': 3, 'Extreme': 4}age_string_index = {'0 to 17': 1, '18 to 29': 2, '30 to 49': 3, '50 to 69': 4, '70 or Older': 5}df['Age Group'] = df['Age Group'].apply(lambda x: age_string_index[x])df['APR Risk of Mortality'] = df['APR Risk of Mortality'].apply(lambda x: mort_string_index[x])
Now, let’s take a look at our predictor feature, length of stay. Length of stay ranges from 1 to 120 and only takes on integer values. I decide to treat predicting length of stay as a multi-class classification problem (instead of regression). Multi-class classification yields one large advantage in comparison to regression that I utilize: the ability to manually define class bins to manually control prediction specificity. Instead of treating length of stay as having 120 different classes, we can group these values into bins that make more sense for predictions without a significant loss in specificity of predictions. Following the exploration of multiple options for these bins, which involved trading off usefulness of the model (if bins are too large, the predictions will no longer be useful) and model accuracy, I ultimately settled on the following bin format:
1–5 days, 6–10 days, 11–20 days, 21–30 days, 31–50 days, and 50–120+ days
From this visualization, a large class imbalance is apparent. Upon revisiting the univariate distribution of the raw length of stay data that was plotted at the beginning of this analysis, the same trend is apparent. This imbalance in the data must be handled carefully, as it can lead to misleading accuracy scores. Class imbalance can lead to a model over-predicting the class that occurs the most often, as simply predicting this class for most of the data instances, regardless of their features, will lead to the highest overall accuracy score. Thus, a model aiming to optimize accuracy score may fall into this trap. For example, even if the model were to always predict an inpatient’s stay would be 1 to 5 days (the most common length of stay in this dataset), it would achieve an accuracy of ~1.6e6/2.3e6 = 69%. In comparison to the baseline accuracy of the model (the accuracy the model would achieve if it randomly guessed), which is 1/6 or ~17%, this is significantly better model performance at first look.
Under-sampling the most occurring class and assigning penalties for over-predicting classes are two of the most common methods for dealing with class imbalance, both of which were explored on this dataset. Ultimately, I found that assigning penalties was the most effective method, resulting in higher model accuracies than the under sampling method, while equally preventing the associated risks with class imbalance as when under sampling was performed. In order to assign these penalties to classes, I made use of the ‘class weight’ parameter in scikit learn’s ML functions, and set this parameter to ‘balanced’. This effectively assigns a weight to each class inversely proportional to the frequency with which it appears.
PCA:
Prior to training any models, I perform PCA on the data after using the StandardScaler() function to normalize the train and test data sets. PCA is a powerful tool that allows one to reduce the dimensionality of a dataset, which can be extremely beneficial for large datasets such as the one we are dealing with here. In addition, it removes any multicollinearity in the data. Although PCA is not intended for use on datasets with a mix of numerical and categorical features (that have been one-hot encoded/string indexed), performance was found to not be harmed significantly with the use of PCA (more can be read here about when to use PCA vs. MFA). It is important always normalize the data before applying PCA because PCA projects the raw data onto directions which maximize variance by calculating relative distances between points within a feature. Thus, when features are on different scales, PCA can be thrown off.
import numpy as npfrom sklearn.decomposition import PCAfrom sklearn.preprocessing import StandardScalerx_train = StandardScaler().fit_transform(X_train)x_test = StandardScaler().fit_transform(X_test)pca1 = PCA()pca1.fit(x_train)explained_variance_ratio = pca1.explained_variance_ratio_pc_vs_variance = np.cumsum(pca1.explained_variance_ratio_)
Below, I plot the % explained variance against the number of components from PCA that are used. From this graph, we can decide how many components to use for the rest of the analysis in order to retain a certain percentage of explained variance in the data. Here, I choose to keep 29 components as this is the minimum number of components required to explain 95% of the variance in the data.
Logistic Regression:
Next, let’s train a one-vs-rest logistic regression classifier. Logistic regression classifiers are strong models for classification problems. Let’s train a model after applying PCA to the standardized data, as shown below.
pca = PCA(n_components=29) #29 components, as found abovex_train = pca.fit_transform(x_train)x_test = pca.transform(x_test)from sklearn.linear_model import LogisticRegressionfrom sklearn.metrics import accuracy_scoreimport numpy as nplog_reg = LogisticRegression(multi_class='ovr').fit(x_train, y_train)y_train_pred = log_reg.predict(x_train)y_pred = log_reg.predict(x_test)test_acc = accuracy_score(y_test, y_pred)train_acc = accuracy_score(y_train, y_train_pred)print('Test accuracy:', test_acc)print('Train accuracy:', train_acc)
Train accuracy: 0.7315117403879348Test accuracy: 0.7318516452316673
We get very good results given the baseline accuracy and nature of this dataset. However, this model was trained on the data without balancing any class weights. It is good practice to create a confusion matrix after making predictions to allow for closer examination of model performance, as an accuracy score alone can be misleading.
The confusion matrix showcases the dangers of ignoring class imbalance in a dataset. Although we get a very high accuracy score, the model is significantly over-predicting prediction label 5 as it is the label with the highest frequency. On the other hand, the model never predicted labels 30 and 50. Both these trends are evident in the darker colors within the heat map for the first column in contrast to the lightest colors in the 30 and 50 columns (also denoted by 0's). Now, let’s see if we can deal with this issue.
Logistic Regression with balanced class weight parameter:
from sklearn.linear_model import LogisticRegressionfrom sklearn.metrics import accuracy_scoreimport numpy as nplog_reg = LogisticRegression(class_weight='balanced', multi_class='ovr').fit(x_train, y_train)y_train_pred = log_reg.predict(x_train)y_pred = log_reg.predict(x_test)test_acc = accuracy_score(y_test, y_pred)train_acc = accuracy_score(y_train, y_train_pred)print('Test accuracy:', test_acc)print('Train accuracy:', train_acc)
Train accuracy: 0.5892498813701775Test accuracy: 0.5889674625391544
As we can see, defining the class weight parameter had huge improvements on the performance of our model, preventing the classifier from simply over-predicting the highest frequency classes. Although the overall accuracy drops, our model will now be a better predictor for any given instance during future “real-world” testing. Furthermore, depicted by the darker colors along the diagonal of the heat map, even when the model does predict the wrong label, it is usually off by one label to the right or left.
Let’s print out some more classification metrics to get an even closer look at how our model is performing:
from sklearn.metrics import classification_reportprint(classification_report(y_test, y_pred))
Let’s use the f1-score to compare the logistic regression classifier to the rest of the models we will create. F1-score is a measure of a model’s accuracy that is calculated by taking the harmonic mean of precision and recall, thus combining both precision and recall, into one metric. If you are unfamiliar with the concepts of precision and recall, refer to this article to learn more. In essence, f1-score provides a quantitative measure for assessing the tendency to over-predict certain class labels and the ability for a model to separate classes.
Decision Tree:
For our next model, let’s try training a decision tree classifier on our data. Decision trees tend to perform very well in multi-class classification problems and are intuitive to understand. However, before we create the model, let’s optimize the hyperparameters for the decision tree classifier first. To do this, we can run a randomized search across a range of hyperparameter value combinations defined by a dictionary. From this search, we can then return the optimal values that are found. This hyperparameter search is enabled through the scikit RandomizedSearchCV function. Here, I choose to optimize the max_depth and max_leaf_nodes parameters, as they are key parameters in preventing overfitting, a property that decision trees are especially prone to. 3-fold cross validation is utilized.
from sklearn.tree import DecisionTreeClassifierfrom sklearn.model_selection import RandomizedSearchCVdtree = DecisionTreeClassifier(class_weight='balanced')search_vals = dict(max_depth=[35,50,75,100], max_leaf_nodes=[800,1000,1500,2000])dtree_search = RandomizedSearchCV(dtree, search_vals, cv=3)search = dtree_search.fit(X_train,y_train)search.best_params_
Output: {‘max_depth’: 50, ‘max_leaf_nodes’: 1000}
I also plot a validation curve while utilizing 3-fold cross validation in order to visualize the effect of hyperparameter optimization and model complexity against model accuracy. The validation curve for the max_depth parameter is shown below.
The validation curve highlights tendency for decision trees to overfit as the max_depth parameter becomes less stringent (allowing deeper trees). However, it is interesting to note that although there is a large gap between training and validation accuracy, validation accuracy does not go down between this max depth range.
Using the hyperparameters found above, let’s train a decision tree classifier and use it to create predictions. Below is the code I used to train the model. Let’s set the ‘class_weight’ parameter to ‘balanced’ just as before. Note that I train the model on the raw training data, and not the PCA scaled + transformed data, as decision trees are scale-invariant. Thus, they do not require the training data to be scaled. In addition, lower accuracy was achieved when the decision tree was trained on the PCA transformed data.
from sklearn.tree import DecisionTreeClassifierdtree=DecisionTreeClassifier(max_depth= 50, max_leaf_nodes=1000, class_weight='balanced')dtree.fit(X_train,y_train)from sklearn import metricstrain_predictions = dtree.predict(X_train)test_predictions = dtree.predict(X_test)print("Train Accuracy:",metrics.accuracy_score(y_train, train_predictions))print("Test Accuracy:",metrics.accuracy_score(y_test, test_predictions))
Train Accuracy: 0.6170775710955541Test Accuracy: 0.6134612839367724
Ultimately, we can see that accuracy has gone up with the decision tree classifier! F1 scores for each class label has also increased, indicating an increase in model performance. Let’s see if we can do better with a decision tree ensemble, otherwise known as a random forest model.
Random Forest:
We can improve upon the decision tree classifier by bagging the trees into an ensemble and utilizing the power of majority voting between all of the trees built. This is called a random forest model. Below is the code and model performance for this model. Note that model parameters were chosen by trial and error (observing f1 and accuracy scores simultaneously) as a grid search/randomized search was very computationally expensive. However, parameters could be further optimized using said techniques.
from sklearn.ensemble import RandomForestClassifierrf = RandomForestClassifier(n_estimators=150, max_depth=15, class_weight='balanced')rf.fit(X_train,y_train)train_predictions = rf.predict(X_train)test_predictions = rf.predict(X_test)print("Train Accuracy:",metrics.accuracy_score(y_train, train_predictions))print("Test Accuracy:",metrics.accuracy_score(y_test, test_predictions))
Train Accuracy: 0.6729519125323106Test Accuracy: 0.6436519717021008
We have improved on the accuracy of the decision tree, as well as on the f1 scores for each class. Next, let’s take advantage of one of the functions in scikit to generate a plot of feature importances in the random forest model. Here, feature importance is calculated as the decrease in node impurity weighted by the probability of reaching that node in the tree.
From this plot, we can see that the APR DRG Code (a classification system that classifies patients according to their reason of admission, severity of illness and risk of mortality) and the APR Severity of Illness Code are the two most important features in predicting a patient’s length of stay. Interestingly, the payment typology medicare subgroup has a relatively high importance in predicting length of stay in comparison to the other payment typology groups. Average income of the patient also played a significant role.
Boosted Decision Trees:
Lastly, let’s see what performance we can achieve with another type of ensemble method: boosting. Using the AdaBoostClassifier in scikit learn, we can create a boosted decision tree classifier. This model works by creating n short decision tree stumps, where each successive tree stump that is built aims to improve on the classification error of the one before it. The majority vote of these trees is then combined to produce final predictions. The code used to achieve this is shown below, where hyperparameters were optmized using equivalent methods to those used for the random forest model.
from sklearn.ensemble import AdaBoostClassifierfrom sklearn.tree import DecisionTreeClassifierfrom sklearn.metrics import accuracy_scoredtree = DecisionTreeClassifier(random_state = 1, class_weight = "balanced", max_depth = 15)boost = AdaBoostClassifier(dtree, n_estimators=75, random_state=0)boost.fit(X_train, y_train)train_predictions = boost.predict(X_train)test_predictions = boost.predict(X_test)print("Train Accuracy:", accuracy_score(y_train, train_predictions))print("Test Accuracy:", accuracy_score(y_test, test_predictions))
Train Accuracy: 0.9397049441494455Test Accuracy: 0.7065437824744865
Here, we can see boosted decision trees achieved the highest model accuracy, beating out both the logistic regression classifier and the random forest ensemble. However, most of the f1 scores for the minority classes (less frequently occurring), are slightly lower. Thus, changes in how severe a false positive or false negative may be on a patient to patient basis may warrant switching between the use of the random forest model or the boosted decision tree classifier as the optimal model.
Spark Cluster Environment:
Due to the size of this data, some modeling aspects such as hyperparameter optimization took a long time to run. Thus, I wanted to include a note on the benefits of Apache Spark. We can achieve significant code speedup if we take advantage of Apache Spark and AWS clusters by parallelizing many computationally intensive tasks. Check out the full code I have linked at the beginning of this blog post for scripts that can be used to accomplish this. This code includes how to set Spark up in a Python environment, connect an AWS cluster to the notebook, retrieve the dataset from an S3 bucket, and create models. One of the drawbacks I found of using Spark for this project was the lack of the class weight parameter in the Spark ML model functions. In the full code, I also show a method for manually assigning class weights. This code was adapted from a blog post that can be viewed here.
Ultimately, through the steps highlighted in this blog post, I was able to predict a patient’s length of stay using data only available from the moment they enter the hospital and are diagnosed with an accuracy of ~70%. Such a model has the ability to profoundly improve hospital management and patient well-being. Future directions could look to a develop a second model that would allow for a sequence of model predictions to be made that would ultimately predict how much a given patient will cost to treat over the course of their stay. This could be accomplished through first predicting length of stay through models such as the one created here. Next, a second model could be created to predict “Total Costs” from length of stay.
Before putting a model such as the one created here into use, important ethical considerations need to be taken into account as well, since features such as patient income and race were used to predict length of stay. More on this topic can be read about here.
Takeaways:
Benefits of classification vs. regression in a multi-class classification setting
Importance of string indexing vs. one-hot encoding categorical variables
Importance of dealing with class imbalance
The effectiveness of the class_weights parameter, which removes the need to under sample the highest frequency class (under sampling has the drawback of reducing the amount of training data available to train on)
General steps required to take a raw dataset and use it to gather insights and generate predictions
APR DRG Codes and APR Severity of Illness Codes are the two most important features in predicting a patient’s length of stay. In addition, the payment typology medicare subgroup has a relatively high importance in predicting length of stay in comparison to the other payment typology groups. Average income also had a significant influence on length of stay.
References:
[1]. Baek, H., Cho, M., Kim, S., Hwang, H., Song, M., & Yoo, S. (2018). Analysis of length of hospital stay using electronic health records: A statistical and data mining approach. PloS one, 13(4), e0195901. https://doi.org/10.1371/journal.pone.0195901
[2]. Scikit-Learn. (n.d.). Retrieved from https://scikit-learn.org/stable/
The work in this article expanded upon my final project and leveraged concepts I learned in CIS 545: Big Data Analytics at the University of Pennsylvania