Predicting How Expensive A Healthcare Provider is for the Government | by Tristan Bonds | Towards Data Science

As Metis Data Science Bootcamp students, we were tasked to build a linear regression model for our first individual project. Simple enough right? Just draw a line through data.
Not so fast!
As a healthcare professional, I was excited to use this algorithm to solve a problem in the medical field. But what I quickly found out was, before any modeling, a successful machine learning project starts with picking the right kind of data. Linear regression works best with continuous numerical data which excludes several sources of publicly available healthcare data. However, it turns out that Medicare payment data is a perfect fit.
But why care about Medicare? Medicare is a government-funded health insurance program currently covering 44 million people, or 1 in 8 individuals in the U.S. (1). That carries with it some huge public health and financial implications for the government. As soon as you turn 65, you’re eligible for it with a few exceptions such as younger people with disabilities or those suffering from end stage renal disease. And this population is only getting bigger. In fact, the Census Bureau estimates that by 2030, the elderly population alone will nearly double to 78 million people or 1 in 5 Americans (2).
On top of this, it seems likely that Medicare is going to be increasingly more important in the coming years due to its prevalence in political discussions around the country. If the United States were to adopt a single-payer system, such as the proposed Medicare-for-All, it would be essential for us to lower costs wherever possible, and the effective usage of data could help to achieve this.
One way to do this would be to look at healthcare provider costs. You can follow along with my code on my GitHub repository; I’ve organized it chronologically with this article for your convenience.
After searching the Center for Medicare and Medicaid Services website, I was able to acquire the most recent provider payment data which had over 1 million rows, each corresponding to a healthcare organization or individual provider, and 70 features.
But as expected, this data was quite a mess. So in order to glean the most reliable insight I could, I decided to narrow down what I was measuring to only individual providers in the U.S, excluding territories and military zones.
I then replaced all missing values in columns with count or percentage data with zeroes. It is likely that if a provider doesn’t have patients with a certain disease, the field is just left blank; this means it should have been relatively safe to impute nulls without losing much fidelity.
There was also some documentation provided for the meaning of each column name. I decided the target variable should be total_medicare_payment_amt, which is the total amount the government paid for all the provider’s services per patient after deductible and coinsurance amounts have been subtracted. Additionally, I removed all columns that were either unnecessary (like provider names) or that would lead to possible data leakage (like other price-based columns).
Finally I was left with a little over 990,000 clean rows with 38 features. Let’s begin!
Using the Statsmodels python library, I just threw all my data into the ordinary least squares (OLS) linear regression to see how it would perform initially with no modifications.
As previously mentioned, linear regression tries to find a linear relationship between independent variables and a dependent variable. Above, you can see the simplest univariate form with only one independent variable or feature. It uses the equation y = mx + b to find the best fit with the data; m is the slope coefficient and b is the y intercept.
But obviously with 38 features, this linear regression problem is a lot more complex. In this case, there will be 38 “mx” terms added together, with each m term corresponding to the size and direction of the effect that specific variable is having on the dependent variable. In geometric terms, we will be fitting a 38th dimensional hyperplane to 39th dimensional space (instead of a line). If you find a way to visualize this, let me know!
Ok, so now we have some intuition for the model, but how do we determine how well the model is doing?
The metric that is commonly used here is called the coefficient of determination or R-squared. Essentially, it is the percentage of variance of the target variable that is predicted by the features. We want an R-squared close to 1 which indicates that the model is very predictive.
But let’s dive a little deeper into the actual formula of R-squared because it helps us to understand how we are evaluating the model. The most naive method we could use to predict Medicare costs would be to just guess the average cost. That is the green ȳ (called y-bar) in the the diagram above. This will be our baseline.
But we can do a lot better than that by using linear regression or the red ŷ (called y-hat). Now we just find how off these two predictions are from the actual value and divide them by each other (SSE/SST). This will tell us the percentage of variance the model cannot explain. But what we really want to know is what percentage of variance this model does explain. Subtracting that value from 1 will get us there.
1 - (Error Sum of Squares/Total Sum of Squares) or 1 - (SSE/SST)
And after running the initial model, the R-squared was 0.619. That means that our model only accounts for about 62% of the variation in the data. That’s not too good.
But wait! Linear regression has many assumptions and it is important to check if our data is actually working for this model.
Assumption #1: Is there a linear relationship between the target variable and the features?
For illustration purposes, if we use the feature of total medicare cost, it’s not entirely clear. To rectify this situation, we could do some feature engineering. One option is to do a logarithmic transformation of both the feature and the target.
Wow! That’s a dramatic improvement. Anyone can draw a line through that! As you can see, it is often the case that we need to transform the data in specific ways to make it conform to the assumptions of the model we are using.
Note: Always remember to undo this transformation afterwards in order to return your value back to the original context. Because after all, what does the logarithm of Medicare costs even mean?
Assumption #2: Are the target and the features normally distributed?
In the above figure, the left plot shows the target variable before using a logarithmic transformation; as you can see, it is terribly right skewed. The right plot, on the other hand, shows how applying this transformation results in a remarkably normal distribution.
Assumption #3: Is there little to no multicollinearity among the features?
Multicollinearity is when the features are highly correlated with each other. Above, we see a heat map where the darker colors indicate strong positive correlations. Ideally, we would see only light colors everywhere else except the diagonal line across the middle, as obviously a variable will be perfectly correlated with itself.
But in reality, we see darker colors popping up all over the place, which indicates we are violating this assumption. This can lead to imprecise regression coefficients or worse, changes in sign for the same features in different samples, which makes it difficult to reliably extract meaning out of those coefficients.
The way to go about solving this is to remove features until there is no longer any collinearity. As will be discussed later, regularization techniques do this for you by zeroing out coefficients of some of the features that are collinear with each other.
Assumption #4: Are the residuals correlated with themselves?
An autocorrelation happens when the residuals for a specific feature are not independent from each other. This is considered bad because it indicates the model is not extracting all the information possible from the data, and thus, we see it in the residuals.
This can be measured through the Durbin-Watson test. Values near 2 indicate no autocorrelation, while values near 0 or 4 indicate strong autocorrelations. Our initial model has a value of 1.998, indicating that the model is extracting as much information as possible and the assumption has been met.
Assumption #5: Is the data homoskedastic?
What we want to avoid here is heteroskedasticity, a big word with a simple explanation. This is when the variance of the residuals change across the range of values in a feature.
As you can see in this hypothetical example, it is very clear that the variance gets wider as age increase. This is not good as it means that our model will get worse at making predictions the older someone gets. What we really want is a consistent predictability and variance across the entire range of values, known as homoskedasticity. In other words, the two dotted red lines would be parallel to each other.
Here we see the predicted values versus the residuals for our model on the Medicare data. This doesn’t look good at all. There is a harsh cut off in the negative residuals (due to government cost always being greater than or equal to 0) and the variance is completely inconsistent across the range of values.
But after applying the logarithmic transformation that we did before, the plot looks relatively homoskedastic now and we have met this assumption. Boom!
So after checking the assumptions of all the features, I decided to apply a log transformation to 3 features and the target variable.
Now, I put this newly transformed data back into the model and after training, it produced an R-squared of 0.92. Fantastic! This is a solid result as the new model can explain 30% more of the variance in the data versus the baseline model. This demonstrates how important it is to transform your data to meet the assumptions of the model you have chosen.
But this was just an OLS model. We can apply the regularization techniques briefly mentioned before which should further strengthen our model. These add an extra term to the cost function, penalizing the model for complexity. This is a good idea because simpler models are typically better than complex ones as they tend to be less susceptible to overfitting.
In other words, complex models tend to fit training data super well but perform poorly on unseen data. I switched over to the scikit-learn library to do this regularization, along with adding in more rigor to the process with a test-train split and cross validation.
I experimented with both ridge and LASSO regression and did hyper parameter tuning of the alpha terms which determine how strong the regularization will be. Surprisingly, both models with optimized alphas performed basically exactly the same as the OLS model with an R-squared of 0.92, with ridge being insignificantly better than LASSO. This indicates that regularization did not significantly help the model.
The LASSO coefficients support this finding as well. LASSO typically zeroes out any redundant features, leaving only a few remaining. In contrast, the best LASSO model only zeroed out 1 out of the 38 features. This is a surprising result, indicating that most features contribute to the predictability of the model and thus stronger regularization would only hurt the model’s performance.
Speaking of coefficients, we can determine the importance of each feature by looking at the sign and magnitude of the coefficients. This allows us to provide valuable business insights to our stakeholders, in this case the Center for Medicare and Medicaid Services.
From the top 10 features, I found it interesting that the fourth most important feature was the number of white patients a provider has. This is alarming as the model actually seems to care about race in some meaningful way.
This could possibly be exposing an underlying fault of the system, indicating that the white population is over-represented and thus, makes up a significantly larger percentage of the Medicare cost in comparison to other races. Other research needs to be done to determine the root cause but it is very possible that lack of access for underserved populations may contribute to this.
This is a powerful result and a example of the value that data science has for society. I was amazed to find that not only could I use this algorithm to improve a stakeholder’s bottom line but to also reveal social disparities. This is one of the main reasons I love data science; it can have immensely powerful effects on our society.
Looking at the top most expensive specialties, surgery clearly stands out. And it makes sense; surgery is incredibly expensive. This means that the government would best spend their efforts reducing surgery costs in order to most significantly impact their bottom line.
In terms of medical conditions, largely preventable chronic diseases take the cake. This finding is a double-edged sword and falls in line with what we already know. Sadly, it means that most of these Medicare patients are suffering from diseases that they didn’t ever have to suffer from if they had just had different lifestyle choices. From both a financial and an ethical perspective, this is absolutely terrible.
But on a more positive note, this means that the government can save incredible amounts of money while also reducing immense amounts of suffering by beginning to focus on preventive lifestyle medicine instead of reactive treatments, such as surgery.
Obviously, we didn’t need data science to tell us to eat better and move more. But this further supports what we all know is necessary for the well being of everyone in society.
And in the process, we built a model to accurately predict how expensive a healthcare provider is for the government. Saving money and saving lives, what more could we ask for?