HERC Econometrics with Observational Data- 15 -Department of Veterans Affairs

**Department of Veterans Affairs**

**HERC Econometrics with Observational Data**

**Cyberseminar 0402502012**

**Cost as the Dependent Variable (Part 1)**

**Paul G. Barnett**

Paul G. Barnett:  Welcome, sorry that little glitch there.  We are going to talk about statistical analysis econometrics when we use cost as the dependent variable.  And this is the first of two talks on this topic.  

And so what do we mean by healthcare costs?  And there are obvious different ways of thinking about it.  It could be something very specific like a specific intermediate product, a chest x-ray or perhaps a day of stay in the hospital, other ideas here, other time in the operating room probably or dispense of prescription, but usually we are thinking more about a bundle of products that bundles up to the level of outpatient visit, or hospital stay or maybe an entire treatment episode, the visits and stays over an entire time period or something like the annual cost, all the care received in a year.  

So there is lots of different ways of slicing, and dicing and thinking about healthcare costs.  And this is some VA data from fiscal year ‘10.  This is the data from the HERC average cost data set. We simply took the five percent random sample of all of the users of VHA, Veterans Health Administration during fiscal ’10 and plotted them on this probability distribution.  And you can see this is kind of a classic case for cost which is that lots of people down on the left-hand side, but then there is this skewness of people who have high annual cost and in fact some people will of there on the extreme far extreme right of this distribution will cost about $30,000 that they account for a lot of the health care costs.  

And so here are some descriptive statistics of that that go along with that graph.  And the mean cost is about $6,000 and the median is down about $1,700.  And so that is an interesting property where the mean is greater than the median.  

It has to due with skewness and here are statistics from or skewness in kurtosis.  I apologize if you are hearing a leaf blower in the background.  They seem to know exactly the right time to show up outside my window so a little noisy here.   

So skewness is also called the third moment, but it talks about the degree of symmetry in that distribution. And if this were the normal bell-shaped curve the skewness statistic would be zero, but in fact it’s a number like and I think it was three, right, or thirteen, excuse me, fourteen.  

So 14 was quite a bit more than zero and that shows that there is more observations in the right tail.  There is also this other measure of departure from normality, kurtosis, and the peakness, how high is the peak and how thick is the tail is really what kurtosis represents.  

And for the normal distribution we would expect to see a number like three.  And so in this case I will just go back, so here we have 336 so obviously our tails are a lot thicker and our peak is a lot shorter.  That would be in a normal distribution.  

So those are one way just to understand that cost is not normal.  Cost has got problems with skewness.  And those are costs by these rare but extremely high costs event.  Some individuals get hospitalized and have very high costs or there are some individuals who have chronic illnesses that are quite expensive.  And we think about positive skewness being skewed to the right.  

Now here’s a kind of question for you to think about.  As an analyst we could, we have the mean which reflects the skewness or the median which is just the middle value.  It’s the middle cost of the value. 

So when we are doing an analysis of health care costs do we care about the mean or the median?  And so let’s see how we can set this up as a question.  I guess we could ask people to raise their hand if they think that it is the median that is more important.  And so maybe you can help people, direct people on that so they go to their—

Unidentified Speaker:  There is a tab on the right.  

Paul G. Barnett:  So, Heidi, you can help me out a little bit with that? 

Moderator:  Sure.  There is a tab.  Can you guys hear me?  There is a tab on the right-hand side of your screen, the one with the orange arrow at the top.  And at the bottom of that tab there is a hand with a little arrow on it.  Just click on that to raise your hand.  

Paul G. Barnett:  And I, so and I guess I would be able to see in the participants. 

Moderator:  In the attendee list you can see we are definitely getting some raised hands in there.  

Paul G. Barnett:  And so and then is it possible for people to comment?  Or well maybe let’s not do that right now.  So the idea of median I think people appeal to that because they think well that is a good statistic because it is not driven by these rare events that it’s somehow robust, but actually in terms of cost we are care about those skewed events.  

And so the mean is probably the better answer because the one person who might consume five percent of the budget in the hospital today that one person is important.  And so we want to know about that, their contribution to the thickness of the tail.  So really the mean is thing that we want to estimate and not the—and....

And then I want to raise with next graph another concept that’s so we first with skewness, and not it’s the issue of about this left truncation.  So we selected a cohort of all the people who used VA in one fiscal year, fiscal ’09, oh excuse me in fiscal ’10, and then we looked at their cost in the prior year, fiscal ‘09.  

And you will see that there is a bunch of people who didn’t use VA at all.  So over on the left-hand side about, oh I don’t know, sixteen, seventeen percent people who incurred no cost, so they have zero.  And so this is a zero cost.  And this is another feature of cost data is this truncated distribution or observations with zero cost.  

So if you are looking at a say a health plan and you want to know what’s the mean cost there are some people don’t utilize in a given year and that raises another problem in terms of just the statistical problem property.  So we have kind of it is not only not the bell-shaped curve, but we truncated or kind of cut off that last end of the distribution and had all those people piled up on zero.  So these are like I say enrollees who don’t use care represent of truncation of the distribution.

So what kind of hypotheses involve in cost that people want to test?  And I have written down some ideas here that we have gathered for people in past seminars that people would like to know how treatment, some sort of intervention affects healthcare cost or maybe it’s the characteristics of the patient or provider, could be randomization to some intervention, so lots of different things that we would like to study to evaluate cost.  

Now before we go into the cost model, and so I would like to talk a little bit about the ordinarily least squares.  And I believe that this was covered in the last lecture, this call to classic linear model

And we assume that the dependent variable can be expressed as a linear function of some independent variables, not just put X here.  It is that independent variable.  

So ordinarily least squares Y are a dependent variable is a function of some fixture of alpha that we estimate plus beta, another term that we estimate, a parameter of the estimate.  And some values and X’s could be continuous variable, could be indicator variable, or called dumb variables if that plus this E, an error term of some unexplained part.  

So we estimate those coefficients, those parameters, alpha and beta, by minimizing the sum of squared errors.  And that is that this is between our plot of data points and that regression line. 

So this is the classic linear model and we could use cost as the dependent variable as Y.  Now I want to—and then the beta in that case would be interpretable in some sort of raw dollar.  So for each unit change in X beta represents how much Y or dollar, how much extra cost in dollars there are.  

So for example if beta were in a value of ten for each unit increase in X is an extra $10 in cost.  So this is how we interpret that coefficient if we use the linear model.  And Y is our dependent variables in dollars to cost.  

Now I wanted to do a little definition of something what is called the expectations operator.  And it’s the expectation of random variables, just a weighted average the of all the values that variable could take times the probability each value occurs.  

So in rough terms it is the mean and the probabilities of course are between zero and one.  And so for each value of this W of this random variable that we are talking about, this expectation operator is hopeful I get by that probability that that particular value occurs, so that is—the expectations operator we use in the next slide just to talk about these assumptions that we make when we use ordinarily least squares.  

So remember that error term that was that unexplained part of the classic linear model.  Our assumption is the expected value of that is zero.  So overall the probabilities and all the chances as such mean is going to be zero or all the observations.  

It is also that the errors in from one observation are not correlated with the errors of another observation, that that the observations are independent of each other.  So that’s another assumption we make.  The assumption we make is that the errors have identical variance that is it doesn’t matter what value X takes that error turn is going to be the same, or for that matter what, yeah, what value so the errors have an identical variance, finally the assumptions are errors are normally distributed and that they are not correlated with independent variables.  

So those are the five assumptions of ordinarily squares.  And it turns out that the cost data not all of the assumptions work out so well.  And so I just relisted these and the rhetorical question here, which of these are likely to be violated by cost data.  

And there are certain situations in some studies where every one of these is going be violated.  And so we have to have methods to deal with that.  And I think really that’s what a lot of this course is about is there are these five assumptions we make to use ordinarily squares or classic linear model and how do we cope when those assumptions can’t be, we really can’t make those assumptions.  

Now we’ll give an example of how to use the model. Say we have just a single explanatory variable which is a group membership variable. So X in this case is an indicator variable takes the value of one if the people in the experimental group or if they’re in the control group. 

And so if we estimated ordinarily squares with this we have a model of this sort.  And so and if we were wanting to predict the value of Y and someone wasn’t a member of the group it would simply be alpha.  

And this is a bit of notational the fitted value of Y, that’s the Y is conditional on the value of X being zero is equal to alpha.  That is the intercept term in our regression.  And if the person is in the group then the fitted value of Y conditional on X being one that is of course being the experimental group.  It’s just alpha plus beta, so beta is the extra cost of being in the group.  

And really analysis of variance is the same regression, is the same as this regression with one that kind of this independent variable.  It also relays on ordinarily squares assumptions.  So that is just another way of thinking about an over, ANOVA, and ordinarily squares regression are pretty much the same thing.  

Now we can also add some covariant, say we thought that these groups we are comparing had some case mix differences, well so we would include a case mix variable to try to control for the underlying differences in the groups.  So I have notated that here as Z.  And then we could project what is the cost conditional and not being in the group assignment X=zero.  We would have to add back in the mean value of Z.  We would be estimating the mean cost at the mean value or mean case fixed value, and the same way for being part of the experimental group.  

We would add alpha plus beta plus this term for the beta two times the what’s the mean value of the case mix variable.  So those are sort of ways to project the cost or synthesize the cost while controlling for case mix.  

So let me before I go about these assumptions I would like to see if I can’t show you an example of this.  So this, Heidi, can people see this screen now, the SaaS interactive session?

Moderator:  Yes.  We can see it, Paul. 

Paul G. Barnett:  Great.  So I have taken this data.  If you look down on the bottom you will see the SaaS program editor and I have already loaded my program in here.  And so this, the data down here are from this study that is published, and we’d be happy to send you a copy of the paper.  

This doesn’t really follow exactly the analysis we did in the paper, but the data are the same.  So I just going to—the first thing I am just going to do here is highlight this code, and then run it.  And so this is just going to take a means of the three variables.  

So what this study was about was there is a variable you see here called concordance.  And concordance was our group assignment.  Now this was not a randomized study.  These were ninety-one patients who were selected from studies from methadone treatment programs that were less concordant with treatment guidelines than this 164 patients who were recruited from sites that were more in concordant of treatment guides.  

And if we look at the means we can see that the less concordant, this is the total cost in the follow up period it is about $16,000, so that was the mean in these ninety-one patients versus about 23,000 more than $23,000 for the people at the more concordant sites, but the more concordant sites had more people with HIV AIDs and more people with schizophrenia.   

About five percent had HIV AIDS at these concordant sites and six percent had six schizophrenia.  So this indicator variable mean value of the indicator variable is takes a value of one or zero, so 0.06 it’s saying six percent.  

So now we have these group comparisons, so now I am going to go down here and I will just do—and let’s do just do univariate, whoops, here we go.  My computer bonked there.  Here we go.  So I am going to do a prop univariate which will look at some of those statistics that we were previously looking at of mean, median skewness and kurtosis, and to scroll back up here to find that, too far.  That’s what our prior one.  

So our next page is here we go.  So now this is our all cost, total cost during this study for all 255 pages, so we follow it.  So now we can if we look down here we see that the mean among all the pages is 21,000 and the median quite a bit less.  

So there is skewness in the data and our skewness statistic is 2.3, so that saying there is positive skewness.  So the assumptions of ordinarily least squares there is some departure from the normality.  Kurtosis is seven.  

So ordinarily least squares may not be the best thing to do, but that said we will go forward and let me go down here with the, scroll down here and run this regression.  So now this regression that I am going to highlight here, I’ll try to highlight, there we go, is saying we are going to run a model where that all cost is our dependent variable.  And our explanatory or independent variables are this, are they concordant in a program that is concordant with treatment guidelines? 

Does the person have an indicator for HIV AIDS?  Do they have the indicator for schizophrenia?   

So let’s run this model, submit this.  And so here is our results of our regression.  So what is this saying?  It is saying so if they are in the concordance treatment group, that is the highly concordant sights, they have $6,600 extra total health care costs, if they have HIV AIDs about $16,000 extra health care costs, if they have schizophrenia about $14,000 extra health care costs.  

Leave the follow-up period in terms of year.  And all of these parameters are statistically significant.  Now if we get the—there is our squared.  It’s not a whole lot of the explanatory powers not super great, but there are better ways we have root mean square, things that we will talk about next time about how assessing goodness of model fit, so maybe not the best fitting model, but our parameters are certainly the expected sign and we do see some extra costs associated with guideline concordance.  

Now go back to our—I’ll talk a little bit more about the ordinarily least squared assumptions. And so we-re—the assumptions that we make in doing ordinarily least squares are about the error turn and the residuals, that is the estimated errors that we have that if we took our fitted value of the line and compared it to the actual value of life those are the residuals.  And they often have a similar distribution to the dependent variable.  

So we—so that, sorry about that, flipping back and forth, made me lose my place.  Here we go.  So why worry about this problem departure of more normality?

And so I’m quoting from Will Manning here just to indicate that in small and moderate size samples a single case can have a big influence and I’ll illustrate this with a graphic in just a moment.  So and the reason for this is there is no value skewed to the left to balance this influence.  And Manning observed that in the Rand Health Insurance experiment, that classic study that was done some years ago that he worked on that there was one observation that accounted for almost twenty percent of the cost at one of the health plans in the experiment, so and whether that lightning bolt happened or not would have really affected, really could have affected the results of the analysis. 

And so graphically we can look at this.  So here is our linear model.  We are trying to fit this line to these points.  Our X in this case is a continuous variable and our Y say it was cost on the left-hand side there.  And so here we fitted this equation and this is just totally made of data, but just to illustrate this point.  

So here we have the alpha is 0.7 and the beta is about 0.9, right, so and notice that that where that one point is slightly above the line.  Now in this next plot we are just going to move that one point and see what happens to our fitted line.  

So we move that one point.  Now our alpha is twenty-three and our beta is 0.4.  All the rest of the points are identical.   So let me flip back.  

So that just that one point changing, that one influential outline really drives a linear model and really changes our parameter in essence, so here we go from less than one to twenty-three for our alpha.  And our beta goes from 0.8 to 0.42.  It’s cut in half.  So that is the concern about the departure from normality.  

So one way to cope with this is to transform costs to try to make it a more normal variable nd the normally distributed variable.  And one way to do this is to take the natural log of costs.  And so if we take the natural log of cost is $10 and the log cost is 2.3, et cetera.  A $100,000 costs it would take its log.  It becomes, takes a value of a 11.5.  

So what happens here? Well let’s look at our back to our VA, back to our synthetic data where we are taking the log of costs.  And we estimate a model where log, it’s log Y on the access sum and the X log wise, our dependent variable and same otherwise everything is the same.  

And so now this one—and so these are the same data so before our alpha is about three.  Our data is in this case 0.1 and we have to make a different interpretation of what beta means by doing what alpha means, but these are how we fit our line to this log data.  And now beta changes from 2.87 to 2.99, not so influence, excuse me, alphas changes from 2.87 to 2.99, and our beta for 0.011 to 0.008.  So it changes, but not quite to the same degree this outlier is much less influence, influential.  So these are identical data, just the difference is we transformed taking along Y.  

Moderator 2:  Hey Paul. 

Paul G. Barnett:  So let’s look at our—yes?  

Moderator 2:  Okay.  So we had a question and I was just trying to figure out the best time to interject.  And I thought this might be a good time.  

Paul G. Barnett:  Okay. 

Moderator 2:  Isn’t larger skewness really indicating that there are distinctly different routes, perhaps having different underlying causes and thus different means, those with typical costs and those with very high costs?  Wouldn’t very different underlying causes explain these two groups and thus require separate means, separate analyses? 

Paul G. Barnett:  Well what do you think, John?    

Moderator 2:  So my take on it is so first the question is if we had full information about the causes that is probably true, but we often are doing an analysis where we are testing a hypothesis, and we have people in each group.  And so it’s not always clear why they have, why they are train wreck if you will, why they are using so much care, but if you go to—

Paul G. Barnett:  Hit by a bus on the street, could be something that has nothing to do with our intervention.  

Moderator 2:  Right.  Or it could be something that’s completely related to the intervention.  So often are faced with these mixtures of people and we just don’t know for a good reason to be able to explain it fully why somebody is using so much resources.  

Paul G. Barnett:  But that’s so to the degree that we can’t explain it it goes—they have a big residual.  And then we have to make assumptions about the residual.  And they’re kind of getting at the heart of this issue is what are the reasonable assumptions to make about this residual.  

Moderator 2:  Thank you.

Paul G. Barnett:  But I think the impossible dream would be to explain all of the, all of costs.  That’s unfortunately that we never have enough data to really do more than a pretty good job of that, and yes.  

So we’re back on the just to talk a little bit about we were just talking about log costs.  So I’ve forward to this next slide.  So this slide you have seen before, which was the annual person costs of people in VA.  And remember this is skewed distribution.  It has got that right tail and all that. 

Now if we take the log of this now it looks much more like a normal curve, much more well behaved data.  So this is the [disha] and has left and right tails and peakedness. And if we look at our data we can see these are our descriptive statistics and our log costs on the right that the median, the median are almost the same.  And our skewness is actually a little bit, I guess that highlight that on this, negative, slightly negative.  So there is no longer a skewness effect.  We have almost overcorrected.  

Kurtosis is actually we’re so remember normal is three.  So we’re actually a little bit more peak and less tails than with the expected in the normal distribution.  

So but we certainly got in the way with from a lot of dispute, so this is quite desirable to have done this log transformation.  And so we could do this again, do run an regression just like we did before using our ordinarily least squares model, and but the coefficients aren’t interpretable any more in raw dollars.  

And the beta represents the relative change in cost for each unit change of X, that is if the beta is 0.1 then for every unit change in X there is a ten percent increase in cost.  So it is relative rather than an absolute value.  

So we have a different interpretation on that data coefficient.  And if—so before I think I go to this I think I will return to our old SaaS example here, run our SaaS session.  And so let’s, let me scroll down here, the next part of the code.  

And now you see here is a data set step where I have got, let me highlight this whole section.  I have to thank [Adam Chou] from our center to teach me all the keys strokes in the SaaS center active.  I don’t usually work with this, so sorry if I am a little fumble fingers here.  

So now I have highlighted this next section.  So the first part is a data step and you see that I am reading in my data in one from the disk the utmost, and then I am just making a transformation taking the log, and my all cost variable and I am labeling it as good documentation.  

And then I will run up PROC MEANS on this.  So let’s submit this statement.  And so here is our new value, and so now log costs, remember these were what was it $15,000 to $20,000 were the range.  Now we have these log costs as these values of 9.3, 9.6, still a higher value in the concordance group.  

So that is what happens after we transformed it.  And then let’s go down here and take our univariate.  So let’s see what our, what happens to our descriptive statistics for the log of cost.  

So I have to go back up here a little bit.  So here’s our descriptive statistics.  So our mean and median which they are almost the same, in fact the mean is a little less than the median now.  So we are kind of under skewed as it were.  

Our skewness is just like in the prior example with all the VA health care costs we actually now have negative skewness.  And our kurtosis is quite a bit less than three.  So this in essence we are a little bit more peaked and thinner tails and under skewed compared to the other data, but we are certainly not nearly in so much trouble in terms of the departure from normality.  So the assumptions to run ordinarily squares are not as bad in this case.  

So let me go down here to the just to run this regression.  So now this regression is a model of the log of cost and same explanatory variables.  So now run, submit, and here we are.  

So again we have a new set of parameters, all significant, not as quite as significant as before.  Well I guess the HIV AIDS parameter is just fails significant to P 0.05.  Over here you’ll see that it’s 0.053 so it’s not any longer significant at the P 0.5 level.   

So what are these parameters saying?  They are saying that if people are in the concordance group their costs were, their total health care costs during the study were about twenty-seven percent higher than the people who had HIV AIDS at fifty-six percent higher costs, so that we can’t rule out that they were not statically a significant difference, even with schizophrenia at fifty-four percent higher costs.  

So those are the interpretations of these parameters.  So that’s model might be a little bit more tenable because we—it is more consistent with running log costs more consistent with the assumptions needed to run ordinarily squares.  

So is that zone, just trying to avoid that problem, well I guess we will just see the slide here for the time being.  So the question is what is the mean cost controlling for case mix?  So if we wanted to actually say what is the fitted value of Y conditional and being group membership, how do we find that Y half?  

So remember we were working in logs.  What now is the value for Y hat?  And is it just simply a matter of taking that model, finding the fitted value for log Y and then exponentiating it, taking E to that alpha plus beta X plus beta 2Z, or the mean value of C?  Is that what Y hat is?  

And it turns out that seems reasonable, but it is a problem.  It doesn’t actually work that way.  And this has to do with so we can work through the math of this how the expectation operator is applied to this.  And we can only—this only applies that the [dispitted] value of Y would work this way only if we consume that E that each of U, which is U is our error term in this example the error term from the long model, each of the U equals one.  Well our expected value of U is zero so E to zero that should equal one, right? 

Well it turns out we have to think about the probability of each and that the fact that it’s not, here we go, since the expected value of U or the error term is zero.  Shouldn’t the expected value of E to the U = 1 and that so here is this question.  

Does—are these things equivalent?  And if you work through this and here is the kind of a counter example, say with that we have U = 1 and U2 = a negative one, so that their sum of the mean value there is clearly zero, but once you work through it you see the expectations.  Operator is it works out that is not actually equal to one.  

And so it has to do with the non linearity of this retransformation.  And so there is one way, several ways to eliminate this.  This is basically works out probably the most robust method, which is to use what is called the smearing estimator.  And basically if you work down to the last line you realize what this is is you take so this E to the U, which is that term in the lower right-hand corner.  U is your residual.  And so if you exponentiate your residuals and if you take each residual from your observation, so in SaaS if you’re using your out statement after you have run your regression, save your residuals and you could exponentiate them, take EXP function of the residual.  

And then you find the mean value of that.  And you will have this one number, some number that is slightly greater than one, could be quite a bit greater than one but two, three, but usually it’s between one and two.  And that number, that smearing estimator you multiply to correct for retransformation by us so that fitted value that we thought we could use we need to multiply by the smearing estimator.  And so here is the smearing estimator.  It’s just the mean of the exponentiated residuals.  So yes, John?

Moderator 2:  So there is a question that has come in.  When you do this working with on a log scale and then retransform, does that mean that you’re working with medians or with means?

Paul G. Barnett:  So you’re really still working with the mean.  

Moderator 2:  That’s correct, yes right. 

Paul G. Barnett:  Right.  So....

Moderator 2:  It’s just that it’s a log mean.  Part of the confusion I think means when people use the log in your dependent variable is that people are often talk about elasticities and percentages if you are reading just the log, the variables.  

Paul G. Barnett:  But it is the parameters, the parameters in the log regression.  

Moderator 2:  That is correct.  

Paul G. Barnett:  Correct.  

Moderator 2:  That doesn’t mean you are estimating a median.  

Paul G. Barnett:  Right.  So if you are thinking about—we will talk next time a little bit about what if we just cared about in essence the ranking of the costs and which we really get?  What if we just cared about the median and not so much about how skewed they are?  

There are non parametric methods for comparing variables without that are not sensitive to the outliers and for better or for worse.  And so that would be a good question.  We want to know how to evaluate differences in the median.  Let’s think about that when we talk about non parametrics in the next lecture on May 9th.  

So the take-home message here is that if you want to do a simulation or find a fitted value after running your log cost regression you have to realize that there is retransformation bias and then a way to deal with a retransformation bias is to multiply the fitted value by the smearing estimator which is the mean exponentiated residual, and so meaning of the antilog of the residuals.   When I said exponentiated antilog is another way of talking about that.  And most statistical programs allow you to save the residual so I mentioned in SaaS it’s the out statement after regression and you find and take the residuals, find their analog and then the mean over all the observation.  

And it’s usually a value that is greater than one.  And the original cite here is from Duan and this paper.  And actually this was worked out in the context of the [Iran] health insurance study. 

Now the smearing estimator makes some assumption about the variance that the variance of the errors does not depend on the value of the X.  And there are other methods that can be used when this assumption can’t be made and we will talk about it next time.  

So basically the message is that log models are useful when data are skewed, but the fitted values have to be corrected for this retransformation bias.  And we are trying to simulate what the costs would be.  

Say we wanted to know what is the mean cost of each group controlling for the case mix, the average value of the case mix?  Now the problem, one limitation with the cost data is when you get back to the—remember we said sometimes our cost data sets had zero value.   So if we are looking at all the people who used VA and in fiscal year ‘10 and we want to know how much cost they incurred in the prior year, actually I think it’s the other way around, fiscal ‘09 and obviously occurred in the subsequent year we found that a lot of people had zero value.  It’s eighteen percent.

And so how can we find the log and why would Y equal zero?  And log of zero is undefined as you can’t take the log of zero.  And so what people have done as a practice is they substitute a small positive number for in place of that zero, and something like a penny, or dime or a dollar.

And so how well does this work?  And this is can be problematic.  So here we are going to estimate a regression where we have taken all our zero observations and we substituted $1.00 for them.  

And so you see our regression is, our log regression here, and again this is just made up data, but to illustrate this point.  And now if we and we have alphas negative 0.4, beta is 0.12.  

Now say we substitute a dime, now really drives our results.  Our alpha has gone from 0.4 to 2.47, our beta from 0.12 to 0.15, not quite as badly affected, but the alpha definitely.  

So these—it is the same sort of issue that there is this kind of that line is trying to accommodate these dime and the dollar, right?  It makes a big difference.  In other words the log model is assuming that the parameters are linear in logs and so it’s assuming that that change from a penny to a dime or a dime to a dollar is the same as the change from $1,000 to $10,000.  And we’re trying to fit that.  

So it is possible to do this substitution of a small positive number in the place of the zero, but it’s only advisable if you have got just a few serial cost records involved, and if you find out that your results are not sensitive to which small positive you value you use penny, dime, dollar, but they are really much better ways.  One is that you can use a transformation that allows the zeros like a zero like a square root of cost.  

You could use a two-part model which is especially useful if you want to know about the probability of incurring any cost, and then the cost conditional on incurring some cost and will in other types of regressions, and these will be covered next time.  We’ll talk exactly about better ways than this when especially with zero cost data. 

So then just to kind of a recapitulate it, can we ever use ordinarily squares with just the raw costs without taking any log transformation?  And that can be okay if we don’t have much skew or zero observations or when there’s a large number of observations, so we don’t violate those assumptions or ordinarily squares.  

And one advantage of using untransformed costs or just the raw cost as our dependent variable is on Y is that the parameters are easy to explain in terms of dollars.  And but I would say it is not advisable unless reviewers are going to watch it to know that you looked at other things and it’s okay that you really found that this worked out okay and that is your results are, that your analysis is not too naïve.  

Just to recapitulate what I hope the take home message here is is that the big point is that cost is a continuous variable, but it is not usually normal and distributed.  It’s got skewness from high cost outliers and it has often has left truncation from individuals who have zero values.  

And the ordinarily squares, the classical linear model actually makes five assumptions, but about the error term, but important ones is that it assumes that it is normally distributed and so is that cost data are normal than the ordinarily squares don’t work out so well, it doesn’t work out so well with untransformed cost.

So applying OLS to data that aren’t normal can result in bias.  And the outliers are just too influential. Remember that one outlier really where, what it’s value is really can drive the value of the—and Manning saying that that one observation in the Rand health experiment accounted for twenty percent of the cost of one of the health plans, so you don’t want to get too carried away with our inferences based on just one or two outliers. 

So log transformation really helped out, but it’s not the only way of dealing with skewed cost.  The next time we’ll report on general linear models, so in other ways of dealing with skewness.  

And then the meaning of the parameter obviously depends on the model. With a linear dependent variable, that is it’s the raw costs it’s our Y and our regression data is represents that effect of X absolute number of units of & change for a unit change in X, whereas the log gets a proportionate change. What’s the percentage change in Y for a unit change in X?

And then if we’re going to find a fitted value we can find this linear combination of the parameters that we’ve estimated.  So for instance if we want to know in this example what I’ve written out here is is what’s the cost conditional of being in the group, having X =’s one, and the mean value of case mix.  And so you can take this, figure this based on your parameters and mean value of Z, say here’s what’s the mean cost controlling for case mix in an untransformed case. 

But if we’re going to use the log dependent variable we just can’t simply take the antilog of that linear combination.  We have to make some correction for retransformation bias.  And one way to do this is the smearing estimator, the mean antilog of residuals.  

And then just to mention that since cost data have observations with zero values and it’s a truncated distribution the log model has its limits because log zero is not defined.  And it’s sometimes possible to substitute small positive values for zero and that’s take the log and those small positive values, but this can also result in bias.  And there are better matters, better methods like general linear models that we will cover the next time, May 7th.

So what we’re going to cover next time is two-part models sometimes called hurdle models, which is where the hurdle is if you incur any costs.  So there’s one part is what’s the probability of occurring any cost and then a conditional estimate of how much cost additional and having been a health care user. 

Then we’ll look at regressions with link functions, the general linear models.  We’ll look at non-parametric statistical tests, so that’s sort of like where you really are just considered about the relative ranks, advantage of non-parametrics is there’s you don’t have to make any assumptions about the distribution. They are conservatives. 

And then finally the question for next session is how to determine which method is best.  And I have some suggestions for reading if you can get this paper.  If you can’t let us know.  We’ll have it send you.  There’s the HERC e-mail address.  This is a good, pretty basic overview, [Paul Andeer’s] paper that’s and review public health on methods of analyzing cost.

I have also put up here some references on the smearing estimator and alternatives to the smearing estimator, [Dave Duvant’s] original paper back in 1983, and then so much more in advanced stuff that Will Manning has come up since then from the Journal of Health Economics, other approaches, and then just for those who are especially keen why is the parameter in the log [ma] model represent proportional change, any who remember the calculus from their school days. 

Do you have any more questions, John?  

Moderator 2:  Yes.  We have one that relates to software.  So you have shown SaaS examples here.  Do you have a position on SaaS versus—the question related to the SPSS and relation, and maybe speak more generally about software. 

Paul G. Barnett:  So right.  I have not used SPSS so I don’t really know it’s advantages or disadvantages.  In the next session we’ll be talking about these models that are based on link functions, general linear models.  And I since I gave this lecture last time I have looked into—somebody got back to me and said I’m not sure this works with SaaS.  And they’re right.  

SaaS does not play well with general linear models and actually kicks out any observations with zeros unfortunately.  And so right now I wouldn’t advise anyone who has a model that has a lot of zero values in it who wants to use one of the more robust techniques, which is this general model, to use SaaS, and that really the only—well the best software to do it right now is Stata.  And so that would be the direction I would go, if you’re a user of [Vinci] both SaaS and Stata are at the [Vinci] that our research analytic service that’s located in Austin and administered by the people in Salt Lake City. 

So those tools are available to inside, don’t know if [Zinti] has SPPS.  And I really can’t advise people on that.  I know there are other packages out there, but I think that between SaaS and Stata they have the big market share.  And I would find that for just about every study you end up using Stata, and even though we do all of our data manipulation with SaaS

Moderator 2:  Thanks.  That’s the only question that has come up so far.  We are just approaching the hour so I just wanted to thank you.  And if other people are wanting to ask a question they can type it in.  We can answer it for you.   

Paul G. Barnett:  Yeah.  Please feel free if something occurs to you later to write to our HERC e-mail address or to me.  

Moderator 2:  We have two more.  

Paul G. Barnett:  We’ll do the best we can.  

Moderator 2:  Can you give an example of a case mix variable?

Paul G. Barnett:  Well so in our—in the SaaS example that I showed those were the AIDS.  It was an indicator of AIDS, so that’s a very case mixed indicator of schizophrenia.  We could have had some sort of continuous variable like say body mass index.  We could have another indicator of a recurrent smoker, whatever might be associated with higher health care costs, all of those things. 

And then there are actually some case mixed indices, and I’m blanking on the name right now, the....

Moderator 2:  [Charleson Howser]. 

Paul G. Barnett:  [Charleson], yeah. 

Moderator 2:  Yeah. 

Paul G. Barnett:  DO co-morbidity indexes which are basically a combination of a whole bunch of different diagnoses and are a continuous variable.  

Moderator 2:  Or even as you point out costs in the last year is often your prior year of study is often a good case that can be it as well.  

Paul G. Barnett:  And so in this example I have here I have costs in the prior year.  And it’s very highly explanatory in this particular data set, but I was worried that I was over controlling because the concordant sites there is some chance that they actually—

Moderator 2:  Sure. 

Paul G. Barnett:  —that that was part of what the nature of being at the concordant sites, so it’s an interesting conceptual problem.  

Moderator 2:  So we have a few more that have come in.  If you have extreme outliers other than transformation will you consider dropping some data points?

Paul G. Barnett:  Some people do that.  And I’m not sure it’s appropriate because I’m always worried that you’re throwing out important information.  How did you select those the data points?

There is a kind of a boot strapping analysis.  I know you might be more qualified to answer this than me, John.  What are people...?

Moderator 2:  Different, yes, there’s different techniques, one called [windsorizing] people talk about where you bring in the what you think of as the outliers.  And I should point out that there is a data generating process here for which generating our data. 

And there’s the question about when you see a high cost is it truly a high cost that is valid to the data generating process, or is it an error?  And so some of the first things that we would recommend if you are working with VA data would be to try to determine whether it’s a data error because if you see like a $5 million inpatient stay and $4,999,000 is in pharmacy and it’s a three-day stay you might say that looks like it’s an error.  And that would be a valid time to figure out what to do with that error, but if you believe it’s a valid case I would argue that you should keep it.

Paul G. Barnett:  Yeah.  I think that’s—those are both good points.  And there is this [windsorizing], but I worry that you’re actually throwing out important information. 

Moderator 2:  When we talk about large sample sizes how big is enough?

Paul G. Barnett:  Well so that’s all about goes to your sample size calculation and what is the effect that you hope that is clinically significant.  And what’s the variance in the data set?  So that’s a whole another lecture about much data you have to gather.  

Moderator 2:  And when we get into certain models in G11 it’s also particularly finicky about over fitting in small samples.  And so I’ve heard under [bahn] that Will Manning and others talk about easily needing hundreds of thousands of cases.  

Paul G. Barnett:  Right, well especially with our using their flexible form I imagine. 

Moderator 2:  Exactly. 

Paul G. Barnett:  Right, the flexible form because they are estimating lots of additional parameters that we don’t really actually care about, but are help to fit the data.  And the problem is that maybe they are driven too much by that specific data.  I guess one way to get around that would be to split the sample, see how it works.  

Moderator 2:  And then some people think about doing post hoc power analyses, but I generally think about if you believe that you’re not over fitting the data, and this relates to a question that just came in, is to use your confidence intervals as your sort of your power.  So the comments often don’t have this hard threshold of 0.05 as your significant level and often talk about your level of confidence and your confidence regions, as well as your mean of fact. 

Paul G. Barnett:  So if it’s not an experiment I actually think most of us are working with in health services and we are working with data that are not necessarily the result of a experiment, but randomization.  And I think the bigger problem is the bias from being an observational study, not random assignment of treatments.  I’m worried more about that than I am about the statistical significance, but....

Moderator 2:  Great.  And then can you...?

Paul G. Barnett:  So I think it’s....

Moderator 2:  Can you use Excel to achieve the same results as SaaS?

Paul G. Barnett:  I guess so.  I mean yes depending on how much, how valuable your time is.  

Moderator 2:  So there are things that you can do easily in Excel.  And I think linear models are one of them, but to do anything that you might want to do statistical tests or put in robust standard errors, it’s very hard to do that in Excel I think.  And if you’re constrained by budget consider [R] which is a user-generated system that people like a lot that’s free.  

Paul G. Barnett:  Does that run on a PC?

Moderator 2:  It does.  It also runs on Apple, yeah.  

Paul G. Barnett:  Yeah.  That’s—that would be much better if you’re budget constrained, but I know SaaS is not accessible to some people, but I think having a real statistics package is you’re just reinventing the wheel.  It’s a lot of effort.  And then you really don’t know whether you’ve made some sort of mistake along the way too in terms of a program.  You would have to kind of validate your programming efforts.  So I think that would be the wrong approach. 

I know some classes have people program their own estimators that way, but I think only very basic stuff like standard deviation overall. 

Moderator 2:  We had a question about flexible form, but I believe we’ll address that mostly in the next seminar.  

Paul G. Barnett:  Yes.  So there’s other ways to estimate as well, so I think that’s a just to defer it’s a good idea.  

Moderator 2:  Because we’re actually over the hour here.  So I want to let people—I know a number of people had to run, but I just wanted to make sure that people had the opportunity.  I think that’s all the questions we had and I think that’s all the time we have.  So I just to thank you all for a great presentation on cost data.  And I will....

Paul G. Barnett:  Well thank you, John, for organizing the seminar and answering the questions that I couldn’t.  

Moderator 2:  Tag team.  Thanks everyone and thank you Heidi.  

Paul G. Barnett:  Thanks Heidi. 

Moderator:  You’re welcome.  Thank you to both of you.  We’ll talk with you next time.  

**[End of Recording]**