**Transcript of Cyberseminar**

**HERC Econometrics with Observational Data**

**Cost as the Dependent Variable (Part I)**

**Presented by Paul Barnett, PhD**

**November 20, 2013**

*This is an unedited transcript of this session. As such, it may contain omissions or errors due to sound quality or misinterpretation. For clarification or verification of any points in the transcript, please refer to the audio version posted at www.hsrd.research.va.gov/cyberseminars/catalog-archive.cfm or contact HERC at herc@va.gov.*

Dr. Barnett: I'm going to talk today about econometric evaluation statistical analysis. When the dependent variable in your analysis is cost, and what do we mean by healthcare cost? Well, it could be something as simple as a particular product that is used in producing healthcare like a chest x-ray, or a day of stay, time in the OR, or a dispensed prescription, but it could be products bundled together in a particular visit or stay, or the cost of a particular episode for a treatment for a particular condition, that is a bundle of stays and visits over a period of time. Or it could be the total cost over a period, like, the cost per member over the year or cost per employee. 

And so, I want to just graph. This is actual data about the actual cost per person who used Veteran's Health Administration now a little bit old, but the important message here is just that it's not a normally shaped distribution of the cost. It's a whole bunch of people with costs that are down there about $1,000 or $2,000, and then, off to the right, people who have much higher costs. In fact, the way we had to draw this graph because the tail of the right hand was so extreme as we just put all of the observations for people who had more than $30,000 cost as one point, and that's that blip up to the right. But, had we graphed it, that would have been ... the tail would have extended two or three slides to the right, and with a few observations still not represented. 

So, there are some statistics that we can use to characterize this distribution of cost. This is basically the numbers that go with that picture we just looked at. So the mean was about $6,000, and the median that is the 50th percentile was about $1,700. So the mean was quite a bit to the right of the median. The standard deviation was many times the median, $19,000, so that's more three times the size of the mean is the standard deviation. So that tells you that it's really quite dispersed. 

And then, there's these other two mysterious statistics called skewness and kurtosis, the skewness at 14 and kurtosis at 336. So, what do those mean? Well, skewness is a kind of measure of symmetry, and in the normal distribution skewness is zero. And so, when we have a number like 14, that's a positive skew that just simply means it's a measure of how many observations are in that right tail. Kurtosis is a measure of how the peak is relative to the tail. And, when there's a normal distribution, kurtosis has a value of the 3.0. And as you see we had a value in this VA Cost data of kurtosis was 336, so clearly it's a much higher peak than you would expect for a normal distribution. So, all of these things say that this cost data, that is the annual cost per person in VA is not normally distributed. 

 And really what's going on here, the important thing is the skewness, which is driven by rare but very high-cost events. So some individuals are hospitalized. They incur a lot of costs, there are some individuals who may have some expensive, but rare chronic illnesses where they have a lot of utilization, a lot of pharmacy. And so those are what, those points that are out on the skew to the right a positive skewness. 

Now, here's a poll that I'm hoping to ask just to kind of think a little bit about, if we're comparing two groups, say people who differ by oh, say they were randomized to experimental or control, and we care about the mean, or is it the median that we want to know about? Do we want to know about the mean or the median, that is the 50th percentile? What's the most important way to characterize the difference in the groups, and so we'll let the poll run here for a few minutes. We've got about 90 participants, and now we have about 40 - 50 voting now, 40 voting now, so give ourselves another minute or two. So the median is winning out about 2/3 of the votes so far. Let's see. Well so I think that's give people like 10 seconds more to vote on what they think. But it does seem like there's a kind of consensus that the median is more important.

Well, it's interesting if we think about it. Now, say we have two interventions. So we can probably close the poll now Heidi. Now it's 3⁄4 for the median. Say we have two interventions, and they both have a median cost of say, $1,000. And one of the interventions has a few outliers that drive up the mean. So say that one, Intervention B has a couple of million-dollar cases, and Intervention A, doesn't have those million-dollar cases. Well, the median will be not affected at all, but the mean will be quite a bit more expensive than Intervention B because of those million-dollar cases. Well, the policy maker really does care about those cost outliers. And so, the real answer to this is that we care about the mean than we do about the median?

I think that people probably answered that way because they thought that well, the median's not so influenced by those outliers. But we actually care about the outliers, and we'd like to avoid them if we could. So the answer to this really is about the mean. 

And so, we'll talk a little bit about how to compare groups, statistical methods for comparing groups. So I want to also talk about one other attribute that's typical in cost data. Now, this is another example. We're looking at a sample of people who were, essentially people who used VA care. I think that actually this is mislabeled. This is about: Cost in Fiscal '10 about people who used care in the prior year fiscal '09. I think I've got the labels switched here. But in any case, we have a group of people. Some don't use care in a given year. Care in a give year. They're ... see. You can see there on the left-hand side that there's about 15% of the people who had no cost in fiscal '10 among people who had used VA in the prior year. So this is what we call a truncated distribution, and it's often true in cost data that we have on people who are enrollees in a healthcare system. 

Typically, there's a bunch of people who don't use care in a given year. And those represent a truncation, or kind of a cutting off of the left side of the distribution. So I'd like to ask one other question. Heidi, this is for our whiteboard if you could help me set that up, what hypothesis. What do you want to compare, or are there groups that you want to compare about costs or affects of providers that sort of thing. I was just wondering, so Heidi, maybe you can give the instructions on using the whiteboard. 

Moderator:Yes. I'm actually still working on giving some people access to it, but for those of you who I have already given the rights to, at the top of your screen, you have some drawing tools. You've got a pencil. You have a "t" up there. If you click on the "t," and then go down to the screen, you'll be able to type your responses here. If you want do that, and you don't have that option, just raise your hand, and I'll be able to get to you very quickly here. To raise your hand, it's right near the top of the screen. It's the guy with the hand sticking up there. 

Dr. Barnett:So people want to click on the "t," and then type in a few words about what it is that they're wanting to evaluate?

Moderator:Exactly! 

Dr. Barnett:We're not getting too much yet. 

Moderator:Well, we don't see it until they have completely finished typing. So it may take a minute or two. 

Dr. Barnett:And, hopefully not everybody will choose the same part of the board. 

Moderator:Yeah. That's the tough thing with this whiteboard is we used to be able to see as people were typing, and we don't see that any longer. 

Dr. Barnett:So I'm hoping some will hit. So they hit return, and then it dropped.

Moderator:And then, we'll be able to see it. There we go. So how does the type of treatment influence different types of costs. I think that's interesting how, not the total cost, but the wanting to look at the subtotal cost too. Think of treatment as being an explanatory variable for cost, or a way to characterize groups. Then, what is the effect of program on the cost. Of course, we might be concerned in that case that the programs might have different kinds of patients. 

So, we might want to control for patient severity. So back up to the top, how do you include the cost saved by cost that has been paid for? So of that's run off to the side. How do you improve future outcome by? Well, so I think what they're bidding at here is the idea that some treatments have an off setting effect. You may spend cost on a treatment improvement, and some of that cost may be offset by downstream reductions in cost of treating illness. Is the cost of Treatment A more than the cost of Treatment B? Exactly! We'll see if we've got any others coming up here. We have almost 100 participants. I'm surprised we don't have a few more. 

How does government funding influence cost? Interesting? Well, so we might be there comparing a group that has government funding to a group that doesn't have government funding, so it's a kind of an attribute isn't it of the person or of the treatment plan. Or we could think about comparing Medicare to the private sector. We would have groups defined that way. Well, I think these are all great ideas for studies. And, hopefully this talk will help you. Okay, is your initial slide The Medical Cost versus The Medical Cost. How can medical costs ... so we add it to the medical costs that's sort of a question. But the slides were about the cost of medical care without prescriptions in their medical care with permissions.

Unidentified Female:Paul. I have another question that's come up in the question list, and the question is: What component of the costs are cost drivers? 

Dr. Barnett:Yes, well so in essence that's back to that person who said how does the treatment influence different types of cost? So we're saying, well, if treatment has, say that the outpatient visit costs are the same in the two treatment groups, but the inpatient costs are different, then aren't you saying inpatient costs are cost drivers, so it's really kind of that same question. Isn't it? What type of event is more influential stroke or heart attack, myocardial infarction? 

Unidentified Female:I'm sorry. Did you see the question in the middle in your initial slide: The Medical Cost Versus Medical Cost Plus Meds is lower. How can medical costs alone be lower than medical costs plus meds. 

Dr. Barnett:Well, we're adding the prescription to the medical cost, so that why. I think that the idea that it's medical costs with prescriptions compared to medical costs without prescriptions. So obviously, with prescriptions it's higher than medical cost alone. What's the effect of the icer on two treatments? So the icer is the incremental-cost/effectiveness ratio. We would need to calculate the change in cost divided by the change in outcomes. And so, we'll talk a little bit about, at least, the top half of that ratio, which is the change in cost. Does VA have a willingness to pay thresholds for dollars per quality? No, and that's kind of not exactly this lecture or even this course. 

But we will have that course in I believe in spring that we're teaching the cost effectiveness course. We can talk a little bit about what the willingness to pay in a cost-effectiveness study is, but it's not quite this topic. Well, I think Friday we should move back to the slides. I appreciate that feedback. And so, I thought myself what are the likely responses here, and basically how is cost effected by either a type or quantity of treatment characteristics that the patient or provider, there are some other things we had. Like is it cost more expensive than an MI or a stroke. So those are all kind of hypotheses about essentially differences in groups. And, maybe we need compare groups while controlling for case mix. So let's set this up as a kind of econometric analysis. But first I want to talk about just to review the classic linear model that we call, Ordinary Least Squares, which we assume that the dependent variable, in this setup we're saying, that's Cost is some linear function of independent variables. And so, that's the x. And x could be the treatment group. It could be Stroke versus MI, Treatment A versus Treatment B. It could be Facility One versus Facility Two. 

All of hose things, x could be an indicator that takes a value of one if it's stroke or zero if it's MI, and then we would no the extra cost of stroke relative to MI by the data. But, so that's our classic linear model. And, we are going to estimate our coefficients are sometimes called parameters α or intercept β our slop variable. And The Ordinary Least Squares, minimizes the sum of squared errors. That is the distance between the data points, and that fitted regression. And the linear model with cost is a linear dependent variable. We can interpret β as interpretable in terms of raw dollars. So if x has units and β is a value of 10 then for each unit change in x there's a $10 increase in cost. And, if x is the indicator variable like is this Site A or not Site A, then Site A has $10 extra cost. Or if it's Treatment Group A versus Not-Treatment Group A, that is Treatment Group B, then has a β that represents the extra cost of that treatment that the indicator variable represents. 

Now, I want to define something here, which is going to be useful. And, just to make sure everybody understands what I'm doing, which is this "e" or expectations operator is something that is applied to a random variable. So the example I'm giving here of w is the random variable. And so, the expected value of w is each value that w can possibly take, the probability that that occurs. So say for example, w was the number that shows up on the side of a dice. And so, w could take a value of one, two, three, four, five, or six, and there's a 1/6 probability of each. So, I multiply each of those numbers by 1/6. If you add that up, it's 21/6 or the expected value is 3.5. 

That makes pretty intrinsic sense. It's somewhere between three and four is what you're going to get if you've rolled fair dice. So the probabilities are between zero and one sum up to one. Just like 1/6, probability for each side adds up to one. So, you're sure to get one side showing. So the expectations operator is basically the weighted average of the value that the random variable takes where those weights are by the probability that the variable has that particular value. So this expectations operator can be used to kind of express some of the assumptions that we make when we use ordinary least squares. So we have that error term Ɛ well on average, the expected value over all of the possible values that Ɛ could take is 0. 

We assume that the errors are independent. That is if we multiply and Ɛ times any other Ɛ, on average, they're going to end up being probability of weighted average be 0. All the errors are going to have on average the same variance. They will be normally distributed, and they won't be correlated with our explanatory variable. So these are the assumptions that we make in that we make in that classical linear model. So if we use the classic linear model to measure costs, which of these assumptions are likely to be violated, and really it turns out that all of these that I've listed here in some way or shape or another are depending on the particular case but are likely to be violated. We'll go through this, and what do we do about each of these problems when we're doing a cost data. Today we'll just mainly focus on the whole issue about the assumption of normality and how we can cope with that, or one way to cope with that. So the other thing I think I alluded to is, is that if we have a dichotomous explanatory variable, like x is representing an indicator of group membership. We set it up like this say that x takes a value of 1 if this person's in the experimental group or 0 if they're in the control group. Then, our β represents the effective x. So, if we think about the predicted value of y, conditional on x that is the mean for the control group, that is simply α, the intercept term. Ɛ, remember the expected value of that is 0 over all the observations. 

And, if it's in the treatment group, then we think that the estimated cost we get it from the regression is α + β. So this Ŷ term is a prediction based on our regression, and the β represents the extra cost of being in the experimental group and if β is significantly different from 0 in our regression, then we're saying that the experimental group has a significantly higher mean cost, or lower cost if β < 0 than the control group. And just as kind of a little aside, this way that I've just described it is really the same thing as an Analysis of Variance. It's a regression with one dichotomous variable, and Analysis of Variance relies on those same ordinary least squares regressions. That's a little bit of an aside, but interesting to note. Now we can include a case-mixed variable. 

And so, this might be useful, and some of those people were talking about, well, I'd like to know whether one site has a different cost than another site, well, we want to know if the sites ... we want to control for the fact that perhaps there are differences in the severity of illness of the patients who are treated at those sites. We could have a variable like z or we could have multiple different z variables to capture the severity of illness of the patients at those sties, and try to control for that and say that once we've controlled for that is β​​1 still different, and that is is one site different from the other? In fact we can have lots of different sites and indicators for each of them, and use one site as the reference group. 

And so, we're often wanting to predict what is the mean cost for some variable of the case-mix variable, and I've put ... the over-bar here is to say that just the mean value of the case-mix variable. So we could say that here's the cost of the incurred at the site where x is 0. That is the comparison site. At the mean value for the case mix variable. And, we could do the same thing for the experimental group, or for the site where the indicator variable takes a value of 1. So, we're predicting cost Ŷ and it's some function of the linear function of the parameters α + β + β1 + β2 x that mean value of the case mix variable. That's the cost of the ... I said experimental group here, but it could be that site that is represented by x. 

So, let's talk a little bit about relying on this ordinary least squares. The assumptions are about the error terms, and the residuals or the estimated errors, often have a similar distribution to the dependent variable. So this is where we get an inkling that ordinary least squares may not be the best model. So, Will Manning talks about this that in small and moderate-sized samples, a single case can have a tremendous influence on an estimate. And, that's because there's no thing ... so that case is way out there on the right-hand side of the distribution. There's no value skewed to the left to balance. That's all truncated by $0. There's not such thing as negative cost. And so, he gives an example that in the Rand health insurance that classic experiment, there was observation in a particular health plan that accounted for 17% of the cost of the entire plan. So, that was obviously very skewed and very influential outlier. And, I'd like to just graph this. What does this mean? 

So here we have cost on our y axis, and some measure x here of a quantity that say it is well, we could think about some clinical laboratory test where a higher number is worse. And, this one observation where x takes a value of 300 and this person has a $300 cost. So all the rest of the points there are clustered around the left-hand side, and this is kind of a skewed one. Now let's take that same point, and say, well what if that person who has x = 300, actually only had $100 cost. Well, you can see the line gets drawn quite differently. So, I'm going to flip back and forth between these just so you can see. All the rest of the dots are unchanged. All we've changed is one outlier, and you can see how the line has changed. And the value for our β, which is 0.88 become 0.42, here, so it's cut in half just by this one influential outlier. Our α, which was 0.72 now becomes 22.9. So this one outlier really just drives the whole evaluation. And, so this is the problem with relying on ordinary least squares when we have skewness in the data. So it's one outlier just can drive everything. So, one approach is to take a transformation of the cost. Take the natural log, ln. 

So, that's the log of that napierian log e of cost. And so, if it's a $10 cost if we take it's log it's 2.3. And $100,00 the log of that cost is 11.5. So this has a very nice property which is when we do our regression. And now, instead of having cost on our y axis, we have log of cost. These are exactly the same data as we looked at before. Now our cost regression changes. This one outlier has much less influence. See the line is only slightly perturbed by the fact that this outlier changes because we've basically kind of tamped down it's influence by taking the log of cost. You can see here that in the case when the outlier was high cost. The x is about 0.11, and here it's 0.08, so there β is not very influenced. The intercept term goes from 2.87 to 2.99. So much less influential on what happens with a regression. 

And so this has a number of desirable properties, but let's just look at how our distribution ... so remember this was our distribution of cost that we did before of VHA cost of people who used VHA in fiscal '10. And now, we're going to take this log of this. And now, we have much more like the bell shaped curve that we saw before. And in fact, let's look at our measures of the distribution. Comparing the cost and the log cost. 

So you can see before the median and the mean. The mean was much higher than the median. Now, it's only very slightly higher, so that's evidence that the skewness is not so bad. Our standard deviation now, is some fraction of the mean. Our skewness is very close to 0, which says that there's not big right tail. In fact, it's actually saying that slightly the tail to the left, and our kurtosis statistic rather than being that crazy 300 value is now a 1. Actually, so for normal kurtosis would be three. So we're actually saying the peak is slightly flatter than what would be in a normal distribution, but these numbers just confirm what's in the picture. That this is much more normal than that, so our log cost is much better behaved. And so, we can run that regression with log cost, and when we do that, the parameters no longer represent the effect of x on cost. So before we said that β was for each unit change in x was the number of dollars cost. 

Well now, β is really the percentage change in cost for each unit increase in x. And the last slide in the handout is for those who had the Calculus. You can work out that in terms of derivatives, and how that works out. So we are back to the kind of case that we were on to find the fitted value of y. That is: what is the predicted cost conditional on x being 1. So, that's saying our site that is represented by the indicator variable while controlling for the co-variants, what is the value of fitted cost? What is the predicted cost for the mean patient at this site, or we could also do this for the site were x = 0 and drop the β1. What is the predicted value? 

So we often want to simulate out of our data. And so, one idea that you'd have in order to find that fitted value. Well, since we're not really interested in log cost, the policy maker wants to know about cost. So could we take the antilog of each side, so we know the antilog of log y is going to be y, but is the antilog of the other stuff, that is we take eαβx + β2z is that going to be the predicted value of the cost. And we can work through this using the expectations operator. 

And, it turns out that this work only if we can assume that ... so here in this setup we've used μ as our error term, is if the exponentiated value of the error term has an expected value of 1. So is that reasonable to assume. Well so μ we think that μ has an expected value of 0. So can't we assume that if we exponentiate that μ, so e0 is 1, right, so doesn't the exponentiated value of this work out. And does this last line, is that assumption right, and if you work through this with your expected operation and I offer an example here of why this is not true. If we take, say there's just two observation μ1 and μ2, well so the average value of those is 0, so we met our condition to use ordinary least squares. But, the exponentiate the expected value of the exponentiated error term is not equal to 1. And, this has to do with the non-linearities of it. 

And, in summary, we just have to say the expected value of the antilog of the residuals is not equal to antilog of the expected value of the residuals. Those are in words just saying what's at the bottom. The bottom line here is is that we can't do that simple retransformation, taking the antilog to find the expected cost. 

Becky:Paul. 

Dr. Barnett:Yes, Becky. 

Becky:I'm sorry. There's a request here just to speak a little more clearly especially at the end of your sentences. You're dropping off a little bit. 

Dr. Barnett:Okay. All righty. We'll try to articulate a little better. So, one way to eliminate this retransformation is by using this smearing estimator. This slide explains how this smearing estimator works or how it's derived. So we're calculating the expected value of y that is cost. And we want to do that by taking the exponentiated value of our regression including the error term. If you work through this, you see that you need to multiply by that term on the right-hand side, 1/m time the sum of the exponentiated residuals, those actual error terms as they're observed, which is basically saying it's the mean of the antilog of the residuals. And, I'll summarize that. There is the smearing estimator. Actually this turns out to be a pretty simple thing to derive, the mean antilog of the residuals because most statistical programs allow you to save the residuals from your regression. 

So you simple run your regression, save the residuals, find their antilog, and find the mean of that value those antilog residuals. The smearing estimator is often > 1. And so then, we take that smearing estimator and we multiply it times that eαβx + βz or whatever that simulation is. And that gives us our predicted cost. And, this is worked out in The Seminal Papers and this JSET Paper by Duan. There are some assumptions made when you use the smearing estimator, and in the next lecture in two weeks, we'll talk about what you do if those assumptions can't be made, but basically using that multiplying by the mean of the antilog of the residuals is what you need to do to make sense of that retransformed value. 

So the log model is useful when your data is skewed, but if what you need is a simulation of what the costs or what the effect of your intervention or your site or your disease state is, at the mean value. You've got to correct for this retransformation bias. There are other ways beside the smearing estimator. That's a pretty robust one. I want to raise one other problem that you can incur. So we talking about the 0 values in cost data. We gave that example that there was like 15% of the people in VHA who used care in one year, don't use it in the next year. So you've got this problem that the distribution that's truncated where there are some people with no-cost values. 

So how can we estimate a log model when we've got 0 values? You can't take the log of 0. That is undefined. It's just not possible to do. So what people have proposed is: well, let's just substitute a small positive number that's close to 0. We can say well, that person may not have had any cost, but we can take the log of $.01 or the log of $.10 or the log of $1, and then have a value for them. Why don't we just use that and substitute that in the log regression, and then we've got a way of dealing both with the 0 values and with skewed values. So there are some reasons why this doesn't work out very well. And, I want to just graphically show what happens if we take the 0 value. So here's an example of regression where we've got some 0 values, and instead of, we can't take the log of 0, so we say that well, let's substitute $1 for all those people who had no cost, and estimate our regression. And, let's do it again, but this time we're going to substitute $.10. Well, you can see that the regression is quite influence by our choice of $.10 or $1. And, that's because they're way out here on the left, the low end of the distribution, and the choice of $.10 or $1 is pretty influential. 

So our β when we had $1 as our value of these 0 values, is 0.12. Now it's 0.15. So it has some influence, and our intercept is really influenced a lot. Here's it's -0.4. When we substitute $.10, it's 2.47, actually I think that's -2.47. I think there's a little error there. So that choice of the small positive value can be very influential especially if you have a lot of 0 cost records. So that's a limitation of the log model. So intuitively, why is this true? Well, the log model assumes that the parameters are linear in logs, and that means that a change from $.01 to $.10 has the same affect on the regression as a change of some observation for $1,000 to $10,000. So the assumption that we make about these 0-cost records is pretty important, pretty influential. And, it's possible to do this if we've just got a few 0-cost records. But, the results could be sensitive to that choice of small positive values, and that's something you want to check before you do this method. 

That said, there are better methods, and we'll talk about these in two weeks at the next lecture. Transformations that allow us to have 0's like square root. There are two-part models, and there are other types of regressions that allow you to get around this problem. Now the question is back to our original classic model where we don't transform the cost. And, yes. Ordinary least squares can be used if the cost data aren't very skewed, or there aren't too many 0 observations. So, there are some examples of some clinical trials where everybody is pretty similar. There are not high-cost rare events that allow you to use ordinary least squares. It has the advantage that the parameter is much easier to explain. That β is just interpretable in dollars cost. And we can include those for observations with 0 costs. My only caution is that, if you do this, you really want to try out some of the alternatives, not only log costs, but some of the more sophisticated methods we'll talk about in two weeks. Just the reviewer is going to want to know, gee. Did you think about these other things, and I've got to say that I can't think of very many recently published papers that are estimating costs with ordinary least squares. There are just so many better ways to do it now. 

So just to review what we've said, some of the basic points or the take home are that cost data are not normally distributed. They can be skewed by high-cost outliers and truncated by some observations where no costs are incurred that are 0's. The ordinary least squares assumes that the error term, and really the dependent variable are really normally distributed, and that is not consistent with what we see in these cost data. And so, applying ordinary least squares when the data aren't normally distributed is going result in parameter that are biased. We're going to misstate our β that we want to know. And this is especially true in very small or moderate-sized samples. Now, if we take the log of our cost variable, the cost is more normally distributed, and we can use ordinary least squares. But it's not the only method of dealing with this problem of skewness. 

When we do these two different methods the parameters have different meanings. If we're using a linear-dependent variable, ordinary least squares of untransformed costs, β is the absolute effect of x on y. That is it's denominated in dollars of cost. If we do ordinary least squares with the dependent variable transformed, that is if we take the log of cost then β becomes the proportionate or percent change in y for each unit change in x. And we can use either of these methods to predict the value of cost case, with a particular value of x or y. 

So, if x is an indicator of a variable, we can predict y conditional on x taking a value of 1 at the mean value of the co-variants. We have these fitted values. So we do this with untransformed cost. If we have log dependent variable, we just can't simple take the antilog of this equation. We have to correct for retransformation bias, and we learned about this smearing estimator, which is the mean antilog of the residuals. So the smearing estimator is really ... something like that smearing estimator needs to be used to adjust that antilog of the fitted value. So typically makes it a bigger number because the smearing estimator somewhat above one. If we have cost data with 0 values, the log of 0 is not defined, and so this idea of substituting 0's with small positive values is possible, but can have its own problems with bias, and there are better methods, which we will get to week after next. 

And so, next session on December 4th, we're going to talk about two-part models where one part is the predicting whether the person incurred any cost, and then how much cost conditional on incurring cost. There are regressions with link functions. These are general linear models that don't have that well, allow you to do lots of different things, and have a lot of advantages over what we talked about the simple models we talked about today. There are non-parametric statistical tests. Don't require any assumptions about distribution normal or not. And then, we'll also review methods to determine kind of criteria for choosing among these alternatives. 

There is a basic overview, a good paper by Paul Adeere, and co-authors Mark Hernberg and others at analyzing healthcare utilization and cost with those methodologies. If you have VA email address, and would like to get a copy of this, please send us an email, and we'll forward it along to you. There's our email address. And, I think that's a good reading for this topic. I would also just want to provide you with these two sites Nathan Duan's paper about the smearing estimator and Will Mannings work about using ln cost and the whole problem of this retransformation bias. I'm so in debt to those econometricians in figuring this all out. So Patsy, are there any questions that I've overlooked here, or any other people who have question that they want to ask at this point. 

Moderator:There's nothing pending. 

Dr. Barnett:Well, I'm glad to entertain any questions or talk to you again with the Part Two after. 

Moderator:If anyone does have any questions, please use The Q & A Screen in Adobe Connect to submit those questions into us located in the lower-right-hand corner of your screen. We've got about 13 minutes to go here, so a great opportunity to get your questions answered. 

Dr. Barnett:Well, hopefully somebody's typing one up right now, and we'll see it in the 

Moderator:Is the smearing estimator necessary for the parameter estimates predicted values or both?Dr. Barnett:So if you take the ln of cost, and then you use it in an ordinary least-square regression, you get parameter estimates, and you can interpret those as percent change in cost. You don't need a smearing estimator to estimate those parameters or to interpret them. The issue is now if I want to take all of those parameters, and I want to say, so I report in my paper, log costs. People are going to go what are log costs. I don't know what that means. I want to know what the costs are in dollars. And so, I want to predict costs given certain characteristics. Say I want to predict the cost of the person in Group A and predict the cost of people in Group B, then I've got to find that fitted value of cost. That is what is Ŷ, and that's where the smearing estimator comes in. So you could take α + βx + β2z, and if you take the antilog of that, you're going to underestimate the cost. 

And so, the smearing estimator, you multiply times that antilog, and that is the predicted cost. And so that's the importance of the smearing estimator is really if you want to predict sample, and maybe you want to predict out of your sample. You want to use those parameters that you estimated in your study, and apply it to a different group with different case-mix co-variates. Well, you could predict those, but you still will need to use the smearing estimator to do that.

Moderator:And another question is there a SES proct for the smearing estimator?Dr. Barnett:Well, no. Not per se, but I mean. If you use the SAS proct reg, there is a way to save your residuals. So you run your regression, and there is an out statement that allows you to save different things from your regression. You can save both the fitted value, and you can save the regression, excuse me, the residual value. So, if you use that out statement, you could save both of those things, and then, you can manipulate them so what you'll want to do is to, once you have your residuals, exponentiated them, so use you can use a subsequent data step to manipulate those values. You're going to take your residuals, exponentiated them with an exponentiation operator, and then find the mean of that, and that mean is the smearing estimator. Now, take that multiply it times all of the observations, the exponentiated fitted values, and it's a little bit hard to talk about in the. I guess. I could use the whiteboard couldn't I?

Moderator:Do you want a whiteboard Paul.

Dr. Barnett:Well, there's a danger of trying too fast like this. It's too bad that we don't have a worked example for you. In fact, when we had the old system -- I think that one reason why we're a little bit short is we're not demonstrating this, but essentially you can do all of these things fairly simply, and fast. There's not canned procedure, but you just need to be able to save out of your proct reg, your residuals and your fitted values, and then you can manipulate them in the way that I've suggested. Stata will do this pretty easily too, and the idea that you can predict the values of people in your study, you can predict the value of a particular observation that is somebody who has all the characteristics of a named patient enrolled in your study, or you could predict something that's out of your sample someone from a different site. What would there costs be if these parameters were true. What's the next question? I hope that's a good answer. 

Moderator:The next question is can we walk through the expectation slide, the dice example. I stepped away, and I want to have that slide firm in my hand. I've asked for the slide number, but you're probably more familiar than I am. 

Dr. Barnett:Um, so let's see. 

Moderator:It's pretty early on I believe. 

Dr. Barnett:Yes, there was a way to navigate among them. 

Moderator:Oh, cool!

Dr. Barnett:Yes. 

Moderator:Slide 16 the asker says. 

Dr. Barnett:Yes. There how do I get rid of this thing now. 

Moderator:Can you?

Dr. Barnett:There. Thank you.

Moderator:So that's 14, 15, 16. 

Dr. Barnett:So, this is the expectations operator, I think is what the question is about. So the idea is is that if we have a random variable like here we've just given a letter w, then what is the expected value? It's essentially the mean of that variable. But so it's the probability. So , we could think about that as well I actually said that we could think of the probability of a dice. The probability of wi​ coming up one, of w taking a value of 1 is 1/6. And so, we and the probability of it taking a value of 2 is also 1/6, etc. And so, if we sum that over all of the possible values that a dice could take that will end up to be 3.5. But you could take the probability of say w is someone's height, and so you say, well. What percentage of the people are 5' 6"? What percentage of the people are 5' 7", etc. You would end up with what's the mean height. 

And then, the same question with a residual and a regression. A lot of them are near 0, some of them are above 0, and some are below 0, but on average, the expected value of an ordinary-least-squares regression of the error term, which is the next slide, is 0. And so, that's what this expectations operator is about. And we just really used this to derive this whole question that because of the non-linearities involved, we need to be careful about how we're handling the exponentiated values of the residuals that they don't turn out to be, there's a non-linear relationship there. And so, we need to be careful, and that's the basis of this smearing estimator is that we can't assume that that exponentiated value of the residuals have an expected value of one. That's the later slide. I think the take-home message is you don't really need to understand how we showed this, but just remember that if you need to predict costs when you're using a log-cost model, you can't just take the antilog of the value. That is the predicted value. So, you need to apply some sort of correction retransformation bias. I see another one from Nick.

Moderator:He is I think the person who asked the SAS procedure. 

Dr. Barnett: So, what about other procedures, other than proc. Reg? Well so, we'll talk about the GLM procedure next time. And, one of the nifty things about doing a general linear model with a link function is that you don't have to do this retransformation. You don't need a smearing estimator. There's not retransformation bias. So, for proc. GLM it doesn't work the same way, but anything that's based on an ordinary least-squares model where we have transformation of the dependent variable, we have to think through if there's a possibility of retransformation bias. So there are some SAS procedures that use for example panel data. I think that's the only other thing I can think of other that proc. Reg. I think we need to help you out if you have, give us an email if you've got a particular way you want to set this up and now how to do it, but also maybe you're just anticipating next lecture. 

Moderator:Are there any other questions?Dr. Barnett:Well, these are all great questions. Sounds like people are engaged in doing some interesting projects. It's getting pretty close to the top of the hour. Maybe one more. 

Moderator:For the smearing estimator, why do we multiple by 1/m x the sum of exponentiated error, but not certain weighted average?

Dr. Barnett:Well, this really is a weighted average. So what's the weight? So each exponentiated error term gets the weight, which is 1/m, so if there's 1,000 observations, each residual goes into that, it's 1/1,000 of the weight. So its frequency is ... and so, if there are two identical residuals they get twice the weight. And, that particular value gets twice the weight. But basically, each error terms gets weighted equally. They're an exponentiated value of the error term. I should say it gets weighted equally in finding the error term. And since it's a non-linear relationship, it turns out that the ones that are positive have higher values and non-linearities involved mean that that smearing estimator is slightly above 1. But, we do use a weighted average, and the weight is just 1/m.

Moderator:Can we use other weights?

Dr. Barnett:Well, it's the frequency. I don't think you want to. I think this is how it works. This is what makes it make your predicted value controls for this retransformation bias. There are other retransformation ways to correct for the retransformation bias other than the smearing estimator. That paper by Manning goes into them. Each of the methods has certain assumptions involved. The smearing estimator probably makes the least assumptions, but there is an important assumption that's made with the smearing estimator, and that is that all the error, the third assumption listed on this slide, errors have identical variance, which we call homoscedasticity and so next time we'll talk about what do you do when it's not appropriate to assume homoscedasticity? When there is heteroscedasticity that errors don't have identical variance. That's back to that general linear model. Really this is sort of step one is to think about using log transformation. And then, there's better ways to do log transformations in the context of general linear model. So, I'll just use that as the teaser. They always do that with the evening news. At 11:00 will tell you. So on December 4th, we'll tell you the second part of the story.

Moderator:Thank you. Anything else? Any other questions?

Dr. Barnett:So, Heidi you have some final, I see it’s now noon our time. You have some a little bit of housekeeping to do, I think. 

Moderator:I have a little bit of housekeeping, yes. Paul, thank you very much for presenting today. We are looking forward to your next and the final session in this series on December 4th. We will be sending registration information out to everyone on that next week. If you're not already registered, keep an eye out for that. As I close the meeting today, you will have a feedback form pop up on your screen. If you could take a few moments to fill that out, we really do read through all of your feedback, and we use it to incorporate into our current and upcoming sessions. Thank you everyone for joining us for today's: HSRND Cyber Seminar, and we hope to see you at a future session. Thank you. 

1 of 14