**Transcript of Cyberseminar**

**HERC Econometrics with Observational Data**

**Econometrics Course: Introduction & Identification**

**Presenter: Todd Wagner PhD**

**October 2, 2013**

**Todd Wagner:**I just wanted to welcome everybody.  This is Todd Wagner.  I don't normally sound like this.  When you have two kids in elementary school, this is the colds you get and, of course, the laryngitis is hitting me today.  I appreciate people's patience.  I have about 50 slides to get through.  This is the first class on the econometrics with observational data.  I'm really thrilled about the class this year.  

One of the things that we're trying to do is revise some of the talks, and we also have some new staff here who are going to be giving some of the new presentations.  For example, you'll hear from Christine Chee later in the course.  

I also appreciate people's patience with this new platform.  It's something we're getting used to, so hopefully you can see the next slide here.  It's the goals of the course.  Really, we're trying to provide people with an opportunity to understand more about econometrics and understand how to think about observational data and conduct careful analyses with existing VA data.  There's this discussion about big data, especially here in Silicon Valley, but across the nation.  In healthcare we have been working with big data all the time and we have some amazing data in VA.  

We're going to describe some econometric tools.  Some of my colleagues have called this sort of a buffet, if you will, of different econometric tools, and we'll talk about their strengths and limitations and we'll try to use examples to reinforce learning.  When possible, we'll use examples from VA.  There are, I know, people who are from academic affiliates and non-VA so I apologize if I use lingo and jargon that's specific to VA.  

As we go through the class today, here's the goals for the class.  I also want to make sure people understand there's a question and answer thing.  Patsi Sinnott who is one of the other health economists here at HERC, is monitoring that.  As we go through I might have to take a break to drink some water or cough, and that'll open up a time for people to ask questions.  Feel free, and, Patsi, if you want to jump in, and I'll let you parse if it's a question that's a clarification versus you want to hold it to the end.  

Today's class, the goals are to understand causation with observational data.  That's really what people like myself and Health Services Research are trying to do when they're using observational data.  We're going to describe an equation and the elements of an equation and give you an example of equation, and then sort of work that through the assumptions that are built into that equation when we think about modeling with data and the classic linear model.  

What we talk about here is going to set you up for each of the other classes, so we're going to talk about the five main assumptions in a classic linear model.  Now, each of those assumptions, when they're violated, it's going to invoke some other questions and other methods, and other people will talk about those later on in the course.  

Terminology.  It can be incredibly confusing.  I find that I do a lot of multidisciplinary work.  It's one of the reasons I love VA, but it also is a major challenge.  There's questions of confounding, endogeneity, interaction, moderation, mediation, multivariable, multivariate, right or wrong.  It's one of the things that we're going to have to, as we present our slides, get over.  If you have questions about our terminology, please ask a question about it.  Don't be shy.  It's not that we're trying to snow you.  What we're trying to do is make sure that it's understandable for everybody.  

There's also a great paper a distinguished colleague, Matt Maciejewski and colleagues from—he's from Durham VA—has written this great paper.  It's actually an update to an earlier paper where they try to talk about how these different concepts fit together.  I highly recommend it.  

I have a poll question for you, speaking of—perfect.  I'm curious if you have any graduate level statistical training, and it's fine if you don't, but what your background is, because that's going to contribute to this interesting discussion we have.  Take your time to fill out the course.  This is a new poll for us.  What I'm seeing is real-time responses.  I feel a little bit like watching a reality TV show in action here where the people are watching and voting.  

We're seeing about half of the people are trained in biostatistics, about 10 percent are trained in econometrics, 22 percent, so about a quarter, are trained in math or statistics, and about 10 percent are trained in psychology or psychometrics, and then another 10 percent have no graduate level statistical training.  I was actually expecting a very different split for the course, so it's a very interesting split for me.  I was expecting a lot more in psychology because I know that a lot of our attendees in the past have been in the field of psychology.  I'm going to end the poll, so thank you all for voting.  Perfect.  

Again, if we use jargon or terms that confuse you, please let us know.  I also recognize that many people had statistics many years ago.  That's okay, too, but it sometimes colors the way we think about data and colors the terms that we use.  

In econometrics, one of the things that we're particularly interested in is understanding causation, whether it's the causal relationship for individuals, what drives individual behavior or organizational behavior.  How policy affects people, for example.  In many cases, we understand that the randomized control trial is the gold-standard research design for assessing causality.  

It might go without saying, but when I ask the question what is unique about a randomized trial, it really is this idea that you're experimentally controlling and manipulating an independent variable.  The treatment or the exposure is randomly assigned, so it's not someone's choice.  It's not the participant's desire to say, "Well, I want to get more intensive treatment," or, "I want to be on the experimental drug right now."  If that's randomly assigned and it's done well, provides the best evidence on causality.  

All right.  There we go.  The benefits of randomization, it provides us some inferences on causality.  If you see a large effect it might tell you something about what's driving what in what direction.  It really distinguishes experimental and non-experimental designs, and for people who are trained in experimental designs, moving to a non-experimental world where you're worried about confounding and some other problems, it can be a challenge.  Likewise, the other way is true for people who are trained in observational data.  It's often tough to go and think about how to really control, in a good way, an experimental design.  

I'm not going to be talking much about randomized trials.  Most of what we're going to talk about is what you're using with observational data.  I don't want to say that random assignment, I don't want to confuse it with random selection, because there are many times that you can do random selection in an observational trial, which is different from random assignment.  It's the random assignment that's important for causation.  

Now, there's many limitations of randomized control trials.  The generalizability to real life maybe low, and the key criteria here are often that we use inclusion or exclusion criteria in a trial.  One, perhaps to make it feasible or less expensive, but that might also hinder its ability to generalize to a broader sample.  

You can end up with a Hawthorne effect such that just observing patients, patients know that they're being observed.  It changes their behaviors.  We hear this more and more often, especially in the tight budget years, the randomized trials are expensive and slow.  If you're working in operations, you might not have the luxury of starting a large trial that's going to take five years and be six or seven by the time you're done to know the answer.  

There can also be questions about that you really can't use randomization to answer because it's unethical, and it might be a clinical question that's very important, but it's unethical to randomize it to people.  You can think about smoking would be the obvious one.  Right?  You can't randomize someone to smoke or not smoke, but you can think about it in treatment settings, too.  There was a very classic study looking at how people sought treatment for heart attacks, and there was a question about does intensive cardiac care improve outcomes.  Well, you might have a hard time randomizing people to intensive cardiac care and non-intensive cardiac care, but you can observe that because hospitals differ in their patterns.  Quasi-experimental design can often fill an important role.  The real challenges here have to do with understanding causation, because that's the chief limitation with observational data.  

Can secondary help us understand causation?  My hope is that if you're here listening to me and my gravelly voice, is that you believe that there is some way that we can do that.  I'm a big coffee drinker, so all these examples are pulled from headlines about coffee.  Coffee is linked to or is not linked to psoriasis.  May make you lazy.  It's a good thing, a bad thing.  If you follow coffee, for example, but I'm just sort of pulling this one out as an example, you can see that just using observational data can create some very disparate answers, but this is true about almost everything we observe in life.  

Observational data.  It's widely available, especially the VA.  It permits quick—and I should put quick in quotes—careful, quick analyses compared to randomized trials at a relatively low cost.  It's not to say that you should be sloppy with your data, or that you can be and get away with it with observational data.  It's just that it's faster than randomized trials.  Because you're often pulling from a broad sample, it may be realistic and generalizable, perhaps more so than a randomized trial.  

Now, the key challenge, of course, is that perhaps you're interested in a key independent variable that may not be randomized.  It's most likely not randomized, and so we think of it as not being exogenous, not being external to the person or the organization, and it being endogenous.  Now, throughout the course we're going to talk a lot about what does exogeneity and endogeneity mean, in part because there are some specific tools for addressing this issue, but this is also one of the perhaps chief limitations that observational data encounter when they're working in doing analyses.  

Let me define endogeneity.  A variable is said to be endogenous when it is correlated with the error term.  Now, if you're not familiar with equations or the classic linear model—this is assumption 4—this is going to be confusing to you, but think of it this way.  If there exists a loop of causality between the independent and the dependent variables, this can lead to endogeneity.  I'm going to give you an example here of what this means.  It can come from measurement error.  It can come with autoregression or autocorrelated errors.  It can come from simultaneity.  It can come from omitted variables or even sample selection.  

Typically, we think of sample selection when we're working in health care.  People choose certain things and more interested in it.  For example, you might be interested in saying what's the link between smoking and health outcomes.  Now, I would hope that everybody knows that we believe there to be a causal link between smoking and health outcomes, although there's never been a randomized trial on it.  We know that that link exists because there's biological plausibility.  We can do all the bench science.  There's also been enough observational data that we believe that this to be the case, but we're still stuck with this question about when you put smoking on the right-hand side of your equation and you're looking at it, people choose to smoke for various reasons and you can't control for all those reasons.  That makes it endogenous.  It makes it very tricky to understand the true link between smoking and health outcomes.  

Here's an example of an endogeneity question.  Perhaps you're in pulmonology and you're interested in the question of PET screening and the question of does greater use of PET screening—it's positronic emission tomography—decrease lung cancer mortality?  The idea is that you're going to give people these scans.  You can better stage the person, better understand what treatment they should get, and perhaps that would decrease mortality.  You might observe that some facilities do a lot of PET screening, while others do very little PET screening.  One question might be, well, let's just compare patients and their outcomes across where they go to care.  

You're going to put PET screening or use of PET at the facility level on the right-hand side of your equation, but you should be very careful because right away you should see that PET screening intensity is endogenous.  Patients choose their facilities.  More than that, clinicians choose where they want to work.  When clinicians think about taking a job, they might think about the resources that are available to them.  They might choose to work there because other people believe in the same types of technology.  They might, when they're working there, also realize that there's this new really cool technology called a PET screener and that they have a better ability to lobby the facility to buy and invest in that technology.  Those things might all be correlated with things like the quality of the physician or the quality of the surgeon or radiology.  That can bias our analysis if we're not very careful.  

That key issue of endogeneity is going to come up time and time again.  I'm sorry, I heard something.  I wasn't sure—

**Moderator:**No, I was trying not to sneeze into the microphone.

**Todd Wagner:**Sneeze?  No worries.  I wish I didn't sound like this.  I can hear myself in this, my gravelly voice, and I apologize to people.  

**Moderator:**Todd, it's nowhere near as bad as you think it is.  It's not a problem.

**Todd Wagner:**All right.  I'll just think of it as being extra cool.  

*[Laughter]*

**Todd Wagner:**Econometrics versus statistics.  Some people, when I say econometrics, my daily job is econometrics.  I get these blank stares.  Then if I say I do statistics, applied statistics, people seem to understand it.  There are cultural norms about statistics and econometrics.  In economics, for example, one of the cultural norms is if your independent variable seems like it potentially is endogenous, it probably is.  You'll find in biostatistics and other health services that that's not always the case.  People say, well, it could be but that's a limitation of the study and we move on.  In economics they would say that's a fatal flaw.  

The other probably key issue that I want to bring out is that there's an underlying data generating model.  In statistics, it's a general field of applied math that's interested in these relationships.  In econometrics, we're often assuming a rational actor is concerned with some sort of behavior, and that actor can be an organization or a person, things interested, for example, in profit maximization, quantity maximization—let's assume that they're a nonprofit—or time minimization that you, as an employee, are trying to get—thank you.  

All right.  Let's move on.  Terms.  I just want to get back to make sure people are understanding the terms here as we get to an equation.  A univariate statistic is a statistical expression of one variable.  Uni, one.  Bivariate is an expression of two variables.  You're interested in a relationship between two variables, hence the bi.  Multivariate is the expression of more than one variable.  It can be more than one dependent variable or more than one—a dependent variable with more than one independent variable, hence the multi.  I am not a believer in multivariable.  I think that's a concocted term, but that's my personal opinion.  

Here we go.  Hopefully people recognize that this is an equation of a line.  Note the note there underneath.  Y is our dependent variable.  The i means it's a subscript denoting what the unit of analysis is.  In this case we could think of it as being as a person, so each i is a person.  The beta 0 is the intercept.  Perhaps we're interested in, at this point, just the relationship between Y and X, and the X is our covariate.  If this were a perfect line, you wouldn't have an error term because it's not measured—a line is not measured with error, but in statistics, we often think about there is a level of error involved in measurement and behavior, and hence we have an error term here.  

There are many terms, many descriptive terms for X.  Some people call it a covariate.  Some people call it a right-hand side variable, a predictor variable, independent variable.  I tend to call it a right-hand side variable to covariate because when I feel like if I make the case that it's independent, that's making a mathematical claim that it's an exogenous variable or truly independent, which I don't necessarily want to make.  

Like I said, i is an index.  If we're analyzing people, then this would refer to the people.  If we're analyzing medical centers, this could refer to a medical center.  You can also have other indices.  You can have like a T, which is a common indice 18:48 for time, so you might be able to do that as well.  

I've just extended this as a multivariate equation now.  We have two covariates.  We have a dependent variable, we have the intercept, and we have two covariates and an error term.  Obviously this can get bigger.  It just means that my writing on the slide has to get a little bit smaller.  

Different notation, and you'll often see this in journals.  Sometimes we might want to highlight two independent variables, X and Z, and then you might say, well, there's a whole bunch of other variables that we're controlling for, and so people would use the summations—let's see if I can actually—can I draw on this, Heidi?  Ah, there we go.  I can pull up my—so this thing here, let's see if that works.  That's a common way of looking at a vector of variables that you just want to highlight quickly and say that they're in the equation, but you're not going to talk about each of them, whereas you might want to say this is our key independent variable, and perhaps this is a key other variable that you want to talk about.  

Again, we have our error term.  It's just a shorthand way of making it easier to understand.  Error exists because, for example, some important variables might be omitted.  Perhaps we're just measuring the height of kids in a class, and there's measurement error in our ruler.  You measure the same kid four times and you come up with slightly different estimates, and that can be true with height.  It can be true with hemoglobin A1c.  There's also just this idea of human indeterminacy, that everything is sort of interconnected to everything else and provides a little bit of error.  

What I will say is that understanding the error structure is critical for econometrics when you're working with cost data.  That error structure can be very fundamental to how you think about your statistical model.  Most of the time we're trying to minimize error and make a precise estimate, and handling this error is important.  Error can be additive.  It can be multiplicative.  It can have different structures, different shapes.  It's all very important.  

Here's an example.  This is all made up data, but the idea is real.  Is height associated with income?  Here the dependent variable is income and X is height.  The idea is do people who are taller, is that associated with a higher annual income?  You might have your null hypothesis, that height is not related to income, and if beta 1—so let me get my drawing tools again.  If this is zero, my question for you all would be what is this?  Now, I don't know if people can take a chance to write into the Q&A what they think that's going to be if beta 1 is zero.  

**Moderator:**Todd, can you see the answers?

**Todd Wagner:**I can see the answers.

**Moderator:**Great.

**Todd Wagner:**Y is income and X is height.  If there is no association between height and income, then beta 0 is your intercept, and that is—somebody says the Y intercept, which is correct.  It's actually just your average income.  There are people who are putting that down, mean income.  That's correct.  It's your average income, because there's no relationship here between this line—think of it as just the horizontal line crossing at the average income.  That's a great way of thinking about that if there's no relationship.  Now, of course, if there is a relationship we're going to see a line here with a slope, so let's move on to the next slide.  

Here is the data that I've made up.  It's always easier when you make up the data.  How do we want to describe the data?  Estimators.  You'll often hear this term in econometrics.  Let's think of a good estimator.  If I go back a slide, I could say how do I want to describe that data?  What's a good estimator for that data?  

Now, I think everybody knows what an estimator is, or at least they have—we've already talked about some.  Many common estimators are things like means and medians.  They're ways, statistical ways, of describing the sample that we have.  A mean provides information on the central tendency.  Median also provides information on the central tendency.  It's just a slightly different formation of it.  Those are sort of univariate estimators.  There can be multivariate estimators.  What you're doing is applying a function to the data to describe the data.  

Here's an ordinary least squares.  Here's fitting a line, the best fitting line.  Now, ordinary least squares, what it's trying to do, and hence the name least squares, it's minimizing the distance between each of those points in the line, and I'll show that to you in a second.  This would suggest that there is a relationship between height measured in inches and your annual income.  Now, if you're short this might be rather a bummer to you and you might say, "Well, I don't like that."  This is just an association.  This isn't to say it's causally related at all, but there have been studies that show, particularly among certain people, that age is associated with—or, sorry, height is associated with greater income.  

You might decide that you want to think about other estimators.  Maybe you want to talk about a maximum likelihood, least absolute deviations.  Maybe it's a curve you want to fit to it.  This is an idea that here's a curve, and I'm sorry if you can hear my sore throat lozenge here.  I'm trying not to lose my voice here.  You can come up with other estimators.  Your job as an analyst is to think about what's the best estimator that I can do that's going to provide the best fit that's the most true to the data.  That's often the hardest thing that you're going to have to do.  Then you might have the questions about maybe you don't know and you're going to have to do sensitivity or robustness checks to figure out is it robust to that estimator.  

There are many methods for choosing an estimator.  You can talk about least squares.  You can talk about unbiasedness.  Efficiency, which is the idea of the minimum variance.  There can be asymptotic properties of the estimator.  You can talk about maximum likelihood and goodness of fit.  Throughout the whole course we're going to talk a lot more about choosing the "right," and I put "right" in quotes, estimator, and it's often going to depend on the assumptions that we're willing to make with the data.  

Here is the OLS, and I told you what OLS is trying to do is it's trying to minimize those green arrows.  By default, that line minimizes the green arrows, so it's minimizing the least squares.  Now, you can think about the difference, if you were coming down—let me get my drawing tool here.  Coming down on that arrow is a negative, but because you're squaring it it turns positive, so you can look at all of the different ways of fitting this line, and this is going to come up with the minimum variance.  

What about gender?  Right now you could think about maybe we're missing some things in that model.  Maybe it's age and you make more as you get older, or we hope we do.  As I get older, I hope I make more.  You can think about other variables that are also omitted from this current bivariate relationship.  What about gender?  How could gender affect the relationship between height and income?  Men and women have different heights.  You might think about a gender specific intercept.  You might say, well, let's think about what would it look like if we wanted to measure this for women separately from men.  Maybe it's an interaction effect.  Maybe it's only true for men but it's not true for women.  You're thinking about a change in the shape of the curve.  

Going back to the equation, in this line equation here we have Y is our income, X is our height, and Z is our gender intercept.  Let me show you what this is going to look like when we implement this.  If you were to implement this, this is basically saying that you've got two parallel lines.  They just have a different intercept.  Beta 1 is the slope of the lines.  It's the same for both groups.  You just have that it's now that there's a higher intercept for one group versus the other group.  You would see that the red line fits the, I believe it's the women, and the blue line fits the men, if I'm not mistaken.  

Now, you might say that's a terrible fit.  Maybe we need an interaction.  Now here's the interaction.  We've got the main effects, the main effects being height and gender, but now we want to multiply them.  Maybe we're saying if we don't like this assumption, that the two lines, the men and women, have parallel lines.  Maybe we want to allow them to have different slopes.  By doing this, beta 3 provides a different slope for the men and the women.  Note that the main effects are both still in the models.  You get very different results if you drop out height and/or gender as a main effect.  

Let's walk this over and show you what this looks like.  This looks very different all of a sudden.  The red line, being the women, looks like it's quite flat, that there is not much of a relationship between height and income.  In men, it looks to be quite strong that there is an association between height and income.  

What I was trying to do there, that's the last segue for the equation.  There are different ways of thinking about the equations.  I don't want people to be afraid of equations.  They're often very helpful in thinking about what's in the regression model that people are talking about.  It's also a really good chance to think about what's being omitted here, because you can see right away—and I'll get to this later on—if we omit something that's important it has a big effect on the results.  I don't want people to freak out.  You can get much more complicated equations, but that's the idea behind a lot of these things.  

Now, there are assumptions built into the ordinary least squares, the classic linear model.  There are five assumptions.  I'm hoping this takes the last 25 minutes of today's class to walk you through those assumptions.  Make sure you understand them.  That's going to help you decide what classes you want to come back for.  If you're particularly interested in certain assumptions, you're going to say, okay, I want to hear that more, and you'll know what class to come back to.  Hopefully you'll want to come back to them all.  

The classic linear regression, there's no superestimator in all of the things that we do.  There's no one time that you can say this model is better than everything else.  This is the art of econometrics or the art of statistics.  You, as an analyst, have to think a lot about what you're modeling, how you're modeling it.  There's assumptions embedded in each of the models.  Classic linear models are often used as the starting point for analyses.  Historically that was true because when computing power was much easier, these were the fastest to compute.  Now you could do that on your cell phone if you wanted to, the computers are so powerful.  

The other reason that they're used now is that the results are easy to understand even if they're wrong.  They tend to be quite robust so people like starting with them.  Then when you get into even when you have a dependent variable that's a dichotomous dependent variable, the interactions in those types of models are not easy to understand, and so the classic linear model can sometimes fit quite well even when the dependent variable is dichotomous and provide information that gets you—and people will talk about a linear probability model.  People often use the classic linear model, just OLS.  

I'm going to talk about the five assumptions and then variations in the assumptions.  The variations in the assumptions is really going to guide your choice of estimator, and then if you go on to publish it, really going to affect where you're going to publish it and who likes it.  

Assumption one, that the dependent variable can be calculated as a linear function of a specific set of independent variables, plus an error term.  The idea is that you can model this using a linear function.  Now, I don't mean, when I say linear function, that it's a single line.  This is the same one which we said, if went back to the age—so height, gender, and height and gender—that you can still model this in terms of a linear function.  The other thing to keep in mind is that you can often transform variables that may not be linear on the raw data but they're linear on the transformed data.  

Violations to assumption one are omitted variables, and I'll give you an example of that in a second.  The non-linearities you can often think about of just maybe I could transform this variable.  If it's non-linear in raw dollars, maybe in log dollars it is linear.  

There are empirical tests for assumption one.  There are theory-based—maybe you can say, well, I'm really interested in estimating a Cobb-Douglas production function.  Well, right away that'll tell you you're going to log your dependent variable and you're going to log your independent variable.  There are empirically -based transformation tests.  There's this thing called common sense, which works surprisingly well.  

The challenge with a lot of these tests is they're not always particularly strong.  Sometimes they have low power, but here's common tests.  The Ramsey RESET test, the Pregibon Link test.  If you're particularly interested in a lot of these, you can easily look them up and they're published, Daryl Pregibon's work from, I think, '81 and Ramsey's RESET test.  

Assumption one and stepwise regression—and I sort of interject this here—many people are familiar with this idea of I want to include a whole bunch of data terms and then let the statistical software keep the variables that are significant, and you can choose the significance level.  You might say, regression, keep any statistical variable that's got a significance of less than .2 or .1, and in stepwise fashion you can have it move forward and include things or move backwards and have it drop things out.  

My general take on it is be careful when using this approach, and the reason for that is often we're working with very large data sets, in VA especially.  There's very little penalty for adding a nuisance variable, but there's a big penalty for dropping something that's important, even if it's just a confounder that you should be controlling for.  Sometimes those confounders are not significant at .2 but they are still important in the variable.  

For example, if we had ignored gender in our last equation, we're interested in this idea of what's the relationship between height and income, you can see the red line is the estimate without gender being included.  The other ones—and right away you can say, wow, without doing that everything is wrong.  The slope is wrong.  The intercept is wrong.  Yes, if you omit an important variable you have a biased regression.  Your estimates are wrong.  Omitting something is potentially catastrophic, including something, that nuisance that just doesn't associate with anything.  It's just one degree of freedom.  Now, if you've got a thousand people in your data set, you have a lot of degrees of freedom.  

Assumption number two, the expected value of the error term is zero.  A lot of the questions that we're going to get coming on from here are dealing with the error term and how we think about the error term.  E, this notation here—let me circle it—is an expectation.  The idea here is that—and you'll often see this in econometrics—that the expectation of that error term, the ui error term, is zero, so the average of that error term is zero.  Violations lead to a biased intercept.  

Now, the common way that we'll think about this is that if you're working a lot with cost data, you'll often end up in, whether it's, let's say, log function of cost, and you're going to get to this problem where you can't just retransform the data because the expectation of the error term is not zero and so you have to use these things called smearing estimators if you're working with logged costs.  This is a common assumption that's violated with cost data, so if you're particularly interested in using cost data we're going to have Paul Barnett is going to give two classes on analyzing cost data, and it's specifically referring to the violation of assumption number two in this model.  

Assumption three is that the error terms are independent and identically distributed error terms, so that let's say all of us are in a data set.  The error term associated with me, Todd, is independent of Patsi, who is also on this call, is independent from other people.  They are identically distributed error terms.  Now, you can think about ways in which this would be violated.  Let's say you've surveyed Todd and surveyed me ten times over ten points in time.  Well, that clearly, those relationship between those error terms are not independent.  They are correlated at some level, so that would break that down as autocorrelation.  

You can also have this problem with homoskedasticity where the error terms are not identically distributed, and cost data often gets back to that.  I'll give you a picture of that.  Here's heteroskedasticity.  Let's say you're interested in the relationship between the national cost, or cost n, which on the Y axis, and length of stay in a VA bedsection.  This is a very common thing that when you stay very short periods in a hospital, your costs tend to be small.  When you stay long, you get much more variability in the cost because there's a lot of stuff going on here that you have to take into account.  This could also create problems with your statistical model, your OLS model.  

The effects are is that if you're using linear model or OLS is your coefficients are unbiased, but your standard errors are inefficient so your standard errors are biased, and is often, I find, very helpful.  If you thought about plotting your data, especially if you're just looking at common associations like this, it's often helpful to plot it out and say, oh, yeah.  That does appear to be a problem.  

There are different statistical tests for heteroskedasticity.  Groupwise heteroskedasticity, GWHet, is a test on Stata that's common for this.  There are more general tests for heteroskedasticity, but the more general the test the less power you have to actually estimate that test.  

There are ways of fixing and dealing with this.  One might be, for example, that you can just transform your dependent variable.  Let's say you're interested in length of stay.  Maybe it's like, well, maybe if I log length of stay or log the cost, those will help.  In that case it was possible that your dependent variable was a log of the cost, but you could also log your independent variable.  

Another way of handling this is what's known as the robust standard errors.  If you think about you have this variance, covariance matrix.  The robust standard errors is working off the diagonal on that, and it's also known as Huber White or sandwich estimators, to get an unbiased estimate of your standard errors.  I will say, and I'll mention this later on as well, is that I often use Stata for my analyses and I know people who use R [statistics language].  I often work in SAS but I often do my analyses in Stata because using, for example, the robust standard errors in Stata is just a single comma R, and I always find it hard for me to do what I want to do in Stata and very—hard to do what I want to do in SAS and very easy what I want to do in Stata.  

All right.  Assumption number four, observations on independent variables are considered fixed in repeated samples.  Boy, that sounds like a mouthful, and there is an even crazier expectation, but really the question is, is if you're familiar with—you've got this problem that your independent variable is not uncorrelated, or is correlated with your error term, and the idea of endogeneity, and we worked through this.  This is a classic problem with many health services research analytical models where you're, for example, you're including something on the right-hand side that you think is really interesting.  We give the example of intensity of PET use.  

There are many reasons that you can come up with these violations and Christine Chee, who is a new health economist here, is going to walk through these in a future class where she's going to talk about what does it mean to have the errors and variables.  That just means that you've got measurement error in how you think about your right-hand side variables, and that measurement error is not consistent across every episode of variables, that it's actually correlated with the error term.  You can have autoregression.  You can have simultaneity.  You can also have sample selection, or selection biases, built in.  That creates a violation in assumption four.  

Here's the discussion about the errors in variables.  You've got this measurement error in the dependent variable is maintained in the error term, and OLS assumes that the covariates are measured without error, and the error in measuring the covariates can be problematic, especially if it's correlated.  

Common ways that we encounter this are if you lag your dependent variable—so let's say you're working with time series data, repeated time series data.  If you're interested in somebody's behavior today and you put their behavior yesterday as an independent variable, that creates a problem right off the bat.  There are ways to test for these violations.  The Hausman test is a very common test, but it's weak, especially in small samples.  The solution for this is instrumental variables, and Christine is going to talk at length about instrumental variables.  You might be familiar with this idea of propensity scores.  Propensity scores is not a solution for this, even though everybody wants it to be, in part because it's so hard to find good instruments to use in instrumental variables.  

Assumption five is probably the easiest assumption, is that you've got—the observations have to be greater than the covariates.  If you have three people, you can't estimate a model with an intercept and three covariates.  You just can't do it.  Now, in VA that is almost never a problem, but that's a problem, for example, if you're a macroeconomist and you're interested in the annual GDP, gross domestic product of the US, and you're interested in how maybe the treasury policies affect the GDP.  Well, you could imagine quite quickly if you want to control for a couple of things, you have to have a huge time series or you're just not going to have enough power to do it.  Your observations are not going to be bigger than your covariates.  You also have to assume that there's no multicollinearity.  Sometimes we'll get variables on the right-hand side that are highly collinear.  

Now, the solutions, one is to remove, if there's perfect linear variables, Stata will drop it right off the bat.  It's often, it's not intended.  It's usually, it's by accident you have perfectly collinear variables, but if you've got perfectly collinear variables it's to remove one.  You just can't estimate two variables that are perfectly collinear, move exactly the same way.  The other solution is to increase your sample size, so if you're interested in GDP maybe you can just pool data across multiple years.  

If you're interested in relationships in your—for example, said, well, I'm really interested in the intensity of our minimally invasive surgery techniques on patient outcomes, and in one year of data—and I'm particularly interested in kidney surgery—in one year of data I've got 200 patients.  You might say, all right, that's too small.  Well, then, you can pool your data across the years.  Pooling data across time increases your sample size.  

Any questions?  Got about five minutes left.  I'm so glad my voice has held out to this point, and I appreciate everybody's patience.  Patsi, I can't—any questions?

**Moderator:**There was a hand up for a while, but I think that went away.  

**Todd Wagner:**I don't know if we lost all the questions when the system went down for a bit.  When we lost—

**Moderator:**I have a question that says, "Yes, we lost sound.  Could you revisit the elements of the equation slide?"  I assume that's the first equation slide, and please correct me if I'm wrong.

**Todd Wagner:**All right.  Let me see if I can find it quickly in this new environment.  Okay.  Hopefully people can see this.  The elements of an equation.  I started, actually, with the slide that didn't have the Z in it, didn't have this one here, but for right now I'm just going to make it this slide because it's a little bit easier.  

Here's the equation.  If you remember geometry, you talked about a line.  Many of the times, what we're trying to do with statistics is fit lines to the data.  Here we have our dependent variable, Y.  Y is commonly used as a dependent variable in statistics, in econometrics.  Now, Y is just a generalized notation for whatever is the dependent variable.  You'll often hear people say, well, I'm interested in a relationship between covariates and income, so income would be, for example, your Y.  It could also be your health outcome.  Perhaps you're interested in mortality, and so Y would be your mortality.  It could be cost, and Y would be cost.  

You then have this beta nought, which is your intercept, and we had talked about if everything else falls out of your equation, that what—and all you're left with is this—I apologize.  I'll get rid of all the lines in a second.  Is all you're left with this is Y equals beta nought, that what beta nought is, is just the average of Y.  Let's say you're interested in your students' heights or heights in your kids' classrooms and you measure all the kids, and you're saying, well, does, I don't know, parental income associate with the kids' height?  You find, no, it doesn't.  In your equation, what you're going to find is that your beta nought, your intercept for your line, is just the average height of the kids, and it's going to be a flat line just at the average.  Let me get rid of all those lines so it doesn't get too—

**Moderator:**Todd, I have a question.  Why is endogeneity a problem?

**Todd Wagner:**Why is endogeneity a problem?  

**Moderator:**Mm-hmm.

**Todd Wagner:**Because endogeneity, well, the classic linear model in OLS assumed that all of your right-hand side variables are not associated with your error term.  When you have a variable, let's say like smoking, implicitly there are things about smoking that are—we can't control for.  Why do you smoke?  Well, you smoke for all sorts of reasons.  Perhaps your parents were smokers.  Perhaps you grew up in a poor neighborhood.  Perhaps your friends at age three were smokers.  Perhaps it was your friends at age five were smokers.  Now, if you were able to control and observe everything that's associated with smoking, you could do away with that problem that smoking is correlated with the error term, but in every data set that I know of we're not able to control for everything.  

Implicitly, we've got this embedded correlation between smoking and the error term, and so that create a bias in our smoking variable.  That bias can be big or small.  It's not clear.  It can change the entire direction of the effect.  You might come up with and say, in your results, smoking is good for you.  Well, we know that that is actually wrong, but you could—there's many analyses where there's no right or wrong.  If it's endogenous, it gives you pause and reason to think about how do I want to identify the effect?  Often when econometricians talk about identification, they're really trying to identify the causal linking between independent variables and the dependent variable.  

That's a particular case in health services research that frequently comes up and we really struggle with, and so most graduate students in econometrics or graduate students in health services are often looking for these situations that create a pseudo-randomized trial where it's quite plausibly exogenous on the independent variable.  Those are hard to find, so that's the trick.  Christine, she, like I said, is going to have a whole lecture on endogeneity and ways to get around it and what instrumental variables is trying to do.  If you're particularly interested in that I would urge you to come back in a couple of weeks.  

**Moderator:**Okay.  I have another question.

**Todd Wagner:**Sure.

**Moderator:**Could you go over the graph showing heteroskedasticity?  How does it show heteroskedasticity?

**Todd Wagner:**Ah, that's a great question.  I appreciate the person bringing that up.  Let's see if I can find it.  Heidi, if there's a way that I can see the thumbnails of my slides and then move to them, but I think I don't—

**Moderator:****I wish there was but right now there is not.

**Todd Wagner:**Okay.  Here we go.  All right.  Let's say you run your regression.  I've got to get my tool.  You run your regression.  Assume this is a straight line.  My hand's not perfect.  That would be your, the line that gets fit to that function that says that, yes, there is a positive association between length of stay and cost.  That's obvious.  Now, if you think about what OLS is trying to do, imagine each of those points getting dropped to that line.  There's a very tight correlation at this end.  Out here it's a very wide correlation, so that there is, if you were to look at the variance at this end, it's very different than the variance at this end.  That's heteroskedasticity.  Over length of stay, the variance changes.  That's what's creating your heteroskedasticity.  I hope that was clear.  Patsi, any other questions?  Are you just answering as we go here?

**Moderator:**No, the rest of the questions so far have been about availability of slides and things like that—I've got to find them—that Heidi's going to**—

**Todd Wagner:**Why does my voice sound so bad.  *[Laughter]*

**Moderator:**No.  *[Laughter]*  Actually no one so far has complained.

**Todd Wagner:**I actually got an email from somebody—thank you very much—for saying it didn't sound so bad.  It sounds bad to me.

**Moderator:**And a thank you.

**Todd Wagner:**Well, thank you.  I appreciate everybody coming.  Like I said, go to the HERC website and you can see the lecture courses that we're going to have, and we're trying to have them all tie back into how they—for example, the cost data, how that ties back into what assumption it's breaking. I'm really trying to encourage the conceptual ideas behind these things.  We've had discussions here about should we show the math, and I think we've erred on not showing the math and just trying to show what the concepts are.  If you're particularly interested in the math, we can give you additional readings or point you in the right direction, but we're particularly interested in making sure people understand that—why endogeneity is a problem and then ways of, when you get to that class, ways of addressing it.  Thank you all for coming.  I don't think there's anything else.

**Moderator:**No, there are no other questions.

**Todd Wagner:**And I'm just amazed—

**Moderator:**I'm just going to put a plug in that Christine Chee is presenting next Wednesday on research design, and we will be sending registration information on that any moment.  It's kind of been a crazy day around here so I haven't gotten that out yet, but you should all see that in your email boxes later on today or first thing tomorrow morning.

**Todd Wagner:**Can you remind me, Heidi, I know that at some point we start losing VANTs Operations and we start losing—I think we're good through October and if the—

**Moderator:**No, VANTs is only funded through this Friday so—

**Todd Wagner:**Okay, then we're going to start losing some capabilities to run these courses if the government doesn't get funded.

**Moderator:**I always have a backup plan in mind, so if VANTs is not running next week, if the furloughs are continuing to happen next week, we will be using a toll number for the cyber seminars but we will be continuing to stream the audio for the people that are not able to call in on a toll number.  We will still have the option to stream the audio through your computer or, if you do need to use a telephone, it will be a toll number if we don't have VANTs capability.

**Todd Wagner:**Okay, and you've been so fantastic on this.  I'm sure that you'll let everybody know ahead of time whether we're—

**Moderator:**Yes.  Yes, we will.

**Todd Wagner:**- on or off, so thank you so much.

**Moderator:**We send that reminder the morning of the session, and we will make sure that all of that information is included in there, so everyone, keep your eye on that reminder that we send out the morning of the session.

**Todd Wagner:**Yes, and—

**Moderator:**I just want to—go ahead.

**Todd Wagner:**I was going to say, in a world where some people are not getting their payments and their checks to live on, this is sort of a first world problem but we'll work through it.

**Moderator:**Exactly.  Exactly.

**Todd Wagner:**I just wanted to highlight that there were a couple of questions about the slides.  The slides are also included in the reminder that arrives the morning of the cyber seminar, so if you want to download the slides ahead of time you can do so.  Generally, after the fact, you can also find them on the HERC website.

**Moderator:**Yes, and they're also included on the archives page which we typically send out to everyone the day after the session, so there's a few different resources to be able to find those slides before and after the session.

**Todd Wagner:**Perfect.  Hopefully I'll have my voice back before I have to teach again.

**Moderator:**Hopefully.  For our audience, I want to thank everyone for joining us for today's session.  As I close out the session today, I will be putting up a feedback form for everyone.  We are very aware of the issues that we had with VANTs today.  I wanted to apologize for that, but I'm hoping in your feedback you will give us some feedback on the content of the session.  Todd would appreciate that a lot more than hearing that we are continuing to have trouble with VANTs.  Thank you, everyone, for joining us for today's HSR&D cyber seminar, and we hope to see you at a future session.  Thank you.

**Todd Wagner:**And thank you, Heidi and Patsi, and hopefully for everybody out there who stayed with us it was worth your hour, so thank you so much.

**Moderator:**Thank you.

*[End of Audio]*

Page 1 of 17