Christine Pal Chee

October 9, 2013

Research Design
Health Services Research

 Many questions in health services research aim to 

establish causality

– Does the adoption of electronic medical records reduce 

health care costs?

– Did the transition to Patient Aligned Care Teams 

(PACT) improve quality of care and health outcomes?

– What effect will the Affordable Care Act (ACA) have on 

the demand for VHA services?

 Ideally studied through randomized controlled trials

 When can regression analysis of observational data 

answer these questions?

2
Poll: Familiarity with Regressions

 How would you describe your familiarity 

with regression analysis?

A. Regression is my middle name.

B. I’ve run a few regressions and get the gist 

of how they work.

C. I took a statistics class many years ago.

D. What is a regression?

3
Objectives

 Provide a conceptual framework for research 

design

 Review the linear regression model

 Define exogeneity and endogeneity

 Discuss three forms of endogeneity

– Omitted variable bias

– Sample selection

– Simultaneous causality

4
Research Question

 Start with a research question:

– What is the effect of X on Y?

 For example:

– What effect does receiving antiretroviral 

therapy (ARVs) have on substance use?

5
Linear Regression Model

Yi = β0 + β1X1i + β2X2i + ei

 Y: outcome variable of interest

 X1: explanatory variable of interest

 X2: additional control variable

 e: error term

– e contains all other factors besides X1 and X2 that determine the value 

of Y

– e is the difference between the observed and predicted values of Y

 β1: the change in Y associated with a unit change in 

X1, holding constant X2

– β 

1 is our estimate of β1

 Model specifies all meaningful determinants of Y

6
Linear Regression Model (2)

 In our example:

substancei = β0 + β1ARVi + ei

 substance: dependent variable

 ARV: independent variable

 e: error term

– e contains all other factors besides receiving ARVs that 

determine substance use

 β1: the change in the likelihood of substance use 

associated with receiving ARVs

 When does β1 estimate the causal effect of receiving 

ARVs on substance use?

7
Exogeneity

 Assumption: E ei Xi = 0

– Conditional mean of ei given Xiis zero 

 Conditional mean independence

– X is “exogenous”

 Knowing Xi does not help us predict ei

– ei: the difference between the observed and predicted 

values of Yi, contains other factors besides Xithat 

determine the value of Yi

– Information other than Xi does not tell us anything more 

about Yi

 Implies that Xi and ei cannot be correlated

8
Exogeneity (2)

 In the context of a randomized controlled trial:

outcomei = β0 + β1treatmenti + ei

 ei can include things like age, gender, pre-existing 

conditions, income, education, etc.

 Because treatment is randomly assigned, treatment

and e are independent

– This implies treatment is exogenous

 In observational studies, treatment is not randomly 

assigned

– The best we can hope for is that treatment is as if

randomly assigned

9
Exogeneity (3)

 In our example:

substancei = β0 + β1ARVi + ei

 In order for β

 

1 to estimate the causal effect of ARVs 

on substance use, ARV must be exogenous

– All factors other than receiving ARVs do not tell us 

anything more about substance use

 In the context of a randomized controlled trial, ARV

is exogenous

– Is the same true in the context of observational studies?

10
Endogeneity

 Violation of the exogeneity assumption 

– X is endogenous

– Always true when Xiis correlated with ei

 β

 

1 is biased

– β 

1 is unbiased if the expected value of β 1 is equal to 

the true value of β1

 β

 

1 will not estimate a causal effect of X on Y

– β 

1 is a measure of the correlation between X and Y

– Correlation does not imply causation

11
Forms of Endogeneity

 Omitted variable bias

 Sample selection

 Simultaneous causality

12
Omitted Variable Bias

 Arises when:

– A variable omitted from the regression model 

is a determinant of the dependent variable, Y

– The omitted variable is correlated with the 

regressor, X

 Leads β

 

1 to be biased

– β

 

1 also captures the correlation between the 

omitted variable and the dependent variable

13
Omitted Variable Bias (2)

 Regression model: Yi = β0 + β1Xi + ei

 Say another factor, Wi, determines Yi

– Wiis included in the error term, ei

 If Xi and Wi are correlated

– Xi and ei are correlated

 Xiis endogenous

– β

 

1 is biased

 β 

1 captures the correlation between Wi and Yi

14
Omitted Variable Bias: Example

 In our example:

substancei = β0 + β1ARVi + ei

 Two questions:

– Besides receiving ARVs, do any other factors 

determine substance use?

– Are those factors correlated with receiving 

ARVs?

 Consider two factors: education and health

15
Omitted Variable Bias: Example (2)

 Education

– Individuals who are more highly educated are 

less likely to engage in substance use

– Individuals who are more highly educated are 

more likely to receive ARVs

 Health

– Individuals who are sick are less likely to 

engage in substance use

– Individuals who are sick are more likely to 

receive ARVs

16
Omitted Variable Bias: Example (3)

 A change in β

 

1 suggests omitted variable 

bias

17
Omitted Variable Bias: Solutions

 Randomized controlled trial

 Multiple linear regression

– Include all relevant factors in the regression model 

so that we have conditional mean independence

– What if it is not possible to include omitted 

variables in the regression?

18
Omitted Variable Bias: Solutions (2)

 Utilize panel data (same observational unit observed 

at different points in time)

– Fixed effects regression: Possible to control for 

unobserved omitted variables as long as those omitted 

variables do not change over time 

– For more information: Stock and Watson, Chapter 10

 Instrumental variables regression

– Utilize an instrument variable that is correlated with the 

independent variable of interest but is uncorrelated with 

the omitted variables

– More on this in the Instrumental Variables Regression 

lecture on Oct 30

19
Sample Selection

 Arises when:

– A selection process influences the 

availability of data

– The selection process is related to the 

dependent variable, Y, beyond depending on 

X

 Leads β

 

1 to be biased

20
Sample Selection (2)

 Form of omitted variable bias

– The selection process is captured by the 

error term

– Induces correlation between the regressor, 

X, and the error term, e

21
Sample Selection: Example

 Classic examples from economics:

– The effect of union membership on wages

 wagesi = β0 + β1unioni + ei

 Problem: a worker’s decision to join a union is 

endogenous; it may depend on many factors, including 

the difference between his union vs. non-union wages

– The effect of college education on wages

 wagesi = β0 + β1collegei + ei

 Problem: a person’s decision to go to college is 

endogenous; it may depend on many factors, including 

her anticipated return to college education

22
Sample Selection: Example (2)

 In the VA:

– Want to evaluate the effect of a new tobacco dependence 

treatment program on quitting

 quiti = β0 + β1treatmenti + ei

 Individuals who participate in the program may be more likely to 

quit to begin with

– Want to evaluate the effect of home-based primary care on 

nursing home use

 nursingi = β0 + β1HBPCi + ei

 Individuals who use home-based primary care may be more 

plugged in and likely to know about other available programs

 Facilities that adopt home-based primary care may already have 

other supportive programs to begin with

23
Sample Selection: Solutions

 Randomized controlled trial

 Sample selection and treatment effect models

– For more information: 

 Greene, 2000 Chapter 20

 Wooldridge, 2010, Chapter 17

 Instrumental variables regression

– More on this in the Instrumental Variables 

Regression lecture on Oct 30

24
Simultaneous Causality

 Arises when:

– There is a causal link from X to Y

– There is also a causal link from Y to X

 Also called simultaneous equations bias

 Leads β

 

1 to be biased

– Reverse causality leads β

 

1 to pick up both 

effects

25
Simultaneous Causality: Example

 We want to estimate the effect of receiving 

ARVs on substance use

 Regression model: 

substancei = β0 + β1ARVi + ei

 If substance use also affects the likelihood of 

receiving ARVs:

ARVi = γ0 + γ1substancei + εi

 Both equations are necessary to understand the 

relationship between ARVs and substance use

26
Simultaneous Causality: Example (2)

 We now have two simultaneous equations:

substancei = β0 + β1ARVi + ei(1)

ARVi = γ0 + γ1substancei + εi(2)

 Suppose a positive error eileads to a higher value of substancei

substancei = β0 + β1ARVi + ei(1)

 A higher value of substanceileads to a higher value of ARVi

ARVi = γ0 + γ1substancei + εi(2)

 Therefore, a positive error eileads to a higher value of ARVi

– ei ↑ → ARVi ↑

– ARVi and ei are correlated

– β 

1 is biased

27
Simultaneous Causality: Solutions

 Randomized controlled trial where the 

reverse causality channel is eliminated

 Instrumental variables regression

– Utilize an instrumental variable that is 

correlated with X (determines ARV) but is 

uncorrelated with the error term (does not 

otherwise determine Y, substance use)

– More on this in the Instrumental Variables 

Regression lecture on Oct 30

28
Summary

 Good research design requires an understanding of how the 

dependent variable is determined

 Need to ask: is the explanatory variable of interest exogenous? 

– Are there omitted variables?

– Is there sample selection?

– Is there simultaneous causality?

 Exogeneity is necessary for the estimation of a causal treatment 

effect

 Understanding sources of endogeneity can:

– Help us understand what our regression estimates actually estimate and 

the limitations of our analyses

– Can point us to appropriate methods to use to answer our research 

question

29
Resources

 Stock and Watson, Introduction to 

Econometrics, 3rd edition (2011)

 Green, Econometric Analysis, 7th edition 

(2012) 

 Wooldridge, Econometric Analysis of 

Cross Section and Panel Data, 2nd edition 

(2010)

30