Right Hand Side Right Hand Side

(Independent) Variab(Independent) Variab

Ciaran S Phibbs Ciaran 

S. Phibbs

June 6, 2012
Independent Variable

 Regression models make several 

assumptions about the independenassumptions about the independen

variables

 The purpose of this talk is to exam

some of the more common 

proble

some methods of fixing them 



Focus on things not lik t t lik

e

ly 

to 

be co

in standard econometric classes
Outline 

 Heteroskedasticity Heteroskedasticity  Clustering of observations Clustering of observations  Functional form Functional form  Testing for multicollinearity Testing for multicollinearity
Heteroskedasticity

Y = 

β

0 + 

β

X + 

ε

Yi + 

βX+ 

ε

 Assumes that the error terms are 

i de e de t i

nde

p

e

nde

n

t o

f x

i. C tte ommon patte

g gg ets bigg er, e 

gets bigger.
Heteroskedasticity 

 Biased standard errors  Parameter estimates unbiased, but 

inefficient 
Heteroskedasticity 

 Simple solution robust option in S

Simple solution, robust option in S

uses Huber-White method to correstandard errors. 

 Ma

y also consider transformation 

variables, e.g., log(X) instead of X 

RHS variable RHS variable. 
β β

Clustering 

Yi = β0 + βX + ε i 0 i 

 Assumes that the error terms are Assumes that the error terms are 

uncorrelated 

 Clustering is a common problem, 

example, patients are cl i 

ustered wi

hospitals 
C i lustering

Yi = β0 + β1x1 + β2x2 

 x1 is a patient level variable, and x2 is 

hospital level variable hospital level variable 

 Regression assumes there are as many

hospitals as patients 

 Result standard errors for Result, standard errors for β2 are too s

are too s

effect on parameter estimate. 
Correcting for Clusterin

 Generalized Estimating Equations 

or other hierarchical methods can or other hierarchical methods can 

 Alternatively, Stata “cluster” optio

a Huber-White correction of the sterrors. 

 Both methods can yield essentially 

same resul i t,t depends on the data 

structure structure 
Correcting for Clusterin

 Hierarchical Linear Modeling. M

of formally incorporating hierarchof formally incorporating hierarch

structures into the model. 

 Can use for non-linear models also. 

 Need for HLM, vs. other methods 

depend on structure of the data depend on structure of the data. O

very similar answers. 
Example of Clustering

 I had a research project that looke

effec

ts of NICU t f NICU patien

t vo

lume an

NICU level on mortalit

y. NEJM 2

 I apolo

gize for not havinga VA ex

but I had already carefully address

these issues for this project these issues for this project. 
Clustering

 Failure to make this correction happe

too often. It is easy to fix 

 Extent of the correction varies with s

size, and with the number of clustersto the number of observations. 

 With big samples, the effects are fair

My example, N>48,000, >200 hospit

years of data, with repeat observation
Example of Cl i Clustering

Level of Care/VLBW volume Level 1 Level 1 ≤10 VLBW infants 10 VLBW infantsLevel 2 11-25 VLBW infantsLevel 2 >25 VLBW infantsLevel 3B or 3C ≤25 VLBW 

Level 3B or 3C 26-50 VLBW 

Level 3B, 3C, or 3D 51 Level 3B, 3C, or 3D 51 -100

OR 95% C.I. 

2 72**. (2 37. , 3 13) .

1.88** (1.56, 2.26)

1.22 (0.98, 1.52)

1.51** (1.17, 1.95)

1.30** (1.12, 1.50)

1.19* (1.04, 1.37) (1.04, 1.37) 

u
Functional Form

Yi = 

β

0 + 

β

X + 

Y

i 0 

β

X 

β

X assumes that each variable in X 

assumes that each variable in X 

linear relationshi

p with Y 

 This is not alwa

ys the case,can res

mis-specified model 
 You should check for the function

for every non for every non-binary variable in y

binary variable in y

model. 

 There are formal tests for model There are formal tests for model 

specification, some of which you have been exposed to in classes. B

these tests don these tests don t ’really show you w

really show you w

you are looking at. 
Using Dummy Variables 

Using Dummy Variables 

Examine Functional ForExamine Functional For

1. L k oo

k care

full y at t

h ib e di stribut

ion 

each variable 

2. Create a set of dummy variables 

re bl ll i te l ith easonably small 

i

nterva

ls, with 

excluded cate

gy or

y

3. Run model with no intercept 
Example of Using Dumm

Example of Using Dumm

Variables to Examine Variables to Examine

Functional Form

 Same NICU dataset as before Same NICU dataset as before.  Effect of NICU patient volume on Effect of NICU patient volume on 

mortality 

 Overall, and within level of care 
Example of Using Dum

Example of Using Dum

Variables to Examine 

Functional Form

 Graph out the parameter estimates for 

dummy variable dummy variable. 

 Gives you a good idea of what the fun

loo

ks lik e.

 Can use to determine which functiona

would be good starting points, or whemake the cuts for cate

gorical variable
Dummy Variable Look Dummy Variable Look

at Functional Form

3.5 2.5 

3 

1.5 

2 

0.5 

1 

0 

0 50 100 150 200 
3.00 

2.00 2.50 1.00 1.50 0.00 0.50 

Dummy Variable Look Dummy Variable Look

at Functional Form
Example of Using Dumm

Variables to Examine 

Functi onalF orm

 For some applications you may just w

For some applications you may just w

use dummy variables, instead of a con

functional form This may be especiafunctional form. This may be especia

when there are complex relationships

be very difficult to 

get a continuous fu

to accurately predict across the entire values. 

 Aside, categorical variables frequentl

to present to medical audiences to present to medical audiences. 
-

-

Dummy Variables to Ca p

Complex Functional For

Level of Care/VLBW volume OR 95% C.I. Level 1 ≤10 VLBW infants 2.72** (2.37, 3.13Level 2 11 Level 2 11-25 VLBW infants 25 VLBW infants 1 88 . ** (1 56 2 26

(1.56, 2.26

Level 2 >25 VLBW infants 1.22 (0.98, 1.52Level 3B or 3C ≤25 VLBW 1.51** (1.17, 1.95

Level 3B or 3C 26-50 VLBW 1.30** (1.12, 1.50

Level 3B 3C Level 3B, 3C, or 3D 51-100 1 19 . * (1 04 1 37

(1.04, 1.37
β β β

–

Multicollinearity

Yi = β0 + β1x1 + β2x2 

i 0 1 1 2 2 

 What if x What if x and x are strongly corr

1 and x2 are strongly corr

Regression has trouble attributing to each variable. 

– Increases standard errors Increases standard errors – Can affect parameter estimates 
M lti lli it M

ulticollinearit

y

 Strong s

imp

le corre

l

ati on, you 

have a 

But, can be hidden problems not detec

simple correlations simple correlations. 

 Variance Inflation Factor (/VIF SAS, v

Stata Regression Diagnostics) measureStata Regression Diagnostics) measure

inflation in the variances of each param

estimate due to collinearities among thestimate due to collinearities among th

regressors

 Tolerance which is 1/VIF Tolerance, which is 1/VIF  VIF > 10 implies significant collineari

problem problem 
Testing for Multicollinea

 Look at the simple correlations Look at the simple correlations. 

– General rule of thumb

, need to look 

– Note, can still have collinearity prob

with r<0 5 with r<0.5 
Example ofC i orrelation an

 Study of nurse staffing and patien

outcomes. 

– Problem variables. RN HPBD and (

HPBD)**2 

– R 0= 92 . – VIF range, 10-40, depending on subResult, many fewer statistically signif

results than we expected. 
Fii i i x

ing mult

icoll

inear

it

 More observations. As long as there is

perf t l ti dditi l b 

fec

t corre

l

ation, additiona

l o

bserva

help.

 Revise data in ways that reduce the co In nurse staffing example, used set of 

to capture non-linearity, eliminated coproblem, more significant results 
Example of parameter effe

Example of parameter effe

multicollinearit

y 

 Same nurse staffing study.  Average RN tenure on unit and av

age of RN s on unit , corr = 

0 46 .

– Tenure onl

y -0.00011** 

– Age only 0.000012 ns 
Example of parameter effe

Example of parameter effe

multicollinearit

y

 Same nurse staffing study. Average RN tenure on unit and av

age of RN s on unit , corr = 

0 46 .

– Tenure onl

y -0.00011** 

– Age only 0.000012 ns 

– Both tenure -0.00003 ns 

Age 0.00051 ns 0.00051 ns
M lti lli it M

ulticollinearit

y

 Strong simple correlation, you have a 

But, can be hidden problems not detecsimple correlations. 



Regress

ion, n-space, corre

l ti 

ation on ea

regression planes can matter. 

 Collin option in SAS, looks at how mu

the variation in each eigen vector is exthe variation in each eigen vector is ex

by each variable. Intuitively, the corre

the Nth dimension of the regression. the Nth dimension of the regression. 
SAS Collin option 

 SAS Model Y= var1 ... varN/col Continue newborn example  Birth weight and gestational age v

correlated. R 0= 56 . 

 Simple model, only BW, GA, Bla
Interpreting Collin outp

 Condition index >10 indicates a 

collinearity problem 

 Condition index >100 indicates an Condition index >100 indicates an 

extreme problem 

 There is strong correlation in the v

p p ro

portion if 2 or more variables h

values >0.50. 
SAS C lli t SAS 

C

ollin ou

tpu

t

Eigen value 

Condition index 

Const Black BW 

3.18 1.00 0.00 0.02 0.00 0.77 2.03 0.00 0.96 0.00 0.04 9.27 0.19 0.01 0.80 0.001 18.41 0.81 0.00 0.19 
Fixing multicollinearity Fixing multicollinearity

NICU exam

ple

 Used dummy variables for BW in 100

i t l t 1000 th 250 i t l i

n

terva

ls 

to 1000g, then 250g 

i

n

terva

ls

 Se

parate BW dummies for sin

gleton m

singleton females, and multiple births, 

 Gestation in 2 week intervals. Gestation in 2 week intervals.  Max condition index < 8, i.e. no seriou

collinearity problem collinearity problem. 

 Model predictions also improved. 
Odds Ra

18 12 14 16 

tios 

6 8 

10

t 

650 750 850 950 1125 1375 

Birth Weight 

0 2 4 

y

Dummy Variables To FDummy Variables To F

Collinearity
References 

 Belsley D A Kuh E and Welsc

Belsley, D.A., Kuh, E., and Welsc

(1980) Regression Diagnositics. N

Y k or , Joh &S hn Wil ey & Sons.
Next lecture

Research Design Research Design

Jean Yoon Jean Yoon

July 11, 2012