Right Hand Side 

(Independent) Variables

Ciaran S. Phibbs

November 6, 2013
Independent Variables

 Regression models make several 

assumptions about the independent 

variables 

 The purpose of this talk is to examine 

some of the more common problems, and 

some methods of fixing them

 Focus on things not likely to be covered 

in standard econometric classes
Outline 

 Heteroskedasticity 

 Clustering of observations 

 Functional form 

 Testing for multicollinearity 
Heteroskedasticity 

Yi = β0 + βX + εi

 Assumes that the error terms are 

independent of xi. Common pattern, as x 

gets bigger, e gets bigger.
Heteroskedasticity 

 Biased standard errors 

 Parameter estimates unbiased, but 

inefficient 
Heteroskedasticity 

 Simple solution, robust option in Stata 

uses Huber-White method to correct 

standard errors. 

 May also consider transformation of 

variables, e.g., log(X) instead of X as a 

RHS variable.
Clustering 

Yi = β0 + βX + εi

 Assumes that the error terms are 

uncorrelated 

 Clustering is a common problem, for 

example, patients are clustered within 

hospitals 
Clustering 

Yi = β0 + β1x1 + β2x2 + εi 

 x1 is a patient level variable, and x2 is a 

hospital level variable

 Regression assumes there are as many 

hospitals as patients

 Result, standard errors for β2 are too small, no 

effect on parameter estimate.
Correcting for Clustering

 Generalized Estimating Equations (GEE) 

or other hierarchical methods can be used

 Alternatively, Stata “cluster” option uses 

a Huber-White correction of the standard 

errors. 

 Both methods can yield essentially the 

same result, it depends on the data 

structure 
Correcting for Clustering

 Hierarchical Linear Modeling. Method 

of formally incorporating hierarchical 

structures into the model. 

 Can use for non-linear models also.

 Need for HLM, vs. other methods will 

depend on structure of the data. Often 

very similar answers.
Example of Clustering

 I had a research project that looked at the 

effects of NICU patient volume and 

NICU level on mortality. NEJM 2007. 

 I apologize for not having a VA example, 

but I had already carefully addressed 

these issues for this project. 
Clustering 

 Failure to make this correction happens all 

too often. It is easy to fix

 Extent of the correction varies with sample 

size, and with the number of clusters, relative 

to the number of observations.

 With big samples, the effects are fairly small. 

My example, N>48,000, >200 hospitals, 10 

years of data, with repeat observations.
Example of Clustering

Level of Care/VLBW volume OR 95% C.I. unadjusted

Level 1 ≤10 VLBW infants 2.72** (2.37, 3.13) 2.40, 3.07

Level 2 11-25 VLBW infants 1.88** (1.56, 2.26) 1.64, 2.15

Level 2 >25 VLBW infants 1.22 (0.98, 1.52) 1.09, 1.36

Level 3B or 3C ≤25 VLBW 1.51** (1.17, 1.95) 1.25, 1.78

Level 3B or 3C 26-50 VLBW 1.30** (1.12, 1.50) 1.17, 1.42

Level 3B, 3C, or 3D 51-100 1.19* (1.04, 1.37) 1.10, 1.29
Functional Form

Yi = β0 + βX + εi

βX assumes that each variable in X has a 

linear relationship with Y 

 This is not always the case, can result in a 

mis-specified model
 

 You should check for the functional form 

for every non-binary variable in your 

model. 

 There are formal tests for model 

specification, some of which you may 

have been exposed to in classes. But, 

these tests don’t really show you what 

you are looking at.
Using Dummy Variables to 

Examine Functional Form

1. Look carefully at the distribution of 

each variable 

2. Create a set of dummy variables for 

reasonably small intervals, with no 

excluded category 

3. Run model with no intercept 
Example of Using Dummy 

Variables to Examine 

Functional Form

 Same NICU dataset as before. 

 Effect of NICU patient volume on 

mortality

 Overall, and within level of care 
Example of Using Dummy 

Variables to Examine 

Functional Form

 Graph out the parameter estimates for each 

dummy variable. 

 Gives you a good idea of what the function 

looks like.

 Can use to determine which functional forms 

would be good starting points, or where to 

make the cuts for categorical variables.
Dummy Variable Look 

at Functional Form

0

0.5

1

1.5

2

2.5

3

3.5

0 50 100 150 200

Series1
Dummy Variable Look 

at Functional Form

0.00

0.50

1.00

1.50

2.00

2.50

3.00

Levels 1-2

Levels 3A-3B

Levels 3C-3D
Example of Using Dummy 

Variables to Examine 

Functional Form

 For some applications you may just want to 

use dummy variables, instead of a continuous 

functional form. This may be especially useful 

when there are complex relationships. It can 

be very difficult to get a continuous function 

to accurately predict across the entire range of 

values.

 Aside, categorical variables frequently easier 

to present to medical audiences. 
Dummy Variables to Capture 

Complex Functional Forms

Level of Care/VLBW volume OR 95% C.I. 

Level 1 ≤10 VLBW infants 2.72** (2.37, 3.13)

Level 2 11-25 VLBW infants 1.88** (1.56, 2.26)

Level 2 >25 VLBW infants 1.22 (0.98, 1.52)

Level 3B or 3C ≤25 VLBW 1.51** (1.17, 1.95)

Level 3B or 3C 26-50 VLBW 1.30** (1.12, 1.50)

Level 3B, 3C, or 3D 51-100 1.19* (1.04, 1.37)
Multicollinearity 

Yi = β0 + β1x1 + β2x2 + εi 

 What if x1 and x2 are strongly correlated? 

Regression has trouble attributing effect 

to each variable. 

– Increases standard errors

– Can affect parameter estimates
Multicollinearity 

 Strong simple correlation, you have a problem. 

But, can be hidden problems not detected by 

simple correlations.

 Variance Inflation Factor (/VIF SAS, vif in 

Stata Regression Diagnostics) measures the 

inflation in the variances of each parameter 

estimate due to collinearities among the 

regressors 

 Tolerance, which is 1/VIF

 VIF > 10 implies significant collinearity 

problem
Testing for Multicollinearity

 Look at the simple correlations. 

– General rule of thumb, need to look of r>0.5

– Note, can still have collinearity problems 

with r<0.5 
Example of Correlation and VIF

 Study of nurse staffing and patient 

outcomes. 

– Problem variables. RN Tenure and RN Age

– R=0.46

– VIF range, 18-30, depending on subset

Result, many fewer statistically significant 

results than we expected. 
Fixing multicollinearity

 More observations. As long as there isn’t 

perfect correlation, additional observations 

help.

 Revise data in ways that reduce the correlation

 In nurse staffing example, dropped age from 

model
Example of parameter effects of 

multicollinearity 

 Average RN tenure on unit and average 

age of RNs on unit, corr = 0.46 

– Tenure only -0.013**

– Age only -0.003 **

 
Example of parameter effects of 

multicollinearity 

 Average RN tenure on unit and average 

age of RNs on unit, corr = 0.46 

– Tenure only -0.013**

– Age only -0.003 **

– Both tenure -0.003 ns

 Age -0.0051 ns

 
Multicollinearity 

 Strong simple correlation, you have a problem. 

But, can be hidden problems not detected by 

simple correlations.

 Regression, n-space, correlation on each of the 

regression planes can matter. 

 Collin option in SAS, looks at how much of 

the variation in each eigen vector is explained 

by each variable. Intuitively, the correlation in 

the Nth dimension of the regression.
SAS Collin option

 SAS Model Y= var1 ... varN/collin;

 Continue newborn example

 Birth weight and gestational age very 

correlated. R=0.56

 Simple model, only BW, GA, Black
Interpreting Collin output

 Condition index >10 indicates a 

collinearity problem

 Condition index >100 indicates an 

extreme problem

 There is strong correlation in the variance 

proportion if 2 or more variables have 

values >0.50.
SAS Collin output

Eigen 

value

Condition 

index

Black BW GA

3.18 1.00 0.02 0.00 0.00

0.77 2.03 0.96 0.00 0.00

0.04 9.27 0.01 0.80 0.02

0.001 18.41 0.00 0.39 0.98
Fixing multicollinearity,

NICU example

 Used dummy variables for BW in 100g 

intervals to 1000g, then 250g intervals.

 Separate BW dummies for singleton males, 

singleton females, and multiple births,

 Gestation in 2 week intervals.

 Max condition index < 8, i.e. no serious 

collinearity problem. 

 Model predictions also improved.
Dummy Variables To Fix 

Collinearity

0

2

4

6

8

10

12

14

16

18

650 750 850 950 1125 1375

Odds Ratios

Birth Weight

Female

Multiple

Male
References 

 Belsley, D.A., Kuh, E., and Welsch, R.E. 

(1980) Regression Diagnositics. New 

York, John Wiley & Sons. 
Next lecture

Limited Dependent Variables

Ciaran Phibbs

November 13, 2013