Paul G. B ar

May 9, 

Econometrics Course: Econometrics Course:

Cost at the 

Dependent Variable (II)

nett, P

2012 
Rev ew o O d e 

ev

iew 

of 

O

r

dinaril

y 

L

e

a

Squares (OLS) 

 Class

ic li near mo

d

el

 Assume de

pendent variable can be 

expressed as a linear function of the 

chosen independent variables e g : chosen independent variables, 

e.g.:



Yi = 

α + 

β

Xi + 

εi 
R i f OLS Rev

iew of OLS assumpti o



Expecte

d va

lue o

f error 

is zero E( ε

i l 

 Errors are inde

pendent E

(

ε

i

εj

)=0 

i

 Errors have identical variance E(

ε

i

2



Errors are normall y di strib ute

d 

 Errors are not correlated with 

independent variables E(X

i

ε

i)=0 
C t i diffi lt i bl Cost 

is a difficult var

i

abl

 Sk d b b l hi h Skewed 

by rare 

but extreme

ly hi

g

h 

events 

 Zero cost incurred by enrollees who 

use care use care 

 No negative values 
R i ev ew from last session

 Applying Ordinary Least Squares OLS 

Applying Ordinary Least Squares OLS 

that aren’t normal can result in biased 

parameters 

– OLS can predict negative costs 

 Log transformation can make cost more 

normall di ib d lly distributed 

 Predicted cost is affected by re-transform

bias 

– Corrected using smearing estimator – Assumes constant error (homoscedasticity) Assumes constant error (homoscedasticity) 
T i op cs for today’s cours

 Wh at to do when there is 

heteroscedasticity? 

 What to do when there are many ze

values? values? 

 How to test differences in groups w

assumpti b di ib i ? ions about distribution? 

 How to determine which method is 
T i op cs for today’s cours

 Wh at to do when there is 

heteroscedasticity? 

 What to do when there are many ze

values? values? 

 How to test differences in groups w

assumpti b di ib i ? ions about distribution? 

 How to determine which method is 
Properties o v ce o 

of 

variance 

of t

errors



Homosce

dast

i

c

ity 

– Identical variance E(

ε

i

2)=

σ2 

 Heteroscedasticity 

– Variance depends on x (or on predict

Variance depends on x (or on predict
H d it Homosce

dasti city 

– 

Errors 

have identicalvar

iance E(

εi

2

)

=

σ

2 

4321013

0 5,000 10,000 15,000 20,

e 

-3 -2 -1 -4 
H te eroscedasti it c y

– E d d ( di t d ) Errors depend on x (or on predicted y)

3 4 1 2 

-1 

0 

0 5,000 10,000 15,000 20,0

e 

4 

-3 -2 

-4 
Why worry about Why worry about

heteroscedasticity?

 OLS with homosce

dastic re

trans

formati

– “If error term 

ε is heteroscedastic, estimate

appreci bl bi d” i

ably biased” 

– Reminding Manning and Mullahy of Longfellow

rhyme: 

“When she was good, she was very, very good, b

she was bad, she was horrid” 

JHE 20:46
Ge e ed e ode 

Ge

n

eralized Lin

ear Mode

(GLM)

 A l ifi li k f i ( 

Ana

lyst specifies a li

nk 

funct

ion g( 

 Anal

yst s

pecifies a variance functio

– Key reading: “Estimating log models

transform or not to transform transform or not to transform, ” Mulla

Mulla

Manning JHE 20:461, 2001 
– =

Li k f ti ( ) i GLM Link function g( ) in GLM 

 g (E( |y x) )=α + βx  Link function can be natural log, sq

root, or other function

Eg. . ln (E ( y | x) E ( y | x)) = α + βx 

– When link function is natural log, the

represents percent c hange in y 
GLM OLS GLM vs. OLS 

 OLS f l i E ( l ( ) | )

OLS of log estimate: E ( ln ( y) | x)

 GLM estimate: ln ( (y| E x)) 

– Log of expectation of y is not the sam

expectation of log Y! expectation of log Y! 

 With GLM to find predicted Y 

– No retransformation bias with GLM – Smearingestimator not used 
V i ar ance functi on 

 GLM does not assume constant var GLM assumes there is function that 

explains the relationship between tvariance and mean variance and mean 

– v (y | x) 
Variance assumptions for GLM 

Variance assumptions for GLM 

models 

 G Di ib i ( 

Gamma Distribut

ion 

(most common

– Variance is proportional to the square 

mean 

 Poisson Distribution Poisson Distribution 

– Variance is proportional to the mean 
E ti ti th d E

stimation meth

o

ds 



How 

to specif y 

log li n

k an

d gamma 

distribution with dependent variabland i

n

depen

d en

t var

i

ables X1 

, X2, 

 Stata 

GLM Y X1 X2 X3, FAM(GAM) LINK(

 SAS 

(warnin

g: SAS dro

ps zero cos

observations!!!!!!!) 

PROC GENMOD MODEL Y=X1 X2 X

DIST=GAMMA LINK=LOG; 
Choice between GLM Choice between GLM

and OLS of log transform

 GLM da

dvantages: 

– GLM can correct for heteroscedastici– GLM does not lead to retransformatio

 OLS of log transform advantages OLS of log transform advantages 

– OLS is more efficient (standard error

smaller t

han w

i h GLM) th GLM)


Which link f ?unction?

–Box-Cox regression 

–Stata command: boxcox cost {indep. vars} if y > 

COST  1 

   x   
Link function Link function Theta 

Inverse (1/cost) -1 Log(cost) 0 Square root (cost) .5 Cost 1 Cost Squared Cost Squared 2 

Which link f ?unction?

 Box-Cox parameter 
Whi h i t t ith G

Whi

c

h var

iance s

truc

ture with G

Modified Park test 

 GLM regression & find residual  Square the residuals Square the residuals  Second regression by OLS 

– Dependent variable squared residua

– Independent variable predicted y 

2

(Y 



Yˆ i 

i ) 





0 



 1 

Yˆ 

i 



i
Whi h i t t ith G

Whi

c

h var

iance s

truc

ture with G

 Parameter from GLM famil

y test 

(modified Park test) 

(

Yi

γ1 0 1 2 3 



Y ˆ 

)2 i 



 0 1 



Yˆ i i 



Variance Gaussian (Norma)PoissonGammaWald (Inverse Normal)
Other models for skewed d

 Generali zed gamma models 

– Estimate link function, distribution, a

parameters in single model 

– See: Basu & Rathouz (2005)
Questi ons?
T i op cs for today’s cours

 Wh at to do when there is 

heteroscedasticity? (GLM models) 

 What to do when there are many ze

values? values? 

 How to test differences in groups w

assumpti b di ib i ? ions about distribution? 

 How to determine which method is 
What to do when there are m

What to do when there are m

zeros values? 



Examp

le o

f part

i

c

ipants enroll edi n 

health plan who have no utilization 
Annual per person VHA costs F

Annual per person VHA costs F

among those who used VHA in F

0.40 

Medical Only Medical+Rx 

0 30 . 0.20 0.10 0.00 

no cost

$1K

$5K 

$10K 

$15K 

$20K 

$25K 
The two-part model 

 Part 1: Dependent variabl e is indi ca

any cost is incurred 

– 1 if cost is incurred (Y > 0) – 0 if no cost is incurred (Y 0 if no cost is incurred (Y 0) =

 Part 2: Regression of how much co

among those who incurred any cost
0 

The two-part model

 E t d l f Y diti l X Expected value of Y conditional on X 

E(Y | X )  P(Y  0) | X )E(Y | Y  

Is the product of: 

Part 2. Part 1. Expected valu

The probability that 

conditi l 

ditional o

Y is greater than zero, greater than zconditional on X 

conditional o
P di t t i Predi

c

t

e

d cost 

in 

two-part 

 m

 P di t d f Y Predi

c

t

e

d va

lue of Y 

E

(Y| X ) 



P

( 

Y 



0) | X E) | 0) | E

( Y Y | 

 

Is the product of: 

Part 2. Part 1. Predicted cos

Probability of any cost conditional o

conditional o

being incurred incurring any 
Q ti ues on for class 

P(Y 0) | X )

 Part one estimates probability Y > 0 

– Y > 0 is dichotomous indicator Y > 0 is dichotomous indicator – 1 if cost is incurred (Y > 0) – 0 if no cost is incurred (Y=0) 

 What type of regression should be used when 

dependent variable is dichotomous (takes a vaeither zero or one)? either zero or one)? 
First part of model First part of model

Regression with dichotomous va



L

ogi

sti c re

gress

ion or probit 

 Logistic regression uses maximum 

likelihood function to estimate lo

g odds 

ratio: 

Pi log 









X

1



Pi 

1 
L i ti og s c regressi t i 

on syntax in 

Proc Logistic; Proc Logistic;Model Y = X1 X2 X3 / Descending;Output out={dataset} prob={variable name};

 Output statement saves the predicted probabil

the dependent variable equals one (cost was d l in

 Descending option in model statement is requ

otherwise SAS estimates the probability that totherwise SAS estimates the probability that t

dependent variable equals zero
Logis c ess o stic regression syntax 

 Predict statement generates the pred

og eg sy 

Stata 

Logit Y = X1 X2 X3 Logit Y = X1 X2 X3 Predict {variable name}, pr 

P di t tt t t th d

probability that the dependent variaequals one (cost was incurred) 
Second part of model Second part of model

Conditional quantity



Regress

ion 

invo

lves on

ly o

bservat

i

with non-zero cost (conditional cosregression) 

 Use GLM or OLS with log cost Use GLM or OLS with log cost 
–

Two-part l mode s 

 Separate parameters for participation an

Separate parameters for participation an

conditional quantity 

– How independent variables predict How independent variables predict

 participation in care quantity of cost conditional on participa

quantity of cost conditional on participa

– each parameter may have its policy 

relevance 

 Disadvantage: hard to predict confidenc

interval around predicted Y given X 
Alternate to two-part mo d

 OLS i h OLS with untransformed cost  OLS with log cost , using small pos

values in place of zero 

 Certain GLM models Certain GLM models 
T i op cs for today’s cours

 Wh at to do when there is 

heteroscedasticity? (GLM models) 

 What to do when there are many ze

values? (Two -part models) part models) 

 How to test differences in groups w

assumpti b di ib i ? ions about distribution? 

 How to determine which method is 
Non-paramet i r c stati sti cal t 

 Make no assumptions about distribu

variance 

 Wilcoxon rank-sum test  Assigns rank to every observation Assigns rank to every observation  Compares ranks of groups  Calculates the probability that the r

order occurred by chance alone order occurred by chance alone 
Exte s o o o e w

e

n

s

i

on t

o m

o

r

e than tw

groups 

 G i bl ith th t Group var

i

able with more than 

two 

mutually exclusive values 

 Kruskall Wallis test 

– is there an

y difference between an

y 

p

the mutuall

y exclusive 

groups? 

 If KW is si

gnificant,then a series of 

Wilcoxon tests allows comparison o

pairs of 

groups 
Li it f Limits of non-parameti t 

r c t

 It is too conservative It is too conservative 

– Compares ranks, not means – Ignores influence of outliers Ignores influence of outliers – E.g. all other ranks being equal, Wilcoxon 

give same result regardless of whether 

 T k d b i i $1 illi 

Top ranked observation is $1 million more cos

second observation, or 

 Top ranked observation just $1 more costly 

 Doesn’t allow for additional explanatory

variables 
T i op cs for today’s cours

 What to do when there is 

heteroscedasticity? (GLM models) 

 What to do when there are many ze

What to do when there are many ze

values? (Two-part models) 

 How to test differences in groups w

assumptions about distribution? (N

parametric statistical tests) 

 How to determine which method is 

How to determine which method is 
Whi h th d i b t? Whi

c

h meth

od 

is 

best?

 Fi

n

d predict

ive accuracy o

f mo

d

e

l 

s

 Estimate re

gressions with half the d

test their predictive accuracy on the half of the data half of the data 

 Find 

– Mean Absolute Error (MAE) – Root Mean S

quare Error 

(RMSE

) 
–



n

MAE Y Y

ˆ 1 



 

i

Y Yii

n

MAE

1

Mean Ab solute Error 

 For each observati on 

– find difference between observed and predicted co

take absolute value take absolute value 

– find the mean 

 Model with smallest value is best Model with smallest value is best 
  

n

RMSE Y Y 2)ˆ ( 1





i

Y Yii

n

RMSE

1

( )

R t M Root Mean Square Erro

 S th diff b t di t 

Square the differences between predicte

and observed, find their mean, find its square root 

 Best model has smallest value 
E l va uati ons of resid uals

 M id l ( di t d l b 

Mean residual (predicted less obser

or 

 Mean predicted ratio (ratio of predi

observed) 

– calculate separately for each decile of 

observed Y

– A good model should have equal resi

(or equal mean ratio) for all deciles 
F l t t f id l Formal 

tes

ts o

f residua

ls 

 V i ar

iant of H osmerLemes

how 

Test 

– F test of whether residuals in raw sca

each decile are significantly different 

 Pregibon Pregibon s ’ Link Test Link Test 

– Tests if linearity assumption was viol

 S M i B & M ll h 2005 See 

Mann

ing, 

Basu, & 

M

ull

a

hy, 2005
Questi ons?
R i f ti Rev

iew o

f presen

t

ation 

 C i diffi l d d i b

Cost 

is a difficu

lt 

depen

dent var

i

ab

– Skewed to the right by high outliers– May have many observations with ze

values 

– Cost is not-negative 
Wh t i d When cost 

is s

kewe

d

 OLS f o

f raw cost 

is prone to bias 

– Especially in small samples with influ

outliers 

– “A sin

gle case can have tremendous influe
Wh t i k d ( 

When cost 

is s

kewed 

(con



Log trans

forme

d cost 

– Log cost is more normally distributed 

raw cost 

– Lo

g cost can be estimated with OLS 
Wh t i k d ( 

When cost 

is s

kewed 

(con

 T fi o

n

d predi cte

d cost, must correct 

retransformation bias 

– Smearing estimator assumes errors ar

homoscedastic

– Biased if errors are heteroscedasctic 



“When she was good she was very ver

When she was good, she was very, ver

but when she was bad, she was horrid” 
–

When cost is skewed When cost is skewed

and errors are heteroscedas

 GLM i h l li k d i GLM with log link and gamma vari

– Considers heteroscedasctic errors – Not subject to retransformation bias – May not be very efficient May not be very efficient – Alternative specification 

 P i o sson instead of gamma variance func

 Square root instead of log link function
Wh t hen cost has many zero va

 Two part model 

– Logit or probit is the first part – Conditional cost regression is the sec

part 
Comparison without distributi

Comparison without distributi

assumptions



Non-parametr

ic tests can 

be use

ful

 Ma

y be too conservative 

 Don’t allow co-variates 
E l ti l Eva

luating mo

d

e

ls 

 M Ab E Mean Abso

lute 

Error 

 Root Mean S

quare Error 

 Other evaluations and tests of resid
N t l t Next lecture

Non-li d d i bl linear dependent variables

Ciaran Phibbs

May 30, 2012
Key sources on GLM

 MANNING, W. G. (1998) The logged dependent var

(1998) The logged dependent var

heteroscedasticity, and the retransformation problemEcon, 17, 283-95. 

 * MANNING W G & MULLAHY J (2001) E ti 

* MANNING, W. G. & MULLAHY, J. (2001) Estim

models: to transform or not to transform?, J Health E461-94. 

 * MANNING, W. G., BASU, A. & MULLAHY, J. (

Generalized modeling approaches to risk adjustment skewed outcomes data, J Health Econ, 24, 465-88. 

 BASU, A. & Rathouz P.J. (2005) Estimating margina

incremental effects on health outcomes using flexible variance function models variance function models, Biostatistics Biostatistics 6(1): 93 6(1): 93-109 

109, 
Key sources on two-part mo

 * MULLAHY J * MULLAHY, J. (1998) Much b 

h ado abou

reconsidering retransformation and the t

part mod l i h l h del in health econometrics, JH 

Econ, 17, 247-81 

 JONES, A. (2000) Health econometrics

Culyer, A. & Newhouse, J. y, ( Eds.)HandHealth Economics, pp. 265-344 (AmsteElsevier). 
R fe erences to worked exam

 FLEISHMAN, J. A., COHEN W COHEN, J. W., MANN

G. & KOSINSKI, M. (2006) Using the SF-12 

status measure to improve predictions of med

expenditures, Med Care, 44, I54-63. 

 MONTEZ-RATH, M., CHRISTIANSEN, C. 

ETTNER S L LOVELAND S & ROSEN ETTNER, S. L., LOVELAND, S. & ROSEN

(2006) Performance of statistical models to prmental health and substance abuse cost, BMC 

Res Methodol, 6, 53. 
R f t k l ( 

R

e

ferences 

to wor

k examp

les 

(c

 MORAN J L SOLOMON P J PEISACH 

MORAN, 

J. 

L., SOLOMON, 

P. 

J., PEISACH

& MARTIN, J. (2007) New models for old qu

generalized linear models for cost 

prediction, 

Clin Prac

t, 13, 381-9.

 DIER, P., YANEZ D., ASH, A., HORNBRO

LIN, 

D. 

Y . (1999) . M th d f l i h 

M

eth

o

ds 

for ana

lyz

ing 

h

care utilization and costs Ann Rev Public Hea

(1999)20:125-144 (

( Also 

gives accessible ove

of methods, but lacks information from more developments) 
Link to HERC Link to HERC Cybersemin

HSR&D study of worked exam

Performance of Statistical Models to PredPerformance of Statistical Models to Pred

Mental Health and Substance Abuse Co

Maria Montez -Rath, M.S. 11/8/2006 M.S. 11/8/2006 The audio:  http://vaww.hsrd.research.va.gov/for http://vaww.hsrd.research.va.gov/for_re

re

ers/cyber_seminars/HERC110806.asx 

The Power point slides:  http://vaww.hsrd.research.va.gov/for_re

ers/cyber_seminars/HERC110806.pdf 
B k oo chapters 

 MANNING W G (2006) Dealing ith MANNING, W. G. (2006) Dealing with 

skewed data on costs and expenditures, 

Jones A (Ed ) Jones, A. (Ed.) The Elgar Companion to The Elgar Companion to 

Health Economics, pp. 439-446 (ChelteUK, Edward El gar).