Mini Course on Structural Estimation ofStatic and Dynamic Games
Junichi Suzuki
University of Toronto

June 1st, 20091

Part II: Estimation of Single-Agent DynamicOptimization Problem

2Motivation
I

Most economic agents seem forward-looking (β 6= 0)

Households care current consumption as well as future
consumptions
I Firms make huge investment for developing newtechnology
II

Current decisions often a¤ect future state variables
If you spend too much today, you have to spend less inthe futureI If a drug company does not invest enough, it cannot
attain a pro…table pro…tI

3

Motivation
I

Static analysis is irrelevant when these two conditions
holdI

Need to pursue a dynamic analysis

IThis section looks at the models of single-agent dynamic
optimization problem (i.e., no strategic interaction withothers)

I

Dynamic programming (DP) is a standard tool to analyze
this framework (Stokey, Lucas and Prescott)4

GoalI

Overview the very basic of single-agent maximization
problem using the model of Rust (1989)

I

Present di¢ culty in estimating this model bystraightforward conventional methods

IShow how to apply a two-step method/BBL in thisframework

5

Example: Rust’s Engine Replacement ModelI

Harold Zurcher is a maintenance guy at a bus company inMadison, Wisconsin

I

He is responsible for deciding when each bus replaces itsengine

I

The chance of engine trouble increases as engines
accumulates mileage

I

Replacing engine incurs signi…cant expense

I

Replacing decisions need to take into account thetrade-o¤ between current expenditure and future trouble
6

Example: Rust’s Engine Replacement ModelI

Static model is NOT appropriate sinceII

Harold Zurcher should be forward-looking
Current replacement decision a¤ects future pro…tI

Need to construct a dynamic modelIConsider a simple version of his model

7

Model: Primitives
IConsider a particular bus

IObservable state variable: x

IUnobservable state variables: e = fe (0) , e (1)gI

Choice variable: ii=I

1 if replaced
0 otherwise

Notationx: accumulated mileage on the bus engine at the current
period since the last replacement
I e: cost factor observable to Zurcher but not to8econometricians
I i: replacement decisionI

Model: Period Pro…tI

Period pro…t functionu (x, i, θ 1 ) =I

[RC + c (0, θ 1 )] + e (1) if i = 1c (xt , θ 1 ) + e (0)if i = 0

Operating cost increases as mileage accumulates∂c (x, θ 1 )>0∂xI

NotationsII

RC : net replacement costc (0, θ 1 ): operating cost9Model: Transition

I

Additional mileage is a random draw from a certain
probability distribution:p x 0 jx, i, θ 3 =Ig (x 0g (x 0x, θ 3 ) if i = 00, θ 3 ) if i = 1

This function represents the uncertainty aboutI
I

when this bus is in operation
demand shock etc..
10Epsilon as a Structural Error TermI

In structural analysis, error terms e has a direct economic
interpretationIe re‡ects whatever relevant factors Zurcher observes but
econometricians do notI

Large values of e (0) may represent bus driver’s positive
report on this bus

I

Large values of e (1) may represent engine inventory11Conditional Independence Assumption (CI)ITo make estimation feasible, need to impose the following
restriction on the distribution of ep x 0 , e0 jx, e, i, , θ 2 , θ 3 = q e0 jx 0 , θ 2 p x 0 jx, i, θ 3

IRust calls this assumption Conditional Independence
Assumption (CI)IThis assumption implies
The value of x 0 is a su¢ cient statistic to characterize thedistribution of e0
I The value of e a¤ects the value of x 0 only throughinvestment iII

12Type I extreme valued i.i.d e and above transition
function satis…es this condition

Model: Value Function
IValue function calculates the sum of expected discounted
pro…ts when Zurcher makes the pro…t-maximizing
decision every period:
V (x, e, θ )= max [u (x, i, θ 1 ) + e (i, θ 2 )i 2f0,1g+β

Z Z

V x 0 , e0 , θ p x 0 jx, i, θ 3 dF e0 jx 0 , θ 2 d

where θ = (θ 1 , θ 2 , θ 3 )13

Model: Optimal Strategy

There exists an optimal policy σ (x, e, θ ) that maps the statevariable to f0, 1g
V (x, e, θ )= u (x, σ (x, e) , θ 1 ) + e (σ (x, e) , θ 2 )+β

Z Z

V x 0 , e0 , θ p x 0 jx, σ (x, e) , θ 3 dF e0 jx 0 , θ 2 dx 014

Model: Value Function

IOnce we know the optimal policy function σ (x, e, θ ), thechance of engine replacement at this period isPr ( i = 1j x ) = Pr (σ (x, e, θ ) = 1)R∞
exp u (x, 1, θ 1 ) + β 0 V (x 0 ) p (x 0 jx, f (x, θ ) , θ 3 ) dx 0
R∞=∑i 0 2f0,1g exp u (x, i 0 , θ 1 ) + β 0 V (x 0 ) p (x 0 jx, i 0 , θ 3 ) dx
15The Data
ISuppose we observe the maintenance record of M
di¤erent buses fxtm , itm gMm =1

I

For simplicity, assume all M buses are observationaly
equal except their mileages

I

Want to recover the parameter of θ 1 and θ 3 from the data

IStart with a brute-force method, nested-…xed point
algorithm
16

Step 1: Estimating the Transition Function
IDiscretize the space of x into several intervals

I

Assume x never decreases without replacing an engine

I

Assume an increase in x in one period is no more thantwo intervals

I

These two assumptions imply x 0I

Using the data, estimate
θ 30 = Pr (x 0θ 31 = Pr (x 0I

x 2 f0, 1, 2g

x = 0j i = 0)
x = 1i = 0)

17 to CI
This estimation can be done independently, thanks
assumption

Step 2: Evaluating the Likelihood
I

Let the value θ 1 as givenI

For a given set of parameters, …nd the optimal policy
f (x, e, θ ) and V (x, e)This step requires solving DP numericallyV (x, e) is often approximated by Chebyshev polynomials
I See Judd (1989) for its implementation
IIICalculate the likelihood of the observed event Lit (θ )ITake log and summing up
ln L (θ ) =∑ ∑ ln Lit
i=t∑ ∑[1 (iit = 0) ln Pr (iit = 0jx )it

18+1 (iit = 1) ln Pr (iit = 1jx )]Step 3: Maximizing the Likelihood
IFind a set of parameters that maximizes ln L (θ )IThis algorithm is straightforward but requires heavycomputation

I

Need to solve the dynamic programming for every set ofparameters evaluatedIEstimation can be very slow

IAgain, the two-step methods is very useful19

Estimating the Policy Function
IBoth BBL and NPL require estimating the reduced-form
policy function of Zurcher

I

Ideally, nonparametric methods are appealing

I

In practice, ‡exible logit are often usediI= α1 x + α2 x 2 + α3 x 3 + υ0 if i < 0i =1 if i0

Used to approximate his observed/optimal behavior
20

Estimation by BBL
IBBL estimates the structural parameters without solving
DP even onceIFor estimating the parameter, the following optimization
condition is exploited:V (x, σ)V (x, σ̃ ) for all xI

Generate fake policies fσ0 (x )g that are slightly di¤erent
from the observed one σ̂ (x )I

Find a set of parameters that let σ (x ) beat as many asσ̂ (x ) possible

IBBL use the data only to estimate policy function σ̂ (x )21

Estimation by BBLI

The basic idea is to transform a dynamic discrete choiceproblem to the conventional (static) discrete choiceproblemI

Approximate EV (x ) by implementing the forward
simulation

IFind a set of parameters that rationalizes the observed
policy22

Implementing BBL Step by StepI

1
Generate fake policies fσ̃m (x )gNI
m =1I

2
Pick several initial values fx0n gNIn =1

I

Calculate EV (x0 , σ; θ ) by forward simulation

I

Calculate EV (x0 , σ̃; θ ) by forward simulation

IFind θ that solves
NI 1 NI 2min ∑ ∑ (min fEV (x0n , σ; θ )θm =1 n =1

EV (x0n , σ̃m ; θ ) , 0g)223

Implementing Forward Simulation

IPick T so that βT is su¢ ciently smallI

By using p̂ (x 0 jx ), σ (x ) and F (e), simulate a stream of
his period pro…t for T periods

ItCalculate ∑Tt =1 β u (xt , σ (xt , et ) , θ 1 )IIterating this process
h many times, calculateitn
EV (x0 , σ; θ ) = E ∑T
βux,
σx,e,θ(())
ttt
1t =124

SummaryI

Go over a single-agent dynamic optimization problem byusing Rust (1989)

INested-…xed point is straightforward but its
computational burden can be prohibitiveITwo-step methods are very useful to obtain consistentestimates by maintaining computational burden topractical level25