BLS WORKING PAPERS

U.S. DEPARTMENT OF LABOR

Bureau of Labor Statistics

 

OFFICE OF PRICES AND LIVING

CONDITIONS

An Anticipative Feedback Solution for the Infinite-Horizon,

Linear-Quadratic, Dynamic, Stackelberg Game

Baoline Chen, Rutgers University

Peter A. Zadrozny, U.S. Bureau of Labor Statistics

Working Paper 350

November 2001

 

The views expressed are those of the authors and do not necessarily reflect the policies of the U.S. Bureau of Labor

Statistics or the views of other staff members.
AN ANTICIPATIVE FEEDBACK SOLUTION FOR THE INFINITE-HORIZON,

LINEAR-QUADRATIC, DYNAMIC, STACKELBERG GAME*

 

*

The paper represents the authors’ views and does not represent any

official positions of the Bureau of Labor Statistics. We thank the

editors, referees, Jacek Krawczyk, and Randal Verbrugge for comments.

Baoline Chen Peter A. Zadrozny

Department of Economics

Rutgers University

Camden, NJ 08102

e-mail: baoline@crab.rutgers.edu

Bureau of Labor Statistics

2 Massachusetts Ave., NE

Washington, DC 20212

e-mail: zadrozny_p@bls.gov

September 17, 2001

 JEL Classification: C61, C63, C73

Additional key words: noncooperative games, solving Riccati-type

nonlinear algebraic equations.

ABSTRACT

This paper derives and illustrates a new suboptimal-consistent feedback

solution for an infinite-horizon, linear-quadratic, dynamic, Stackelberg

game. This solution lies in the same solution space as the infinitehorizon, dynamic-programming, feedback solution but puts the leader in a

preferred equilibrium position. The idea comes from Kydland (1977) who

suggested deriving a consistent feedback solution for an infinitehorizon, linear-quadratic, dynamic, Stackelberg game by varying the

coefficients in the player’s linear constant-coefficient decision rules.

Here feedback is understood in the sense of setting a current control

vector as a function of a predetermined state vector. The proposed

solution is derived for discrete- and continuous-time games and is

called the anticipative feedback solution. The solution is illustrated

with a numerical example of a duopoly model.
1

1. Introduction.

Dynamic Stackelberg (or leader-follower) games are useful tools

for studying dynamic economic behavior in equilibrium settings in which

some player is dominant. Because of their tractability, infinitehorizon, linear-quadratic, dynamic, Stackelberg (LQDS) games have

received particular attention. LQDS games have been used to study noncompetitive behavior in specific markets and to evaluate and design

macroeconomic policies. For example, Sargent (1985) contains studies of

energy markets based on LQDS games; Kydland and Prescott (1977) and

Fischer (1980) studied optimal tax policy using DS games; Canzoneri and

Gray (1985), Miller and Salmon (1985), and Turnovsky, Basar, and d’Orey

(1988) studied international macroeconomic policy coordination using DS

games; section 4 illustrates the present anticipative feedback solution

in a LQDS game of a hypothetical industry. The anticipative feedback

solution could be applied to the LQ approximation of any dynamic

economic setting with a dominant agent.

Three decision spaces have been considered in dynamic games:

open-loop, feedback, and closed-loop. In open-loop decisions, players

set their control vectors as functions of time; in feedback decisions,

players set their control vectors as functions of the current (or most

recently determined or observed) state vector; in closed-loop

decisions, players set their control vectors as functions of the

history of the state vector, from the start of the game to the moment

of decision. For example, Hansen, Epple, and Roberds (1985) considered

open-loop solutions of discrete-time LQDS games, computed using Eulertype equations. Simaan and Cruz (1973) considered feedback solutions of

DS games, computed using backwards recursions of dynamic programming.

To emphasize the dynamic programming nature of these feedback

solutions, we refer to them as dynamic programming feedback (DPF)

solutions. Basar and Selbuz (1979), Basar and Olsder (1980), and

Tolwinski (1981) considered classes of closed-loop solutions for

discrete- and continuous-time, DS games, computed using non-standard

(non-DP) recursions and differential equations. See Basar and Olsder

(1995, ch. 7) for a comprehensive discussion of DS games.

A potential problem in DS games is that the solution which is

optimal for the leader at the beginning of the game is time

inconsistent. That is, it ceases to be optimal for the leader in
2

subsequent periods. Consequently, the leader has an incentive to

restart the game. In a rational-expectations setting, followers would

recognize continual restarts. Such a succession of restarted leaderoptimal solutions would be unsustainable and, hence, unappealing as a

solution concept. The time inconsistency problem in DS games was first

noted by Simaan and Cruz (1973), Kydland (1975, 1977), and Kydland and

Prescott (1977) for open-loop solutions of DS games. In response,

Simaan and Cruz (1973), Kydland (1975, 1977), and Kydland and Prescott

(1977) considered DPF solutions of DS games. DPF solutions are time

consistent by construction, but do not entirely solve the time

consistency problem because in them the leader is continually tempted

to switch to an optimal, open- or closed-loop, solution.

Basar and Selbuz (1979) and Basar and Olsder (1980) proposed

closed-loop solutions for discrete- and continuous-time, DS games. The

Basar-Selbuz-Olsder solutions require additional structural

restrictions, beyond the usual concavity (or convexity), playability,

and stability conditions (see section 2). However, whenever they are

applicable, the Basar-Selbuz-Olsder solutions are time consistent.

Nevertheless, even when applicable, the Basar-Selbuz-Olsder solutions

are not subgame perfect. Tolwinski (1981) proposed a more general

closed-loop solution for LQDS games (under weaker structural

restrictions) which is nearly subgame perfect: if the follower deviates

from the optimal solution path for some reason, the leader induces them

to return to it after one period. The common feature of these closedloop solutions is that they are incentive (or trigger) strategies in

which the leader induces the follower to be on the solution path. In

economics, the time consistency problem has similarly been addressed

using incentive strategies (Barro and Gordon, 1983; Backus and

Driffill, 1985; Rogoff, 1989).

The present paper introduces a new feedback solution for

infinite-horizon LQDS games, called the anticipative feedback (AF)

solution. The name is explained further in this section. Like the

infinite-horizon DPF solution, the AF solution lies in the space of

constant-coefficient, linear, feedback, decision rules and is subgame

perfect, hence, is time consistent. However, the AF solution puts the

leader closer to an open- or closed-loop optimal solution than the DPF

solution. Thus, in the AF solution, the leader less tempted to switch

to an optimal solution. The idea of the AF solution comes from Kydland
3

(1977, p.310), who suggested deriving a feedback solution for a

discrete-time, infinite-horizon, LQDS game by varying the coefficients

in players’ linear, constant-coefficient, decision rules. The AF

solution is derived for both discrete- and continuous-time versions of

the LQDS games. Compared with the open-loop, DPF, and closed-loop

solutions proposed by Hansen, Epple, and Roberds (1985), Simaan and

Cruz (1973), and Tolwinski (1981), the AF solution has the following

three merits:

1. The AF solution is in the same space as the infinite-horizon

DPF solution, namely, the space of linear, constant-coefficient

decision rules in which the current control vector feeds back only on

the current state vector. This solution space is a product space of

real-valued matrices of finite dimensions. As explained below, when

anticipative effects are suppressed, the AF solution reduces to the

infinite-horizon DPF solution. Compared to this DPF solution, the

leader is better off in the AF solution and is, therefore, less tempted

to switch to an optimal, time-inconsistent solution.

2. Like the DPF solution, the AF solution is subgame perfect by

construction, hence, is time consistent.

3. Although open- and closed-loop solutions generally are

preferred by the leader, their solution spaces of sequences of control

vectors or decision functions are much more complicated. In infinitehorizon games, the sequences are infinite. To be practical as policy

prescriptions, DS game solutions should involve simple and easily

understood decision rules. The lower-dimensional AF solution is simpler

and more easily understood.

Anticipative control is the leader’s ability to influence the

state vector’s evolution by accounting for the follower’s current

reactions to changes in the leader’s current and expected future

control settings. The effect, manifested in the dependence of the

optimal solution on the initial state vector (cf., Hansen, Epple, and

Roberds, 1985), causes DS games to be time inconsistent. The AF

solution is consistent by construction, through the assumption that

coefficient matrices of decision rules are independent of the initial

state vector. By virtue of the principle of optimality upon which it is

based, the DPF solution cannot account for anticipative control

effects.
4

In the present AF solution, anticipative control manifests itself

through the matrix Ψ, that measures how the follower’s optimal

valuation matrix W2 varies in response to variations in the leader’s

policy rule. Here W2 = lim

h→ ∞ W2(h) is the limit of the follower's optimal

valuation matrix as the planning horizon, h, goes to infinity. In the

DPF solution at backwards recursion h, by taking W2(h-1) as given, the

leader ignores anticipative control effects and, in effect, sets Ψ

identically equal to zero. In the AF solution, the leader sets Ψ so as

to induce the follower to make decisions which put the leader in a more

favorable solution. Thus, Ψ is an incentive tool of the leader so that,

like Basar-Selbuz-Olsder's and Tolwinski's solutions, the AF solution

is an incentive-based enhancement of the DPF solution.

The AF solution relates more directly to the following previous

work. Zadrozny (1988) reported the discrete-time AF solution in summary

form, without giving any derivations or applications. Medanic (1978)

derived a related continuous-time solution, using the maximum

principle. Whereas Medanic randomized the initial state vector, here,

as usual, it is taken as given. Otherwise, the AF solution appears not

to have been reported before.

This paper is organized as follows: For simplicity, only the twoplayer game is treated. The extension to n-player games is conceptually

straightforward but notationally tedious. Section 2 presents the

discrete-time game. Section 3 derives nonlinear algebraic (or

nonrecursive) Riccati-type solution equations for the discrete-time AF

game. Appendix B derives analogous continuous-time solution equations.

Section 4 describes a trust-region gradient method for solving the

discrete-time AF equations and presents illustrative, numerical, DPF

and AF, solutions of a duopoly model. Section 5 contains concluding

remarks.

2. The Discrete-Time Game.

Two players are indexed by an ordered pair (i,j) ∈ {(1,2),

(2,1)}, where i is the player on whom attention is focused and j is the

opponent. Player 1 is the leader.

Let y(t) be an n×1 vector of "outputs" generated by the ARMAX

process
5

(2.1) y(t) = A1y(t-1) + ... + Apy(t-p) + B0u(t) + ... + Bqu(t-q)

 + C0ε(t) + ... + Crε(t-r),

where p, q, and r are positive integers, u(t) is an m×1 vector of

players’ controls, and ε(t) is an n×1 vector of independent (whitenoise) disturbances with zero mean and constant covariance matrix, Σε,

i.e., ε(t) ∼ IID(0,Σε).

Let u(t) = (u1(t)T

, u2(t)T)T, where ui(t) is the mi×1 subvector

controlled by player i and superscript T denotes vector or matrix

transposition. Coefficients B0, ..., Bq are partitioned conformably.

There are no equality or inequality restrictions on u(t), so that it

ranges over an m-dimensional Euclidean space. By appropriately zeroing

out coefficients, a subvector of y(t) can be made exogenous to the

game. For example, if y(t) = (y1(t)T

, y2(t)T)T and conformable (2,1)

blocks in A1, ..., Cr are zero and Σε is block diagonal, then, y2(t) is

exogenous.

To write process (2.1) in state-space form, following Ansley and

Kohn (1983), first, let x(t) be the s×1 state vector x(t) = (x1(t)T

,

..., xν(t)T

)T where ν = max(p,q+1,r+1) and each x1(t) is n×1, so that s =

νn. Then, with x1(t) = y(t), equation (2.1) is equivalent to

(2.2) x(t) = Fx(t-1) + Gu(t) + Hε(t),

 F = 

























⋅ ⋅ ⋅ ⋅

Aν 0 . . . . 0 . . . . . . I . . . . . . 0 . 0 .

A1 I 0 . . . 0

, G = 

























ν −1

0

B

.

.

.

B

, H = 

























ν −1

0

C

.

.

.

C

,

where Ai = 0 for i > p, Bi = 0 for i > q, and Ci = 0 for i > r.

The state-space representation of process (2.1), thus, comprises

state equation (2.2) and observation equation y(t) = Mx(t), where M =

[I, 0, ..., 0] is an n×s selection matrix. Partition G = [G1, G2]

conformably with u(t) = (u1(t)T

, u2(t)T)T, so that Gu(t) = G1u1(t) +

G2u2(t).

In each period t, player i maximizes
6

(2.3) Vi(t,h) = Et-1∑ =

h

k 0 δk

πi(t+k)

with respect to linear feedback policies on ui(t), defined below, where

Et[⋅] = E[⋅|Ω(t)], Ω(t) = {y(t-k), u(t-k) | k ≥ 0}, δ is a real discount

factor satisfying 0 < δ < 1, and

(2.4) πi(t) = u(t)T

Riu(t) + 2u(t)T Si y(t-1) + y(t-1)T Qi y(t-1).

The matrices Ri, Si , and Qi determine players’ profits (or losses) for

different values of u(t) and y(t-1). For example, in the illustrative

example of section 4, Ri, Si , and Qi are derived from the output demand

curve and the production function and, therefore, depend on the

parameters of these structural components. Without loss of generality,

we assume that Ri and Qi are symmetric. Although we focus on h = ∞,

initially we assume h is finite. Allowing some element of y(t) to be

identically equal to one introduces a constant term into process (2.1)

and linear terms into objective (2.3).

Defining Si = Si M and Qi = Qi M, we write

(2.5) πi(t) = u(t)T

Riu(t) + 2u(t)TSix(t-1) + x(t-1)TQix(t-1).

In accordance with the partition of u(t) into ui(t) and uj(t), partition

Ri into ii Ri , ij Ri , and jj Ri , Si into 1 Si and j Si , and, hence, Si into i Si and

j Si , to obtain

(2.6) πi(t) = ui(t)T ii Ri ui(t) + 2ui(t)T ij Ri uj(t) + uj(t)T jj Ri uj(t)

 + 2ui(t)T i Si x(t-1) + + 2uj(t)T j Si x(t-1) + x(t-1)T

Qix(t-1),

for (i,j) = (1,2) and (2,1).

We assume that each player knows: (a) Ω (t-1) at the beginning of

period t; (b) the game’s structure, and (c) the game’s parameters,

i.e., the coefficients of process (2.2) and objective (2.3), for (i,j)
7

= (1,2) and (2,1). Each player, thus, has the same information about

exogenously given quantities. The information set is complete except

for knowledge of x(t), which might have to be inferred from Ω (t). Games

in which different players have different information sets are

substantially more complicated (Townsend, 1983) and are not considered

here.

We assume that the players follow constant linear feedback

policies ui(t+k) = Pix(t+k-1|t+k-1), k = 0, ..., h, where x(t+k-1|t+k-1)

= Et+k-1x(t+k-1). We make this assumption for reasons of simplicity and

understandability, as discussed in the introduction. In particular,

"constant" means that the policy coefficient matrices, Pi, are

nonstochastic functions of the parameters and are independent of the

initial state vector. We write the players’ policy rules jointly as

u(t+k) = Px(t+k-1|t+k-1), where P = [ T P1 , T P2 ]T

.

Because the game has a linear-quadratic structure and the players

have identical information sets, the principle of certainty equivalence

(also called the separation principle) applies (Astrom, 1970, pp. 278-

279). Certainty equivalence says that the equilibrium value of P is

independent of the probability distributions of ε(t) and v(t), and hence

can be computed independently of x(t+k-1|t+k-1). Because computation of

x(t+k-1|t+k-1) in the present case where both players have the same

information is a familiar Kalman filtering exercise (Anderson and

Moore, 1979, pp. 165-192), we focus on computing P and set Σε = 0, so

that x(t+k-1|t+k-1) = x(t+k-1).

Infinite-horizon dynamic games generally require the following

three types of assumptions: (i) second-order concavity assumptions to

ensure that players’ optimization problems have locally unique

solutions, (ii) playability assumptions (the term comes from Lukes and

Russell, 1971) to ensure that an equilibrium exists, i.e., that

players’ reaction functions "intersect;" and (iii) stability

assumptions to ensure that players’ objectives remain finite as h . → ∞

Following standard practice (Basar and Selbuz, 1979; Basar and

Olsder, 1980), we make a broad concavity assumption. The concavity

assumption also serves to maintain playability. The stability

assumption that we make is the familiar stabilizability condition in

linear optimal control theory.
8

Thus, to cover concavity and playability, we assume that (A) πi(t)

is concave in u(t) and the endogenous variables in y(t) and (B) πi(t) is

strictly concave in ui(t). Assumption (A) is equivalent to assuming that

the matrix which defines the purely endogenous part of πi(t) in terms of

u(t) and y(t) is negative semi-definite and assumption (B) is

equivalent to assuming that ii Ri is negative definite.

In the case of stability, first, Φ = F + GP is the closed-loop

matrix of the game. Second, we account for discounting by multiplying Φ

by δ . That is, we define the discounted closed-loop matrix Φ

~ = δ Φ

or, equivalently, Φ

~ = F

~ + G

~P, where F

~ = δ F and G

~ = δ G. Φ

~ is said

to be discrete-time stable if its eigenvalues are less than one in

modulus. A sufficient (but not always necessary) condition for Vi(t) =

lim

h→ ∞ Vi(t,h) to be finite is that Φ

~ is stable. We assume P is restricted

to values that imply that Φ

~ is stable. P’s that imply stable Φ

~ ’s are

themselves called stable. To ensure that the set of stable P’s is

nonempty, we assume that (C) the ordered pair [F

~ , G

~ ] is stabilizable.

Stabilizability ensures existence of stable, constant, linear, feedback

policies (Wonham, 1967).

Defining stabilizability is somewhat involved (e.g., Kwakernaak

and Sivan, 1972, pp. 53-65 and 459-462). However, let y(t) = (y1(t)T

,

y2(t)T

)T, where y1(t) is endogenous and y2(t) is exogenous. Then, first,

stabilizability implies that, abstracting from exogenous variables and

disturbances, for any initial value of y1(t), and for any target value

y1

* , there is a control sequence, {u*

(t+k)} N 1

k 0

−

= that takes y1 from the

initial value to y1

* in a finite number of periods N. Second,

stabilizability implies that the conditional mean of y2(t) is of

exponential order 1/ δ , i.e., ||Ety2(t+k)|| < c||y2(t)||/ δk , for k =

0, 1, ..., where c is a positive constant and ||⋅|| is a vector norm.

Because 0 < δ < 1, the second condition of stabilizability implies that

exogenous variables can be nonstationary within this limit.

Detectability, which is dual to stabilizability, is usually

assumed for the underlying optimal-control problem (i.e., cooperative

solution of the game) to help ensure that Φ

~ is stable (Kwakernaak and

Sivan, pp. 65-81, 247-283, 462-466, and 495-501). Because concavity
9

assumptions (A) and (B) and detectability may be insufficient to ensure

stability in the present game setting, detectability is not assumed

formally. Instead, we assume directly that P is stable.

A Stackelberg equilibrium can now be defined. In addition to what

each player has been assumed to know, player i has conjecture Ξij about

player j’s policy, Pj, and conjectural variation ∂Ξij/∂Pi about how Pj,

reacts to infinitesimal changes in Pi. Given this information,

conjectures, and conjectural variations, player i maximizes objective

(2.3) with respect to Pi subject to state equation (2.2). Let Pi =

Γi(Ξij, ∂Ξij/∂Pi) denote player i’s resulting optimal feedback matrix as a

function of Ξij and ∂Ξij/∂Pij and let Γ21 denote first-partial derivatives

of Γ2 with respect to the first argument Ξ12. Recall that player 1 is

the leader. A Stackelberg equilibrium occurs when: (I) P = [PT

1 , PT2 ]T

 is

such that the players optimize (P1 = Γ1 and P2 = Γ2), (II) players’

conjectures are confirmed (Ξ12 = P2 and Ξ21 = P1), (III) the leader’s

conjectural variations are confirmed (∂Ξ12/∂P1 = Γ21), and (IV) the

follower’s conjectural variations are null (∂Ξ21/∂P2 = 0). Section 3

derives computationally useful forms of these equilibrium conditions

for the discrete-time infinite-horizon linear-quadratic game. Appendix

B derives analogous continuous-time solution equations.

3. Derivation of Discrete-Time AF Solution Equations.

First, we state definitions and rules of matrix differentiation

and, then, use the rules to derive discrete-time AF solution equations.

3.1. Definitions and Rules of Matrix Differentiation.

First, motivated by Magnus and Neudecker (1988), we define matrix

derivatives in terms of matrix differentials. Then, we state two rules

of matrix differentiation, a product rule and a trace rule, that we use

to derive the AF solution equations.

Let A(θ) = {Aij(θ)} denote a real, differentiable, m×n, matrix

function of a real, p×1 vector θ = (θ1, ..., θp)T

, where m, n, and p are

any positive integers (previous uses of m, n, p, and q are temporarily

suspended). Vector θ could be the vectorization of a matrix with a total
10

of p elements. Let the m×n matrices ∂kA = {∂Aij/∂θk}, for k = 1, ..., p,

collect first-order derivatives of A(θ) in partial-derivative form. Let

dAij = ∑ =

p

k 1 (∂Aij/∂θk)dθk, where dθk is an infinitesimal variation in θk.

It suffices to consider dθ = ( 1

ˆθ , ..., pˆθ )T

dε, where θ

ˆ = ( 1ˆθ , ..., pˆθ )T

is a real vector of finite-valued elements and dε is a real,

infinitesimal, scalar variation. Then, dA = {dAij} is the m×m

differential form of first-order derivatives of A(θ).

Product Rule. Let A(θ) and B(θ) be real, m×n and n×q,

differentiable, matrix functions of the real p×1 vector θ, where m, n,

p, and q are any positive integers. Componentwise application of the

scalar product rule of differentiation yields the matrix product rule

of differentiation

(3.1) d(AB) = dA⋅B + A⋅dB

(Magnus and Neudecker, 1988, p. 148). For example, setting θ = P, we use

rule (3.1) to derive equation (3.5) from equation (3.4).

Trace Rule. Let A and B be real-valued, m×n and n×m, matrices, so

that dA⋅B is a meaningful matrix product, where dA is an infinitesimal

variation in A. dA may be expressed as Aˆ dε, where Aˆ is a finitevalued, nonzero, m×n matrix and dε is an infinitesimal, nonzero, scalar

variation. Let tr(⋅) denote the trace of a square matrix. The trace rule

is:

(3.2) If tr(dA⋅B) = 0, for all dA = Aˆ dε ∈ Rm×n

, then, B = 0.

To prove trace rule (3.2), choose some (i,j) ∈ {(1,1), ..., (m,n)}.

Consider Aˆ such that Aij ˆ ≠ 0, Akl ˆ = 0, for (k,l) ≠ (i,j), and remember

that dε ≠ 0. Then, tr(dA⋅B) = ∑ = ∑ =

n

1

m

k 1 l dAklBlk = Aij ˆ Bjidε = 0 implies Bji

= 0. Repeating this argument for all other (i,j) ∈ {(1,1), ..., (m,n)},

we complete the proof.

In subsection 3.2, we use rule (3.2) to convert first-order

conditions from the unsolvable form tr(dA⋅B) = 0 to the solvable form B

= 0. In these applications, dA represents all possible variations in a
11

policy rule coefficient matrix. For example, setting dA = di T Pi = d2 T P2 ,

we use rule (3.2) to convert equation (3.5) to (3.6). Because the

variations, dA ≠ 0, are used to determine first-order conditions of LQ

maximization problems, their Aˆ ’s conceptually assume any nonzero

values.

3.2. Derivation of Solution Equations.

We now derive the discrete-time AF solution equations. Because

u(t+k) = Px(t+k-1) and the disturbance ε(t) is being ignored, state

equation (2.2) implies x(t+k-1) = ~k Φ x(t-1), for k = 0, ..., h. Thus,

objective (2.3) implies Vi(t,h) = x(t-1)T

Wi(h)x(t-1), where Wi(h) is

generated recursively by

(3.3) Wi(k) = ~ T Φ Wi(k-1) Φ

~ + PT

RiP + PTSi + T Si P + Qi,

for k = 1, ..., h, and Wi(0) = PT

RiP + PTSi + T Si P + Qi. Because Φ

~ is

stable, in the limit as h → ∞, Vi(t, ∞ ) = x(t-1)T

Wix(t-1), where Wi =

lim

h→ ∞ Wi(h) satisfies

(3.4) Wi = ~ T Φ Wi Φ

~ + PT

RiP + PTSi + T Si P + Qi.

Let di denote the differential induced by infinitesimal variations

in Pi. Then, the immediate first-order necessary condition for

maximizing Vi(t,∞) with respect to player i’s policy rule Pi is

diVi(t,∞) = 0. Because x(t-1) is independent of Pi and can assume any

value, diVi(t,∞) = x(t-1)T

diWix(t-1) = 0 implies diWi = 0. To see what

diWi = 0 implies, we use product rule (3.1) to differentiate equation

(3.4) with respect to Pi, impose diWi = 0, and obtain

(3.5) di T Pi [ ii Ri Pi + ij Ri Pj + i Si + T Gi

~ Wi Φ

~ ]

 + di T Pj [ ij T (Ri ) Pi + jj Ri Pj + j Si + T Gj

~ Wi Φ

~ ] + ... = 0,
12

where (i,j) ∈ {(1,2), (2,1)}, the dots here and below denote asymmetric

terms repeated in transposed form, diPi denotes player i’s variation of

their policy rule coefficient matrix, and diPj denotes player i’s

conjectural variation about player j, i.e., player i’s assumption about

how Pj responds to infinitesimal variations in Pi.

Next, we specialize equation (3.5) as the first-order necessary

condition for player 2, the follower in the game. Being a Nash player,

player 2 has a null conjectural variation about the leader, i.e., d2P1 =

0. Thus, we set (i,j) = (2,1) in (3.5), impose d2P1 = 0, take the trace

of (3.5), use tr(A) = tr(AT

) and tr(AB) = tr(BA) to consolidate terms

(including those represented by repeated dots), divide by 2, apply

trace rule (3.2) with dA = d2P2, and obtain

(3.6) 22 R2 P2 + 21 R2 P1+ 2 S2 + T G2

~ W2 Φ

~ = 0.

To obtain the first-order condition of the leader, we express d1P2

in terms of d1P1. First, we use product rule (3.1) to differentiate

equation (3.6) with respect to P1 and obtain

(3.7) d1P2 = M1d1P1 + M2d1W2 Φ

~ ,

where M1 = -[ 22 R2 + T G2

~ W2 G2~ ]-1[ 21 R2 + T G2~ W2 G1~ ],

 M2 = -[ 22 R2 + T G2

~ W2 G2~ ]-1 T G2~ .

Concavity assumptions (A) and (B) stated in section 2 imply [ 22 R2 +

T G2

~ W2 G2~ ] is negative definite and, hence, is nonsingular.

Next, to express d1W2 in terms of d1P1, we differentiate equation

(3.4) for i = 2 with respect to P1, simplify the result using equation

(3.6) (an envelope theorem), and obtain

(3.8) d1W2 = d1 T P1 N12 + ~ T Φ d1W2 Φ

~ + ...,

where N12 = ( 21 R2 )T

P2 + 11 R2 P1 + 1 S2 + T G1

~ W2 Φ

~ .
13

Because Φ

~ is stable, equation (3.8) is equivalent to

(3.9) d1W2 = ∑ ∞

= Φ j 0

T j

) ~( [d1 T P1 N12 + T N12 d1P1] ~j Φ .

Next, we use equation (3.9) to eliminate d1W2 from equation (3.7),

use the result to eliminate d1P2 from equation (3.5), for (i,j) = (1,2),

and obtain

(3.10) d1 T P1 [N11 + T M1 N21] + ∑ ∞

=

+ Φ j 0

T j 1 ) ~

( [d1 T P1 N12 + T N12 d1P1] ~j Φ T M2 N21 + ... = 0,

where N11 = 11 R1 P1 + 12 R1 P2 + 1 S1 + T G1

~ W1 Φ

~ ,

 N21 = ( 12 R1 )T

P1 + 22 R1 P2 + 2 S1 + T G2

~ W1 Φ

~ .

Next, we take the trace of equation (3.10), use tr(A) = tr(AT

) and

tr(AB) = tr(BA) to consolidate terms, divide by 2, and obtain

(3.11) tr{d1 T P1 [N11 + T M1 N21 + N12∑ ∞

j=0 Φ

~ j

( Φ

~ T N21 M2 + T M2 N21 Φ~ T

)( Φ

~ T

)j]} = 0.

Because equation (3.11) is in the form of tr(dA⋅B) = 0, where dA = d1 T P1

can assume any n×m1 value, trace rule (3.2) implies

(3.12) N11 + T M1 N21 + N12Ψ = 0,

where Ψ = ∑ ∞

j=0 Φ

~ j

[ Φ

~ T N21 M2 + T M2 N21 Φ~ T

]( Φ

~ T

)j.

Because Φ

~ is a stable matrix,

(3.13) Ψ = Φ

~ Ψ Φ~ T

 + T M2 N21 Φ

~ T

 + Φ

~ T N21 M2.

The anticipative control effects manifest themselves through Ψ,

which measures how the follower’s valuation matrix, W2, varies in

response to variations in the leader’s policy rule. In the DPF solution
14

at backwards recursion h, by taking W2(h-1) as given, the leader ignores

anticipative control effects and, in effect, sets Ψ = 0. Thus, by

dropping equation (3.13) and setting Ψ = 0, the AF solution equations

reduce to the DPF solution equations. Furthermore, if we set M1 = 0 in

equation (3.12), so that it reduces to N11 = 0, the DPF solution

equations reduce to Nash equilibrium solution equations.

We have derived algebraic Riccati-type solution equations for the

anticipative feedback solution of the discrete-time, linear-quadratic,

infinite-horizon, Stackelberg game: equations (3.4), for i = 1 and 2,

(3.6), (3.12), and (3.13). Equations (3.4), for i = 1, (3.12), and

(3.13) are the leader’s complete first-order conditions and equations

(3.4), for i = 2, and (3.6) are the follower’s complete first-order

conditions. Let θ = ( T θ1 , T θ2 )T

, where θ1 = (vec(P1)T, vech(W1)T,

vech(Ψ)T

)T, θ2 = (vec(P2)T, vech(W2)T)T, vec(⋅) denotes the columnwise

vectorization of a matrix, and vech(⋅) denotes the columnwise

vectorization of the nonredundant lower half of a symmetric matrix.

Then, the AF solution equations comprise ms + (3/2)s(s+1) scalar-level

equations for determining the same number of elements of θ.

4. Numerical Solution of Discrete-Time AF Equations.

First, we describe a numerical trust-region method for solving

the AF equations and, then, illustrate the method with a duopoly model.

4.1. Numerical Solution Method.

The AF solution equations are nonlinear differentiable equations

in θ and, therefore, are solvable using gradient methods such as the

trust-region method (More′ et al., 1980). The trust-region method

requires an initial value of θ, θ0, which should be close to the AF

solution value and, therefore, should satisfy its regularity

conditions. Although the full AF regularity conditions are unknown, at

a minimum θ0 should imply that P is such that Φ

~ is stable (otherwise W1

and W2 are likely to be undefined) and that W1 and W2 are negative semidefinite in endogenous state variables (otherwise second-order

concavity conditions are violated).
15

A cautious approach in setting θ0 for the AF solution is to

compute a sequence of Pareto (or cooperative), Nash equilibrium (NE),

and DPF solutions. The Pareto solution is a convenient starting point

because its solution can be computed without initial values. The Pareto

solution solves the problem of maximizing the weighted average of the

players’ expected present values, V (t,∞) = θV1(t,∞) + (1-θ)V2(t,∞), for

some value of θ ∈ [0,1], with respect to the joint feedback matrix P,

subject to the state law of motion (2.2). Computing the Pareto solution

involves solving the standard discrete-time algebraic matrix Riccati

equation, which can be done accurately and quickly using a Schurdecomposition method (Laub, 1979). Under the concavity and

stabilizability conditions, the Pareto solution yields the desired

stable P and the negative semi-definite endogenous part of W = θW1 +

(1-θ)W2. We could use the Pareto solution as an initial value for

computing the AF solution. A more cautious approach computes successive

Pareto, NE, DPF, and AF solutions, using the Pareto, NE, and DPF

solutions as initial values for the NE, DPF, and AF solutions. The idea

here is that the Pareto, NE, DPF, and AF solutions should "line up" in

the solution space because the leading player has ever greater

dominance in the sequence of solutions. The last step of computing the

AF solution is greatly simplified by using the fact that, given the

first player’s solution values θ1, the second player solves a standard

Riccati equation.

4.2. Illustrative Numerical Solutions of a Duopoly Model.

The model is a modification of the model in Chen and Zadrozny

(2001). As before, subscripts i = 1 and 2 refer to the leading and

following players, respectively. The players are firms that produce q1(t)

and q2(t) amounts of a good. The demand for the good is given by

(4.1) pq(t) = -ηq(t) + d(t),

where q(t) = q1(t) + q2(t), η > 0 is a slope parameter, and d(t) is the

demand state generated by the AR(1) process

(4.2) d(t) = φdd(t-1) + ζd(t),
16

where |φd| < 1/ δ and the disturbance ζd(t) is distributed IID with zero

mean.

The firms use capital, ki(t), labor, li(t), and materials, mi(t), to

produce output and invest in capital. Investment, ni(t), has two stages:

purchasing capital goods and installing them. Installing capital is an

"output activity" because it uses resources that could otherwise be used

to produce output. The output activities are restricted according to the

production function

(4.3) h(qi(t), ni(t)) = g(ki(t), li(t), mi(t)),

where g(·) and h(·) are the constant elasticity functions,

(4.4) g(ki(t), li(t), mi(t)) = (α1ki(t)β

 + α2li(t)β + α3mi(t)β)1/β,

 h(qi(t), ni(t)) = (γ1qi(t)ρ

 + γ2ni(t)ρ)1/ρ,

where αi > 0, α1 + α2 + α3 = 1, β < 1, γi > 0, γ1 + γ2 = 1, and ρ > 1.

|β-1|-1 is the constant elasticity of substitution among inputs and

|ρ-1|-1 is the constant elasticity of transformation between output

activities. Including ni in h(⋅) is a parsimonious way of specifying

internal adjustment costs: for given input resources, ever more units of

output must be forgone as investment increases. Adjustment costs arise

only during the installation of capital goods. Mathematically, ρ > 1 is a

necessary and sufficient condition for the output transformation curves

to be concave. The transformation curves become more curved, and, hence,

adjustment costs increase as ρ increases. Similarly, β < 1 is a necessary

and sufficient condition for the input isoquants to be convex to the

origin. The isoquants become more curved and input substitutability

decreases as β decreases.

To obtain linear-quadratic optimization problems for the firms, we

describe the production function in terms of the quadratic approximation

of its dual variable production cost function (DVPCF). The variable

production costs are cqi(t) = pl(t)li(t) + pm(t)mi(t), where pl(t) and pm(t)

are the hiring and purchase prices of labor and materials. The labor and
17

materials costs are called variable because these inputs are free of

adjustment costs. The DVPCF is denoted by cq(wi(t)) and defined as

follows: given wi(t) = (wi1(t), ..., wi5(t))T = (qi(t), ni(t), ki(t), pl(t),

pm(t))T

, cq(wi(t)) = minimum of pl(t)li(t) + pm(t)mi(t) with respect to li(t)

and mi(t), subject to the production function (4.3)-(4.4).

The firms’ remaining costs are the purchase costs of capital goods

cni(t) = pn(t)ni(t), where pn(t) is the purchase price of capital goods.

Thus, the firms’ profits are πi(t) = rqi(t) – cqi(t) - cni(t), where rqi(t)

= pq(t)qi(t) = -ηq(t)qi(t) + d(t)qi(t) is sales revenue. The quadratic

approximation of cqi(t) is (1/2)wi(t)T

∇2cq(wi0)wi(t), where ∇2cq(wi0) is the

Hessian matrix of second-partial derivatives of cqi(t) evaluated at wi0 =

(1, 1, 1, α2, α3)T

, a value that results in the simplest expression for

∇2

cq(wi0). ∇2cq(wi0) is stated in appendix A in terms of the parameters of

the production function. For simplicity, we write ∇2

cq(wi0) as ∇2cq.

Therefore,

(4.5) πi(t) = -ηq(t)qi(t) + d(t)qi(t) – (1/2)wi(t)T

∇2cqwi(t) – pn(t)ni(t).

The input prices are generated by the AR(1) processes

(4.6) pn(t) = φpnpn(t-1) + ζpn(t),

 pl(t) = φplpl(t-1) + ζpl(t),

 pm(t) = φpmpm(t-1) + ζpm(t),

where the coefficients φpn, φpl, and φpm are less than 1/ δ in absolute

value and the disturbances ζpn, ζpl, and ζpm are distributed IID with zero

means.

Each firm's capital accumulates according to the law of motion

(4.7) ki(t) = φkki(t-1) + ni(t) + ζk(t),

where 0 < φk < 1 and the disturbance ζk is distributed IID with mean zero.

The model's structural components have thus been specified.
18

Next, we simplify the firms’ dynamic optimization problems by first

solving for li(t) and mi(t). We can do this because li(t) and mi(t) are not

control variables in the capital law of motion. Optimal values of li(t)

and mi(t), conditional on qi(t) and ni(t) being at their optimal values,

are obtained using Shepard’s lemma (an envelope theorem),

(4.8) li(t) = c41qi(t) + c42ni(t) + c43ki(t) + c44pl(t) + c45pm(t),

(4.9) mi(t) = c51qi(t) + c52ni(t) + c53ki(t) + c54pl(t) + c55pm(t),

where (c41, ..., c45) and (c51, ..., c55) are the 4th and 5th rows of ∇2cq.

Then, to state the firms’ remaining optimization problems in the

general notation of sections 2 and 3, we define the 4×1 control vector

u(t) = (u1(t), u2(t))T

 = (q1(t), n1(t), q2(t), n2(t))T and the 6×1 state

vector x(t) = (k1(t), k2(t), pi(t), pl(t), pm(t), d(t))T

, and then assemble

the dynamic equations (4.2), (4.6), and (4.7) as the state equation

(4.10) xt = Fxt-1 + 

G

x

0

04 4





 



 ut,

where F = diag[φk, φk, φpi, φpl, φpm, φd], G0 = 0100

0001





 



 , and 04×4 is the 4×4

zero matrix. The matrices Ri, Si, and Qi, which define the profit function

in the general notation, are stated in appendix A in terms of η and the

elements of ∇2

cq.

We computed examples of AF solutions for the model using a trustregion method described in section 4.1. The solutions were computed in

less than 5 seconds on a personal computer using a 150-megahertz

processor. A computed solution $

θ satisfies (3.4) for i = 1 and 2,

(3.6), (3.12), and (3.13) up to error matrices. The solution has kdigit precision if all elements of the error matrices are ≤ 10-k.

To compute the AF solution, first, we computed a Pareto solution

for equally weighted player’s objective functions, by solving a

standard algebraic Riccati equation using a Schur-decomposition method.

Second, we used the Pareto solution as an initial value for computing

the DPF solution using the trust-region method. Finally, we used the

DPF solution as an initial value for computing the AF solution using
19

the trust-region method. We skipped the NE solution between the Pareto

and DPF solutions. In all cases, ~

Φ was stable and W1 and W2 were

negative semi-definite in the two endogenous state variables, i.e.,

firms’ capital stocks.

Table 1 reports Pareto, DPF, and AF solutions for five different

pairs of production function elasticities. The remaining parameters are

fixed at the following values: δ = .935, η = .5, γ1 = γ2 = .5, α1 = α2 =

α3 = .333, and φk = φpi = φpl = φpm = .9. In the Pareto solutions, players’

objective functions are weighted equally, with θ = .5. The table reports

the firms’ optimized values, Vi(t,∞) = x(t-1)T

Wix(t-1), for x(t-1) = (1,

1, 1, 1, 1, 1)T

, in the different solutions.

Table 1: Pareto, DPF, and AF Solutions of the Duopoly Model

Solution V1 V2 V1 V2 V1 V2

(β, ρ) = (-5, 5)

(CES,CET)=(.17,.2)

(β, ρ) = (-5, 7.5)

(CES,CET)=(.17,.15)

(β, ρ) = (-5, 10)

(CES,CET)=(.17,.11)

Pareto .4292 .4292 .2611 .2611 .1417 .1417

AF .4215 .4107 .2568 .2530 .1383 .1360

DPF .4206 .4093 .2567 .2535 .1383 .1366

(β, ρ) = (-5, 5)

(CES,CET)=(.17,.2)

(β, ρ) = (-3, 5)

(CES,CET)=(.77,.2)

(β, ρ) = (-1, 5)

(CES,CET)=(1.1,.2)

Pareto .4292 .4292 2.6286 2.6286 5.0481 5.0481

AF .4215 .4107 2.6199 2.6031 5.0368 4.9990

DPF .4206 .4093 2.6188 2.6083 5.0345 5.0196

As expected, both players achieve the highest values in the

Pareto solutions and the leader achieves higher values in the AF

solutions than in the DPF solutions. At least in this model, for the

parameter values considered, the values differ slightly among the five

cases and between the players. As expected, the firms’ values decline
20

as β declines and ρ increases. This occurs because lower β’s and higher

ρ’s make the firms’ operations less flexible. First, lower β’s imply

lower input substitutability, which reduces firms’ flexibility in

changing input proportions when input prices change. Second, higher ρ’s

imply higher adjustment costs, which makes capital adjustments more

costly. Generally, both firms’ values increase as the leader shifts

from DPF to AF decision rules.

5. Conclusion.

We have derived algebraic solution equations for a new solution,

called the anticipative feedback (AF) solution, for discrete- and

continuous-time, linear-quadratic, infinite-horizon, Stackelberg, twoplayer, dynamic games. The AF solution puts the leading player in a

better position in comparison with the familiar dynamic programming

feedback (DPF) solution. The paper illustrates discrete-time AF

solutions for a duopoly model. The solutions are accurately and quickly

computed using a trust-region method. The illustrations show that the

leading firm indeed increases its value by switching from DPF to AF

decisions, but that the value increases are small, at least in the

duopoly model for the parameter values that are considered.

Appendix A: Coefficient Values of the Duopoly Model.

In this appendix, we state the Ri, Si, Qi matrices, for i = 1 and

2, that define the players’ objective functions in the illustrative

duopoly model. First, we state the elements of ∇2

cq, denoted cij, in

terms of the parameters of the production function. Then, we state the

Ri, Si, and Qi matrices in terms of the output-demand slope η and the

elements of ∇2

cq.

The nonredundant upper-triangular elements of ∇2

cq are

c11 = γ1(1-γ1)(ρ-1)

 + γ 2

1 α1(1-β)/(1-α1),

c12 = -γ1γ2(ρ-1) - 2

1γ γ2(1-β)/(1-α1),

c13 = -γ1α1(1-β)/(1-α1)],

c14 = c15 = γ1/(1-α1),

c22 = γ2(1-γ2)(ρ-1)
21

 + γ 2

2 α1(1-β)/(1-α1),

c23 = -γ2 α1(1-β)/(1-α1),

c24 = c25 = γ2/(1-α1),

c33 = α1(1-β)[1+α1(2-α1)/(1-α1)],

c34 = c35 = -α1/(1-α1),

c44 = -α3/α2(1-α1)(1-β),

c45 = 1/(1-α1)(1-β),

c55 = -α2/α3(1-α1)(1-β).

Let Ri,jk denote the (j,k) element of Ri and similarly for Si and

Qi. For compactness, we state only nonzero elements. The nonzero,

nonredundant, upper-triangular elements of R1 and R2 are

R1,11 = -η - c11,

R1,12 = -c12,

R1,13 = -η,

R1,22 = -c22,

R2,13 = -η,

R2,33 = -η - c11,

R2,34 = -c12,

R2,44 = -c22.

The nonzero elements of S1 and S2 are

S1,11 = -c13,

S1,14 = -c14,

S1,15 = -c15,

S1,21 = -c23,

S1,23 = -1,

S1,24 = -c24,

S1,25 = -c25,

S2,32 = -c13,

S2,34 = -c14,

S2,35 = -c15,

S2,36 = 1,

S2,42 = -c23,

S2,43 = -1,

S2,44 = -c24,

S2,45 = -c25.

The nonzero, nonredundant, upper-triangular elements of Q1 and Q2

are

Q1,11 = -c33,

Q1,14 = -c34,

Q1,15 = -c35,

Q1,44 = -c44,

Q1,45 = -c45,

Q1,55 = -c55,
22

Q2,22 = -c33,

Q2,24 = -c34,

Q2,25 = -c35,

Q2,44 = -c44,

Q2,45 = -c45,

Q2,55 = -c55.

Appendix B: Continuous-Time Solution Equations.

In continuous time, analogously to the ARMAX process (2.1), y(t)

is a n×1 vector generated by the stochastic differential equation

(B.1) DP

y(t) = A1DP-1y(t) + ... + Apy(t) + B0Dqu(t) + ... + Bqu(t)

 + C0Dr

ε(t) + ... + Crε(t),

where Dj

 denotes the j-th mean-squared time derivative, p, q, and r are

positive integers, u(t) = (u1(t)T

, u2(t)T)T is an m×1 = (m1 + m2)×1 vector

of players’ controls, and ε(t) is an n×1 vector of continuous-time

white-noise disturbances.

To say that ε(t) is continuous-time white noise means that it has

independent increments, zero mean, and autocovariance function Σε∆(t2-

t1), where Σε is an n×n, symmetric, positive definite, intensity matrix,

∆(t2-t1) is the Dirac delta function, and t2 ≥ t1 are points in time.

Strictly, ε(t), its derivatives, and (B.1) are not well-defined, but for

p ≥ 1 + max(q,r), which is assumed, there is a well-defined (meansquared) stochastic integral equation corresponding to (B.1) that gives

it a rigorous foundation (Astrom 1970, pp. 13-90).

To put (B.1) in state-space form, we define a state vector x(t)

exactly as in section 2 and, again, let x1(t) = y(t). Then, by a

recursive-substitution argument similar to that yielding (2), we obtain

(B.2) Dx(t) = Fx(t) + Gu(t) + Hε(t),

where coefficient matrices F, G, and H are exactly as in the discretetime state equation (2.2). Thus, the state-space representation of

process (B.1) comprises state equation (B.2) and the observation
23

equation y(t)= Mx(t), where M = [I, 0, ..., 0]. As before, u(t), G, and

P partition as u(t) = (u1(t)T

, u2(t)T)T, G = [G1, G2], and P = [ T P1 , T P2 ]T.

Constant linear feedback policies are ui(t) = Pix(t|t), where

x(t|t) = Et[x(t)] = E{x(t)|Ω(t)] and Ω(t) = {y(τ), u(τ)| τ ≤ t}. As in

the discrete-time case, we suppress disturbances so that x(t|t) = x(t).

At each moment t, given information Ω(t), assumptions about

opponents’ positions and reactions, and state equation (B.2), player i

maximizes

(B.3) Vi(t,h) = Et ∫

+

τ =

t h

t exp[-δ(τ-t)]πi(τ)dτ

with respect to Pi, subject to ui(t) = Pix(t) and state equation (B.2),

where δ > 0 is a real discount factor and πi(τ) = u(τ)T

Riu(τ)+ 2u(τ)T Si (τ)

+ y(τ)T Qi y(τ). As before, we assume that Qi is symmetric, h is initially

taken to be finite, Si = Si M and Qi = MT Qi M, and we consider πi(τ) in the

more detailed representation analogous to equation (2.4).

The discrete-time assumptions on parameters are retained, except

for technical changes necessitated by the switch to continuous time.

That is, as before, we assume (A) πi(τ) is concave in u(t) and in the

endogenous variables in y(t), (B) πi(t) is strictly concave in ui(t),

and (C) after incorporating discounting, state equation (B.2) is

stabilizable. Assumptions (A) and (B) impose the same restrictions as

in the discrete-time case. However, whereas the assumption that ii Ri is

negative definite is generally not necessary in discrete time, it is

necessary in continuous time.

In continuous time, the discounting and stability conditions are

different. Φ = F + GP is still the closed-loop matrix, but in

continuous time the discounted closed-loop matrix is Φ

~ = Φ - (δ/2)In,

where In is the n×n identity matrix, or, equivalently, Φ

~ = F

~ + GP,

where F

~ = F - (δ/2)In. Φ

~ is continuous-time stable if the real parts of

its eigenvalues are negative. As before, P is restricted to values that

imply that Φ

~ is stable and the stabilizability of [F

~ , G

~] ensures that

the set of stable P’s is nonempty.
24

Constant linear feedback policies imply that Dx(τ) = Φ

~ x(τ),

hence, that x(τ) = exp[Φ

~(τ-t)]x(t), for τ ≥ t (Graham, 1981, pp. 108-

110), where exp[Φ

~(τ-t)] = I + Φ (τ-t) + (1/2!) ~2 Φ (τ-t)2

 + ... is the

matrix exponential. Thus, Vi(t,h) = x(t)T

Wi(t,h)x(t), where

(B.4) Vi(t,h) = ∫

+

τ =

t h

t exp[Φ

~ T

(τ-t)][PTRiP + PTSi + T Si P + Qi]exp[ Φ

~ (τ-t)]dτ.

We premultiply equation (B.4) by Φ

~ T

, postmultiply it by Φ

~ , add the

products together, and integrate the sum by parts. Because Φ

~ is a

stability matrix, in the limit as h→∞, we obtain

(B.5) Φ

~ T

Wi + Wi Φ

~ + PT

RiP + PTSi + T Si P + Qi = 0,

where Wi = lim

h → ∞ Wi(t,h).

The immediate first-order necessary condition for maximizing

Vi(t, ∞ ) with respect to Pi is diVi(t, ∞ ) = 0. Because x(t) is given

independently of Pi and can assume any value, diVi(t, ∞ ) = x(t)T

diWix(t) =

0 implies diWi = 0. To see what diWi = 0 implies, we use product rule

(3.1) to differentiate equation (B.5) with respect to Pi, impose diVi =

0, and obtain

(B.6) di T Pi [R ii

i Pi + ij Ri Pj + i Si + T Gi Wi]

 + di T Pj [( ij Ri )T

Pi + R jj

i Pj + j Si + T Gj Wi] + ... = 0,

where (i,j) ∈ {(1,2,), (2,1)}, and the dots denote asymmetric terms

repeated in transposed form.

To specialize equation (B.6) as the first-order necessary

condition for the follower, we set (i,j) = (2,1), impose d2P1 = 0, take

the trace, use tr(B) = tr(BT

) and tr(AB)= tr(BA) to consolidate terms,

divide by 2, apply trace rule (3.2) with dA = d2P2, and obtain

(B.7) 22 R2 P2 + 21 R2 P1 + 2 S2 + T G 2 W2 = 0.
25

To obtain the first-order necessary condition of the leader, we

use product rule (3.1) to differentiate equation (B.7) with respect to

P1 and obtain

(B.8) d1P2 = M1d1P1 + M2d1W2,

where M1 = -( 22 R2 )-1 21 R2 ,

 M2 = -( 22 R2 )-1 T G 2 .

To express d1W2 in terms of d1P1, we differentiate equation (B.5),

for i = 2, with respect to P1, simplify the result using equation (B.7),

and obtain

(B.9) Φ

~ T

d1W2 + d1W2 Φ

~ + d1 PT

1 N12 + T N12 d1P1 = 0,

where N12 = ( 21 R2 )T

P2 + ij R2 P1 + 1 S2 + T G 1 W2.

Because Φ

~ is a stable matrix, equation (B.9) is equivalent to

(B.10) d1W2 = ∫

∞

τ =0 exp(Φ

~ T

τ)[d1 T P1 N12 + T N12 d1P1]exp( Φ

~ τ)dτ.

We use equation (B.10) to eliminate d1W2 from equation (B.8), then,

use the result to eliminate d1P2 from equation (B.6), for (i,j) = (1,2),

and obtain

(B.11) d1 T P1 [N11 + T M1 N21]

 + ∫

∞

τ =0 exp(Φ

~ Tτ)[d1 T P1 N1 + T N12 d1P1]exp( Φ~ τ) T M2 N21dτ = 0,

where N11 = ij Ri P1 + 12 R 1 P2 + 1 S1 + T G 1 W1,

 N21 = ( 12 R 1 )T

P1 + 22 R 1 P2 + 2 S1 + T G 2 W1.
26

Next, we take the trace of equation (B.11), use tr(B) = tr(BT

) and

tr(AB) = tr(BA) to consolidate terms, divide by 2, and obtain

(B.12) tr{d1 T P1 [N11 + T M1 N21

 + N12 ∫

∞

τ =0 exp(Φ

~ τ)[ T M2 N21 + N12M2]exp( Φ~ T

τ)dτ]} = 0.

Because equation (B.12) is in the form of tr(dA⋅B) = 0, where dA = d T P1

can assume any n×m1 value, trace rule (3.2) implies

(B.13) N11 + T M1 N21 + N12Ψ = 0,

where Ψ = ∫

∞

τ =0 exp(Φ

~ τ)[ T M2 N21 + T N21 M2]exp(Φ~ T

τ)dτ,

or, equivalently,

(B.14) Φ

~ Ψ + ΨΦ~ T

 + T M2 N21 + T N21 M2 = 0.

Thus, we have derived algebraic Riccati-type solution equations

for the anticipative feedback solution of the continuous-time, linearquadratic, infinite-horizon, Stackelberg, dynamic game: equations

(B.5), for i = 1 and 2, (B.7), (B.13), and (B.14). Equations (B.5), for

i = 1, (B.13), and (B.14) are the leader’s complete first-order

conditions and equations (B.5), for i = 2, and (B.7) are the follower’s

complete first-order conditions.

As in the discrete-time solution, if we set Ψ ≡ 0 and drop

equation (B.14), the continuous-time Stackelberg AF solution reduces to

the Stackelberg nonanticipative solution, which is analogous to the

discrete-time DPF solution. In addition, if we set M1 ≡ 0, the

nonanticipative solution reduces to the Nash equilibrium solution.
27

REFERENCES

Anderson, B.D.O. and J.B. Moore (1979), Optimal Filtering, Englewood

Cliffs, NJ: Prentice-Hall.

Ansley, C.F. and R. Kohn (1983), "Exact Likelihood of Vector

Autoregressive Moving-Average Process with Missing or Aggregated Data,"

Biometrika 70: 275-278.

Astrom, K.J. (1970), Introduction to Stochastic Control Theory,

Orlando, FL: Academic Press.

Backus, D. and J. Driffill (1985), "Rational Expectations and Policy

Credibility Following a Change in Regime," Review of Economic Studies

52: 211-221.

Barro, R.J. and D.B. Gordon (1983), "Rules, Discretion and Reputation

in a Model of Monetary Policy," Journal of Monetary Economics 12: 101-

121.

Basar, T. and G.J. Olsder (1980), "Team-Optimal Closed-Loop Stackelberg

Strategies in Hierarchical Control Problems," Automatica 16: 409-414.

Basar, T. and G.J. Olsder (1995), Dynamic Noncooperative Game Theory,

second edition, London, UK: Academic Press.

Basar, T. and H. Selbuz (1979), "Closed-Loop Stackelberg Strategies

with Applications in the Optimal Control of Multilevel Systems," IEEE

Transactions on Automatic Control AC-24: 166-178.

Canzoneri, M. and J. Gray (1985), "Monetary Policy Games and the

Consequences of Noncooperative Behavior," International Economic Review

26: 547-564.

Chen, B. and P.A. Zadrozny (2001), "Estimated U.S. Manufacturing

Capital and Technology Based on an Estimated Dynamic Economic Model,"

typescript.

Fischer, S. (1980), "Dynamic Inconsistency, Cooperation, and the

Benevolent Dissembling Government," Journal of Economic Dynamics and

Control 2: 93-107.

Graham, A. (1981), Kronecker Products and Matrix Calculus with

Applications, Chichester, UK: Ellis Horwood.

Hansen, L.P., D. Epple, and W. Roberds (1985), "Linear-Quadratic

Duopoly Models of Resource Depletion," pp. 101-142 in Energy,

Foresight, and Strategy, T.J. Sargent (ed.), Washington, DC: Resources

for the Future.

Kwakernaak, H. and R. Sivan (1972), Linear Optimal Control Systems, New

York, NY: Wiley-Interscience.

Kydland, F. (1975), "Noncooperative and Dominant Player Solutions in

Discrete Dynamic Games," International Economic Review 16: 321-335.
28

Kydland, F. (1977), "Equilibrium Solutions in Dynamic Dominant-Player

Models," Journal of Economic Theory 15: 307-324.

Kydland, F. and E.C. Prescott (1977), "Rules Rather than Discretion:

The Inconsistency of Optimal Plans," Journal of Political Economy 85:

473-491.

Laub, A.J. (1979), "A Schur Method for Solving Algebraic Riccati

Equations," IEEE Transactions on Automatic Control 24: 913-921.

Lukes, D.L. and D.L. Russell (1971), "A Global Theory for LinearQuadratic Differential Games," Journal of Mathematical Analysis and

Applications 33: 96-123.

Magnus, J.R. and H. Neudecker (1988), Matrix Differential Calculus with

Applications in Statistics and Econometrics, New York, NY: John Wiley &

Sons.

Medanic, J. (1978), "Closed-Loop Stackelberg Strategies in LinearQuadratic Problems," IEEE Transactions on Automatic Control AC-23: 632-

637.

Miller, M. and M. Salmon (1985), "Dynamic Games and the Time

Inconsistency of Optimal Policy in Open Economies," Economic Journal 95:

124-137

More′ , J.J., B.S. Garbow, and K.E. Hillstrom (1980), "User Guide for

MINPACK-1," Report ANL-80-74, Argonne National Laboratory, Argonne, IL.

Rogoff, K. (1989), "Reputation, Coordination, and Monetary Policy," pp.

236-264 in Modern Business Cycle Theory, R.J. Barro (ed.), Cambridge,

MA: Harvard University Press.

Sargent T.J. (ed.) (1985), Energy, Foresight, and Strategy, Washington,

DC: Resources for the Future.

Simaan, M. and J.B. Cruz, Jr. (1973), "Additional Aspects of the

Stackelberg Strategy in Nonzero-Sum Games," Journal of Optimization

Theory and Applications 11: 613-626.

Townsend, R.M. (1983), "Forecasting the Forecasts of Others," Journal

of Political Economy 91: 546-588.

Tolwinski, B. (1981), "Closed-Loop Stackelberg Solution to a Multistage

Linear-Quadratic Game," Journal of Optimization Theory and Applications

34: 485-501.

Turnovsky, S.J., T. Basar, and V. d’Orey (1988), "Dynamic Strategic

Monetary Policies and Coordination in Interdependent Economies,"

American Economic Review 78: 341-361.

Wonham, W.M. (1967), "On Pole Assignment in Multi-Input Controllable

Linear Systems," IEEE Transactions on Automatic Control AC-12: 660-665.
29

Zadrozny, P.A. (1988), "A Consistent Closed-Loop Solution for InfiniteHorizon Linear-Quadratic Dynamic Stackelberg Games," Journal of

Economic Dynamics and Control 12: 155-159.