Office of Tax Analysis 

U.S. Treasury Department 

Washington, D.C. 20220 

Issued: September 1976 

ESTIMATION OF A SIMULTANEOUS SYSTEM OF 

EQUATIONS WHEN THE SAMPLE IS UNDERSIZED 

K.R. Kadiyala 

Purdue University 

James R. Nunns 

U.S. Treasury Department 

OTA Paper 14 August 1976 
TABLE OF CONTENTS 

page 

I. Introduction ..................... I1. The Problem ..................... I11. Alternative Approaches to the Problem . . 

IV. A Proposed Class of Estimators ............ V . Estimation of Klein's Model I - An Illustration . . 
I. INTRODUCTION 

In most large and many medium-sized econometric models, the number 

of predetermined variables exceeds the number of observations on each 

variable. Estimation procedures such as two-stage least squares and 

other k-class (k>O) procedures, as well as three-stage least squares and 

certain other full-information procedures are therefore inapplicable. 

In this paper, a class of modified two-stage least squares estimators 

is derived which exhibits several desirable properties in comparison to 

alternative estimators which have been proposed for models with undersized samples. 

11. THE PROBLEM 

The jth structural equation of a linear simultaneous equation system 

may be written as: 

+XB yj 'YYjj jj +Ej (1) 

or more conveniently as: 

=Z6 + E

'j jj j 

where Z 

j 

= (Yj Xj)* 6; = (yi 6i), yj is the nxl vector of observations 

on the jth dependent variable, Y

j 

is the nxL

j 

matrix of observations on the 
-2-

jointly dependent variables which are explanatory in the jth equation, 

X

d is the nxKj 

matrix of observations on the predetermined variables 

entering the jth equation, yj and 6 j 

are parameter vectors to be estimated, 

and 5j is an nxl vector of disturbances. The system contains L jointly 

dependent variables, and K (>K.) predetermined variables; X is the nxK

J 

matrix of observations on all predetermined variables in the system. It 

is assumed throughout the paper that the predetermined variables are 

"fixed", 5

j 

has a zero mean and covariance matrix ajjI (0 < ajj e EO), 

the jth equation is identified, and the rank of X'Zj is L +K which

jj 

requires min(K, n) 2 Lj+Kj. 

Multiplying equation (2) by X' gives: 

X'Yj = X'Z.6 

~j 

+ X'Sj . (3) 

The transformed disturbance vector X'S.: has mean zero and covariance 

J 

matrix u. .X'X. Assuming X has rank K (which requires nlK), the two-stage JJ 

least squares estimator of 6 j, tj, is derived from (3) by applying 

Aitken's theorem, giving: 

where E = X(X'X)-lX'. When the rank of X is less than K, X'X is singular 

and the two-stage least squares estimator (as well as all other estimators 

which depend on the inverse of X'X) fails to exist. The rank of X is 

always less than K when n<K, i,e., when the sample is undersized. 
-3-

111. ALTERNATIVE APPROACHES TO THE PROBLEM 

A number of estimation procedures have been proposed that do not 

rely on the inverse of X'X, and are therefore at least potentially 

applicable when the sample is undersized. These procedures will only be 

discussed briefly here; more extensive discussions may be found in 

Theil [9] and Dutta and Lyttkens [l]. Our primary interest is in those 

procedures which are truly "limited-information'' - requiring only 

specification of the jth equation and the list of predetermined variables 

occurring in the system. Other procedures, while usually more efficient, 

have the undesirable property of requiring a more detailed knowledge of 

the entire system. Estimation of the jth equation is therefore sensitive 

to misspecification in the remainder of the system. 

Among the limited information procedures, the following three are 

widely known and illustrate the difficulties of estimation when the 

sample is undersized. 

1. Kloek and Mennes [3] suggested replacing X with T = (Xj d P) where P 

is a matrix of principal components of some linear combination of some or 

all of the columns of X. This leads to the estimator: 

A major disadvantage of this procedure is that the size of P, the columns 

of X from which the full set of principal components is derived, and the 

* normalization chosen are all arbitrary. Thus, 6 j may be highly sensitive 
-4-

to the P matrix used in its estimation, A lesser disadvantage is that 

the procedure requires considerably greater computational effort than 

the two-stage least squares procedure. Further, as is the case with all 

other limited-information procedures which we are aware of, short of 

specifying and estimating the entire system no estimates of the reduced 

form coefficients is possible using this procedure. Thus, projections of 

the dependent variables included in Y cannot be obtained simply on the 

j 

basis of projections of the predetermined variables. 

From Takeuchi's results [7] it is known that in certain cases, 

if P is of rank r then the even moments of order less than r - L. + 3 J

* of 6j exist, but little else is known about its small sample properties, 

It has the desirable large sample property of consistency. 

2. Applying a generalization of Aitken's,theorem to equation (3), 

Swamy and Holmes [6] and Fischer and Wadycki [2] obtain the estimator: 

where E- = X(X'X)-X' and (X'X)' is any (weak) generalized inverse of X'X. 

Normally when the sample is undersized, the rank of X is n in which case 

-1 X(X'X)-X = I so that 6 - = (Z'Z ) Z'y , the ordinary least squares j jj jj 

estimator for 6 3' Since 6 -

j 

becomes the two-stage least squares estimator 

when n>K - (assuming that the rank of X is then K), it does not share the 

property of inconsistency with the ordinary least squares estimator. 

Consistency, however, is a large sample property; it is the small sample 

properties of 6 - which are relevant in the present context. Mariano [4]

j 
-5-

has shown that in the general case, the even moments of order less than 

n-(IC j

+L.)+l of the ordinary least squares estimator exist. However, J 

Sawa [SI has shown that, for an equation with L

5"1, the ordinary least 

squares estimator has a lower mean square error than other k-class 

(0 <k 51) estimators only in rather specialized circumstances. Reduced 

form parameters cannot be directly computed following this procedure. An 

advantage of the procedure, however, is its computational simplicity. 

3. Partitioning X as (Xj 

! ij), where zj is the m(K-K j ) matrix of observations on the predetermined variables excluded from the j th equation, 

equation (3) may be written: 

Theil's D -class estimator (a") is based on constrained estimation from 

j j 

the second subset of (3'), using some positive definite matrix Q

jjDjin 

place of ujjzigj which is singular when n< (K-K5).; see Theil [9J. The 

constraint, from the systematic part of the first subset of (3'), is 

* * 

J j j' Defining C j 

= f D -'?; , d 

j = XIZ d is obtained by solving: jj 

where X is a vector of Lagrangian multipliers. In practice, Theil 

j 

suggests that D be diagonal, with diagonal elements taken from the 

j 

diagonal of z'z . There are several disadvantages to the D

3- classjj 
- 

-6-

* estimators. The choice of D .

J 

is arbitrary and d

j 

is sensitive to this 

choice; Theil's suggested choice disposes of a fair amount of information 

contained in X'X, The reduced form is explicitly bypassed, The computational burden is roughly the same as for the two-stage least squares 

estimator. The small sample properties of djf are unknown; Theil shows J 

that it is a consistent estimator, but since its asymptotic covariance 

matrix differs from that of two-stage least squares it is not efficient 

(in the limited information sense). 

IV. A PROPOSED CLASS OF ESTIMATORS 

In partitioned form, we have: 

x'x = 

Since XIX is positive definite by assumption, if we "disturb" g!Z Jj Jj 

slightly by adding to it any (symmetric) positive definite matrix A

3 

(so that in Theil's notation, Dj = z!z. + A.) , a comparison of the

3J J 

quadratic forms associated with X'X and 

vj - (9) 

shows that V. is positive definite. J 

The partitioned.inverse of V

j may be written: 

. v-1 = 

(X!X Ji3j .Jj - x*a.x)-~x!E.D? Jj - x!c.x )-I I -(x:x jJj JJJ 

j -(Dj - f!E.Z.)'lF!X.(X'X )-1 : (Dj - ?!E g.)"

JJJ JJ jj JjJ # 
-7-

where E

j = X jjj (X'X )-4' j 3 f and (retaining Theil's notation) Cj = ZaD-'Ztj * 

Using (lo), we define: 

Nj = XVjmlX' = [X (X!X -X'C X )-'Xj][I-C j]+ [X.(D.-?.E.z.)-l?j] J J JJJ [I-E33. (11) Jj jjj 

It follows immediately from equation (11) that N

j 

is symmetric and that 

NjX

j = Xj' jj = X'j . The estimator for 6j based on N

j therefore X'N

-

(6j) ,'isobtained by simply replacing E with N

j 

in equation (4), giving: 

- The estimator 63 has several desirable properties. It is a true 

limited-information estimator. In terms of computational difficulty, it 

is equivalent to two-stage least squares. Under the usual assumptions 

(see, for example, Theil [8, Chapter lo]), it is also asymptotically 

equivalent to two-stage least squares, assuming plim n 

-1 = 0 since then

Aj I 

plim n-lvj = plim n-h'x. 'nus,6 j is consistent, asymptotically efficient 

(in the limited information sense), and asymptotically normally distributed 

with mean 6j and a covariance matrix which is consistently estimated by: 

where 

is; by the above, a consistent estimator for u 

Further, a consistent (but biased) estimate of the reduced form 

parameters of the system (n) is obtained from: 

jj 
-8-

n = v

j 

-lX'Y . (15) 

.. N 

Let TI.J represent the columns of II corresponding to Y

j' 

Note that 

,u N - XII = N.Y = Yj , so N.2 = (Yj ,i Xj). Given projections of the pre- j 35 ~j 

determined variables of the system, Xp = (xi i x! , we may Project 

Y from 

j 

.. Y; = xpnj 

and then, defining Zp = (Y: i X;)s project y from 2's" . J j j j N 

While the proposed estimator 6 j is defined for any suitable choice 

of Aj in practice we suggest specifying A

j = a1 where 0 <a e=. 

Simplicity is, of course, a major advantage of this specification. In 

addition, our (quite limited) experience with this specification, 

.. reported below, suggests that the elements of 8 j 

are reasonably stable 

over fairly large ranges of a. Our current research is directed in 

part toward finding the "optimal" value of a for a given equation. A

- second direction for research is the small sample properties of 6 j* 

V. ESTIMATION OF KLEIN'S MODEL I - AN ILLUSTRATION 

Although the sample underlying Klein's Model I is not undersized 

(n=21, K=8), it has the advantage that it has been estimated using all 

of the alternative procedures previously discussed, including two-stage least 

squares, so that a numerical comparison of the various procedures is 

possible. The model consists of three behavioral equations: 
-9-

where t is measured in calendar years, C is consumption, P profits, 

W the private wage bill, W' the government wage bill, I net investment, 

K capital stock at the end of the year, and X the output of the private 

sector. The six endogenous variables are C, P, I, W, X and K; the model 

is closed by three definitional equations. The eight predetermined 

variables consist of three lagged endogenous variables, p-1' K-1' x-1 

and t, W', 1 (the constant), T (business taxes), and G (government nonwage 

expenditure). In (17), WW' is considered one endogenous variable. The 

underlying data is available in Theil [8, page 4561. 

Point estimates of coefficients, their asymptotic standard errors, 

and estimated variances are shown in the accompanying table. For the 

procedure proposed in this paper, coefficient point estimates are from 

equation (12), standard errors are square roots of the diagonals from 

equation (U), variances are from equation (14),and we have specified 

A

3 = aI. 

Using the full sample (n=21), the proposed estimator with a-1 gives 

results which are virtually identical to. two-stage least squares. This 

result is to be expected, since when n>K, the proposed procedure 

converges to the two-stage least squares procedure as a+O. With a=21, 

coefficients on the highly correlated variables P and P-1 in equations (17) 
- 10 -

and (18) and X and X-l in equation (19) diverge somewhat from the twostage least squares estimates. Standard errors, however, are quite similar. 

For Theil's D.-class procedure, standard errors tend to be larger in all 3 

equations, and the divergence of coefficients on P and P-l in equations 

(17) and (18) from the two-stage least squares estimates is greater than 

for the proposed procedure, but there is no divergence for any coefficient 

in equation (19). The Kloek and Mennes principal components procedure 

performs quite well in equation (18), but the coefficient of P in 

equation (17) and of X and X-l in equation (19) diverge somewhat from the 

two-stage least squares estimates. These results, of course, are no more 

than suggestive of the relative merits of the alternative procedures. 

To illustrate the proposed procedure when the sample is 

undersized, Klein's Model I was estimated for n=7, where the observations 

are those for 1922, 25, 28, 31, 34, 37 and 1940. These years are fairly 

representative of the full 21 year observation period. Note that when 

n<K, as a$ ttie proposed procedure converges to ordinary least squares, 

which normally coincides with the procedure of Swamy and Holmes [6] 

and Fischer and Wadycki [2]. 
rl 

00 

W 

um 

wm 

n 

mr­v "E ?? 

n 

hh 

=rl n ?9 W

+z d .... .. ... a 

IW

-NU ? w 0 w 

" ... m 

-0n ?? 0) 

h" 3 NW bD 

(r

v 

n a n O D - n r 

?? 

mm w 

1 - a U U 

In 

m n 0)

w-rl v wm "a &I '97 r 

W W 

B 

V 

n 

-An drl 0 

*& .. W W $ .... .. 1. . c N Lr I-

:: 0

m 

.. ... .o n 

" e

On wm 

6 .. a mN n 4- 8 V 0 

U 

n I-r h 5 dd 

6 

I mN 

8 g" . .W 

C 

W 

m 

3 

n rl

'93 

NrE.

2 W Y 

U h N Ah OD* d

'2. 9: a v 

m -a

M 

m 

rl 

N 

m 

6 

2 d

d 

Y 

rz* 

.s 

mu m 
- 12 -

REFERENCES 

1. 

2. 

3. 

4. 

5. 

6. 

7. 

8. 

9. 

Dutta, M. and E. Lyttkens, "Iterative Instrumental Variables Method 

and Estimation of a Large Simultaneous System", Journal of the 

American Statistical Association, 69,348 (December 1974), 977-986. 

Fischer, Walter D. and Walter J. Wadycki, "Estimating a Structural 

Equation in a Large System", Econometrica, 39,3 (May 1971), 461-465. 

Kloek, T. and L.B.M. Mennes, "Simultaneous Equations Estimation 

Based on Principal Components of Predetermined Variables", 

Econometrica, 28,l (January 1960), 45-61. 

Mariano, Roberto S., "The Existence of Moments of the Ordinary

Least Squares and Two-Stage Least Squares Estimators", Econometrica, 

40,4 (July 1972), 643-652. 

Sawa, Takamitsu, "Finite-Sample Properties of the k-Class Estimators", 

Econometrica, 40,4 (July 1972), 653-680. 

Swamy, P.A.V.B. and James Holmes, "The Use of Undersized Samples in 

the Estimation of Simultaneous Equation Systems", Econometrica, 

39,3 (May 1971), 455-459. 

Takeuchi, K., "Exact Sampling Moments of the Ordinary Least Squares, 

Instrumental Variable, and Two-Stage Least Squares Estimators", 

International Economic Review, 11,l (February 1970), 1-12. 

Theil, Henri, Principles of Econometrics, New York: John Wiley and Sons, 

1971. 

Theil, Henri, "A Simple Modification of the Two-Stage Least-Squares 

Procedure for Undersized Samples", in Arthur S. Goldberger and Otis 

Dudley Duncan (eds.), Structural Equation Models in the Social Sciences, 

New York: Seminar Press, 1973, 113-129.