Abstract

Key Words

1. Introduction
The study of estimating a single structural equation in econometric models has led to develop several estimation methods as the alternatives to the least squares estimation method. The classical examples in the econometric literatures are the limited informa- tion maximum likelihood (LIML) method and the instrumental variables (IV) method including the two-stage least squares (TSLS) method. See Anderson, Kunitomo, and Sawa (1982) and Anderson, Kunitomo, and Morimune (1986) for their finite sample properties, for instance. In addition to these classical methods the maximum empirical likelihood (MEL) method has been proposed and has gotten some attention recently in the statistical and econometric literatures. It is probably because the MEL method gives asymptotically efficient estimator in the semi-parametric sense and also improves the serious bias problem known in the generalized method of moments (GMM) method when the number of instruments is large in econometric models. See Owen (2001), Qin and Lawless (1994), Kitamura and Stutzer (1997), and Kitamura, Tripathi, and Ahn (2004) on the details of the MEL method. In the econometric literatures the generalized method of moments (GMM) estima- tion method has been quite popular in the past two decades. The GMM method was originally proposed by Hansen (1982) in the econometric literature and it is essentially the same as the estimating equation (EE) method proposed by Godambe (1960) which has been used in statistical applications. It has been known that both the MEL estima- tor and the GMM estimator are asymptotically normally distributed and efficient when the sample size is large. Because we have two semi-parametric estimation methods for econometric models and they are asymptotically equivalent, it is interesting to make comparison of the finite sample properties of alternatives estimation methods. The main purpose of this study is to develop the method of deriving the asymp- totic expansions of the distributions of a class of semi-parametric estimators for the coefficients of a single structural equation in the linear simultaneous equations system. The estimation methods under the present study include both the MEL estimator and the GMM estimator as special cases. Since it is quite difficult to investigate the exact distributions of these estimators in the general case, their asymptotic expansions give some information on their finite sample properties. In this paper the asymptotic ex- pansions shall be carried out in terms of the sample size which is proportional to the non-centrality parameters.  Comparisons of the distributions of the MEL and GMM methods can be made. We shall illustrate the usefulness of the asymptotic expan- sion method developed in this paper by giving some information on the distribution functions of the MEL and GMM estimators. Also we shall relate our results to the ear- lier studies on the asymptotic expansions of the distributions of the limited information maximum likelihood (LIML) and the two-stage least squares (TSLS) estimators. It will give some insights on the statistical properties of the alternative estimation methods for a single structural equation. In order to compare alternative estimators, it is much more easier to investigate the asymptotic expansions of their mean and mean squared errors (MSE) than their exact distribution functions.  Since the exact distributions of estimators can be quite different from the normal distribution, it should be certainly better to investigate the asymptotic expansions of their exact distributions directly. Also the asymptotic expansions of the mean and the MSE of estimators are not necessarily the same as the mean and the MSE of the asymptotic expansions of the distributions of estimators. In fact it has been known in econometrics that the LIML estimator does not possess any moments of positive integer order under a set of reasonable assumptions. This paper may be the first attempt to develop the asymptotic expansions of the distribution functions of semi-parametric estimators and to find their explicit form in the estimating equation model and the simultaneous equation models in econometrics. Our formulation and method of this paper are intentionally parallel to the classical studies on the single equation estimation methods in the linear simultaneous equations by Fujikoshi et.  al.  (1982) and Anderson et.  al.  (1986).  It is mainly because useful interpretation can be drawn in the light of past studies on the finite sample properties of alternative estimators in the classical parametric framework as well as the semi- parametric framework. The problem dealt with our paper would be related to the studies of higher order asymptotic efficiency of estimation by Takeuchi and Morimune (1985), and Newey and Smith (2004). In Section 2 we define the single equation econometric model and their estimation methods.  Then in Section 3 we give the stochastic expansions of a class of estimators.  In Section 4, we give the results of the asymptotic expansions of the distribution functions of estimators under a set of assumptions on the disturbances. Then we shall compare the asymptotic expansions of alternative estimators including their asymptotic biases and the mean-squared errors in a simple case and a more general case. Some discussion and concluding remarks are given in Section 5. The proofs of Lemmas and Theorems and some useful formulas for our results will be given in Appendices.
2. Estimating a Single Structural Equation by the Maximum Empirical
Likelihood Method
Let a linear structural equation in the econometric model be given by
where y 1 i and y 2 i are 1 × 1 and G 1 × 1 (vector of) endogenous variables, Z 1 is a K 1 × 1 vector of exogenous variables, θ ′ = (β ′ , γ ′ ) is a 1 × p (p = K 1 + G 1 ) vector of unknown coefficients, and { u i } are mutually independent disturbance terms with E(u i ) = 0 (i = 1, . . ,n). We assume that (2.1) is the first equation in a system of ( G 1 + 1) structural equations relating the vector of G 1 + 1 endogenous variables Y i = ( y 1 i, y 2 i ) and the vector of K (= K 1 + K 2 ) exogenous variables { z i } which includes { Z ı ;} . The set of exogenous variables { Z ;} are often called the instrumental variables and we have the orthogonal condition
Because we do not specify the equations except (2.1) and we only have the limited information on the set of instrumental variables (or instruments), we only consider the limited information estimation methods. When all structural equations in the econometric model are linear, the reduced form equations for y'; = ( Y i ; Y'2i ) can be represented as
もwhere V i = ( v 1 i , v 2 i ) is a 1 × (1 + G 1 ) disturbance terms with E[v i ] = 0 and  Π ′ = ( π 1 , Π 2 ) ′ is a (1 + G 1 ) × K partitioned matrix of the linear reduced form coefficients. By multiplying (1, − β ′ ) to (2.3) from the left-hand side, we have the linear restriction (1, − β ′ )Π ′ = ( γ ′ , 0′ )  and u i = ν 1 i − β ′ ν 2 i ( i = 1, · · · , n ) . The maximum empirical likelihood (MEL) estimator for the vector of unknown parameters θ in (2.1) is defined by maximizing the Lagrange form
where μ and λ are a scalar and a K × 1 vector of Lagrange multipliers, and p i ( i = 1, ･ ･ ･ , n ) are the weighted probability functions to be chosen. It has been known (see Qin and Lawless (1994) or Owen (2001)) that the above maximization problem is the same as to maximize
where we used the conditions ˆ ı = n and [ nˆ ı ]− 1 = 1 + λ ′ z i [ y 1 i − ( y 2 i ; z ′ i )θ ] . By differentiating (2.5) with respect to λ and combining the resulting equation with the restriction ∑ n =1 P i = 1, we have the relation
and
where u i ( ğ ) = Y 1 i − ( Y 2 i , Z 1 i ) ĝ and ˆ θ is the maximum empirical likelihood (MEL) estimator for the vector of unknown parameters θ . From (2.6) and (2.7) the MEL estimator of θ is the solution of the set of p equations
If we substitute 1/n for ˆ ρ i ( i = 1, ･ ･ , n ) in (2.8) and set an initial (consistent) estimator , θ of θ , then we have the generalized method of moments (GMM) estimator for the vector of coefficient parameters θ ′ = ( β ′ , γ ′ ), which can be written as the solution of
(See Hayashi (2000) on the details of the GMM method in econometrics, for instance.) 4 By generalizing the weight probabilities p i ( i = 1, . . , n ) in (2.8), we can introduce a class of estimators. Let
where δ is a positive constant (0 ≤ δ ≤ 1) and θ is the MEL estimator of θ . Then we define the modification of the MEL estimator (the MMEL estimator) by substituting ˆ β i (i = 1, · · , n ) into (2.6)-(2.8). We shall denote the resulting Lagrange multiplier and the modified estimator as λ and θ whenever we can avoid any confusion. In the rest of the present paper, we shall consider the standardized error of estima- tors as (2.11) where θ  = ( β , γ ′ ).  We sometimes denote ê for the MEL estimator and its modification whenever we do not make any confusion.  Under a set of regularity conditions, the asymptotic variance-covariance matrix of the asymptotically efficient estimators in the semi-parametric framework is given by
where K × K matrices M n and C n are defined by
and
We assume that the (constant) matrices M and C are positive definite, and the rank condition (2.15) rank(D) = p (= G 1 + K 1 ) . These conditions assure that the limiting variance-covariance matrix Q is non-degenerate. The above rank condition implies the order condition  L = K − p ≥ 0 , which is the degree of over-identification. When the disturbance terms are (conditionally or uncon- ditionally) homoscedastic random variables such as an i.i.d. sequence, then we have C = σ 2 M and E(u 2 ) = σ 2 In order to compare alternative efficient estimation methods in the finite sample sense, we need to derive the asymptotic expansions of the density functions of the standardized estimators (2.16) in the form of
where ξ = (ξ 1 , ･ ･ , ξ p ) ′ , φ Q (ξ) is the multivariate normal density function with mean 0 and the variance-covariance matrix Q , and H i ( ξ ) ( i = 1, 2) are some polynomial func- tions of elements of ξ . In order to derive the asymptotic expansions of the distributions of estimators in a simple manner, however, we need a set of regularity conditions.
Assumption I :
(i) The sequences { v i } (and hence { u i }) are mutually independent random vectors which have the positive density with respect to Lebesgue measure with E[||vi||7+c] < ∞ for some ε > 0. (ii) The (constant) matrices M and C are positive definite and the rank condition is satisfied with
(iii) The sequence of z i = ( z i ) ( i = 1, . . . , n ; j = 1, . . . , K ) are independent ran- dom variables (or non-random) vectors, which are mutually independent of Vi (i = 1, ･ ･ ･ , n ). The vectors { z i } are bounded or E[|| z i || 4 + c ] for some ϵ > 0. There exist finite M3(j1,j2,j3) such that (1/n) ∑ n =1 u ˙ z(j1) z(j2) z(j3) = M3(j1,j2,j3) + op( ˙ ω). We need some moment conditions on disturbance terms to derive higher order stochastic expansions of the associated random variables up to O ( n −1 ). The condi- tions in (ii) and (iii) of Assumption I are rather strong and it is possible to weaken them. Then the resulting formulas and their derivations become more complicated than those reported in this paper while the essential method of derivations will not to be changed. We can treat both cases when { z i } are stochastic and deterministic, and also it is possible to replace the independence assumption with { u i } by using a martingale assumption on the random vector sequence of ∑ i =1 Ziui . In order to avoid the inessential arguments, however, we mostly treat { Z i } as if they were deterministic variables. We shall use the mean operator AM n (ê), which is defined as the mean of ê with respect to the asymptotic expansion of its density function of the standardized estimator up to O (n-1) in the form of (2.16). We write the asymptotic bias and the asymptotic MSE of the standardized estimator by
and
These quantities are useful because the asymptotic expansion of the distribution of estimators are quite complicated in the general case. However, it should be noted that they are not necessarily the same as the asymptotic expansions of the exact moments and some care should be taken in this respect 1 .
3. Stochastic Expansions of Estimators
3.1
Stochastic Expansions
First, we apply the similar arguments used in Owen (1990) and Qin and Lawless (1994) on the probability limits and the consistency of the MEL estimator. Then we have nˆı → 1, ˆ θει → θο, (θο is the true value of θ) and √πλ converges to a random vector as n → ∞. By substituting (2.1) into (2.8), we have the representation of the standardized estimator ê as
where we use the notation ˆ θ for ˆ θ EL without any subscript whenever we do not have any confusion. As n → ∞ , we write the first order term of ê as ê 0 , which is given by
In the following derivation we shall utilize the asymptotic equivalence under Assump- tion I that  ẽ 0 — e 0 = o p (1) with
By using the central limit theorem (CLT) to the last term, we have the weak convergence of the random vector
and then * N p (0, Q), where a p × p matrix Q has been defined by (2.12) and means the convergence of distribution as n → ∞ . Also we notice that
By utilizing the representation of (2.7) for Â , we have  √πλ (3.6) λ 0 = C − 1/2 [I K − C − 1/2M n D(D ′ M n C − 1 M n D)− 1 D ′ M n C − 1/2 Since the limiting distribution of B n = C − 1 / 2 X n is given by N K (0,1 K ), we have the convergence  C 1/2 √πλ −→ NK(0, PD+) , where the projection operator is defined by P D • = I K − D * (D* ′ D* ′ which is constructed by a K × p matrix D* = C − 1 / 2 MD The variance-covariance matrix of the limiting distribution of random vector λ 0 is given by (3.7) which plays an important role in our analysis. The sample analogue of D* is given by D* = C~1/2MnD ~ > D* as n ~ +∞ . The method we shall use to derive the asymptotic expansion of the density function of the standardized estimator ê is similar to the one used in Fujikoshi et. al. (1982) and Anderson et. al. (1986). We shall expand ê by the perturbation method in terms 7 of the components of random matrices X n = ( X ( j ), Y n = ( γ ( j,k)), Z n = ( Z ( j,k)) and U n = (U ( jk ) which are defined by
where we have p × 1 random vectors w i = (v 2i, 0 ′ ) ′ − q i u i + O p (n − 1 ) and q i E[( ¥2i )'ui]/E[ui]  (i = 1, ···,n).  In order to facilitate our analysis, we assume that there exists a vector q such that
in the following analysis which can be relaxed. Then if E[| u i | 5 ] < ∞ for s ≥ 3, we can take a positive (bounded) constant c n (1, n ) depending on n which satisfies
where Λ n (1) as the maximum of the characteristic roots of C n . Also for the random variables Y n , Z n and U n we can take positive constants c n (i, s) (i = 2, 3, 4) and similar inequalities for s ≥ 3 under Assumption I with Condition (3.9). These arguments on the validity of the asymptotic expansions of random variables have been given by Bhattacharya and Rao (1976), and Bhattacharya and Ghosh (1978) for the i.i.d. ran- dom vector sequences. They can be easily extended to the present situation while their derivations and resulting explanations become quite lengthy. The validity of the asymptotic expansions based on the inversion of the characteristic functions, which will be utilized in Section 4 of this paper, was also briefly discussed by Fujikoshi et. al. (1982). In the econometric literatures the asymptotic expansion method has been previously discussed by Sargan and Mikhail (1971), and Phillips (1983), for instance. By expanding (2.7) and (2.8) with respect to e 0 , formally we can write
and
By substituting these expansions into p i ( i = 1, . . . , n ), we can also expand the esti- mated probability function as
where p. λρΖι , and
8 In these representations it is possible to show that max 1 ≤ i ≤ n | ˆ ˆ β i − 1/n| = o p (1/n) since (npi)-1 = 1 + λ ′ z i u i (see Owen (1990)). By using the similar arguments, we have max1< i<n | ˆ β i − 1/n − p ( 1 ) /( n √ π ) − p ( 2 ) /n 2 | = o ρ (1/n 2 ). By using the representation ( Y ′ 2 ′ 2 ′ i ) = z ′ D + w ′ i + q ′ u i ( i = 1 ,۰۰, n ) and recursive substitutions, we expand
= C... E(0) where we have recursively defined the random matrices as C ( 0 ) and
By substituting the above expressions into (3.1) for ê , Î, and î i (i = 1, . . . , n ), we can determine each terms of the stochastic expansions of ê in the recursive way. The leading two terms are given by
where we have used the corresponding random matrices Q − 1 = D ′ M n C − 1 M n D,

3.2
Effects of C n
We need to investigate the effects of estimating the variance-covariance matrix C by C n in the semi-parametric estimation methods. We notice that under a set of regularity conditions, each components of Y n have the asymptotic normality as n → +∞ . The covariance of the ( j , k )-th elements of Y n and the l-th element of X n is given by
From this relation we see that X n and Y n are asymptotically independent when E( u ? ) = 0 ( i = 1, . . . , n ) and then our analyses can be simplified considerably in this situation. It is the case when the disturbances are normally distributed, for instance.
9
Next, we expand the inverse of the asymptotic variance-covariance matrix of efficient estimators as
Then by using the inversion of the matrix Q n , we can represent
and
By using the above relations, the first term of the stochastic expansion of ê can be represented as
where ẽ 0 = QD'MC-1X n , e(1) = −QD'MC-1Y n AX n and e(2) = QD'MC-1Y n AY n AX n Also by using the stochastic expansions of C n and Q n , we can derive a representation of λ 0 (= [C − 1 − C − 1 MDQ n D ′ MC − 1 ] X n ) as
The above formulas help simplifying our derivations considerably.
3.3
Terms involving e 1
We shall investigate each terms involving e 1 . For this purpose we decompose e 1 as e1 = e1.1 + e1.2 + e1.3, where e1.1 = Qn[A1][Xn − MDe0], e1.2 = −e0(q'e0) and e 1.3 = −Q n D ′ MC − 1 U ′ n e 0 . The last two terms can be investigated rather easily and we treat these terms first. By using the representations in Section 3.2, we first rewrite e1.2
and
In the above expressions we have defined e (1) and e (1) implicitly. The analysis of e 1.1 becomes substantially more complicated because there are many terms involved in the random matrices C (1) and E (1) . We rewrite

11
On the other hand, we represent
where E(1.0) = Un + q(X'n − X'oCn) and
Then we have
In order to derive the asymptotic expansions of the distributions of the class of the modified estimators, we need to use ρ π instead of ρ i ( i = 1, 2). For an arbitrary (fixed) δ (0 ≤ δ ≤ 1), we substitute δλ 0 (and δλ 1 ) into λ 0 (and λ 1 ) in the stochastic expansion of the MEL estimator. Thus we can further express e 1.1 as

12

and
(3.24) where e e(°) + e(°) + e(°) and e(°) + e 1 , 2 + e 1 , 3 Then the leading terms of e 1 can be represented as
The random vector é o is asymptotically normal and it is asymptotically uncorrelated with the random vector AX n by using CLT. Then by using Lemma 4.3 in Section 4, given ẽ 0 = x the conditional expectation of e ( 0 ) is given by 2 (3.25)
where (3.26)

13
Also the conditional second order moments of e( 0 ) can be calculated as
where we use the notations C t = qq ′ and Q * = D ′ MC − 1 MC − 1 MD. In the above calculations we have used some relations on moments by applying Lemma 4.2 in Section 4 as E[(X ′ , AX n )²] = L(L+2)+O(n−1/2) and E[(z ′ , AX n )²(X ′ , AX n )] = (L+2)z ′ , Az i + O(n-1/2). It is because we use the fact that the limiting random vectors é 0 and AX n are asymptotically normal and uncorrelated.
Terms involving e 2
3.4
We shall investigate the terms associated with e 2 . For this purpose we decompose e 2 = e 2.1 + e 2.2 + e 2.3 and e 2.i (i = 1, 2, 3) correspond to each terms of (3.17). Because we can estimate Q and C consistently by using Q n and C n , their estimates do not affect the terms much involving e 2 . We first consider e 2 ,3 and by using the stochastic expansion of e1, we have
Because ẽ 0 and AX n are asymptotically orthogonal, the conditional expectation of the first term, e 2.3.1 = −QD ′ MC−1 U nq ′ e ( 0 ′ , given ξ 0 = x can be calculated as
The second term of e2.3 can be also expressed as e2.3.2
Hence its conditional expectation given e 0 = x can be expressed as
By combining these terms, we have
Secondly, we shall evaluate the terms involving e2.2. For this purpose we notice that we only need to investigate two terms as e2.2.1 = −Q[A1]MDe(0) and e2.2.2 = −Q[A1][U'n + Xnq']ěo because it is straightforward to show that the rest of terms e2.2 — e2.2.1 — e2.2.2 = Op (n-1/2). The second term can be further rewritten as
Then the conditional expectation is given by
Now we use the decomposition X ′ n C− 1 X n = X ′ n AX n + ε ′ 0 Q− 1 δ 0 , and also we notice the relation
Then the conditional expectation can be rewritten as

15
On the other hand, the first term of e2.2 can be expressed as
Given e o = x, the conditional expectation is given by
In order to evaluate each terms in the above expression, we use the relations ACMD = O, E[X'nAë 0 | ë 0 = x] = O p (n − 1 /2 ), and
Then we have the conditional expectation (3.30)
Hence we obtain the explicit expression of the conditional expectation E[e 2.2 | ẽ 0 = x] = E[e2.2.1|x] + E[e2.2.2|x] up to Op(n-1/2) Thirdly, we shall evaluate all terms involving e2.1 which is the first term of (3.17) and we need more complicated evaluations. We notice that it is asymptotically equivalent to (3.31) where we use the notations e2.1(A) = -QD'MC-1C(2)AXn, e2.1(B) = QD'MC-1C(1)C-1C(1)AXn, e2.1(C) = −QE(1)C-1C(1)AXn and e2.1(D) = QE( 2 ) AX n . Because the above terms contain p( 2 ) ( i = 1, . . . , n ), we need to use the explicit expression of λ 1 , which is the solution of the equation
In the above representation we have used the representation of residuals as u i ( θ ) = y 1 i − ( y 2 i, 2 i i ) θ = u i − ( - π )[z i D + ( v 2 i, 0 ' )]ê.  Then by using the stochastic expansion of C (1) , we have the representation as
By using the relations C-1Xn = AXn + C-1MDQD'MC-1Xn and 2AXn − C-1Xn = AX n − C−1MD e, we have
Then we need to evaluate each terms by using the expression for e 1 and λ 1 . By using the explicit representation of p ( 1 ) ( i = 1, . . . , n ), it can be written as
Since the random vector AX n is asymptotically uncorrelated with é 0 , we have the convergence in probability as 2QD'MC-1( n ∑ π =1 u͡z i z i z i )AX n (q'e 1 ) → O, we use the
17
relation E { π ∑ i = 1 ZiZi j [( zi D ē 0)² + (w i ē 0)² + u ç (q ′ ē 0)²]AXn | ẽ 0 = x } = O p (n-1/2). Hence for e2.1( A ) we only need to evaluate the conditional expectation of the last four terms as −QD′MC−1 { π ∑ π =1 z i z i [−2δ(z i AX n )( u ? z i ,D̃δ + u ? w ĩδ 0 + u ? q́ẽ 0 ) + u?p( 2 )] } AX n up to the order of O p ( n − 1/2 ). For the last term involving p( 2 ), we use the stochastic expansion of λ 1 and it becomes − QMC− 1 times
Here we illustrate the typical argument of order evaluations.  For the final term, we write
and the second term is of order O p ( n −1/2). Then we take the conditional expectations of each terms.  By applying Lemma 4.3, we find that the first term in the above equation is of order Op (n-1/2) . Also we use the relation
Then we can show that E[ = ∑ i = 1 z i u ? w i ē 0(z i AX n ) 2 | x | =  ≠ ∑ i = 1 z i E[ u ? w i ] z i , Az i x and E[ π ∑ ï=1 z i u ³ q ′ ẽ 0 (z ′ ,AX π )² | x ] = π ∑ ï=1 E( u ³ ) z iq ′ xz ′ ,Az i , where we have ignored the terms of O n −1/2 ).  After tedious calculations of conditional expectations, we then can
expression
(3.32)
Similarly, the second term of e2.1 is expressed as
Then by using the asymptotic uncorrelatedness of AX n and é 0 , the conditional expec- tation can be reduced to (3.34)
For the third term, we write e2.1(C) = −QE(1)C−1C(1)AXn, and we use the fact that QE(1)C-1 = [QUn + Qq(X'n - X'oCn)]C-1 + Op(-)n) and C(1)AXn = δ(1/n) ∑ n =1 E(u 3 ) z i z i (z i AX n )] AX n + O p ( J n ).  Then the conditional expectation of e2.1( C ) is given by (3.35)
The fourth term of e 2.1 is expressed as
Since the first term of e 2.1( D ) is similar to the last term of e 2.1( A ) and we can utilize the expression of λ 1 in (3.32), it is easy to find that the conditional expectation, which is given by

19

For the second term of e 2.1 ( D ), we rewrite
e2.1.2(D), e2.1.3(D), respectively.  Then
Also we find that
But the random vector AX n is asymptotically normal, and its limiting distribution is uncorrelated with that of ẽ 0 . Hence we find E [e 2.1.3( D ) | ẽ 0 = x ] = O p (n − 1/2 ). Then we evaluate the conditional expectation of e 2.1.2 ( D ). Because the pairs of random vectors (w i , u i ) are uncorrelated, we have the convergence in probability (1/n) ∑ π=1 w i u i z ( i ) z ( k )  P + 0. Therefore, as for the remaining conditional expectation terms, we use the explicit expression for λ 1 , we find

20

Finally, we can derive the conditional expectation of E[e2.1| ẽ 0 = x] by collecting the conditional expectations of E[e2.1(A)|x], E[e2.1(B)|x], E[e2.1(C)|x] and E[e2.1(D)|x]. At the first glance of these many terms, it looks formidable to calculate them. However, the resulting formulas become relatively simple if we can ignore the third order mo- ments because it has turned out that many terms have disappeared in the conditional expectation formulas eventually.
4. Asymptotic Expansions of Distributions
Conditional Expectation Formulas
4.1
In order to derive the asymptotic expansions of the distribution functions, we prepare some useful formulas on the conditional expectations and their proofs are given in Appendix A. They are the result of straightforward calculations, but we shall give the derivations of Lemma 4.1 and Lemma 4.3 in Appendix A for the expository purpose. Lemma 4.1 : Let the random vectors ξ 0 , X n , and Y n be defined as in Section 3. Then
where m3 is given by (3.26). Lemma 4.2  : Let a set of random vectors X = ( X i ) and Z = ( Z i ) be normally distributed and the conditional expectation of X i given Z be E(X i | Z ). Then (4.2) E[X i X i X i ]
and
E[X i X i X i X i ]
The above formulas are used repeatedly in our derivations by setting Z = é 0 , which is
21
the leading term of our stochastic expansions of estimators. In order to evaluate the conditional expectation operations appeared in the stochastic expansions of estimators, we also need the next formula. Lemma 4.3  : Let u n = ( u i ) and v n be p × 1 random vector and a random variable with E( u i ) = 0, E( v n ) = 0, E( u i u j ) = δ( i, j ), E( v n ) = 1 and they have finite fourth order moments.  Assume that they are sum of i.i.d.  (non-lattice) random vectors and asymptotically normally distributed and admit the asymptotic expansion of their distribution function up to Op (n-1) . Then
if l = l'  and δ(l,l') = 0 if l ≠ l'), and ρ = Cov(v,un). In particular, if E(u i u j u k ) = 0 (i ≠ j ≠ k ), then βιι ι ι ι = 0 and the second term in the order of O p (n-1/2) in (4.4) vanishes. Although we have obtained several conditional expectation formulas associated with the terms e ( 0 ) and e 2 in Section 3, we need to evaluate several additional terms for deriving the asymptotic expansions of density functions of estimators up to O ( n −1 ). By applying Lemma 4.3, the conditional expectation of e ( 1 ) in (3.20) given X n can be calculated as
By decomposing AXnX'nC-1 = AXnX'nA + AXnfeoD'MC-1 and taking the condi- tional expectations, we have
(4.6E[e]
22
Then it is immediate to obtain the (unconditional) asymptotic bias of the class of standardized estimators up to the order o (1/ √ n ). It is possible to continue our procedure to derive the asymptotic expansions of the density function of estimators in the general case, but the resulting expressions become more messy due to the terms associated with e( 1 ) and e( 2 ) . It is because the conditional expectations of some random variables of order O p ( n − 1/2 ) have the terms of O p (n − 1/2 ) and also some additional terms of O p (n − 1), which make our derivation some complications. When we can ignore the effects of the third order moments of the disturbance, the asymptotic expansions of estimators for an arbitrary δ (0 ≤ δ ≤ 1) can be simplified greatly in evaluating many terms of order O p ( n −1 ). Then we shall make the following assumption and show how our derivations can be simplified greatly.
Assumption II :
(i) For the second and fourth moments of disturbances in (2.1) and (2.3) we assume E( u i ) = σ 2 > 0, C 2 = E( w i w i ), the condition given by (3.9) and κ = E( u i )/σ 4 − 3 for i = 1, . . . , n . (ii) For the third order moments of disturbances in (2.1) and (2.3) we assume E[ u 3 ] = 0 and E[u ? w i ] = 0 for i = 1, · · · , n. It is immediate that the conditions in Assumption II can be relaxed as
and the similar conditions on the third order moments on { u 2 w i } . In this section we shall derive the asymptotic distribution functions of estimators under Assumption I and Assumption II. Now we shall evaluate the conditional expectations of e (1) = e(1) + e(1) + e( = x . This term plays an important role and makes some complications of our analyses. We first note that the conditional expectations of e(1) and e(1) given ẽ 0 = x can be evaluated easily. By using Lemma 4.3, their conditional expectations are given by
and (4.9)E[e [ 1 ] | ẽ 0 = x]
which are both the terms of O (1/n) under Assumption II. Next, we evaluate the conditional expectation of e( 1 ) in (3.23) which has many terms. We notice that in e ( 1 ) two terms associated with the random matrices C ( 1 ) and E ( 1 ) have been cancelled out. Then we try to evaluate each remaining terms of order O p (n-1/2) and the terms in the first two lines are given by
Similarly, the terms in the third line are given by
which are of order O p ( n-1/2 ) under Assumption II . The most important terms in Op(n-1/2) are the next two terms in the fourth and fifth lines because they are depen- dent on the fourth order moments of { u i }, which are given by
up to O p ( n −1/2 ). When we have homoscedastic disturbances, we can use the fourth order cumulant given by κ = [ E ( u t ) − 3σ 4 )] /σ 4 . The conditional expectations of the first term and the second term are given by
and

24
For the expression of e( 1 ), there are many remaining terms in the last three lines, which are given by +
which are of the order Op(n-1/2). Finally, after many terms involving the third order moments disappear due to Assump- tion II , we have only two terms involving the fourth order moments terms in (4.10) and (4.11). Hence under Assumption II in this section the conditional expectation of e (1) is given by
where we define (4.13) We note that (4.12) is one of key aspects in our analysis of the semi-parametric esti- mation of a single structural equation because we do not have the corresponding terms Op(n-1) in the asymptotic expansions of the classical estimation methods. Similarly, under Assumption II,
In order to evaluate E[e] [ ξ 0 = x], we need the relation that for a constant matrix A (= ( A j k ))
under Assumption II. Hence we have
because the random vector AX n is asymptotically normally distributed and uncorre- lated with ̃o. By using direct calculations we have
in the general case. For the random matrix U n = ( u j k ) = (1/ √ n ) ∑ i =1 w i z j i , we apply Lemma 4.3 and use the fact that Cov(u jk , e 0. Then we have
Then by decomposing C-1 = A + C-1MD'QD'MC-1 and z',C-1XnX',C-1zi = 2z',AXné 0 D'MC- 1z' + z',C- 1 MDé 0 é 0 D'MC- 1z', and taking the conditional expec- tation, we have
By using the above type of the conditional expectations formulas and lengthy calcula- tion under Assumption II, we find that
which is of Op (n-1/2).
4.2
Asymptotic Expansions of Density Functions
Since there are many terms appeared in the stochastic expansion of ê, at first it looks formidable to evaluate all terms and to derive the asymptotic expansions of the density functions of alternative estimators. Under Assumption I and Assumption II, how- ever, it is possible to derive them in a compact form which gives useful information on the exact distributions of estimators. In order derive the asymptotic expansions of density functions, we define e ξ = ẽ 0 as the leading term.  As the second term, we define e f (x) as the sum of constant
26
order terms of the conditional expectations E[e( 0 ) + e( 1 ) | ẽ 0 = x]. As the third order term, we define e ξ ( x ) as the sum of O p ( n − 1/2 ) terms of the conditional expectations + e( 1 ) | ě 0 = x ] plus the constant order terms of the conditional expectations E[e( 2 ) + e( 1 ) + e 2 | ẽ 0 = x]. As the cross order term, we define e 2 1 (x) as the sum of constant order terms of the conditional expectations E[(e(0) +e(1)(e(0) +e(1))'|ē0 = x]. By using the conditional expectation formulae we have obtained under Assump-
resented as
Next, we consider the characteristic function of the standardized estimator ê in order
Fourier Inversion Formulae developed by  Appendix B of Fujikoshi et.  al.  (1982), we can invert the characteristic function in (4.20). Although the intermediate computations are quite tedious but straightforward. We first consider the asymptotic expansion of the density function of ξ 0 and we know that its limiting distribution as n → +∞ is normal. By expanding its characteristic function E[exp( it'é 0 )] and inverting it as the standard practice in the asymptotic theory, we have (4.21) n3/2
27
where φ Q (ξ) is the p-dimensional normal density function with means 0 and the variance-covariance matrix Q , where z t = ( z ( t )* ) = QD'MC-1z i ( i = 1, . . . , n ), and ∑ ι t , t , t , t , t , t , t , t , means the combina- tions of two pairs such as ( l , l ′ ) and ( l ′ , l ′′′ ) (it is 3 when l = l ′ = l ′′ = l ′′′ , for instance). Also h 3 ( x 1 , x 1 ′′ ) and h 4 ( x 1 , x 1 ′′ , x 1 ′′ , x 1 ′′′ ) are the Hermitian polynomials as
We notice that (4.21) is common for all efficient estimators and then it does not make any effects on the comparisons of alternative efficient estimators. When the third order moments of disturbances are zeros, the terms of O p ( n −1/2 ) on the right-hand side vanish (i.e.  βייויויויויויוי= 0) and we only have extra terms in the order of n-1. In these cases we can directly use the Fourier inversion formulae reported in Appendix B because only terms on the effects of non-normality of disturbance terms appear as QD'FDx in the order of O p ( n −1 ) and the resulting expressions become considerably simplified. Also we notice that when the disturbance terms are conditionally homoscedastic as in Assumption II, we have C = σ 2 M, Q = σ 2 (D'MD)− 1 , Q* = D'MC− 1 MC− 1 MD = σ− 2 Q− 1 , and tr(MA) = σ−2L . Then we can state our main result after lengthy but straightforward computations by using the formulas given in Appendix B. Theorem 4.1  : Under Assumption I and Assumption II , the asymptotic expansion of the joint density function of ê for the class of modified MEL estimators as n → ∞ is given by

28
where ξ is a p × 1 (p = G 1 + K 1 ) vector, φ ξ (ξ) and F are given by (4.21) and (4.13), respectively, and C 1 = C t (= qq'), C 2 = σ− 2 C z (= σ− 2 E [w i w i ]), and κ = [E(u t ) − 3σ 4 )/σ 4 . The first term φ δ (ξ) are common among all asymptotically efficient estimators and we need to make comparison on the terms of the second term of O(n-1/2) and the third term of O ( n −1 ). When the disturbance terms are normally distributed all terms except the leading term vanish in (4.21) and φ ∞ (x) = φ Ω (x). There is an interesting observation in Theorem 4.1 that if we further drop the last term
and the disturbance terms are normally distributed, the resulting formulas are identical to those for the limited information maximum likelihood (LIML) estimator and the two stage least squares (TSLS) estimator, which have been reported by Fujikoshi et. al. (1982). By using the asymptotic expansion of the density function, we can evaluate the asymp- totic mean and the asymptotic mean squared errors of the modified MEL estimator. We summarize the resulting formulas. Theorem 4.2 : Under the assumptions of Theorem 4.1 , the asymptotic bias and the asymptotic mean squared errors of ê for the modified (MEL) estimators based on the asymptotic expansion of the density function as n → ∞ up to O ( n − 1 ) are given by
and
where we use the notations in Theorem 4.1.
A Simple Case
4.3
We notice that the exact density functions of estimators and their asymptotic expan- sions are quite complicated in the general case. Hence it is interesting to derive the asymptotic expansions of the distribution functions of estimators in the simplest case when G 1 = 1. We take the estimator on the coefficient of an endogenous variable in the right hand side and standardize
since its limiting distribution is the standard normal.
as the (1, 1) − element of Q and we partition a [1 + (p − 1)] × [1 + (p − 1)] matrix as
The right-hand side of φ * ( x ) for the standardized estimator in (4.26) can be simplified and it is in the form of
where β 3 and β 4 are the third and fourth order cumulants in (4.21) by replacing z 3 Q − 1 / 2 z ( 1 )*) for z † ( i = 1 , . . , n ) and φ ( x ) is the density function of the standard normal distribution. For any p − dimensional normal density φ (ξ), we partition any [1+(p−1)] × [1+(p−1)] matrix B and p (= [1 + (p − 1)]) vector ξ ′ = (ξ 1 ,ξ ′ 2) ′ , we can write
By using the fact that ξ 2 | ξ 1 follows the ( p − 1) − dimensional (conditional) normal distri- bution N p –1 [ Q21Q11 ξ1 , Q22.1 ] and Q22.1 = Q22 − Q21Q11 Q21 , we can evaluate (4.28) explicitly and it is rewritten as
When the third order moments of disturbances are zeros, the asymptotic expansion of the density function of the standardized estimator can be simplified. We set the standardized form as (4.26) and notice that two matrices C 1 and C 2 appeared in the asymptotic expansions have non-zero elements only in the upper-left parts. By evaluating the integrations with respect to the ( p – 1) last elements of ξ in Theorem 4.1, the asymptotic expansion of the density becomes
where φ*( · ) is given by (4.28) with β 3 = 0 , α* = − Cov(v2i, ui)/|Ω|1/2, η* = |Ω|/σ², and
If we further assume the normal disturbances, then the formula can be further simpli- fied. By setting δ = 1 for the MEL estimator, we have
where Φ( · ) and φ( · ) are the cdf and the density function of the standard normal distri- bution, respectively, and α = ω 22 [β − ω 22 )] / [Ω | 1 / 22
Also under the normal disturbances and we set δ = 0 for the GMM estimator, the resulting asymptotic expansion of the distribution function becomes
Furthermore if we set τ = 0 in the above expressions, the resulting formulas in (4.31) and (4.34) are identical to those for the limited information maximum likelihood (LIML) estimator and the two stage least squares (TSLS) estimator obtained by Anderson (1974), and Anderson and Sawa (1973), respectively.
A More General Case
4.4
When we remove Assumption II , it is straightforward to obtain the asymptotic ex- pansion of the density function of the MEL and GMM estimators up to the order O(n-1/2). We find that the explicit form of the asymptotic expansion of the density function of the MEL estimator given in Theorem 4.1 is unchanged up to the order O(n-1/2) with φ δ (ξ). We also see that the factor φ Q (ξ)(q'ξ)[p + 1 — ξ'Q-1ξ] in the term O(1/ √ n ) is symmetric around zeros. Then if we write ˆ ε is the i-th component of ê for the MEL estimator,
when the third order moments of disturbances are zeros.  On the other hand, the asymptotic expansion of the density function of the GMM estimator has an additional
31
term involving the third order moments and the term of O ( n −1/2 ) is proportional to L(1/ √ n ), where L = K 2 − G 1 . Hence if K 2 (the number of excluded instruments) is large, the probability bias of the GMM (or the TSLS) estimator becomes sustantial while the MEL (or the LIML) estimator concentrates its probability around the true parameter values (see Tables in Anderson et. al. (2005) when G 1 = 1). Although it is possible to derive the explicit expressions of the asymptotic ex- pansions of the terms of O ( n −1 ), they become quite tedious because there are many additional terms as we have illustrated in the derivation of the asymptotic expansions in Sections 3 and 4. However, it is rather straightforward to derive the asymptotic bias of the estimator in the general case. By using (4.6) we have
The above formula agrees with the recent result by Newey and Smith (2004) on the asymptotic bias of the MEL and GMM estimators in the more general nonlinear setting of estimating equation model. Although it is straightforward to proceed our step to the mean-squared errors of alternative estimators, it is quite tedious to obtain the explicit formula of the additional terms in the asymptotic MSE of the modified estimators in the general case.  In principle it can be calculated by
In additon to AMSE*(ê) of AM(êê') in Theorem 4.2 , there are several additional terms for an arbitrary δ (0 ≤ δ ≤ 1) when we cannot ignore the effects of third order moments of disturbance terms. For the MEL estimator case, however, there are only a few additional terms. Although it is straightforward to write down those terms, we have omitted to report the details since they are complicated and may not be useful at the present stage of our investigation.
5. Concluding Remarks
In this paper we have developed the asymptotic expansions of the density functions for the class of semi-parametric estimators including the MEL estimator and the GMM estimator. Although the general forms of the asymptotic expansions are quite compli- cated at the first glance, it is possible to obtain some interesting observations which make possible to compare alternative estimation methods. It would be desirable to investigate the finite sample distributions of estimators and Anderson et. al. (2005), for instance, have investigated the finite sample properties of the distribution functions of the MEL and GMM estimators and have given extensive tables when G 1 = 1 in a systematic way. In the more general case, however, it would not be possible to investi- gate the finite sample properties directly and the asymptotic expansion method we have developed should be useful for comparing different estimators. The explicit formulas we have derived in this paper give some useful information on the exact ditributions of alternative estimators in more general cases. The issue of comparing the finite sample distributions of alternative estimators are closely related to the problem of higher order asymptotic efficiency of estimation. Takeuchi and Morimune (1985) gave the classic result on the simultaneous equations system in the parametric framework and shown that the LIML estimator is third or- der asymptotically efficient when the dusturbances are normally distributed. Recently, Newey and Smith (2004) concluded that the MEL estimator is third order asymptot- ically efficient when the disturbances are the multinomial (lattice) distribution in the more general nonlinear estimating equation framework. In general, the asymptotic ex- pansions of distributions of estimators in the lattice case are different from those in the non-lattice case due to the lack of so-called the Cramér condition (see Chapter 5 of Bhattacharya and Rao (1976), for instance). However, it is important to note that the differences between the distributions of the LIML and MEL estimators are very small as we have discussed in Section 4 at least when the disturbances are i.i.d. non- lattice random variables with zero third moments. It may be interesting to see if these differences would be substantial for the practical purposes. [1] Anderson, T.W. (1974), "An Asymptotic Expansion of the Distribution of the Limited Information Maximum Likelihood Estimate of a Coefficient in a Simulta- neous Equation System," Journal of the American Statistical Asociation , Vol. 69, 565-573. [2] Anderson, T.W. and T. Sawa (1973), "Distributions of Estimates of Coefficients of a Single Equation in a Simultaneous System and Their Asymptotic Expansion," Econometrica , Vol. 41, 683-714. [3] Anderson, T.W., N. Kunitomo, and T. Sawa (1982), "Evaluation of the Distribu- tion Function of the Limited Information Maximum Likelihood Estimator,"  Econo- metrica , Vol. 50, 1009-1027. [4] Anderson, T.W., N. Kunitomo, and K. Morimune (1986), "Comparing Single Equation Estimators in a Simultaneous Equation System," Econometric Theory, Vol. 2, 1-32. T.W., N. Kunitomo, Y. Matsushita “A New [5] Anderson, and (2005), Light from Old Wisdoms : Alternative Estimation Method of Simultane- ous Equations and Microeconometric Models," Discussion Paper CIRJE-F- 321, Graduate School of Economics, University of Tokyo (http://www.e.u- tokyo.ac.jp/cirje/research/dp/2005). [6] Bhattacharya, R.N. and Ghosh, J.K. (1978), "On the Validity of the Formal Edge- worth Expansion," The Annals of Statistics , Vol. 6, 434-451. [7] Bhattacharya, R.N. and R. Rao (1976) Normal Approximations and Asymptotic Expansions, Jhon-Wiley. [8] Fujikoshi, Y., K. Morimune, N. Kunitomo, and M. Taniguchi (1982), "Asymptotic Expansions of the Distributions of the Estimates of Coefficients in a Simultaneous Equation System,"  Journal of Econometrics , Vol. 18, 2, 191-205. [9] Hansen, L. (1982), "Large Sample Properties of Generalized Method of Moments Estimators," Econometrica , Vol. 50, 1029-1054. [10] Hayashi, F. (2000) Econometrics , Princeton University Press. [11] Godambe, V.P. (1960), "An Optimum Property of Regular Maximum Likelihood Equation," Annals of Mathematical Statistics, Vol. 31, 1208-1211. [12] Kitamura, Y. and M. Stutzer (1997), "An Informational-Theoretic Alternative to Genaralized method of Moments," Econometrica , 65, 861-874. [13] Kitamura, Y. G. Tripathi, and H. Ahn (2004), "Empirical Likelihood-Based Infer- ence in Conditional Moment Restriction Models," Econometrica , 72, 1667-1714. [14] Newey, W. K. and R. Smith (2004), "Higher Order Properties of GMM and Gen- eralized Empirical Likelihood Estimator," Econometrica, 72, 219-255. [15]  Owen, A. B. (1990), "Empirical Likelihood Ratio Confidence Regions,"  The Annals of Statistics, Vol. 22, 300-325. [16] Owen, A. B. (2001), Empirical Likelihood, Chapman and Hall. [17] Phillips, P.C.B. (1983), "Exact Small Sample Theory in the Simultaneous Equa- tions Model,"  Handbook of Econometrics , Vol. 1, 449-516, North-Holland. [18] Qin, J. and Lawless, J. (1994), "Empirical Likelihood and General Estimating Equations," The Annals of Statistics, Vol. 22, 300-325. [19] Sargan, J. D. and Mikhail, W.M. (1971), "A General Approximation to the Dis- tribution of Instrumental Variables Estimates," Econometrica , Vol. 39, 131-169. [20] Takeuchi, K. and Morimune, K. (1985), "Third-Order Efficiency of the Extended Maximum Likelihood Estimators in a Simultaneous Equations System," Econo- metrica, Vol. 53, 177-200. In Appendix A, we give the proofs of two lemmas used in Section 4. In Appendix B we gather some useful formulae on the Fourier inversion because it seems they are not readily available. [A-1] : Proof of Lemma 4.1 Let X 1 = (Y n ) ij, X 2 = (AX n ) k and X 3 = ( ě 0 ) t. Since the limiting distribution of random vector (X 1 , X 2 , X 3 ) ' is normal, the conditional distribution of (X 1 , X 2 ) ' given X 3 is also asymptotically normal. Then we have
Because X 2 and X 3 are asymptotically orthogonal, E [ X 2 ] = 0 and Cov ( X 2 , X 3 ) = 0 . Also by using the notation z ( j ) and noting that
we have the result. (Q.E.D) [A-2] :  Proof of Lemma 4.3 Let z n = (u n ,v n )' be a (p + 1) × 1 random vector which is a sum of i.i.d.  random vectors z ( π ) ( j = 1, . . ., n ) :
and E[z ( n )] = 0, E[z ( n ) z ( n ) = Σ > 0. Then under a set of regularity conditions (Bhattacharya and Rao (1976) for instance) the characteristic function of z n can be expressed as
. Then the density function of z n has where βι ′ו′וare the third order moments of z ( n ) a representation
where h 3 ( zι, z r , z r ′ ) are the third-order Hermitian polynomials and we set a ( p + 1) × (p + 1) variance-covariance matrix of z n as
for the mathematical convenience. Let fn(un) be the marginal density and fn(vn|un) be the conditional density, which can be represented as
where φ(v|ρ′ u n , 1 − ρ′ρ) is the conditional density function, and h 3 , ( · ) are the third order Hermitian polynomials for the ( p + 1)−dimensional random vector ( u n, v ) and h3( · ) are the third order Hermitian polynomials for the p –dimensional random vector u n . Then the conditional expectation is given by
By using the integral-by-parts calculations, the third term and the fourth term of the right-hand side of O(n-1/2) are zeros. Hence
ρ Un
where h2( u 1 , u r ) are the second order Hermite polynomials of p —dimensional vector u n . Since two terms in the above expressions on the right-hand side are cancelled out, we have the desired result.  (Q.E.D.) This appendix gives the useful formulas, which correspond to the inversion of the characteristic function from the conditional expectations given z and z follows the p- dimensional normal distribution N p (0, Q) . All inversion results we have needed in Sections 3 and 4 can be reduced the Fourier Inversion formulas for the density function as
for any polynomials h( · ) and g( · ), where t = ( i ) is a p × 1 vector, i 2 = −1, and the differentiation vector The method adopted here was developed by Fujikoshi et.  al.  (1982) and they were given by Anderson et. al. (1986). We present some useful results. Lemma B.1 : Let α′ = (α 1 , · · ,α p ) be a 1 × p constant vector, A be a symmetric constant matrix, and tr : [ · ] stands for ∑ i ∑ j ∂² /∂ξ i ∂ξ j [ · ] ij . Then