# Moment Conditions Selection Based on Adaptive Penalized Empirical Likelihood

Authors: Yunquan Song
Journal: Abstract and Applied Analysis
 (2014)

---

## Abstract

Empirical likelihood is a very popular method and has been widely used in the fields of artificial intelligence (AI) and data mining as tablets and mobile application and social media dominate the technology landscape. This paper proposes an empirical likelihood shrinkage method to efficiently estimate unknown parameters and select correct moment conditions simultaneously, when the model is defined by moment restrictions in which some are possibly misspecified. We show that our method enjoys oracle-like properties; that is, it consistently selects the correct moment conditions and at the same time its estimator is as efficient as the empirical likelihood estimator obtained by all correct moment conditions. Moreover, unlike the GMM, our proposed method allows us to carry out confidence regions for the parameters included in the model without estimating the covariances of the estimators. For empirical implementation, we provide some data-driven procedures for selecting the tuning parameter of the penalty function. The simulation results show that the method works remarkably well in terms of correct moment selection and the finite sample properties of the estimators. Also, a real-life example is carried out to illustrate the new methodology.


---

## Body

## 1. Introduction

As Xie et al. [1] show, growing attention is being paid to the fields of artificial intelligence (AI) and data mining as tablets and mobile application and social media dominate the technology landscape. Moment conditions often appear in the study of artificial intelligence (AI) and data mining. We all know that empirical likelihood is a very practical tool for the study of moment conditions. When a parametric likelihood function is not specified for a model, estimating equations may provide an alternative instrument for statistical inference. For example, let Z
1
,
…
,
Z
n be independent and identically distributed random vectors from a distribution, and let θ
∈
R
p be a vector of unknown parameters. Suppose that the information of the distribution is available in the form of an unbiased estimating function g
(
z
;
θ
)
=
{
g
1
(
z
;
θ
)
,
…
,
g
r
(
z
;
θ
)
}
T satisfying E
{
g
(
Z
;
θ
)
}
=
0 and r
≥
p. When r
=
p, θ can be estimated by solving the estimating equations 0
=
n
-
1
∑
i
=
1
r
g
(
Z
i
;
θ
). Allowing r
>
p provides a useful device to combine available information for improving estimation efficiency, but directly solving 0
=
n
-
1
∑
i
=
1
r
g
(
Z
i
;
θ
) may not be feasible.The generalized method of moments (GMM) and empirical likelihood (EL) are two popular methodologies for estimating the parameters in the structural equations. As was introduced by Hansen [2], the GMM estimator θ
^
n is defined as
(1)
θ
^
G
=
arg
min
⁡
θ
∈
Θ
[
∑
i
=
1
n
g
(
Z
i
,
θ
)
n
]
T
W
n
[
∑
i
=
1
n
g
(
Z
i
,
θ
)
n
]
,
where Θ is the parameter space where θ
0 lies and W
n is a given q
×
q weight matrix. Unlike the GMM, the EL uses likelihood to optimally combine information given in the estimating equations. More specifically, the estimator θ
^
E is defined by maximizing the following empirical likelihood:
(2)
L
(
θ
)
=
sup
⁡
{
∏
i
=
1
n
‍
n
ω
i
:
ω
i
≥
0
,
∑
i
=
1
n
‍
ω
i
=
1
,
∑
i
=
1
n
‍
ω
i
g
(
Z
i
;
θ
)
=
0
}
.
The estimator θ
^
E is optimal in the sense of Godambe and Heyde [3]. It is known that maximizing (2) is equivalent to minimizing the empirical likelihood ratio
(3)
l
(
θ
)
=
-
[
log
⁡
{
L
(
θ
)
-
n
log
⁡
(
n
)
}
]
=
∑
i
=
1
n
‍
log
⁡
{
1
+
λ
θ
T
g
(
Z
i
;
θ
)
}
,
where λ
θ satisfies n
-
1
∑
i
=
1
n
g
(
Z
i
;
θ
)
{
1
+
λ
θ
T
g
(
Z
i
;
θ
)
}
-
1
=
0.Both the GMM and the EL have been successfully used for parameter estimation and variable selection in general estimating equations. The statistical properties of the GMM and the EL estimators rely heavily on the quality of these moment conditions. The strong and valid moment conditions can help to reduce finite-sample bias and improve efficiency of the GMM and the EL estimators. However, when some moment conditions are misspecified, the GMM and the EL estimators may be inconsistent. In this paper, we are interested in estimating some unknown parameterθ
0 identified by a set, set-1, of some moment restrictions which can be used to estimate θ
0 consistently. Meanwhile, it is supposed that there is another set, set-2, of possibly misspecified moment conditions. When the moment conditions in set-2 (or some of them) are correctly specified, including them into estimation equations can improve the asymptotic efficiency of the estimator for θ
0. However, if they are misspecified, then using these moment conditions will lead to inconsistent estimation. Hence, whenever an empirical researcher has a set of moment conditions and there is no prior information about their validity, it is important to have some procedures to select the correctly specified moment conditions in that set and include them in the estimation equations.Note that both the GMM estimators and the EL estimators are defined through moment restrictions. They generally have the same asymptotic distributions, but possibly different higher order asymptotic properties; see Newey and Smith [4] and Schennach [5]. As discussed in Newey and Smith [4], the small sample performance of the GMM is poor in some applications and the EL has advantages over the GMM estimators. First, unlike GMM, the asymptotic bias of the EL estimator does not grow with the number of moment restrictions. Consequently, with many moment conditions, the bias of EL will be less than the bias of GMM. The relatively low asymptotic bias of the EL indicates that it is an important alternative to the GMM. Second, unlike the GMM, the EL does not require weight matrix estimation and is invariant to nonsingular linear transformations of the moment conditions. The third theoretical advantage of EL is that after it is bias corrected, it is higher efficient relative to the GMM bias corrected estimators. The reason is that the biased corrected EL estimators inherit the higher order property of maximum likelihood estimators.Inspired by the idea of Liao [6] and considering the above advantages of the EL estimators relative to the GMM estimators, we propose a novel method for moment selection and parameter estimation simultaneously. The new method attaches a penalty function to the EL criterion and the resulting estimator of θ
0 is then called the EL shrinkage estimator. Our method embeds the moment selection in EL estimation and once a certain moment condition is selected, it will be automatically included into estimating θ
0. Hence, our method not only selects the correct moment conditions in the set-2 in one step but also deals with the moment selection and efficient estimation simultaneously. Under some regularity conditions, we show that the EL shrinkage estimator of θ
0 is root-n consistent and asymptotically normal. Moreover, we show that consistent moment selection is automatically achieved in the penalized EL estimation and the EL shrinkage estimator of θ
0 is asymptotically oracle-efficient (i.e., as efficient as the oracle EL estimator based on all valid moment conditions). Unlike the GMM, our proposed method allows us to carry out confidence regions for parameters included in the model without estimating the covariances of the estimators.The rest of the paper is organized as follows. In Section2, based on the EL and penalty method, the parameter estimation and moment condition selection are introduced. The theoretical properties of the EL shrinkage estimators and the empirical likelihood ratio are presented in Section 3. Section 4 provides simple and data-driven procedures of selecting the tuning parameters. Simulation studies and a real-life example are given in Section 5. Proofs and the technical derivations are included in the appendix.


## 2. Methodology

Suppose that we are interested in estimating some unknown parameterθ
0 identified by the following moment restrictions:
(4)
E
[
g
q
(
Z
,
θ
0
)
]
=
0
,
where Z is a d
z-dimensional random vector, θ
0 is a d
θ-dimensional parameter vector, the subscript q of g
q
(
·
,
·
) denotes the number of moment conditions, and g
q
(
·
,
·
)
:
R
d
z
×
R
d
θ
→
R
q. The moment conditions in (4) can be used to estimate θ
0 consistently. Suppose there is another set of possibly misspecified moment conditions as
(5)
E
[
g
k
(
Z
,
θ
0
)
]
=
?
0
,
where “=
?” signifies that equality may hold for some elements but not others, the subscript k of g
k
(
·
,
·
) denotes the number of moment conditions, and g
k
(
·
,
·
)
:
R
d
z
×
R
d
θ
→
R
k. The goal of this paper is to consistently select the correct moment conditions in the set-2 and automatically include them into the empirical likelihood estimation to improve the efficiency of estimating θ
0.To incorporate moment selection into the estimation procedure, we first introduce a set of auxiliary unknown parametersβ
0 and reparametrize the moment conditions in the set-2 as
(6)
E
[
g
k
(
Z
,
θ
0
)
-
β
0
]
=
0
.
From (6), we see that if the jth (
j
=
1
,
…
,
k
) moment condition in (5) is correctly specified (or misspecified), then β
0
,
j
=
0
(
or
β
0
,
j
≠
0
). Hence, the zero/nonzero components in β
0 can be used to identify the correctly specified/misspecified moment conditions in the set-2 and consistent moment selection is equivalent to consistent selection of the zero components in β
0. We thus stack the moment conditions in (4) and (6) to get
(7)
E
[
ρ
(
Z
,
θ
0
,
β
0
)
]
≡
E
[
(
g
q
(
z
,
θ
0
)
g
k
(
z
,
θ
0
)
-
β
0
)
]
=
0
.Let{
Z
i
}
i
≤
n be a sample of Z. The EL shrinkage estimator (
θ
^
n
S
,
β
^
n
S
) of (
θ
0
,
β
0
) is defined as
(8)
(
θ
^
n
S
,
β
^
n
S
)
=
arg
min
⁡
(
θ
,
β
)
∈
Θ
×
B
∑
i
=
1
n
‍
log
⁡
{
1
+
λ
θ
T
ρ
(
Z
i
,
θ
,
β
)
}
+
n
∑
j
=
1
k
‍
P
τ
n
(
β
j
)
,
where λ
θ satisfies n
-
1
∑
i
=
1
n
ρ
(
Z
i
,
θ
,
β
)
{
1
+
λ
θ
T
ρ
(
Z
i
,
θ
,
β
)
}
-
1
=
0, Θ
×
B is the parameter space where (
θ
0
,
β
0
) lies, and τ
n is the tuning parameter in some general penalty function P
τ
n
(
·
). The success of our method in simultaneous moment selection and efficient estimation relies on the “oracle properties” of the shrinkage techniques. That is to say, if β
0
,
j
=
0, for some j
∈
{
1
,
…
,
k
}, our method will estimate β
0
,
j as zero with probability approaching 1 (w.p.a.1.). When β
0
,
j is estimated as zero w.p.a.1., the information contained in the jth moment condition of (5) is automatically used in estimating θ
0 w.p.a.1. On the other hand, the nonzero components in β
0 are consistently estimated and their estimators are nonzero w.p.a.1. Hence, our method can consistently distinguish the zero and nonzero components in β
0 and is consistent in moment selection. Moreover, it estimates θ
0 as if we knew all potentially correct moment conditions in the set-2.There are many popular choices for the penalty functionP
τ
n
(
·
). For example, the bridge penalty is defined as P
τ
n
(
β
)
=
τ
n
|
β
|
γ, where γ
∈
(
0,1
); the adaptive Lasso penalty is defined as P
τ
n
(
β
)
=
τ
n
ω
^
β
|
β
|, where ω
^
β
=
|
β
^
n
|
-
ω  (
ω
>
0
) and β
^
n is some first-step consistent estimator of β
0; and the smoothly clipped absolute deviation (SCAD) penalty is defined as
(9)
P
τ
n
(
β
)
=
τ
n
|
β
|
I
(
0
≤
|
β
|
≤
τ
n
)
+
a
τ
n
|
β
|
-
(
β
2
+
τ
n
2
)
/
2
a
-
1
×
I
(
τ
n
≤
|
β
|
≤
a
τ
n
)
+
(
a
+
1
)
τ
n
2
2
I
(
|
β
|
≥
a
τ
n
)
,
where a is some positive real number strictly larger than 2. The above penalty functions differ in their empirical implementations, although the related EL shrinkage estimators may have the same asymptotic properties (see the results in Section 3). We focus on the SCAD penalty in this paper.


## 3. Asymptotic Theory

This section establishes the oracle property of the adaptive empirical likelihood (EL) shrinkage estimator. We state our theorems here, but their proofs are relegated to the appendix.LetS
β
≡
{
j
:
β
0
,
j
≠
0
,
j
=
1
,
…
,
k
} and S
β
,
n
≡
{
j
:
β
^
n
,
j
≠
0
,
j
=
1
,
…
,
k
} be the index set of the nonzero components in β
0 and β
^
n, respectively. For ease of notation, we sort the elements in β
0 in the following way: β
0
′
=
(
β
0
,
-
′
,
β
0
,
+
′
), where β
0
,
-
=
0 and β
0
,
+
≠
0. Let k
0 denote the number of valid moment conditions in the set-2. By definition, we know that β
0
,
- and β
0
,
+ are k
0 and k
-
k
0 dimensional vectors, respectively. We define α
′
=
(
θ
′
,
β
′
) and
(10)
m
(
α
)
≡
E
[
ρ
(
Z
,
θ
,
β
)
]
≡
E
(
g
q
(
Z
,
θ
)
g
k
(
Z
,
θ
)
-
β
)
≡
(
G
q
(
θ
)
G
k
(
θ
)
-
β
)
for any (
θ
,
β
)
∈
Θ
×
B. We use ∥
·
∥ to denote the Euclidean norm in the Euclidean space.We first present and discuss the sufficient conditions for consistency ofα
^
n.Assumption 1.
(i)E
[
g
l
(
z
,
θ
0
)
g
l
T
(
z
,
θ
0
)
] is positive definite for l
=
q
,
k;
(ii)∂
g
l
(
z
,
θ
)
/
∂
θ and ∂
2
g
l
(
z
,
θ
)
/
∂
θ
∂
θ
T are continuous in a neighborhood of the true value θ
0 for l
=
q
,
k;
(iii)∥
∂
g
l
(
z
,
θ
)
/
∂
θ
∥, ∥
∂
2
g
l
(
z
,
θ
)
/
∂
θ
∂
θ
T
∥, and ∥
g
l
(
z
,
θ
)
∥
3 are bounded by some integrable function H
(
z
) in this neighborhood of the true value θ
0 for l
=
q
,
k, and the rank of E
[
∂
g
q
+
k
(
z
,
θ
)
/
∂
θ
] is p;
(iv)E
{
sup
⁡
(
θ
,
β
)
∈
Θ
×
B
(
∥
ρ
(
Z
i
,
θ
,
β
)
∥
(
q
+
k
)
-
1
/
2
)
κ
}
<
∞ for some κ
>
10
/
3 when n is large.Assumption1 is similar to those of Qin and Lawless [7]. We emphasize that the dimensionality q
+
k cannot exceed n because the convex hull of {
g
q
+
k
(
Z
i
,
θ
0
)
}
i
=
1
n is at most at a subset in R
n.Assumption 2.
(i)G
k
(
θ
) is continuous in θ and for any ɛ
>
0 there exists some δ
ɛ such that
(11)
inf
⁡
{
θ
∈
Θ
:
|
|
θ
-
θ
0
|
|
≥
ɛ
}
∥
G
q
(
Z
,
θ
)
∥
>
δ
ɛ
;
(ii) the following uniform law of large numbers (ULLN) holds:(12)
sup
⁡
θ
∈
Θ
[
n
-
1
∑
i
=
1
n
‍
{
g
l
(
Z
i
,
θ
)
-
E
[
g
l
(
Z
i
,
θ
)
]
}
]
=
o
p
(
1
)
for l
=
q
,
k;
(iii)W
0
=
(
1
/
2
)
E
[
ρ
(
Z
,
θ
,
β
)
ρ
T
(
Z
,
θ
,
β
)
] is positive definite; W
n is a symmetric and real matrix and its eigenvalues are bounded from below and above by some fixed finite positive constants for all n;
(iv) the penalty functionP
τ
n
(
·
) is nonnegative and P
τ
n
(
β
0
,
j
)
=
o
p
(
1
) for j
=
1
,
…
,
k.Condition (11) in Assumption 2(i) is the identifiable uniqueness condition for θ
0. By definition, β
0
=
G
k
(
θ
0
); thus, β
0 is locally uniquely identified under (11) and the continuity of G
k
(
θ
). Assumptions 2(ii) and (iii) are two conditions whose application range is very wide because it does not specify the data structure, the properties of the moment functions, and the form of the weight matrix W
n. It is clear that when W
n is an identity matrix this assumption holds automatically. We choose W
n
=
(
1
/
2
n
)
∑
i
=
1
n
ρ
(
Z
i
,
θ
,
β
)
ρ
T
(
Z
i
,
θ
,
β
) usually. Assumption 1(iv) implies that the shrinkage effect of the penalty function on the moment selection coefficients (i.e. β
0
,
+) converges in probability to zero as n
→
∞. It states that the nonzero parameters cannot converge to zero too fast. This is reasonable because otherwise the noise is too strong. This condition includes the case that P
τ
n
(
β
0
,
j
)
=
0 for j
=
1
,
…
,
k as a special example.Assumption 3.
(i) The following functional central limit theorem (FCLT) holds:(13)
sup
⁡
θ
∈
Θ
[
n
-
1
/
2
∑
i
=
1
n
‍
{
g
l
(
Z
i
,
θ
)
-
E
[
g
l
(
Z
i
,
θ
)
]
}
]
=
O
P
(
1
)
,
for l
=
q
,
k;
(ii)G
l
(
θ
) is continuously differentiable in some neighborhood of θ
0 for l
=
q
,
k;
(iii)∂
G
l
(
θ
0
)
/
∂
θ
′ has full column rank;
(iv) the penalty functionP
τ
n
(
·
) satisfies P
τ
n
(
0
)
=
0 and is continuously twice differentiable at β
0
,
j for any j
∈
S
β with
(14)
max
⁡
j
∈
S
β
|
P
τ
n
′′
(
β
0
,
j
)
|
=
o
p
(
1
)
.Assumption3(i) can be verified by applying Donsker’s theorem in specific models. Assumption 3(ii) imposes a local differentiability condition on the expectation of the moment function g
l
(
Z
,
θ
),  l
=
q
,
k. Assumption 3(iii) is a local identification condition for θ
0. If this assumption fails, the resulting estimator α
^
n may not be n-consistent. Assumption 3(iv) imposes some local smoothness conditions on the penalty function P
τ
n
(
·
). Intuitively, this condition implies that attaching a penalty function to the empirical likelihood criterion function does not cause any local identification problem for the unknown parameter (
θ
0
,
β
0
). It can be verified that the bridge, adaptive Lasso, and SCAD penalty functions satisfy Assumption 3(iv).Theorem 4.
Under Assumptions1, 2, and 3, as n
→
∞ and with probability tending to 1, the EL shrinkage estimator defined in (8) satisfies(a)
α
^
n
→
α
0 and
(b)
∥
α
^
n
-
α
0
∥
=
O
p
(
δ
n
),where δ
n
=
max
⁡
{
b
n
,
n
-
1
/
2
} and b
n
=
max
⁡
j
∈
S
β
|
P
τ
n
′
(
β
0
,
j
)
|.It is clear from Theorem4 that, by choosing a proper τ
n, there exists a consistent EL shrinkage estimator α
^
n whose convergence rate is of the order δ
n. We now show that this estimator must possess the sparsity property β
^
n
,
j
S
=
0 for all j
∈
S
β
c, which is stated in Theorem 6.Assumption 5.
(i) The tuning parameterτ
n satisfies
(15)
n
max
⁡
j
∈
S
β
|
P
τ
n
′
(
β
0
,
j
)
|
=
o
p
(
1
)
;
(ii) for anyj
∈
S
β
c and any random sequence {
β
j
,
n
}
n with β
j
,
n
≠
0 a.e. for all n and β
j
,
n
=
O
p
(
n
-
1
/
2
), there is
(16)
liminf
⁡
n
→
∞
[
|
P
τ
n
′
(
β
j
,
n
)
|
r
n
τ
n
]
>
0
a.e
.
,
where r
n is some nonnegative sequence such that n
1
/
2
τ
n
r
n
→
∞.Assumption5(i) indicates that the convergence rate of |
P
τ
n
′
(
β
0
,
j
)
| for all j
∈
S
β is faster than n. Under this assumption, Theorem 4 implies that
(17)
n
(
α
^
n
-
α
0
)
=
O
p
(
1
)
;
that is, the convergence rate of α
^
n is n. Assumption 5(ii) is a generalized version of condition (3.5) in Fan and Li [8]. Intuitively, Assumption 5(ii) implies that the shrinkage estimator β
^
n
,
j of β
0
,
j
j
∈
S
β
c is the minimizer of P
τ
n
(
·
) w.p.a.1. From Assumptions 2(iv) and 3(iv), we know that P
τ
n
(
·
) is locally minimized at 0. Hence, Assumption 5(ii) is the key condition needed for showing consistent moment selection. It can be verified that the bridge, adaptive Lasso, and SCAD penalty functions satisfy Assumption 5.Theorem 6.
Under Assumptions1, 2, 3, and 5, one has
(18)
lim
⁡
n
→
∞
P
r
(
β
^
n
,
j
=
0
)
=
1
f
o
r
a
n
y
j
∈
S
β
c
.From the consistency ofβ
^
n and Theorem 6, we can immediately get
(19)
lim
⁡
n
→
∞
Pr
(
S
β
=
S
β
,
n
)
=
1
,
that is, the consistent moment selection. We next provide the conditions needed for deriving the limiting distribution of the EL shrinkage estimator.Assumption 7.
(i) There exists a symmetric, nonrandom, and positive definite matrixW
0 such that
(20)
W
n
⟶
p
W
0
;
(ii) the following central limit theorem (CLT) holds:(21)
n
-
1
/
2
∑
i
=
1
n
‍
{
g
q
+
k
(
Z
i
,
θ
0
)
-
E
[
g
q
+
k
(
Z
i
,
θ
0
)
]
}
⟶
d
Ψ
(
θ
0
)
,
where Ψ
(
θ
0
) is some Gaussian random vector.Assumption7(i) is a regularity condition. Assumption 7(ii) can be verified by applying CLTs in models with specific moment functions and data structure.Next we will consider the oracle properties of the EL shrinkage estimation. The oracle properties state that the EL shrinkage estimation can consistently identify all potentially valid moment conditions in set-2 and its estimator ofθ
0 is as efficient as the oracle EL estimator based on all valid moment conditions. As the consistent moment selection is directly implied by Theorems 4 and 6, the oracle properties follow if we can show that the asymptotic variance-covariance matrix of β
^
n coincides with that of the oracle EL estimator.Letg
d
β
-
(
Z
,
θ
) and g
d
β
+
(
Z
,
θ
) denote the potentially valid and misspecified moment functions in set-2, respectively. We define
(22)
g
q
+
d
β
-
(
Z
,
θ
)
≡
(
g
q
(
Z
,
θ
)
g
d
β
-
(
Z
,
θ
)
)
.
If we had prior information about the validity of the moment conditions in set-2, then there would be q
+
k
0 moment conditions to estimate θ
0. We can stack these moment conditions as
(23)
m
e
(
θ
0
)
=
E
[
g
q
+
d
β
-
(
Z
,
θ
0
)
]
=
0
.
From the moment conditions in (23), we can compute the asymptotic variance-covariance matrix of the optimally oracle EL estimator as
(24)
Σ
*
=
(
[
∂
m
e
(
θ
)
∂
θ
0
′
]
T
V
e
,
o
-
1
[
∂
m
e
(
θ
)
∂
θ
0
′
]
)
-
1
,
where V
e
,
o is the leading (
q
+
k
0
)
×
(
q
+
k
0
) submatrix of E
[
Ψ
(
θ
0
)
Ψ
T
(
θ
0
)
].In the EL shrinkage estimation, if we choose a weight matrixW
n
* such that
(25)
W
n
*
⟶
p
W
0
=
{
E
[
Ψ
(
θ
0
)
Ψ
T
(
θ
0
)
]
}
-
1
,
then an interesting question is whether the resulting empirical likelihood (EL) shrinkage estimator θ
^
n of θ
0 could be as efficient as the optimally weighted oracle EL estimator. The answer to the above question is affirmative, as illustrated in the following theorem.Theorem 8 (Oracle Property).
Under Assumptions1–5, one has
(26)
lim
⁡
n
→
∞
P
r
(
S
β
=
S
β
,
n
)
=
1
.
Furthermore, if the weight matrix W
n satisfies (25) and Assumption 7 holds, then one has
(27)
n
(
θ
^
n
S
-
θ
0
)
⟶
d
N
(
0
,
Σ
*
)
,
where Σ
* is defined in (24).The empirical likelihood method is capable of finding estimators, constructing confidence regions, and testing hypotheses. The following theorem, a generalization of the Wilks theorem, allows us to carry out inference for parameters included in the model without estimating their estimators’ covariance for our proposed method.Theorem 9.
Suppose Assumptions1–7 hold. The empirical likelihood ratio statistic for testing H
0
:
θ
=
θ
0 is
(28)
W
E
(
θ
0
)
=
2
l
E
(
θ
0
)
-
2
l
E
(
θ
^
n
S
)
,
where l
E
(
θ
) is given by l
E
(
θ
)
=
∑
i
=
1
n
log
⁡
[
1
+
λ
T
g
q
+
k
0
(
Z
i
,
θ
)
]. Under Assumptions 1–7, W
E
(
θ
0
)
→
χ
p
2 as n
→
∞, where H
0 is true.Theorem9 allows us to use the EL ratio statistic for testing or obtaining confidence limits for parameters in a completely analogous way to that for parametric likelihood. The asymptotic confidence region of level 1
-
δ for θ
0 is
(29)
{
θ
0
∣
W
E
(
θ
0
)
≤
χ
p
2
(
δ
)
}
,
where χ
p
2
(
δ
) is the (
1
-
δ
) quantile of the chi-square distribution with p degrees of freedom.


## 4. Adaptive Selection of Tuning Parameter

From the results of the previous sections, we see that the tuning parameterλ
n plays an important role in deriving the oracle properties of the EL shrinkage estimator. Assumptions 2(iv), 3(iv), and 5(i)-(ii) are sufficient conditions imposed on λ
n for the oracle properties to hold. However, these conditions do not provide a straightforward mechanism for choosing the tuning parameter λ
n in finite samples. For practical implementation of the shrinkage techniques, it is important to have some procedures of selecting λ
n such that the EL shrinkage estimator not only enjoys the oracle properties asymptotically but also has good finite-sample properties.To choose the penalty parameterλ, some data-driven approaches for selecting tuning parameters need to be proposed. In the following, we will propose empirical likelihood based AIC-type criterion (EmAIC), BIC-type criterion (EmBIC), and Hannan-Quinn information criterion- (HQIC-) type criterion (EmHQIC). They are defined, respectively, as
(30)
EmAIC
(
λ
)
=
GE
L
n
(
λ
)
-
2
|
S
β
,
λ
c
|
,
(31)
EmBIC
(
λ
)
=
GE
L
n
(
λ
)
-
{
log
⁡
(
n
)
}
|
S
β
,
λ
c
|
,
(32)
EmHQIC
(
λ
)
=
GE
L
n
(
λ
)
-
Q
{
log
⁡
log
⁡
(
n
)
}
|
S
β
,
λ
c
|
,
where S
β
,
λ is the index set of nonzero elements in β
^
λ
,
n, S
β
,
λ
c is the complement set of S
β
,
λ, and |
S
β
,
λ
c
| denotes the cardinality of the index set of S
β
,
λ
c and it stands for the number of moment conditions selected by the EL shrinkage method given λ, Q
>
2. GE
L
n
(
λ
) is the generalized empirical likelihood (GEL) statistic proposed in Hong et al. [9], which is defined as GE
L
n
(
λ
)
=
-
2
min
⁡
α
S
β
,
λ
⁡
max
⁡
π
⁡
∑
i
=
1
n
ν
[
π
′
ρ
(
Z
i
,
α
S
β
,
λ
)
], where ν
(
·
) is some concave function and its domain contains 0, π is some q
+
k dimensional vector, and α
S
β
,
λ
=
(
θ
,
β
S
β
,
λ
,
0
) is transferred from α by setting the elements of β whose index belongs to S
β
,
λ
c to be zero.


## 5. Numerical Studies

In this section, we first carry out simulations to demonstrate the performance of our method for finite data sets. We then apply our method to one real dataset. We compare our proposed method with the adaptive EL shrinkage method and the GMM shrinkage method. We find that both the adaptive EL and GMM shrinkage methods can consistently select the correct moment conditions in set-2 and automatically include them into the estimation to improve the efficiency of estimatingθ
0. However, the adaptive EL shrinkage is more efficient relative to the adaptive GMM estimators because of the advantages of EL relative to GMM.


### 5.1. Simulation Example

Example 1.
In this simulation study, the data are generated from the following linear model:(33)
Y
i
=
θ
10
+
θ
20
X
i
+
u
i
,
where
(34)
u
i
~
N
(
0
,
σ
u
2
)
,
X
i
~
N
(
0
,
σ
x
2
)
,
E
[
X
i
u
i
]
≠
0
,
for all i. The available IVs are (
Z
1
,
i
,
Z
2
,
i
), where Z
1
,
i is a scale random variable and Z
2
,
i
=
(
Z
21
,
i
,
Z
22
,
i
) is a random vector. There are two elements in Z
21
,
i which denote the potentially valid IVs and there are eight elements in Z
22
,
i which are misspecified IVs.
In (33), we take (
θ
10
,
θ
20
)
=
(
0.8,0.8
). The random variables X
i, Z
1
,
i, Z
21
,
i, Z
22
,
i
*, and u
i are generated from the following joint normal distribution:
(35)
(
X
i
,
Z
1
,
i
,
Z
21
,
i
,
u
i
,
Z
22
,
i
*
)
′
~
N
(
0
,
Σ
)
,
where the diagonal elements of Σ are 1, E
(
X
i
Z
1
,
i
)
=
σ
z
1
x
,
E
(
X
i
Z
21
,
i
)
=
(
σ
z
2
x
,
σ
z
2
x
), E
(
X
i
u
i
)
=
0.4, and all other elements in Σ are zero. Z
22
,
i is generated by the following equation:
(36)
Z
22
,
i
=
Z
22
,
i
*
+
0.5
u
i
*
l
,
where l is a 1
×
8 vector of ones. The correlation σ
z
j
x of X
i and Z
j
,
i
(
j
=
1,2
) measures the signal strength of the IV Z
j
,
i about the endogenous variable X
i. There is one specification of (
σ
z
1
x
,
σ
z
2
x
) used in the simulation; that is, (
σ
z
1
x
,
σ
z
2
x
)
=
(
0.4,0.4
).
We assume the econometrician knows thatZ
1
,
i is a valid IV, while being unsure about validity of the IVs in Z
2
,
i. Hence, the moment conditions in set-1 are
(37)
E
[
(
Y
i
-
θ
10
-
θ
20
X
i
)
]
=
0
,
E
[
(
Y
i
-
θ
10
-
θ
20
X
i
)
Z
1
,
i
]
=
0
,
while the moment conditions in set-2 are
(38)
E
[
(
Y
i
-
θ
10
-
θ
20
X
i
)
Z
2
,
i
′
]
=
?
0
.
The SCAD penalty is used in the empirical likelihood shrinkage estimation, where the first-step estimators of the moment selection coefficients are from the empirical likelihood estimation using the moment conditions in (37) and the reparametrized moment conditions in (38).For the specification of(
σ
z
1
x
,
σ
z
2
x
), we use the simulated samples with sample sizes n
=
100 and n
=
500, respectively, in our simulation study, and for each sample size, 2000 simulated samples are drawn from the data generating mechanism. With each simulated sample, we calculate four different types of estimators, which include the oracle estimator, empirical likelihood estimator, empirical likelihood shrinkage estimator using λ
n selected by EmAIC, and EL shrinkage estimator using λ
n selected by EmBIC. The oracle estimator is an EL estimator based on the moment conditions in set-1 and all valid moment conditions in set-2. The EL estimator is an EL estimator based only on the moment conditions in the set-1. Given the specification of (
σ
z
1
x
,
σ
z
2
x
) and the sample size n, we can get 2000 estimators of (
θ
1,0
,
θ
2,0
) for each type of estimator using the 2000 simulated samples. Hence, we can estimate the finite sample marginal densities of different estimators for (
θ
1,0
,
θ
2,0
) and the simulation results are presented in Figures 1 and 2. Tables 1 and 2 contain the selection probabilities of the correct, underselected, and overselected sets of moment conditions in EL (GMM) shrinkage estimation using the tuning parameters from EmAIC (GMM-AIC) and EmBIC (GMM-BIC), respectively.Table 1
The selection probabilities of adaptive EL shrinkage estimation.
The selection probabilities
Case
The correct
Underselected
Overselected
P-LA
n
=
100
0.46
0.53
0.01
n
=
500
0.64
0.36
0.00
P-LB
n
=
100
0.61
0.30
0.09
n
=
500
0.89
0.11
0.00
1.P-LA and P-LB contain the selection probabilities of the correct, underselected, and overselected sets of moment conditions in EL (GMM) shrinkage estimation using the tuning parameters from EmAIC (GMM-AIC) and EmBIC (GMM-BIC), respectively.Table 2
The selection probabilities of adaptive GMM shrinkage estimation.
The selection probabilities
Case
The correct
Underselected
Overselected
P-LA
n
=
100
0.44
0.55
0.01
n
=
500
0.62
0.38
0.00
P-LB
n
=
100
0.58
0.31
0.11
n
=
500
0.86
0.14
0.00
1.P-LA and P-LB contain the selection probabilities of the correct, underselected, and overselected sets of moment conditions in EL (GMM) shrinkage estimation using the tuning parameters from EmAIC (GMM-AIC) and EmBIC (GMM-BIC), respectively.Finite sample densities of EL and EL shrinkage estimators forθ
2,0 in the cases of example (1). The SCAD penalty is used in the EL shrinkage estimation and the first-step estimators of moment selection coefficients are the EL estimators; (2) oracle estimators are the EL estimators using the moment conditions in set-1 and all correct moment conditions in set-2; (3) EL estimators only use the moment conditions in set-1; (4) SEL-A refers to the EL shrinkage estimators using tuning parameters selected by minimizing EmAIC; (5) SEL-B refers to the EL shrinkage estimators using tuning parameters selected by minimizing EmBIC.
(a)
(b)Finite sample densities of EL and EL shrinkage estimators forθ
1
,
0 in the cases of example (1). The SCAD penalty is used in the EL shrinkage estimation and the first-step estimators of moment selection coefficients are the EL estimators; (2) oracle estimators are the EL estimators using the moment conditions in set-1 and all correct moment conditions in set-2; (3) EL estimators only use the moment conditions in set-1; (4) SEL-A refers to the EL shrinkage estimators using tuning parameters selected by minimizing EmAIC; (5) SEL-B refers to the EL shrinkage estimators using tuning parameters selected by minimizing EmBIC.
(a)
(b)There are several remarks we can make based on the simulation results presented in Figures1 and 2. First, when the signal strength of the moment conditions in set-1 is E
[
X
i
Z
1
,
i
]
=
0.4, the EL shrinkage method selects all valid moment conditions in set-2 with high probability and selects the overselected sets of moment conditions with low probability. Second, when the sample size n is increased from 100 to 500, the probability of selecting the set of valid moment conditions in set-2 increases greatly and the probability of selecting the overselected or underselected sets of moment conditions decreases sharply. Third, if we compare the EL shrinkage estimators based on different data-driven procedures of selecting the tuning parameter, we see that the EL shrinkage estimation using the tuning parameters from EmAIC has lower probability of selecting inconsistent sets of moment conditions. But it has nontrivial probability of selecting underselected sets of moment conditions, even when the sample size is increased from 100 to 500. On the other hand, the EL shrinkage estimation using the tuning parameters from EmBIC has lower probability of selecting the overselected sets of moment conditions and higher probability of selecting the set of correct moment conditions, but its probability of selecting the overselected sets of moment conditions is higher. Fourth, the finite sample densities of the EL shrinkage estimators behave much better than those of the EL estimators in all scenarios of this simulation study. Comparing the EL shrinkage estimator with the EL estimator, the most obvious improvement is the reduction of the variance, as we can see from the finite sample densities depicted in Figures 1 and 2. Also note that when the sample size is increased, the finite sample densities of the EL shrinkage estimators are approaching those of the oracle EL estimators. Finally, when the moment conditions in (37) are E
[
X
i
Z
1
,
i
]
=
0.4, the densities of the EL shrinkage estimators θ
^
1
,
n of θ
1,0 are almost the same as those of the oracle estimators and the EL estimators. This is because the moment conditions in set-2 only contain the information about θ
2,0. Hence, when θ
2,0 could be reliably estimated using the set-1 moment conditions, the extra valid moment conditions in set-2 do not help to reduce the variances of the estimators of θ
1,0.


### 5.2. Real Data Example

Example 2.
We apply the EL shrinkage method to study the following labor supply equation in the life-cycle labor supply model [6, 10, 11]:
(39)
Δ
log
⁡
(
h
i
,
t
)
=
α
t
+
Δ
log
⁡
(
ω
i
,
t
)
δ
0
+
ɛ
i
,
t
,
where h
i
,
t is the annual hours working for money, ω
i
,
t is the hourly wage rate of individual i at period t, α
t is a time varying constant, ɛ
i
,
t is the time varying error term, and δ
0 measures the intertemporal substitution elasticity of labor supply with respect to the evolutionary wage changes and the theoretical prediction for its sign is positive.
Due to the measurement errors inω
i
,
t, the OLS estimator of (39) may be inconsistent. MaCurdy [10] proposes to use a set of family background variables to construct the set-1 moment conditions; we only use the parents’ economic status as the credibly valid IV and include the rest of them into set-2. We also include the alternative measure of wage ω
i
,
t
* and the wage ω
i
,
t itself into set-2. Our sample is constructed from the Michigan Panel Study of Income Dynamics (PSID) dataset from year 1970 to year 1981.
We next apply the EL shrinkage estimation to the labor supply equation (39). The estimators of the moment selection coefficients are included in Table 3. As a comparison, we also include the EL estimators of the moment selection coefficients in different specifications of α
t in Table 3. In the first two rows of Table 3, the constant term α
t in (39) is treated to be time variant, while in its last two rows, α
t is taken to be a time invariant constant. From Table 3, we see that the EL estimators of the moment selection coefficients are nonzero and it is hard to determine which moment conditions are valid (misspecified) based on these estimators. On the other hand, the EL shrinkage estimation gives the same moment selection result in the different specifications of α
t. The moment conditions constructed from the IVs by MaCurdy [10] and Altonji [11] are picked up by our shrinkage method, while the moment condition constructed using the imputed wage ω
i
,
t is not selected, which implies that Δ
log
⁡
(
ω
i
,
t
) is an endogenous variable in the labor supply equation (39).
The results of the EL shrinkage estimation of the labor supply equation (39) are contained in Table 4. As a comparison, we also include the EL estimators of δ
0 based on the moment condition in set-1 and the postmoment selection EL (PEL) estimators of δ
0 in Table 4. Columns (1)-(2) of Table 4 present the EL estimators of δ
0 based on the following IV: parent’s economic status when individual was young, which provides the moment condition in set-1. Compared with other estimators in Table 4, the EL estimators in columns (1)-(2) are larger in magnitude and have larger standard errors. On the other hand, the EL shrinkage estimators in columns (3)-(4) have much smaller standard errors, because some moment conditions in set-2 are selected and automatically included into estimation by the EL shrinkage method.
From Table4, we see that, compared with the EL estimators, the EL shrinkage estimators of β
7,0 are closer to zero, which implies that part of the information in the moment condition constructed by ω
i
,
t is indeed used in the EL shrinkage estimation. Based on the above reasoning, we can deduce that the shrinkage effect of the penalty function on the estimators of β
7,0 may introduce some bias to the estimator of δ
0. To get rid of this bias, we conduct another EL estimation based on the moment condition in set-1 and the moment conditions in set-2 selected by our method. These PEL estimators are included in columns (5)-(6) of Table 4. We can see that the PEL estimators are slightly larger in magnitude than the EL shrinkage estimators and their standard errors are almost the same.Table 3
EL and GMM shrinkage estimators of the moment selection coefficients1.
Method
IV
edu
edu2
edu_f
Age
edu* age
ω
i
,
t
*
ω
i
,
t
β
0
β
1,0
β
2,0
β
3,0
β
4,0
β
5,0
β
6,0
β
7,0
ada-EL
EL2
−0.0016
−0.0145
−0.0022
0.0049
−0.0123
−0.0008
−0.0441
(0.0071)
(0.0518)
(0.0057)
(0.1145)
(0.4698)
(0.0074)
(0.0996)
0.0000
0.0000
0.0000
0.0000
0.0000
0.0000
−0.0306
SEL2,4
(0.0000)
(0.0000)
(0.0000)
(0.0000)
(0.0000)
(0.0000)
(0.0097)
EL3
−0.0030
0.0251
−0.0023
0.0357
0.0385
−0.0015
0.0496
(0.0094)
(0.0553)
(0.0065)
(0.2165)
(0.6342)
(0.0091)
(0.1143)
0.0000
0.0000
0.0000
0.0000
0.0000
0.0000
−0.0273
SEL3,5
(0.0000)
(0.0000)
(0.0000)
(0.0000)
(0.0000)
(0.0000)
(0.0081)
ada-GMM
GMM2
−0.0019
−0.0145
−0.0021
0.0048
−0.0123
−0.0007
−0.0442
(0.0073)
(0.0518)
(0.0057)
(0.1145)
(0.4696)
(0.0071)
(0.0995)
0.0000
0.0000
0.0000
0.0000
0.0000
0.0000
−0.0306
SGMM2,4
(0.0000)
(0.0000)
(0.0000)
(0.0000)
(0.0000)
(0.0000)
(0.0097)
GMM3
−0.0034
0.0253
−0.0025
0.0357
0.0382
−0.0018
0.0496
(0.0095)
(0.0558)
(0.0063)
(0.2166)
(0.6348)
(0.0094)
(0.1142)
0.0000
0.0000
0.0000
0.0000
0.0000
0.0000
−0.0273
SGMM3,5
(0.0000)
(0.0000)
(0.0000)
(0.0000)
(0.0000)
(0.0000)
(0.0076)
For detailed instructions see Remark3.Table 4
EL and GMM shrinkage estimation of the labor supply equation1.
Method
IV
EL3
EL3
SEL4
SEL4
P-EL5
P-EL5
(1)
(2)
(3)
(4)
(5)
(6)
Adaptive-EL
a
−0.0157
−0.0226
−0.0117
−0.0226
−0.0165
−0.0187
(0.0253)
(0.0238)
(0.0052)
(0.0196)
(0.0195)
(0.0191)
δ
0
0.3243
0.2685
0.0697
0.1832
0.1395
0.1726
(1.227)
(1.145)
(0.1469)
(0.1923)
(0.1455)
(0.1397)
d
t
?
2
No
Yes
No
Yes
No
Yes
Adaptive-GMM
a
−0.0157
−0.0223
−0.0116
−0.0228
−0.0165
−0.0192
(0.025)
(0.0238)
(0.0049)
(0.0199)
(0.0196)
(0.0191)
δ
0
0.3240
0.2679
0.0697
0.1827
0.1398
0.1721
(1.223)
(1.141)
(0.1469)
(0.1918)
(0.1455)
(0.1393)
d
t
?
2
No
Yes
No
Yes
No
Yes
For detailed instructions see Remark4.Remark 3.
We now give detailed instructions about some marks in Table3.(1)
Standard errors are in parentheses andn
=
3487.
(2)
EL (GMM) estimation with the time dummy variables.
(3)
EL (GMM) estimation without time dummy variables.
(4)
EL (GMM) shrinkage estimation with time dummy variables, where the penalty function is the SCAD and the tuning parameter equals 0.000374 (selected by EmAIC (GMM-AIC), EmBIC (GMM-BIC), and EmHQIC (GMM-HQ)).
(5)
EL (GMM) shrinkage estimation without time dummy variables, where the penalty function is the SCAD and the tuning parameters equals 0.000948 (selected by EmAIC (GMM-AIC), EmBIC (GMM-BIC), and EmHQIC (GMM-HQ)).Remark 4.
We now give detailed instructions about some marks in Table4.(1)
Standard errors are in parentheses and sample sizen
=
3487.
(2)
d
t refers to the set of time dummy variables for the years from 1971 to 1981.
(3)
EL (GMM) is the EL (GMM) estimation only using the moment conditions in set-1.
(4)
SEL (SGMM) denotes the EL (GMM) shrinkage estimation based on the SCAD penalty. In column (3) the tuning parameter equals 0.000949 and in column (4) the tuning parameter equals 0.000374. EmAIC (GMM-AIC), EmBIC (GMM-BIC), and EmHQIC (GMM-HQ) produce the same number of the tuning parameter in each case.
(5)
PEL (P-GMM) denotes the EL (GMM) estimation based on the moment conditions selected by the EL (GMM) shrinkage estimation. The results in columns (5) and (6) are based on the moment conditions selected in (3) and (4), respectively.


## 5.1. Simulation Example

Example 1.
In this simulation study, the data are generated from the following linear model:(33)
Y
i
=
θ
10
+
θ
20
X
i
+
u
i
,
where
(34)
u
i
~
N
(
0
,
σ
u
2
)
,
X
i
~
N
(
0
,
σ
x
2
)
,
E
[
X
i
u
i
]
≠
0
,
for all i. The available IVs are (
Z
1
,
i
,
Z
2
,
i
), where Z
1
,
i is a scale random variable and Z
2
,
i
=
(
Z
21
,
i
,
Z
22
,
i
) is a random vector. There are two elements in Z
21
,
i which denote the potentially valid IVs and there are eight elements in Z
22
,
i which are misspecified IVs.
In (33), we take (
θ
10
,
θ
20
)
=
(
0.8,0.8
). The random variables X
i, Z
1
,
i, Z
21
,
i, Z
22
,
i
*, and u
i are generated from the following joint normal distribution:
(35)
(
X
i
,
Z
1
,
i
,
Z
21
,
i
,
u
i
,
Z
22
,
i
*
)
′
~
N
(
0
,
Σ
)
,
where the diagonal elements of Σ are 1, E
(
X
i
Z
1
,
i
)
=
σ
z
1
x
,
E
(
X
i
Z
21
,
i
)
=
(
σ
z
2
x
,
σ
z
2
x
), E
(
X
i
u
i
)
=
0.4, and all other elements in Σ are zero. Z
22
,
i is generated by the following equation:
(36)
Z
22
,
i
=
Z
22
,
i
*
+
0.5
u
i
*
l
,
where l is a 1
×
8 vector of ones. The correlation σ
z
j
x of X
i and Z
j
,
i
(
j
=
1,2
) measures the signal strength of the IV Z
j
,
i about the endogenous variable X
i. There is one specification of (
σ
z
1
x
,
σ
z
2
x
) used in the simulation; that is, (
σ
z
1
x
,
σ
z
2
x
)
=
(
0.4,0.4
).
We assume the econometrician knows thatZ
1
,
i is a valid IV, while being unsure about validity of the IVs in Z
2
,
i. Hence, the moment conditions in set-1 are
(37)
E
[
(
Y
i
-
θ
10
-
θ
20
X
i
)
]
=
0
,
E
[
(
Y
i
-
θ
10
-
θ
20
X
i
)
Z
1
,
i
]
=
0
,
while the moment conditions in set-2 are
(38)
E
[
(
Y
i
-
θ
10
-
θ
20
X
i
)
Z
2
,
i
′
]
=
?
0
.
The SCAD penalty is used in the empirical likelihood shrinkage estimation, where the first-step estimators of the moment selection coefficients are from the empirical likelihood estimation using the moment conditions in (37) and the reparametrized moment conditions in (38).For the specification of(
σ
z
1
x
,
σ
z
2
x
), we use the simulated samples with sample sizes n
=
100 and n
=
500, respectively, in our simulation study, and for each sample size, 2000 simulated samples are drawn from the data generating mechanism. With each simulated sample, we calculate four different types of estimators, which include the oracle estimator, empirical likelihood estimator, empirical likelihood shrinkage estimator using λ
n selected by EmAIC, and EL shrinkage estimator using λ
n selected by EmBIC. The oracle estimator is an EL estimator based on the moment conditions in set-1 and all valid moment conditions in set-2. The EL estimator is an EL estimator based only on the moment conditions in the set-1. Given the specification of (
σ
z
1
x
,
σ
z
2
x
) and the sample size n, we can get 2000 estimators of (
θ
1,0
,
θ
2,0
) for each type of estimator using the 2000 simulated samples. Hence, we can estimate the finite sample marginal densities of different estimators for (
θ
1,0
,
θ
2,0
) and the simulation results are presented in Figures 1 and 2. Tables 1 and 2 contain the selection probabilities of the correct, underselected, and overselected sets of moment conditions in EL (GMM) shrinkage estimation using the tuning parameters from EmAIC (GMM-AIC) and EmBIC (GMM-BIC), respectively.Table 1
The selection probabilities of adaptive EL shrinkage estimation.
The selection probabilities
Case
The correct
Underselected
Overselected
P-LA
n
=
100
0.46
0.53
0.01
n
=
500
0.64
0.36
0.00
P-LB
n
=
100
0.61
0.30
0.09
n
=
500
0.89
0.11
0.00
1.P-LA and P-LB contain the selection probabilities of the correct, underselected, and overselected sets of moment conditions in EL (GMM) shrinkage estimation using the tuning parameters from EmAIC (GMM-AIC) and EmBIC (GMM-BIC), respectively.Table 2
The selection probabilities of adaptive GMM shrinkage estimation.
The selection probabilities
Case
The correct
Underselected
Overselected
P-LA
n
=
100
0.44
0.55
0.01
n
=
500
0.62
0.38
0.00
P-LB
n
=
100
0.58
0.31
0.11
n
=
500
0.86
0.14
0.00
1.P-LA and P-LB contain the selection probabilities of the correct, underselected, and overselected sets of moment conditions in EL (GMM) shrinkage estimation using the tuning parameters from EmAIC (GMM-AIC) and EmBIC (GMM-BIC), respectively.Finite sample densities of EL and EL shrinkage estimators forθ
2,0 in the cases of example (1). The SCAD penalty is used in the EL shrinkage estimation and the first-step estimators of moment selection coefficients are the EL estimators; (2) oracle estimators are the EL estimators using the moment conditions in set-1 and all correct moment conditions in set-2; (3) EL estimators only use the moment conditions in set-1; (4) SEL-A refers to the EL shrinkage estimators using tuning parameters selected by minimizing EmAIC; (5) SEL-B refers to the EL shrinkage estimators using tuning parameters selected by minimizing EmBIC.
(a)
(b)Finite sample densities of EL and EL shrinkage estimators forθ
1
,
0 in the cases of example (1). The SCAD penalty is used in the EL shrinkage estimation and the first-step estimators of moment selection coefficients are the EL estimators; (2) oracle estimators are the EL estimators using the moment conditions in set-1 and all correct moment conditions in set-2; (3) EL estimators only use the moment conditions in set-1; (4) SEL-A refers to the EL shrinkage estimators using tuning parameters selected by minimizing EmAIC; (5) SEL-B refers to the EL shrinkage estimators using tuning parameters selected by minimizing EmBIC.
(a)
(b)There are several remarks we can make based on the simulation results presented in Figures1 and 2. First, when the signal strength of the moment conditions in set-1 is E
[
X
i
Z
1
,
i
]
=
0.4, the EL shrinkage method selects all valid moment conditions in set-2 with high probability and selects the overselected sets of moment conditions with low probability. Second, when the sample size n is increased from 100 to 500, the probability of selecting the set of valid moment conditions in set-2 increases greatly and the probability of selecting the overselected or underselected sets of moment conditions decreases sharply. Third, if we compare the EL shrinkage estimators based on different data-driven procedures of selecting the tuning parameter, we see that the EL shrinkage estimation using the tuning parameters from EmAIC has lower probability of selecting inconsistent sets of moment conditions. But it has nontrivial probability of selecting underselected sets of moment conditions, even when the sample size is increased from 100 to 500. On the other hand, the EL shrinkage estimation using the tuning parameters from EmBIC has lower probability of selecting the overselected sets of moment conditions and higher probability of selecting the set of correct moment conditions, but its probability of selecting the overselected sets of moment conditions is higher. Fourth, the finite sample densities of the EL shrinkage estimators behave much better than those of the EL estimators in all scenarios of this simulation study. Comparing the EL shrinkage estimator with the EL estimator, the most obvious improvement is the reduction of the variance, as we can see from the finite sample densities depicted in Figures 1 and 2. Also note that when the sample size is increased, the finite sample densities of the EL shrinkage estimators are approaching those of the oracle EL estimators. Finally, when the moment conditions in (37) are E
[
X
i
Z
1
,
i
]
=
0.4, the densities of the EL shrinkage estimators θ
^
1
,
n of θ
1,0 are almost the same as those of the oracle estimators and the EL estimators. This is because the moment conditions in set-2 only contain the information about θ
2,0. Hence, when θ
2,0 could be reliably estimated using the set-1 moment conditions, the extra valid moment conditions in set-2 do not help to reduce the variances of the estimators of θ
1,0.


## 5.2. Real Data Example

Example 2.
We apply the EL shrinkage method to study the following labor supply equation in the life-cycle labor supply model [6, 10, 11]:
(39)
Δ
log
⁡
(
h
i
,
t
)
=
α
t
+
Δ
log
⁡
(
ω
i
,
t
)
δ
0
+
ɛ
i
,
t
,
where h
i
,
t is the annual hours working for money, ω
i
,
t is the hourly wage rate of individual i at period t, α
t is a time varying constant, ɛ
i
,
t is the time varying error term, and δ
0 measures the intertemporal substitution elasticity of labor supply with respect to the evolutionary wage changes and the theoretical prediction for its sign is positive.
Due to the measurement errors inω
i
,
t, the OLS estimator of (39) may be inconsistent. MaCurdy [10] proposes to use a set of family background variables to construct the set-1 moment conditions; we only use the parents’ economic status as the credibly valid IV and include the rest of them into set-2. We also include the alternative measure of wage ω
i
,
t
* and the wage ω
i
,
t itself into set-2. Our sample is constructed from the Michigan Panel Study of Income Dynamics (PSID) dataset from year 1970 to year 1981.
We next apply the EL shrinkage estimation to the labor supply equation (39). The estimators of the moment selection coefficients are included in Table 3. As a comparison, we also include the EL estimators of the moment selection coefficients in different specifications of α
t in Table 3. In the first two rows of Table 3, the constant term α
t in (39) is treated to be time variant, while in its last two rows, α
t is taken to be a time invariant constant. From Table 3, we see that the EL estimators of the moment selection coefficients are nonzero and it is hard to determine which moment conditions are valid (misspecified) based on these estimators. On the other hand, the EL shrinkage estimation gives the same moment selection result in the different specifications of α
t. The moment conditions constructed from the IVs by MaCurdy [10] and Altonji [11] are picked up by our shrinkage method, while the moment condition constructed using the imputed wage ω
i
,
t is not selected, which implies that Δ
log
⁡
(
ω
i
,
t
) is an endogenous variable in the labor supply equation (39).
The results of the EL shrinkage estimation of the labor supply equation (39) are contained in Table 4. As a comparison, we also include the EL estimators of δ
0 based on the moment condition in set-1 and the postmoment selection EL (PEL) estimators of δ
0 in Table 4. Columns (1)-(2) of Table 4 present the EL estimators of δ
0 based on the following IV: parent’s economic status when individual was young, which provides the moment condition in set-1. Compared with other estimators in Table 4, the EL estimators in columns (1)-(2) are larger in magnitude and have larger standard errors. On the other hand, the EL shrinkage estimators in columns (3)-(4) have much smaller standard errors, because some moment conditions in set-2 are selected and automatically included into estimation by the EL shrinkage method.
From Table4, we see that, compared with the EL estimators, the EL shrinkage estimators of β
7,0 are closer to zero, which implies that part of the information in the moment condition constructed by ω
i
,
t is indeed used in the EL shrinkage estimation. Based on the above reasoning, we can deduce that the shrinkage effect of the penalty function on the estimators of β
7,0 may introduce some bias to the estimator of δ
0. To get rid of this bias, we conduct another EL estimation based on the moment condition in set-1 and the moment conditions in set-2 selected by our method. These PEL estimators are included in columns (5)-(6) of Table 4. We can see that the PEL estimators are slightly larger in magnitude than the EL shrinkage estimators and their standard errors are almost the same.Table 3
EL and GMM shrinkage estimators of the moment selection coefficients1.
Method
IV
edu
edu2
edu_f
Age
edu* age
ω
i
,
t
*
ω
i
,
t
β
0
β
1,0
β
2,0
β
3,0
β
4,0
β
5,0
β
6,0
β
7,0
ada-EL
EL2
−0.0016
−0.0145
−0.0022
0.0049
−0.0123
−0.0008
−0.0441
(0.0071)
(0.0518)
(0.0057)
(0.1145)
(0.4698)
(0.0074)
(0.0996)
0.0000
0.0000
0.0000
0.0000
0.0000
0.0000
−0.0306
SEL2,4
(0.0000)
(0.0000)
(0.0000)
(0.0000)
(0.0000)
(0.0000)
(0.0097)
EL3
−0.0030
0.0251
−0.0023
0.0357
0.0385
−0.0015
0.0496
(0.0094)
(0.0553)
(0.0065)
(0.2165)
(0.6342)
(0.0091)
(0.1143)
0.0000
0.0000
0.0000
0.0000
0.0000
0.0000
−0.0273
SEL3,5
(0.0000)
(0.0000)
(0.0000)
(0.0000)
(0.0000)
(0.0000)
(0.0081)
ada-GMM
GMM2
−0.0019
−0.0145
−0.0021
0.0048
−0.0123
−0.0007
−0.0442
(0.0073)
(0.0518)
(0.0057)
(0.1145)
(0.4696)
(0.0071)
(0.0995)
0.0000
0.0000
0.0000
0.0000
0.0000
0.0000
−0.0306
SGMM2,4
(0.0000)
(0.0000)
(0.0000)
(0.0000)
(0.0000)
(0.0000)
(0.0097)
GMM3
−0.0034
0.0253
−0.0025
0.0357
0.0382
−0.0018
0.0496
(0.0095)
(0.0558)
(0.0063)
(0.2166)
(0.6348)
(0.0094)
(0.1142)
0.0000
0.0000
0.0000
0.0000
0.0000
0.0000
−0.0273
SGMM3,5
(0.0000)
(0.0000)
(0.0000)
(0.0000)
(0.0000)
(0.0000)
(0.0076)
For detailed instructions see Remark3.Table 4
EL and GMM shrinkage estimation of the labor supply equation1.
Method
IV
EL3
EL3
SEL4
SEL4
P-EL5
P-EL5
(1)
(2)
(3)
(4)
(5)
(6)
Adaptive-EL
a
−0.0157
−0.0226
−0.0117
−0.0226
−0.0165
−0.0187
(0.0253)
(0.0238)
(0.0052)
(0.0196)
(0.0195)
(0.0191)
δ
0
0.3243
0.2685
0.0697
0.1832
0.1395
0.1726
(1.227)
(1.145)
(0.1469)
(0.1923)
(0.1455)
(0.1397)
d
t
?
2
No
Yes
No
Yes
No
Yes
Adaptive-GMM
a
−0.0157
−0.0223
−0.0116
−0.0228
−0.0165
−0.0192
(0.025)
(0.0238)
(0.0049)
(0.0199)
(0.0196)
(0.0191)
δ
0
0.3240
0.2679
0.0697
0.1827
0.1398
0.1721
(1.223)
(1.141)
(0.1469)
(0.1918)
(0.1455)
(0.1393)
d
t
?
2
No
Yes
No
Yes
No
Yes
For detailed instructions see Remark4.Remark 3.
We now give detailed instructions about some marks in Table3.(1)
Standard errors are in parentheses andn
=
3487.
(2)
EL (GMM) estimation with the time dummy variables.
(3)
EL (GMM) estimation without time dummy variables.
(4)
EL (GMM) shrinkage estimation with time dummy variables, where the penalty function is the SCAD and the tuning parameter equals 0.000374 (selected by EmAIC (GMM-AIC), EmBIC (GMM-BIC), and EmHQIC (GMM-HQ)).
(5)
EL (GMM) shrinkage estimation without time dummy variables, where the penalty function is the SCAD and the tuning parameters equals 0.000948 (selected by EmAIC (GMM-AIC), EmBIC (GMM-BIC), and EmHQIC (GMM-HQ)).Remark 4.
We now give detailed instructions about some marks in Table4.(1)
Standard errors are in parentheses and sample sizen
=
3487.
(2)
d
t refers to the set of time dummy variables for the years from 1971 to 1981.
(3)
EL (GMM) is the EL (GMM) estimation only using the moment conditions in set-1.
(4)
SEL (SGMM) denotes the EL (GMM) shrinkage estimation based on the SCAD penalty. In column (3) the tuning parameter equals 0.000949 and in column (4) the tuning parameter equals 0.000374. EmAIC (GMM-AIC), EmBIC (GMM-BIC), and EmHQIC (GMM-HQ) produce the same number of the tuning parameter in each case.
(5)
PEL (P-GMM) denotes the EL (GMM) estimation based on the moment conditions selected by the EL (GMM) shrinkage estimation. The results in columns (5) and (6) are based on the moment conditions selected in (3) and (4), respectively.



---

*Source: 391719-2014-07-13.xml*