# The Empirical Cressie-Read Test Statistics for Longitudinal Generalized Linear Models

Authors: Junhua Zhang; Ruiqin Tian; Suigen Yang; Sanying Feng
Journal: Journal of Applied Mathematics
 (2014)

---

## Abstract

For the marginal longitudinal generalized linear models (GLMs), we develop the empirical Cressie-Read (ECR) test statistic approach which has been proposed for the independent identically
distributed (i.i.d.) case. The ECR test statistic includes empirical likelihood as a special case. By adopting this ECR test statistic approach and taking into account the within-subject correlation, the efficiency theory results of estimation and testing based on ECR are established under some regularity conditions. Although a working
correlation matrix is assumed, there is no need to estimate the nuisance parameters in the working correlation matrix based on the quadratic inference function (QIF). Therefore, the proposed ECR test statistic is asymptotically a standardχ
2 limit under the null hypothesis. It is shown that the proposed method is more efficient even when the working correlation matrix is misspecified. We also evaluate the finite sample performance of the proposed methods via simulation studies and a real data analysis.


---

## Body

## 1. Introduction

Longitudinal studies are increasingly common in many scientific research fields, including epidemiology, econometrics, medicine, and life and social sciences. For example, longitudinal studies are often used in psychology to study developmental trends across the life span, in sociology to study life events throughout lifetimes or generations, and in medical area to take different treatments at the start of the study and to see what kind of effects the assigned patients have with each treatment by week or by year.Longitudinal data modeling is a statistical method often used in the experiments that are designed such that responses on the same experimental units are observed at each repetition. However, generalized linear models (GLMs [1]) have become a favored tool for the modelling of clustered and longitudinal data by allowing for the non-Gaussian data and nonlinear link functions, such as binomial or Poisson’s type responses. The intrinsic complexity of longitudinal GLMs makes them challenging. First, one characteristic of longitudinal data is the within-subject correlation among the repeated measurements. Ignoring this within-subject correlation causes a loss of efficiency in general problems, and the presence of correlation makes it hard to establish the underlying asymptotic theory. Second, the full likelihood for longitudinal data is often difficult to specify, particularly for the correlated non-Gaussian data. Researchers had developed appropriate methods and criteria for longitudinal GLMs, where generalized estimating equations (GEE [2, 3]) or a recently proposed related method based on quadratic inference functions (QIF; see [4]) is an attractive option for longitudinal regression, particularly when marginal relationships rather than within-subject correlation are of primary interest.In the present paper, we consider the popular marginal longitudinal GLMs. Suppose thatY
i
=
(
y
i
1
,
…
,
y
i
m
i
)
T is the multivariate response for the i subject and X
i
=
(
x
i
1
,
…
,
x
i
m
i
)
T is the m
i
×
p matrix of the covariates for the ith subject (i
=
1
,
…
,
n). Observations from different subjects are independent, but those from the same subjects are correlated. Assume that the marginal mean of y
i
j is
(1)
E
(
y
i
j
∣
x
i
j
)
=
h
(
β
T
x
i
j
)
,
i
=
1
,
…
,
n
,
j
=
1
,
…
,
m
i
,
where h
(
·
) is a known link function, β
=
(
β
1
,
…
,
β
p
)
T is the unknown parameter vector of interest, and x
i
j
=
(
x
i
j
1
,
…
,
x
i
j
p
)
T is a p
×
1 vector for j
=
1
,
…
,
m
i. In order to simplify notation, but without loss of generality, we assume m
i
≡
m. Recently, there has been considerable interest of investigating the GLMs, such as Liang and Zeger [2], Zeger and Liang [3], Fitzmaurice [5], Qu et al. [4], Pan [6], Balan and Schiopu-Kratina [7], Qu and Li [8], Wang and Qu [9], Wang [10], and Li et al. [11].The empirical likelihood method was introduced by Owen [12–14] as a nonparametric method of inference based on a data-driven likelihood ratio function in the i.i.d. case and had been applied to various statistical models. For example, Kolaczyk [15] and Chen and Cui [16] made an extension to the generalized linear models. As for other studies, see Baggerly [17], DiCiccio et al. [18], Chen and Qin [19], Qin and Lawless [20], Zhu and Xue [21], and Li et al. [22] among others. Owen [23] is one of the best references for this field. Although the empirical likelihood method had been investigated by many authors, those researches had mostly focused on independent observations. For the analysis of longitudinal data, Xue and Zhu [24] applied the empirical likelihood approach to the varying coefficient models with longitudinal data, but they did not consider the within-subject correlation structure of the longitudinal data. Li et al. [25] and Li et al. [26] proposed the generalized empirical likelihood method for longitudinal data by introducing the known within-subject correlation structure. Wang et al. [27] proposed two generalized empirical likelihood methods for the longitudinal linear model by taking into consideration within-subject correlations. They showed that, by plugging the estimator of the working correlation matrix into the empirical log-likelihood ratio (ELR), the resulting ELR is asymptotically a weighted sum of independent χ
1
2-variables with unknown weights. Thus, the empirical likelihood method needs to be adjusted so that it can be efficiently used to accommodate the correlation inherent in longitudinal data. However, in many applications, how to estimate the working correlation matrix effectively is a challenging problem.Instead of the empirical likelihood ratio statistic, Baggerly [17] used the empirical Cressie-Read (ECR) test statistic and showed that it is also asymptotically a Chi-squared distribution in the i.i.d. case. Bravo [28] extended the ECR approach to inference for α-mixing processes. The empirical Cressie-Read test statistic has user-specified parameter λ
∈
(
-
∞
,
∞
) and encompasses several commonly used testing statistics as special cases, such as the empirical likelihood statistic (λ
=
0), the maximum entropy, minimum information or the Kullback-Leibler statistic (λ
=
-
1), the Neyman-modified χ
2 statistic or the Euclidean likelihood statistic (λ
=
-
2), the Freeman-Tukey statistic (λ
=
-
1
/
2), and Pearson’s χ
2 statistic (λ
=
1), the first two being defined in a limiting sense. Thus, we can use these results to construct confidence regions of the parameter β of interest.In this paper we develop Baggerly’s results on the ECR test statistic for the longitudinal data by combining the idea of QIF in Qu et al. [4] and apply the proposed method to the marginal longitudinal GLMs (1). Specifically, we derive the asymptotic distributions of the maximum empirical Cressie-Read likelihood estimator (MECRLE) and the ECR test statistic. Although the working correlation matrix is assumed, the proposed method needs not to estimate the nuisance parameters associated with correlations. This is because the inverse of the commonly used working correlation matrix can be exactly represented or approximated by a linear combination of basis matrices such that the dimension of the unbiased estimating functions is greater than the number of unknown parameters. Thus, the efficiency results can be obtained by combining the ECR method with the generalized method of moments (GMM) proposed by Hansen [29]. Therefore, the proposed method inherits the advantages of the empirical likelihood, QIF, and GMM methods, and the proposed method represents a good alternative in this area as well.The paper is organized as follows. In Section2, we propose the more generalized method of the ECR test statistic for marginal longitudinal GLMs by combining with the QIF approach. The technical conditions and some asymptotic properties are given in Section 3. In Section 4, we present the results for simulation studies and a real dataset to illustrate the proposed methods. The technical proofs of the main results are presented in Section 5.


## 2. Model and ECR Test Statistic

Suppose that the population(
X
,
Y
) comes from the marginal longitudinal generalized linear model (1). Then the marginal mean of y
i
j is
(2)
μ
i
j
=
E
(
y
i
j
∣
x
i
j
)
=
h
(
β
T
x
i
j
)
and the marginal variance of y
i
j is
(3)
Var
⁡
(
y
i
j
∣
x
i
j
)
=
ϕ
ν
(
μ
i
j
)
,
where ν
(
·
) is a variance function and ϕ is a scale parameter.With assumptions on the first marginal moments, the GEE [2] estimator of β is defined as the solution of
(4)
∑
i
=
1
n
‍
μ
˙
i
T
V
i
-
1
(
Y
i
-
μ
i
)
=
0
,
where μ
i
=
(
μ
i
1
,
…
,
μ
i
m
)
T, μ
˙
i
=
∂
μ
i
/
∂
β, is an m
×
p matrix and V
i
=
A
i
1
/
2
R
(
α
)
A
i
1
/
2 is a working covariance matrix with A
i being the m
×
m diagonal matrix of marginal variances Var
⁡
(
Y
i
) and R
(
α
) as the working correlation matrix, where α is a vector which fully characterizes R
(
α
). The main advantage of the GEE method is that it yields a consistent estimator even if the working correlation matrix is misspecified.Qu et al. [4] introduced the quadratic inference function (QIF) by assuming that the inverse of the working correlation can be approximated by a linear combination of several basis matrices; that is,
(5)
R
-
1
≈
a
1
T
1
+
a
2
T
2
+
⋯
+
a
k
T
k
,
where T
1 is the identity matrix, T
2
,
…
,
T
k are symmetric basis matrices which are determined by the structure of R
(
α
), and a
1
,
…
,
a
k are constant coefficients. The advantage of this approach is that it does not require estimation of linear coefficients a
i’s which can be viewed as nuisance parameters.To implement the QIF approach, we need to choose the basis for the inverse of the correlation matrixR
(
α
). Qu et al. [4] and Qu and Song [30] found that, if R
(
α
) is the exchangeable working correlation matrix, R
-
1
=
a
1
T
1
+
a
2
T
2, where T
1 is the identity matrix and T
2 is a matrix with 0 on the diagonal and 1 off diagonal. If the working correlation matrix is AR(1) with R
i
j
=
α
|
i
-
j
|, then R
-
1 can be written as a linear combination of three basis matrices, that is, a
1
T
1
*
+
a
2
T
2
*
+
a
3
T
3
*, where T
1
* is the identity matrix, T
2
* can be 1 on the subdiagonal and 0 elsewhere, and T
3 can be 1 on the corners (
1,1
) and (
m
,
m
) and 0 elsewhere. However, T
3
* can often be dropped out of the model, as removing T
3
* does not affect the efficiency of the estimator too much, and this can simplify the estimation procedure. More details can be found in Qu et al. [4], Qu and Song [30], and Qu and Li [8] among others. In addition, Qu and Lindsay [31] developed an adaptive estimating equation approach to find a reliable approximation to the inverse of the variance matrix.Let(6)
g
i
(
β
)
=
(
μ
˙
i
T
A
i
-
1
/
2
T
1
A
i
-
1
/
2
(
Y
i
-
μ
i
)
μ
˙
i
T
A
i
-
1
/
2
T
2
A
i
-
1
/
2
(
Y
i
-
μ
i
)
⋮
μ
˙
i
T
A
i
-
1
/
2
T
k
A
i
-
1
/
2
(
Y
i
-
μ
i
)
)
.
It is easy to check that the GEE defined in (4) becomes a linear combination of the above extended score vector of ∑
i
=
1
n
‍
g
i
(
β
). Note that the dimension of g
i
(
β
) is r
=
k
p, and it is greater than the number of unknown parameters; thus the GEE method is unavailable. Therefore, Qu et al. [4] proposed the quadratic inference function by extending the method of GMM proposed by Hansen [29] to obtain the estimator of β.In this paper, we consider an alternate method using the following empirical Cressie-Read test statistics [17, 32]:
(7)
CR
(
λ
)
=
2
λ
(
1
+
λ
)
∑
i
=
1
n
‍
{
(
n
p
i
)
-
λ
-
1
}
,
-
∞
<
λ
<
∞
,
where λ is a user-specified parameter. ECR (7) is a generalization of the empirical log-likelihood ratio statistic and can also be defined as
(8)
CR
(
λ
)
=
{
2
λ
(
1
+
λ
)
∑
i
=
1
n
‍
{
(
n
p
i
)
-
λ
-
1
}
,
for
λ
≠
0
,
-
1
,
-
2
∑
i
=
1
n
‍
log
⁡
(
n
p
i
)
,
for
λ
=
0
,
2
n
∑
i
=
1
n
‍
p
i
log
⁡
(
n
p
i
)
,
for
λ
=
-
1
.
Let p
i be a multinomial likelihood supported on the observed data, and suppose that we are interested in testing the null hypothesis H
0
:
β
=
β
0, where β
0 is the true parameter vector. The empirical Cressie-Read family of test statistics for H
0 can be based on the following constrained minimization problem:
(9)
C
R
λ
(
β
)
=
min
⁡
{
∑
i
=
1
n
‍
CR
(
λ
)
∣
∑
i
=
1
n
‍
p
i
=
1
,
p
i
≥
0
,
∑
i
=
1
n
‍
p
i
g
i
(
β
)
=
0
}
.
As noted by Baggerly [17], a unique value for the right-hand side of (9) exists, provided that zero is inside the convex hull of the g
1
(
β
)
,
…
,
g
n
(
β
) for given β. Applying the Lagrange multiplier method, we can obtain the solution of β by minimizing (9). Let
(10)
G
=
2
λ
(
1
+
λ
)
∑
i
=
1
n
‍
{
(
n
p
i
)
-
λ
-
1
}
+
κ
1
(
∑
i
=
1
n
‍
p
i
-
1
)
+
n
κ
2
T
∑
i
=
1
n
‍
p
i
g
i
(
β
)
,
where κ
1 and κ
2
=
(
κ
1
,
…
,
κ
k
p
)
T are the Lagrange multipliers. Let ∂
G
/
∂
p
i
=
0; we have
(11)
p
i
=
{
1
n
{
1
+
s
+
t
T
g
i
(
β
)
}
-
1
/
(
1
+
λ
)
,
(
λ
≠
-
1
)
,
s
exp
⁡
{
t
T
g
i
(
β
)
}
,
(
λ
=
-
1
)
,
where s and the vector t
=
(
t
1
,
…
,
t
k
p
)
T are determined by
(12)
1
n
∑
i
=
1
n
‍
{
1
+
s
+
t
T
g
i
(
β
)
}
-
1
/
(
1
+
λ
)
=
1
,
1
n
∑
i
=
1
n
‍
{
1
+
s
+
t
T
g
i
(
β
)
}
-
1
/
(
1
+
λ
)
g
i
(
β
)
=
0
.
In the present paper, we only are interested in the case of λ
≠
-
1. When λ
→
-
1, the estimator of β is viewed as the empirical exponential tilting estimator (see [33, 34]). When λ
≠
-
1 and substituting (11) into (7), we obtain the following empirical Cressie-Read test statistic:
(13)
C
R
λ
(
β
)
=
2
λ
(
1
+
λ
)
∑
i
=
1
n
‍
[
{
1
+
s
+
t
T
g
i
(
β
)
}
λ
/
(
1
+
λ
)
-
1
]
.
Baggerly [17] had applied the empirical Cressie-Read test statistic to the general parameter case and shown that all members of the empirical Cressie-Read family have a Chi-squared calibration and enjoy the advantages of the empirical likelihood method. From the proof of Theorem 1 in this paper, we can obtain that
(14)
C
R
λ
(
β
)
=
n
g
-
n
T
(
β
)
Σ
^
-
1
g
-
n
(
β
)
+
o
P
(
1
)
,
s
=
1
2
λ
(
1
+
λ
)
g
-
n
(
β
)
Σ
^
-
1
g
-
n
(
β
)
+
O
P
(
n
-
1
/
2
ζ
n
)
,
t
(
β
)
=
(
1
+
λ
)
Σ
^
-
1
g
-
n
(
β
)
+
ζ
n
,
ζ
n
=
o
P
(
n
-
1
/
2
)
,
where g
-
n
(
β
)
=
(
1
/
n
)
∑
i
=
1
n
‍
g
i
(
β
) and Σ
^
=
(
1
/
n
)
∑
i
=
1
n
‍
g
i
(
β
)
g
i
T
(
β
). If β
^ satisfies C
R
λ
(
β
^
)
=
inf
⁡
β
∈
ℬ
C
R
λ
(
β
), the estimator β
^ is calledthe maximum empirical Cressie-Read likelihood estimator (MECRLE) of the parameter β.


## 3. Asymptotic Properties

To establish the asymptotic properties, we need the following regularity conditions.(C1)
The matrixΣ
^
=
(
1
/
n
)
∑
i
=
1
n
‍
g
i
(
β
)
g
i
T
(
β
) converges a.s. to an invertible positive definite matrix Σ
(
β
). This condition holds based on the weak law of large number when n tends to infinity and the cluster size is fixed.
(C2)
The domainℬ is a compact subset of ℝ
p and the true parameter value β
0 is in its interior. There exists a unique β
0
∈
ℬ satisfying mean zero model assumption E
{
g
1
(
β
0
)
}
=
0.
(C3)
Letμ
i
=
(
μ
i
1
,
…
,
μ
i
m
)
T where μ
i
j
=
E
(
y
i
j
∣
x
i
j
); then μ
i is a.s. continuously differentiable in β. Denote this m
×
p derivative matrix by μ
˙
i; then μ
˙
i has full column rank p a.s.
(C4)
Assume that∂
g
i
(
β
)
/
∂
β is continuous in a neighborhood of the true value β
0 and the rank of E
(
∂
g
1
(
β
0
)
/
∂
β
) is p. In addition, (
∂
g
-
n
/
∂
β
)
(
β
^
) converges in probability to E
[
(
∂
g
1
/
∂
β
)
(
β
0
)
] when β
^ converges in probability to β
0.Conditions (C1)–(C4) are actually quite mild and can be easily satisfied, and these conditions are also found in Qin and Lawless [20] and Qu et al. [4]. Conditions (C1) and (C4) ensure that the asymptotic variance exists for the proposed estimator. Condition (C2) ensures that there exists a n-consistent solution in the compact subset ℬ. (C3) is a common condition used in GLMs with longitudinal data.Theorem 1.
Assume that conditions (C1–C4) hold. Then, asn
→
∞, with probability tending to 1, C
R
λ
(
β
) attains its minimum value at some point β
^ in the interior of the ball ∥
β
-
β
0
∥
≤
n
-
1
/
3 and β
^, s
^, and t
^
=
t
(
β
^
) satisfy
(15)
Q
1
n
(
β
^
,
s
^
,
t
^
)
=
0
,
Q
2
n
(
β
^
,
s
^
,
t
^
)
=
1
,
Q
3
n
(
β
^
,
s
^
,
t
^
)
=
0
,
where
(16)
Q
1
n
(
β
,
s
,
t
)
=
1
n
∑
i
=
1
n
‍
{
1
+
s
+
t
T
g
i
(
β
)
}
-
1
/
(
1
+
λ
)
g
i
(
β
)
,
Q
2
n
(
β
,
s
,
t
)
=
1
n
∑
i
=
1
n
‍
{
1
+
s
+
t
T
g
i
(
β
)
}
-
1
/
(
1
+
λ
)
,
Q
3
n
(
β
,
s
,
t
)
=
1
n
1
(
1
+
λ
)
2
×
∑
i
=
1
n
‍
{
1
+
s
+
t
T
g
i
(
β
)
}
-
1
/
(
1
+
λ
)
×
(
∂
g
i
(
β
)
∂
β
)
T
t
.Theorem 2.
Suppose that the conditions of Theorem1 hold, and further assume that ∂
2
g
(
β
)
/
∂
β
∂
β
T is continuous about β in a neighborhood of the true value β
0. Then, as n
→
∞, one has
(17)
n
(
β
^
-
β
0
)
⟶
L
N
(
0
,
Ω
)
,
where “→
L” denotes the convergence in distribution and
(18)
Ω
=
[
E
(
∂
g
1
(
β
0
)
∂
β
)
T
Σ
-
1
(
β
0
)
E
(
∂
g
1
(
β
0
)
∂
β
)
]
-
1
.Remark 3.
Whenλ
→
0, the empirical log-likelihood ratio statistic is the special case of the empirical Cressie-Read test statistic C
R
λ
(
β
).Remark 4.
The proposed method provides a way to find efficient estimates for marginal longitudinal GLMs when the within-subject correlation is considered. It is known that we can construct the confidence regions ofβ
0 using Theorem 2. The asymptotic variance Ω of n
(
β
^
-
β
0
) does not depend on any λ and can be consistently estimated by
(19)
[
{
∑
i
=
1
n
‍
p
^
i
∂
g
i
(
β
^
)
∂
β
}
T
{
∑
i
=
1
n
‍
p
^
i
g
i
(
β
^
)
g
i
T
(
β
^
)
}
-
1
×
{
∑
i
=
1
n
‍
p
^
i
∂
g
i
(
β
^
)
∂
β
}
{
∑
i
=
1
n
‍
p
^
i
∂
g
i
(
β
^
)
∂
β
}
T
]
-
1
or by the same expression with the p
^
i’s, replaced by 1
/
n.Theorem 5.
Suppose that the conditions of Theorem2 hold; then, under the hypothesis H
0
:
β
=
β
0, C
R
λ
(
β
0
) is asymptotically a Chi-squared distribution with r degrees of freedom as n
→
∞.Theorem5 allows us to use the empirical Cressie-Read test statistic for testing or obtaining the confidence regions for the parameter β
0. For any 0
<
α
<
1, the confidence region of β
0 with asymptotic coverage 1
-
α can be determined by
(20)
P
(
C
R
λ
(
β
0
)
≤
χ
r
2
)
=
1
-
α
.We can construct a goodness-of-fit statistic to test the model assumption:(21)
H
0
:
E
{
g
i
(
β
)
}
=
0
.
We call C
R
λ
(
β
0
)
-
C
R
λ
(
β
^
) the empirical Cressie-Read likelihood ratio statistic and call C
R
λ
(
β
^
) the model test statistic.Theorem 6.
Suppose that the conditions of Theorem2 hold and g
i
(
β
) has dimension r
=
k
p and β has dimension p with p
<
r. Then, under the model assumption (21), C
R
λ
(
β
^
) is asymptotically a Chi-squared distribution with r
-
p degrees of freedom as n
→
∞; under the null hypothesis H
0
:
β
=
β
0, C
R
λ
(
β
0
)
-
C
R
λ
(
β
^
) is asymptotically a Chi-squared distribution with p degrees of freedom as n
→
∞.


## 4. Numerical Studies

### 4.1. Simulation Studies

In this subsection, we report the simulation study to illustrate the finite sample properties of the proposed ECR test statistic. For simplicity, we only compute the empirical likelihood (EL,λ
=
0) and the empirical exponential tilting (ET, λ
→
-
1) and compare the proposed methods with the GEE and QIF methods. Throughout the simulation study, each dataset comprised n
=
60 and 100 subjects and m
=
5 observations per subject over time. For each case, we repeat the experiment 500 times.Consider the following logistic regression model. The response variabley
i
j is binary and its marginal expectation given x
i
j is
(22)
logit
⁡
(
μ
i
j
)
=
∑
k
=
1
3
x
i
j
,
k
β
0
k
,
i
=
1
,
…
,
n
,
j
=
1
,
…
,
5
,
where β
0
=
[
1,2
,
-
0.8
]
T and the covariate x
i
j
=
(
x
i
j
,
1
,
…
,
x
i
j
,
3
)
T has a multivariate normal distribution with mean zero, marginal variance 1, and an AR(1) correlation matrix with autocorrelation coefficient 0.7. The binary response vector for each cluster has mean specified by (22) and an AR(1) correlation structure with an autocorrelation coefficient α
=
0.7. Table 1 reports the simulation results for the average coverage probabilities and the average lengths of the confidence intervals for β
1, β
2, and β
3 when the nominal level is 0.95 and the true correlation structure is AR(1) structure.Table 1
The average coverage probabilities (ACP) and the average lengths (ALEN) of the confidence intervals forβ
1, β
2, and β
3 when the nominal level is 0.95 and the true correlation structure is AR(1) structure.
n
β
EL
ET
GEE
QIF
ACP
ALEN
ACP
ALEN
ACP
ALEN
ACP
ALEN
CS
60
β
1
0.9120
0.4647
0.9060
0.4741
0.9000
0.4658
0.9120
0.4879
β
2
0.9140
0.6375
0.8820
0.6539
0.8400
0.5561
0.8700
0.6642
β
3
0.9280
0.4520
0.9150
0.4600
0.9100
0.4565
0.8960
0.4378
100
β
1
0.9160
0.4646
0.9120
0.4713
0.9100
0.4625
0.9120
0.4742
β
2
0.9180
0.6396
0.9080
0.6512
0.8940
0.6152
0.8740
0.6569
β
3
0.9240
0.4518
0.9180
0.4577
0.9100
0.4432
0.9040
0.4432
AR(1)
60
β
1
0.9220
0.3608
0.9180
0.3650
0.9200
0.3663
0.9480
0.3848
β
2
0.9240
0.4959
0.9120
0.5033
0.8760
0.4288
0.9020
0.5149
β
3
0.9320
0.3513
0.9280
0.3550
0.9240
0.3599
0.9280
0.3585
100
β
1
0.9300
0.3613
0.9200
0.3641
0.9220
0.3541
0.9460
0.3716
β
2
0.9360
0.4964
0.9320
0.5016
0.9160
0.4857
0.9140
0.5128
β
3
0.9420
0.3514
0.9380
0.3541
0.9240
0.3597
0.9340
0.3537From Table1, it is easy to see that the empirical likelihood method performs much better in terms of coverage accuracies of the confidence intervals even when the working correlation structure is misspecified. And when the working correlation structure is correctly specified, the performances of all the methods are usually slightly better. Table 1 also shows that the average coverage probabilities obtained by all methods tend to the nominal level 0.95 and the average lengths decrease as n increases.When sample size is 60 and the true correlation structure is AR(1), the histograms and the QQ plots of the 500 maximum empirical likelihood estimatorsβ
^
1, β
^
2, and β
^
3 based on the empirical likelihood method are plotted in Figure 1 and the histograms and the QQ plots of the 500 maximum empirical exponential tilting estimators β
^
1, β
^
2, and β
^
3 based on the empirical exponential tilting method are plotted in Figure 2 under the misspecified and correct correlation structures, respectively. Figures 1 and 2 show that empirically these estimators are asymptotically normal even when the working correlation structure is misspecified.The histograms and the QQ plots for the maximum empirical likelihood estimators based on the empirical likelihood method forn
=
60. When the true correlation structure is AR(1), the left plots are obtained by using the misspecified CS working correlation structure and the right plots are obtained by using the true AR(1) working correlation structure.
(a)
(b)The histograms and the QQ plots for the maximum empirical exponential tilting estimators based on the empirical exponential tilting method forn
=
60. When the true correlation structure is AR(1), the left plots are obtained by using the misspecified CS working correlation structure and the right plots are obtained by using the true AR(1) working correlation structure.
(a)
(b)


### 4.2.  Application to Real Data

To examine the performance of the proposed method, we analyze real data [35, 36] from a six-week frequent magnetic resonance imaging (MRI) substudy of the Betaseron clinical trial conducted at the University of British Columbia in relapsing-remitting multiple sclerosis involving 52 patients. The real data concerns a longitudinal clinical trial to assess the effects of neutralizing antibodies on interferon beta-1b (IFNB) in relapsing-remitting multiple sclerosis (MS), which is a disease that destroys the myelin sheath that surrounds the nerves. All patients were randomized into three treatment groups with allocation of 17 patients being treated by placebo, 17 by low dose, and 16 by high dose. This dataset has been studied by Song [37] and Li et al. [11]. There exist the missing values in this dataset; for convenience, we only analyze the balanced longitudinal data which contain 39 patients.For the analysis of this data, the binary response variable is exacerbation, which refers to whether an exacerbation began since the previous MRI scan, and is 1 for yes and 0 for no. Several baseline covariates are included in the model. They are treatment (trt), time (T) in weeks, squared time (T
2), and duration of disease (dur) in years. Here trt is treated as an ordinal covariate with scale 0, 1, 2 representing zero (placebo), low, and high dosage of the drug treatment. We consider the following marginal logistic model for the data:
(23)
logit
⁡
(
μ
i
j
)
=
β
0
+
β
1
trt
i
+
β
2
T
j
+
β
3
T
j
2
+
β
4
dur
i
,
where μ
i
j is the probability of exacerbation at visit j for subject i. Two correlation structures (exchangeable (CS) and AR(1)) are considered in this analysis. Table 2 reports the coefficient estimators and the corresponding standard errors for CS and AR(1) correlation structures, respectively. Table 3 reports the 95% confidence intervals and the lengths of the confidence intervals for CS and AR(1) correlation structures.Table 2
The parameter estimators  (the corresponding standard errors) for CS and AR(1) correlation structures.
EL
ET
GEE
QIF
CS
β
^
0
−0.3834 (0.3820)
−0.3051 (0.3872)
−0.4030 (0.4096)
−0.5039 (0.4005)
β
^
1
−0.0321 (0.1239)
−0.0454 (0.1259)
−0.0809 (0.1510)
−0.0667 (0.1206)
β
^
2
−0.0344 (0.0141)
−0.0349 (0.0143)
−0.0301 (0.0146)
−0.0296 (0.0146)
β
^
3
0.00031 (0.00013)
0.00031 (0.00013)
0.00027 (0.00013)
0.00027 (0.00010)
β
^
4
−0.0574 (0.0189)
−0.0700 (0.0200)
−0.0617 (0.0227)
−0.0530 (0.0255)
AR(1)
β
^
0
−0.4212 (0.4016)
−0.4382 (0.4041)
−0.4009 (0.3682)
−0.5043 (0.4145)
β
^
1
−0.0590 (0.1236)
−0.0599 (0.1242)
−0.0767 (0.1195)
−0.0711 (0.1506)
β
^
2
−0.0295 (0.0146)
−0.0278 (0.0147)
−0.0309 (0.0138)
−0.0290 (0.0089)
β
^
3
0.00026 (0.00013)
0.00025 (0.00013)
0.00028 (0.00012)
0.00027 (0.00008)
β
^
4
−0.0620 (0.0206)
−0.0645 (0.0208)
−0.0606 (0.0179)
−0.0558 (0.0249)Table 3
95% confidence intervals (CI) and the lengths (LEN) of the confidence intervals for CS and AR(1) correlation structures.
EL
ET
GEE
QIF
CS
β
0
CI
[
-
1.1321,0.3654
]
[
-
1.0640,0.4539
]
[
-
1.2058,0.3998
]
[
-
1.1504,0.3596
]
LEN
1.4975
1.5179
1.6056
1.5100
β
1
CI
[
-
0.2749,0.2107
]
[
-
0.2922,0.2014
]
[
-
0.3768,0.2150
]
[
-
0.3680,0.2065
]
LEN
0.4856
0.4936
0.5918
0.5745
β
2
CI
[
-
0.0620
,
-
0.0068
]
[
-
0.0629
,
-
0.0069
]
[
-
0.0587
,
-
0.0016
]
[
-
0.0604
,
-
0.0039
]
LEN
0.0552
0.0560
0.0571
0.0565
β
3
CI
[
0.0001,0.0006
]
[
0.0001,0.0006
]
[
0.00002,0.00052
]
[
0.00008,0.00056
]
LEN
0.0005
0.0005
0.0005
0.00048
β
4
CI
[
-
0.0945
,
-
0.0203
]
[
-
0.1092
,
-
0.0308
]
[
-
0.1061
,
-
0.0172
]
[
-
0.1030
,
-
0.0030
]
LEN
0.742
0.0784
0.0889
0.09995
AR(1)
β
0
CI
[
-
1.2084,0.3659
]
[
-
1.2302,0.3537
]
[
-
1.1226,0.3208
]
[
-
1.2268,0.2760
]
LEN
1.5743
1.5839
1.4434
1.5028
β
1
CI
[
-
0.3012,0.1832
]
[
-
0.3033,0.1834
]
[
-
0.3109,0.1576
]
[
-
0.3663,0.2240
]
LEN
0.4844
0.4867
0.4685
0.5903
β
2
CI
[
-
0.0580
,
-
0.0009
]
[
-
0.0566,0.0010
]
[
-
0.0580
,
-
0.0037
]
[
-
0.0465
,
-
0.0115
]
LEN
0.0571
0.0576
0.0543
0.0350
β
3
CI
[
0,0.0005
]
[
0,0.0005
]
[
0.00004,0.00052
]
[
0.00011,0.00052
]
LEN
0.0005
0.0005
0.00048
0.00041
β
4
CI
[
-
0.1023
,
-
0.0217
]
[
-
0.1053
,
-
0.0238
]
[
-
0.0957
,
-
0.0255
]
[
-
0.1047
,
-
0.0070
]
LEN
0.0806
0.0815
0.0702
0.0977From Tables2 and 3, we can see that the parameter estimators are very similar for all four methods under the CS and AR(1) working correlation structures. For the CS working correlation structure, the empirical likelihood and the empirical exponential tilting methods have smaller standard errors and smaller interval lengths than the GEE and QIF methods. However, the GEE method has the smaller standard errors and the smaller interval lengths under the AR(1) working correlation structure. Similar to the results in Song [37], we also find that the baseline disease severity measured as the duration of disease before the trial is an important explanatory variable associated with the risk of exacerbation, and we also do not find strong evidence that the drug treatment is effective in reducing the risk of exacerbation. Therefore, all the methods have comparable performance, and these findings are close to the existing analysis in Song [37].


## 4.1. Simulation Studies

In this subsection, we report the simulation study to illustrate the finite sample properties of the proposed ECR test statistic. For simplicity, we only compute the empirical likelihood (EL,λ
=
0) and the empirical exponential tilting (ET, λ
→
-
1) and compare the proposed methods with the GEE and QIF methods. Throughout the simulation study, each dataset comprised n
=
60 and 100 subjects and m
=
5 observations per subject over time. For each case, we repeat the experiment 500 times.Consider the following logistic regression model. The response variabley
i
j is binary and its marginal expectation given x
i
j is
(22)
logit
⁡
(
μ
i
j
)
=
∑
k
=
1
3
x
i
j
,
k
β
0
k
,
i
=
1
,
…
,
n
,
j
=
1
,
…
,
5
,
where β
0
=
[
1,2
,
-
0.8
]
T and the covariate x
i
j
=
(
x
i
j
,
1
,
…
,
x
i
j
,
3
)
T has a multivariate normal distribution with mean zero, marginal variance 1, and an AR(1) correlation matrix with autocorrelation coefficient 0.7. The binary response vector for each cluster has mean specified by (22) and an AR(1) correlation structure with an autocorrelation coefficient α
=
0.7. Table 1 reports the simulation results for the average coverage probabilities and the average lengths of the confidence intervals for β
1, β
2, and β
3 when the nominal level is 0.95 and the true correlation structure is AR(1) structure.Table 1
The average coverage probabilities (ACP) and the average lengths (ALEN) of the confidence intervals forβ
1, β
2, and β
3 when the nominal level is 0.95 and the true correlation structure is AR(1) structure.
n
β
EL
ET
GEE
QIF
ACP
ALEN
ACP
ALEN
ACP
ALEN
ACP
ALEN
CS
60
β
1
0.9120
0.4647
0.9060
0.4741
0.9000
0.4658
0.9120
0.4879
β
2
0.9140
0.6375
0.8820
0.6539
0.8400
0.5561
0.8700
0.6642
β
3
0.9280
0.4520
0.9150
0.4600
0.9100
0.4565
0.8960
0.4378
100
β
1
0.9160
0.4646
0.9120
0.4713
0.9100
0.4625
0.9120
0.4742
β
2
0.9180
0.6396
0.9080
0.6512
0.8940
0.6152
0.8740
0.6569
β
3
0.9240
0.4518
0.9180
0.4577
0.9100
0.4432
0.9040
0.4432
AR(1)
60
β
1
0.9220
0.3608
0.9180
0.3650
0.9200
0.3663
0.9480
0.3848
β
2
0.9240
0.4959
0.9120
0.5033
0.8760
0.4288
0.9020
0.5149
β
3
0.9320
0.3513
0.9280
0.3550
0.9240
0.3599
0.9280
0.3585
100
β
1
0.9300
0.3613
0.9200
0.3641
0.9220
0.3541
0.9460
0.3716
β
2
0.9360
0.4964
0.9320
0.5016
0.9160
0.4857
0.9140
0.5128
β
3
0.9420
0.3514
0.9380
0.3541
0.9240
0.3597
0.9340
0.3537From Table1, it is easy to see that the empirical likelihood method performs much better in terms of coverage accuracies of the confidence intervals even when the working correlation structure is misspecified. And when the working correlation structure is correctly specified, the performances of all the methods are usually slightly better. Table 1 also shows that the average coverage probabilities obtained by all methods tend to the nominal level 0.95 and the average lengths decrease as n increases.When sample size is 60 and the true correlation structure is AR(1), the histograms and the QQ plots of the 500 maximum empirical likelihood estimatorsβ
^
1, β
^
2, and β
^
3 based on the empirical likelihood method are plotted in Figure 1 and the histograms and the QQ plots of the 500 maximum empirical exponential tilting estimators β
^
1, β
^
2, and β
^
3 based on the empirical exponential tilting method are plotted in Figure 2 under the misspecified and correct correlation structures, respectively. Figures 1 and 2 show that empirically these estimators are asymptotically normal even when the working correlation structure is misspecified.The histograms and the QQ plots for the maximum empirical likelihood estimators based on the empirical likelihood method forn
=
60. When the true correlation structure is AR(1), the left plots are obtained by using the misspecified CS working correlation structure and the right plots are obtained by using the true AR(1) working correlation structure.
(a)
(b)The histograms and the QQ plots for the maximum empirical exponential tilting estimators based on the empirical exponential tilting method forn
=
60. When the true correlation structure is AR(1), the left plots are obtained by using the misspecified CS working correlation structure and the right plots are obtained by using the true AR(1) working correlation structure.
(a)
(b)


## 4.2.  Application to Real Data

To examine the performance of the proposed method, we analyze real data [35, 36] from a six-week frequent magnetic resonance imaging (MRI) substudy of the Betaseron clinical trial conducted at the University of British Columbia in relapsing-remitting multiple sclerosis involving 52 patients. The real data concerns a longitudinal clinical trial to assess the effects of neutralizing antibodies on interferon beta-1b (IFNB) in relapsing-remitting multiple sclerosis (MS), which is a disease that destroys the myelin sheath that surrounds the nerves. All patients were randomized into three treatment groups with allocation of 17 patients being treated by placebo, 17 by low dose, and 16 by high dose. This dataset has been studied by Song [37] and Li et al. [11]. There exist the missing values in this dataset; for convenience, we only analyze the balanced longitudinal data which contain 39 patients.For the analysis of this data, the binary response variable is exacerbation, which refers to whether an exacerbation began since the previous MRI scan, and is 1 for yes and 0 for no. Several baseline covariates are included in the model. They are treatment (trt), time (T) in weeks, squared time (T
2), and duration of disease (dur) in years. Here trt is treated as an ordinal covariate with scale 0, 1, 2 representing zero (placebo), low, and high dosage of the drug treatment. We consider the following marginal logistic model for the data:
(23)
logit
⁡
(
μ
i
j
)
=
β
0
+
β
1
trt
i
+
β
2
T
j
+
β
3
T
j
2
+
β
4
dur
i
,
where μ
i
j is the probability of exacerbation at visit j for subject i. Two correlation structures (exchangeable (CS) and AR(1)) are considered in this analysis. Table 2 reports the coefficient estimators and the corresponding standard errors for CS and AR(1) correlation structures, respectively. Table 3 reports the 95% confidence intervals and the lengths of the confidence intervals for CS and AR(1) correlation structures.Table 2
The parameter estimators  (the corresponding standard errors) for CS and AR(1) correlation structures.
EL
ET
GEE
QIF
CS
β
^
0
−0.3834 (0.3820)
−0.3051 (0.3872)
−0.4030 (0.4096)
−0.5039 (0.4005)
β
^
1
−0.0321 (0.1239)
−0.0454 (0.1259)
−0.0809 (0.1510)
−0.0667 (0.1206)
β
^
2
−0.0344 (0.0141)
−0.0349 (0.0143)
−0.0301 (0.0146)
−0.0296 (0.0146)
β
^
3
0.00031 (0.00013)
0.00031 (0.00013)
0.00027 (0.00013)
0.00027 (0.00010)
β
^
4
−0.0574 (0.0189)
−0.0700 (0.0200)
−0.0617 (0.0227)
−0.0530 (0.0255)
AR(1)
β
^
0
−0.4212 (0.4016)
−0.4382 (0.4041)
−0.4009 (0.3682)
−0.5043 (0.4145)
β
^
1
−0.0590 (0.1236)
−0.0599 (0.1242)
−0.0767 (0.1195)
−0.0711 (0.1506)
β
^
2
−0.0295 (0.0146)
−0.0278 (0.0147)
−0.0309 (0.0138)
−0.0290 (0.0089)
β
^
3
0.00026 (0.00013)
0.00025 (0.00013)
0.00028 (0.00012)
0.00027 (0.00008)
β
^
4
−0.0620 (0.0206)
−0.0645 (0.0208)
−0.0606 (0.0179)
−0.0558 (0.0249)Table 3
95% confidence intervals (CI) and the lengths (LEN) of the confidence intervals for CS and AR(1) correlation structures.
EL
ET
GEE
QIF
CS
β
0
CI
[
-
1.1321,0.3654
]
[
-
1.0640,0.4539
]
[
-
1.2058,0.3998
]
[
-
1.1504,0.3596
]
LEN
1.4975
1.5179
1.6056
1.5100
β
1
CI
[
-
0.2749,0.2107
]
[
-
0.2922,0.2014
]
[
-
0.3768,0.2150
]
[
-
0.3680,0.2065
]
LEN
0.4856
0.4936
0.5918
0.5745
β
2
CI
[
-
0.0620
,
-
0.0068
]
[
-
0.0629
,
-
0.0069
]
[
-
0.0587
,
-
0.0016
]
[
-
0.0604
,
-
0.0039
]
LEN
0.0552
0.0560
0.0571
0.0565
β
3
CI
[
0.0001,0.0006
]
[
0.0001,0.0006
]
[
0.00002,0.00052
]
[
0.00008,0.00056
]
LEN
0.0005
0.0005
0.0005
0.00048
β
4
CI
[
-
0.0945
,
-
0.0203
]
[
-
0.1092
,
-
0.0308
]
[
-
0.1061
,
-
0.0172
]
[
-
0.1030
,
-
0.0030
]
LEN
0.742
0.0784
0.0889
0.09995
AR(1)
β
0
CI
[
-
1.2084,0.3659
]
[
-
1.2302,0.3537
]
[
-
1.1226,0.3208
]
[
-
1.2268,0.2760
]
LEN
1.5743
1.5839
1.4434
1.5028
β
1
CI
[
-
0.3012,0.1832
]
[
-
0.3033,0.1834
]
[
-
0.3109,0.1576
]
[
-
0.3663,0.2240
]
LEN
0.4844
0.4867
0.4685
0.5903
β
2
CI
[
-
0.0580
,
-
0.0009
]
[
-
0.0566,0.0010
]
[
-
0.0580
,
-
0.0037
]
[
-
0.0465
,
-
0.0115
]
LEN
0.0571
0.0576
0.0543
0.0350
β
3
CI
[
0,0.0005
]
[
0,0.0005
]
[
0.00004,0.00052
]
[
0.00011,0.00052
]
LEN
0.0005
0.0005
0.00048
0.00041
β
4
CI
[
-
0.1023
,
-
0.0217
]
[
-
0.1053
,
-
0.0238
]
[
-
0.0957
,
-
0.0255
]
[
-
0.1047
,
-
0.0070
]
LEN
0.0806
0.0815
0.0702
0.0977From Tables2 and 3, we can see that the parameter estimators are very similar for all four methods under the CS and AR(1) working correlation structures. For the CS working correlation structure, the empirical likelihood and the empirical exponential tilting methods have smaller standard errors and smaller interval lengths than the GEE and QIF methods. However, the GEE method has the smaller standard errors and the smaller interval lengths under the AR(1) working correlation structure. Similar to the results in Song [37], we also find that the baseline disease severity measured as the duration of disease before the trial is an important explanatory variable associated with the risk of exacerbation, and we also do not find strong evidence that the drug treatment is effective in reducing the risk of exacerbation. Therefore, all the methods have comparable performance, and these findings are close to the existing analysis in Song [37].


## 5. Proofs of the Theorems

In this section, we present rigorous proofs of our results stated in Section3.Proof of Theorem1.
Assume thatλ
≠
-
1; G defined by (10) takes derivatives with respect to p
i; we have
(24)
∂
G
∂
p
i
=
-
2
(
1
+
λ
)
n
λ
p
i
-
(
1
+
λ
)
+
κ
1
+
n
κ
2
T
g
i
(
β
)
=
0
,
which implies that
(25)
p
i
-
(
1
+
λ
)
=
(
1
+
λ
)
n
λ
(
κ
1
+
n
κ
2
T
g
i
(
β
)
)
2
.
Multiplying p
i at both sides of (25) and taking sum and noting that ∑
i
=
1
n
p
i
=
1 and ∑
i
=
1
n
p
i
g
i
(
β
)
=
0, we can derive that
(26)
∑
i
=
1
n
‍
p
i
-
λ
=
(
1
+
λ
)
n
λ
κ
1
2
.
Substituting (26) into (25), we have
(27)
p
i
-
(
1
+
λ
)
=
∑
i
=
1
n
‍
p
i
-
λ
+
(
1
+
λ
)
n
1
+
λ
κ
2
T
g
i
(
β
)
2
.
By Theorem 1 in Baggerly [17], there exists a constant c
>
0 such that
(28)
2
λ
(
1
+
λ
)
∑
i
=
1
n
‍
[
(
n
p
i
)
-
λ
-
1
]
≤
c
,
which implies that ∑
i
=
1
n
p
i
-
λ
=
n
λ
+
1
+
η, where η
=
O
P
(
n
λ
). Substituting this expression into (27), we have
(29)
p
i
-
(
1
+
λ
)
=
n
1
+
λ
+
η
+
(
1
+
λ
)
n
1
+
λ
κ
2
T
g
i
(
β
)
2
.
Therefore, we can obtain that
(30)
p
i
=
1
n
[
1
+
s
+
t
T
g
i
(
β
)
]
-
1
/
(
1
+
λ
)
,
where s
=
η
/
n
1
+
λ
=
O
P
(
n
-
1
) and t
=
(
(
1
+
λ
)
/
2
)
κ
2. The next step is to bound the convergence rate of t. Let t
=
ρ
t
0, where ∥
t
0
∥
=
1. Invoking Lemma 3.2 in Künsch [38] and E
(
g
i
(
β
)
)
=
0, it is easy to show that t
0
T
g
i
(
β
)
=
O
P
(
n
1
/
2
). Applying the Taylor expansion to ∑
i
=
1
n
p
i
g
i
(
β
)
=
0, we have
(31)
1
n
∑
i
=
1
n
‍
[
g
i
(
β
)
-
1
1
+
λ
(
s
g
i
(
β
)
+
ρ
g
i
(
β
)
g
i
T
(
β
)
t
0
)
+
ζ
n
]
=
0
,
where ζ
n
=
ζ
n
(
g
i
(
β
)
,
s
,
ρ
). Assume that ρ
=
O
P
(
n
-
ν
), ν
>
0. By the central limit theorem, we have (
1
/
n
)
∑
i
=
1
n
g
i
(
β
)
=
O
P
(
n
-
1
/
2
) and (
1
/
n
)
∑
i
=
1
n
s
g
i
(
β
)
=
O
P
(
n
-
3
/
2
). Let Σ
(
β
)
=
Var
⁡
{
g
i
(
β
)
}; it is easy to show that
(32)
Σ
^
=
Δ
1
n
∑
i
=
1
n
‍
g
i
(
β
)
g
i
T
(
β
)
⟶
Σ
(
β
)
,
a
.
s
.
If ν
>
1
/
2 (or ν
<
1
/
2), then the dominant term in the right-hand side of (31) is (
1
/
n
)
∑
i
=
1
n
g
i
(
β
)
=
O
P
(
n
-
1
/
2
), while ρ
Σ
^
t
0
=
O
P
(
n
-
ν
), which implies that the sum cannot be zero unless ν
=
1
/
2. Therefore, we obtain that ρ
=
O
P
(
n
-
1
/
2
). By (31), we have
(33)
ρ
t
0
=
(
1
+
λ
)
Σ
^
-
1
(
1
n
∑
i
=
1
n
‍
g
i
(
β
)
)
+
ζ
n
,
ζ
n
=
o
P
(
n
-
1
/
2
)
.
For convenience, letg
-
n
(
β
)
=
Δ
(
1
/
n
)
∑
i
=
1
n
g
i
(
β
). We now find an approximation for s in terms of ρ
t
0
T
g
i
(
β
)
=
O
P
(
1
). By ∑
i
=
1
n
p
i
=
1 and applying the Taylor expansion, we have
(34)
1
n
∑
i
=
1
n
[
1
-
d
n
1
+
λ
+
(
2
+
λ
)
d
n
2
2
(
1
+
λ
)
2
+
O
P
(
n
-
1
/
2
ζ
n
)
]
=
1
,
where d
n
=
s
+
ρ
t
0
T
g
i
(
β
) and ζ
n is defined by (33). Rearranging (34) and ignoring terms of order O
P
(
s
2
), we have
(35)
s
+
ρ
t
0
T
g
-
n
(
β
)
=
2
+
λ
2
(
1
+
λ
)
[
∑
i
=
1
n
‍
2
s
ρ
t
0
T
g
-
n
(
β
)
h
h
h
h
h
h
h
h
h
h
h
h
+
1
n
ρ
2
t
0
T
∑
i
=
1
n
‍
g
i
(
β
)
g
i
T
(
β
)
t
0
]
+
O
P
(
n
-
1
/
2
ζ
n
)
.
That is,
(36)
s
=
2
+
λ
2
(
1
+
λ
)
ρ
2
t
0
T
Σ
^
t
0
-
ρ
t
0
T
g
-
n
(
β
)
+
O
P
(
n
-
1
/
2
ζ
n
)
.
From (33) and (36), we have
(37)
s
=
λ
2
ρ
t
0
T
g
-
n
(
β
)
+
O
P
(
n
-
1
/
2
ζ
n
)
.
By (33) and (37) and applying the Taylor expansion for C
R
λ
(
β
), it is easy to show that
(38)
C
R
λ
(
β
)
=
2
λ
(
1
+
λ
)
∑
i
=
1
n
[
(
1
+
s
+
ρ
t
0
T
g
i
(
β
)
)
λ
/
(
1
+
λ
)
-
1
]
=
2
(
1
+
λ
)
2
∑
i
=
1
n
[
s
+
ρ
t
0
T
g
i
(
β
)
-
1
2
(
1
+
λ
)
‍
h
h
h
h
h
h
h
h
h
h
×
ρ
2
t
0
T
g
i
(
β
)
g
i
T
(
β
)
t
0
+
o
P
(
n
-
1
)
1
2
(
1
+
λ
)
]
=
n
(
1
+
λ
)
2
[
2
s
+
2
ρ
t
0
T
g
-
n
(
β
)
-
1
1
+
λ
ρ
2
t
0
T
Σ
^
t
0
h
h
h
h
h
h
h
h
+
o
P
(
n
-
1
)
1
1
+
λ
]
=
n
(
1
+
λ
)
2
[
λ
ρ
t
0
T
g
-
n
(
β
)
kkkkkkkkk
+
2
(
1
+
λ
)
g
-
n
T
(
β
)
Σ
^
-
1
g
-
n
(
β
)
kkkkkkkkk
+
2
O
P
(
n
-
1
/
2
)
ζ
n
kkkkkkkkk
-
(
1
+
λ
)
g
-
n
T
(
β
)
Σ
^
-
1
g
-
n
(
β
)
kkkkkkkkk
-
2
g
-
n
T
(
β
)
Σ
^
-
1
ζ
n
-
ζ
n
T
Σ
^
-
1
ζ
n
kkkkkkkkk
+
o
P
(
n
-
1
)
]
=
n
(
1
+
λ
)
2
[
λ
(
1
+
λ
)
g
-
n
T
(
β
)
Σ
^
-
1
g
-
n
(
β
)
kkkkkkkk
+
2
(
1
+
λ
)
g
-
n
T
(
β
)
Σ
^
-
1
g
-
n
(
β
)
kkkkkkkk
-
(
1
+
λ
)
g
-
n
T
(
β
)
Σ
^
-
1
g
-
n
(
β
)
]
+
o
P
(
1
)
=
n
g
-
n
T
(
β
)
Σ
^
-
1
g
-
n
(
β
)
+
o
P
(
1
)
.
Similar to the proof of Lemma 1 in Qin and Lawless [20], let β
=
β
0
+
u
n
-
1
/
3, for β
∈
{
β
∣
∥
β
-
β
0
∥
=
n
-
1
/
3
}, where ∥
u
∥
=
1. We first give a lower bound for C
R
λ
(
β
) on the surface of the ball. When E
∥
g
i
(
β
)
∥
3
<
∞ and ∥
β
-
β
0
∥
≤
n
-
1
/
3, we can obtain that
(39)
t
(
β
)
=
(
1
+
λ
)
[
1
n
∑
i
=
1
n
‍
g
i
(
β
)
g
i
T
(
β
)
]
-
1
×
[
1
n
∑
i
=
1
n
‍
g
i
(
β
)
]
+
o
(
n
-
1
/
3
)
=
O
(
n
-
1
/
3
)
,
a
.
s
.
holds uniformly for β
∈
{
β
∣
∥
β
-
β
0
∥
=
n
-
1
/
3
}. From (32), (38), and (39), we have
(40)
C
R
λ
(
β
)
=
n
[
1
n
∑
i
=
1
n
‍
g
i
(
β
)
]
T
[
1
n
∑
i
=
1
n
‍
g
i
(
β
)
g
i
T
(
β
)
]
-
1
×
[
1
n
∑
i
=
1
n
‍
g
i
(
β
)
]
+
o
(
n
-
1
/
3
)
=
n
[
1
n
∑
i
=
1
n
‍
g
i
(
β
0
)
+
1
n
∑
i
=
1
n
‍
∂
g
i
(
β
0
)
∂
β
u
n
-
1
/
3
]
T
×
[
1
n
∑
i
=
1
n
‍
g
i
(
β
)
g
i
T
(
β
)
]
-
1
×
[
1
n
∑
i
=
1
n
‍
g
i
(
β
0
)
+
1
n
∑
i
=
1
n
‍
∂
g
i
(
β
0
)
∂
β
u
n
-
1
/
3
]
+
o
(
n
-
1
/
3
)
=
n
[
∂
g
1
(
β
0
)
∂
β
O
(
n
-
1
/
2
(
log
⁡
log
⁡
n
)
1
/
2
)
+
E
(
∂
g
1
(
β
0
)
∂
β
)
u
n
-
1
/
3
]
T
Σ
-
1
(
β
0
)
×
[
∂
g
1
(
β
0
)
∂
β
O
(
n
-
1
/
2
(
log
⁡
log
⁡
n
)
1
/
2
)
+
E
(
∂
g
1
(
β
0
)
∂
β
)
u
n
-
1
/
3
]
+
o
(
n
-
1
/
3
)
≥
(
ϕ
-
ϵ
)
n
1
/
3
,
a
.
s
.
,
where ϕ
-
ϵ
>
0 and ϕ is the smallest eigenvalue of
(41)
E
(
∂
g
1
(
β
0
)
∂
β
)
T
Σ
-
1
(
β
0
)
E
(
∂
g
1
(
β
0
)
∂
β
)
.
Similarly,
(42)
C
R
λ
(
β
0
)
=
n
[
1
n
∑
i
=
1
n
‍
g
i
(
β
0
)
]
T
[
1
n
∑
i
=
1
n
‍
g
i
(
β
0
)
g
i
T
(
β
0
)
]
-
1
×
[
1
n
∑
i
=
1
n
‍
g
i
(
β
0
)
]
+
o
(
1
)
=
O
(
log
⁡
log
⁡
n
)
,
a.s
.
It is known that, when β
∈
{
β
∣
∥
β
-
β
0
∥
≤
n
-
1
/
3
}, C
R
λ
(
β
) is a continuous function about β and has a minimum value in the interior of this ball. In addition, β
^ satisfies
(43)
∂
C
R
λ
(
β
)
∂
β
|
β
=
β
^
=
2
(
1
+
λ
)
2
∑
i
=
1
n
‍
[
1
+
s
+
t
T
g
i
(
β
)
]
-
1
/
(
1
+
λ
)
×
(
∂
g
i
(
β
)
∂
β
)
T
t
|
β
=
β
^
=
0
.Proof of Theorem2.
We take derivatives forQ
1
n, Q
2
n, and Q
3
n with respect to β, s, and t
T; we have
(44)
∂
Q
1
n
(
β
,
0,0
)
∂
β
=
1
n
∑
i
=
1
n
‍
∂
g
i
(
β
)
∂
β
,
∂
Q
1
n
(
β
,
0,0
)
∂
s
=
-
1
n
(
1
+
λ
)
∑
i
=
1
n
‍
g
i
(
β
)
,
∂
Q
1
n
(
β
,
0,0
)
∂
t
T
=
-
1
n
(
1
+
λ
)
∑
i
=
1
n
‍
g
i
(
β
)
g
i
T
(
β
)
;
∂
Q
2
n
(
β
,
0,0
)
∂
β
=
0
,
∂
Q
2
n
(
β
,
0,0
)
∂
s
=
-
1
1
+
λ
,
∂
Q
2
n
(
β
,
0,0
)
∂
t
T
=
-
1
n
(
1
+
λ
)
∑
i
=
1
n
‍
g
i
T
(
β
)
;
∂
Q
3
n
(
β
,
0,0
)
∂
β
=
0
,
∂
Q
3
n
(
β
,
0,0
)
∂
s
=
0
,
∂
Q
3
n
(
β
,
0,0
)
∂
t
T
=
1
n
(
1
+
λ
)
2
∑
i
=
1
n
‍
(
∂
g
i
(
β
)
∂
β
)
T
.
By the conditions and Theorem 1 and the Taylor expanding of Q
1
n
(
β
^
,
s
^
,
t
^
), Q
2
n
(
β
^
,
s
^
,
t
^
), and Q
3
n
(
β
^
,
s
^
,
t
^
) at (
β
0
,
0,0
), we have
(45)
0
=
Q
1
n
(
β
^
,
s
^
,
t
^
)
h
=
Q
1
n
(
β
0
,
0,0
)
+
∂
Q
1
n
(
β
0
,
0,0
)
∂
β
(
β
^
-
β
0
)
hhh
+
∂
Q
1
n
(
β
0
,
0,0
)
∂
s
(
s
^
-
0
)
hhh
+
∂
Q
1
n
(
β
0
,
0,0
)
∂
t
T
(
t
^
-
0
)
+
o
P
(
δ
n
)
,
1
=
Q
2
n
(
β
^
,
s
^
,
t
^
)
h
=
Q
2
n
(
β
0
,
0,0
)
+
∂
Q
2
n
(
β
0
,
0,0
)
∂
β
(
β
^
-
β
0
)
hhh
+
∂
Q
2
n
(
β
0
,
0,0
)
∂
s
(
s
^
-
0
)
hhh
+
∂
Q
2
n
(
β
0
,
0,0
)
∂
t
T
(
t
^
-
0
)
+
o
P
(
δ
n
)
,
0
=
Q
3
n
(
β
^
,
s
^
,
t
^
)
h
=
Q
3
n
(
β
0
,
0,0
)
+
∂
Q
3
n
(
β
0
,
0,0
)
∂
β
(
β
^
-
β
0
)
hhh
+
∂
Q
3
n
(
β
0
,
0,0
)
∂
s
(
s
^
-
0
)
hhh
+
∂
Q
3
n
(
β
0
,
0,0
)
∂
t
T
(
t
^
-
0
)
+
o
P
(
δ
n
)
,
where δ
n
=
∥
β
^
-
β
0
∥
+
∥
s
^
∥
+
∥
t
^
∥. We further have
(46)
(
-
Q
1
n
(
β
0
,
0,0
)
+
o
P
(
δ
n
)
1
-
Q
2
n
(
β
0
,
0,0
)
+
o
P
(
δ
n
)
-
Q
3
n
(
β
0
,
0,0
)
+
o
P
(
δ
n
)
)
=
(
∂
Q
1
n
(
β
0
,
0,0
)
∂
t
T
∂
Q
1
n
(
β
0
,
0,0
)
∂
s
∂
Q
1
n
(
β
0
,
0,0
)
∂
β
∂
Q
2
n
(
β
0
,
0,0
)
∂
t
T
∂
Q
2
n
(
β
0
,
0,0
)
∂
s
∂
Q
2
n
(
β
0
,
0,0
)
∂
β
∂
Q
3
n
(
β
0
,
0,0
)
∂
t
T
∂
Q
3
n
(
β
0
,
0,0
)
∂
s
∂
Q
3
n
(
β
0
,
0,0
)
∂
β
)
×
(
t
^
s
^
β
^
-
β
0
)
.
By the above expression, we can obtain that
(47)
(
t
^
s
^
β
^
-
β
0
)
=
S
n
-
1
(
-
Q
1
n
(
β
0
,
0,0
)
+
o
P
(
δ
n
)
o
P
(
δ
n
)
o
P
(
δ
n
)
)
,
where
(48)
S
n
=
Δ
(
∂
Q
1
n
(
β
0
,
0,0
)
∂
t
T
∂
Q
1
n
(
β
0
,
0,0
)
∂
s
∂
Q
1
n
(
β
0
,
0,0
)
∂
β
∂
Q
2
n
(
β
0
,
0,0
)
∂
t
T
∂
Q
2
n
(
β
0
,
0,0
)
∂
s
∂
Q
2
n
(
β
0
,
0,0
)
∂
β
∂
Q
3
n
(
β
0
,
0,0
)
∂
t
T
∂
Q
3
n
(
β
0
,
0,0
)
∂
s
∂
Q
3
n
(
β
0
,
0,0
)
∂
β
)
⟶
S
,
a.s.
,
S
=
(
S
11
S
12
S
13
S
21
S
22
0
S
31
0
0
)
S
=
(
-
1
1
+
λ
Σ
(
β
0
)
-
1
1
+
λ
E
(
g
1
(
β
0
)
)
E
(
∂
g
1
(
β
0
)
∂
β
)
-
1
1
+
λ
E
(
g
1
T
(
β
0
)
)
-
1
1
+
λ
0
1
(
1
+
λ
)
2
E
(
∂
g
1
(
β
0
)
∂
β
)
T
0
0
)
.
By the above expression and Q
1
n
(
β
0
,
0,0
)
=
(
1
/
n
)
∑
i
=
1
n
g
i
(
β
0
)
=
O
P
(
n
-
1
/
2
), we can derive that δ
n
=
O
P
(
n
-
1
/
2
). Note that E
(
g
i
(
β
0
)
)
=
0, and Σ
(
β
0
) is the positive definite matrix; it is easy to show that
(49)
S
-
1
=
(
S
11
-
1
+
S
11
-
1
[
S
13
S
33.1
-
1
S
31
]
S
11
-
1
0
-
S
11
-
1
S
13
S
33.1
-
1
0
S
22
-
1
0
-
S
33.1
-
1
S
31
S
11
-
1
0
S
33.1
-
1
)
,
where S
33.1
=
-
S
31
S
11
-
1
S
13. Substituting (49) into (47), we have
(50)
n
(
β
^
-
β
0
)
=
S
33.1
-
1
S
31
S
11
-
1
n
Q
1
n
(
β
0
,
0,0
)
+
o
P
(
1
)
,
n
(
t
^
-
0
)
=
-
{
S
11
-
1
+
S
11
-
1
[
S
13
S
33.1
-
1
S
31
]
S
11
-
1
}
×
n
Q
1
n
(
β
0
,
0,0
)
+
o
P
(
1
)
.
Note that n
Q
1
n
(
β
0
,
0,0
)
→
L
N
(
0
,
Σ
(
β
0
)
); we can derive that
(51)
n
(
β
^
-
β
0
)
⟶
L
N
(
0
,
Ω
)
,
n
(
t
^
-
0
)
⟶
L
N
(
0
,
Φ
)
,
where
(52)
Ω
=
[
E
(
∂
g
1
(
β
0
)
∂
β
)
T
Σ
-
1
(
β
0
)
E
(
∂
g
1
(
β
0
)
∂
β
)
]
-
1
,
Φ
=
(
1
+
λ
)
Σ
-
1
(
β
0
)
×
{
I
-
E
(
∂
g
1
(
β
0
)
∂
β
)
Ω
E
(
∂
g
1
(
β
0
)
∂
β
)
T
Σ
-
1
(
β
0
)
}
.Proof of Theorem5.
Sincen
g
-
n
(
β
0
)
→
L
N
(
0
,
Σ
(
β
0
)
) and Σ
^
(
β
0
)
=
Δ
(
1
/
n
)
∑
i
=
1
n
g
i
(
β
0
)
g
i
T
(
β
0
)
→
Σ
(
β
0
), a.s., by (38), it is easy to show that
(53)
C
R
λ
(
β
0
)
=
[
n
g
-
n
(
β
0
)
]
T
Σ
^
-
1
(
β
0
)
×
[
n
g
-
n
(
β
0
)
]
+
o
P
(
1
)
⟶
L
χ
r
2
.Proof of Theorem6.
From (33), (38), and (50) and by the Taylor expansion and some simple calculations, we have
(54)
C
R
λ
(
β
^
)
=
n
g
-
n
T
(
β
^
)
Σ
^
-
1
(
β
^
)
g
-
n
(
β
^
)
+
o
P
(
1
)
=
n
1
+
λ
g
-
n
T
(
β
^
)
t
^
+
o
P
(
1
)
=
n
1
+
λ
[
g
-
n
(
β
0
)
+
∂
g
-
n
(
β
0
)
∂
β
(
β
^
-
β
0
)
h
h
h
h
h
h
+
o
P
(
n
-
1
/
2
)
∂
g
-
n
(
β
0
)
∂
β
]
T
t
^
+
o
P
(
1
)
=
n
1
+
λ
[
(
n
-
1
/
2
)
Q
1
n
(
β
0
,
0,0
)
+
S
13
S
33.1
-
1
S
31
S
11
-
1
Q
1
n
(
β
0
,
0,0
)
+
o
P
(
n
-
1
/
2
)
]
T
×
[
-
{
S
11
-
1
+
S
11
-
1
[
S
13
S
33.1
-
1
S
31
]
S
11
-
1
}
Q
1
n
×
(
β
0
,
0,0
)
+
o
P
(
n
-
1
/
2
)
]
+
o
P
(
1
)
=
[
n
Σ
-
1
/
2
(
β
0
)
Q
1
n
(
β
0
,
0,0
)
]
T
×
G
1
[
n
Σ
-
1
/
2
(
β
0
)
Q
1
n
(
β
0
,
0,0
)
]
+
o
P
(
1
)
,
since G
1
2
=
G
1, where G
1
=
Δ
[
I
-
Σ
-
1
/
2
(
β
0
)
E
(
∂
g
1
(
β
0
)
/
∂
β
)
Ω
E
(
∂
g
1
(
β
0
)
/
∂
β
)
T
Σ
-
1
/
2
(
β
0
)
]. From n
Σ
-
1
/
2
(
β
0
)
Q
1
n
(
β
0
,
0,0
)
→
L
N
(
0
,
I
) and
(55)
tr
⁡
(
G
1
)
=
tr
⁡
{
I
-
Σ
-
1
/
2
(
β
0
)
E
(
∂
g
1
(
β
0
)
∂
β
)
Ω
E
×
(
∂
g
1
(
β
0
)
∂
β
)
T
Σ
-
1
/
2
(
β
0
)
}
=
r
-
tr
⁡
{
Σ
-
1
/
2
(
β
0
)
E
(
∂
g
1
(
β
0
)
∂
β
)
Ω
E
×
(
∂
g
1
(
β
0
)
∂
β
)
T
Σ
-
1
/
2
(
β
0
)
}
=
r
-
p
,
it is easy to show that C
R
λ
(
β
^
)
→
L
χ
r
-
p
2.
On the other hand, invoking the same argument, we have(56)
C
R
λ
(
β
0
)
-
C
R
λ
(
β
^
)
=
n
g
-
n
T
(
β
0
)
Σ
^
-
1
(
β
0
)
g
-
n
(
β
0
)
-
n
g
-
n
T
(
β
^
)
Σ
^
-
1
(
β
^
)
g
-
n
(
β
^
)
+
o
P
(
1
)
=
n
Q
1
n
T
(
β
0
,
0,0
)
Σ
-
1
(
β
0
)
Q
1
n
(
β
0
,
0,0
)
-
[
n
Σ
-
1
/
2
(
β
0
)
Q
1
n
(
β
0
,
0,0
)
]
T
×
G
1
[
n
Σ
-
1
/
2
(
β
0
)
Q
1
n
(
β
0
,
0,0
)
]
+
o
P
(
1
)
=
[
n
Σ
-
1
/
2
(
β
0
)
Q
1
n
(
β
0
,
0,0
)
]
T
×
G
2
[
n
Σ
-
1
/
2
(
β
0
)
Q
1
n
(
β
0
,
0,0
)
]
+
o
P
(
1
)
,
since G
2
2
=
G
2, where G
2
=
Δ
Σ
-
1
/
2
(
β
0
)
E
(
∂
g
1
(
β
0
)
/
∂
β
)
Ω
E
(
∂
g
1
(
β
0
)
/
∂
β
)
T
Σ
-
1
/
2
(
β
0
). Further,
(57)
tr
⁡
{
Σ
-
1
/
2
(
β
0
)
E
(
∂
g
1
(
β
0
)
∂
β
)
Ω
E
(
∂
g
1
(
β
0
)
∂
β
)
T
×
Σ
-
1
/
2
(
β
0
)
(
∂
g
1
(
β
0
)
∂
β
)
T
∂
g
1
(
β
0
)
∂
β
}
=
p
.
Thus, we can obtain that CR
λ
(
β
0
)
-
CR
λ
(
β
^
)
→
L
χ
p
2.



---

*Source: 738953-2014-03-18.xml*