# Confidence Intervals for the Mean Based on Exponential Type Inequalities and Empirical Likelihood

Authors: Sandra Vucane; Janis Valeinis; George Luta
Journal: ISRN Biomathematics
 (2013)

---

## Abstract

For independent observations, recently, it has been proposed to construct the confidence intervals for the mean using exponential type inequalities. Although this method requires much weaker assumptions than those required by the classical methods, the resulting intervals are usually too large. Still in special cases, one can find some advantage of using bounded and unbounded Bernstein inequalities. In this paper, we discuss the applicability of this approach for dependent data. Moreover, we propose to use the empirical likelihood method both in the case of independent and dependent observations for inference regarding the mean. The advantage of empirical likelihood is its Bartlett correctability and a rather simple extension to the dependent case. Finally, we provide some simulation results comparing these methods with respect to their empirical coverage accuracy and average interval length. At the end, we apply the above described methods for the serial analysis of a gene expression (SAGE) data example.


---

## Body

## 1. Introduction

Although the classicalt-test is based on the assumption that observations are normally distributed, it is well known that it is a robust test and works well at least for symmetric distributions (see [1]). However, for skewed or heavy tailed distributions, the confidence intervals for the mean based on the inversion of the t-test may give poor coverage. Using exponential type inequalities (such as Bernstein’s inequality), Rosenblum and van der Laan [2] presented a simple approach for constructing confidence intervals for the population mean. Rosenblum and van der Laan [2] dealt with the bounded version of Bernstein’s inequality which is applicable only for a few distributions such as, the bounded uniform distribution. To use it for general distributions, we need to use the unbounded version of Bernstein’s inequality, which was analyzed by Shilane et al. [3] for the negative binomial distribution. The confidence intervals based on exponential type inequalities have a guaranteed coverage probability under much weaker assumptions than required by the standard methods. Although the obtained confidence intervals are usually too large, there are situations when they give better coverage accuracy than the classical methods.In this paper, our goal is to use the empirical likelihood method introduced by Owen [4, 5] for the inference on the mean and compare it with the Rosenblum and van der Laan [2] method. The empirical likelihood method is based on the nonparametric likelihood ratio statistic, which similarly to the parametric case has asymptotically a limiting chi-square distribution. The confidence intervals obtained by the empirical likelihood method have some very appealing characteristics. There are no prespecified parametric assumptions on the distribution of observations and no constraints on the shape of the confidence intervals. Empirical likelihood intervals are Bartlett correctable in most cases. This means that a simple correction for the mean reduces the coverage error from order n-1 to order n-2, where n denotes the sample size (see, e.g., [6, 7]). Owen [7] suggested (see Section 2 for more details) that for small sample sizes the F distribution may provide a better approximation for the limiting distribution of the test statistic. We found in our simulation study that this approach can provide a significant improvement (see Tables 1 and 3).Table 1
Empirical coverage accuracy for95% confidence intervals for the mean using different methods for the iid data generated with a marginal negative binomial distribution Negbinom(μ,θ), based on 10 000 samples.
n
t
EL
E
L
B
E
L
F
E
L
F
B
B
perc
B
norm
B
basic
B
stud
B
bn
B
ubn
θ
=
0.1
20
0.797
0.808
0.821
0.829
0.840
0.797
0.783
0.744
0.926
0.898
0.889
50
0.848
0.884
0.898
0.897
0.902
0.861
0.837
0.807
0.945
0.961
0.944
100
0.889
0.917
0.924
0.922
0.928
0.907
0.888
0.856
0.948
0.979
0.966
θ
=
0.5
20
0.893
0.890
0.901
0.905
0.918
0.881
0.874
0.853
0.943
0.980
0.965
50
0.916
0.921
0.928
0.929
0.932
0.916
0.910
0.906
0.945
0.993
0.980
100
0.937
0.947
0.950
0.950
0.952
0.936
0.929
0.920
0.955
0.998
0.995
θ
=
1.0
20
0.918
0.905
0.914
0.920
0.931
0.900
0.902
0.894
0.940
0.985
0.972
50
0.931
0.934
0.940
0.939
0.944
0.931
0.924
0.919
0.947
0.995
0.985
100
0.941
0.946
0.948
0.950
0.952
0.936
0.936
0.936
0.955
0.995
0.990Time series data are often correlated. For this situation, either modified or different statistical procedures of statistical inference are required. There exist many exponential type inequalities for weakly dependent data, characterized byα-mixing or β-mixing sequences (e.g., see [8–10]). However, it is problematic to use them for practical data problems. First, such inequalities typically contain the α- or β-mixing coefficients which have to be estimated from the data. Only recently McDonald et al. [11] introduced the estimation procedure for β-mixing coefficients. Second, usually these inequalities are established for bounded distributions. The applicability of exponential type inequalities for the dependent case is therefore limited. The blockwise empirical likelihood method introduced by Kitamura [12] provides a good alternative for the inference regarding the mean. It seems that the Bartlett correction can be applied also in this case.For our simulation study, we use the negative binomial distribution, which has been analyzed by several authors (e.g., see [3, 13]) and has been widely used for applications involving insurance data. There are several ways how the negative binomial distribution can be parametrized. We will use the parametrization described by Hilbe [13]; that is, for any μ∈ℝ+ and θ∈ℝ+, the resulting probability mass function for the negative binomial random variable X~Negbinom(μ,θ) is
(1)ℙ(X=k)=μkk!Γ(θ+k)Γ(θ)(μ+θ)k(1+μθ)-θ,k∈ℤ+.
We also simulate some autoregressive processes representing data with a marginal negative binomial distribution. For independent data, we compared the empirical coverage accuracy for the confidence intervals based on exponential type inequalities, the t-test, and several empirical likelihood and bootstrap methods. For dependent data, we used a simulation study to compare the blockwise empirical likelihood method, with and without the Bartlett correction, and we also applied the empirical likelihood method to the serial analysis of gene expression (SAGE) data analyzed by Shilane et al. [3].This paper is organized as follows. Section2 deals with the independent case. It contains three subsections which introduce Bernstein inequalities and the empirical likelihood method while also including a simulation study. In Section 3, weakly dependent data are discussed. Subsequent three subsections introduce the blockwise empirical likelihood method, describe two exponential-type inequalities, provide a simulation study, and analyze the SAGE data example. We finish with the main conclusions presented in Section 4.


## 2. Independent Case

### 2.1. Confidence Intervals Using Bernstein’s Inequality

Rosenblum and van der Laan [2] described the method for generating confidence intervals, which bounds the tails of the distribution of the sample mean of n independent, bounded random variables. This method is based on exponential type inequalities such as Bernstein’s inequality: for all x>0,
(2)ℙ(1n|∑i=1n‍(Xi-𝔼Xi)|>xn)≤2exp(-nx22(v+Wnx/3)),
where W and v are constants defined in Theorem 1.Theorem 1 (Rosenblum and van der Laan [2]).
LetX1,…,Xn be independent and identically distributed (iid) random variables with mean μ, and let W be such that ℙ(|X-μ|≤W)=1. Let σ*2 be a number satisfying σ*2≥Var(X). Then for the function
(3)q(1-α,W,nσ*2)≡n-1log(2α)(Wn-1/2+(W2n)9+(2n2σ*2)log(2/α)),ℙ(X--q(1-α,W,nσ*2)n≤μ≤X-+q(1-α,W,nσ*2)n)≥1-α.
In particular,
(4)(X--q(1-α,W,nσ*2)n,X-+q(1-α,W,nσ*2)n)
is the (1-α)100% confidence interval for μ.The accuracy of coverage probabilities depends on the estimation ofW and v, where v=nσ*2. Rosenblum and van der Laan [2] suggested that a bound W should be set at least as large as maxi|Xi-X-| and proposed also to use for σ*2 an estimator s2+SE(s2), where SE(s2) is the standard error of s2 (for details, see [2]).To examine exponential type inequalities for distributions like the negative binomial distribution, we will need a version of Bernstein’s inequality that does not require the assumption of boundedness.Lemma 2 (Birgé and Massart [14]).
LetZ1,…,Zn be independent random variables satisfying the moments conditions
(5)1n∑i=1n‍𝔼|Zim|≤m!2v2cm-2
for all m>2, for some positive constants v and c. Then, for any positive ε and Sn=∑i=1n‍Zi,
(6)ℙ[Sn-E(Sn)≥nε]≤exp(-nx)≤exp(-nε2/2v2+nε),
where x is defined by the equation ε=v2x+cx.Suppose thatZi, 1≤i≤n, are iid negative binomial with parameters μ and θ, and let Yi=Zi-μ. In this case, Shilane et al. [3] obtained the confidence interval for the mean of the form X-±ε, where ε is the solution to the equation
(7)(θ+μnε+θ+μ)-θ((θ+μ)(nε+μ)μ(nε+θ+μ))-nε-μ=α2.
To estimate the parameter θ, we can use the method of moments; that is, θ^=X-/((s2/X-)-1).


### 2.2. Empirical Likelihood for the Mean

LetX1,…,Xn be iid observations with the distribution F and mean μ=𝔼X1. To obtain confidence intervals for μ, we define the profile empirical likelihood function
(8)R(μ)=sup{∏i=1n‍npi|pi≥0,∑i=1n‍pi=1,∑i=1n‍piXi=μ}.
Owen [4, 5] showed that a unique value for the right–hand side of (8) exists when μ is inside the convex hull of the data points X1,…,Xn. This maximization problem can be solved using the Lagrange multiplier method.Theorem 3 (Owen [7]).
IfX1,…,Xn are iid random variables with the distribution function F0, μ0=𝔼(Xi), and 0<Var(Xi)<∞, then
(9)-2log(R(μ0))⟶dχ12.The empirical likelihood(1-α)100% confidence interval for μ is of the following form:
(10){μ∣-2log(R(μ))≤c},
where c is the 1-α quantile of the χ12 distribution. Owen [7] stated that the proof of Theorem 3 and some simulations suggest that the quantiles χ12,1-α can be replaced by the quantiles of the F distribution F1,n-11-α. The F calibration usually gives better results for small sample sizes.The Bartlett correction replaces the1-α quantile of the limiting chi-square distribution χ12,1-α of the test statistic -2log(R(μ)) by (1+an-1)χ12,1-α, where a=(1/2)μ4-(1/3)μ32 and μi is ith moment of X1 (see [6]). The Bartlett correction improves the asymptotic error rate for all coverage levels 1-α. A Bartlett-corrected empirical likelihood confidence interval has the following form:
(11){μ∣-2log(R(μ))≤(1+an-1)χ12,1-α}.


### 2.3. Simulation Study

Our simulation study is based on10,000 independent trials. In every trial, we generated iid random variables from the negative binomial distribution Negbinom(μ,θ) with different sample sizes n={20,50,100}, μ=5, and θ={0.1,0.5,1.0}. To simulate data, we used the command rnegbin from the package MASS in the R statistical programming language.In Table1, the empirical coverage accuracy for nominal 95% confidence intervals for the mean is evaluated for the following methods: the t-test (t), the empirical likelihood method (EL), the empirical likelihood with Bartlett correction (ELB), the empirical likelihood method with the F distribution calibration (ELF), and the empirical likelihood method with the F distribution calibration and Bartlett correction (ELFB). For comparison, we have also simulated the coverage accuracy of several bootstrap resampling procedures: the percentile (Bperc), normal (Bnorm), basic (Bbasic), and studentized (Bstud) bootstrap resampling methods. The R package boot and command boot.ci are used to construct the confidence intervals for all the above mentioned bootstrap methods. Although only the unbounded version of Bernstein inequalities is valid for the negative binomial distribution, we also apply the bounded version in order to see the possible effects of the violation of this assumption. In Table 2, we present average confidence interval lengths for the same methods as described in Table 1.Table 2
Average confidence interval lengths for the95% confidence intervals for the mean using different methods for the iid data generated with the marginal negative binomial distribution Negbinom(μ,θ), based on 10 000 samples.
n
t
EL
E
L
B
E
L
F
E
L
F
B
B
perc
B
norm
B
basic
B
stud
B
bn
B
ubn
θ
=
0.1
20
12.80
12.11
12.91
12.98
13.85
11.11
11.67
11.11
69.33
28.53
26.59
50
8.17
8.33
8.75
8.56
8.99
7.76
7.89
7.76
15.27
18.07
15.49
100
5.95
6.17
6.40
6.25
6.49
5.80
5.85
5.80
8.15
12.74
10.53
θ
=
0.5
20
6.63
6.21
6.50
6.65
6.96
6.00
6.06
6.00
9.33
13.39
11.20
50
4.07
4.06
4.18
4.17
4.29
3.93
3.93
3.93
4.66
8.02
6.53
100
2.90
2.93
2.98
2.97
3.02
2.86
2.85
2.86
3.12
5.48
4.49
θ
=
1.0
20
4.93
4.60
4.79
4.93
5.13
4.48
4.50
4.48
5.87
9.53
7.76
50
3.07
3.04
3.11
3.12
3.19
2.97
2.96
2.97
3.30
5.79
4.70
100
2.15
2.15
2.98
2.97
3.02
2.12
2.11
2.12
2.23
3.88
3.21Table 3
Empirical coverage accuracy for the95% confidence intervals for the mean using the blockwise EL method (EL) and the blockwise EL method with the Bartlett correction (ELB) for the data generated from an AR(1) process with ϕ=0.5 and with a marginal negative binomial distribution Negbinom(μ,θ), based on 10 000 samples.
n
M
=
L
θ
=
0.1
θ
=
0.5
θ
=
1.0
EL
E
L
B
EL
E
L
B
EL
E
L
B
50
2
0.630
0.655
0.759
0.772
0.808
0.825
100
2
0.671
0.698
0.783
0.791
0.830
0.837
4
0.674
0.697
0.797
0.808
0.839
0.852
5
0.658
0.679
0.793
0.802
0.839
0.859
200
2
0.686
0.693
0.805
0.815
0.835
0.839
4
0.692
0.714
0.826
0.833
0.859
0.867
5
0.684
0.706
0.833
0.845
0.866
0.877
500
2
0.678
0.693
0.832
0.838
0.882
0.886
4
0.710
0.719
0.858
0.872
0.912
0.915
5
0.706
0.712
0.870
0.883
0.920
0.921
10
0.714
0.730
0.881
0.894
0.919
0.927
1000
2
0.724
0.735
0.827
0.836
0.871
0.875
4
0.746
0.757
0.871
0.877
0.902
0.907
5
0.751
0.762
0.869
0.872
0.909
0.913
10
0.749
0.765
0.878
0.887
0.923
0.926
20
0.749
0.770
0.882
0.897
0.924
0.932For smallerθ values, the degree of skewness of the negative binomial distribution is higher. Therefore the coverage accuracy increases along with the increase of the parameter θ. From the results from Tables 1 and 2, we can see that the coverage accuracy is higher for the confidence intervals based on the bounded Bernstein’s inequality than for those using the unbounded version. However, the resulting confidence intervals have larger average length when the bounded version is used.Comparing the empirical likelihood and the bootstrap methods, we can see that even the uncorrectedEL method works better than the bootstrap methods (the only exception is the studentized bootstrap, which also has one of the widest intervals) and also the average lengths of the EL based confidence intervals are only negligibly larger than those for the bootstrap methods.When the parameterθ=0.1 and data become much more asymmetric, the exponential type inequalities give the best coverage. However, as they are exact methods, for greater θ values, the empirical coverage becomes much larger than the nominal 95% confidence level. When using the Bartlett correction and the calibration with the F distribution, the empirical likelihood method provides much better coverage accuracy compared with the t-test and bootstrap methods.


## 2.1. Confidence Intervals Using Bernstein’s Inequality

Rosenblum and van der Laan [2] described the method for generating confidence intervals, which bounds the tails of the distribution of the sample mean of n independent, bounded random variables. This method is based on exponential type inequalities such as Bernstein’s inequality: for all x>0,
(2)ℙ(1n|∑i=1n‍(Xi-𝔼Xi)|>xn)≤2exp(-nx22(v+Wnx/3)),
where W and v are constants defined in Theorem 1.Theorem 1 (Rosenblum and van der Laan [2]).
LetX1,…,Xn be independent and identically distributed (iid) random variables with mean μ, and let W be such that ℙ(|X-μ|≤W)=1. Let σ*2 be a number satisfying σ*2≥Var(X). Then for the function
(3)q(1-α,W,nσ*2)≡n-1log(2α)(Wn-1/2+(W2n)9+(2n2σ*2)log(2/α)),ℙ(X--q(1-α,W,nσ*2)n≤μ≤X-+q(1-α,W,nσ*2)n)≥1-α.
In particular,
(4)(X--q(1-α,W,nσ*2)n,X-+q(1-α,W,nσ*2)n)
is the (1-α)100% confidence interval for μ.The accuracy of coverage probabilities depends on the estimation ofW and v, where v=nσ*2. Rosenblum and van der Laan [2] suggested that a bound W should be set at least as large as maxi|Xi-X-| and proposed also to use for σ*2 an estimator s2+SE(s2), where SE(s2) is the standard error of s2 (for details, see [2]).To examine exponential type inequalities for distributions like the negative binomial distribution, we will need a version of Bernstein’s inequality that does not require the assumption of boundedness.Lemma 2 (Birgé and Massart [14]).
LetZ1,…,Zn be independent random variables satisfying the moments conditions
(5)1n∑i=1n‍𝔼|Zim|≤m!2v2cm-2
for all m>2, for some positive constants v and c. Then, for any positive ε and Sn=∑i=1n‍Zi,
(6)ℙ[Sn-E(Sn)≥nε]≤exp(-nx)≤exp(-nε2/2v2+nε),
where x is defined by the equation ε=v2x+cx.Suppose thatZi, 1≤i≤n, are iid negative binomial with parameters μ and θ, and let Yi=Zi-μ. In this case, Shilane et al. [3] obtained the confidence interval for the mean of the form X-±ε, where ε is the solution to the equation
(7)(θ+μnε+θ+μ)-θ((θ+μ)(nε+μ)μ(nε+θ+μ))-nε-μ=α2.
To estimate the parameter θ, we can use the method of moments; that is, θ^=X-/((s2/X-)-1).


## 2.2. Empirical Likelihood for the Mean

LetX1,…,Xn be iid observations with the distribution F and mean μ=𝔼X1. To obtain confidence intervals for μ, we define the profile empirical likelihood function
(8)R(μ)=sup{∏i=1n‍npi|pi≥0,∑i=1n‍pi=1,∑i=1n‍piXi=μ}.
Owen [4, 5] showed that a unique value for the right–hand side of (8) exists when μ is inside the convex hull of the data points X1,…,Xn. This maximization problem can be solved using the Lagrange multiplier method.Theorem 3 (Owen [7]).
IfX1,…,Xn are iid random variables with the distribution function F0, μ0=𝔼(Xi), and 0<Var(Xi)<∞, then
(9)-2log(R(μ0))⟶dχ12.The empirical likelihood(1-α)100% confidence interval for μ is of the following form:
(10){μ∣-2log(R(μ))≤c},
where c is the 1-α quantile of the χ12 distribution. Owen [7] stated that the proof of Theorem 3 and some simulations suggest that the quantiles χ12,1-α can be replaced by the quantiles of the F distribution F1,n-11-α. The F calibration usually gives better results for small sample sizes.The Bartlett correction replaces the1-α quantile of the limiting chi-square distribution χ12,1-α of the test statistic -2log(R(μ)) by (1+an-1)χ12,1-α, where a=(1/2)μ4-(1/3)μ32 and μi is ith moment of X1 (see [6]). The Bartlett correction improves the asymptotic error rate for all coverage levels 1-α. A Bartlett-corrected empirical likelihood confidence interval has the following form:
(11){μ∣-2log(R(μ))≤(1+an-1)χ12,1-α}.


## 2.3. Simulation Study

Our simulation study is based on10,000 independent trials. In every trial, we generated iid random variables from the negative binomial distribution Negbinom(μ,θ) with different sample sizes n={20,50,100}, μ=5, and θ={0.1,0.5,1.0}. To simulate data, we used the command rnegbin from the package MASS in the R statistical programming language.In Table1, the empirical coverage accuracy for nominal 95% confidence intervals for the mean is evaluated for the following methods: the t-test (t), the empirical likelihood method (EL), the empirical likelihood with Bartlett correction (ELB), the empirical likelihood method with the F distribution calibration (ELF), and the empirical likelihood method with the F distribution calibration and Bartlett correction (ELFB). For comparison, we have also simulated the coverage accuracy of several bootstrap resampling procedures: the percentile (Bperc), normal (Bnorm), basic (Bbasic), and studentized (Bstud) bootstrap resampling methods. The R package boot and command boot.ci are used to construct the confidence intervals for all the above mentioned bootstrap methods. Although only the unbounded version of Bernstein inequalities is valid for the negative binomial distribution, we also apply the bounded version in order to see the possible effects of the violation of this assumption. In Table 2, we present average confidence interval lengths for the same methods as described in Table 1.Table 2
Average confidence interval lengths for the95% confidence intervals for the mean using different methods for the iid data generated with the marginal negative binomial distribution Negbinom(μ,θ), based on 10 000 samples.
n
t
EL
E
L
B
E
L
F
E
L
F
B
B
perc
B
norm
B
basic
B
stud
B
bn
B
ubn
θ
=
0.1
20
12.80
12.11
12.91
12.98
13.85
11.11
11.67
11.11
69.33
28.53
26.59
50
8.17
8.33
8.75
8.56
8.99
7.76
7.89
7.76
15.27
18.07
15.49
100
5.95
6.17
6.40
6.25
6.49
5.80
5.85
5.80
8.15
12.74
10.53
θ
=
0.5
20
6.63
6.21
6.50
6.65
6.96
6.00
6.06
6.00
9.33
13.39
11.20
50
4.07
4.06
4.18
4.17
4.29
3.93
3.93
3.93
4.66
8.02
6.53
100
2.90
2.93
2.98
2.97
3.02
2.86
2.85
2.86
3.12
5.48
4.49
θ
=
1.0
20
4.93
4.60
4.79
4.93
5.13
4.48
4.50
4.48
5.87
9.53
7.76
50
3.07
3.04
3.11
3.12
3.19
2.97
2.96
2.97
3.30
5.79
4.70
100
2.15
2.15
2.98
2.97
3.02
2.12
2.11
2.12
2.23
3.88
3.21Table 3
Empirical coverage accuracy for the95% confidence intervals for the mean using the blockwise EL method (EL) and the blockwise EL method with the Bartlett correction (ELB) for the data generated from an AR(1) process with ϕ=0.5 and with a marginal negative binomial distribution Negbinom(μ,θ), based on 10 000 samples.
n
M
=
L
θ
=
0.1
θ
=
0.5
θ
=
1.0
EL
E
L
B
EL
E
L
B
EL
E
L
B
50
2
0.630
0.655
0.759
0.772
0.808
0.825
100
2
0.671
0.698
0.783
0.791
0.830
0.837
4
0.674
0.697
0.797
0.808
0.839
0.852
5
0.658
0.679
0.793
0.802
0.839
0.859
200
2
0.686
0.693
0.805
0.815
0.835
0.839
4
0.692
0.714
0.826
0.833
0.859
0.867
5
0.684
0.706
0.833
0.845
0.866
0.877
500
2
0.678
0.693
0.832
0.838
0.882
0.886
4
0.710
0.719
0.858
0.872
0.912
0.915
5
0.706
0.712
0.870
0.883
0.920
0.921
10
0.714
0.730
0.881
0.894
0.919
0.927
1000
2
0.724
0.735
0.827
0.836
0.871
0.875
4
0.746
0.757
0.871
0.877
0.902
0.907
5
0.751
0.762
0.869
0.872
0.909
0.913
10
0.749
0.765
0.878
0.887
0.923
0.926
20
0.749
0.770
0.882
0.897
0.924
0.932For smallerθ values, the degree of skewness of the negative binomial distribution is higher. Therefore the coverage accuracy increases along with the increase of the parameter θ. From the results from Tables 1 and 2, we can see that the coverage accuracy is higher for the confidence intervals based on the bounded Bernstein’s inequality than for those using the unbounded version. However, the resulting confidence intervals have larger average length when the bounded version is used.Comparing the empirical likelihood and the bootstrap methods, we can see that even the uncorrectedEL method works better than the bootstrap methods (the only exception is the studentized bootstrap, which also has one of the widest intervals) and also the average lengths of the EL based confidence intervals are only negligibly larger than those for the bootstrap methods.When the parameterθ=0.1 and data become much more asymmetric, the exponential type inequalities give the best coverage. However, as they are exact methods, for greater θ values, the empirical coverage becomes much larger than the nominal 95% confidence level. When using the Bartlett correction and the calibration with the F distribution, the empirical likelihood method provides much better coverage accuracy compared with the t-test and bootstrap methods.


## 3. Dependent Case

### 3.1. Blockwise Empirical Likelihood for the Mean

In practice, the assumption of independence may be invalid in many situations, and by relaxing this strong assumption, we extend the scope of our research. As first shown by Kitamura [12], the empirical likelihood method can be applied not only for independent but also for stationary weakly dependent data characterized by some mixing coefficients.Let(Xt)t∈ℤ be a real-valued strictly stationary process defined on a probability space (Ω,ℱ,P). For any two σ-fields 𝒜 and ℬ⊂ℱ, define the following coefficient of dependence:
(12)α(𝒜,ℬ):=sup|P(A∩B)-P(A)P(B)|,
where the supremum is taken over all A∈𝒜 and B∈ℬ. For -∞≤J≤L≤∞, define FJL:=σ(Xk,J≤k≤L). (Xt)t∈ℤ is called strongly mixing or α-mixing (see, e.g., [15]) if α(n):=α(F-∞0,Fn∞)→0 when n→∞.Further, we also define another coefficient for any twoσ-fields 𝒜 and ℬ⊂ℱ as follows:
(13)β(𝒜,ℬ):=sup12∑i=1I‍∑j=1J‍|P(Ai∩Bj)-P(Ai)P(Bj)|,
where the supremum is taken over all {A1,…,AI} and {B1,…,BJ} from Ω such that Ai∈𝒜 for all i and Bj∈ℬ for all j. Define FJL:=σ(Xk,J≤k≤L), when -∞≤J≤L≤∞. (Xt)t∈ℤ is called absolutely regular or β-mixing if β(n):=supJ∈ℤβ(F-∞J,FJ+n∞)→0 when n→∞.Under certain (usually mild) conditions, ARMA, GARCH, nonlinear time series models, and Markov chains areβ-mixing processes (see, e.g., [15]). It is important to note that if a process is β-mixing, then it follows that it is also an α-mixing sequence.Further, we will use the notation and the results from two papers: Zhang et al. [16] and Kitamura [12]. Let G(X,μ) be a real-valued function such that 𝔼{G(X,μ)}=0, where μ=E(Xt). Let M and L be the integers depending on n, M→∞,  M=o(n1/2), and L=O(M) as n→∞. Set Q=[(n-M)/L]+1, where [x] is the integer part of x. The Q blocks of M consecutive observations
(14)Bi:=(X(i-1)L+1,…,X(i-1)L+M),i=1,…,Q
will be used for the estimation. Note that M is the block length and L is the separation between the block starting points. Furthermore, we will use nonoverlapping blocks with L=M.In order to address the dependence amongXi Kitamura, [12] proposed to use the profile empirical likelihood method to the blockwise sample {∑j=1M‍G(X(i-1)L+j,μ)}i=1Q instead of {G(Xi,μ)}i=1n. More specifically, Kitamura [12] defined the blockwise empirical likelihood function for μ as (15)L(μ)=sup{∏i=1Q‍(Qpi):p1≥0,…,pQ≥0,∑i=1Q‍pi=1,∑i=1Q‍pi∑j=1M‍G(X(i-1)L+j,μ)=0},which generates the blockwise empirical likelihood ratio statistic
(16)LR(μ)=(QMn)-1(2L(μ)-2L(μ^)).
Under some regularity conditions, Kitamura [12] showed that LR(μ0) also has a χ12 limiting distribution.The coverage error of confidence intervals(17)ℙ{LR(μ0)≤z}=ℙ{χ12≤z}+O(n-2/3)
can be improved up to the order O(n-5/6) by using the Bartlett correction; that is,
(18)ℙ{LR(μ0)(1-n-1a)≤z}=ℙ{χ12≤z}+O(n-5/6),
where a=(1/2)k4-(1/3)k32 and ki denotes the ith moment of Bi. This rate is slower than the rate obtained for the independent data due to the nonparametric treatment of dependence.


### 3.2. Exponential Type Inequalities

In this section, we introduce several exponential type inequalities forα- and β-mixing sequences and discuss their applicability in practice.Theorem 4 (Doukhan [17]).
Let(Xt)t∈ℤ be a zero-mean real-valued process such that ∃σ2∈ℝ+, for all n,m∈ℕ:(1/m)(Xn+⋯+Xn+m)2≤σ2 and for all t∈ℕ:|Xt|≤M. Then for each ε>0,  θ=ε2/4, and q≤n/(1+θ),
(19)ℙ(|Sn|≥x)≤4exp(-(1-ε)x22(nσ2+qMx/3))+2nβ(⌈qθ⌉-1)q.Theorem 5 (Bosq [18]).
Let(Xt)t∈ℕ be a zero-mean real-valued process such that sup1≤t≤n∥Xt∥∞≤b. Then, for each integer q∈[1,n/2] and each ε>0,
(20)ℙ(|Sn|>nε)≤4exp(-ε2q8b2)+22(1+4bε)1/2qα([n2q]).Both inequalities involve mixing coefficients, which in practice need to be estimated. Recently McDonald et al. [11] proposed a method for the β-mixing coefficient that uses the histogram estimator for stationary time series data. To our knowledge, this is the first result in this area that allows us to estimate such general mixing coefficients. The estimator has the following form:
(21)β^(a)=12∫‍|f^a2-f^⊗f^|,
where f^ is the histogram estimator of the joint density of observations and f^a2 is the 2-dimensional histogram estimator of the joint density of the two sets of observations separated by a time points. Typically mixing coefficients decay exponentially fast; therefore, it is possible to get reasonable estimates only for small values of a.In a recent paper, Merlevède et al. [10] obtained the exponential type inequality depending on the following condition on a strong mixing coefficient; that is,
(22)α(n)≤exp(-2cn),
where c is some constant. Here, we present the inequality given in Corollary 12 by Merlevède et al. [10].Corollary 6 (Merlevède et al. [10]).
Let(Xt)t∈ℕ be a sequence of centred real-valued random variables. Suppose that the sequence satisfies (22) and that there exists a positive M such that supi≥1∥Xi∥∞≤M. For all n≥2(c∨2) and x≥0,
(23)ℙ(|Sn|≥x)≤exp(x2n(logn)4CM2+4Mx/(c∧1)),
where
(24)C=6.2K+(1c+8c2)+2clog2,K=1+∑i≥1‍αi, and αi is defined in (22).Note that the inequality (23) only depends on the value of the constant c. Using the β-mixing estimator (21) from McDonald et al. [11], we propose to estimate c by the linear regression estimator of the slope from the equation
(25)log(β^(a))=c1-2cn,
where c1 is some constant. The inequalities defined in Theorems 4 and 5 can be used for practical applications after the estimation of c, by exploiting the relation α(a)≤β(a).


### 3.3. Simulation Study

We simulateX1,…,Xn from a stationary AR(1) process {Xt}t∈Z defined as
(26)Xt-ϕXt-1=Zt,
where {Zt}t∈Z is an innovation process which is weakly stationary with mean 0 and autocovariance 𝔼(ZtZt+h)=σZ2<∞ if h=0 and 0 otherwise, and |ϕ|≤1 is the coefficient of the process. The data are generated with a marginal negative binomial distribution using the R package gsarima. We used the command garsim and selected the parameter link=“identity.” We used different sample sizes n={50,100,200,500,1000} and selected different values for the parameter θ={0.1,0.5,1.0}. The minimal value of μ was set equal to 0.5 for the algorithm used by the command garsim.In Table3, we present the empirical coverage accuracies for the 95% confidence intervals constructed using the blockwise empirical likelihood method with and without the Bartlett correction. We report the corresponding average lengths of the 95% confidence intervals in Table 4. Similar to the independent case the, Bartlett correction provides some improvement in all cases. Furthermore, the results strongly depend on the block length parameter M.Table 4
Empirical average interval lengths for the95% confidence intervals for the mean using the blockwise EL method (EL) and the blockwise EL method with the Bartlett correction (ELB) for the data generated from an AR(1) process with ϕ=0.5 and with a marginal negative binomial distribution Negbinom(μ,θ), based on 10 000 samples.
n
M
=
L
θ
=
0.1
θ
=
0.5
θ
=
1.0
EL
E
L
B
EL
E
L
B
EL
E
L
B
50
2
1.62
1.70
0.85
0.88
0.65
0.68
100
2
1.47
1.54
0.68
0.71
0.51
0.52
4
1.49
1.55
0.73
0.76
0.54
0.56
5
1.48
1.55
0.73
0.76
0.55
0.57
200
2
1.25
1.31
0.52
0.53
0.38
0.39
4
1.32
1.37
0.57
0.58
0.41
0.43
5
1.31
1.36
0.58
0.59
0.42
0.44
500
2
0.97
1.02
0.34
0.35
0.25
0.25
4
1.04
1.08
0.38
0.39
0.28
0.28
5
1.03
1.08
0.39
0.40
0.28
0.29
10
1.06
1.10
0.40
0.41
0.30
0.31
1000
2
0.84
0.88
0.25
0.26
0.18
0.18
4
0.90
0.93
0.29
0.29
0.20
0.20
5
0.91
0.95
0.29
0.30
0.20
0.21
10
0.92
0.95
0.31
0.31
0.21
0.22
20
0.92
0.95
0.31
0.32
0.22
0.22A general autoregressive process AR(1) has the mixing property if the process Zt defined in (26) has a continuous marginal density (see [15]). Thus, it might not be appropriate to use exponential type inequalities in the case of the negative binomial distribution. We performed more simulations using a continuous distribution such as the normal distribution. In this case, the confidence intervals using Bernstein type inequalities typically are too large in comparison with those produced by the blockwise empirical likelihood method.Finally, we analyze the serial analysis of gene expression data (SAGE) from Shilane et al. [3]. This dataset is used in molecular biology to estimate the relative abundance of messenger ribonucleic acid (mRNA) molecules based upon the frequency of corresponding 14 base pair tag sequences that are extracted from a cell. Due to the fact that the cost of sequencing can be quite high, the sample size is often quite small for such data situations.Shilane et al. [3] picked the 20 most frequent tags from the whole data set and constructed the confidence intervals for the mean of the corresponding counts using several methods: bounded and unbounded Bernstein type inequalities, χ2, Gamma, Wald, t-test, and bias corrected and accelerated bootstrap methods. Additionally, we report the confidence intervals obtained by the empirical likelihood method in Table 5.Table 5
95% confidence intervals for the mean of the 20 most frequent tag counts for the SAGE data.
Method
Lower limit
Upper limit
Length
Unbounded Bernstein
182.14
430.06
247.92
Bounded Bernstein
−455.02
1067.22
1522.24
Chi square
259.53
356.46
96.93
Gamma
160.82
497.35
336.53
Wald
−38.00
650.20
688.20
t-test
−61.36
673.56
734.92
Bootstrap
111.95
1015.75
903.80
EL
123.17
833.04
709.86Shilane et al. [3] outlined some strengths and weaknesses of the methods from Table 5. Both the intervals based on the bounded Bernstein inequality and the Wald method include a range of negative numbers as possible values for the mean μ, which is unreasonable for the SAGE data. By the contrast, the χ2, Gamma, bootstrap, and empirical likelihood methods give only nonnegative values.All the methods used in Table5 require the independence of the data. However, if we look at the whole data set, there are significant correlations (see Figure 1). Still in this case we can use the blockwise empirical likelihood method for statistical inference (see Table 6).Table 6
95% confidence intervals for the mean of the 20 most frequent tag counts for different block sizesM for the SAGE data.
M
=
L
EL
E
L
B
Lower
Upper
Length
Lower
Upper
Length
2
2.13
4.60
2.47
2.11
4.74
2.62
4
2.06
4.86
2.80
2.06
4.86
2.80
5
2.04
4.92
2.88
2.04
4.92
2.88
10
1.96
5.16
3.20
1.96
5.16
3.20
15
1.91
5.30
3.39
1.91
5.30
3.39
20
1.87
5.40
3.53
1.87
5.40
3.53
25
1.84
5.49
3.65
1.87
5.40
3.53
30
1.82
5.57
3.75
1.82
5.57
3.75
50
1.74
5.82
4.08
1.74
5.82
4.08Figure 1
Empirical autocorrelations for the SAGE data with different sample sizes,n=20 and n=7726.


## 3.1. Blockwise Empirical Likelihood for the Mean

In practice, the assumption of independence may be invalid in many situations, and by relaxing this strong assumption, we extend the scope of our research. As first shown by Kitamura [12], the empirical likelihood method can be applied not only for independent but also for stationary weakly dependent data characterized by some mixing coefficients.Let(Xt)t∈ℤ be a real-valued strictly stationary process defined on a probability space (Ω,ℱ,P). For any two σ-fields 𝒜 and ℬ⊂ℱ, define the following coefficient of dependence:
(12)α(𝒜,ℬ):=sup|P(A∩B)-P(A)P(B)|,
where the supremum is taken over all A∈𝒜 and B∈ℬ. For -∞≤J≤L≤∞, define FJL:=σ(Xk,J≤k≤L). (Xt)t∈ℤ is called strongly mixing or α-mixing (see, e.g., [15]) if α(n):=α(F-∞0,Fn∞)→0 when n→∞.Further, we also define another coefficient for any twoσ-fields 𝒜 and ℬ⊂ℱ as follows:
(13)β(𝒜,ℬ):=sup12∑i=1I‍∑j=1J‍|P(Ai∩Bj)-P(Ai)P(Bj)|,
where the supremum is taken over all {A1,…,AI} and {B1,…,BJ} from Ω such that Ai∈𝒜 for all i and Bj∈ℬ for all j. Define FJL:=σ(Xk,J≤k≤L), when -∞≤J≤L≤∞. (Xt)t∈ℤ is called absolutely regular or β-mixing if β(n):=supJ∈ℤβ(F-∞J,FJ+n∞)→0 when n→∞.Under certain (usually mild) conditions, ARMA, GARCH, nonlinear time series models, and Markov chains areβ-mixing processes (see, e.g., [15]). It is important to note that if a process is β-mixing, then it follows that it is also an α-mixing sequence.Further, we will use the notation and the results from two papers: Zhang et al. [16] and Kitamura [12]. Let G(X,μ) be a real-valued function such that 𝔼{G(X,μ)}=0, where μ=E(Xt). Let M and L be the integers depending on n, M→∞,  M=o(n1/2), and L=O(M) as n→∞. Set Q=[(n-M)/L]+1, where [x] is the integer part of x. The Q blocks of M consecutive observations
(14)Bi:=(X(i-1)L+1,…,X(i-1)L+M),i=1,…,Q
will be used for the estimation. Note that M is the block length and L is the separation between the block starting points. Furthermore, we will use nonoverlapping blocks with L=M.In order to address the dependence amongXi Kitamura, [12] proposed to use the profile empirical likelihood method to the blockwise sample {∑j=1M‍G(X(i-1)L+j,μ)}i=1Q instead of {G(Xi,μ)}i=1n. More specifically, Kitamura [12] defined the blockwise empirical likelihood function for μ as (15)L(μ)=sup{∏i=1Q‍(Qpi):p1≥0,…,pQ≥0,∑i=1Q‍pi=1,∑i=1Q‍pi∑j=1M‍G(X(i-1)L+j,μ)=0},which generates the blockwise empirical likelihood ratio statistic
(16)LR(μ)=(QMn)-1(2L(μ)-2L(μ^)).
Under some regularity conditions, Kitamura [12] showed that LR(μ0) also has a χ12 limiting distribution.The coverage error of confidence intervals(17)ℙ{LR(μ0)≤z}=ℙ{χ12≤z}+O(n-2/3)
can be improved up to the order O(n-5/6) by using the Bartlett correction; that is,
(18)ℙ{LR(μ0)(1-n-1a)≤z}=ℙ{χ12≤z}+O(n-5/6),
where a=(1/2)k4-(1/3)k32 and ki denotes the ith moment of Bi. This rate is slower than the rate obtained for the independent data due to the nonparametric treatment of dependence.


## 3.2. Exponential Type Inequalities

In this section, we introduce several exponential type inequalities forα- and β-mixing sequences and discuss their applicability in practice.Theorem 4 (Doukhan [17]).
Let(Xt)t∈ℤ be a zero-mean real-valued process such that ∃σ2∈ℝ+, for all n,m∈ℕ:(1/m)(Xn+⋯+Xn+m)2≤σ2 and for all t∈ℕ:|Xt|≤M. Then for each ε>0,  θ=ε2/4, and q≤n/(1+θ),
(19)ℙ(|Sn|≥x)≤4exp(-(1-ε)x22(nσ2+qMx/3))+2nβ(⌈qθ⌉-1)q.Theorem 5 (Bosq [18]).
Let(Xt)t∈ℕ be a zero-mean real-valued process such that sup1≤t≤n∥Xt∥∞≤b. Then, for each integer q∈[1,n/2] and each ε>0,
(20)ℙ(|Sn|>nε)≤4exp(-ε2q8b2)+22(1+4bε)1/2qα([n2q]).Both inequalities involve mixing coefficients, which in practice need to be estimated. Recently McDonald et al. [11] proposed a method for the β-mixing coefficient that uses the histogram estimator for stationary time series data. To our knowledge, this is the first result in this area that allows us to estimate such general mixing coefficients. The estimator has the following form:
(21)β^(a)=12∫‍|f^a2-f^⊗f^|,
where f^ is the histogram estimator of the joint density of observations and f^a2 is the 2-dimensional histogram estimator of the joint density of the two sets of observations separated by a time points. Typically mixing coefficients decay exponentially fast; therefore, it is possible to get reasonable estimates only for small values of a.In a recent paper, Merlevède et al. [10] obtained the exponential type inequality depending on the following condition on a strong mixing coefficient; that is,
(22)α(n)≤exp(-2cn),
where c is some constant. Here, we present the inequality given in Corollary 12 by Merlevède et al. [10].Corollary 6 (Merlevède et al. [10]).
Let(Xt)t∈ℕ be a sequence of centred real-valued random variables. Suppose that the sequence satisfies (22) and that there exists a positive M such that supi≥1∥Xi∥∞≤M. For all n≥2(c∨2) and x≥0,
(23)ℙ(|Sn|≥x)≤exp(x2n(logn)4CM2+4Mx/(c∧1)),
where
(24)C=6.2K+(1c+8c2)+2clog2,K=1+∑i≥1‍αi, and αi is defined in (22).Note that the inequality (23) only depends on the value of the constant c. Using the β-mixing estimator (21) from McDonald et al. [11], we propose to estimate c by the linear regression estimator of the slope from the equation
(25)log(β^(a))=c1-2cn,
where c1 is some constant. The inequalities defined in Theorems 4 and 5 can be used for practical applications after the estimation of c, by exploiting the relation α(a)≤β(a).


## 3.3. Simulation Study

We simulateX1,…,Xn from a stationary AR(1) process {Xt}t∈Z defined as
(26)Xt-ϕXt-1=Zt,
where {Zt}t∈Z is an innovation process which is weakly stationary with mean 0 and autocovariance 𝔼(ZtZt+h)=σZ2<∞ if h=0 and 0 otherwise, and |ϕ|≤1 is the coefficient of the process. The data are generated with a marginal negative binomial distribution using the R package gsarima. We used the command garsim and selected the parameter link=“identity.” We used different sample sizes n={50,100,200,500,1000} and selected different values for the parameter θ={0.1,0.5,1.0}. The minimal value of μ was set equal to 0.5 for the algorithm used by the command garsim.In Table3, we present the empirical coverage accuracies for the 95% confidence intervals constructed using the blockwise empirical likelihood method with and without the Bartlett correction. We report the corresponding average lengths of the 95% confidence intervals in Table 4. Similar to the independent case the, Bartlett correction provides some improvement in all cases. Furthermore, the results strongly depend on the block length parameter M.Table 4
Empirical average interval lengths for the95% confidence intervals for the mean using the blockwise EL method (EL) and the blockwise EL method with the Bartlett correction (ELB) for the data generated from an AR(1) process with ϕ=0.5 and with a marginal negative binomial distribution Negbinom(μ,θ), based on 10 000 samples.
n
M
=
L
θ
=
0.1
θ
=
0.5
θ
=
1.0
EL
E
L
B
EL
E
L
B
EL
E
L
B
50
2
1.62
1.70
0.85
0.88
0.65
0.68
100
2
1.47
1.54
0.68
0.71
0.51
0.52
4
1.49
1.55
0.73
0.76
0.54
0.56
5
1.48
1.55
0.73
0.76
0.55
0.57
200
2
1.25
1.31
0.52
0.53
0.38
0.39
4
1.32
1.37
0.57
0.58
0.41
0.43
5
1.31
1.36
0.58
0.59
0.42
0.44
500
2
0.97
1.02
0.34
0.35
0.25
0.25
4
1.04
1.08
0.38
0.39
0.28
0.28
5
1.03
1.08
0.39
0.40
0.28
0.29
10
1.06
1.10
0.40
0.41
0.30
0.31
1000
2
0.84
0.88
0.25
0.26
0.18
0.18
4
0.90
0.93
0.29
0.29
0.20
0.20
5
0.91
0.95
0.29
0.30
0.20
0.21
10
0.92
0.95
0.31
0.31
0.21
0.22
20
0.92
0.95
0.31
0.32
0.22
0.22A general autoregressive process AR(1) has the mixing property if the process Zt defined in (26) has a continuous marginal density (see [15]). Thus, it might not be appropriate to use exponential type inequalities in the case of the negative binomial distribution. We performed more simulations using a continuous distribution such as the normal distribution. In this case, the confidence intervals using Bernstein type inequalities typically are too large in comparison with those produced by the blockwise empirical likelihood method.Finally, we analyze the serial analysis of gene expression data (SAGE) from Shilane et al. [3]. This dataset is used in molecular biology to estimate the relative abundance of messenger ribonucleic acid (mRNA) molecules based upon the frequency of corresponding 14 base pair tag sequences that are extracted from a cell. Due to the fact that the cost of sequencing can be quite high, the sample size is often quite small for such data situations.Shilane et al. [3] picked the 20 most frequent tags from the whole data set and constructed the confidence intervals for the mean of the corresponding counts using several methods: bounded and unbounded Bernstein type inequalities, χ2, Gamma, Wald, t-test, and bias corrected and accelerated bootstrap methods. Additionally, we report the confidence intervals obtained by the empirical likelihood method in Table 5.Table 5
95% confidence intervals for the mean of the 20 most frequent tag counts for the SAGE data.
Method
Lower limit
Upper limit
Length
Unbounded Bernstein
182.14
430.06
247.92
Bounded Bernstein
−455.02
1067.22
1522.24
Chi square
259.53
356.46
96.93
Gamma
160.82
497.35
336.53
Wald
−38.00
650.20
688.20
t-test
−61.36
673.56
734.92
Bootstrap
111.95
1015.75
903.80
EL
123.17
833.04
709.86Shilane et al. [3] outlined some strengths and weaknesses of the methods from Table 5. Both the intervals based on the bounded Bernstein inequality and the Wald method include a range of negative numbers as possible values for the mean μ, which is unreasonable for the SAGE data. By the contrast, the χ2, Gamma, bootstrap, and empirical likelihood methods give only nonnegative values.All the methods used in Table5 require the independence of the data. However, if we look at the whole data set, there are significant correlations (see Figure 1). Still in this case we can use the blockwise empirical likelihood method for statistical inference (see Table 6).Table 6
95% confidence intervals for the mean of the 20 most frequent tag counts for different block sizesM for the SAGE data.
M
=
L
EL
E
L
B
Lower
Upper
Length
Lower
Upper
Length
2
2.13
4.60
2.47
2.11
4.74
2.62
4
2.06
4.86
2.80
2.06
4.86
2.80
5
2.04
4.92
2.88
2.04
4.92
2.88
10
1.96
5.16
3.20
1.96
5.16
3.20
15
1.91
5.30
3.39
1.91
5.30
3.39
20
1.87
5.40
3.53
1.87
5.40
3.53
25
1.84
5.49
3.65
1.87
5.40
3.53
30
1.82
5.57
3.75
1.82
5.57
3.75
50
1.74
5.82
4.08
1.74
5.82
4.08Figure 1
Empirical autocorrelations for the SAGE data with different sample sizes,n=20 and n=7726.


## 4. Conclusions

We conclude that large confidence intervals are typically obtained when Bernstein type inequalities are used. This method can still provide good results for highly skewed situations when the sample sizes are small. For the independent observations, the empirical likelihood method is a good alternative not only to the classicalt-test statistic but also to the bootstrap methods. The Bartlett correction and the calibration using the F distribution can provide a significant improvement for small sample sizes. For dependent data, in principle, it is possible to use Bernstein type inequalities for inference for the mean. However, for correlated time series data, usually bigger sample sizes are necessary. Second, for the negative binomial distribution, these inequalities are not valid for ARMA type processes. A reasonable alternative is to use the blockwise empirical likelihood method.



---

*Source: 765752-2013-10-08.xml*