# General Bootstrap for Dualϕ-Divergence Estimates

Authors: Salim Bouzebda; Mohamed Cherfi
Journal: Journal of Probability and Statistics
 (2012)

---

## Abstract

A general notion of bootstrappedϕ-divergence estimates constructed
by exchangeably weighting sample is introduced. Asymptotic properties of these
generalized bootstrapped ϕ-divergence estimates are obtained, by means of the
empirical process theory, which are applied to construct the bootstrap confidence
set with asymptotically correct coverage probability. Some of practical problems
are discussed, including, in particular, the choice of escort parameter, and several
examples of divergences are investigated. Simulation results are provided to illustrate
the finite sample performance of the proposed estimators.


---

## Body

## 1. Introduction

Theϕ-divergence modeling has proved to be a flexible tool and provided a powerful statistical modeling framework in a variety of applied and theoretical contexts (refer to [1–4] and the references therein). For good recent sources of references to the research literature in this area along with statistical applications, consult [2, 5]. Unfortunately, in general, the limiting distribution of the estimators, or their functionals, based on ϕ-divergences depends crucially on the unknown distribution, which is a serious problem in practice. To circumvent this matter, we will propose, in this work, a general bootstrap of ϕ-divergence-based estimators and study some of its properties by means of sophisticated empirical process techniques. A major application for an estimator is in the calculation of confidence intervals. By far the most favored confidence interval is the standard confidence interval based on a normal or a Student’s t-distribution. Such standard intervals are useful tools, but they are based on an approximation that can be quite inaccurate in practice. Bootstrap procedures are an attractive alternative. One way to look at them is as procedures for handling data when one is not willing to make assumptions about the parameters of the populations from which one sampled. The most that one is willing to assume is that the data are a reasonable representation of the population from which they come. One then resamples from the data and draws inferences about the corresponding population and its parameters. The resulting confidence intervals have received the most theoretical study of any topic in the bootstrap analysis.Our main findings, which are analogous to that of Cheng and Huang [6], are summarized as follows. The ϕ-divergence estimator α̂ϕ(θ) and the bootstrap ϕ-divergence estimator α̂ϕ*(θ) are obtained by optimizing the objective function h(θ,α) based on the independent and identically distributed (i.i.d) observations X1,…,Xn and the bootstrap sample X1*,…,Xn*, respectively,(1.1)α̂ϕ(θ):=argsup⁡α∈Θ1n∑i=1nh(θ,α,Xi),α̂ϕ*(θ):=argsup⁡α∈Θ1n∑i=1nh(θ,α,Xi*),
where X1*,…,Xn* are independent draws with replacement from the original sample. We will mention that α̂ϕ*(θ) can alternatively be expressed as(1.2)α̂ϕ*(θ)=argsup⁡α∈Θ1n∑i=1nWnih(θ,α,Xi),
where the bootstrap weights are given by(1.3)(Wn1,…,Wnn)~Multinomial(n;n-1,…,n-1).In this paper, we will consider the more general exchangeable bootstrap weighting scheme that includes Efron’s bootstrap [7, 8]. The general resampling scheme was first proposed in [9] and extensively studied by Bickel and Freedman [10], who suggested the name “weighted bootstrap”; for example, Bayesian Bootstrap when (Wn1,…,Wnn)=(Dn1,…,Dnn) is equal in distribution to the vector of n spacings of n-1 ordered uniform (0,1) random variables, that is,(1.4)(Dn1,…,Dnn)~Dirichlet(n;1,…,1).The interested reader may refer to [11]. The case(1.5)(Dn1,…,Dnn)~Dirichlet(n;4,…,4)was considered in [12, Remark 2.3] and [13, Remark 5]. The Bickel and Freedman result concerning the empirical process has been subsequently generalized for empirical processes based on observations in ℝd, d>1 as well as in very general sample spaces and for various set and function-indexed random objects (see, e.g., [14–18]). In this framework, [19] developed similar results for a variety of other statistical functions. This line of research was continued in the work of [20, 21]. There is a huge literature on the application of the bootstrap methodology to nonparametric kernel density and regression estimation, among other statistical procedures, and it is not the purpose of this paper to survey this extensive literature. This being said, it is worthwhile mentioning that the bootstrap as per Efron’s original formulation (see [7]) presents some drawbacks. Namely, some observations may be used more than once while others are not sampled at all. To overcome this difficulty, a more general formulation of the bootstrap has been devised: the weighted (or smooth) bootstrap, which has also been shown to be computationally more efficient in several applications. We may refer to [22–24]. Holmes and Reinert [25] provided new proofs for many known results about the convergence in law of the bootstrap distribution to the true distribution of smooth statistics employing the techniques based on Stein’s method for empirical processes. Note that other variations of Efron’s bootstrap are studied in [26] using the term “generalized bootstrap.” The practical usefulness of the more general scheme is well documented in the literature. For a survey of further results on weighted bootstrap, the reader is referred to [27].The remainder of this paper is organized as follows. In the forthcoming section we recall the estimation procedure based onϕ-divergences. The bootstrap of ϕ-divergence estimators is introduced, in detail, and their asymptotic properties are given in Section 3. In Section 4, we provide some examples explaining the computation of the ϕ-divergence estimators. In Section 5, we illustrate how to apply our results in the context of right censoring. Section 6 provides simulation results in order to illustrate the performance of the proposed estimators. To avoid interrupting the flow of the presentation, all mathematical developments are relegated to the appendix.


## 2. Dual Divergence-Based Estimates

The class ofdual divergence estimators has been recently introduced by Keziou [28] and Broniatowski and Keziou [1]. Recall that the ϕ-divergence between a bounded signed measure ℚ and a probability measure ℙ on 𝒟, when ℚ is absolutely continuous with respect to ℙ, is defined by(2.1)Dϕ(Q,P):=∫Dϕ(dQdP)dP,whereϕ(·) is a convex function from ]-∞,∞[ to [0,∞] with ϕ(1)=0. We will consider only ϕ-divergences for which the function ϕ(·) is strictly convex and satisfies the domain of ϕ(·), domϕ:={x∈ℝ:ϕ(x)<∞} is an interval with end points(2.2)aϕ<1<bϕ,ϕ(aϕ)=lim⁡x↓aϕϕ(x),ϕ(aϕ)=lim⁡x↑bϕϕ(x).The Kullback-Leibler, modified Kullback-Leibler,χ2, modified χ2, and Hellinger divergences are examples of ϕ-divergences; they are obtained, respectively, for ϕ(x)=xlogx-x+1, ϕ(x)=-logx+x-1, ϕ(x)=(1/2)(x-1)2, ϕ(x)=(1/2)((x-1)2/x), and ϕ(x)=2(x-1)2. The squared Le Cam distance (sometimes called the Vincze-Le Cam distance) and ℒ1-error are obtained, respectively, for(2.3)ϕ(x)=(x-1)22(x-1),ϕ(x)=|x-1|.We extend the definition of these divergences on the whole space of all bounded signed measures via the extension of the definition of the correspondingϕ(·) functions on the whole real space ℝ as follows: when ϕ(·) is not well defined on ℝ- or well defined but not convex on ℝ, we set ϕ(x)=+∞ for all x<0. Notice that, for the χ2-divergence, the corresponding ϕ(·) function is defined on whole ℝ and strictly convex. All the above examples are particular cases of the so-called “power divergences,” introduced by Cressie and Read [29] (see also [4, Chapter 2], and also Rényi’s paper [30] is to be mentioned here), which are defined through the class of convex real-valued functions, for γ in ℝ∖{0,1},(2.4)x∈R+*⟶ϕγ(x):=xγ-γx+γ-1γ(γ-1),ϕ0(x):=-logx+x-1, and ϕ1(x):=xlogx-x+1. (For all γ∈ℝ, we define ϕγ(0):=limx↓0ϕγ(x).) So, the KL-divergence is associated to ϕ1, the KLm to ϕ0, the χ2 to ϕ2, the χm2 to ϕ-1, and the Hellinger distance to ϕ1/2. In the monograph by [4], the reader may find detailed ingredients of the modeling theory as well as surveys of the commonly used divergences.Let{ℙθ:θ∈Θ} be some identifiable parametric model with Θ a compact subset of ℝd. Consider the problem of estimation of the unknown true value of the parameter θ0 on the basis of an i.i.d sample X1,…,Xn. We will assume that the observed data are from the probability space (𝒳,𝒜,ℙθ0). Let ϕ(·) be a function of class 𝒞2, strictly convex such that(2.5)∫|ϕ′(dPθ(x)dPα(x))|dPθ(x)<∞,∀α∈Θ.
As it is mentioned in [1], if the function ϕ(·) satisfies the following conditions:(2.6)thereexists0<δ<1suchthatforallcin[1-δ,1+δ],wecanfindnumbersc1,c2,c3suchthatϕ(cx)≤c1ϕ(x)+c2|x|+c3,∀realx,
then the assumption (2.5) is satisfied whenever Dϕ(θ,α)<∞, where Dϕ(θ,α) stands for the ϕ-divergence between ℙθ and ℙα; refer to [31, Lemma 3.2]. Also the real convex functions ϕ(·) (2.4), associated with the class of power divergences, all satisfy the condition (2.5), including all standard divergences. Under assumption (2.5), using Fenchel duality technique, the divergence Dϕ(θ,θ0) can be represented as resulting from an optimization procedure, this result was elegantly proved in [1, 3, 28]. Broniatowski and Keziou [31] called it the dual form of a divergence, due to its connection with convex analysis. According to [3], under the strict convexity and the differentiability of the function ϕ(·), it holds(2.7)ϕ(t)≥ϕ(s)+ϕ′(s)(t-s),
where the equality holds only for s=t. Let θ and θ0 be fixed, and put t=dℙθ(x)/dℙθ0(x) and s=dℙθ(x)/dℙα(x) in (2.7), and then integrate with respect to ℙθ0, to obtain(2.8)Dϕ(θ,θ0):=∫ϕ(dPθdPθ0)dPθ0=sup⁡α∈Θ∫h(θ,α)dPθ0,
where h(θ,α,·):x↦h(θ,α,x) and(2.9)h(θ,α,x):=∫ϕ′(dPθdPα)dPθ-[dPθ(x)dPα(x)ϕ′(dPθ(x)dPα(x))-ϕ(dPθ(x)dPα(x))].
Furthermore, the supremum in this display (2.8) is unique and reached in α=θ0, independently upon the value of θ. Naturally, a class of estimators of θ0, called “dual ϕ-divergence estimators” (DϕDEs), is defined by(2.10)α̂ϕ(θ):=argsup⁡α∈ΘPnh(θ,α),θ∈Θ,
where h(θ,α) is the function defined in (2.9) and, for a measurable function f(·),(2.11)Pnf:=n-1∑i=1nf(Xi).The class of estimatorsα̂ϕ(θ) satisfies(2.12)Pn∂∂αh(θ,α̂ϕ(θ))=0.
Formula (2.10) defines a family of M-estimators indexed by the function ϕ(·) specifying the divergence and by some instrumental value of the parameter θ. The ϕ-divergence estimators are motivated by the fact that a suitable choice of the divergence may lead to an estimate more robust than the maximum likelihood estimator (MLE) one; see [32]. Toma and Broniatowski [33] studied the robustness of the DϕDEs through the influence function approach; they treated numerous examples of location-scale models and give sufficient conditions for the robustness of DϕDEs. We recall that the maximum likelihood estimate belongs to the class of estimates (2.10). Indeed, it is obtained when ϕ(x)=-logx+x-1, that is, as the dual modified KLm-divergence estimate. Observe that ϕ′(x)=-(1/x)+1 and xϕ′(x)-ϕ(x)=logx, and hence(2.13)∫h(θ,α)dPn=-∫log⁡(dPθdPα)dPn.Keeping in mind definitions (2.10), we get(2.14)α̂KLm(θ)=argsup⁡α-∫log⁡(dPθdPα)dPn=argsup⁡α∫log⁡(dPα)dPn=MLE,independently uponθ.


## 3. Asymptotic Properties

In this section, we will establish the consistency of bootstrapping under general conditions in the framework of dual divergence estimation. Define, for a measurable functionf(·),(3.1)Pn*f:=1n∑i=1nWnif(Xi),whereWni’s are the bootstrap weights defined on the probability space (𝒲,Ω,ℙW). In view of (2.10), the bootstrap estimator can be rewritten as(3.2)α̂ϕ*(θ):=argsup⁡α∈ΘPn*h(θ,α).
The definition of α̂ϕ*(θ), defined in (3.2), implies that(3.3)Pn*∂∂αh(θ,α̂ϕ*(θ))=0.
The bootstrap weights Wni’s are assumed to belong to the class of exchangeable bootstrap weights introduced in [23]. In the sequel, the transpose of a vector x will be denoted by x⊤. We will assume the following conditions.(W.1)
The vectorWn=(Wn1,…,Wnn)⊤is exchangeable for alln=1,2,…; that is, for any permutationπ=(π1,…,πn)of(1,…,n), the joint distribution ofπ(Wn)=(Wnπ1,…,Wnπn)⊤is the same as that ofWn.(W.2)
Wni≥0for alln,iand∑i=1nWni=nfor alln.(W.3)
limsupn→∞∥Wn1∥2,1≤C<∞, where(3.4)‖Wn1‖2,1=∫0∞PW(Wn1≥u)du.(W.4)
One has(3.5)lim⁡λ→∞lim⁡sup⁡n→∞sup⁡t≥λt2PW(Wn1>t)=0.(W.5)
(1/n)∑i=1n(Wni-1)2→ℙWc2>0.In Efron’s nonparametric bootstrap, the bootstrap sample is drawn from the nonparametric estimate of the true distribution, that is, empirical distribution. Thus, it is easy to show thatWn~Multinomial(n;n-1,…,n-1) and conditions (W.1)–(W.5) are satisfied. In general, conditions (W.3)–(W.5) are easily satisfied under some moment conditions on Wni; see [23, Lemma 3.1]. In addition to Efron’s nonparametric bootstrap, the sampling schemes that satisfy conditions (W.1)–(W.5) include Bayesian bootstrap, Multiplier bootstrap, Double bootstrap, andUrn bootstrap. This list is sufficiently long to indicate that conditions (W.1)–(W.5), are not unduly restrictive. Notice that the value of c in (W.5) is independent of n and depends on the resampling method, for example, c=1 for the nonparametric bootstrap and Bayesian bootstrap and c=2 for the double bootstrap. A more precise discussion of this general formulation of the bootstrap can be found in [23, 34, 35].There exist two sources of randomness for the bootstrapped quantity, that is,α̂ϕ*(θ): the first comes from the observed data and the second is due to the resampling done by the bootstrap, that is, random Wni’s. Therefore, in order to rigorously state our main theoretical results for the general bootstrap of ϕ-divergence estimates, we need to specify relevant probability spaces and define stochastic orders with respect to relevant probability measures. Following [6, 36], we will view Xi as the ith coordinate projection from the canonical probability space (𝒳∞,𝒜∞,ℙθ0∞) onto the ith copy of 𝒳. For the joint randomness involved, the product probability space is defined as(3.6)(X∞,A∞,Pθ0∞)×(W,Ω,PW)=(X∞×W,A∞×Ω,Pθ0∞×PW).Throughout the paper, we assume that the bootstrap weightsWni’s are independent of the data Xi’s, thus(3.7)PXW=Pθ0×PW.
Given a real-valued function Δn defined on the above product probability space, for example, α̂ϕ*(θ), we say that Δn is of an order OℙWo(1) in ℙθ0-probability if, for any ϵ,η>0, as n→0,(3.8)Pθ0{PW∣Xo(|Δn|>ϵ)>η}⟶0and thatΔn is of an order OℙWo(1) in ℙθ0-probability if, for any η>0, there exists a 0<M<∞ such that, as n→0,(3.9)Pθ0{PW∣Xo(|Δn|≥M)>η}⟶0,where the superscript “o” denotes the outer probability; see [34] for more details on outer probability measures. For more details on stochastic orders, the interested reader may refer to [6], in particular, Lemma 3 of the cited reference.To establish the consistency ofα̂ϕ*(θ), the following conditions are assumed in our analysis.(A.1)
One has(3.10)Pθ0h(θ,θ0)>sup⁡α∉N(θ0)Pθ0h(θ,α)
for any open set N(θ0)⊂Θ containing θ0.(A.2)
One has(3.11)sup⁡α∈Θ|Pn*h(θ,α)-Pθ0h(θ,α)|→PXWo0.
The following theorem gives the consistency of the bootstrapped estimateα̂ϕ*(θ).Theorem 3.1.
Assume that conditions (A.1) and (A.2) hold. Suppose that conditions (A.3)–(A.5) and (W.1)–(W.5) hold. Thenα̂ϕ*(θ) is a consistent estimate of θ0; that is,
(3.12)α̂ϕ*(θ)→PWoθ0inPθ0-probability.The proof of Theorem3.1 is postponed until the appendix.We need the following definitions; refer to [34, 37] among others. If ℱ is a class of functions for which, we have almost surely,(3.13)‖Pn-P‖F=sup⁡f∈F|Pnf-Pf|⟶0,then we say thatℱ is a ℙ-Glivenko-Cantelli class of functions. If ℱ is a class of functions for which(3.14)Gn=n(Pn-P)⟶Ginl∞(F),where𝔾 is a mean-zero ℙ-Brownian bridge process with (uniformly) continuous sample paths with respect to the semimetric ρℙ(f,g), defined by(3.15)ρP2(f,g)=VarP(f(X)-g(X)),then we say thatℱ is a ℙ-Donsker class of functions. Here(3.16)l∞(F)={v:F⟼R∣‖v‖F=sup⁡f∈F|v(f)|<∞}and𝔾 is a ℙ-Brownian bridge process on ℱ if it is a mean-zero Gaussian process with covariance function(3.17)E(G(f)G(g))=Pfg-(Pf)(Pg).Remark 3.2.
(i) Condition (A.1) is the “well-separated” condition, compactness of the parameter spaceΘ and the continuity of divergence imply that the optimum is well separated, provided the parametric model is identified; see [37, Theorem 5.7].
(ii) Condition (A.2) holds if the class(3.18){h(θ,α):α∈Θ}
is shown to be ℙ-Glivenko-Cantelli, by applying [34, Lemma 3.6.16] and [6, Lemma A.1].For any fixedδn>0, define the class of functions ℋn and ℋ̇n as(3.19)Hn:={∂∂αh(θ,α):‖α-θ0‖≤δn},Ḣn:={∂2∂α2h(θ,α):‖α-θ0‖≤δn}.
We will say a class of functions ℋ∈M(ℙθ0) if ℋ possesses enough measurability for randomization with i.i.d multipliers to be possible, that is, ℙn can be randomized, in other words, we can replace (δXi-ℙθ0) by (Wni-1)δXi. It is known that ℋ∈M(ℙθ0), for example, if ℋ is countable,  if {ℙn}n∞ are stochastically separable in ℋ, or if ℋ is image admissible Suslin; see [21, pages 853 and 854].To state our result concerning the asymptotic normality, we will assume the following additional conditions.(A.3)
The matrices(3.20)V:=Pθ0∂∂αh(θ,θ0)∂∂αh(θ,θ0)⊤,S:=-Pθ0∂2∂α2h(θ,θ0)
are nonsingular.(A.4)
The classℋn∈M(ℙθ0)∩L2(ℙθ0) and is ℙ-Donsker.(A.5)
The classℋ̇n∈M(ℙθ0)∩L2(ℙθ0) and is ℙ-Donsker.Conditions (A.4) and (A.5) ensure that the “size” of the function classesℋn and ℋ̇n are reasonable so that the bootstrapped empirical processes(3.21)Gn*≡n(Pn*-Pn)indexed, respectively, byℋn and ℋ̇n, have a limiting process conditional on the original observations; we refer, for instance, to [23, Theorem 2.2]. The main result to be proved here may now be stated precisely as follows.Theorem 3.3.
Assume thatα̂ϕ(θ) and α̂ϕ*(θ) fulfil (2.12) and (3.3), respectively. In addition suppose that
(3.22)α̂ϕ(θ)→Pθ0θ0,α̂ϕ*(θ)→PWoθ0inPθ0-probability.
Assume that conditions (A.3)–(A.5) and (W.1)–(W.5) hold. Then one has
(3.23)‖α̂ϕ*(θ)-θ0‖=OPWo(n-1/2)
in ℙθ0-probability. Furthermore,
(3.24)n(α̂ϕ*(θ)-α̂ϕ(θ))=-S-1Gn*∂∂αh(θ,θ0)+oPWo(1)
in ℙθ0-probability. Consequently,
(3.25)sup⁡x∈Rd|PW∣Xn((nc)(α̂ϕ*(θ)-α̂ϕ(θ))≤x)-P(N(0,Σ)≤x)|=oPθ0(1),
where “≤” is taken componentwise and “c” is given in (W.5), whose value depends on the used sampling scheme, and
(3.26)Σ≡S-1V(S-1)⊤,
where S and V are given in condition (A.3). Thus, one has
(3.27)sup⁡x∈Rd|PW∣Xn((nc)(α̂ϕ*(θ)-α̂ϕ(θ))≤x)-Pθ0(n(α̂ϕ(θ)-θ0)≤x)|→Pθ00.The proof of Theorem3.1 is captured in the forthcoming appendix.Remark 3.4.
Note that an appropriate choice of the bootstrap weightsWni’s implicates a smaller limit variance; that is, c2 is smaller than 1. For instance, typical examples are i.i.d-weighted bootstraps and the multivariate hypergeometric bootstrap; refer to [23, Examples 3.1 and 3.4].Following [6], we will illustrate how to apply our results to construct the confidence sets. A lower ϵth quantile of bootstrap distribution is defined to be any qnϵ*∈ℝd fulfilling(3.28)qnϵ*:=inf⁡{x:PW∣Xn(α̂ϕ*(θ)≤x)≥ϵ},wherex is an infimum over the given set only if there does not exist a x1<x in ℝd such that(3.29)PW∣Xn(α̂ϕ*(θ)≤x1)≥ϵ.Keep in mind the assumed regularity conditions on the criterion function, that is,h(θ,α) in the present framework, we can, without loss of generality, suppose that(3.30)PW∣Xn(α̂ϕ*(θ)≤qnϵ*)=ϵ.Making use of the distribution consistency result given in (3.27), we can approximate the ϵth quantile of the distribution of(3.31)(α̂ϕ(θ)-θ0)byqnϵ*-α̂ϕ(θ)c.Therefore, we define thepercentile-type bootstrap confidence set as(3.32)C(ϵ):=[α̂ϕ(θ)+qn(ϵ/2)*-α̂ϕ(θ)c,α̂ϕ(θ)+qn(1-ϵ/2)*-α̂ϕ(θ)c].
In a similar manner, the ϵth quantile of n(α̂ϕ(θ)-θ0) can be approximated by q̃nϵ*, where q̃nϵ* is the ϵth quantile of the hybrid quantity (n/c)(α̂ϕ*(θ)-α̂ϕ(θ)), that is, (3.33)PW∣Xn((nc)(α̂ϕ*(θ)-α̂ϕ(θ))≤q̃nϵ*)=ϵ.Note that(3.34)q̃nϵ*=(nc)(qnϵ*-α̂ϕ(θ)).Thus, thehybrid-type bootstrap confidence set would be defined as follows:(3.35)C̃(ϵ):=[α̂ϕ(θ)-q̃n(1-ϵ/2)*n,α̂ϕ(θ)-q̃n(ϵ/2)*n].
Note that qnϵ* and q̃nϵ* are not unique by the fact that we assume θ is a vector. Recall that, for any x∈ℝd,(3.36)Pθ0(n(α̂ϕ(θ)-θ0)≤x)⟶Ψ(x),PW∣Xn((nc)(α̂ϕ*(θ)-α̂ϕ(θ))≤x)→Pθ0Ψ(x),where(3.37)Ψ(x)=P(N(0,Σ)≤x).According to the quantile convergence theorem, that is, [37, Lemma 21.1], we have, almost surely,(3.38)q̃nϵ*→PXWΨ-1(ϵ).When applying quantile convergence theorem, we use the almost sure representation, that is, [37, Theorem 2.19], and argue along subsequences. Considering Slutsky’s Theorem which ensures that(3.39)n(α̂ϕ(θ)-θ0)-q̃n(ϵ/2)*weaklyconvergestoN(0,Σ)-Ψ-1(ϵ/2),we further have(3.40)PXW(θ0≤α̂ϕ(θ)-q̃n(ϵ/2)*n)=PXW(n(α̂ϕ(θ)-θ0)≥q̃n(ϵ/2)*)⟶PXW(N(0,Σ)≥Ψ-1(ϵ2))=1-ϵ2.The above arguments prove the consistency of thehybrid-type bootstrap confidence set, that is, (3.42), and can also be applied to the percentile-type bootstrap confidence set, that is, (3.41). For an in-depth study and more rigorous proof, we may refer to [37, Lemma 23.3]. The above discussion may be summarized as follows.Corollary 3.5.
Under the conditions in Theorem3.3, one has, as n→∞,
(3.41)PXW(α̂ϕ(θ)+qn(ϵ/2)*-α̂ϕ(θ)c≤θ0≤α̂ϕ(θ)+qn(1-ϵ/2)*-α̂ϕ(θ)c)⟶1-ϵ,(3.42)PXW(α̂ϕ(θ)-q̃n(1-ϵ/2)*n≤θ0≤α̂ϕ(θ)-q̃n(ϵ/2)*n)⟶1-ϵ.It is well known that the above bootstrap confidence sets can be obtained easily through routine bootstrap sampling.Remark 3.6.
Notice that the choice of weights depends on the problem at hand: accuracy of the estimation of the entire distribution of the statistic, accuracy of a confidence interval, accuracy in large deviation sense, and accuracy for a finite sample size; we may refer to [38] and the references therein for more details. Barbe and Bertail [27] indicate that the area where the weighted bootstrap clearly performs better than the classical bootstrap is in term of coverage accuracy.


### 3.1. On the Choice of the Escort Parameter

The very peculiar choice of the escort parameter defined throughθ=θ0 has the same limit properties as the MLE one. The DϕDE α̂ϕ(θ0), in this case, has variance which indeed coincides with the MLE one; see, for instance, [28, Theorem 2.2, (1) (b)]. This result is of some relevance, since it leaves open the choice of the divergence, while keeping good asymptotic properties. For data generated from the distribution 𝒩(0,1), Figure 1 shows that the global maximum of the empirical criterion ℙnh(θ̂n,α) is zero, independently of the value of the escort parameter θ̂n (the sample mean X¯=n-1∑i=1nXi, in Figure 1(a) and the median in Figure 1(b)) for all the considered divergences which is in agreement with the result of [39, Theorem 6], where it is showed that all differentiable divergences produce the same estimator of the parameter on any regular exponential family, in particular the normal models, which is the MLE one, provided that the conditions (2.6) and Dϕ(θ,α)<∞ are satisfied.Criterion for the normal location model.
(a)(b)Unlike the case of data without contamination, the choice of the escort parameter is crucial in the estimation method in the presence of outliers. We plot in Figure2 the empirical criterion ℙnh(θ̂n,α), where the data are generated from the distribution(3.43)(1-ϵ)N(θ0,1)+ϵδ10,Criterion for the normal location model under contamination.
(a)(b)whereϵ=0.1, θ0=0, and δx stands for the Dirac measure at x. Under contamination, when we take the empirical “mean,”  θ̂n=X¯, as the value of the escort parameter θ, Figure 2(a) shows how the global maximum of the empirical criterion ℙnh(θ̂n,α) shifts from zero to the contamination point. In Figure 2(b), the choice of the “median” as escort parameter value leads to the position of the global maximum remaining close to α=0, for Hellinger (γ=0.5), χ2(γ=2),  and KL-divergence (γ=1), while the criterion associated to the KLm-divergence (γ=0,themaximumistheMLE) is still affected by the presence of outliers.In practice, the consequence is that if the data are subject to contamination the escort parameter should be chosen as a robust estimator ofθ0, say θ̂n. For more details about the performances of dual ϕ-divergence estimators for normal density models, we refer to [40].


## 3.1. On the Choice of the Escort Parameter

The very peculiar choice of the escort parameter defined throughθ=θ0 has the same limit properties as the MLE one. The DϕDE α̂ϕ(θ0), in this case, has variance which indeed coincides with the MLE one; see, for instance, [28, Theorem 2.2, (1) (b)]. This result is of some relevance, since it leaves open the choice of the divergence, while keeping good asymptotic properties. For data generated from the distribution 𝒩(0,1), Figure 1 shows that the global maximum of the empirical criterion ℙnh(θ̂n,α) is zero, independently of the value of the escort parameter θ̂n (the sample mean X¯=n-1∑i=1nXi, in Figure 1(a) and the median in Figure 1(b)) for all the considered divergences which is in agreement with the result of [39, Theorem 6], where it is showed that all differentiable divergences produce the same estimator of the parameter on any regular exponential family, in particular the normal models, which is the MLE one, provided that the conditions (2.6) and Dϕ(θ,α)<∞ are satisfied.Criterion for the normal location model.
(a)(b)Unlike the case of data without contamination, the choice of the escort parameter is crucial in the estimation method in the presence of outliers. We plot in Figure2 the empirical criterion ℙnh(θ̂n,α), where the data are generated from the distribution(3.43)(1-ϵ)N(θ0,1)+ϵδ10,Criterion for the normal location model under contamination.
(a)(b)whereϵ=0.1, θ0=0, and δx stands for the Dirac measure at x. Under contamination, when we take the empirical “mean,”  θ̂n=X¯, as the value of the escort parameter θ, Figure 2(a) shows how the global maximum of the empirical criterion ℙnh(θ̂n,α) shifts from zero to the contamination point. In Figure 2(b), the choice of the “median” as escort parameter value leads to the position of the global maximum remaining close to α=0, for Hellinger (γ=0.5), χ2(γ=2),  and KL-divergence (γ=1), while the criterion associated to the KLm-divergence (γ=0,themaximumistheMLE) is still affected by the presence of outliers.In practice, the consequence is that if the data are subject to contamination the escort parameter should be chosen as a robust estimator ofθ0, say θ̂n. For more details about the performances of dual ϕ-divergence estimators for normal density models, we refer to [40].


## 4. Examples

Keep in mind the definitions (2.8) and (2.9). In what follows, for easy reference and completeness, we give some usual examples of divergences, discussed in [41, 42], of divergences and the associated estimates; we may refer also to [43] for more examples and details.(i)
Our first example is the Kullback-Leibler divergence:(4.1)ϕ(x)=xlog⁡x-x+1,ϕ′(x)=log⁡x,xϕ′(x)-ϕ(x)=x-1.
The estimate of DKL(θ,θ0) is given by
(4.2)D̂KL(θ,θ0)=sup⁡α∈Θ{∫log⁡(dPθdPα)dPθ-∫(dPθdPα-1)dPn(dPθdPα-1)},
and the estimate of the parameter θ0, with escort parameter θ, is defined as follows:
(4.3)α̂KL(θ):=argsup⁡α∈Θ{∫log⁡(dPθdPα)dPθ-∫(dPθdPα-1)dPn}.(ii)
The second one is theχ2-divergence:
(4.4)ϕ(x)=12(x-1)2,ϕ′(x)=x-1,xϕ′(x)-ϕ(x)=12x-12.
The estimate of Dχ2(θ,θ0) is given by
(4.5)D̂χ2(θ,θ0)=sup⁡α∈Θ{∫(dPθdPα-1)dPθ-12∫((dPθdPα)2-1)dPn},
and the estimate of the parameter θ0, with escort parameter θ, is defined by
(4.6)α̂χ2(θ):=argsup⁡α∈Θ{∫(dPθdPα-1)dPθ-12∫((dPθdPα)2-1)dPn}.(iii)
Another example is the Hellinger divergence:(4.7)ϕ(x)=2(x-1)2,ϕ′(x)=2-1x,xϕ′(x)-ϕ(x)=2x-2.
The estimate of DH(θ,θ0) is given by
(4.8)D̂H(θ,θ0)=sup⁡α∈Θ{∫(2-2dPαdPθ)dPθ-∫2(dPθdPα-1)dPn},
and the estimate of the parameter θ0, with escort parameter θ, is defined by
(4.9)α̂H(θ):=argsup⁡α∈Θ{∫(2-2dPαdPθ)dPθ-∫2(dPθdPα-1)dPn}.(iv)
All the above examples are particular cases of the so-called “power divergences,” which are defined through the class of convex real-valued functions, for γ in ℝ∖{0,1},(4.10)x∈R+*⟶φγ(x):=xγ-γx+γ-1γ(γ-1).
The estimate of Dγ(θ,θ0) is given by
(4.11)D̂γ(θ,θ0)=sup⁡α∈Θ{∫1γ-1((dPθdPα)γ-1-1)dPθ-∫1γ((dPθdPα)γ-1)dPn},
and the parameter estimate is defined by(4.12)α̂γ(θ):=-argsup⁡α∈Θ{∫1γ-1((dPθdPα)γ-1-1)dPθ-∫1γ((dPθdPα)γ-1)dPn((dPθdPα)γ-1)}.Remark 4.1.
The computation of the estimateα̂ϕ(θ) requires calculus of the integral in the formula (2.9). This integral can be explicitly calculated for the most standard parametric models. Below, we give a closed-form expression for Normal, log-Normal, Exponential, Gamma, Weibull, and Pareto density models. Hence, the computation of α̂ϕ(θ) can be performed by any standard nonlinear optimization code. Unfortunately, the explicit formula of α̂ϕ(θ), generally, cannot be derived, which also is the case for the ML method. In practical problems, to obtain the estimate α̂ϕ(θ), one can use the Newton-Raphson algorithm taking as initial point the escort parameter θ. This algorithm is a powerful technique for solving equations numerically, performs well since the objective functions α∈Θ↦ℙθ0h(θ,α) are concave and the estimated parameter is unique for functions α∈Θ↦ℙnh(θ,α); for instance, refer to [1, Remark 3.5].


### 4.1. Example of Normal Density

Consider the case of power divergences and the Normal model(4.13){N(θ,σ2):(θ,σ2)∈Θ=R×R+*}.Set(4.14)pθ,σ(x)=1σ2πexp⁡{-12(x-θσ)2}.Simple calculus gives, forγ in ℝ∖{0,1},(4.15)1γ-1∫(dPθ,σ1(x)dPα,σ2(x))γ-1dPθ,σ1(x)dx=1γ-1σ1-(γ-1)σ2γγσ22-(γ-1)σ12exp⁡{γ(γ-1)(θ-α)22(γσ22-(γ-1)σ12)}.This yields(4.16)D̂γ((θ,σ1),(θ0,σ0))=sup⁡α,σ2{1γ-1σ1-(γ-1)σ2γγσ22-(γ-1)σ12exp⁡{γ(γ-1)(θ-α)22(γσ22-(γ-1)σ12)}-1γn∑i=1n(σ2σ1)γexp⁡{-γ2((Xi-θσ1)2-(Xi-ασ2)2)}-1γ(γ-1)(Xi-ασ2)2}.In the particular case,ℙθ≡𝒩(θ,1), it follows that, for γ∈ℝ∖{0,1},(4.17)D̂γ(θ,θ0):=sup⁡α∫h(θ,α)dPn=sup⁡α{{γ(γ-1)(θ-α)22}1γ-1exp⁡{γ(γ-1)(θ-α)22}-1γn∑i=1nexp⁡{-γ2(θ-α)(θ+α-2Xi)}-1γ(γ-1)}.Forγ=0,(4.18)D̂KLm(θ,θ0):=sup⁡α∫h(θ,α)dPn=sup⁡α{12n∑i=1n(θ-α)(θ+α-2Xi)},which leads to the maximum likelihood estimate independently uponθ.Forγ=1,(4.19)D̂KL(θ,θ0):=sup⁡α∫h(θ,α)dPn=sup⁡α{-12(θ-α)2-1n∑i=1nexp⁡{-12(θ-α)(θ+α-2Xi)}+1}.


### 4.2. Example of Log-Normal Density

Consider the case of power divergences and the log-Normal model(4.20){pθ,σ(x)=1xσ2πexp⁡{-12(log⁡(x)-θσ)2}:(θ,σ2)∈Θ=R×R+*,x>0}.Simple calculus gives, forγ in ℝ∖{0,1},(4.21)1γ-1∫(dPθ,σ1(x)dPα,σ2(x))γ-1dPθ,σ1(x)dx=1γ-1σ1-(γ-1)σ2γγσ22-(γ-1)σ12exp⁡{γ(γ-1)(θ-α)22(γσ22-(γ-1)σ12)}.This yields(4.22)D̂γ((θ,σ1),(θ0,σ0))=sup⁡α,σ2{1γ-1σ1-(γ-1)σ2γγσ22-(γ-1)σ12exp⁡{γ(γ-1)(θ-α)22(γσ22-(γ-1)σ12)}-1γn∑i=1n(σ2σ1)γexp⁡{-γ2((log⁡(Xi)-θσ1)2-(log⁡(Xi)-ασ2)2)((log⁡(Xi)-θσ1)2-(log⁡(Xi)-ασ2)2)}-1γ(γ-1)(log⁡(Xi)-ασ2)2}.


### 4.3. Example of Exponential Density

Consider the case of power divergences and the Exponential model(4.23){pθ(x)=θexp⁡(-θx):θ∈Θ=R+*}.We have, forγ in ℝ∖{0,1},(4.24)1γ-1∫(dPθ(x)dPα(x))γ-1dPθ(x)dx=(θα)(γ-1)(θθγ(γ-1)-α(γ-1)2).
Then, using this last equality, one finds(4.25)D̂γ(θ,θ0)=sup⁡α{(θα)(γ-1)(θθγ(γ-1)-α(γ-1)2)-1γn∑i=1n(θα)γexp⁡{-γ((θXi)-(αXi))}-1γ(γ-1)}.In more general case, we may consider the Gamma density combined with the power divergence. The Gamma model is defined by(4.26){pθ(x;k):=θkxk-1exp⁡(-xθ)Γ(k):k,θ≥0},whereΓ(·) is the Gamma function(4.27)Γ(k):=∫0∞xk-1exp⁡(-x)dx.Simple calculus gives, forγ in ℝ∖{0,1},(4.28)1γ-1∫(dPθ;k(x)dPα;k(x))γ-1dPθ;k(x)dx=(θα)k(γ-1)(θθγ-α(γ-1))k1γ-1,which implies that(4.29)D̂γ(θ,θ0)=sup⁡α{(θα)k(γ-1)(θθγ-α(γ-1))k1γ-1-1γn∑i=1n(θα)kγexp⁡{-γ((θXi)-(αXi))((θXi)-(αXi))}-1γ(γ-1)}.


### 4.4. Example of Weibull Density

Consider the case of power divergences and the Weibull density model, with the assumption thatk∈ℝ+* is known and θ is the parameter of interest to be estimated, and recall that(4.30){pθ(x)=kθ(xθ)k-1exp⁡(-(xθ)k):θ∈Θ=R+*,x≥0}.Routine algebra gives, forγ in ℝ∖{0,1},(4.31)1γ-1∫(dPθ;k(x)dPα;k(x))γ-1dPθ;k(x)dx=(αθ)k(γ-1)(1γ-(θ/α)k(γ-1))1γ-1,which implies that(4.32)D̂γ(θ,θ0)=sup⁡α{(Xiθ)k(αθ)k(γ-1)(1γ-(θ/α)k(γ-1))1γ-1-1γn∑i=1n(αθ)kγexp⁡{-γ((Xiθ)k-(Xiα)k)}-1γ(γ-1)(Xiα)k}.


### 4.5. Example of the Pareto Density

Consider the case of power divergences and the Pareto density(4.33){pθ(x):=θxθ+1:x>1;θ∈R+*}.Simple calculus gives, forγ in ℝ∖{0,1},(4.34)1γ-1∫(dPθ(x)dPα(x))γ-1dPθ(x)dx=(θα)(γ-1)(θθγ(γ-1)-α(γ-1)2).
As before, using this last equality, one finds(4.35)D̂γ(θ,θ0)=sup⁡α{(θα)(γ-1)(θθγ(γ-1)-α(γ-1)2)-1γn∑i=1n(θα)γXi{-γ(θ-α)}-1γ(γ-1)}.Forγ=0,(4.36)D̂KLm(θ,θ0):=sup⁡α∫h(θ,α)dPn=sup⁡α{-1n∑i=1n{log⁡(θα)-(θ-α)log⁡(Xi)}},which leads to the maximum likelihood estimate, given by(4.37)(1n∑i=1nlog⁡(Xi))-1,independently uponθ.Remark 4.2.
The choice of divergence, that is, the statistical criterion, depends crucially on the problem at hand. For example, theχ2-divergence among various divergences in the nonstandard problem (e.g., boundary problem estimation) is more appropriate. The idea is to include the parameter domain Θ into an enlarged space, say Θe, in order to render the boundary value an interior point of the new parameter space, Θe. Indeed, Kullback-Leibler, modified Kullback-Leibler, modified χ2, and Hellinger divergences are infinite when dℚ/dℙ takes negative values on nonnegligible (with respect to ℙ) subset of the support of ℙ, since the corresponding ϕ(·) is infinite on (-∞,0), when θ belongs to Θe∖Θ. This problem does not hold in the case of χ2-divergence, in fact, the corresponding ϕ(·) is finite on ℝ; for more details refer to [41, 42, 44], and consult also [1, 45] for related matter. It is well known that when the underlying model is misspecified or when the data are contaminated the maximum likelihood or other classical parametric methods may be severely affected and lead to very poor results. Therefore, robust methods, which automatically circumvent the contamination effects and model misspecification, can be used to provide a compromise between efficient classical parametric methods and the semiparametric approach provided they are reasonably efficient at the model; this problem has been investigated in [46, 47]. In [41, 42], simulation results show that the choice of χ2-divergence has good properties in terms of efficiency robustness. We mention that some progress has been made on automatic data-based selection of the tuning parameter α>0, appearing in formula (1) of [47]; the interested reader is referred to [48, 49]. It is mentioned in [50], where semiparametric minimum distance estimators are considered, that the MLE or inversion-type estimators involve solving a nonlinear equation which depends on some initial value. The second difficulty is that the objective function is not convex in θ, in general, which give the situation of multiple roots. Thus, in general, “good” consistent initial estimates are necessary and the DϕDE should serve that purpose.


## 4.1. Example of Normal Density

Consider the case of power divergences and the Normal model(4.13){N(θ,σ2):(θ,σ2)∈Θ=R×R+*}.Set(4.14)pθ,σ(x)=1σ2πexp⁡{-12(x-θσ)2}.Simple calculus gives, forγ in ℝ∖{0,1},(4.15)1γ-1∫(dPθ,σ1(x)dPα,σ2(x))γ-1dPθ,σ1(x)dx=1γ-1σ1-(γ-1)σ2γγσ22-(γ-1)σ12exp⁡{γ(γ-1)(θ-α)22(γσ22-(γ-1)σ12)}.This yields(4.16)D̂γ((θ,σ1),(θ0,σ0))=sup⁡α,σ2{1γ-1σ1-(γ-1)σ2γγσ22-(γ-1)σ12exp⁡{γ(γ-1)(θ-α)22(γσ22-(γ-1)σ12)}-1γn∑i=1n(σ2σ1)γexp⁡{-γ2((Xi-θσ1)2-(Xi-ασ2)2)}-1γ(γ-1)(Xi-ασ2)2}.In the particular case,ℙθ≡𝒩(θ,1), it follows that, for γ∈ℝ∖{0,1},(4.17)D̂γ(θ,θ0):=sup⁡α∫h(θ,α)dPn=sup⁡α{{γ(γ-1)(θ-α)22}1γ-1exp⁡{γ(γ-1)(θ-α)22}-1γn∑i=1nexp⁡{-γ2(θ-α)(θ+α-2Xi)}-1γ(γ-1)}.Forγ=0,(4.18)D̂KLm(θ,θ0):=sup⁡α∫h(θ,α)dPn=sup⁡α{12n∑i=1n(θ-α)(θ+α-2Xi)},which leads to the maximum likelihood estimate independently uponθ.Forγ=1,(4.19)D̂KL(θ,θ0):=sup⁡α∫h(θ,α)dPn=sup⁡α{-12(θ-α)2-1n∑i=1nexp⁡{-12(θ-α)(θ+α-2Xi)}+1}.


## 4.2. Example of Log-Normal Density

Consider the case of power divergences and the log-Normal model(4.20){pθ,σ(x)=1xσ2πexp⁡{-12(log⁡(x)-θσ)2}:(θ,σ2)∈Θ=R×R+*,x>0}.Simple calculus gives, forγ in ℝ∖{0,1},(4.21)1γ-1∫(dPθ,σ1(x)dPα,σ2(x))γ-1dPθ,σ1(x)dx=1γ-1σ1-(γ-1)σ2γγσ22-(γ-1)σ12exp⁡{γ(γ-1)(θ-α)22(γσ22-(γ-1)σ12)}.This yields(4.22)D̂γ((θ,σ1),(θ0,σ0))=sup⁡α,σ2{1γ-1σ1-(γ-1)σ2γγσ22-(γ-1)σ12exp⁡{γ(γ-1)(θ-α)22(γσ22-(γ-1)σ12)}-1γn∑i=1n(σ2σ1)γexp⁡{-γ2((log⁡(Xi)-θσ1)2-(log⁡(Xi)-ασ2)2)((log⁡(Xi)-θσ1)2-(log⁡(Xi)-ασ2)2)}-1γ(γ-1)(log⁡(Xi)-ασ2)2}.


## 4.3. Example of Exponential Density

Consider the case of power divergences and the Exponential model(4.23){pθ(x)=θexp⁡(-θx):θ∈Θ=R+*}.We have, forγ in ℝ∖{0,1},(4.24)1γ-1∫(dPθ(x)dPα(x))γ-1dPθ(x)dx=(θα)(γ-1)(θθγ(γ-1)-α(γ-1)2).
Then, using this last equality, one finds(4.25)D̂γ(θ,θ0)=sup⁡α{(θα)(γ-1)(θθγ(γ-1)-α(γ-1)2)-1γn∑i=1n(θα)γexp⁡{-γ((θXi)-(αXi))}-1γ(γ-1)}.In more general case, we may consider the Gamma density combined with the power divergence. The Gamma model is defined by(4.26){pθ(x;k):=θkxk-1exp⁡(-xθ)Γ(k):k,θ≥0},whereΓ(·) is the Gamma function(4.27)Γ(k):=∫0∞xk-1exp⁡(-x)dx.Simple calculus gives, forγ in ℝ∖{0,1},(4.28)1γ-1∫(dPθ;k(x)dPα;k(x))γ-1dPθ;k(x)dx=(θα)k(γ-1)(θθγ-α(γ-1))k1γ-1,which implies that(4.29)D̂γ(θ,θ0)=sup⁡α{(θα)k(γ-1)(θθγ-α(γ-1))k1γ-1-1γn∑i=1n(θα)kγexp⁡{-γ((θXi)-(αXi))((θXi)-(αXi))}-1γ(γ-1)}.


## 4.4. Example of Weibull Density

Consider the case of power divergences and the Weibull density model, with the assumption thatk∈ℝ+* is known and θ is the parameter of interest to be estimated, and recall that(4.30){pθ(x)=kθ(xθ)k-1exp⁡(-(xθ)k):θ∈Θ=R+*,x≥0}.Routine algebra gives, forγ in ℝ∖{0,1},(4.31)1γ-1∫(dPθ;k(x)dPα;k(x))γ-1dPθ;k(x)dx=(αθ)k(γ-1)(1γ-(θ/α)k(γ-1))1γ-1,which implies that(4.32)D̂γ(θ,θ0)=sup⁡α{(Xiθ)k(αθ)k(γ-1)(1γ-(θ/α)k(γ-1))1γ-1-1γn∑i=1n(αθ)kγexp⁡{-γ((Xiθ)k-(Xiα)k)}-1γ(γ-1)(Xiα)k}.


## 4.5. Example of the Pareto Density

Consider the case of power divergences and the Pareto density(4.33){pθ(x):=θxθ+1:x>1;θ∈R+*}.Simple calculus gives, forγ in ℝ∖{0,1},(4.34)1γ-1∫(dPθ(x)dPα(x))γ-1dPθ(x)dx=(θα)(γ-1)(θθγ(γ-1)-α(γ-1)2).
As before, using this last equality, one finds(4.35)D̂γ(θ,θ0)=sup⁡α{(θα)(γ-1)(θθγ(γ-1)-α(γ-1)2)-1γn∑i=1n(θα)γXi{-γ(θ-α)}-1γ(γ-1)}.Forγ=0,(4.36)D̂KLm(θ,θ0):=sup⁡α∫h(θ,α)dPn=sup⁡α{-1n∑i=1n{log⁡(θα)-(θ-α)log⁡(Xi)}},which leads to the maximum likelihood estimate, given by(4.37)(1n∑i=1nlog⁡(Xi))-1,independently uponθ.Remark 4.2.
The choice of divergence, that is, the statistical criterion, depends crucially on the problem at hand. For example, theχ2-divergence among various divergences in the nonstandard problem (e.g., boundary problem estimation) is more appropriate. The idea is to include the parameter domain Θ into an enlarged space, say Θe, in order to render the boundary value an interior point of the new parameter space, Θe. Indeed, Kullback-Leibler, modified Kullback-Leibler, modified χ2, and Hellinger divergences are infinite when dℚ/dℙ takes negative values on nonnegligible (with respect to ℙ) subset of the support of ℙ, since the corresponding ϕ(·) is infinite on (-∞,0), when θ belongs to Θe∖Θ. This problem does not hold in the case of χ2-divergence, in fact, the corresponding ϕ(·) is finite on ℝ; for more details refer to [41, 42, 44], and consult also [1, 45] for related matter. It is well known that when the underlying model is misspecified or when the data are contaminated the maximum likelihood or other classical parametric methods may be severely affected and lead to very poor results. Therefore, robust methods, which automatically circumvent the contamination effects and model misspecification, can be used to provide a compromise between efficient classical parametric methods and the semiparametric approach provided they are reasonably efficient at the model; this problem has been investigated in [46, 47]. In [41, 42], simulation results show that the choice of χ2-divergence has good properties in terms of efficiency robustness. We mention that some progress has been made on automatic data-based selection of the tuning parameter α>0, appearing in formula (1) of [47]; the interested reader is referred to [48, 49]. It is mentioned in [50], where semiparametric minimum distance estimators are considered, that the MLE or inversion-type estimators involve solving a nonlinear equation which depends on some initial value. The second difficulty is that the objective function is not convex in θ, in general, which give the situation of multiple roots. Thus, in general, “good” consistent initial estimates are necessary and the DϕDE should serve that purpose.


## 5. Random Right Censoring

LetT=T1,…,Tn be i.i.d survival times with continuous survival function 1-Fθ0(·)=1-ℙθ0(T≤·), and let C1,…,Cn be independent censoring times with d.f. G(·). In the censoring setup, we observe only the pair Yi=min(Ti,Ci) and δi=1{Ti≤Ci}, where 1{·} is the indicator function of the event {·}, which designs whether an observation has been censored or not. Let (Y1,δ1),…,(Yn,δn) denote the observed data points, and let(5.1)t(1)<t(2)<⋯<t(k)be thek distinct death times. Now define the death set and risk set as follows: for j=1,…,k,(5.2)D(j):={i:yi=t(j),δi=1},R(j):={i:yi≥t(j)}.Kaplan and Meier’s [51] estimator of 1-Fθ0(·), denoted here by 1-F̂n(·), may be written as follows:(5.3)1-F̂n(t):=∏j=1k(1-∑q∈D(j)1∑q∈R(j)1){T(j)≤t}.One may define a generally exchangeable weighted bootstrap scheme for the Kaplan-Meier estimator and related functionals as follows (cf. [38, page 1598]):(5.4)1-F̂n*(t):=∏j=1k(1-∑q∈D(j)Wnq∑q∈R(j)Wnq){T(j)≤t}.Letψ be Fθ0-integrable, and put(5.5)Ψn:=∫ψ(u)dP̂n*(u)=∑j=1kΥjnψ(T(j)),where(5.6)Υjn:=(∑q∈D(j)Wnq∑q∈R(j)Wnq)∏k=1j-1(∑q∈D(k)Wnq∑q∈R(k)Wnq).Note that we have used the following identity. Letai, i=1,…,k, bi, i=1,…,k, be real numbers(5.7)∏i=1kai-∏i=1kbi=∑i=1k(ai-bi)∏j=1i-1bj∏h=1+ikah.In the similar way, we define a more appropriate representation, which will be used in the sequel, as follows:(5.8)Ψn=∫ψ(u)dP̂n*(u)=∑j=1nπjnψ(Yj:n),where, for1≤j≤n,(5.9)πjn:=δj:n(∑q∈D(j)Wnq∑q∈R(j)Wnq)∏k=1j-1(∑q∈D(k)Wnq∑q∈R(k)Wnq)δk:n.Here,Y1:n≤⋯≤Yn:n are ordered Y-values and δi:n denotes the concomitant associated with Yi:n. Hence, we may write(5.10)P̂n*:=∑j=1nπjnδYi:n.
For the right censoring situation, the bootstrap DϕDEs, is defined by replacing ℙn in (2.10) by ℙ̂n*, that is,(5.11)α̂n(θ):=argsup⁡α∈Θ∫h(θ,α)dP̂n*,θ∈Θ.
The corresponding estimating equation for the unknown parameter is then given by(5.12)∫∂∂αh(θ,α)dP̂n*=0,
where we recall that(5.13)h(θ,α,x):=∫ϕ′(dPθdPα)dPθ-[dPθ(x)dPα(x)ϕ′(dPθ(x)dPα(x))-ϕ(dPθ(x)dPα(x))].Formula (5.11) defines a family of M-estimators for censored data. In the case of the power divergences family (2.4), it follows that from (4.11)(5.14)∫h(θ,α)dP̂n=1γ-1∫(dPθdPα)γ-1dPθ-1γ∫[(dPθdPα)γ-1]dP̂n-1γ-1,where(5.15)P̂n:=∑j=1nωjnδYi:n,and, for1≤j≤n,(5.16)ωjn=δj:nn-j+1∏i=1j-1[n-in-i+1]δi:n.Consider the lifetime distribution to be the one-parameter exponentialexp(θ) with density θe-θx, x≥0. Following [52], the Kaplan-Meier integral ∫h(θ,α)dℙ̂n may be written as(5.17)∑j=1nωjnh(θ,α,Yj:n).The MLE ofθ0 is given by(5.18)θ̂n,MLE=∑j=1nδj∑j=1nYj,
and the approximate MLE (AMLE) of [53] is defined by(5.19)θ̂n,AMLE=∑j=1nδj∑j=1nωjnYj:n.
We infer from (4.24), that, for γ∈ℝ∖{0,1},(5.20)∫h(θ,α)dP̂n=θγα1-γ(γ-1)[γθ+(1-γ)α]-1γ∑j=1nωjn[(θα)γexp⁡{-γ(θ-α)Yj:n}-1].Forγ=0,(5.21)∫h(θ,α)dP̂n=∑j=1nωjn[(θ-α)Yj:n-log⁡(θα)].Observe that this divergence leads to the AMLE, independently upon the value ofθ.Forγ=1,(5.22)∫h(θ,α)dP̂n=log⁡(θα)-(θ-α)θ-∑i=1nωjn[θαexp⁡(-(θ-α)Yj:n)-1].For more details about dualϕ-divergence estimators in right censoring, we refer to [54]; we leave this study open for future research. We mention that the bootstrapped estimators, in this framework, are obtained by replacing the weights ωjn by πjn in the preceding formulas.


## 6. Simulations

In this section, series of experiments were conducted in order to examine the performance of the proposed random weighted bootstrap procedure of the DϕDEs, defined in (3.2). We provide numerical illustrations regarding the mean-squared error (MSE) and the coverage probabilities. The computing program codes were implemented in R.The values ofγ are chosen to be -1,0,0.5,1,2, which corresponds, as indicated above, to the well-known standard divergences: χm2-divergence, KLm, the Hellinger distance, KL, and the χ2-divergence, respectively. The samples of sizes considered in our simulations are 25, 50, 75, 100, 150, and 200, and the estimates, DϕDEs α̂ϕ(θ), are obtained from 500 independent runs. The value of escort parameter θ is taken to be the MLE, which, under the model, is a consistent estimate of θ0, and the limit distribution of the DϕDE α̂ϕ(θ0), in this case, has variance which indeed coincides with the MLE; for more details on this subject, we refer to [28, Theorem 2.2, (1) (b)], as it is mentioned in Section 3.1. The bootstrap weights are chosen to be(6.1)(Wn1,…,Wnn)~Dirichlet(n;1,…,1).In Figure3, we plot the densities of the different estimates, it shows that the proposed estimators perform reasonably well.Densities of the estimates.
(a)(b)Tables1 and 2 provide the MSE of various estimates under the Normal model N(θ0=0,1). Here, we mention that the KL-based estimator (γ=1) is more efficient than the other competitors.Table 1
MSE of the estimates for the normal distribution,B=500.
γn=25n=50n=75n=100n=150n=200−10.06870.04190.02880.02100.01350.010700.06470.03730.02550.01920.01270.01010.50.06680.03790.02570.01940.01280.010110.04190.02170.01430.01080.00700.005720.09310.05140.03310.02380.01480.0112Table 2
MSE of the estimates for the normal distribution,B=1000.
γn=25n=50n=75n=100n=150n=200−10.07160.04320.02850.02240.01470.009900.06700.03850.02550.02020.01360.00930.50.06840.03910.02580.02030.01370.009310.04410.02300.01430.01160.00780.004920.09000.05220.03350.02460.01560.0103Tables3 and 4 provide the MSE of various estimates under the Exponential model exp(θ0=1). As expected, the MLE produces most efficient estimators. A close look at the results of the simulations show that the DϕDEs perform well under the model. For large sample size n=200, the estimator based on the Hellinger distance is equivalent to that of the MLE. Indeed in terms of empirical MSE, the DϕDE with γ=0.5 produces the same MSE as the MLE, while the performance of the other estimators is comparable.Table 3
MSE of the estimates for the exponential distribution,B=500.
γn=25n=50n=75n=100n=150n=200−10.07290.04350.03130.02150.01460.011700.07080.04050.02800.01950.01310.01040.50.07270.04150.02820.01970.01310.010510.07860.04460.02960.02070.01360.010820.11090.06640.04240.02890.01780.0132Table 4
MSE of the estimates for the exponential distribution,B=1000.
γn=25n=50n=75n=100n=150n=200−10.06700.04440.02950.02430.01460.011100.06590.04170.02690.02160.01330.01020.50.06770.04270.02720.02160.01350.010210.07350.04580.02870.02250.01400.010620.10740.06970.04290.03060.01830.0133Tables5, 6, 7, and 8, provide the empirical coverage probabilities of the corresponding 0.95 weighted bootstrap confidence intervals based on B=500,1000 weighted bootstrap estimators. Notice that the empirical coverage probabilities as in any other inferential context, the greater the sample size, the better. From the results reported in these tables, we find that for large values of the sample size n, the empirical coverage probabilities are all close to the nominal level. One can see that the DϕDE with γ=2 has the best empirical coverage probability which is near the assigned nominal level.Table 5
Empirical coverage probabilities for the normal distribution,B=500.
γn=25n=50n=75n=100n=150n=200−10.880.910.930.920.950.9200.910.920.940.940.940.930.50.940.940.940.960.940.9310.440.470.540.460.480.5120.970.970.960.970.950.95Table 6
Empirical coverage probabilities for the normal distribution,B=1000.
γn=25n=50n=75n=100n=150n=200−10.870.900.930.920.930.9600.910.940.940.930.940.960.50.930.930.950.930.940.9610.460.450.480.460.450.5020.960.970.960.950.960.96Table 7
Empirical coverage probabilities for the exponential distribution,B=500.
γn=25n=50n=75n=100n=150n=200−10.670.830.870.910.930.9200.730.870.910.930.960.930.50.760.880.910.940.960.9310.760.880.900.950.970.9320.760.890.910.960.960.94Table 8
Empirical coverage probabilities for the exponential distribution,B=1000.
γn=25n=50n=75n=100n=150n=200−10.700.790.900.910.920.9100.760.840.910.920.930.920.50.780.850.930.940.940.9310.780.870.940.940.950.9420.780.880.950.950.960.95


### 6.1. Right Censoring Case

This subsection presents some simulations for right censoring case discussed in Section5. A sample is generated from exp(1) and an exponential censoring scheme is used; the censoring distribution is taken to be exp(1/9) that the proportion of censoring is 10%. To study the robustness properties of our estimators, 20% of the observations are contaminated by exp(5). The DϕDE’s α̂ϕ(θ) are calculated for samples of sizes 25, 50, 100, and 150, and the hole procedure is repeated 500 times. We can see from Table 9 that the DϕDEs perform well under the model in terms of MSE and are an attractive alternative to the AMLE.Table 9
MSE of the estimates for the exponential distribution under right censoring.
γn=25n=50n=100n=150−10.10880.08770.07060.056300.10600.08430.06790.05380.50.10800.08600.06890.054410.11500.09140.07240.056720.15350.12760.10190.0787Table10 shows the variation in coverage of nominal 95% asymptotic confidence intervals according to the sample size. Clearly, under coverage of the confidence intervals, the DϕDEs have poor coverage probabilities due to the censoring effect. However, for small- and moderate-sized samples, the DϕDEs associated to γ=2 outperform the AMLE.Table 10
Empirical coverage probabilities for the exponential distribution under right censoring.
γn=25n=50n=100n=150−10.550.630.630.6400.590.660.640.640.50.610.660.640.6510.630.670.660.6620.640.700.680.67Under contamination the performances of our estimators decrease considerably. Such findings are evidences for the need for more adequate procedures for right-censored data (Tables11 and 12).Table 11
MSE of the estimates for the exponential distribution under right censoring, 20% of contamination.
γn=25n=50n=100n=150−10.14480.15100.15610.159100.14820.14360.14090.14050.50.14570.14020.13600.134210.14620.13890.13320.130020.15720.14420.13380.1266Table 12
Empirical coverage probabilities for the exponential distribution under right censoring, 20% of contamination.
γn=25n=50n=100n=150−10.440.490.540.5700.460.490.530.570.50.460.490.530.5710.450.490.530.5720.450.490.520.53Remark 6.1.
In order to extract methodological recommendations for the use of an appropriate divergence, it will be interesting to conduct extensive Monte Carlo experiments for several divergences or investigate theoretically the problem of the choice of the divergence which leads to an “optimal” (in some sense) estimate in terms of efficiency and robustness, which would go well beyond the scope of the present paper. Another challenging task is how to choose the bootstrap weights for a given divergence in order to obtain, for example, an efficient estimator.


## 6.1. Right Censoring Case

This subsection presents some simulations for right censoring case discussed in Section5. A sample is generated from exp(1) and an exponential censoring scheme is used; the censoring distribution is taken to be exp(1/9) that the proportion of censoring is 10%. To study the robustness properties of our estimators, 20% of the observations are contaminated by exp(5). The DϕDE’s α̂ϕ(θ) are calculated for samples of sizes 25, 50, 100, and 150, and the hole procedure is repeated 500 times. We can see from Table 9 that the DϕDEs perform well under the model in terms of MSE and are an attractive alternative to the AMLE.Table 9
MSE of the estimates for the exponential distribution under right censoring.
γn=25n=50n=100n=150−10.10880.08770.07060.056300.10600.08430.06790.05380.50.10800.08600.06890.054410.11500.09140.07240.056720.15350.12760.10190.0787Table10 shows the variation in coverage of nominal 95% asymptotic confidence intervals according to the sample size. Clearly, under coverage of the confidence intervals, the DϕDEs have poor coverage probabilities due to the censoring effect. However, for small- and moderate-sized samples, the DϕDEs associated to γ=2 outperform the AMLE.Table 10
Empirical coverage probabilities for the exponential distribution under right censoring.
γn=25n=50n=100n=150−10.550.630.630.6400.590.660.640.640.50.610.660.640.6510.630.670.660.6620.640.700.680.67Under contamination the performances of our estimators decrease considerably. Such findings are evidences for the need for more adequate procedures for right-censored data (Tables11 and 12).Table 11
MSE of the estimates for the exponential distribution under right censoring, 20% of contamination.
γn=25n=50n=100n=150−10.14480.15100.15610.159100.14820.14360.14090.14050.50.14570.14020.13600.134210.14620.13890.13320.130020.15720.14420.13380.1266Table 12
Empirical coverage probabilities for the exponential distribution under right censoring, 20% of contamination.
γn=25n=50n=100n=150−10.440.490.540.5700.460.490.530.570.50.460.490.530.5710.450.490.530.5720.450.490.520.53Remark 6.1.
In order to extract methodological recommendations for the use of an appropriate divergence, it will be interesting to conduct extensive Monte Carlo experiments for several divergences or investigate theoretically the problem of the choice of the divergence which leads to an “optimal” (in some sense) estimate in terms of efficiency and robustness, which would go well beyond the scope of the present paper. Another challenging task is how to choose the bootstrap weights for a given divergence in order to obtain, for example, an efficient estimator.



---

*Source: 834107-2012-02-09.xml*