# Joint Behaviour of Semirecursive Kernel Estimators of the Location and of the Size of the Mode  of a Probability Density Function

Authors: Abdelkader Mokkadem; Mariane Pelletier; Baba Thiam
Journal: Journal of Probability and Statistics
 (2011)

---

## Abstract

Letθ and μ denote the location and the size of the mode of a probability density. We study the joint convergence rates of semirecursive kernel estimators of θ and μ. We show how the estimation of the size of the mode allows measuring the relevance of the estimation of its location. We also enlighten that, beyond their computational advantage on nonrecursive estimators, the semirecursive estimators are preferable to use for the construction of confidence regions.


---

## Body

## 1. Introduction

LetX1,…,Xn be independent and identically distributed ℝd-valued random variables with unknown probability density f. The aim of this paper is to study the joint kernel estimation of the location θ and of the size μ=f(θ) of the mode of f. The mode is assumed to be unique, that is, f(x)<f(θ) for any x≠θ, and nondegenerated, that is, the second order differential D2f(θ) at the point θ is nonsingular (in the sequel, Dmg will denote the differential of order m of a multivariate function g).The problem of estimating the location of the mode of a probability density was widely studied. Kernel methods were considered, among many others, by Parzen [1], Nadaraya [2], Van Ryzin [3], Rüschendorf [4], Konakov [5], Samanta [6], Eddy ([7, 8]), Romano [9], Tsybakov [10], Vieu [11], Mokkadem and Pelletier [12], and Abraham et al. ([13, 14]). The problem of estimating the size of the mode was brought up by several authors (see, e.g., Romano [9] and Vieu [11]), but, at our knowledge, the behaviour of estimators of the size of the mode has not been investigated in detail, whereas there are at least two statistical motivations for estimating this parameter. First, the use of an estimator of the size is necessary for the construction of confidence regions for the location of the mode (see, e.g., Romano [9]). As a more important motivation, the estimation of the high of the peak gives information on the shape of a density in a neighbourhood of its mode and, consequently, allows measuring the pertinence of the parameter location of the mode; this motivation must be related to the remark made by Vieu [11], who pointed out that the location of the mode is more related to the shape of the derivative of f, whereas the size of the mode is more related to the shape of the density itself.Let us mention that, even though the problem of estimating the size of the mode was not investigated in the framework of density estimation, it was studied in the framework of regression estimation. Müller [15] proves in particular the joint asymptotic normality and independence of kernel estimators of the location and of the size of the mode in the framework of nonparametric regression models with fixed design. In the framework of nonparametric regression with random design, a similar result is obtained by Ziegler ([16, 17]) for kernel estimators and by Mokkadem and Pelletier [18] for estimators issued from stochastic approximation methods.This paper is focused on semirecursive kernel estimators ofθ and f(θ). To explain why we chose this option of semirecursive estimators, let us first recall that the (nonrecursive) wellknown kernel estimator of the location of the mode introduced by Parzen [1] is defined as a random variable θn* satisfying (1.1)fn*(θn*)=sup⁡y∈Rdfn*(y),
where fn* is Rosenblatt's estimator of f; more precisely, (1.2)fn*(x)=1nhnd∑i=1nK(x-Xihn),
where the bandwidth (hn) is a sequence of positive real numbers going to zero and the kernel K is a continuous function satisfying lim∥x∥→+∞K(x)=0, ∫ℝdK(x)dx=1. The asymptotic behaviour of θn* was widely studied (see, among others, [1–9, 11, 12]), but, on a computational point of view, the estimator θn* has a main drawback: its update, from a sample size n to a sample size n+1, is far from being immediate. Applying the stochastic approximation method, Tsybakov [10] introduced the recursive kernel estimator of θ defined as (1.3)Tn=Tn-1+γn1hnd+1∇K(Tn-1-Xnhn),
where T0∈ℝd is arbitrarily chosen and the stepsize (γn) is a sequence of positive real numbers going to zero. The great property of this estimator is that its update is very rapid. Unfortunately, for reasons inherent to stochastic approximation algorithms properties, very strong assumptions on the density f must be required to ensure its consistency. A recursive version fn of Rosenblatt's density estimator was introduced by Wolverton and Wagner [19] (and discussed, among others, by Yamato [20], Davies [21], Devroye [22], Menon et al. [23], Wertz [24], Wegman and Davies [25], Roussas [26], and Mokkadem et al. [27]). Let us recall that fn is defined as(1.4)fn(x)=1n∑i=1n1hidK(x-Xihi).
Its update from a sample of size n to one of size n+1 is immediate since fn clearly satisfies the recursive relation (1.5)fn(x)=(1-1n)fn-1(x)+1nhndK(x-Xnhn).
This property of rapid update of the density estimator is particularly important in the framework of mode estimation, since the number of points where f must be estimated is very large. We thus define a semirecursive version of Parzen's estimator of the location of the mode by using Wolverton-Wagner's recursive density estimator, rather than Rosenblatt's density estimator. More precisely, our estimator θn of the location θ of the mode is a random variable satisfying(1.6)fn(θn)=sup⁡y∈Rdfn(y).Let us now come back to the problem of estimating the sizef(θ) of the mode. The ordinarily used estimator is defined as μn*=fn*(θn*) (fn* being Rosenblatt's density estimator and θn* Parzen's mode estimator); the consistency of μn* is sufficient to allow the construction of confidence regions for θ (see, e.g., Romano [9]). Adapting the construction of μn* to the semirecursive framework would lead us to estimate f(θ) by(1.7)μn=fn(θn).
However, this estimator has two main drawbacks (as well as μn*). First, the use of a higher order kernel K is necessary for (μn-μ) to satisfy a central limit theorem and thus for the construction of confidence intervals of μ (and of confidence regions for (θ,μ)). Moreover, in the case when a higher order kernel is used, it is not possible to choose a bandwidth for which both estimators θn and μn converge at the optimal rate. These observations lead us to use two different bandwidths, one for the estimation of θ, the other one for the estimation of μ. More precisely, let f̃n be the recursive kernel density estimator defined as (1.8)f̃n(x)=1n∑i=1n1h̃idK(x-Xih̃i),
where the bandwidth (h̃n) may be different from (hn) used in the definition of fn (see (1.4)); we estimate the size of the mode by(1.9)μ̃n=f̃n(θn),
where θn is still defined by (1.6) and thus with the first bandwidth (hn).The purpose of this paper is the study of the joint asymptotic behaviour ofθn and μ̃n. We first prove the strong consistency of both estimators. We then establish the joint weak convergence rate of θn and μ̃n. We prove in particular that adequate choices of the bandwidths lead to the asymptotic normality and independence of these estimators and that the use of different bandwidths allow obtaining simultaneously the optimal convergence rate of both estimators. We then apply our weak convergence rate result to the construction of confidence regions for (θ,μ) and illustrate this application with a simulations study. This application enlightens the advantage of using semirecursive estimators rather than nonrecursive estimators. It also shows how the estimation of the size of the mode gives information on the relevance of estimating its location. Finally, we establish the joint strong convergence rate of θn and μ̃n.


## 2. Assumptions and Main Results

Throughout this paper,(hn) and (h̃n) are defined as hn=h(n) and h̃n=h̃(n) for all n≥1, where h and h̃ are two positive functions.


### 2.1. Strong Consistency

The conditions we require for the strong consistency ofθn and μ̃n are the following. (A1)(i)
K is an integrable, differentiable, and even function such that ∫ℝdK(z)dz=1.(ii)
There existsζ>0 such that ∫ℝd∥z∥ζ|K(z)|dz<∞.(iii)
K is Hölder continuous.(iv)
There existsγ>0 such that z↦∥z∥γ|K(z)| is a bounded function.(A2)(i)
f is uniformly continuous on ℝd.(ii)
There existsξ>0 such that ∫ℝd∥x∥ξf(x)dx<∞.(iii)
There existsη>0 such that z↦∥z∥ηf(z) is a bounded function.(iv)
There existsθ∈ℝd such that f(x)<f(θ) for all x≠θ.(A3)(i)
The functionh is locally bounded and varies regularly with exponent (-a) with a∈]0,1/d[.(ii)
The functionh̃ is locally bounded and varies regularly with exponent (-ã) with ã∈]0,1/d[.Remark 2.1.
Note that (A1)(iv) implies thatK is bounded.Remark 2.2.
Let us recall that a positive function (not necessarily monotone)ℒ defined on ]0,∞[ is slowly varying if limt→∞ℒ(tx)/ℒ(t)=1 and that a function G varies regularly with exponent ρ, ρ∈ℝ, if and only if it is of the form G(x)=xρℒ(x) with ℒ slowly varying (see, e.g., Feller [28, page 275]). Typical examples of regularly varying functions are xρ, xρlogx, xρloglogx, xρlogx/loglogx, and so on.Proposition 2.3.
Letθn and μ̃n be defined by (1.6) and (1.9), respectively. (i)
Under (A1), (A2), and (A3)(i),limn→∞θn=θ a.s.(ii)
Under (A1)–(A3),limn→∞μ̃n=μ a.s.Let us mention that the assumptions required on the probability density to establish the strong consistency of the semirecursive estimator of the location of the mode are slightly stronger than those needed for the nonrecursive estimatorθn* (see, e.g., [9, 12]), but are much weaker than those needed for the recursive estimator (see [10]). Let us also note that the strong consistency of μ̃n* can be proved in the same way as that of μ̃n.


### 2.2. Weak Convergence Rate

In order to state the weak convergence rate ofθn and μ̃n, we need the following additional assumptions on K, f, h, and h̃. (A4)(i)
K is twice differentiable on ℝd.(ii)
z↦z∇K(z) is integrable.(iii)
For any(i,j)∈{1,…,d}2, ∂2K/∂xi∂xj is bounded integrable and Hölder continuous.(iv)
K is a kernel of order q≥2, that is, for alls∈{1,…,q-1}, for allj∈{1,…,d}, ∫ℝdyjsK(y)dyj=0 and ∫ℝd|yjqK(y)|dy<∞.(A5)(i)
D2f(θ) is nonsingular.(ii)
D2f is q-times differentiable; ∇f and Dqf are bounded.(iii)
For any(i,j)∈{1,…,d}2, supx∈ℝd∥Dq(∂2f/∂xi∂xj)∥<∞, and for any k∈{1,…,d}, supx∈ℝd∥Dq(∂f/∂xk)∥<∞.(A6)(i)
a∈]0,1/(d+4)[.(ii)
ã∈]0,1/(d+2)[.Remark 2.4.
Note that (A4)(ii) and (A4)(iii) imply that∇K is Lipschitz-continuous and integrable; it is thus straightforward to see that lim∥x∥→∞∥∇K(x)∥=0 (and in particular ∇K is bounded).LetG be the d×d matrix defined by G(i,j)=∫ℝd(∂K/∂xi)(x)(∂K/∂xj)(x)dx and set (2.1)βjq=∫RdyjqK(y)dy,Ba,q(θ)={(-1)qq!(1-aq)∇(∑j=1dβjq∂qf∂xjq(θ))ifaq<1,0otherwise,Σa(θ)=f(θ)1+a(d+2)G.
The following theorem gives the weak convergence rate of the semirecursive kernel mode estimator θn.Theorem 2.5 (Weak convergence rate ofθn).
Letθn be defined by (1.6), and assume that (A1), (A2), (A3)(i), (A4), (A5), and (A6)(i) hold. (1)
If there existsc≥0 such that limn→∞nhnd+2+2q=c, then
(2.2)nhnd+2(θn-θ)⟶DN(-c[D2f(θ)]-1Ba,q(θ),[D2f(θ)]-1Σa(θ)[D2f(θ)]-1).(2)
Iflimn→∞nhnd+2+2q=∞, then
(2.3)hn-q(θn-θ)⟶P-[D2f(θ)]-1Ba,q(θ).Remark 2.6.
Iflimn→∞nhnd+2+2q=c≠0 or if limn→∞nhnd+2+2q=∞, then aq<1, and thus Ba,q(θ) is usually nonzero.In order to compare the semirecursive estimatorθn with the well-known nonrecursive Parzen's estimator θn*, let us recall that Theorem 2.5 holds when θn is replaced by θn* and a by zero (see e.g., Parzen [1] in the case d=1, and Mokkadem and Pelletier [12] in the case d≥1). The main advantage of θn on θn* is that, due to the factor [1+a(d+2)]-1 standing in the definition of Σa(θ), the asymptotic covariance of θn* is smaller than that of θn; this property will be discussed again in Section 2.3.In order to state the weak convergence rate ofμ̃n, we need the following notation (where γ>0 and where (bn) is a positive sequence): (2.4)w(bn,γ)=(log⁡⁡n)1+γnbnd+2+bn2q,Pn(γ)=[w(hn,γ)]2+w(hn,γ)w(h̃n,γ),Bã,q(μ)={(-1)qq!(1-ãq)∑j=1dβjq∂qf∂xjq(θ)ifãq<1,0otherwise,Σã(μ)=f(θ)1+ãd∫RdK2(z)dz.Theorem 2.7 (Weak convergence rate ofμ̃n).
Letμ̃n be defined by (1.9), and assume that (A1)–(A6) hold. (1)
If there existsc̃≥0 such that limn→∞nh̃nd+2q=c̃ and if there exists γ>0 such that limn→∞nh̃ndPn(γ)=0, then
(2.5)nh̃nd(μ̃n-μ)⟶DN(-c̃Bã,q(μ),Σã(μ)).(2)
Iflimn→∞nh̃nd+2q=∞ and if there exists γ>0 such that limn→∞h̃n-2qPn(γ)=0, then
(2.6)h̃n-q(μ̃n-μ)⟶PBã,q(μ).Remark 2.8.
Iflimn→∞nh̃nd+2q=c̃≠ or if limn→∞nh̃nd+2q=∞, then ãq<1, so that Bã,q(μ) is usually nonzero.Remark 2.9.
Following the proof of Theorem2.7, it can be shown that the results of Theorem 2.7 also hold when μ̃n is replaced by μ̃n* and ã by zero.Let us consider the case when(hn)=(h̃n), that is, the case when the same bandwidth is used to define θn and μ̃n (or θn* and μ̃n*). If K is a two-order kernel (i.e., if q=2), then the condition limn→∞nh̃nd+2q=c̃≥0 implies that limn→∞nh̃ndPn(γ)=∞ for all γ>0, so that the condition required in Part 1 of Theorem 2.7 is not satisfied: the limit of (μ̃n-μ) (or of (μ̃n*-μ)) suitably normalized is then necessary degenerated. This is the first main drawback of using the same bandwidth to estimate the location and the size of the mode: to construct confidence intervals for μ, the use of higher-order kernels is unavoidable.The estimation of the size of the mode is of course not independent of the estimation of the location, since the estimatorμ̃n is constructed with the help of the estimator θn. To get a good estimation of the size of the mode, it seems obvious that θn should be computed with a bandwidth (hn) leading to its optimal convergence rate (or, at least, to a convergence rate close to the optimal one). The main information given by Theorem 2.7 is that, for μ̃n to converge at the optimal rate, the use of a second bandwidth (h̃n) is then necessary.Now, set(2.7)Ba,ã,q=(Ba,q(θ)Bã,q(μ)),Σa,ã=(Σa(θ)00Σã(μ)),A=(-[D2f(θ)]-1001),
and, for any c,c̃≥0, D(c,c̃)=(cId00c̃) where Id is the d×d identity matrix. The following corollary gives a central limit theorem for the couple (θn,μ̃n).Corollary 2.10 (Joint weak convergence rate ofθn and μ̃n).
Letθn and μ̃n be defined by (1.6) and (1.9), respectively, and let the assumptions of the first parts of Theorems 2.5 and 2.7 hold. Then,
(2.8)(nhnd+2(θn-θ)nh̃nd(μ̃n-μ))⟶DN(D(c,c̃)ABa,ã,q,AΣa,ãA).Remark 2.11.
Following the proof of Corollary2.10, it can be shown that the result of Corollary 2.10 also holds when θn is replaced by θn*, μ̃n by μ̃n*, and a and ã by zero.Let us enlighten that, in view of Corollary2.10, in the case when the couple (θn,μ̃n) satisfies a central limit theorem, the estimators θn and μ̃n are asymptotically independent, although, in its definition, the estimator of the size of the mode is heavily connected to that of the location of the mode. This property is not quite surprising since, as pointed out by Vieu [11], the location of the mode gives information on the shape of the density derivative, whereas the size of the mode gives information on the shape of the density itself. This constatation must be related to the fact that the weak (and strong) convergence rate of θn is given by that of the gradient of fn, whereas the weak (and strong) convergence rate of μ̃n is given by that of f̃n itself; the variance of the density estimators converging to zero faster than that of the estimators of the density derivatives, the asymptotic independence of θn and μ̃n is completely explained.Let us finally say one word on our assumptions on the bandwidths. In the framework of nonrecursive estimation, there is no need to assume that(hn) and (h̃n) are regularly varying sequences. In the case of semirecursive estimation, this assumption cannot obviously be omitted, since the exponents a and ã stand in the expressions of the asymptotic bias Ba,ã,q and variance Σa,ã. This might be seen as a slight inconvenient of semirecursive estimation; however, as it is enlightened in the following section, it turns out to be an advantage, since the asymptotic variances of the semirecursive estimators are smaller than those of the nonrecursive estimators.


### 2.3. Construction of Confidence Regions and Simulations Studies

The application of Theorems2.5, 2.7, and Corollary 2.10 allows the construction of confidence regions of the location and of the size of the mode, as well as confidence ellipsoids of the couple (θ,μ). Hall [29] shows that, in order to construct confidence regions, avoiding bias estimation by a slight undersmoothing is more efficient than explicit bias correction. In the framework of undersmoothing, the asymptotic bias of the estimator is negligible in front of its asymptotic variance; according to the estimation by confidence regions point of view, the parameter to minimize is thus the asymptotic variance. Now, set (2.9)Σ*=([1+a(d+2)]Id00[1+ãd])Σa,ã,
and note that, in view of Corollary 2.10 and of Remark 2.11, AΣa,ãA (resp. AΣ*A) is the asymptotic covariance matrix of the semirecursive estimators (θn,μ̃n) (resp., of the nonrecursive estimators (θn*,μ̃n*)). In order to construct confidence regions for the location and/or size of the mode, it is thus much preferable to use semirecursive estimators rather than nonrecursive estimators. Simulations studies confirm this theoretical conclusion, whatever the parameter (θ, μ or (θ,μ)) for which confidence regions are constructed is. For sake of succinctness, we do not give all these simulations results here but focus on the construction of confidence ellipsoid for (θ,μ); the aim of this example is of course to enlighten the advantage of using semirecursive estimators rather than nonrecursive estimators but also to show how this confidence region gives information on the shape of the density and consequently allows measuring the pertinence of the parameter location of the mode.To construct confidence regions for(θ,μ), we consider the case d=1. The following corollary is a straightforward consequence of Corollary 2.10.Corollary 2.12.
Letθn and μ̃n be defined by (1.6) and (1.9), respectively, and let the assumptions of the first parts of Theorems 2.5 and 2.7 hold. We then have
(2.10)(1+3a)nhn3[f′′(θ)]2f(θ)∫RK'2(x)dx(θn-θ)2+(1+ã)nh̃nf(θ)∫RK2(x)dx(μ̃n-μ)2⟶Dχ2(2).
Moreover, (2.10) still holds when the parameters f(θ) and f′′(θ) are replaced by consistent estimators.Remark 2.13.
In view of Remark2.11, in the case when the nonrecursive estimators θn* and μ̃n* are used, (2.10) becomes
(2.11)nhn3[f′′(θ)]2f(θ)∫RK'2(x)dx(θn*-θ)2+nh̃nf(θ)∫RK2(x)dx(μ̃n*-μ)2⟶Dχ2(2)
(and, again, this convergence still holds when the parameters f(θ) and f′′(θ) are replaced by consistent estimators).Letf̌n′′ (resp., f̌n*″) be the recursive estimator (resp., the nonrecursive Rosenblatt's estimator) of f′′ computed with the help of a bandwidth ȟn, and set (2.12)Pn=(1+3a)nhn3[f̌n′′(θn)]2f̃n(θn)∫RK'2(x)dx,Qn=(1+ã)nh̃nf̃n(θn)∫RK2(x)dx,Pn*=nhn3[f̌n*″(θn*)]2f̃n*(θn*)∫RK'2(x)dx,Qn*=nh̃nf̃n*(θn*)∫RK2(x)dx.
Moreover, let cα be such that ℙ(Z≤cα)=1-α, where Z is χ2(2)-distributed; in view of Corollary 2.12 and Remark 2.13, the sets (2.13)Eα={(θ,μ)Pn(θn-θ)2+Qn(μ̃n-μ)2≤cα}Eα*={(θ,μ)Pn*(θn*-θ)2+Qn*(μ̃n*-μ)2≤cα}
are confidence ellipsoids for (θ,μ) with asymptotic coverage level 1-α. Let us dwell on the fact that both confidence regions have the same asymptotic level, but the lengths of the axes of the first one (constructed with the help of the semirecursive estimators θn and μ̃n) are smaller than those of the second one (constructed with the help of the nonrecursive estimators θn* and μ̃n*).We now present simulations results. In order to see the relationship between the shape of the confidence ellipsoids and that of the density, the densityf we consider is the density of the 𝒩(0,σ2)-distribution, the parameter σ taking the values 0.3, 0.4, 0.5, 0.7, 0.75, 1, 1.5, 2, and 2.5. We use the sample size n=100 and the coverage level 1-α=95% (and thus cα=5.99). In each case, the number of simulations is N=5000. The kernel we use is the standard Gaussian density; the bandwidths are (2.14)hn=n-1/7(log⁡⁡n),h̃n=n-1/5(log⁡⁡n),ȟn=n-1/9.
Table 1 gives, for each value of σ, the empirical values of θn, θn*, μn, μn* (with respect to the 5000 simulations) andTable 1σ0.30.40.50.70.7511.522.5θn−0.0020.0040.0010.0030.0020.014−0.005−0.0090.014θn*0.0030.0050.0010.005−0.0080.0160.003−0.020−0.046b1.1541.3461.8052.8983.1605.21810.09417.86617.405b*1.1661.4581.9683.3003.5825.92512.94321.94623.715μn1.3350.9890.7820.5640.5220.4010.2630.1960.155μn*1.3120.9790.7830.5620.5120.3880.2690.1930.163a0.4440.3990.3650.3220.3150.2830.2470.2240.210a*0.5140.4590.4200.3690.3630.3270.2870.2610.246p98.7%97.8%98.2%98.4%97.7%97.8%97.5%97.2%98.4%p*98.6%98.1%98.4%98.2%96.8%96.6%96.9%97.7%98.2%b: the empirical length of the θ-axis of the confidence ellipsoid ℰ5%;b*: the empirical length of the θ-axis of the confidence ellipsoid ℰ5%*;a: the empirical length of the μ-axis of the confidence ellipsoid ℰ5%;a*: the empirical length of the μ-axis of the confidence ellipsoid ℰ5%*;p: the empirical coverage level of the confidence ellipsoid ℰ5%;p*: the empirical coverage level of the confidence ellipsoid ℰ5%*.Confirming our theoretical results, we see that the empirical coverage levels of both confidence ellipsoidsℰ5%* and ℰ5% are similar but that the empirical areas of the ellipsoids ℰ5% (constructed with the help of the semirecursive estimators) are always smaller than those of the ellipsoids ℰ5%* (constructed with the help of the nonrecursive estimators).Let us now discuss the interest of the estimation of the size of the mode and that of the joint estimation of the location and size of the mode. Both estimations give information on the shape of the probability density and, consequently, allow measuring the pertinence of the parameter location of the mode. Of course, the parameterθ is significant only in the case when the high of the peak is large enough; since we consider here the example of the 𝒩(0,σ2)-distribution, this corresponds to the case when σ is small enough. Estimating only the size of the mode gives a first idea of the shape of the density around the location of the mode (for instance, when the size is estimated around 0.16, it is clear that the density is very flat). Now, the shape of the confidence ellipsoids allows getting a more precise idea. As a matter of fact, for small values of σ, the length of the μ-axis is larger than that of the θ-axis; as σ increases, the length of the μ-axis decreases, and the one of the θ-axis increases (for σ=2.5, the length of the θ-axis is larger than 20 times the one of the μ-axis). Let us underline that these variations of the lengths of the axes are not due to bad estimations results; Table 2 gives the values of the lengths b (resp., b*) of the θ-axis, a (resp., a*) of the μ-axis of the ellipsoids computed with the semirecursive estimators θn and μ̃n (resp., with the nonrecursive estimators θn* and μ̃n*) in the case when the true values of the parameters f(θ) and f′′(θ) are used (that is, by straightforwardly applying (2.10) and (2.11)).Table 2σ0.30.40.50.70.7511.522.5b0.1590.3270.5711.3571.5723.2278.89518.26031.899b*0.1900.3900.6821.6221.8793.85810.63121.82538.127μ1.3330.9980.7980.5700.5320.3990.2660.1990.159a0.4650.4030.3600.3030.2940.2550.2080.1800.161a*0.5090.4410.3950.3320.3220.2790.2280.1970.176


### 2.4. Strong Convergence Rate

To establish the strong convergence rate ofθn and μ̃n, we need the following additional assumption. (A7)(i)
h is differentiable, and its derivative varies regularly with exponent (-a-1).(ii)
h̃ is differentiable, and its derivative varies regularly with exponent (-ã-1).The following two theorems give the almost sure convergence rate ofθn and of μ̃n, respectively. Before stating them, let us enlighten that Proposition 2.3 in Mokkadem and Pelletier [12] ensures that the matrix G (and thus the matrix Σa(θ)) is nonsingular.Theorem 2.14 (Strong convergence rate ofθn).
Letθn be defined by (1.6), and assume that (A1), (A2), (A3)(i), (A4), (A5), (A6)(i), and (A7)(i) hold. (1)
If there existsc≥0 such that limn→∞nhnd+2+2q/[2loglogn]=c, then, with probability one, the sequence ((nhnd+2/2loglogn)(θn-θ)) is relatively compact and its limit set is the ellipsoid
(2.15)E={ν∈Rdsuch that(-D2f(θ)ν-cBa,q(θ))T[Σa(θ)]-1(-D2f(θ)ν-cBa,q(θ))≤1}.(2)
Iflimn→∞nhnd+2+2q/[2loglogn]=∞, then, with probability one, limn→∞hn-q(θn-θ)=-[D2f(θ)]-1Ba,q(θ).Theorem 2.15 (Strong convergence rate ofμ̃n).
Letμ̃n be defined by (1.9), and assume that (A1)–(A7) hold. (1)
If there existsc̃≥0 such that limn→∞nh̃nd+2q/[2loglogn]=c̃ and if there exists γ>0 such that limn→∞nh̃ndPn(γ)/loglogn=0, then, with probability one, the sequence
(2.16)(nh̃nd2log⁡⁡log⁡⁡n(μ̃n-μ))
is relatively compact, and its limit set is the interval [c̃Bã,q(μ)-Σã(μ);c̃Bã,q(μ)+Σã(μ)].(2)
Iflimn→∞nh̃nd+2q/loglogn=∞ and if there exists γ>0 such that limn→∞h̃n-2qPn(γ)=0, then, with probability one, limn→∞h̃n-q(μ̃n-μ)=Bã,q(μ).To establish a law of the iterated logarithm for the couple(θn,μ̃n), we need the following additional assumption.(A8)
There existsn0∈ℕ such that(2.17)n≥m≥n0⟹max⁡⁡{mhm-(d+2)nhn-(d+2);mh̃m-dnh̃n-d}=min⁡⁡{mhm-(d+2);mh̃m-d}min⁡⁡{nhn-(d+2);nh̃n-d}.Remark 2.16.
Assumption (A8) holds whena≠ã. In the case when a=ã, set ℒθ(n)=nahn and ℒμ(n)=nah̃n; (A8) is then satisfied when ℒθ(n)=(ℒμ(n))d/(d+2) for n large enough.Corollary 2.17 (Joint strong convergence rate ofθn and μ̃n).
Letθn and μ̃n be defined by (1.6) and (1.9), respectively; let the assumptions of Parts 1 of Theorems 2.14 and 2.15 hold, as well as (A8). Then, with probability one, the sequence
(2.18)12log⁡⁡log⁡⁡n(nhnd+2(θn-θ)nh̃nd(μ̃n-μ))
is relatively compact, and its limit set is the ellipsoid
(2.19)E={ν∈Rd+1such that(A-1ν-D(c,c̃)Ba,ã,q)TΣa,ã-1(A-1ν-D(c,c̃)Ba,ã,q)≤1}.Laws of the iterated logarithm for Parzen's nonrecursive kernel mode estimator in the multivariate framework were established by Mokkadem and Pelletier [12]. The technics of demonstration used in the framework of nonrecursive estimators are totally different from those employed to prove Theorem 2.14. This is due to the following fundamental difference between the nonrecursive estimator θn* and the semirecursive estimator θn: the study of the asymptotic behaviour of θn* comes down to the one of a triangular sum of independent variables, whereas the study of the asymptotic behaviour of θn reduces to the one of a sum of independent variables. Of course, this difference is not quite important for the study of the weak convergence rate. But, for the study of the strong convergence rate, it makes the case of the semirecursive estimation much easier than the case of the nonrecursive estimation. In particular, on the opposite to the weak convergence rate, the joint strong convergence rate of the nonrecursive estimators θn* and μ̃n* cannot be obtained by following the lines of the proof of Theorem 2.14 and remains an open question.


## 2.1. Strong Consistency

The conditions we require for the strong consistency ofθn and μ̃n are the following. (A1)(i)
K is an integrable, differentiable, and even function such that ∫ℝdK(z)dz=1.(ii)
There existsζ>0 such that ∫ℝd∥z∥ζ|K(z)|dz<∞.(iii)
K is Hölder continuous.(iv)
There existsγ>0 such that z↦∥z∥γ|K(z)| is a bounded function.(A2)(i)
f is uniformly continuous on ℝd.(ii)
There existsξ>0 such that ∫ℝd∥x∥ξf(x)dx<∞.(iii)
There existsη>0 such that z↦∥z∥ηf(z) is a bounded function.(iv)
There existsθ∈ℝd such that f(x)<f(θ) for all x≠θ.(A3)(i)
The functionh is locally bounded and varies regularly with exponent (-a) with a∈]0,1/d[.(ii)
The functionh̃ is locally bounded and varies regularly with exponent (-ã) with ã∈]0,1/d[.Remark 2.1.
Note that (A1)(iv) implies thatK is bounded.Remark 2.2.
Let us recall that a positive function (not necessarily monotone)ℒ defined on ]0,∞[ is slowly varying if limt→∞ℒ(tx)/ℒ(t)=1 and that a function G varies regularly with exponent ρ, ρ∈ℝ, if and only if it is of the form G(x)=xρℒ(x) with ℒ slowly varying (see, e.g., Feller [28, page 275]). Typical examples of regularly varying functions are xρ, xρlogx, xρloglogx, xρlogx/loglogx, and so on.Proposition 2.3.
Letθn and μ̃n be defined by (1.6) and (1.9), respectively. (i)
Under (A1), (A2), and (A3)(i),limn→∞θn=θ a.s.(ii)
Under (A1)–(A3),limn→∞μ̃n=μ a.s.Let us mention that the assumptions required on the probability density to establish the strong consistency of the semirecursive estimator of the location of the mode are slightly stronger than those needed for the nonrecursive estimatorθn* (see, e.g., [9, 12]), but are much weaker than those needed for the recursive estimator (see [10]). Let us also note that the strong consistency of μ̃n* can be proved in the same way as that of μ̃n.


## 2.2. Weak Convergence Rate

In order to state the weak convergence rate ofθn and μ̃n, we need the following additional assumptions on K, f, h, and h̃. (A4)(i)
K is twice differentiable on ℝd.(ii)
z↦z∇K(z) is integrable.(iii)
For any(i,j)∈{1,…,d}2, ∂2K/∂xi∂xj is bounded integrable and Hölder continuous.(iv)
K is a kernel of order q≥2, that is, for alls∈{1,…,q-1}, for allj∈{1,…,d}, ∫ℝdyjsK(y)dyj=0 and ∫ℝd|yjqK(y)|dy<∞.(A5)(i)
D2f(θ) is nonsingular.(ii)
D2f is q-times differentiable; ∇f and Dqf are bounded.(iii)
For any(i,j)∈{1,…,d}2, supx∈ℝd∥Dq(∂2f/∂xi∂xj)∥<∞, and for any k∈{1,…,d}, supx∈ℝd∥Dq(∂f/∂xk)∥<∞.(A6)(i)
a∈]0,1/(d+4)[.(ii)
ã∈]0,1/(d+2)[.Remark 2.4.
Note that (A4)(ii) and (A4)(iii) imply that∇K is Lipschitz-continuous and integrable; it is thus straightforward to see that lim∥x∥→∞∥∇K(x)∥=0 (and in particular ∇K is bounded).LetG be the d×d matrix defined by G(i,j)=∫ℝd(∂K/∂xi)(x)(∂K/∂xj)(x)dx and set (2.1)βjq=∫RdyjqK(y)dy,Ba,q(θ)={(-1)qq!(1-aq)∇(∑j=1dβjq∂qf∂xjq(θ))ifaq<1,0otherwise,Σa(θ)=f(θ)1+a(d+2)G.
The following theorem gives the weak convergence rate of the semirecursive kernel mode estimator θn.Theorem 2.5 (Weak convergence rate ofθn).
Letθn be defined by (1.6), and assume that (A1), (A2), (A3)(i), (A4), (A5), and (A6)(i) hold. (1)
If there existsc≥0 such that limn→∞nhnd+2+2q=c, then
(2.2)nhnd+2(θn-θ)⟶DN(-c[D2f(θ)]-1Ba,q(θ),[D2f(θ)]-1Σa(θ)[D2f(θ)]-1).(2)
Iflimn→∞nhnd+2+2q=∞, then
(2.3)hn-q(θn-θ)⟶P-[D2f(θ)]-1Ba,q(θ).Remark 2.6.
Iflimn→∞nhnd+2+2q=c≠0 or if limn→∞nhnd+2+2q=∞, then aq<1, and thus Ba,q(θ) is usually nonzero.In order to compare the semirecursive estimatorθn with the well-known nonrecursive Parzen's estimator θn*, let us recall that Theorem 2.5 holds when θn is replaced by θn* and a by zero (see e.g., Parzen [1] in the case d=1, and Mokkadem and Pelletier [12] in the case d≥1). The main advantage of θn on θn* is that, due to the factor [1+a(d+2)]-1 standing in the definition of Σa(θ), the asymptotic covariance of θn* is smaller than that of θn; this property will be discussed again in Section 2.3.In order to state the weak convergence rate ofμ̃n, we need the following notation (where γ>0 and where (bn) is a positive sequence): (2.4)w(bn,γ)=(log⁡⁡n)1+γnbnd+2+bn2q,Pn(γ)=[w(hn,γ)]2+w(hn,γ)w(h̃n,γ),Bã,q(μ)={(-1)qq!(1-ãq)∑j=1dβjq∂qf∂xjq(θ)ifãq<1,0otherwise,Σã(μ)=f(θ)1+ãd∫RdK2(z)dz.Theorem 2.7 (Weak convergence rate ofμ̃n).
Letμ̃n be defined by (1.9), and assume that (A1)–(A6) hold. (1)
If there existsc̃≥0 such that limn→∞nh̃nd+2q=c̃ and if there exists γ>0 such that limn→∞nh̃ndPn(γ)=0, then
(2.5)nh̃nd(μ̃n-μ)⟶DN(-c̃Bã,q(μ),Σã(μ)).(2)
Iflimn→∞nh̃nd+2q=∞ and if there exists γ>0 such that limn→∞h̃n-2qPn(γ)=0, then
(2.6)h̃n-q(μ̃n-μ)⟶PBã,q(μ).Remark 2.8.
Iflimn→∞nh̃nd+2q=c̃≠ or if limn→∞nh̃nd+2q=∞, then ãq<1, so that Bã,q(μ) is usually nonzero.Remark 2.9.
Following the proof of Theorem2.7, it can be shown that the results of Theorem 2.7 also hold when μ̃n is replaced by μ̃n* and ã by zero.Let us consider the case when(hn)=(h̃n), that is, the case when the same bandwidth is used to define θn and μ̃n (or θn* and μ̃n*). If K is a two-order kernel (i.e., if q=2), then the condition limn→∞nh̃nd+2q=c̃≥0 implies that limn→∞nh̃ndPn(γ)=∞ for all γ>0, so that the condition required in Part 1 of Theorem 2.7 is not satisfied: the limit of (μ̃n-μ) (or of (μ̃n*-μ)) suitably normalized is then necessary degenerated. This is the first main drawback of using the same bandwidth to estimate the location and the size of the mode: to construct confidence intervals for μ, the use of higher-order kernels is unavoidable.The estimation of the size of the mode is of course not independent of the estimation of the location, since the estimatorμ̃n is constructed with the help of the estimator θn. To get a good estimation of the size of the mode, it seems obvious that θn should be computed with a bandwidth (hn) leading to its optimal convergence rate (or, at least, to a convergence rate close to the optimal one). The main information given by Theorem 2.7 is that, for μ̃n to converge at the optimal rate, the use of a second bandwidth (h̃n) is then necessary.Now, set(2.7)Ba,ã,q=(Ba,q(θ)Bã,q(μ)),Σa,ã=(Σa(θ)00Σã(μ)),A=(-[D2f(θ)]-1001),
and, for any c,c̃≥0, D(c,c̃)=(cId00c̃) where Id is the d×d identity matrix. The following corollary gives a central limit theorem for the couple (θn,μ̃n).Corollary 2.10 (Joint weak convergence rate ofθn and μ̃n).
Letθn and μ̃n be defined by (1.6) and (1.9), respectively, and let the assumptions of the first parts of Theorems 2.5 and 2.7 hold. Then,
(2.8)(nhnd+2(θn-θ)nh̃nd(μ̃n-μ))⟶DN(D(c,c̃)ABa,ã,q,AΣa,ãA).Remark 2.11.
Following the proof of Corollary2.10, it can be shown that the result of Corollary 2.10 also holds when θn is replaced by θn*, μ̃n by μ̃n*, and a and ã by zero.Let us enlighten that, in view of Corollary2.10, in the case when the couple (θn,μ̃n) satisfies a central limit theorem, the estimators θn and μ̃n are asymptotically independent, although, in its definition, the estimator of the size of the mode is heavily connected to that of the location of the mode. This property is not quite surprising since, as pointed out by Vieu [11], the location of the mode gives information on the shape of the density derivative, whereas the size of the mode gives information on the shape of the density itself. This constatation must be related to the fact that the weak (and strong) convergence rate of θn is given by that of the gradient of fn, whereas the weak (and strong) convergence rate of μ̃n is given by that of f̃n itself; the variance of the density estimators converging to zero faster than that of the estimators of the density derivatives, the asymptotic independence of θn and μ̃n is completely explained.Let us finally say one word on our assumptions on the bandwidths. In the framework of nonrecursive estimation, there is no need to assume that(hn) and (h̃n) are regularly varying sequences. In the case of semirecursive estimation, this assumption cannot obviously be omitted, since the exponents a and ã stand in the expressions of the asymptotic bias Ba,ã,q and variance Σa,ã. This might be seen as a slight inconvenient of semirecursive estimation; however, as it is enlightened in the following section, it turns out to be an advantage, since the asymptotic variances of the semirecursive estimators are smaller than those of the nonrecursive estimators.


## 2.3. Construction of Confidence Regions and Simulations Studies

The application of Theorems2.5, 2.7, and Corollary 2.10 allows the construction of confidence regions of the location and of the size of the mode, as well as confidence ellipsoids of the couple (θ,μ). Hall [29] shows that, in order to construct confidence regions, avoiding bias estimation by a slight undersmoothing is more efficient than explicit bias correction. In the framework of undersmoothing, the asymptotic bias of the estimator is negligible in front of its asymptotic variance; according to the estimation by confidence regions point of view, the parameter to minimize is thus the asymptotic variance. Now, set (2.9)Σ*=([1+a(d+2)]Id00[1+ãd])Σa,ã,
and note that, in view of Corollary 2.10 and of Remark 2.11, AΣa,ãA (resp. AΣ*A) is the asymptotic covariance matrix of the semirecursive estimators (θn,μ̃n) (resp., of the nonrecursive estimators (θn*,μ̃n*)). In order to construct confidence regions for the location and/or size of the mode, it is thus much preferable to use semirecursive estimators rather than nonrecursive estimators. Simulations studies confirm this theoretical conclusion, whatever the parameter (θ, μ or (θ,μ)) for which confidence regions are constructed is. For sake of succinctness, we do not give all these simulations results here but focus on the construction of confidence ellipsoid for (θ,μ); the aim of this example is of course to enlighten the advantage of using semirecursive estimators rather than nonrecursive estimators but also to show how this confidence region gives information on the shape of the density and consequently allows measuring the pertinence of the parameter location of the mode.To construct confidence regions for(θ,μ), we consider the case d=1. The following corollary is a straightforward consequence of Corollary 2.10.Corollary 2.12.
Letθn and μ̃n be defined by (1.6) and (1.9), respectively, and let the assumptions of the first parts of Theorems 2.5 and 2.7 hold. We then have
(2.10)(1+3a)nhn3[f′′(θ)]2f(θ)∫RK'2(x)dx(θn-θ)2+(1+ã)nh̃nf(θ)∫RK2(x)dx(μ̃n-μ)2⟶Dχ2(2).
Moreover, (2.10) still holds when the parameters f(θ) and f′′(θ) are replaced by consistent estimators.Remark 2.13.
In view of Remark2.11, in the case when the nonrecursive estimators θn* and μ̃n* are used, (2.10) becomes
(2.11)nhn3[f′′(θ)]2f(θ)∫RK'2(x)dx(θn*-θ)2+nh̃nf(θ)∫RK2(x)dx(μ̃n*-μ)2⟶Dχ2(2)
(and, again, this convergence still holds when the parameters f(θ) and f′′(θ) are replaced by consistent estimators).Letf̌n′′ (resp., f̌n*″) be the recursive estimator (resp., the nonrecursive Rosenblatt's estimator) of f′′ computed with the help of a bandwidth ȟn, and set (2.12)Pn=(1+3a)nhn3[f̌n′′(θn)]2f̃n(θn)∫RK'2(x)dx,Qn=(1+ã)nh̃nf̃n(θn)∫RK2(x)dx,Pn*=nhn3[f̌n*″(θn*)]2f̃n*(θn*)∫RK'2(x)dx,Qn*=nh̃nf̃n*(θn*)∫RK2(x)dx.
Moreover, let cα be such that ℙ(Z≤cα)=1-α, where Z is χ2(2)-distributed; in view of Corollary 2.12 and Remark 2.13, the sets (2.13)Eα={(θ,μ)Pn(θn-θ)2+Qn(μ̃n-μ)2≤cα}Eα*={(θ,μ)Pn*(θn*-θ)2+Qn*(μ̃n*-μ)2≤cα}
are confidence ellipsoids for (θ,μ) with asymptotic coverage level 1-α. Let us dwell on the fact that both confidence regions have the same asymptotic level, but the lengths of the axes of the first one (constructed with the help of the semirecursive estimators θn and μ̃n) are smaller than those of the second one (constructed with the help of the nonrecursive estimators θn* and μ̃n*).We now present simulations results. In order to see the relationship between the shape of the confidence ellipsoids and that of the density, the densityf we consider is the density of the 𝒩(0,σ2)-distribution, the parameter σ taking the values 0.3, 0.4, 0.5, 0.7, 0.75, 1, 1.5, 2, and 2.5. We use the sample size n=100 and the coverage level 1-α=95% (and thus cα=5.99). In each case, the number of simulations is N=5000. The kernel we use is the standard Gaussian density; the bandwidths are (2.14)hn=n-1/7(log⁡⁡n),h̃n=n-1/5(log⁡⁡n),ȟn=n-1/9.
Table 1 gives, for each value of σ, the empirical values of θn, θn*, μn, μn* (with respect to the 5000 simulations) andTable 1σ0.30.40.50.70.7511.522.5θn−0.0020.0040.0010.0030.0020.014−0.005−0.0090.014θn*0.0030.0050.0010.005−0.0080.0160.003−0.020−0.046b1.1541.3461.8052.8983.1605.21810.09417.86617.405b*1.1661.4581.9683.3003.5825.92512.94321.94623.715μn1.3350.9890.7820.5640.5220.4010.2630.1960.155μn*1.3120.9790.7830.5620.5120.3880.2690.1930.163a0.4440.3990.3650.3220.3150.2830.2470.2240.210a*0.5140.4590.4200.3690.3630.3270.2870.2610.246p98.7%97.8%98.2%98.4%97.7%97.8%97.5%97.2%98.4%p*98.6%98.1%98.4%98.2%96.8%96.6%96.9%97.7%98.2%b: the empirical length of the θ-axis of the confidence ellipsoid ℰ5%;b*: the empirical length of the θ-axis of the confidence ellipsoid ℰ5%*;a: the empirical length of the μ-axis of the confidence ellipsoid ℰ5%;a*: the empirical length of the μ-axis of the confidence ellipsoid ℰ5%*;p: the empirical coverage level of the confidence ellipsoid ℰ5%;p*: the empirical coverage level of the confidence ellipsoid ℰ5%*.Confirming our theoretical results, we see that the empirical coverage levels of both confidence ellipsoidsℰ5%* and ℰ5% are similar but that the empirical areas of the ellipsoids ℰ5% (constructed with the help of the semirecursive estimators) are always smaller than those of the ellipsoids ℰ5%* (constructed with the help of the nonrecursive estimators).Let us now discuss the interest of the estimation of the size of the mode and that of the joint estimation of the location and size of the mode. Both estimations give information on the shape of the probability density and, consequently, allow measuring the pertinence of the parameter location of the mode. Of course, the parameterθ is significant only in the case when the high of the peak is large enough; since we consider here the example of the 𝒩(0,σ2)-distribution, this corresponds to the case when σ is small enough. Estimating only the size of the mode gives a first idea of the shape of the density around the location of the mode (for instance, when the size is estimated around 0.16, it is clear that the density is very flat). Now, the shape of the confidence ellipsoids allows getting a more precise idea. As a matter of fact, for small values of σ, the length of the μ-axis is larger than that of the θ-axis; as σ increases, the length of the μ-axis decreases, and the one of the θ-axis increases (for σ=2.5, the length of the θ-axis is larger than 20 times the one of the μ-axis). Let us underline that these variations of the lengths of the axes are not due to bad estimations results; Table 2 gives the values of the lengths b (resp., b*) of the θ-axis, a (resp., a*) of the μ-axis of the ellipsoids computed with the semirecursive estimators θn and μ̃n (resp., with the nonrecursive estimators θn* and μ̃n*) in the case when the true values of the parameters f(θ) and f′′(θ) are used (that is, by straightforwardly applying (2.10) and (2.11)).Table 2σ0.30.40.50.70.7511.522.5b0.1590.3270.5711.3571.5723.2278.89518.26031.899b*0.1900.3900.6821.6221.8793.85810.63121.82538.127μ1.3330.9980.7980.5700.5320.3990.2660.1990.159a0.4650.4030.3600.3030.2940.2550.2080.1800.161a*0.5090.4410.3950.3320.3220.2790.2280.1970.176


## 2.4. Strong Convergence Rate

To establish the strong convergence rate ofθn and μ̃n, we need the following additional assumption. (A7)(i)
h is differentiable, and its derivative varies regularly with exponent (-a-1).(ii)
h̃ is differentiable, and its derivative varies regularly with exponent (-ã-1).The following two theorems give the almost sure convergence rate ofθn and of μ̃n, respectively. Before stating them, let us enlighten that Proposition 2.3 in Mokkadem and Pelletier [12] ensures that the matrix G (and thus the matrix Σa(θ)) is nonsingular.Theorem 2.14 (Strong convergence rate ofθn).
Letθn be defined by (1.6), and assume that (A1), (A2), (A3)(i), (A4), (A5), (A6)(i), and (A7)(i) hold. (1)
If there existsc≥0 such that limn→∞nhnd+2+2q/[2loglogn]=c, then, with probability one, the sequence ((nhnd+2/2loglogn)(θn-θ)) is relatively compact and its limit set is the ellipsoid
(2.15)E={ν∈Rdsuch that(-D2f(θ)ν-cBa,q(θ))T[Σa(θ)]-1(-D2f(θ)ν-cBa,q(θ))≤1}.(2)
Iflimn→∞nhnd+2+2q/[2loglogn]=∞, then, with probability one, limn→∞hn-q(θn-θ)=-[D2f(θ)]-1Ba,q(θ).Theorem 2.15 (Strong convergence rate ofμ̃n).
Letμ̃n be defined by (1.9), and assume that (A1)–(A7) hold. (1)
If there existsc̃≥0 such that limn→∞nh̃nd+2q/[2loglogn]=c̃ and if there exists γ>0 such that limn→∞nh̃ndPn(γ)/loglogn=0, then, with probability one, the sequence
(2.16)(nh̃nd2log⁡⁡log⁡⁡n(μ̃n-μ))
is relatively compact, and its limit set is the interval [c̃Bã,q(μ)-Σã(μ);c̃Bã,q(μ)+Σã(μ)].(2)
Iflimn→∞nh̃nd+2q/loglogn=∞ and if there exists γ>0 such that limn→∞h̃n-2qPn(γ)=0, then, with probability one, limn→∞h̃n-q(μ̃n-μ)=Bã,q(μ).To establish a law of the iterated logarithm for the couple(θn,μ̃n), we need the following additional assumption.(A8)
There existsn0∈ℕ such that(2.17)n≥m≥n0⟹max⁡⁡{mhm-(d+2)nhn-(d+2);mh̃m-dnh̃n-d}=min⁡⁡{mhm-(d+2);mh̃m-d}min⁡⁡{nhn-(d+2);nh̃n-d}.Remark 2.16.
Assumption (A8) holds whena≠ã. In the case when a=ã, set ℒθ(n)=nahn and ℒμ(n)=nah̃n; (A8) is then satisfied when ℒθ(n)=(ℒμ(n))d/(d+2) for n large enough.Corollary 2.17 (Joint strong convergence rate ofθn and μ̃n).
Letθn and μ̃n be defined by (1.6) and (1.9), respectively; let the assumptions of Parts 1 of Theorems 2.14 and 2.15 hold, as well as (A8). Then, with probability one, the sequence
(2.18)12log⁡⁡log⁡⁡n(nhnd+2(θn-θ)nh̃nd(μ̃n-μ))
is relatively compact, and its limit set is the ellipsoid
(2.19)E={ν∈Rd+1such that(A-1ν-D(c,c̃)Ba,ã,q)TΣa,ã-1(A-1ν-D(c,c̃)Ba,ã,q)≤1}.Laws of the iterated logarithm for Parzen's nonrecursive kernel mode estimator in the multivariate framework were established by Mokkadem and Pelletier [12]. The technics of demonstration used in the framework of nonrecursive estimators are totally different from those employed to prove Theorem 2.14. This is due to the following fundamental difference between the nonrecursive estimator θn* and the semirecursive estimator θn: the study of the asymptotic behaviour of θn* comes down to the one of a triangular sum of independent variables, whereas the study of the asymptotic behaviour of θn reduces to the one of a sum of independent variables. Of course, this difference is not quite important for the study of the weak convergence rate. But, for the study of the strong convergence rate, it makes the case of the semirecursive estimation much easier than the case of the nonrecursive estimation. In particular, on the opposite to the weak convergence rate, the joint strong convergence rate of the nonrecursive estimators θn* and μ̃n* cannot be obtained by following the lines of the proof of Theorem 2.14 and remains an open question.


## 3. Proofs

Let us first note that an important consequence of (A3)(i) is that(3.1)ifβa<1,thenlim⁡n→∞1nhnβ∑i=1nhiβ=11-aβ.
Moreover, for all ɛ>0,(3.2)1n∑i=1nhiq=O(hnq+nɛ-1).
As a matter of fact: (i) if aq<1, (3.2) follows easily from (3.1); (ii) if aq>1, since ∑ihiq is summable, (3.2) holds; (iii) if aq=1, then hnq=O(nɛ-1), so that ∑i=1nhiq=O(nɛ), and thus (3.2) follows.Of course, in view of (A3)(ii), (3.1) and (3.2) also hold when (hn) and a are replaced by (h̃n) and ã, respectively.Our proofs are now organized as follows. Section3.1 is devoted to the proof of Proposition 2.3. In Section 3.2, we state some preliminary lemmas, which are crucial in the proof of the convergence rates of θn and μ̃n and which are proved in Section 3.6. Section 3.3 is reserved to the proof of Theorems 2.5 and 2.14, Section 3.4 to that of Theorems 2.7 and 2.15, and Section 3.5 to that of Corollaries 2.10 and 2.17.


### 3.1. Consistency ofθn and μ̃n: Proof of Proposition 2.3

Sinceθn is the mode of fn and θ the mode of f, we have(3.3)0≤f(θ)-f(θn)=[f(θ)-fn(θn)]+[fn(θn)-f(θn)]≤[f(θ)-fn(θ)]+[fn(θn)-f(θn)]≤|f(θ)-fn(θ)|+|fn(θn)-f(θn)|≤2‖fn-f‖∞.
The application of Theorem 5 in Mokkadem et al. [27] with |α|=0 and vn=lognensures that, for any δ>0, there exists c(δ)>0 such that ℙ[(logn)∥fn-𝔼(fn)∥∞≥δ]≤exp(-c(δ)∑i=1nhid/(logn)2). In view of (3.1), since ad<1, we can write (3.4)n2exp⁡⁡(-c(δ)∑i=1nhid(log⁡⁡n)2)=n2exp⁡⁡(-c(δ)nhnd(log⁡⁡n)2∑i=1nhidnhnd)=o(1).
Borell-Cantelli's Lemma then ensures that limn→∞∥fn-𝔼(fn)∥∞=0 a.s. Since limn→∞∥𝔼(fn)-f∥∞=0, it follows from (3.3) that limn→∞f(θn)=f(θ) a.s. Since f is continuous, since lim∥z∥→∞f(z)=0, and since θ is the unique mode of f, we deduce that limn→∞θn=θ a.s. Now, we have (3.5)|μ̃n-μ|≤|f̃n(θn)-f(θn)|+|f(θn)-f(θ)|≤‖f̃n-f‖∞+2‖fn-f‖∞,
where the last inequality follows from (3.3). Following the proof of the strong uniform convergence of fn, we show that limn→∞∥f̃n-f∥∞=0 a.s. It follows that limn→∞μ̃n=μ a.s., which concludes the proof of Proposition 2.3.


### 3.2. Some Preliminary Lemmas

The aim of this section is to state some properties of the density estimators and of their derivatives, which are crucial in the proof of Theorems2.5–2.15 and of Corollaries 2.10 and 2.17.


#### 3.2.1. Strong Uniform Convergence Rate of the Derivatives of the Density Estimators

For anyd-uplet [α]=(α1,…,αd)∈ℕd, set |α|=α1+⋯+αd and, for any function g, let ∂[α]g(x)=∂|α|g/(∂x1α1…∂xdαd)(x) denote the [α]-th partial derivative of g (if |α|=0, then ∂[α]g=g). In order to prove Theorems 2.5–2.15 and Corollaries 2.10 and 2.17, we need to know the behaviour of ∂[α]fn for |α|∈{0,1,2} and that of ∂[α]f̃n for |α|∈{0,1}. For the sake of conciseness, we state the preliminary lemmas 3.1 and 3.2 for ∂[α]gn where either gn=fn and |α|∈{0,1,2} or gn=f̃n and |α|∈{0,1}. Moreover, we set (bn)=(hn) if gn=fn, and (bn)=(h̃n) if gn=f̃n.Lemma 3.1.
Let (A1), (A2), (A3)(i), (A4), (A5), and (A6)(i) hold. Moreover, ifgn=f̃n, then let (A3)(ii) and (A6)(ii) hold. We have
(3.6)lim⁡n→∞n∑i=1nbiq[E[∂[α]gn(x)]-∂[α]f(x)]=(-1)qq!∂[α](∑j=1dβjq∂qf∂xjq)(x),
where βjq is defined in (2.1). Moreover, if we set Mq=supx∈ℝd∥Dq∂[α]f(x)∥, then
(3.7)lim⁡n→∞n∑i=1nbiqsup⁡x∈Rd|E(∂[α]gn(x))-∂[α]f(x)|≤Mqq!∫Rd‖z‖q|K(z)|dz.Lemma 3.2.
LetU be a compact set of  ℝd, and assume that (A1), (A2), (A3)(i), (A4), (A5), and (A6)(i) hold. Moreover, if gn=f̃n, then let (A3)(ii) and (A6)(ii) hold. Then, for all γ>0, we have
(3.8)sup⁡x∈U|∂[α]gn(x)-E(∂[α]gn(x))|=O((log⁡⁡n)1+γ∑i=1nbid+2|α|)a.s.Lemma3.1 is proved in Mokkadem et al. [27], Lemma 3.2 in Section 3.6.


#### 3.2.2. Convergence Rate of(∇fn(θ)-𝔼(∇fn(θ)),f̃n(θ)-𝔼(f̃n(θ)))

Lemma 3.3.
(1)
Under Assumptions (A1), (A2), (A3)(i), (A4), (A5), and (A6)(i), we have(3.9)nhnd+2[∇fn(θ)-E(∇fn(θ))]⟶DN(0,Σa(θ)).(2)
Under Assumptions (A1)–(A6), we have(3.10)Wn=(nhnd+2[∇fn(θ)-E(∇fn(θ))]nh̃nd[f̃n(θ)-E(f̃n(θ))])⟶DN(0,Σa,ã).Lemma 3.4.
(1)
Under Assumptions (A1), (A2), (A3)(i), (A4), (A5), (A6)(i), and (A7)(i), with probability one, the sequence((nhnd+2[∇fn(θ)-𝔼(∇fn(θ))])/2loglogn) is relatively compact, and its limit set is {ν∈ℝdsuch thatνT[Σa(θ)]-1ν≤1}.(2)
Under Assumptions (A1)–(A7), with probability one, the sequence((nh̃nd[f̃n(θ)-𝔼(f̃n(θ))])/2loglogn) is relatively compact, and its limit set is the interval [-Σã(μ);Σã(μ)].(3)
Under Assumptions (A1)–(A8), with probability one, the sequence(3.11)12log⁡⁡log⁡⁡n(nhnd+2[∇fn(θ)-E(∇fn(θ))]nh̃nd[f̃n(θ)-E(f̃n(θ))])
is relatively compact, and its limit set is {ν∈ℝd+1suchthatνTΣa,ã-1ν≤1}.


### 3.3. Convergence Rate ofθn: Proof of Theorems 2.5 and 2.14

In order to prove Theorems2.5 and 2.14, we first show that the weak and strong asymptotic behaviours of θn-θ are given by those of -[D2f(θ)]-1∇fn(θ) (see Section 3.3.1) and then deduce the convergence rates of θn-θ from those of ∇fn(θ) (see Section 3.3.2).


#### 3.3.1. Relationship between(θn-θ) and ∇fn(θ)

By definition ofθn, we have ∇fn(θn)=0, so that(3.12)∇fn(θn)-∇fn(θ)=-∇fn(θ).
For each i∈{1,…,d}, a Taylor expansion applied to the real valued application ∂fn/∂xi implies the existence of ɛn(i)=(ɛn(1)(i),…,ɛn(d)(i))t such that (3.13)∂fn∂xi(θn)-∂fn∂xi(θ)=∑j=1d∂2fn∂xi∂xj(ɛn(i))(θn(j)-θ(j)),|ɛn(j)(i)-θ(j)|≤|θn(j)(i)-θ(j)|∀j∈{1,…,d}.
Define the d×d matrix Hn=(Hn(i,j))1≤i,j≤d by setting Hn(i,j)=(∂2fn/∂xi∂xj)(ɛn(i)); (3.12) can then be rewritten as(3.14)Hn(θn-θ)=-∇fn(θ).
Now, let U be a compact set of ℝd containing θ. The combination of Lemmas 3.1 and 3.2 with |α|=2, gn=fn, and bn=hn ensures that, for any γ>0 and any ɛ∈]0,1[,(3.15)sup⁡x∈U|∂[α]fn(x)-∂[α]f(x)|=O((log⁡⁡n)1+γ∑i=1nhid+4+∑i=1nhiqn)a.s.=O((log⁡⁡n)1+γnhnd+4+hnq+nε-1)=o(1)a.s.SinceD2f is continuous in a neighbourhood of θ and since limn→∞θn=θ a.s., (3.15) ensures that(3.16)lim⁡n→∞⁡Hn=D2f(θ)a.s.
In view of (3.14), we can thus state the following lemma.Lemma 3.5.
Under Assumptions (A1), (A2), (A3)(i), (A4), (A5), and (A6)(i), the weak and strong asymptotic behaviour of(θn-θ)T is given by that of -[D2f(θ)]-1∇fn(θ).


#### 3.3.2. Proof of Theorems2.5 and 2.14

Theorem2.5 (resp., Theorem 2.14) straightforwardly follows from the application of Lemma 3.5 of the first part of Lemma 3.3 (resp., of the first part of Lemma 3.4) and of the following lemma.Lemma 3.6.
Let (A1), (A2), (A3)(i), (A4), (A5), and (A6)(i) hold.(1)
Iflimn→∞nhnd+2+2q∈]0,∞], then limn→∞hn-q𝔼(∇fn(θ))=Ba,q(θ).(2)
Iflimn→∞nhnd+2+2q=0, then limn→∞nhnd+2𝔼(∇fn(θ))=0.Let us now prove Lemma3.6. The application of Lemma 3.1 (with gn=fn and |α|=1) ensures that(3.17)lim⁡n→∞n∑i=1nhiqE(∇fn(θ))=(-1)qq!∇(∑j=1dβjq∂qf∂xjq(θ)).
Let us first consider the case when aq<1. By application of (3.1), we have (3.18)lim⁡n→∞hn-qE(∇fn(θ))=Ba,q(θ).(i)
Iflimn→∞nhnd+2+2q∈]0,∞], then 1-a(d+2+2q)≥0, and thus aq<1/2; Part 1 of Lemma 3.6 thus follows.(ii)
Iflimn→∞nhnd+2+2q=0, then limn→∞nhnd+2hnq=0, and Part 2 of Lemma 3.6 also follows.It remains to prove Part 2 of Lemma3.6 in the case when aq≥1. The application of (3.2) then ensures that, for all ɛ∈]0,1/2[, (3.19)nhnd+2∑i=1nhiqn=O(nhnd+2[hnq+nɛ-1])=o(1).
In view of (3.17), Part 2 of Lemma 3.6 then follows.


### 3.4. Convergence Rate ofμ̃n: Proof of Theorems 2.7 and 2.15

In order to prove Theorems2.7 and 2.15, we show that the weak and strong convergence rates of μ̃n-μ are given by those of f̃n(θ)-f(θ). More precisely, set(3.20)Rn=(μ̃n-μ)-(f̃n(θ)-f(θ)),
and assume that the following two lemmas hold.Lemma 3.7.
Let (A1)–(A6) hold.(1)
Iflimn→∞nh̃nd+2q∈]0,∞], then limn→∞h̃n-q[𝔼(f̃n(θ))-f(θ)]=Bã,q(μ).(2)
Iflimn→∞nh̃nd+2q=0, then limn→∞nh̃nd[𝔼(f̃n(θ))-f(θ)]=0.Lemma 3.8.
Let (A1)–(A6) hold. For allγ>0, Rn2=O(Pn(γ)) a.s.In order to prove Theorem2.7, we first note that the application of the second part of Lemma 3.3 yields(3.21)nh̃nd[f̃n(θ)-E(f̃n(θ))]⟶DN(0,Σã(μ)).
In view of Lemma 3.7, it follows that if limn→∞nh̃nd+2q=c̃≥0, then(3.22)nh̃nd[f̃n(θ)-f(θ)]⟶DN(c̃Bã,q(μ),Σã(μ)),
and if limn→∞nh̃nd+2q=∞, then(3.23)h̃n-q[f̃n(θ)-f(θ)]⟶PBã,q(μ).
Part 1 (resp., Part 2) of Theorem 2.7 straightforwardly follows from the combination of (3.22) (resp., of (3.23)) and of Lemma 3.8.The proof of Theorem2.15 follows that of Theorem 2.7 (except that the second part of Lemma 3.4 stands instead of (3.21)); this proof is thus omitted.It remains to prove Lemmas3.7 and 3.8. The proof of Lemma 3.7 follows that of Lemma 3.6 and is omitted. We now prove Lemma 3.8. We first note that Rn=f̃n(θn)-f̃n(θ); a Taylor's expansion implies the existence of ζn such that ∥ζn-θn∥≤∥θn-θ∥ and (3.24)Rn=(θn-θ)T∇f̃n(ζn)=(θn-θ)T[∇f̃n(ζn)-∇f(ζn)+∇f(ζn)-∇f(θ)].
Let 𝒱 be a compact set that contains θ; for n large enough, we get (3.25)|Rn|=O(‖θn-θ‖[sup⁡x∈V‖∇f̃n(x)-∇f(x)‖+‖ζn-θ‖])=O(‖θn-θ‖sup⁡x∈V‖∇f̃n(x)-∇f(x)‖+‖θn-θ‖2).
To get an upper bound of |Rn|, we thus need to establish upper bounds of ∥θn-θ∥ and of supx∈𝒱∥∇f̃n(x)-∇f(x)∥.(i)
Let us recall that the a.s. convergence rate of(θn-θ) is given by that of [D2f(θ)]-1∇fn(θ) (see Lemma 3.5). Theorem 2.14 can be applied to obtain the exact a.s. convergence rate of θn-θ. However, to avoid assuming (A7)(i), we apply here Lemmas 3.1 and 3.2 (with |α|=1 and (gn,bn)=(fn,hn)) and get the following upper bound of the a.s. convergence rate of ∇fn(θ) and thus of θn-θ: for any γ>0 and ɛ∈]0,(1+a(d+2))/2[,
(3.26)‖θn-θ‖2=O((log⁡⁡n)1+γ∑i=1nhid+2+[∑i=1nhiqn]2)=O((log⁡⁡n)1+γnhnd+2+[hnq+nɛ-1]2)a.s.=O(w(hn,γ))a.s.(ii)
Noting that(3.27)sup⁡x∈V‖∇f̃n(x)-∇f(x)‖≤sup⁡x∈V‖∇f̃n(x)-E(∇f̃n(x))‖+sup⁡x∈V‖E(∇f̃n(x))-∇f(x)‖,
the application of Lemmas 3.1 and 3.2 with |α|=1, (gn,bn)=(f̃n,h̃n) ensures that, for any γ>0 and ɛ∈]0,(1+ã(d+2))/2[,(3.28)sup⁡x∈V‖∇f̃n(x)-∇f(x)‖2=O((log⁡⁡n)1+γnh̃nd+2+[∑i=1nh̃iqn]2)a.s.=O((log⁡⁡n)1+γnh̃nd+2+[h̃nq+nɛ-1]2)a.s.=O(w(h̃n,γ))a.s.
It follows from (3.25), (3.26), and (3.28) that (3.29)|Rn|2=O(w(hn,γ)w(h̃n,γ)+[w(hn,γ)]2)a.s.=O(Pn(γ))a.s.,
which concludes the proof of Lemma 3.8.


### 3.5. Joint Convergence Rate ofθn and μ̃n: Proof of Corollaries 2.10 and 2.17

In view of (3.14) and (3.20), we have (3.30)(Hn001)(θn-θμ̃n-μ)=(-∇fn(θ)f̃n(θ)-f(θ))+(0Rn),
and, in view of (3.16), the weak and strong asymptotic behaviour of (θn-θμ̃n-μ) is given by that of (3.31)([D2f(θ)]-1001)[(-∇fn(θ)f̃n(θ)-f(θ))+(0Rn)]=([-D2f(θ)]-1[∇fn(θ)-E(∇fn(θ))]f̃n(θ)-E(f̃n(θ)))+([-D2f(θ)]-1E(∇fn(θ))E(f̃n(θ))-f(θ)+Rn).
Under the assumptions of Corollary 2.10, the application of Part 1 of Lemma 3.6, that of Part 1 of Lemma 3.7, and that of Lemma 3.8 ensure that (3.32)lim⁡n→∞(nhnd+2[-D2f(θ)]-1E(∇fn(θ))nh̃nd[E(f̃n(θ))-f(θ)+Rn])=D(c,c̃)ABa,ã,q.
Corollary 2.10 then follows from the application of Part 2 of Lemma 3.3. The proof of Corollary 2.17 follows that of Corollary 2.10 (except that Part 3 of Lemma 3.4 is applied instead of Part 2 of Lemma 3.3) and is omitted.


### 3.6. Proof of Lemmas3.2–3.4

#### 3.6.1. Proof of Lemma3.2

Setvn=[∑i=1nbid+2|α|]1/2[(logn)1+γ]-1/2. Applying Proposition 3 in Mokkadem et al. [27], it holds that for any δ>0, there exists c(δ)>0 such that (3.33)P[sup⁡x∈Uvn|∂[α]gn(x)-E(∂[α]gn(x))|≥δ]≤exp⁡⁡(-c(δ)∑i=1nbid+2|α|2vn2).
Since limn→∞∑i=1nbid+2|α|/(vn2logn)=∞, we have, for n large enough, c(δ)∑i=1nbid+2|α|/2vn2≥2logn, and Lemma 3.2 follows from the application of Borel-Cantelli's Lemma.


#### 3.6.2. Proof of Lemma3.3

We only prove the second part of Lemma3.3: the proof of its first part can be easily deduced. We first establish that(3.34)lim⁡n→∞E(WnWnT)=Σa,ã
and then check that (Wn) satisfies Lyapounov's condition. Set (3.35)Yk,n=1nhn-d-2hk-d-1[∇K(θ-Xkhk)-E(∇K(θ-Xkhk))],Zk,n=1nh̃n-dh̃k-d[K(θ-Xkh̃k)-E(K(θ-Xkh̃k))],
and note that (3.36)E(WnWnT)=∑k=1n(E(Yk,nYk,nT)E(Yk,nZk,n)E(Yk,nTZk,n)E(Zk,n2)).
Now, for any s,t∈{1,…,d}, we have (3.37)E[∂K∂xs(θ-Xkhk)∂K∂xt(θ-Xkhk)]=∫Rd∂K∂xs(θ-yhk)∂K∂xt(θ-yhk)f(y)dy=hkdf(θ)Gs,t+o(hkd),
and since 𝔼[(∂K/∂xs)(θ-Xk/hk)]=O(hkd), we deduce that(3.38)E([∇K(θ-Xkhk)-E(∇K(θ-Xkhk))][∇K(θ-Xkhk)-E(∇K(θ-Xkhk))]T)=f(θ)Ghkd[1+o(1)],
which implies that limn→∞∑k=1n𝔼(Yk,nYk,nT)=f(θ)[1+a(d+2)]-1G. In the same way, we have(3.39)E([K(θ-Xkh̃k)-E(K(θ-Xkh̃k))]2)=h̃kdf(θ)∫RdK2(z)dz[1+o(1)],
and thus limn→∞∑k=1n𝔼(Zk,n2)=f(θ)[1+ãd]-1∫ℝdK2(z)dz. Moreover, set hn*=min(hn,h̃n); we have (3.40)E[∇K(θ-Xkhk)K(θ-Xkh̃k)]=hk*d∫Rd∇K(hk*hkz)K(hk*h̃kz)f(θ-hk*z)dz.
Noting that f(θ-hk*z)=f(θ)+hk*Rk(θ,z) with |Rk(θ,z)|≤∥∇f∥∞∥z∥, we get (3.41)E[∇K(θ-Xkhk)K(θ-Xkh̃k)]=hk*d[f(θ)∫Rd∇K(hk*hkz)K(hk*h̃kz)dz+hk*∫Rd∇K(hk*hkz)K(hk*h̃kz)Rk(θ,z)dz].
Since the function z↦[∇K(z)]K(z is odd (in each coordinate), the first right-handed integral is zero, and since hk* equals either hk or h̃k, we get (3.42)‖E[∇K(θ-Xkhk)K(θ-Xkh̃k)]‖≤hk*(d+1)‖∇f‖∞[‖K‖∞∫Rd‖z‖‖∇K(z)‖dz+‖∇K‖∞∫Rd‖z‖|K(z)|dz]=O(hk*(d+1)).
We then deduce that(3.43)E([∇K(θ-Xkhk)-E(∇K(θ-Xkhk))][K(θ-Xkh̃k)-E(K(θ-Xkh̃k))])=O([min⁡(hk,h̃k)]d+1)+O(hkdh̃kd)=O(hk(d+1)/2h̃k(d+1)/2),
and thus, in view of (3.1), (3.44)∑k=1nE(Yk,nZk,n)=O(1(nhn-d-2)(nh̃n-d)∑k=1nhk-(d+1)/2h̃k(1-d)/2)=o(1),
which concludes the proof of (3.34). Let us now check that (Wn) satisfies the Lyapounov's condition. Set p>2. Since K and ∇K are bounded and integrable, we have ∫ℝd∥∇K(z)∥pdz<∞ and ∫ℝd|K(z)|pdz<∞. It follows that (3.45)∑k=1nE(‖Yk,n‖p)=O(1(nhn-d-2)p/2∑k=1nhk(-d-1)p∫Rd‖∇K(θ-yhk)‖pf(y)dy)=O(1(nhn-d-2)p/2∑k=1nhk(-d-1)phkd)=o(1),∑k=1nE(|Zk,n|p)=O(1(nh̃n-d)p/2∑k=1nh̃k-dp∫Rd|K(θ-yh̃k)|pf(y)dy)=O(1(nh̃n-d)p/2∑k=1nh̃k-dph̃kd)=o(1),
which concludes the proof of Lemma 3.3.


#### 3.6.3. Proof of Lemma3.4

We only prove the third part of Lemma3.4: the proofs of its first two parts can be easily deduced. Set (3.46)Γ=f(θ)(G00∫RdK2(z)dz),Δn=(1nhn-d-2Id001nh̃n-d),Qn=(hn-d-2Id00h̃n-d),
let (ɛn) be a sequence of ℝd+1-valued, independent, and 𝒩(0,Γ)-distributed random vectors, and set Sn=∑k=1nQkɛk and Bn=𝔼(SnSnT). We clearly have limn→+∞ΔnBnΔn=Σa,ã. Moreover, all the eigenvalues ρn(i)(1≤i≤d+1) of Bn satisfy limn→+∞loglogρn(i)/loglogn=1. The application of Theorem 1 in Mokkadem and Pelletier [30] then ensures that, with probability one, the sequence(3.47)(ΔnSn2log⁡⁡log⁡⁡n)
is relatively compact, and its limit set is the ellipsoid {ν∈ℝd+1such thatνTΣa,ã-1ν≤1}. Now, set (3.48)Ṽk=(hk-d/2[∇K(θ-Xkhk)-E(∇K(θ-Xkhk))]h̃k-d/2[K(θ-Xkh̃k)-E(K(θ-Xkh̃k))])
and Γk=𝔼(ṼkṼkT). In view of (3.38), (3.39), and (3.43), we have limk→∞Γk=Γ. It follows that ∃k0≥1 such that for allk≥k0, Γk is inversible; without loss of generality, we assume k0=1, and set Ũk=Γk-1/2Ṽk. Set p∈]2,4[ and let ℒ be a slowly varying function; we have (3.49)E(‖Ũk‖p)(klog⁡⁡log⁡⁡k)p/2=O(hk-dp/2E[‖∇K((θ-Xk)/hk)‖p]+h̃k-dp/2E[|K((θ-Xk)/hk)|p](klog⁡log⁡k)p/2)=O(hkd-dp/2+h̃kd-dp/2(klog⁡⁡log⁡⁡k)p/2)=O(L(k)[k-[1+((p/2)-1)(1-ad)]+k-[1+((p/2)-1)(1-ãd)]])
so that ∑k(kloglogk)-p/2𝔼(∥Ũk∥p)<∞. By application of Theorem 2 of Einmahl [31], we deduce that ∑k=1nŨk-∑k=1nηk=o(nloglogn) a.s., where ηk are independent and 𝒩(0,Id+1)-distributed random vectors. It follows that(3.50)∑k=1nΓ1/2Γk-1/2Ṽk-∑k=1nɛk=o(nlog⁡⁡log⁡⁡n)a.s.
Now, (3.51)Δn[∑k=1nQkΓ1/2Γk-1/2Ṽk-∑k=1nQkɛk]=Δn∑k=1nQk[Γ1/2Γk-1/2Ṽk-ɛk]=Δn∑k=1nQk(∑j=1k[Γ1/2Γj-1/2Ṽj-ɛj]-∑j=1k-1[Γ1/2Γj-1/2Ṽj-ɛj])(with∑j=10=0)=Δn∑k=1n-1(Qk-Qk+1)(∑j=1k(Γ1/2Γj-1/2Ṽj-ɛj))+ΔnQn∑j=1n(Γ1/2Γj-1/2Ṽj-ɛj)=Δn∑k=1n-1(Qk-Qk+1)[o(klog⁡⁡log⁡⁡k)]+ΔnQn[o(nlog⁡⁡log⁡⁡n)]a.s.
Moreover, (3.52)Δn∑k=1n-1(Qk-Qk+1)[o(klog⁡⁡log⁡⁡k)]=(hnd+2n∑k=1n-1(hk-(d+2)/2-hk+1-(d+2)/2)o(klog⁡⁡log⁡⁡k)00h̃ndn∑k=1n-1(h̃k-d/2-h̃k+1-d/2))=(o(hnd+2log⁡⁡log⁡⁡n)∑k=1n-1(hk-(d+2)/2-hk+1-(d+2)/2)00o(h̃ndlog⁡⁡log⁡⁡n)∑k=1n-1(h̃k-d/2-h̃k+1-d/2)).
Set ϕ(s)=[h(s)]-(d+2)/2 and ϕ̃(s)=[h̃(s)]-d/2, and let uk∈[k,k+1]; since ϕ′ and ϕ̃′ vary regularly with exponent (a(d+2)/2-1) and (ãd/2-1), respectively, we have (3.53)∑k=1n-1(hk-(d+2)/2-hk+1-(d+2)/2)=O(∑k=1n-1ϕ′(uk))=O(∫1nϕ′(s)ds)=O(hn-(d+2)/2),∑k=1n-1(h̃k-d/2-h̃k+1-d/2)=O(∑k=1n-1ϕ̃′(uk))=O(∫1nϕ̃′(s)ds)=O(h̃n-d/2)
so that (3.54)Δn∑k=1n-1(Qk-Qk+1)[o(klog⁡⁡log⁡⁡k)]=o(log⁡⁡log⁡⁡n).
Since ΔnQn[o(nloglogn)]=o(loglogn), we deduce that (3.55)Δn∑k=1nQkΓ1/2Γk-1/2Ṽk2log⁡⁡log⁡⁡n-Δn∑k=1nQkɛk2log⁡⁡log⁡⁡n=o(1)a.s.
The application of (3.47) then ensures that, with probability one, the sequence (3.56)(Δn∑k=1nQkΓ1/2Γk-1/2Ṽk2log⁡⁡log⁡⁡n)
is relatively compact, and its limit set is ℰ={ν∈ℝd+1such thatνTΣa,ã-1ν≤1}. Since (3.57)Δn∑k=1nQkṼk2log⁡⁡log⁡⁡n=Δn∑k=1nQkΓ1/2Γk-1/2Ṽk2log⁡⁡log⁡⁡n+Δn∑k=1nQk(Id+1-Γ1/2Γk-1/2)Ṽk2log⁡⁡log⁡⁡n
with limk→∞(Id+1-Γ1/2Γk-1/2)=0, Lemma 3.4 follows.


## 3.1. Consistency ofθn and μ̃n: Proof of Proposition 2.3

Sinceθn is the mode of fn and θ the mode of f, we have(3.3)0≤f(θ)-f(θn)=[f(θ)-fn(θn)]+[fn(θn)-f(θn)]≤[f(θ)-fn(θ)]+[fn(θn)-f(θn)]≤|f(θ)-fn(θ)|+|fn(θn)-f(θn)|≤2‖fn-f‖∞.
The application of Theorem 5 in Mokkadem et al. [27] with |α|=0 and vn=lognensures that, for any δ>0, there exists c(δ)>0 such that ℙ[(logn)∥fn-𝔼(fn)∥∞≥δ]≤exp(-c(δ)∑i=1nhid/(logn)2). In view of (3.1), since ad<1, we can write (3.4)n2exp⁡⁡(-c(δ)∑i=1nhid(log⁡⁡n)2)=n2exp⁡⁡(-c(δ)nhnd(log⁡⁡n)2∑i=1nhidnhnd)=o(1).
Borell-Cantelli's Lemma then ensures that limn→∞∥fn-𝔼(fn)∥∞=0 a.s. Since limn→∞∥𝔼(fn)-f∥∞=0, it follows from (3.3) that limn→∞f(θn)=f(θ) a.s. Since f is continuous, since lim∥z∥→∞f(z)=0, and since θ is the unique mode of f, we deduce that limn→∞θn=θ a.s. Now, we have (3.5)|μ̃n-μ|≤|f̃n(θn)-f(θn)|+|f(θn)-f(θ)|≤‖f̃n-f‖∞+2‖fn-f‖∞,
where the last inequality follows from (3.3). Following the proof of the strong uniform convergence of fn, we show that limn→∞∥f̃n-f∥∞=0 a.s. It follows that limn→∞μ̃n=μ a.s., which concludes the proof of Proposition 2.3.


## 3.2. Some Preliminary Lemmas

The aim of this section is to state some properties of the density estimators and of their derivatives, which are crucial in the proof of Theorems2.5–2.15 and of Corollaries 2.10 and 2.17.


### 3.2.1. Strong Uniform Convergence Rate of the Derivatives of the Density Estimators

For anyd-uplet [α]=(α1,…,αd)∈ℕd, set |α|=α1+⋯+αd and, for any function g, let ∂[α]g(x)=∂|α|g/(∂x1α1…∂xdαd)(x) denote the [α]-th partial derivative of g (if |α|=0, then ∂[α]g=g). In order to prove Theorems 2.5–2.15 and Corollaries 2.10 and 2.17, we need to know the behaviour of ∂[α]fn for |α|∈{0,1,2} and that of ∂[α]f̃n for |α|∈{0,1}. For the sake of conciseness, we state the preliminary lemmas 3.1 and 3.2 for ∂[α]gn where either gn=fn and |α|∈{0,1,2} or gn=f̃n and |α|∈{0,1}. Moreover, we set (bn)=(hn) if gn=fn, and (bn)=(h̃n) if gn=f̃n.Lemma 3.1.
Let (A1), (A2), (A3)(i), (A4), (A5), and (A6)(i) hold. Moreover, ifgn=f̃n, then let (A3)(ii) and (A6)(ii) hold. We have
(3.6)lim⁡n→∞n∑i=1nbiq[E[∂[α]gn(x)]-∂[α]f(x)]=(-1)qq!∂[α](∑j=1dβjq∂qf∂xjq)(x),
where βjq is defined in (2.1). Moreover, if we set Mq=supx∈ℝd∥Dq∂[α]f(x)∥, then
(3.7)lim⁡n→∞n∑i=1nbiqsup⁡x∈Rd|E(∂[α]gn(x))-∂[α]f(x)|≤Mqq!∫Rd‖z‖q|K(z)|dz.Lemma 3.2.
LetU be a compact set of  ℝd, and assume that (A1), (A2), (A3)(i), (A4), (A5), and (A6)(i) hold. Moreover, if gn=f̃n, then let (A3)(ii) and (A6)(ii) hold. Then, for all γ>0, we have
(3.8)sup⁡x∈U|∂[α]gn(x)-E(∂[α]gn(x))|=O((log⁡⁡n)1+γ∑i=1nbid+2|α|)a.s.Lemma3.1 is proved in Mokkadem et al. [27], Lemma 3.2 in Section 3.6.


### 3.2.2. Convergence Rate of(∇fn(θ)-𝔼(∇fn(θ)),f̃n(θ)-𝔼(f̃n(θ)))

Lemma 3.3.
(1)
Under Assumptions (A1), (A2), (A3)(i), (A4), (A5), and (A6)(i), we have(3.9)nhnd+2[∇fn(θ)-E(∇fn(θ))]⟶DN(0,Σa(θ)).(2)
Under Assumptions (A1)–(A6), we have(3.10)Wn=(nhnd+2[∇fn(θ)-E(∇fn(θ))]nh̃nd[f̃n(θ)-E(f̃n(θ))])⟶DN(0,Σa,ã).Lemma 3.4.
(1)
Under Assumptions (A1), (A2), (A3)(i), (A4), (A5), (A6)(i), and (A7)(i), with probability one, the sequence((nhnd+2[∇fn(θ)-𝔼(∇fn(θ))])/2loglogn) is relatively compact, and its limit set is {ν∈ℝdsuch thatνT[Σa(θ)]-1ν≤1}.(2)
Under Assumptions (A1)–(A7), with probability one, the sequence((nh̃nd[f̃n(θ)-𝔼(f̃n(θ))])/2loglogn) is relatively compact, and its limit set is the interval [-Σã(μ);Σã(μ)].(3)
Under Assumptions (A1)–(A8), with probability one, the sequence(3.11)12log⁡⁡log⁡⁡n(nhnd+2[∇fn(θ)-E(∇fn(θ))]nh̃nd[f̃n(θ)-E(f̃n(θ))])
is relatively compact, and its limit set is {ν∈ℝd+1suchthatνTΣa,ã-1ν≤1}.


## 3.2.1. Strong Uniform Convergence Rate of the Derivatives of the Density Estimators

For anyd-uplet [α]=(α1,…,αd)∈ℕd, set |α|=α1+⋯+αd and, for any function g, let ∂[α]g(x)=∂|α|g/(∂x1α1…∂xdαd)(x) denote the [α]-th partial derivative of g (if |α|=0, then ∂[α]g=g). In order to prove Theorems 2.5–2.15 and Corollaries 2.10 and 2.17, we need to know the behaviour of ∂[α]fn for |α|∈{0,1,2} and that of ∂[α]f̃n for |α|∈{0,1}. For the sake of conciseness, we state the preliminary lemmas 3.1 and 3.2 for ∂[α]gn where either gn=fn and |α|∈{0,1,2} or gn=f̃n and |α|∈{0,1}. Moreover, we set (bn)=(hn) if gn=fn, and (bn)=(h̃n) if gn=f̃n.Lemma 3.1.
Let (A1), (A2), (A3)(i), (A4), (A5), and (A6)(i) hold. Moreover, ifgn=f̃n, then let (A3)(ii) and (A6)(ii) hold. We have
(3.6)lim⁡n→∞n∑i=1nbiq[E[∂[α]gn(x)]-∂[α]f(x)]=(-1)qq!∂[α](∑j=1dβjq∂qf∂xjq)(x),
where βjq is defined in (2.1). Moreover, if we set Mq=supx∈ℝd∥Dq∂[α]f(x)∥, then
(3.7)lim⁡n→∞n∑i=1nbiqsup⁡x∈Rd|E(∂[α]gn(x))-∂[α]f(x)|≤Mqq!∫Rd‖z‖q|K(z)|dz.Lemma 3.2.
LetU be a compact set of  ℝd, and assume that (A1), (A2), (A3)(i), (A4), (A5), and (A6)(i) hold. Moreover, if gn=f̃n, then let (A3)(ii) and (A6)(ii) hold. Then, for all γ>0, we have
(3.8)sup⁡x∈U|∂[α]gn(x)-E(∂[α]gn(x))|=O((log⁡⁡n)1+γ∑i=1nbid+2|α|)a.s.Lemma3.1 is proved in Mokkadem et al. [27], Lemma 3.2 in Section 3.6.


## 3.2.2. Convergence Rate of(∇fn(θ)-𝔼(∇fn(θ)),f̃n(θ)-𝔼(f̃n(θ)))

Lemma 3.3.
(1)
Under Assumptions (A1), (A2), (A3)(i), (A4), (A5), and (A6)(i), we have(3.9)nhnd+2[∇fn(θ)-E(∇fn(θ))]⟶DN(0,Σa(θ)).(2)
Under Assumptions (A1)–(A6), we have(3.10)Wn=(nhnd+2[∇fn(θ)-E(∇fn(θ))]nh̃nd[f̃n(θ)-E(f̃n(θ))])⟶DN(0,Σa,ã).Lemma 3.4.
(1)
Under Assumptions (A1), (A2), (A3)(i), (A4), (A5), (A6)(i), and (A7)(i), with probability one, the sequence((nhnd+2[∇fn(θ)-𝔼(∇fn(θ))])/2loglogn) is relatively compact, and its limit set is {ν∈ℝdsuch thatνT[Σa(θ)]-1ν≤1}.(2)
Under Assumptions (A1)–(A7), with probability one, the sequence((nh̃nd[f̃n(θ)-𝔼(f̃n(θ))])/2loglogn) is relatively compact, and its limit set is the interval [-Σã(μ);Σã(μ)].(3)
Under Assumptions (A1)–(A8), with probability one, the sequence(3.11)12log⁡⁡log⁡⁡n(nhnd+2[∇fn(θ)-E(∇fn(θ))]nh̃nd[f̃n(θ)-E(f̃n(θ))])
is relatively compact, and its limit set is {ν∈ℝd+1suchthatνTΣa,ã-1ν≤1}.


## 3.3. Convergence Rate ofθn: Proof of Theorems 2.5 and 2.14

In order to prove Theorems2.5 and 2.14, we first show that the weak and strong asymptotic behaviours of θn-θ are given by those of -[D2f(θ)]-1∇fn(θ) (see Section 3.3.1) and then deduce the convergence rates of θn-θ from those of ∇fn(θ) (see Section 3.3.2).


### 3.3.1. Relationship between(θn-θ) and ∇fn(θ)

By definition ofθn, we have ∇fn(θn)=0, so that(3.12)∇fn(θn)-∇fn(θ)=-∇fn(θ).
For each i∈{1,…,d}, a Taylor expansion applied to the real valued application ∂fn/∂xi implies the existence of ɛn(i)=(ɛn(1)(i),…,ɛn(d)(i))t such that (3.13)∂fn∂xi(θn)-∂fn∂xi(θ)=∑j=1d∂2fn∂xi∂xj(ɛn(i))(θn(j)-θ(j)),|ɛn(j)(i)-θ(j)|≤|θn(j)(i)-θ(j)|∀j∈{1,…,d}.
Define the d×d matrix Hn=(Hn(i,j))1≤i,j≤d by setting Hn(i,j)=(∂2fn/∂xi∂xj)(ɛn(i)); (3.12) can then be rewritten as(3.14)Hn(θn-θ)=-∇fn(θ).
Now, let U be a compact set of ℝd containing θ. The combination of Lemmas 3.1 and 3.2 with |α|=2, gn=fn, and bn=hn ensures that, for any γ>0 and any ɛ∈]0,1[,(3.15)sup⁡x∈U|∂[α]fn(x)-∂[α]f(x)|=O((log⁡⁡n)1+γ∑i=1nhid+4+∑i=1nhiqn)a.s.=O((log⁡⁡n)1+γnhnd+4+hnq+nε-1)=o(1)a.s.SinceD2f is continuous in a neighbourhood of θ and since limn→∞θn=θ a.s., (3.15) ensures that(3.16)lim⁡n→∞⁡Hn=D2f(θ)a.s.
In view of (3.14), we can thus state the following lemma.Lemma 3.5.
Under Assumptions (A1), (A2), (A3)(i), (A4), (A5), and (A6)(i), the weak and strong asymptotic behaviour of(θn-θ)T is given by that of -[D2f(θ)]-1∇fn(θ).


### 3.3.2. Proof of Theorems2.5 and 2.14

Theorem2.5 (resp., Theorem 2.14) straightforwardly follows from the application of Lemma 3.5 of the first part of Lemma 3.3 (resp., of the first part of Lemma 3.4) and of the following lemma.Lemma 3.6.
Let (A1), (A2), (A3)(i), (A4), (A5), and (A6)(i) hold.(1)
Iflimn→∞nhnd+2+2q∈]0,∞], then limn→∞hn-q𝔼(∇fn(θ))=Ba,q(θ).(2)
Iflimn→∞nhnd+2+2q=0, then limn→∞nhnd+2𝔼(∇fn(θ))=0.Let us now prove Lemma3.6. The application of Lemma 3.1 (with gn=fn and |α|=1) ensures that(3.17)lim⁡n→∞n∑i=1nhiqE(∇fn(θ))=(-1)qq!∇(∑j=1dβjq∂qf∂xjq(θ)).
Let us first consider the case when aq<1. By application of (3.1), we have (3.18)lim⁡n→∞hn-qE(∇fn(θ))=Ba,q(θ).(i)
Iflimn→∞nhnd+2+2q∈]0,∞], then 1-a(d+2+2q)≥0, and thus aq<1/2; Part 1 of Lemma 3.6 thus follows.(ii)
Iflimn→∞nhnd+2+2q=0, then limn→∞nhnd+2hnq=0, and Part 2 of Lemma 3.6 also follows.It remains to prove Part 2 of Lemma3.6 in the case when aq≥1. The application of (3.2) then ensures that, for all ɛ∈]0,1/2[, (3.19)nhnd+2∑i=1nhiqn=O(nhnd+2[hnq+nɛ-1])=o(1).
In view of (3.17), Part 2 of Lemma 3.6 then follows.


## 3.3.1. Relationship between(θn-θ) and ∇fn(θ)

By definition ofθn, we have ∇fn(θn)=0, so that(3.12)∇fn(θn)-∇fn(θ)=-∇fn(θ).
For each i∈{1,…,d}, a Taylor expansion applied to the real valued application ∂fn/∂xi implies the existence of ɛn(i)=(ɛn(1)(i),…,ɛn(d)(i))t such that (3.13)∂fn∂xi(θn)-∂fn∂xi(θ)=∑j=1d∂2fn∂xi∂xj(ɛn(i))(θn(j)-θ(j)),|ɛn(j)(i)-θ(j)|≤|θn(j)(i)-θ(j)|∀j∈{1,…,d}.
Define the d×d matrix Hn=(Hn(i,j))1≤i,j≤d by setting Hn(i,j)=(∂2fn/∂xi∂xj)(ɛn(i)); (3.12) can then be rewritten as(3.14)Hn(θn-θ)=-∇fn(θ).
Now, let U be a compact set of ℝd containing θ. The combination of Lemmas 3.1 and 3.2 with |α|=2, gn=fn, and bn=hn ensures that, for any γ>0 and any ɛ∈]0,1[,(3.15)sup⁡x∈U|∂[α]fn(x)-∂[α]f(x)|=O((log⁡⁡n)1+γ∑i=1nhid+4+∑i=1nhiqn)a.s.=O((log⁡⁡n)1+γnhnd+4+hnq+nε-1)=o(1)a.s.SinceD2f is continuous in a neighbourhood of θ and since limn→∞θn=θ a.s., (3.15) ensures that(3.16)lim⁡n→∞⁡Hn=D2f(θ)a.s.
In view of (3.14), we can thus state the following lemma.Lemma 3.5.
Under Assumptions (A1), (A2), (A3)(i), (A4), (A5), and (A6)(i), the weak and strong asymptotic behaviour of(θn-θ)T is given by that of -[D2f(θ)]-1∇fn(θ).


## 3.3.2. Proof of Theorems2.5 and 2.14

Theorem2.5 (resp., Theorem 2.14) straightforwardly follows from the application of Lemma 3.5 of the first part of Lemma 3.3 (resp., of the first part of Lemma 3.4) and of the following lemma.Lemma 3.6.
Let (A1), (A2), (A3)(i), (A4), (A5), and (A6)(i) hold.(1)
Iflimn→∞nhnd+2+2q∈]0,∞], then limn→∞hn-q𝔼(∇fn(θ))=Ba,q(θ).(2)
Iflimn→∞nhnd+2+2q=0, then limn→∞nhnd+2𝔼(∇fn(θ))=0.Let us now prove Lemma3.6. The application of Lemma 3.1 (with gn=fn and |α|=1) ensures that(3.17)lim⁡n→∞n∑i=1nhiqE(∇fn(θ))=(-1)qq!∇(∑j=1dβjq∂qf∂xjq(θ)).
Let us first consider the case when aq<1. By application of (3.1), we have (3.18)lim⁡n→∞hn-qE(∇fn(θ))=Ba,q(θ).(i)
Iflimn→∞nhnd+2+2q∈]0,∞], then 1-a(d+2+2q)≥0, and thus aq<1/2; Part 1 of Lemma 3.6 thus follows.(ii)
Iflimn→∞nhnd+2+2q=0, then limn→∞nhnd+2hnq=0, and Part 2 of Lemma 3.6 also follows.It remains to prove Part 2 of Lemma3.6 in the case when aq≥1. The application of (3.2) then ensures that, for all ɛ∈]0,1/2[, (3.19)nhnd+2∑i=1nhiqn=O(nhnd+2[hnq+nɛ-1])=o(1).
In view of (3.17), Part 2 of Lemma 3.6 then follows.


## 3.4. Convergence Rate ofμ̃n: Proof of Theorems 2.7 and 2.15

In order to prove Theorems2.7 and 2.15, we show that the weak and strong convergence rates of μ̃n-μ are given by those of f̃n(θ)-f(θ). More precisely, set(3.20)Rn=(μ̃n-μ)-(f̃n(θ)-f(θ)),
and assume that the following two lemmas hold.Lemma 3.7.
Let (A1)–(A6) hold.(1)
Iflimn→∞nh̃nd+2q∈]0,∞], then limn→∞h̃n-q[𝔼(f̃n(θ))-f(θ)]=Bã,q(μ).(2)
Iflimn→∞nh̃nd+2q=0, then limn→∞nh̃nd[𝔼(f̃n(θ))-f(θ)]=0.Lemma 3.8.
Let (A1)–(A6) hold. For allγ>0, Rn2=O(Pn(γ)) a.s.In order to prove Theorem2.7, we first note that the application of the second part of Lemma 3.3 yields(3.21)nh̃nd[f̃n(θ)-E(f̃n(θ))]⟶DN(0,Σã(μ)).
In view of Lemma 3.7, it follows that if limn→∞nh̃nd+2q=c̃≥0, then(3.22)nh̃nd[f̃n(θ)-f(θ)]⟶DN(c̃Bã,q(μ),Σã(μ)),
and if limn→∞nh̃nd+2q=∞, then(3.23)h̃n-q[f̃n(θ)-f(θ)]⟶PBã,q(μ).
Part 1 (resp., Part 2) of Theorem 2.7 straightforwardly follows from the combination of (3.22) (resp., of (3.23)) and of Lemma 3.8.The proof of Theorem2.15 follows that of Theorem 2.7 (except that the second part of Lemma 3.4 stands instead of (3.21)); this proof is thus omitted.It remains to prove Lemmas3.7 and 3.8. The proof of Lemma 3.7 follows that of Lemma 3.6 and is omitted. We now prove Lemma 3.8. We first note that Rn=f̃n(θn)-f̃n(θ); a Taylor's expansion implies the existence of ζn such that ∥ζn-θn∥≤∥θn-θ∥ and (3.24)Rn=(θn-θ)T∇f̃n(ζn)=(θn-θ)T[∇f̃n(ζn)-∇f(ζn)+∇f(ζn)-∇f(θ)].
Let 𝒱 be a compact set that contains θ; for n large enough, we get (3.25)|Rn|=O(‖θn-θ‖[sup⁡x∈V‖∇f̃n(x)-∇f(x)‖+‖ζn-θ‖])=O(‖θn-θ‖sup⁡x∈V‖∇f̃n(x)-∇f(x)‖+‖θn-θ‖2).
To get an upper bound of |Rn|, we thus need to establish upper bounds of ∥θn-θ∥ and of supx∈𝒱∥∇f̃n(x)-∇f(x)∥.(i)
Let us recall that the a.s. convergence rate of(θn-θ) is given by that of [D2f(θ)]-1∇fn(θ) (see Lemma 3.5). Theorem 2.14 can be applied to obtain the exact a.s. convergence rate of θn-θ. However, to avoid assuming (A7)(i), we apply here Lemmas 3.1 and 3.2 (with |α|=1 and (gn,bn)=(fn,hn)) and get the following upper bound of the a.s. convergence rate of ∇fn(θ) and thus of θn-θ: for any γ>0 and ɛ∈]0,(1+a(d+2))/2[,
(3.26)‖θn-θ‖2=O((log⁡⁡n)1+γ∑i=1nhid+2+[∑i=1nhiqn]2)=O((log⁡⁡n)1+γnhnd+2+[hnq+nɛ-1]2)a.s.=O(w(hn,γ))a.s.(ii)
Noting that(3.27)sup⁡x∈V‖∇f̃n(x)-∇f(x)‖≤sup⁡x∈V‖∇f̃n(x)-E(∇f̃n(x))‖+sup⁡x∈V‖E(∇f̃n(x))-∇f(x)‖,
the application of Lemmas 3.1 and 3.2 with |α|=1, (gn,bn)=(f̃n,h̃n) ensures that, for any γ>0 and ɛ∈]0,(1+ã(d+2))/2[,(3.28)sup⁡x∈V‖∇f̃n(x)-∇f(x)‖2=O((log⁡⁡n)1+γnh̃nd+2+[∑i=1nh̃iqn]2)a.s.=O((log⁡⁡n)1+γnh̃nd+2+[h̃nq+nɛ-1]2)a.s.=O(w(h̃n,γ))a.s.
It follows from (3.25), (3.26), and (3.28) that (3.29)|Rn|2=O(w(hn,γ)w(h̃n,γ)+[w(hn,γ)]2)a.s.=O(Pn(γ))a.s.,
which concludes the proof of Lemma 3.8.


## 3.5. Joint Convergence Rate ofθn and μ̃n: Proof of Corollaries 2.10 and 2.17

In view of (3.14) and (3.20), we have (3.30)(Hn001)(θn-θμ̃n-μ)=(-∇fn(θ)f̃n(θ)-f(θ))+(0Rn),
and, in view of (3.16), the weak and strong asymptotic behaviour of (θn-θμ̃n-μ) is given by that of (3.31)([D2f(θ)]-1001)[(-∇fn(θ)f̃n(θ)-f(θ))+(0Rn)]=([-D2f(θ)]-1[∇fn(θ)-E(∇fn(θ))]f̃n(θ)-E(f̃n(θ)))+([-D2f(θ)]-1E(∇fn(θ))E(f̃n(θ))-f(θ)+Rn).
Under the assumptions of Corollary 2.10, the application of Part 1 of Lemma 3.6, that of Part 1 of Lemma 3.7, and that of Lemma 3.8 ensure that (3.32)lim⁡n→∞(nhnd+2[-D2f(θ)]-1E(∇fn(θ))nh̃nd[E(f̃n(θ))-f(θ)+Rn])=D(c,c̃)ABa,ã,q.
Corollary 2.10 then follows from the application of Part 2 of Lemma 3.3. The proof of Corollary 2.17 follows that of Corollary 2.10 (except that Part 3 of Lemma 3.4 is applied instead of Part 2 of Lemma 3.3) and is omitted.


## 3.6. Proof of Lemmas3.2–3.4

### 3.6.1. Proof of Lemma3.2

Setvn=[∑i=1nbid+2|α|]1/2[(logn)1+γ]-1/2. Applying Proposition 3 in Mokkadem et al. [27], it holds that for any δ>0, there exists c(δ)>0 such that (3.33)P[sup⁡x∈Uvn|∂[α]gn(x)-E(∂[α]gn(x))|≥δ]≤exp⁡⁡(-c(δ)∑i=1nbid+2|α|2vn2).
Since limn→∞∑i=1nbid+2|α|/(vn2logn)=∞, we have, for n large enough, c(δ)∑i=1nbid+2|α|/2vn2≥2logn, and Lemma 3.2 follows from the application of Borel-Cantelli's Lemma.


### 3.6.2. Proof of Lemma3.3

We only prove the second part of Lemma3.3: the proof of its first part can be easily deduced. We first establish that(3.34)lim⁡n→∞E(WnWnT)=Σa,ã
and then check that (Wn) satisfies Lyapounov's condition. Set (3.35)Yk,n=1nhn-d-2hk-d-1[∇K(θ-Xkhk)-E(∇K(θ-Xkhk))],Zk,n=1nh̃n-dh̃k-d[K(θ-Xkh̃k)-E(K(θ-Xkh̃k))],
and note that (3.36)E(WnWnT)=∑k=1n(E(Yk,nYk,nT)E(Yk,nZk,n)E(Yk,nTZk,n)E(Zk,n2)).
Now, for any s,t∈{1,…,d}, we have (3.37)E[∂K∂xs(θ-Xkhk)∂K∂xt(θ-Xkhk)]=∫Rd∂K∂xs(θ-yhk)∂K∂xt(θ-yhk)f(y)dy=hkdf(θ)Gs,t+o(hkd),
and since 𝔼[(∂K/∂xs)(θ-Xk/hk)]=O(hkd), we deduce that(3.38)E([∇K(θ-Xkhk)-E(∇K(θ-Xkhk))][∇K(θ-Xkhk)-E(∇K(θ-Xkhk))]T)=f(θ)Ghkd[1+o(1)],
which implies that limn→∞∑k=1n𝔼(Yk,nYk,nT)=f(θ)[1+a(d+2)]-1G. In the same way, we have(3.39)E([K(θ-Xkh̃k)-E(K(θ-Xkh̃k))]2)=h̃kdf(θ)∫RdK2(z)dz[1+o(1)],
and thus limn→∞∑k=1n𝔼(Zk,n2)=f(θ)[1+ãd]-1∫ℝdK2(z)dz. Moreover, set hn*=min(hn,h̃n); we have (3.40)E[∇K(θ-Xkhk)K(θ-Xkh̃k)]=hk*d∫Rd∇K(hk*hkz)K(hk*h̃kz)f(θ-hk*z)dz.
Noting that f(θ-hk*z)=f(θ)+hk*Rk(θ,z) with |Rk(θ,z)|≤∥∇f∥∞∥z∥, we get (3.41)E[∇K(θ-Xkhk)K(θ-Xkh̃k)]=hk*d[f(θ)∫Rd∇K(hk*hkz)K(hk*h̃kz)dz+hk*∫Rd∇K(hk*hkz)K(hk*h̃kz)Rk(θ,z)dz].
Since the function z↦[∇K(z)]K(z is odd (in each coordinate), the first right-handed integral is zero, and since hk* equals either hk or h̃k, we get (3.42)‖E[∇K(θ-Xkhk)K(θ-Xkh̃k)]‖≤hk*(d+1)‖∇f‖∞[‖K‖∞∫Rd‖z‖‖∇K(z)‖dz+‖∇K‖∞∫Rd‖z‖|K(z)|dz]=O(hk*(d+1)).
We then deduce that(3.43)E([∇K(θ-Xkhk)-E(∇K(θ-Xkhk))][K(θ-Xkh̃k)-E(K(θ-Xkh̃k))])=O([min⁡(hk,h̃k)]d+1)+O(hkdh̃kd)=O(hk(d+1)/2h̃k(d+1)/2),
and thus, in view of (3.1), (3.44)∑k=1nE(Yk,nZk,n)=O(1(nhn-d-2)(nh̃n-d)∑k=1nhk-(d+1)/2h̃k(1-d)/2)=o(1),
which concludes the proof of (3.34). Let us now check that (Wn) satisfies the Lyapounov's condition. Set p>2. Since K and ∇K are bounded and integrable, we have ∫ℝd∥∇K(z)∥pdz<∞ and ∫ℝd|K(z)|pdz<∞. It follows that (3.45)∑k=1nE(‖Yk,n‖p)=O(1(nhn-d-2)p/2∑k=1nhk(-d-1)p∫Rd‖∇K(θ-yhk)‖pf(y)dy)=O(1(nhn-d-2)p/2∑k=1nhk(-d-1)phkd)=o(1),∑k=1nE(|Zk,n|p)=O(1(nh̃n-d)p/2∑k=1nh̃k-dp∫Rd|K(θ-yh̃k)|pf(y)dy)=O(1(nh̃n-d)p/2∑k=1nh̃k-dph̃kd)=o(1),
which concludes the proof of Lemma 3.3.


### 3.6.3. Proof of Lemma3.4

We only prove the third part of Lemma3.4: the proofs of its first two parts can be easily deduced. Set (3.46)Γ=f(θ)(G00∫RdK2(z)dz),Δn=(1nhn-d-2Id001nh̃n-d),Qn=(hn-d-2Id00h̃n-d),
let (ɛn) be a sequence of ℝd+1-valued, independent, and 𝒩(0,Γ)-distributed random vectors, and set Sn=∑k=1nQkɛk and Bn=𝔼(SnSnT). We clearly have limn→+∞ΔnBnΔn=Σa,ã. Moreover, all the eigenvalues ρn(i)(1≤i≤d+1) of Bn satisfy limn→+∞loglogρn(i)/loglogn=1. The application of Theorem 1 in Mokkadem and Pelletier [30] then ensures that, with probability one, the sequence(3.47)(ΔnSn2log⁡⁡log⁡⁡n)
is relatively compact, and its limit set is the ellipsoid {ν∈ℝd+1such thatνTΣa,ã-1ν≤1}. Now, set (3.48)Ṽk=(hk-d/2[∇K(θ-Xkhk)-E(∇K(θ-Xkhk))]h̃k-d/2[K(θ-Xkh̃k)-E(K(θ-Xkh̃k))])
and Γk=𝔼(ṼkṼkT). In view of (3.38), (3.39), and (3.43), we have limk→∞Γk=Γ. It follows that ∃k0≥1 such that for allk≥k0, Γk is inversible; without loss of generality, we assume k0=1, and set Ũk=Γk-1/2Ṽk. Set p∈]2,4[ and let ℒ be a slowly varying function; we have (3.49)E(‖Ũk‖p)(klog⁡⁡log⁡⁡k)p/2=O(hk-dp/2E[‖∇K((θ-Xk)/hk)‖p]+h̃k-dp/2E[|K((θ-Xk)/hk)|p](klog⁡log⁡k)p/2)=O(hkd-dp/2+h̃kd-dp/2(klog⁡⁡log⁡⁡k)p/2)=O(L(k)[k-[1+((p/2)-1)(1-ad)]+k-[1+((p/2)-1)(1-ãd)]])
so that ∑k(kloglogk)-p/2𝔼(∥Ũk∥p)<∞. By application of Theorem 2 of Einmahl [31], we deduce that ∑k=1nŨk-∑k=1nηk=o(nloglogn) a.s., where ηk are independent and 𝒩(0,Id+1)-distributed random vectors. It follows that(3.50)∑k=1nΓ1/2Γk-1/2Ṽk-∑k=1nɛk=o(nlog⁡⁡log⁡⁡n)a.s.
Now, (3.51)Δn[∑k=1nQkΓ1/2Γk-1/2Ṽk-∑k=1nQkɛk]=Δn∑k=1nQk[Γ1/2Γk-1/2Ṽk-ɛk]=Δn∑k=1nQk(∑j=1k[Γ1/2Γj-1/2Ṽj-ɛj]-∑j=1k-1[Γ1/2Γj-1/2Ṽj-ɛj])(with∑j=10=0)=Δn∑k=1n-1(Qk-Qk+1)(∑j=1k(Γ1/2Γj-1/2Ṽj-ɛj))+ΔnQn∑j=1n(Γ1/2Γj-1/2Ṽj-ɛj)=Δn∑k=1n-1(Qk-Qk+1)[o(klog⁡⁡log⁡⁡k)]+ΔnQn[o(nlog⁡⁡log⁡⁡n)]a.s.
Moreover, (3.52)Δn∑k=1n-1(Qk-Qk+1)[o(klog⁡⁡log⁡⁡k)]=(hnd+2n∑k=1n-1(hk-(d+2)/2-hk+1-(d+2)/2)o(klog⁡⁡log⁡⁡k)00h̃ndn∑k=1n-1(h̃k-d/2-h̃k+1-d/2))=(o(hnd+2log⁡⁡log⁡⁡n)∑k=1n-1(hk-(d+2)/2-hk+1-(d+2)/2)00o(h̃ndlog⁡⁡log⁡⁡n)∑k=1n-1(h̃k-d/2-h̃k+1-d/2)).
Set ϕ(s)=[h(s)]-(d+2)/2 and ϕ̃(s)=[h̃(s)]-d/2, and let uk∈[k,k+1]; since ϕ′ and ϕ̃′ vary regularly with exponent (a(d+2)/2-1) and (ãd/2-1), respectively, we have (3.53)∑k=1n-1(hk-(d+2)/2-hk+1-(d+2)/2)=O(∑k=1n-1ϕ′(uk))=O(∫1nϕ′(s)ds)=O(hn-(d+2)/2),∑k=1n-1(h̃k-d/2-h̃k+1-d/2)=O(∑k=1n-1ϕ̃′(uk))=O(∫1nϕ̃′(s)ds)=O(h̃n-d/2)
so that (3.54)Δn∑k=1n-1(Qk-Qk+1)[o(klog⁡⁡log⁡⁡k)]=o(log⁡⁡log⁡⁡n).
Since ΔnQn[o(nloglogn)]=o(loglogn), we deduce that (3.55)Δn∑k=1nQkΓ1/2Γk-1/2Ṽk2log⁡⁡log⁡⁡n-Δn∑k=1nQkɛk2log⁡⁡log⁡⁡n=o(1)a.s.
The application of (3.47) then ensures that, with probability one, the sequence (3.56)(Δn∑k=1nQkΓ1/2Γk-1/2Ṽk2log⁡⁡log⁡⁡n)
is relatively compact, and its limit set is ℰ={ν∈ℝd+1such thatνTΣa,ã-1ν≤1}. Since (3.57)Δn∑k=1nQkṼk2log⁡⁡log⁡⁡n=Δn∑k=1nQkΓ1/2Γk-1/2Ṽk2log⁡⁡log⁡⁡n+Δn∑k=1nQk(Id+1-Γ1/2Γk-1/2)Ṽk2log⁡⁡log⁡⁡n
with limk→∞(Id+1-Γ1/2Γk-1/2)=0, Lemma 3.4 follows.


## 3.6.1. Proof of Lemma3.2

Setvn=[∑i=1nbid+2|α|]1/2[(logn)1+γ]-1/2. Applying Proposition 3 in Mokkadem et al. [27], it holds that for any δ>0, there exists c(δ)>0 such that (3.33)P[sup⁡x∈Uvn|∂[α]gn(x)-E(∂[α]gn(x))|≥δ]≤exp⁡⁡(-c(δ)∑i=1nbid+2|α|2vn2).
Since limn→∞∑i=1nbid+2|α|/(vn2logn)=∞, we have, for n large enough, c(δ)∑i=1nbid+2|α|/2vn2≥2logn, and Lemma 3.2 follows from the application of Borel-Cantelli's Lemma.


## 3.6.2. Proof of Lemma3.3

We only prove the second part of Lemma3.3: the proof of its first part can be easily deduced. We first establish that(3.34)lim⁡n→∞E(WnWnT)=Σa,ã
and then check that (Wn) satisfies Lyapounov's condition. Set (3.35)Yk,n=1nhn-d-2hk-d-1[∇K(θ-Xkhk)-E(∇K(θ-Xkhk))],Zk,n=1nh̃n-dh̃k-d[K(θ-Xkh̃k)-E(K(θ-Xkh̃k))],
and note that (3.36)E(WnWnT)=∑k=1n(E(Yk,nYk,nT)E(Yk,nZk,n)E(Yk,nTZk,n)E(Zk,n2)).
Now, for any s,t∈{1,…,d}, we have (3.37)E[∂K∂xs(θ-Xkhk)∂K∂xt(θ-Xkhk)]=∫Rd∂K∂xs(θ-yhk)∂K∂xt(θ-yhk)f(y)dy=hkdf(θ)Gs,t+o(hkd),
and since 𝔼[(∂K/∂xs)(θ-Xk/hk)]=O(hkd), we deduce that(3.38)E([∇K(θ-Xkhk)-E(∇K(θ-Xkhk))][∇K(θ-Xkhk)-E(∇K(θ-Xkhk))]T)=f(θ)Ghkd[1+o(1)],
which implies that limn→∞∑k=1n𝔼(Yk,nYk,nT)=f(θ)[1+a(d+2)]-1G. In the same way, we have(3.39)E([K(θ-Xkh̃k)-E(K(θ-Xkh̃k))]2)=h̃kdf(θ)∫RdK2(z)dz[1+o(1)],
and thus limn→∞∑k=1n𝔼(Zk,n2)=f(θ)[1+ãd]-1∫ℝdK2(z)dz. Moreover, set hn*=min(hn,h̃n); we have (3.40)E[∇K(θ-Xkhk)K(θ-Xkh̃k)]=hk*d∫Rd∇K(hk*hkz)K(hk*h̃kz)f(θ-hk*z)dz.
Noting that f(θ-hk*z)=f(θ)+hk*Rk(θ,z) with |Rk(θ,z)|≤∥∇f∥∞∥z∥, we get (3.41)E[∇K(θ-Xkhk)K(θ-Xkh̃k)]=hk*d[f(θ)∫Rd∇K(hk*hkz)K(hk*h̃kz)dz+hk*∫Rd∇K(hk*hkz)K(hk*h̃kz)Rk(θ,z)dz].
Since the function z↦[∇K(z)]K(z is odd (in each coordinate), the first right-handed integral is zero, and since hk* equals either hk or h̃k, we get (3.42)‖E[∇K(θ-Xkhk)K(θ-Xkh̃k)]‖≤hk*(d+1)‖∇f‖∞[‖K‖∞∫Rd‖z‖‖∇K(z)‖dz+‖∇K‖∞∫Rd‖z‖|K(z)|dz]=O(hk*(d+1)).
We then deduce that(3.43)E([∇K(θ-Xkhk)-E(∇K(θ-Xkhk))][K(θ-Xkh̃k)-E(K(θ-Xkh̃k))])=O([min⁡(hk,h̃k)]d+1)+O(hkdh̃kd)=O(hk(d+1)/2h̃k(d+1)/2),
and thus, in view of (3.1), (3.44)∑k=1nE(Yk,nZk,n)=O(1(nhn-d-2)(nh̃n-d)∑k=1nhk-(d+1)/2h̃k(1-d)/2)=o(1),
which concludes the proof of (3.34). Let us now check that (Wn) satisfies the Lyapounov's condition. Set p>2. Since K and ∇K are bounded and integrable, we have ∫ℝd∥∇K(z)∥pdz<∞ and ∫ℝd|K(z)|pdz<∞. It follows that (3.45)∑k=1nE(‖Yk,n‖p)=O(1(nhn-d-2)p/2∑k=1nhk(-d-1)p∫Rd‖∇K(θ-yhk)‖pf(y)dy)=O(1(nhn-d-2)p/2∑k=1nhk(-d-1)phkd)=o(1),∑k=1nE(|Zk,n|p)=O(1(nh̃n-d)p/2∑k=1nh̃k-dp∫Rd|K(θ-yh̃k)|pf(y)dy)=O(1(nh̃n-d)p/2∑k=1nh̃k-dph̃kd)=o(1),
which concludes the proof of Lemma 3.3.


## 3.6.3. Proof of Lemma3.4

We only prove the third part of Lemma3.4: the proofs of its first two parts can be easily deduced. Set (3.46)Γ=f(θ)(G00∫RdK2(z)dz),Δn=(1nhn-d-2Id001nh̃n-d),Qn=(hn-d-2Id00h̃n-d),
let (ɛn) be a sequence of ℝd+1-valued, independent, and 𝒩(0,Γ)-distributed random vectors, and set Sn=∑k=1nQkɛk and Bn=𝔼(SnSnT). We clearly have limn→+∞ΔnBnΔn=Σa,ã. Moreover, all the eigenvalues ρn(i)(1≤i≤d+1) of Bn satisfy limn→+∞loglogρn(i)/loglogn=1. The application of Theorem 1 in Mokkadem and Pelletier [30] then ensures that, with probability one, the sequence(3.47)(ΔnSn2log⁡⁡log⁡⁡n)
is relatively compact, and its limit set is the ellipsoid {ν∈ℝd+1such thatνTΣa,ã-1ν≤1}. Now, set (3.48)Ṽk=(hk-d/2[∇K(θ-Xkhk)-E(∇K(θ-Xkhk))]h̃k-d/2[K(θ-Xkh̃k)-E(K(θ-Xkh̃k))])
and Γk=𝔼(ṼkṼkT). In view of (3.38), (3.39), and (3.43), we have limk→∞Γk=Γ. It follows that ∃k0≥1 such that for allk≥k0, Γk is inversible; without loss of generality, we assume k0=1, and set Ũk=Γk-1/2Ṽk. Set p∈]2,4[ and let ℒ be a slowly varying function; we have (3.49)E(‖Ũk‖p)(klog⁡⁡log⁡⁡k)p/2=O(hk-dp/2E[‖∇K((θ-Xk)/hk)‖p]+h̃k-dp/2E[|K((θ-Xk)/hk)|p](klog⁡log⁡k)p/2)=O(hkd-dp/2+h̃kd-dp/2(klog⁡⁡log⁡⁡k)p/2)=O(L(k)[k-[1+((p/2)-1)(1-ad)]+k-[1+((p/2)-1)(1-ãd)]])
so that ∑k(kloglogk)-p/2𝔼(∥Ũk∥p)<∞. By application of Theorem 2 of Einmahl [31], we deduce that ∑k=1nŨk-∑k=1nηk=o(nloglogn) a.s., where ηk are independent and 𝒩(0,Id+1)-distributed random vectors. It follows that(3.50)∑k=1nΓ1/2Γk-1/2Ṽk-∑k=1nɛk=o(nlog⁡⁡log⁡⁡n)a.s.
Now, (3.51)Δn[∑k=1nQkΓ1/2Γk-1/2Ṽk-∑k=1nQkɛk]=Δn∑k=1nQk[Γ1/2Γk-1/2Ṽk-ɛk]=Δn∑k=1nQk(∑j=1k[Γ1/2Γj-1/2Ṽj-ɛj]-∑j=1k-1[Γ1/2Γj-1/2Ṽj-ɛj])(with∑j=10=0)=Δn∑k=1n-1(Qk-Qk+1)(∑j=1k(Γ1/2Γj-1/2Ṽj-ɛj))+ΔnQn∑j=1n(Γ1/2Γj-1/2Ṽj-ɛj)=Δn∑k=1n-1(Qk-Qk+1)[o(klog⁡⁡log⁡⁡k)]+ΔnQn[o(nlog⁡⁡log⁡⁡n)]a.s.
Moreover, (3.52)Δn∑k=1n-1(Qk-Qk+1)[o(klog⁡⁡log⁡⁡k)]=(hnd+2n∑k=1n-1(hk-(d+2)/2-hk+1-(d+2)/2)o(klog⁡⁡log⁡⁡k)00h̃ndn∑k=1n-1(h̃k-d/2-h̃k+1-d/2))=(o(hnd+2log⁡⁡log⁡⁡n)∑k=1n-1(hk-(d+2)/2-hk+1-(d+2)/2)00o(h̃ndlog⁡⁡log⁡⁡n)∑k=1n-1(h̃k-d/2-h̃k+1-d/2)).
Set ϕ(s)=[h(s)]-(d+2)/2 and ϕ̃(s)=[h̃(s)]-d/2, and let uk∈[k,k+1]; since ϕ′ and ϕ̃′ vary regularly with exponent (a(d+2)/2-1) and (ãd/2-1), respectively, we have (3.53)∑k=1n-1(hk-(d+2)/2-hk+1-(d+2)/2)=O(∑k=1n-1ϕ′(uk))=O(∫1nϕ′(s)ds)=O(hn-(d+2)/2),∑k=1n-1(h̃k-d/2-h̃k+1-d/2)=O(∑k=1n-1ϕ̃′(uk))=O(∫1nϕ̃′(s)ds)=O(h̃n-d/2)
so that (3.54)Δn∑k=1n-1(Qk-Qk+1)[o(klog⁡⁡log⁡⁡k)]=o(log⁡⁡log⁡⁡n).
Since ΔnQn[o(nloglogn)]=o(loglogn), we deduce that (3.55)Δn∑k=1nQkΓ1/2Γk-1/2Ṽk2log⁡⁡log⁡⁡n-Δn∑k=1nQkɛk2log⁡⁡log⁡⁡n=o(1)a.s.
The application of (3.47) then ensures that, with probability one, the sequence (3.56)(Δn∑k=1nQkΓ1/2Γk-1/2Ṽk2log⁡⁡log⁡⁡n)
is relatively compact, and its limit set is ℰ={ν∈ℝd+1such thatνTΣa,ã-1ν≤1}. Since (3.57)Δn∑k=1nQkṼk2log⁡⁡log⁡⁡n=Δn∑k=1nQkΓ1/2Γk-1/2Ṽk2log⁡⁡log⁡⁡n+Δn∑k=1nQk(Id+1-Γ1/2Γk-1/2)Ṽk2log⁡⁡log⁡⁡n
with limk→∞(Id+1-Γ1/2Γk-1/2)=0, Lemma 3.4 follows.



---

*Source: 564297-2011-10-26.xml*