# Asymptotic Normality of the Robust Equivariant Estimator for Functional Nonparametric Models

Authors: Ibrahim M. Almanjahie; Omar Fetitah; Mohammed Kadi Attouch; Hayat Louhab
Journal: Mathematical Problems in Engineering
 (2022)

---

## Abstract

As in parametric regression, nonparametric kernel regression is essential for examining the relationship between response variables and covariates. In both methods, outliers may affect the estimators, and hence robustness is essential to deal with practical issues. This paper proposes a family of robust nonparametric estimators with unknown scale parameters for regression function based on the kernel method. In addition, we establish the asymptotic normality of the estimator under the concentration properties on small balls of probability measure of the functional explanatory variables. The superiority of the proposed methods is shown through numerical and real data studies to compare the sensitivity to outliers between the classical and robust regression (fixed and unknown scale parameter). Such a new proposed method will be useful in the future for analyzing data and making decisions.


---

## Body

## 1. Introduction

Nonparametric kernel regression is a familiar tool to explore the underlying relationship between response variables and covariates. In the functional data studies, these estimators are given by Ferraty and Vieu [1] and Ferraty and Vieu [2]. As in parametric regression estimation, the kernel estimator may be affected by outliers; hence, considering robustness is essential.In this context, letXi,Yii=1,…,n be a sequence of strictly stationary dependent random variables identically distributed as (X, Y), a random pair valued in ℱ×ℝ, where ℱ is a semi-metric space. Let d denote the semi-metric. The purpose of this work is to study the nonparametric estimation of the robust regression θx when the scale parameter is unknown. In fact, for any x∈ℱ, θx is defined as zero with respect to the parameter a of the following equation:(1)Ψx,a,σ=ψxY−aσ|X=x=0,where ψx is a real-valued function that satisfies some regularity conditions, to be stated later, and σ is a robust measure of conditional scale. In what follows, we assume, for all x∈ℱ, that the robust regression θx exists and is unique (see, for instance, Boente and Fraiman [3]).We point out that the robustification method is an old subject in statistics. Among many papers dealing with the robust nonparametric estimates of regression function in finite dimension case, one can refer, for example, to key works of Robinson (1984), and Collomb and Härdle [4]. We refer to Laïb and Ould Saïd [5] for some results on multivariate time series (mixing and ergodicity conditions), and Roozbeh [6] and Roozbeh et al. [6–8] for multicollinearity in the least trimmed squares regression and outliers effects.The robust regression is widely studied in nonparametric functional statistics. Indeed, it was firstly introduced by Azzedine et al. [9]; they proved the almost complete convergence of this model in the independent and identically distributed (i.i.d.) case. The asymptotic normality of their model has been established later by Attouch et al. [10–12] in both dependent and independent cases and unknown scale parameters. However, all these results are obtained in the complete data. In the incomplete data cases, we can refer to Derrar et al. (2020) and the references therein.The main objective of this paper is to generalize the results of Boente and Vahnovanb [13]. Precisely, we prove the asymptotic normality of the constructed estimator by combining the ideas of robustness with those of unknown scale parameters. This result is obtained under standard conditions allowing us to explore the different structural axes of the subject, such as the robustness of the regression function and the correlation of the observation. We emphasize that, contrary to the usual case where the scale parameter is fixed, it must be estimated here, which makes it more difficult to establish the asymptotic properties of the estimator. However, although this difference is more important in the context of this work, we have been able to overcome it.The paper is organized as follows. Section2 is dedicated to the presentation of the robust estimator with unknown scale parameters. The needed assumptions and notations are given in Section 3. We state our main results in Section 4, and the proofs of the main results are relegated to the appendix. Sections 5 and 6 are devoted to the simulation and real data applications of our proposed methods. The conclusion is stated in Section 7.


## 2. The Robust Equivariant Estimators and Their Related Functional

Let us consider a functional stationary ergodic processZi=Xi,Yii=1,…,n (see Laïb and Louani [14] for some definitions and examples). When the scale parameter is unknown, the robust estimator may be constructed based on two steps. Firstly, we estimate the scale parameter σ by the conditional median of the absolute deviation from the conditional median, that is,(2)σ≔sx=MEDY−mx|X=x=MADF⋅|X=x,where F⋅|X=x=ℙY≤.|X=x is the conditional distribution of Y given that X=x and mx=MEDY|X=x=infy∈ℝ:Fy|X=x≥1/2 is the median of the conditional distribution. Then, for x∈ℱ, the kernel estimator sˆx=MADFˆ⋅|X=x, where Fˆy|X=x is given by(3)Fˆy|X=x=∑i=1nKh−1dx,XiI−∞,yYi∑i=1nKh−1dx,Xi,where K is a kernel function and h=hn is a sequence of positive numbers which goes to zero as n goes to infinity. Next, the kernel estimator θˆx, of the robust regression θx, is the zero, with respect to a, of the equation(4)Ψˆx,a,sˆx=0,where(5)Ψˆx,a,sˆx=∑i=1nKh−1dx,XiψxYi−a/sˆx∑i=1nKh−1dx,Xi.


## 3. Notation, Conditions, and Comments

Throughout the paper, when no confusion is possible, we will denote byCi for i∈1,2,3,4 some strictly positive generic constants. x stands for a fixed point in ℱ, Nx denotes a fixed neighborhood of x, and we set λγu,a,σ=EψxγY−a/σ|X=u and Γγu,a,σ=Eψx′γY−a/σ|X=u, for γ∈1,2. We state the following conditions:A1. The functionψx is a continuously differentiable function, being strictly monotone and bounded w.r.t. the second component, and its derivative ∂ψxy−a/σx/∂a is bounded and continuous at θx uniformly in y.A2. There exists a nonnegative differentiable functionϕ and a nonnegative function g such that ℙX∈Bx,r=ϕr.gx+oϕr where Bx,r=x′∈ℱ/dx,x′<r.A3. The functionλγ.,.,. satisfies a Lipschitz condition of order one; that is, there exists a strictly positive constant bγ such that(6)∀u1,u2∈Nx×Nx,∀a∈ℝ,λγu1,a,σ−λγu2,a,σ≤C3du1,u2bγ.A4. The functionΓγ.,.,σ satisfies a Lipschitz condition of order one; that is, there exists a strictly positive constant dγ such that ∀u1,u2∈Nx×Nx,∀a∈ℝ,Γγu1,a,σ−Γγu2,a,σ≤C4du1,u2dγ.A5. The functionsfxx and px are bounded functions on C such that Ap=infx∈Cpx>0 and Af=infx∈CfXx>0. Moreover, px is a continuous function in a neighborhood of C.A6. (i)
Fy|X=x is a continuous function of x in a neighborhood of C. Furthermore, it satisfies the following equicontinuity condition:(7)∀ε>0,∃δ>0:u−v<δ⟹supx∈CFu|X=x−Fv|X=x<ε.(ii)
Fy|X=x is symmetric around θx and a continuous function of y for each fixed x.A7. The kernelK is a differentiable function supported on 0,1. Its derivative K′ exists and is such that there exist two constants C1 and C2 with −∞<C1<K′t<C2<0 for 0≤t≤1.A8. The functionfX is a continuous function in a neighborhood of C.A9. The bandwidthh satisfiesh↓0,∀t∈0,1,limh⟶0ϕth/ϕh=βt and nϕh⟶∞ as n⟶∞.A10. The sequencek=kn is such that kn/n⟶0,kn⟶∞, and kn/logn⟶∞.Comments on conditions A1-A10:(1)
A1 keeps the same conditions on the function ψx given by Collomb and Härdle (1986) and Boente and Rodriguez [15] in the multivariate case. We point out that the boundness required to the score function ψx can be dropped by using the classical truncated method (see Laïb and Ould Saïd [5] for more details).(2)
A2 keeps the same conditions given by Attouch et al. [16].(3)
ConditionsA3-A4 are regularity conditions that characterize the functional space of our model and are needed to evaluate the bias term in our asymptotic properties.(4)
ConditionsA5-A6 state regularity conditions on the marginal density of X and on the conditional distribution function which imply that, for any compact set C,0<infx∈Csx≤supx∈Csx<∞ and that θx is a continuous function of x.(5)
A7–A10 are technical conditions imposed for the brevity of proofs. The function β. defined in A9 intervenes in all asymptotic behavior, in particular in the asymptotic variance term. With a simple algebra, it is possible to specify this function in the above examples by(8)iβu=uγ,iiβu=I0,1u,iiiβu=δ1u,whereδ1. is Dirac function.


## 4. Main Results

The following result in Proposition1 ensures uniform consistency of the regression function θx, when the smoothing is based on either local medians or local M-smoothers, while Theorem 1 deals with the asymptotic normality of the proposed estimator. The proofs of these asymptotic results are postponed to the appendix.Proposition 1.
Assume thatA5toA7hold. Moreover, assume thatA9holds for kernel weights and thatA8andA10hold for nearest neighbor with kernel weights. Then, for any compact setC,(a)
underA1andA6ii, we have thatsupx∈Cθˆx−θx⟶a.s.0;(b)
if, in addition,Fy|X=xhave a unique median atθx,we have that(9)supx∈CθˆMEDx−θx⟶a.s.0.We will now setψσt=σψxt/σ.Lemma 1.
UnderA1andA6(ii), if there exists a real constantC>0and an increasing functionϕhsuch thatnϕh−1E∑i=1nWni2x,X1,...,Xn≤C,withWnix=Ki/∑i=1nKi,then, we have that(10)nϕh1/2∑i=1nWnixψσnxYi−θx−ψσYi−θx⟶P0,for any sequenceσn=σnxsuch thatσnx⟶σx=σ>0in probability.Theorem 1.
UnderA1toA4,A7,andA9, if in additionθˆx⟶Pθxandsˆx⟶Pσx,for anyx∈A,then, we have(11)nϕhσ2x,θx1/2θˆx−θx−Bnx⟶DN0,1asn⟶∞,where(12)Bnx=hϕhα1Γ1x,θx,σ∫01Ktφσ,xthϕ′thdt+o1,σ2x,θx=α2λ2x,θx,σα12gxΓ1x,θx,σ2,with φσ,xt=Eψx′Y−θx/σ|dX,x=t,αj=−∫01Kj′zβzdzforj=1,2.(13)A=z∈ℱ,gzλ2z,θz,σΓ1z,θz,σ≠0,and⟶Dmeans the convergence in distribution.


### 4.1. Application to Conditional Confidence Interval

An important application of the asymptotic normality result is the building of confidence intervals for the true value ofθx given that curve X=x. A plug-in estimate for the asymptotic standard deviation σx,θx can be obtained using the estimators λˆ2x,θˆx,sˆx and Γˆ1x,θˆx,sˆx of λ2x,θˆx,sˆx and Γ1x,θˆx,sˆx, respectively. We get σˆnx,θˆx=βˆ2λˆ2x,θˆx,sˆx/βˆ12gxΓˆ12x,θˆx,sˆx1/2.Then,σˆnx,θˆx can be used to get the following approximate 1−ζ confidence interval for θx:(14)θˆx±t1−ζ/2×σˆn2x,θˆxnϕx,h1/2,where t1−ζ/2 denotes the 1−ζ/2 quantile of the standard normal distribution. Here, we point out that the estimators λ2x,θˆx,sˆx and Γ1x,θˆx,sˆx can be estimated by(15)λˆ2x,θˆx,sˆx=∑i=1nKh−1dx,Xiψx2Yi−θˆx/sˆx∑i=1nKh−1dx,Xi,Γˆ1x,θˆx,sˆx=∑i=1nKh−1dx,Xi∂/∂tψxYi−θˆx/sˆx∑i=1nKh−1dx,Xi.We estimateβ1 and β2 by βˆ1=1/nϕx,hgx∑i=1nKh−1dx,Xi and βˆ2=1/nϕx,hgx∑i=1nK2h−1dx,Xi.This last estimation is justified by the fact that, underA2, A7, and A9, we have (see Ferraty and Vieu (2002, p. 44))(16)1ϕhgxEK1j⟶αj,j=1,2.Then, by the asymptotic normality result in Section4, we haveΛnx=nϕx,h/σ2x,θx1/2θˆx−θx⟶DN0,1 as n⟶∞.


## 4.1. Application to Conditional Confidence Interval

An important application of the asymptotic normality result is the building of confidence intervals for the true value ofθx given that curve X=x. A plug-in estimate for the asymptotic standard deviation σx,θx can be obtained using the estimators λˆ2x,θˆx,sˆx and Γˆ1x,θˆx,sˆx of λ2x,θˆx,sˆx and Γ1x,θˆx,sˆx, respectively. We get σˆnx,θˆx=βˆ2λˆ2x,θˆx,sˆx/βˆ12gxΓˆ12x,θˆx,sˆx1/2.Then,σˆnx,θˆx can be used to get the following approximate 1−ζ confidence interval for θx:(14)θˆx±t1−ζ/2×σˆn2x,θˆxnϕx,h1/2,where t1−ζ/2 denotes the 1−ζ/2 quantile of the standard normal distribution. Here, we point out that the estimators λ2x,θˆx,sˆx and Γ1x,θˆx,sˆx can be estimated by(15)λˆ2x,θˆx,sˆx=∑i=1nKh−1dx,Xiψx2Yi−θˆx/sˆx∑i=1nKh−1dx,Xi,Γˆ1x,θˆx,sˆx=∑i=1nKh−1dx,Xi∂/∂tψxYi−θˆx/sˆx∑i=1nKh−1dx,Xi.We estimateβ1 and β2 by βˆ1=1/nϕx,hgx∑i=1nKh−1dx,Xi and βˆ2=1/nϕx,hgx∑i=1nK2h−1dx,Xi.This last estimation is justified by the fact that, underA2, A7, and A9, we have (see Ferraty and Vieu (2002, p. 44))(16)1ϕhgxEK1j⟶αj,j=1,2.Then, by the asymptotic normality result in Section4, we haveΛnx=nϕx,h/σ2x,θx1/2θˆx−θx⟶DN0,1 as n⟶∞.


## 5. Simulation Study

The purpose of this section is to apply our method to an example of simulated data. More precisely, our main aim is to compare the sensitivity to the outliers of the classical kernel regression estimator (CKE:rˆx=∑i=1nYiKh−1dx,Xi/∑i=1nKh−1dx,Xi; see Ferraty and Vieu [17]; the kernel robust estimator (KRE: θ˜x=argminy∑i=1nρYi−yKh−1dx,Xi/∑i=1nKh−1dx,Xi; see Azzedine et al. [9]); and the equivariant robust regression estimator see Boente and Vahnovanb [13] (ERE: θˆx=argminy∑i=1nρYi−y/s^xKh−1dx,Xi/∑i=1nKh−1dx,Xi associated with ψt=ρ′t=t/1+t2/2; see Boente and Vahnovanb [13]). This score function is known in the literature as L1−′s function. In this example, we consider the functional variable in the interval 0,1, and we take(17)Xt=AcosπtW+1−A2−cosπtW,where W∼N0,1 and A is Bernoulli random variable distributed. We carried out the simulation with 200 samples of the curve X which is represented in Figure 1.Figure 1
The curvesXi=1,…,200t,t∈0,1.The scalar response variable is defined byY=rX+ϵ, where r is the nonlinear regression model, and rX=2.5×∫01X′tdt2, where ϵ is N0,0.5. The selection of the bandwidth h:=hK is an important and basic problem in all kernel smoothing techniques. In this simulation, for both methods, the optimal bandwidths were chosen by the cross-validation method on the k nearest neighbors, where hkopt is the bandwidth corresponding to the optimal number of neighbors obtained by a cross-validation procedure:(18)hk=minh∈ℝ+such that∑i=1nIBx,hXi=k,with(19)kopt=argminkCVkwhereCVk=∑i=1nYi−Y˜−iXi2,where Yˆ−i is the leave-one-out-curve estimator of the CKE, KRE, and ERE (we refer the reader to Ferraty and Vieu [2] for more details). We choose the quadratic kernel:(20)Kx=321−x2I0,1.Another essential point for ensuring a good behavior of the method is using a semi-metric that is well adapted to the kind of data that we have to deal with. Here, we used the semi-metric defined by theL2-distance between the first derivatives of the curves dderiv.,. (for further discussion, see Ferraty and Vieu [2]).(21)dderivxi,xj=∫01xi′t−xj′t2dt,∀xi,xj∈ℱ.This choice is motivated by the regularity of the curvesX. To compare these two methods, we split the data randomly into two subsets: training sample Xi,Yii∈I and test sample Xj,Yjj∈J for measuring the forecasting accuracy of the responses.To verify the theoretical results, it is possible to visualize the data histogram and then compare its shape to the normal density for a fixedXj0∈J (we carry out 100 independent replications).The histogram ofΛnXj0 is almost symmetric around zero point and well-shaped like the standard normal density. Thus, the simulation results indicate that ΛnXj0 obeys the standard normal law when n is large (see Figure 2).Figure 2
Histograms and density curves.In order to construct confidence bands, we proceed using the following algorithm:Step 1.
For eachXj in the test sample, we calculate the estimator θˆXj using the training sample.Step 2.
For allj∈J, we define the confidence bands by(22)θˆXj−z0.975×σˆn2Xj,θˆXjnϕXj,h1/2,θˆXj+z0.975×σˆn2Xj,θˆXjnϕXj,h1/2,where z0.975 is the 2.5% quantile of a standard normal distribution.Step 3.
We present our results by plotting the extremities of the predicted values versus the true values and the confidence bands.
The results are presented in Figures3 and 4, where the solid black curve connects the true values and the dashed blue curve connects the lower and upper predicted confidence interval (IC). Clearly, Figure 3 shows the excellent behavior of our functional forecasting procedure for the ERE method in the absence of outliers.
The performance of the estimators is described by the mean squared error (MSE):(23)MSE=1#J∑j∈JYj−YˆXj2,where #J is the length of testing sample J, and YˆXj means the regression estimators of CKE, KRE, and ERE calculated at the point Xj. The main feature of our approach is illustrated in the second case when we perturb the data by introducing some outliers as indicated below. In this part, following the same approach in Sinova et al. [18], let ϱi=1n be independent sequences of random variables as in Bernoulli (with parameter ϵϱ∈5%;10%) and contamination size constant M be either 5 or 25.
The data generating model is given by(24)Yi∗=Yi+Mϱi1≤i≤n.
In all these cases, we arrived at the same conclusion. Usually in the presence of outliers, the ERE regression shows better behavior than that of the CKE and KRE methods. Even if the MSE of the three methods increases substantially with theϵϱ and with the value of contamination size constant M, it remains very low for the ERE method. The results are given in Table 1 and Figure 4 where we only present the case of ϵϱ=10%andM=25, because the results are very similar to the other cases.Figure 3
Extremities of the predicted values versus the true values and the confidence bands (simulation data without outliers). The black curve connects the true values. The blue curve connects the crossed points which give the predicted IC.Figure 4
Extremities of the predicted values versus the true values and the confidence bands (simulation data with outliers). The black curve connects the true values. The blue curve connects the crossed points, which give the predicted IC.Table 1
Comparison between the three methods in the presence of outliers.
Contamination size constantMParameterϵϱMSE (ERE)MSE (KRE)MSE (CKE)50.051.38920.95010.810850.102.19591.79960.8685250.053.87011.84440.8912250.109.42022.34720.9269


## 6. Real Data Analysis

We focus now on the comparison of our estimator ERE (θˆx) with CKE (rˆx) and KRE (θ˜x) estimators for some spectroscopic datasets. For instance, let us consider a sample of 108 soil samples measured by near-infrared reflectance (NIR). Each sample is illuminated by a light beam at 1050 equally spaced wavelengths t1,…,t1050 in the range of 400–2500 nanometres (nm) with a 2 nm resolution. The dataset is available at http://www.models.kvl.dk/NIRsoil. For each wavelength t and each soil sample i, the absorption Xit of radiation is measured. The ith discretized spectrometric curve is given by Xit1,…,Xit1050; Figure 5 displays the 108 NIR reflectance spectra curves, where the red ones correspond to the outlier curves.Figure 5
Soil curves: 108 near-infrared spectra sampled at 1050 equally spaced wavelengths.Moreover, to determine the chemical and microbiological properties of soil, ergosterol concentration was determined through High-Performance Liquid Chromatography (HPLC), which is taken in the following as a response variableY. The latter variable is affected by the presence of some outliers as shown in Figure 6.Figure 6
The 108 response variableY (ergosterol concentration), where the red ones correspond to the outlier from the Y.In this application, we use the MAD-median rule (cf. Wilcox [19] for detecting outliers). Recall that this method will refer to declaring Yi an outlier if(25)Yi−MMAD∗0.6745>C,where M is the sample median and MAD is the median absolute deviation given by(26)MAD=medianY1−M,Y2−M,…,Yn−M.Moreover,C is taken to be χ2.975: the square root of the quantile of a chi-squared distribution with one degree of freedom. We applied this method to our sample data and the result shows that the MAD-median method identifies 21 outliers. The goal now is to predict the ergosterol concentration from the observation of a new spectrometric curve. To this end, one observes n=108 pairs Xi,Yii=1,…,n where Xi (resp., Yi ) is the ith spectrometric curve (resp., response). The prediction problem is very simple and can be formulated via the model Yi=rXi+εi for i=1,…,n. Then, we split the observations Xi,Yii=1,…,108 into two parts: one is the training sample containing Xi,Yii=1,…,86 for modeling, and the other is Xj,Yjj=87,…,108 which is a testing sample for measuring the forecasting accuracy of the responses. In the modeling procedures, we choose the asymmetrical quadratic kernel function Ku=3/412/11−u2I0,1u and the semi-metric dderiv.,. of the first derivative. As in all smoothing methods, the choice of the smoothing parameter has a crucial role in the computational issues. In this illustration, we use the cross-validation procedure described in Rachdi and Vieu (2007) for which the bandwidth h is chosen via the following rule:(27)hopt=argminhCVh=argminh∑i=1nYi−Yˆ−iXi,where Yˆ−i is the leave-one-out-curve estimator of the CKE, KRE, and ERE. Finally, we predict the Yjj=87,…,108 by the three methods above and check the performance of both models by computing the mean squared prediction error (MSE), and the relative mean squared prediction error (RMSE):(28)RMSE=122∑j∈the testing sampleYj−YˆXjYj2,where YˆXj means the regression estimators of CKE, KRE, and ERE calculated at the point Xj. Then, we display in Table 2 and Figure 7 these errors where it appears clearly that the ERE is quite more performant than the KRE and CKE. That is, the classical kernel method is susceptible to the presence of outliers.Table 2
MSE and RMSE of the CKE, KRE, and ERE.
CKEKREEREMean squared error (MSE)6.8987816.2169574.807382Relative mean squared error (RMSE)0.0009080.0008190.000640Figure 7
Box plots of the MSE (a) and RMSE (b) of the CKE, KRE, and ERE.
(a)(b)Next, the obtained prediction results are shown in Figure8, while we give in Figure 9 the 95% predictive intervals of the concentrations for the ergosterol concentration in the sample test. Figure 9 also shows clearly a good behavior of the ERE estimator compared to the KRE and KRE ones. It appears in the band of confidence intervals which is very large for the CKE (0 to 200) and KRE (60 to 120) as compared to the ERE (80 to 100). This conclusion shows the good performance of our asymptotic normality.Figure 8
Prediction of the last 15 testing samples. The solid black curve connects the true values.Figure 9
Extremities of the predicted values versus the true values and the confidence bands of the last 15 testing samples. The solid black curve connects the true values. The dashed blue curve connects the lower and upper predicted values.


## 7. Conclusion

This work generalizes the results of Boente et al. [13] in real response random variables to the functional response random variables. Our results are applied to derive asymptotic normality of the predictor estimate and build conditional confidence intervals. The results were obtained under sufficient standard conditions that allow one to explore different structural axes of the subject, such as the functional naturalness of the model and the data as well as the robustness of the regression function.



---

*Source: 8989037-2022-05-10.xml*