# Partial Derivative Estimation for Underlying Functional-Valued Process in a Unified Framework

Authors: Yunbei Ma; Fanyin Zhou; Xuan Luo
Journal: Journal of Applied Mathematics
 (2020)

---

## Abstract

We consider functional data analysis when the observations at each location are functional rather than scalar. When the dynamic of underlying functional-valued process at each location is of interest, it is desirable to recover partial derivatives of a sample function, especially from sparse and noise-contaminated measures. We propose a novel approach based on estimating derivatives of eigenfunctions of marginal kernels to obtain a representation for functional-valued process and its partial derivatives in a unified framework in which the number of locations and number of observations at each location for each individual can be any rate relative to the sample size. We derive almost sure rates of convergence for the procedures and further establish consistency results for recovered partial derivatives.


---

## Body

## 1. Introduction

With the rapid advance in computational and analytical technology, many time-dynamic processes are monitored and recoded continuously during a time interval or intermittently at several discrete time points. Functional data analysis (FDA) is a powerful tool to deal with the analysis and theory of data that are in the form of functions, images and shapes, or more general objects. Traditional functional data typically consist of a random sample of independent real-valued functions, which can be viewed as the realization of a one-dimensional stochastic process. In this field of research, a general introduction of the available methods can be found in Ramsay and Silverman [1] and Wang et al. [2].Many recent developments in FDA concern multivariate functional data and spatially indexed functional data. Chen and Müller [3] introduced a methodology for repeatedly observed and thus dependent functional data, covering the case where the recordings of the curves are scheduled on a regular and dense grid with often sparse and random recording times.We consider special situations where the observations at each location are functional rather than scalar. ForS⊂ℝp1 and T⊂ℝp2, we consider the stochastic process X:T⟶L2S and denote its value at time t∈T by X⋅,t, which is a square integrable random function with argument s∈S. Chen et al. [4] proposed marginal FPCA and product FPCA models for Xs,t and developed estimating methods and theoretical results under designs that are dense and regular in s. In practice, we may deal with functional data which are dense and random at the s direction. Under these cases, a presmoothing of individual curve at each location is necessary. However, in practice, it is possible that we are faced with sparse and random designs in s. Moreover, it is also possible that curves at some locations are densely observed, while curves at other locations are sparsely observed. In this paper, we aim to recover X(s,t) by estimating the multivariate mean function, the marginal covariance function, and then the FPCA in a unified framework. This unified framework allows the number of locations and the number of observations at each location for each individual to be any rate relative to the sample size. Thus, the proposed procedure avoids a challenging issue of classifying which scenario we are faced with and hence deciding which methodology to use when dealing with real data.On the other hand, it is often of interest to recover derivatives of a sample of random functions, especially when the dynamics of underlying processes is of interest. Since currently available statistical methods for estimating derivatives require densely observed data, it is quiet challenging to recover derivatives from sparse functional data with noise-contaminated measurements. Liu and Müller [5] proposed an approach based on estimating derivatives of eigenfunctions to obtain a representation for derivatives of a sample of sparsely observed one-dimensional functions. Our further work in this paper is aimed at recovering partial derivatives of underlying functional-valued process at each location, that is the dth partial derivatives of Xs,t with respect to s, which is denoted as Xd,0s,t=∂d/∂sdXs,t. The whole procedure is also in a unified framework in which multiple functional data can be either densely or sparsely observed.The article is organized as follows. In Section2, we introduce the model and all estimation procedures for recovering both functional-valued process and its partial derivatives. We establish the uniformly almost sure convergence rates of the procedures in Section 3, where we also discuss the rates corresponding to some special scenarios. Some relative issues to our proposed procedures are discussed in Section 4. In Section 5, simulation studies are conducted to evaluate the performance of our procedures. All technique lemmas and all proofs are included in Appendix.


## 2. Models and Estimation

### 2.1. Representations

Consider processXs,t with mean μs,t=EXs,t for all s∈S⊂ℝp1 and t∈T⊂ℝp2 and covariance function(1)Cs,t,u,v=EXs,tXu,v−μs,tμu,v=EXcs,tXcu,v.Chen et al. [4] proposed a representation as(2)Xs,t=μs,t+∑j=1∞ξjtψjs=μs,t+∑j=1∞∑k=1∞ζijkϕjktψjs,where ψ:j≥1 are the eigenfunctions of the operator in L2S with kernel(3)GSs,u=∫TCs,t,u,tdt,and ξjt:j≥1 are the random coefficients of the expansion of the centred processes Xc⋅,t in ψjs and(4)ξjt=∑k=1∞ζijkϕjkt,is the Karhunen–Loeve expansion of the random functions ξjt in L2T. Here, for each j≥1, ϕjk,k≥1 are the eigenfunctions of the operator with kernel(5)ΓTt,v=Eξjtξjv,and ζijk,k≥1 are the FPC scores of ξjt.Based on the representation ofXs,t shown as (2), we can write Xd,0s,t as(6)Xd,0s,t=μd,0s,t+∑j=1∞∑k=1∞ζijkϕjktψjds,where μd,0s,t is the dth partial derivative of μs,t with respect to s and ψjd⋅ is the dth derivative of ψj⋅ on S.Denoteλ1≥λ2≥⋯≥0 are eigenvalues of Gu,s, and then the eigenfunctions ψj are the solutions of the eigenequations ∫SGSu,sψjuds=λjψjs, under side conditions of norm 1 and orthogonality on all previous eigenfunctions. Upon taking the dth derivative with respect to s from both side of these eigenequations,(7)∂d∂sd∫SGSs,uψjuds=λjψjds.If∂d/∂sdGSs,uψju exists for all u,s∈S, and(8)∂d∂sdGSs,uψju,is bounded and integrable for all s,u∈S, interchanging integrations and differentiation leads to(9)ψjds=1λj∫S∂d∂sdGSs,uψjuds.One can then estimate derivatives by approximatingXd,0s,t with a truncated representation(10)X˜d,0s,t=μd,0s,t+∑j=1J∑k=1Kjζijkϕjktψjds,with finite Kj≥1, j=1,2,⋯,J, and J≥1.


### 2.2. Estimation

Time-indexed functional data consist of a sample ofn independent subjects or units. For the ith subject, suppose we observe(11)Yiml=XSiml,Tim+εiml,i=1,…,n,m=1,…,Mi,l=1,…,Lim,which means that, on each time point Tim, ∞, Xi⋅,Tim is recorded at a grid of functional points Siml, l=1,…,Lim. Here, εiml are additional measurement errors, assumed to be iid with mean zero and finite variance σ2. We also assume that εiml are independent of all Xi, Tim, and Siml.Our approach is based on the local-polynomial smoother [6].Step 1.
Estimation of the mean function and the partial derivatives of mean functions.
For fixeds,t∈S×T and some bandwidths h1 and h2,(12)μSiml,Tim≈μs,t+h1μ1,0s,tSiml−sh1+⋯+h1d+1d+1!μd+1,0s,tSiml−sh1d+1+h2μ0,1s,tTim−th2≡∑k=0d+1θk0Siml−sh1k+θ01Tim−th2.
Letθ=θ00,θ10,…,θd+10,θ01T, where θl0=h1lμl,0s,t/l!,l=0,1,…,d+1 and θ01=μ0,1s,t. Then, we obtain a smoothing estimator θ^=θ^00,θ^10,…,θ^d+10,θ^01T as(13)θ^=argmin1n∑i=1n1Mi∑m=1Mi1Lim∑l=1LimKh1Siml−sKh2Tim−tYiml−∑k=0d+1θk0Siml−sh1k−θ01Tim−th22,where K⋅ is a symmetric probability density function on 0,1 and Kh⋅=1/hK⋅/h. Here, the kernel function K⋅ can be different at different occasions. Then, local estimators of μs,t and μd,0s,t are given by(14)μ^s,t=θ^00,μ^d,0s,t=d!h1−dθ^d0,respectively.Step 2.
Estimation ofGSs,u, GSd,0s,u, and σ2.
Note that(15)GSs,u=∫TCs,t,u,tdt=∫TDs,u,t−μs,tμu,tdt,where Ds,u,t=EXs,tXu,t. On the contrary, if ∂d/∂sdCs,t,u,t exists for all s,u,t∈S2×T and ∂d/∂sdCs,t,u,t is bounded and integrable for all s,u,t∈S2×T, then(16)GSd,0s,u=∫T∂d∂sdCs,t,u,tdt=∫TDd,0,0s,u,t−μd,0s,tμu,tdt,where Dd,0,0s,u,t=∂d/∂sdDs,u,t. Thus, in order to estimate GSs,u and GSd,0s,u, we estimate Ds,u,t and Dd,0,0s,u,t first.
To this end, we estimateDs,u,t and Dd,0,0s,u,t based on the following procedures. For fixed s,u,t∈S2×T and some bandwidths h3, h4, and h5 (for d = 0, we choose h3=h4),(17)DSiml,Siml′,Tim≈Ds,u,t+h3D1,0,0s,u,tSiml−sh3+⋯+h3d+1d+1!Dd+1,0,0s,u,tSiml−sh3d+1+h4D0,1,0s,u,tSiml′−uh4+h5D0,0,1s,u,tTim−th5≡∑k=0d+1θk00∗Siml−sh3k+θ010∗Siml′−uh4+θ001∗Tim−th5.
Letθ∗=θ000∗,θ100∗,…,θd+100∗,θ010∗,θ001∗T, where θl00∗=h3lDl,0,0s,u,t/l!,l=1,…,d+1, θ010∗=D0,1,0s,u,t, and θ001∗=D0,0,1s,u,t. Then, we obtain an estimator θ^∗=θ^000∗,θ^100∗,…,θ^d+100∗,θ^010∗,θ^001∗T as(18)θ^∗=argmin1n∑i=1n1Mi∑m=1Mi1Lim∗∑l≠l′LimKh3Siml−sKh4Siml′−uKh5Tim−tYimlYiml′−∑k=0d+1θk00∗Siml−sh3k−θ010∗Siml′−uh4−θ001∗Tim−th52,where Lim∗=Lim−1Lim. Then, smoothing estimators of Ds,u,t and Dd,0,0s,u,t are given by(19)D^s,u,t=θ^000∗,D^d,0,0s,u,t=d!h3−dθ^d00∗,respectively. We then can obtain that(20)G^Ss,u=∫TD^s,u,t−μ^s,tμ^u,tdt,G^Sd,0s,u=∫TD^d,0,0s,u,t−μ^d,0s,tμ^u,tdt.
To estimateσ2, we first estimate Vs,t=Ds,s,t+σ2 by V^s,t=a^0, where(21)a^0,a^1,a^2=argmin1n∑i=1n1Mi∑m=1Mi1Lim∑l=1LimKhv1Siml−sKhv2Tim−tYiml2−a0−a1Siml−u−a2Tim−t2,with some bandwidths hv1 and hv2. Ds,s,t is estimated in the same way as estimating Ds,u,t, but with d=0 and h3=h4, we then estimate σ2 by(22)σ^2=1ST∫T∫SV^s,t−D^s,s,tdsdt,where S and T are Lebesgue measures of S and T, respectively.Remark 1.
In practice, the empirical estimator ofGSs,u [4] can be used and remains at convergence rate logn/n1/2 for dense and regular designs in s; that is, all Xi⋅,ts are observed at sll=1l and maxsl−sl−1=On−1. On the contrary, by presmoothing for individual curves, the empirical estimator of GSs,u is also applicable for dense and random designs for s, and as designs get denser, the overall convergence rate logn/n1/2 remains under appropriate regularity conditions. Under these circumstances, a further smoothing estimator of Gd,0s,u can also be obtained based on empirical estimators of GSs,u. Similar results hold for the estimation of σ2.
However, in practice, it is possible that some sample curves are densely observed, while others are sparsely observed at thes direction. Moreover, in dealing with real data, it may even be difficult to classify which scenario we are faced with and hence to decide which methodology to use.Step 3.
Estimation of eigenfunctionsψjs and ψjds and eigenvalues λj of the operator in L2S with kernel GSs,u, as well as estimation of FPC functions ξijt=∫SXics,tψjsds.
The estimated eigenfunctionsψ^js and estimated eigenvalues λ^j can be obtained by standard methods of computing the eigenvalues and eigenfunctions of an integral operator with a symmetric kernel. Then, we have(23)ψ^jds=λ^j−1∫SG^Sd,0s,uψ^judu.
For designs that are dense ins, one can obtain ξ^ijt by interpolating numerical approximations of the integrals:(24)ξ^ijTim=∫Xis,Tim−μ^s,Timψ^jsds.
On the contrary, for designs that are sparse ins, one can estimate ξijt by the PACE approach [7].Step 4.
Estimation of eigenfunctionsϕjkt of the operator with kernel ΓTt,v and FPCs ζijk=∫Tξijtϕjktdt.
This is a standard FPCA of one-dimensional processesξ^ijt,j≥1. For each fixed j, one obtains estimates for the FPCs ζijk and eigenfunctions ϕjkt for designs that are dense in t [1] and for designs that are sparse in t [7]. One can also adapt the approach of Li and Hsing [8], which is suitable for both sparse and dense functional data, to one-dimensional processes ξ^ijt,j≥1.
In this step, for eachj, we are able to approximate ξ^ijt by ∑k=1Kjζ^ijkϕ^jkt.
After selecting appropriate numbers of included componentsJ and Kj, j=1,…,J, we obtain the overall representation:(25)X^s,t=μ^s,t+∑j=1J∑k=1Kjζ^ijkϕ^jktψ^js,(26)X^d,0s,t=μ^d,0s,t+∑j=1J∑k=1Kjζ^ijkϕ^jktψ^jds.
The included number of componentsJ and Kj, j=1,…,J, can be selected via a variety of methods, including fraction of variance explained (FVE) criterion [8], leave-one-curve-out cross-validation [9], pseudo-AIC [10], or pseudo-BIC [7, 11]. We will illustrate these procedures in Section 4.


## 2.1. Representations

Consider processXs,t with mean μs,t=EXs,t for all s∈S⊂ℝp1 and t∈T⊂ℝp2 and covariance function(1)Cs,t,u,v=EXs,tXu,v−μs,tμu,v=EXcs,tXcu,v.Chen et al. [4] proposed a representation as(2)Xs,t=μs,t+∑j=1∞ξjtψjs=μs,t+∑j=1∞∑k=1∞ζijkϕjktψjs,where ψ:j≥1 are the eigenfunctions of the operator in L2S with kernel(3)GSs,u=∫TCs,t,u,tdt,and ξjt:j≥1 are the random coefficients of the expansion of the centred processes Xc⋅,t in ψjs and(4)ξjt=∑k=1∞ζijkϕjkt,is the Karhunen–Loeve expansion of the random functions ξjt in L2T. Here, for each j≥1, ϕjk,k≥1 are the eigenfunctions of the operator with kernel(5)ΓTt,v=Eξjtξjv,and ζijk,k≥1 are the FPC scores of ξjt.Based on the representation ofXs,t shown as (2), we can write Xd,0s,t as(6)Xd,0s,t=μd,0s,t+∑j=1∞∑k=1∞ζijkϕjktψjds,where μd,0s,t is the dth partial derivative of μs,t with respect to s and ψjd⋅ is the dth derivative of ψj⋅ on S.Denoteλ1≥λ2≥⋯≥0 are eigenvalues of Gu,s, and then the eigenfunctions ψj are the solutions of the eigenequations ∫SGSu,sψjuds=λjψjs, under side conditions of norm 1 and orthogonality on all previous eigenfunctions. Upon taking the dth derivative with respect to s from both side of these eigenequations,(7)∂d∂sd∫SGSs,uψjuds=λjψjds.If∂d/∂sdGSs,uψju exists for all u,s∈S, and(8)∂d∂sdGSs,uψju,is bounded and integrable for all s,u∈S, interchanging integrations and differentiation leads to(9)ψjds=1λj∫S∂d∂sdGSs,uψjuds.One can then estimate derivatives by approximatingXd,0s,t with a truncated representation(10)X˜d,0s,t=μd,0s,t+∑j=1J∑k=1Kjζijkϕjktψjds,with finite Kj≥1, j=1,2,⋯,J, and J≥1.


## 2.2. Estimation

Time-indexed functional data consist of a sample ofn independent subjects or units. For the ith subject, suppose we observe(11)Yiml=XSiml,Tim+εiml,i=1,…,n,m=1,…,Mi,l=1,…,Lim,which means that, on each time point Tim, ∞, Xi⋅,Tim is recorded at a grid of functional points Siml, l=1,…,Lim. Here, εiml are additional measurement errors, assumed to be iid with mean zero and finite variance σ2. We also assume that εiml are independent of all Xi, Tim, and Siml.Our approach is based on the local-polynomial smoother [6].Step 1.
Estimation of the mean function and the partial derivatives of mean functions.
For fixeds,t∈S×T and some bandwidths h1 and h2,(12)μSiml,Tim≈μs,t+h1μ1,0s,tSiml−sh1+⋯+h1d+1d+1!μd+1,0s,tSiml−sh1d+1+h2μ0,1s,tTim−th2≡∑k=0d+1θk0Siml−sh1k+θ01Tim−th2.
Letθ=θ00,θ10,…,θd+10,θ01T, where θl0=h1lμl,0s,t/l!,l=0,1,…,d+1 and θ01=μ0,1s,t. Then, we obtain a smoothing estimator θ^=θ^00,θ^10,…,θ^d+10,θ^01T as(13)θ^=argmin1n∑i=1n1Mi∑m=1Mi1Lim∑l=1LimKh1Siml−sKh2Tim−tYiml−∑k=0d+1θk0Siml−sh1k−θ01Tim−th22,where K⋅ is a symmetric probability density function on 0,1 and Kh⋅=1/hK⋅/h. Here, the kernel function K⋅ can be different at different occasions. Then, local estimators of μs,t and μd,0s,t are given by(14)μ^s,t=θ^00,μ^d,0s,t=d!h1−dθ^d0,respectively.Step 2.
Estimation ofGSs,u, GSd,0s,u, and σ2.
Note that(15)GSs,u=∫TCs,t,u,tdt=∫TDs,u,t−μs,tμu,tdt,where Ds,u,t=EXs,tXu,t. On the contrary, if ∂d/∂sdCs,t,u,t exists for all s,u,t∈S2×T and ∂d/∂sdCs,t,u,t is bounded and integrable for all s,u,t∈S2×T, then(16)GSd,0s,u=∫T∂d∂sdCs,t,u,tdt=∫TDd,0,0s,u,t−μd,0s,tμu,tdt,where Dd,0,0s,u,t=∂d/∂sdDs,u,t. Thus, in order to estimate GSs,u and GSd,0s,u, we estimate Ds,u,t and Dd,0,0s,u,t first.
To this end, we estimateDs,u,t and Dd,0,0s,u,t based on the following procedures. For fixed s,u,t∈S2×T and some bandwidths h3, h4, and h5 (for d = 0, we choose h3=h4),(17)DSiml,Siml′,Tim≈Ds,u,t+h3D1,0,0s,u,tSiml−sh3+⋯+h3d+1d+1!Dd+1,0,0s,u,tSiml−sh3d+1+h4D0,1,0s,u,tSiml′−uh4+h5D0,0,1s,u,tTim−th5≡∑k=0d+1θk00∗Siml−sh3k+θ010∗Siml′−uh4+θ001∗Tim−th5.
Letθ∗=θ000∗,θ100∗,…,θd+100∗,θ010∗,θ001∗T, where θl00∗=h3lDl,0,0s,u,t/l!,l=1,…,d+1, θ010∗=D0,1,0s,u,t, and θ001∗=D0,0,1s,u,t. Then, we obtain an estimator θ^∗=θ^000∗,θ^100∗,…,θ^d+100∗,θ^010∗,θ^001∗T as(18)θ^∗=argmin1n∑i=1n1Mi∑m=1Mi1Lim∗∑l≠l′LimKh3Siml−sKh4Siml′−uKh5Tim−tYimlYiml′−∑k=0d+1θk00∗Siml−sh3k−θ010∗Siml′−uh4−θ001∗Tim−th52,where Lim∗=Lim−1Lim. Then, smoothing estimators of Ds,u,t and Dd,0,0s,u,t are given by(19)D^s,u,t=θ^000∗,D^d,0,0s,u,t=d!h3−dθ^d00∗,respectively. We then can obtain that(20)G^Ss,u=∫TD^s,u,t−μ^s,tμ^u,tdt,G^Sd,0s,u=∫TD^d,0,0s,u,t−μ^d,0s,tμ^u,tdt.
To estimateσ2, we first estimate Vs,t=Ds,s,t+σ2 by V^s,t=a^0, where(21)a^0,a^1,a^2=argmin1n∑i=1n1Mi∑m=1Mi1Lim∑l=1LimKhv1Siml−sKhv2Tim−tYiml2−a0−a1Siml−u−a2Tim−t2,with some bandwidths hv1 and hv2. Ds,s,t is estimated in the same way as estimating Ds,u,t, but with d=0 and h3=h4, we then estimate σ2 by(22)σ^2=1ST∫T∫SV^s,t−D^s,s,tdsdt,where S and T are Lebesgue measures of S and T, respectively.Remark 1.
In practice, the empirical estimator ofGSs,u [4] can be used and remains at convergence rate logn/n1/2 for dense and regular designs in s; that is, all Xi⋅,ts are observed at sll=1l and maxsl−sl−1=On−1. On the contrary, by presmoothing for individual curves, the empirical estimator of GSs,u is also applicable for dense and random designs for s, and as designs get denser, the overall convergence rate logn/n1/2 remains under appropriate regularity conditions. Under these circumstances, a further smoothing estimator of Gd,0s,u can also be obtained based on empirical estimators of GSs,u. Similar results hold for the estimation of σ2.
However, in practice, it is possible that some sample curves are densely observed, while others are sparsely observed at thes direction. Moreover, in dealing with real data, it may even be difficult to classify which scenario we are faced with and hence to decide which methodology to use.Step 3.
Estimation of eigenfunctionsψjs and ψjds and eigenvalues λj of the operator in L2S with kernel GSs,u, as well as estimation of FPC functions ξijt=∫SXics,tψjsds.
The estimated eigenfunctionsψ^js and estimated eigenvalues λ^j can be obtained by standard methods of computing the eigenvalues and eigenfunctions of an integral operator with a symmetric kernel. Then, we have(23)ψ^jds=λ^j−1∫SG^Sd,0s,uψ^judu.
For designs that are dense ins, one can obtain ξ^ijt by interpolating numerical approximations of the integrals:(24)ξ^ijTim=∫Xis,Tim−μ^s,Timψ^jsds.
On the contrary, for designs that are sparse ins, one can estimate ξijt by the PACE approach [7].Step 4.
Estimation of eigenfunctionsϕjkt of the operator with kernel ΓTt,v and FPCs ζijk=∫Tξijtϕjktdt.
This is a standard FPCA of one-dimensional processesξ^ijt,j≥1. For each fixed j, one obtains estimates for the FPCs ζijk and eigenfunctions ϕjkt for designs that are dense in t [1] and for designs that are sparse in t [7]. One can also adapt the approach of Li and Hsing [8], which is suitable for both sparse and dense functional data, to one-dimensional processes ξ^ijt,j≥1.
In this step, for eachj, we are able to approximate ξ^ijt by ∑k=1Kjζ^ijkϕ^jkt.
After selecting appropriate numbers of included componentsJ and Kj, j=1,…,J, we obtain the overall representation:(25)X^s,t=μ^s,t+∑j=1J∑k=1Kjζ^ijkϕ^jktψ^js,(26)X^d,0s,t=μ^d,0s,t+∑j=1J∑k=1Kjζ^ijkϕ^jktψ^jds.
The included number of componentsJ and Kj, j=1,…,J, can be selected via a variety of methods, including fraction of variance explained (FVE) criterion [8], leave-one-curve-out cross-validation [9], pseudo-AIC [10], or pseudo-BIC [7, 11]. We will illustrate these procedures in Section 4.


## 3. Asymptotic Theory

We first define the notations and conditions to be used. Assume thatMi and Lim may depend on n as well, namely, Mi=Min and Lim=Limn. However, for simplicity, we continue to use the notation Mi and Lim. Define(27)γnkl=1n∑i=1n1Mik∑m=1Mi1Liml,k=1,2,l=0,1,2.For any bandwidthsh1,h2,andh3, we also define(28)δn1h1,h2=nh1h2−1γn21+2γn20h1+2γn11h2+4h1h2logn,δn2h1,h2,h3=nh1h2h3−1γn22+4γn20h1h2+2γn12h3+8h1h2h3logn.From now on, without loss of generality, we assume that the domainS×T of the process is 0,a×0,b. Some assumptions needed for the asymptotic theory are as follows. We use 0<C<∞ as a generic constant that can take different values at different places.Now, we state the assumptions:Assumption 1.
All second-order partial derivatives ofμs,t are uniformly continuous and bounded on 0,a×0,b. Furthermore, μd+2,0s,t exists and is uniformly continuous and bounded on 0,a×0,b.Assumption 2.
All second-order partial derivatives ofDs,u,t are uniformly continuous and bounded on 0,a2×0,b. Furthermore, Dd+2,0,0s,u,t exists and is uniformly continuous and bounded on 0,a2×0,b.Assumption 3.
LetfS⋅ and fT⋅ be the density distribution functions of S and T, respectively. Assume both of fS⋅ and fT⋅ are lower bounded away from 0 and sup0,afS⋅≤MS and sup0,bfT⋅≤MT for some positive constants MS and MT.Assumption 4.
Letf⋅,⋅ be the joint density distribution function of S,T and f2⋅,⋅,⋅ be the joint density distribution function of S1,S2,T. Both f⋅,⋅ and f2⋅,⋅,⋅ are upper bounded and lower bounded away from 0. Furthermore, assume that both f⋅,⋅ and f2⋅,⋅,⋅ have continuous and bounded second-order derivatives uniformly on their domains.Assumption 5.
E
ε
i
m
l
λ
1
<
∞ and EsupXs,tλ1<∞ for some λ1∈2,∞. h1,h2⟶0 as n⟶∞, and h1−1h2−1γn21+2γn20h1+2γn11h2+4h1h2−1logn/n1−2/λ1=o1.Assumption 6.
E
ε
i
m
l
2
λ
2
<
∞ and EsupXs,t2λ2<∞ for some λ2∈2,∞. h3,h4,h5⟶0 as n⟶∞ and h3−1h4−1h5−1γn22+4γn20h3h4+2γn12h5+8h3h4h5−1logn/n1−2/λ2=o1.Assumption 7.
E
ε
i
m
l
2
λ
3
<
∞ and EsupXs,t2λ3<∞ for some λ3∈2,∞. hv1,hv2⟶0 as n⟶∞ and hv1−1hv2−1γn21+2γn20hv1+2γn11hv2+4hv1hv2−1logn/n1−2/λ3=o1.Assumption 8.
Assume the autocovariance operatorΔS generated by GSs,u is positive definite, such that(29)sups∫GSd,0s,uψjudu<∞,∫∫GSd,0s,udsdu<∞.and with eigenfunctions satisfying sups∈0,aψjs=O1.Assumption 9.
LetK⋅ be a symmetric probability density function on 0,1, νk=∫ukKudu. Let(30)Γ=1ν1⋯νd+1ν1ν2⋯νd+2⋮νd+1νd+2⋯ν2d+2.
Assume thatΓ is a nonsingular matrix.
Assumptions (1) and (2) are regular smoothness conditions on the mean functionµ and the covariance function D. Since we do not impose any parametric structure on the distribution of X, assumptions (3) and (4) are required for the derivation of uniform convergence. The moment conditions in (5)–(7) are similar to that in (C.5)–(C.7) of Li and Hsing [8] and hold rather generally. Assumptions (8) is similar to condition (B4) in Liu and Müller [5] and is needed for Theorem 4. When d=1, the standard normal distribution function is an example for a kernel satisfying (13).


### 3.1. Uniform Convergence Rates ofμ^s,t and μ^d,0s,t

We establish the uniform convergence rates ofμ^s,t and μ^d,0s,t. First, we give some definitions and notations. For p=0,1,…,2d+2, and q=0,1,2, let(31)Snpqs,t=1n∑i=1n1Mi∑m=1Mi1Lim∑l=1LimKh1Siml−sKh2Tim−tSiml−sh1pTim−th2q,S˜npqs,t=1n∑i=1n1Mi∑m=1Mi1Lim∑l=1LimYimlKh1Siml−sKh2Tim−tSiml−sh1pTim−th2q.By some simple algebra, we can obtain thatθ^ satisfies(32)S˜ns,t=Sns,tθ^,where S˜ns,t=S˜n00s,t,S˜n10s,t,…,S˜nd+10s,t,S˜n01s,tT and(33)Sns,t=Sn00Sn10⋯Snd+10Sn01Sn10Sn20⋯Snd+20Sn11⋮⋮⋱⋮⋮Snd+10Snd+20⋯Sn2d+20Snd+11Sn01Sn11⋯Snd+11Sn02s,t.It then follows thatθ^=Sn−1s,tS˜ns,t and θ^−θ=Sn−1s,tS˜ns,t−Sns,tθ.Theorem 1.
Under assumptions (1), (3)–(5), and (9), letγ1dh1,h2=h1d+2+h1h2+h22, then(34)sups∈S,t∈Tμ^s,t−μs,t=Oa.sδn1h1,h2+γ1dh1,h2,sups∈S,t∈Tμ^d,0s,t−μd,0s,t=Oa.sh1−dδn1h1,h2+h1−dγ1dh1,h2.Remark 2.
We discuss special cases of Theorem 1 under dense or sparse designs.(1)
For the designs that are sparse in boths and t: if both max1≤i≤nMi and max1≤i≤n,1≤m≤MiLim are bounded, then γnkl=O1 for all k=1,2 and l=0,1,2. Thus, Theorem 1 implies that(35)sups∈S,t∈Tμ^s,t−μs,t=Oa.snh1h2−1logn+γ1dh1,h2,sups∈S,t∈Tμ^d,0s,t−μd,0s,t=Oa.sh1−dnh1h2−1logn+h1−dγ1dh1,h2.By choosingh1 and h2 satisfying that γ1dh1,h2=Oh1h2=Ologn/n1/3, sups∈S,t∈Tμ^s,t−μs,t can achieve its optimal convergence rate as Ologn/n1/3. Moreover, sups∈S,t∈Tμ^d,0s,t−μd,0s,t can also achieve its optimal convergence rate as Oa.s.logn/n2/3d+6 by further choosing h2=Oh1d+1, that is h2=Oh1d+1=Ologn/nd+1/3d+6.(2)
For the designs that are dense in boths and t: if min1≤i≤nMi=Mn and min1≤i≤n,1≤m≤MiLim=Ln, satisfying Mn−1≲h2 and Ln−1≲h1, then for all k=1,2 and l=0,1,2, γnkl≍Mn1−kLn−l and δn1h1,h2≍logn/n. Thus, Theorem 1 yields that(36)sups∈S,t∈Tμ^s,t−μs,t=Oa.slogn/n+γ1dh1,h2,sups∈S,t∈Tμ^d,0s,t−μd,0s,t=Oa.sh1−dlogn/n+h1−dγ1dh1,h2.Moreover, if the Assumption (5) is replaced by a strong version, in which we assume thatsups,tXs,t and εiml are bounded, then if h1 and h2 satisfying h1h2=Ologn/n1/2 and h2=Oh1d+1, Theorem 1 implies that(37)sups∈S,t∈Tμ^s,t−μs,t=Oa.slogn/n,sups∈S,t∈Tμ^d,0s,t−μd,0s,t=Oa.slogn/n1/d+2.(3)
For the designs that are sparse in s and dense int: if max1≤i≤n,1≤m≤MiLim is bounded and min1≤i≤nMi=Mn, where Mn−1≲h2, then δn1h1,h2≍logn/nh1, that is(38)sups∈S,t∈Tμ^s,t−μs,t=Oa.slogn/nh1+γ1dh1,h2,sups∈S,t∈Tμ^d,0s,t−μd,0s,t=Oa.sh1−dlogn/nh1+h1−dγ1dh1,h2.Furthermore, ifh1=Ologn/n1/2d+5 and h2=Oh1d+1, then Theorem 1 shows that(39)sups∈S,t∈Tμ^s,t−μs,t=Oa.slogn/nd+2/2d+5,sups∈S,t∈Tμ^d,0s,t−μd,0s,t=Oa.slogn/n2/2d+5.(4)
For the designs that are dense ins and sparse in t: if max1≤i≤nMi is bounded and min1≤i≤n,1≤m≤MiLim=Ln, where Ln−1≲h1, then δn1h1,h2≍logn/nh2, that is(40)sups∈S,t∈Tμ^s,t−μs,t=Oa.slogn/nh2+γ1dh1,h2,sups∈S,t∈Tμ^d,0s,t−μd,0s,t=Oa.sh1−dlogn/nh2+h1−dγ1dh1,h2.Furthermore, ifh1=Ologn/n1/3d+5 and h2=Oh1d+1, then Theorem 1 shows that(41)sups∈S,t∈Tμ^s,t−μs,t=Oa.slogn/nd+2/3d+5,sups∈S,t∈Tμ^d,0s,t−μd,0s,t=Oa.slogn/n2/3d+5.


### 3.2. Uniform Convergence Rates ofG^s,u, G^d,0s,u, and σ2^

We next establish the convergence rates ofG^s,u, G^d,0s,u, and σ2^. Since G^s,u=∫TD^s,u,t−μ^s,tμ^u,tdt, integrating D^s,u,t and μ^s,tμ^u,t over t results in extrasmoothing, which leads to a faster convergence rate δn1h1,1 and δn2h3,h4,1 than δn1h1,1 and δn2h3,h4,1, respectively. The similar conclusion holds for G^d,0s,u.Forp=0,1,…,2d+2, q=0,1,2, and r=0,1,2, let(42)Rnpqrs,u,t=1n∑i=1n1Mi∑m=1Mi1Lim∗∑l≠l′LimKh3Siml−sKh4Siml′−uKh5Tim−tSiml−sh3pSiml′−uh4qTim−th5r,R˜npqrs,u,t=1n∑i=1n1Mi∑m=1Mi1Lim∗∑l≠l′LimYimlYiml′Kh3Siml−sKh4Siml′−uKh5Tim−tSiml−sh3pSiml′−uh4qTim−th5r,then we can obtain that(43)θ^∗=Rn−1s,u,tR˜ns,u,t,with R˜ns,u,t=Rn000s,u,t,Rn100s,u,t,…,Rnd+100s,u,t,Rn010s,u,t,Rn001s,u,tT and(44)Rns,u,t=Rn000Rn100⋯Rnd+100Rn010Rn001Rn100Rn200⋯Rnd+200Rn110Rn101⋮⋮⋱⋮⋮⋮Rnd+100Rnd+200⋯Rn2d+200Rnd+110Rnd+101Rn010Rn110⋯Rnd+110Rn020Rn011Rn001Rn100⋯Rnd+101Rn011Rn002s,u,t.Theorem 2.
Under assumptions (1)–(6) and (9), letγ2dh3,h4,h5=h3d+2+h42+h52+h3h4+h3h5, then(45)sups,u∈SG^s,u−Gs,u=Oa.s.δn2h3,h4,1+γ2dh3,h4,h5+δn1h1,1+γ1dh1,h2,(46)sups,u∈SG^d,0s,u−Gd,0s,u=Oa.sh3−dδn2h3,h4,1+γ2dh3,h4,h5+δn1h1,1+γ1dh1,h2.Remark 3.
We discuss special cases of Theorem 2. Actually, whether the design is dense or sparse int, the convergence rate in Theorem 2 is not affected. Hence, we only discuss different designs with respect to s. (1)
For the designs that are sparse ins: if max1≤i≤n,1≤m≤MiLim is bounded, then(47)δn1h1,1≍logn/nh1,δn2h3,h4,1≍logn/nh3h4.On the contrary, since(48)δn2h3,h4,1≍δn1h3,h4,γ2dh3,h4,h5≍γ1dh3,h4+γ1dh3,h5,we choosingh3=Ologn/n1/3d+6 and h4≍h5=Oh3d+1. Either h1orh2 is chosen as in Remark 2(1) or Remark 2(3), Theorem 2 implies that(49)sups,u∈SG^s,u−Gs,u=Oa.s.logn/n1/3,sups,u∈SG^d,0s,u−Gd,0s,u=Oa.slogn/n2/3d+6.(2)
For the designs that are dense ins: if min1≤i≤n,1≤m≤MiLim=Ln, then(50)δn1h1,1≍Ln−1+h1logn/nh1,δn2h3,h4,1≍Ln−2+h3h4logn/nh3h4.Assume thatLn−1=oh1 and Ln−2=oh3h4, where h1≍h3=Ologn/n1/2d+4 and h4≍h5≍h2=Oh1d+1. If the Assumption (5) is replaced by a strong version, in which we assume that sups,tXs,t and εiml are bounded, then Theorem 2 implies that(51)sups,u∈SG^s,u−Gs,u=Oa.s.logn/n,sups,u∈SG^d,0s,u−Gd,0s,u=Oa.slogn/n1/d+2.Theorem 3.
Under assumptions (1)–(7) and (9),(52)σ^2−σ2=Oa.s.δn2h3,1,1+h32+h52+hv12+hv22.Remark 4.
Same as Remark 3, we discuss the convergence rate ofσ^2 under special cases.(1)
For the designs that are sparse ins: if max1≤i≤n,1≤m≤MiLim is bounded, then(53)δn2h3,1,1≍logn/nh3,which results in(54)σ^2−σ2=Oa.s.logn/nh3+h32+h52+hv12+hv22.(2)
For the designs that are dense ins: if min1≤i≤n,1≤m≤MiLim=Ln, satisfying that Ln−1≲h31/2, then(55)δn2h3,1,1≍logn/n.Ifh3,h5,hv1,hv2≲logn/n1/4, then(56)σ^2−σ2=Oa.s.logn/n.


### 3.3. Uniform Convergence Rates in FPCA

We next establish the convergence rates in FPCA. LetJ be a fixed positive constant.Theorem 4.
Under assumptions (1)–(9), forj≤J,(1)
ψ^j−ψj=Oa.s.δn2h4,1,1+δn22h3,h4,1+γ2dh3,h4,h5+δn1h1,1+γ1dh1,h2(2)
λ^j−λj=Oa.s.logn/n1/2+δn22h4,1,1+γ2dh3,h4,h5+δn12h1,1+γ1dh1,h2(3)
sups∈0,aψ^js−ψjs=Oa.s.logn/n1/2+δn2h4,1,1+δn22h3,h4,1+γ2dh3,h4,h5+δn1h1,1+γ1dh1,h2(4)
sups∈0,aψ^jds−ψjds=Oa.s.logn/n1/2+δn22h4,1,1+δn12h1,1+h3−dδn2h4,1,1+γ2dh3,h4,h5+δn1h1,1+γ1dh1,h2(5)
For each1≤i≤n,(57)max1≤m≤Miξ^ijTim−ξijTim⟶a.s.0.The consistency ofξ^ijt guarantees the appropriacy of estimation procedures in Step 4. The proof of the theorems will be given in the Appendix.


## 3.1. Uniform Convergence Rates ofμ^s,t and μ^d,0s,t

We establish the uniform convergence rates ofμ^s,t and μ^d,0s,t. First, we give some definitions and notations. For p=0,1,…,2d+2, and q=0,1,2, let(31)Snpqs,t=1n∑i=1n1Mi∑m=1Mi1Lim∑l=1LimKh1Siml−sKh2Tim−tSiml−sh1pTim−th2q,S˜npqs,t=1n∑i=1n1Mi∑m=1Mi1Lim∑l=1LimYimlKh1Siml−sKh2Tim−tSiml−sh1pTim−th2q.By some simple algebra, we can obtain thatθ^ satisfies(32)S˜ns,t=Sns,tθ^,where S˜ns,t=S˜n00s,t,S˜n10s,t,…,S˜nd+10s,t,S˜n01s,tT and(33)Sns,t=Sn00Sn10⋯Snd+10Sn01Sn10Sn20⋯Snd+20Sn11⋮⋮⋱⋮⋮Snd+10Snd+20⋯Sn2d+20Snd+11Sn01Sn11⋯Snd+11Sn02s,t.It then follows thatθ^=Sn−1s,tS˜ns,t and θ^−θ=Sn−1s,tS˜ns,t−Sns,tθ.Theorem 1.
Under assumptions (1), (3)–(5), and (9), letγ1dh1,h2=h1d+2+h1h2+h22, then(34)sups∈S,t∈Tμ^s,t−μs,t=Oa.sδn1h1,h2+γ1dh1,h2,sups∈S,t∈Tμ^d,0s,t−μd,0s,t=Oa.sh1−dδn1h1,h2+h1−dγ1dh1,h2.Remark 2.
We discuss special cases of Theorem 1 under dense or sparse designs.(1)
For the designs that are sparse in boths and t: if both max1≤i≤nMi and max1≤i≤n,1≤m≤MiLim are bounded, then γnkl=O1 for all k=1,2 and l=0,1,2. Thus, Theorem 1 implies that(35)sups∈S,t∈Tμ^s,t−μs,t=Oa.snh1h2−1logn+γ1dh1,h2,sups∈S,t∈Tμ^d,0s,t−μd,0s,t=Oa.sh1−dnh1h2−1logn+h1−dγ1dh1,h2.By choosingh1 and h2 satisfying that γ1dh1,h2=Oh1h2=Ologn/n1/3, sups∈S,t∈Tμ^s,t−μs,t can achieve its optimal convergence rate as Ologn/n1/3. Moreover, sups∈S,t∈Tμ^d,0s,t−μd,0s,t can also achieve its optimal convergence rate as Oa.s.logn/n2/3d+6 by further choosing h2=Oh1d+1, that is h2=Oh1d+1=Ologn/nd+1/3d+6.(2)
For the designs that are dense in boths and t: if min1≤i≤nMi=Mn and min1≤i≤n,1≤m≤MiLim=Ln, satisfying Mn−1≲h2 and Ln−1≲h1, then for all k=1,2 and l=0,1,2, γnkl≍Mn1−kLn−l and δn1h1,h2≍logn/n. Thus, Theorem 1 yields that(36)sups∈S,t∈Tμ^s,t−μs,t=Oa.slogn/n+γ1dh1,h2,sups∈S,t∈Tμ^d,0s,t−μd,0s,t=Oa.sh1−dlogn/n+h1−dγ1dh1,h2.Moreover, if the Assumption (5) is replaced by a strong version, in which we assume thatsups,tXs,t and εiml are bounded, then if h1 and h2 satisfying h1h2=Ologn/n1/2 and h2=Oh1d+1, Theorem 1 implies that(37)sups∈S,t∈Tμ^s,t−μs,t=Oa.slogn/n,sups∈S,t∈Tμ^d,0s,t−μd,0s,t=Oa.slogn/n1/d+2.(3)
For the designs that are sparse in s and dense int: if max1≤i≤n,1≤m≤MiLim is bounded and min1≤i≤nMi=Mn, where Mn−1≲h2, then δn1h1,h2≍logn/nh1, that is(38)sups∈S,t∈Tμ^s,t−μs,t=Oa.slogn/nh1+γ1dh1,h2,sups∈S,t∈Tμ^d,0s,t−μd,0s,t=Oa.sh1−dlogn/nh1+h1−dγ1dh1,h2.Furthermore, ifh1=Ologn/n1/2d+5 and h2=Oh1d+1, then Theorem 1 shows that(39)sups∈S,t∈Tμ^s,t−μs,t=Oa.slogn/nd+2/2d+5,sups∈S,t∈Tμ^d,0s,t−μd,0s,t=Oa.slogn/n2/2d+5.(4)
For the designs that are dense ins and sparse in t: if max1≤i≤nMi is bounded and min1≤i≤n,1≤m≤MiLim=Ln, where Ln−1≲h1, then δn1h1,h2≍logn/nh2, that is(40)sups∈S,t∈Tμ^s,t−μs,t=Oa.slogn/nh2+γ1dh1,h2,sups∈S,t∈Tμ^d,0s,t−μd,0s,t=Oa.sh1−dlogn/nh2+h1−dγ1dh1,h2.Furthermore, ifh1=Ologn/n1/3d+5 and h2=Oh1d+1, then Theorem 1 shows that(41)sups∈S,t∈Tμ^s,t−μs,t=Oa.slogn/nd+2/3d+5,sups∈S,t∈Tμ^d,0s,t−μd,0s,t=Oa.slogn/n2/3d+5.


## 3.2. Uniform Convergence Rates ofG^s,u, G^d,0s,u, and σ2^

We next establish the convergence rates ofG^s,u, G^d,0s,u, and σ2^. Since G^s,u=∫TD^s,u,t−μ^s,tμ^u,tdt, integrating D^s,u,t and μ^s,tμ^u,t over t results in extrasmoothing, which leads to a faster convergence rate δn1h1,1 and δn2h3,h4,1 than δn1h1,1 and δn2h3,h4,1, respectively. The similar conclusion holds for G^d,0s,u.Forp=0,1,…,2d+2, q=0,1,2, and r=0,1,2, let(42)Rnpqrs,u,t=1n∑i=1n1Mi∑m=1Mi1Lim∗∑l≠l′LimKh3Siml−sKh4Siml′−uKh5Tim−tSiml−sh3pSiml′−uh4qTim−th5r,R˜npqrs,u,t=1n∑i=1n1Mi∑m=1Mi1Lim∗∑l≠l′LimYimlYiml′Kh3Siml−sKh4Siml′−uKh5Tim−tSiml−sh3pSiml′−uh4qTim−th5r,then we can obtain that(43)θ^∗=Rn−1s,u,tR˜ns,u,t,with R˜ns,u,t=Rn000s,u,t,Rn100s,u,t,…,Rnd+100s,u,t,Rn010s,u,t,Rn001s,u,tT and(44)Rns,u,t=Rn000Rn100⋯Rnd+100Rn010Rn001Rn100Rn200⋯Rnd+200Rn110Rn101⋮⋮⋱⋮⋮⋮Rnd+100Rnd+200⋯Rn2d+200Rnd+110Rnd+101Rn010Rn110⋯Rnd+110Rn020Rn011Rn001Rn100⋯Rnd+101Rn011Rn002s,u,t.Theorem 2.
Under assumptions (1)–(6) and (9), letγ2dh3,h4,h5=h3d+2+h42+h52+h3h4+h3h5, then(45)sups,u∈SG^s,u−Gs,u=Oa.s.δn2h3,h4,1+γ2dh3,h4,h5+δn1h1,1+γ1dh1,h2,(46)sups,u∈SG^d,0s,u−Gd,0s,u=Oa.sh3−dδn2h3,h4,1+γ2dh3,h4,h5+δn1h1,1+γ1dh1,h2.Remark 3.
We discuss special cases of Theorem 2. Actually, whether the design is dense or sparse int, the convergence rate in Theorem 2 is not affected. Hence, we only discuss different designs with respect to s. (1)
For the designs that are sparse ins: if max1≤i≤n,1≤m≤MiLim is bounded, then(47)δn1h1,1≍logn/nh1,δn2h3,h4,1≍logn/nh3h4.On the contrary, since(48)δn2h3,h4,1≍δn1h3,h4,γ2dh3,h4,h5≍γ1dh3,h4+γ1dh3,h5,we choosingh3=Ologn/n1/3d+6 and h4≍h5=Oh3d+1. Either h1orh2 is chosen as in Remark 2(1) or Remark 2(3), Theorem 2 implies that(49)sups,u∈SG^s,u−Gs,u=Oa.s.logn/n1/3,sups,u∈SG^d,0s,u−Gd,0s,u=Oa.slogn/n2/3d+6.(2)
For the designs that are dense ins: if min1≤i≤n,1≤m≤MiLim=Ln, then(50)δn1h1,1≍Ln−1+h1logn/nh1,δn2h3,h4,1≍Ln−2+h3h4logn/nh3h4.Assume thatLn−1=oh1 and Ln−2=oh3h4, where h1≍h3=Ologn/n1/2d+4 and h4≍h5≍h2=Oh1d+1. If the Assumption (5) is replaced by a strong version, in which we assume that sups,tXs,t and εiml are bounded, then Theorem 2 implies that(51)sups,u∈SG^s,u−Gs,u=Oa.s.logn/n,sups,u∈SG^d,0s,u−Gd,0s,u=Oa.slogn/n1/d+2.Theorem 3.
Under assumptions (1)–(7) and (9),(52)σ^2−σ2=Oa.s.δn2h3,1,1+h32+h52+hv12+hv22.Remark 4.
Same as Remark 3, we discuss the convergence rate ofσ^2 under special cases.(1)
For the designs that are sparse ins: if max1≤i≤n,1≤m≤MiLim is bounded, then(53)δn2h3,1,1≍logn/nh3,which results in(54)σ^2−σ2=Oa.s.logn/nh3+h32+h52+hv12+hv22.(2)
For the designs that are dense ins: if min1≤i≤n,1≤m≤MiLim=Ln, satisfying that Ln−1≲h31/2, then(55)δn2h3,1,1≍logn/n.Ifh3,h5,hv1,hv2≲logn/n1/4, then(56)σ^2−σ2=Oa.s.logn/n.


## 3.3. Uniform Convergence Rates in FPCA

We next establish the convergence rates in FPCA. LetJ be a fixed positive constant.Theorem 4.
Under assumptions (1)–(9), forj≤J,(1)
ψ^j−ψj=Oa.s.δn2h4,1,1+δn22h3,h4,1+γ2dh3,h4,h5+δn1h1,1+γ1dh1,h2(2)
λ^j−λj=Oa.s.logn/n1/2+δn22h4,1,1+γ2dh3,h4,h5+δn12h1,1+γ1dh1,h2(3)
sups∈0,aψ^js−ψjs=Oa.s.logn/n1/2+δn2h4,1,1+δn22h3,h4,1+γ2dh3,h4,h5+δn1h1,1+γ1dh1,h2(4)
sups∈0,aψ^jds−ψjds=Oa.s.logn/n1/2+δn22h4,1,1+δn12h1,1+h3−dδn2h4,1,1+γ2dh3,h4,h5+δn1h1,1+γ1dh1,h2(5)
For each1≤i≤n,(57)max1≤m≤Miξ^ijTim−ξijTim⟶a.s.0.The consistency ofξ^ijt guarantees the appropriacy of estimation procedures in Step 4. The proof of the theorems will be given in the Appendix.


## 4. Relative Issues

In this section, we discussed a few issues that are related to the implementation of our proposed methods.


### 4.1. Selection of Bandwidths

The performance of the estimators depends on the choice of bandwidths forμ⋅,⋅ and D⋅,⋅,⋅, and the best bandwidths vary with Ms and Ls. The bandwidth selection problem turns out to be very challenging and hence an important problem for future research. For lack of a better approach, we suggest picking the bandwidths by minimizing the integrated mean square error (IMSE). That is, for each function above, one calculated the IMSE over a range of h and selected the one that minimizes the IMSE.


### 4.2. Selection ofKs and J in the Overall Representations (25) and (26)

In practice, the choice of the numbers of componentsJ and Kjs to be included in (25) can be based on the leave-one-curve-out cross-validation method [9] or by the fraction of variance explained (FVE) by the first J components [4]. One can also adopt AIC [10] or BIC [11] type of criteria, see Yao et al. [7] for one-dimensional function data.For bivariate functional data, a pseudo-Gaussian log-likelihood is given by(58)L^J=∑i=1n−Ni2log2π−Ni2logσ^2−12σ^2∑m=1Mi∑l=1LimYiml−μ^Siml,Tim−∑j=1Jξ^jTimψ^jSiml2,where Ni=∑m=1MiLim. One can choose J through minimizing aicJ=−L^J+J (resp., bicJ=−L^J+Jlogn) with respect to J.For each1≤j≤J, define(59)L^jKj=∑i=1n−Mi2log2π−Mi2logσ^2−12σ^2∑m=1Miξ^ijTim−∑k=1Kjζ^ijkϕ^jkTim2.The number of componentsKj is selected by minimizing aicKj=−L^jKj+Kj (resp., bicKj=−L^jKj+Kjlogn) with respect to Kj.


## 4.1. Selection of Bandwidths

The performance of the estimators depends on the choice of bandwidths forμ⋅,⋅ and D⋅,⋅,⋅, and the best bandwidths vary with Ms and Ls. The bandwidth selection problem turns out to be very challenging and hence an important problem for future research. For lack of a better approach, we suggest picking the bandwidths by minimizing the integrated mean square error (IMSE). That is, for each function above, one calculated the IMSE over a range of h and selected the one that minimizes the IMSE.


## 4.2. Selection ofKs and J in the Overall Representations (25) and (26)

In practice, the choice of the numbers of componentsJ and Kjs to be included in (25) can be based on the leave-one-curve-out cross-validation method [9] or by the fraction of variance explained (FVE) by the first J components [4]. One can also adopt AIC [10] or BIC [11] type of criteria, see Yao et al. [7] for one-dimensional function data.For bivariate functional data, a pseudo-Gaussian log-likelihood is given by(58)L^J=∑i=1n−Ni2log2π−Ni2logσ^2−12σ^2∑m=1Mi∑l=1LimYiml−μ^Siml,Tim−∑j=1Jξ^jTimψ^jSiml2,where Ni=∑m=1MiLim. One can choose J through minimizing aicJ=−L^J+J (resp., bicJ=−L^J+Jlogn) with respect to J.For each1≤j≤J, define(59)L^jKj=∑i=1n−Mi2log2π−Mi2logσ^2−12σ^2∑m=1Miξ^ijTim−∑k=1Kjζ^ijkϕ^jkTim2.The number of componentsKj is selected by minimizing aicKj=−L^jKj+Kj (resp., bicKj=−L^jKj+Kjlogn) with respect to Kj.



---

*Source: 6086983-2020-01-13.xml*