# Limit Properties of the Largest Entries of High-Dimensional Sample Covariance and Correlation Matrices

Authors: Xue Ding
Journal: Mathematical Problems in Engineering
 (2021)

---

## Abstract

In this paper, we consider the limit properties of the largest entries of sample covariance matrices and the sample correlation matrices. In order to make the statistics based on the largest entries of the sample covariance matrices and the sample correlation matrices more applicable in high-dimensional tests, the identically distributed assumption of population is removed. Under some moment’s assumption of the underlying distribution, we obtain that the almost surely limit and asymptotical distribution of the extreme statistics as both the dimensionp and sample size n tend to infinity.


---

## Body

## 1. Introduction

The sample covariance matrices play an important role in multivariate statistical inference. Its limiting properties are fundamental in hypothesis testing, principal component analysis, factor analysis, and discrimination analysis. The sample correlation matrix is another popular random matrix that plays an important role in independence test. Under classical statistical settings, where the number of variables is fixed, sample covariance matrix and correlation matrix perform well. However, in modern applications, the dimension of data is often high. Under such high-dimensional settings, sample covariance matrices and correlation matrices become inaccurate due to “curse of dimensionality.” Despite their extensive use in practice, under high-dimensional assumption, there are many theoretical understandings of spectral properties of sample covariance under the high-dimensional setting where the number of samples and variables both grow large at the same rate, see [1, 2]. The sample correlation matrices have many same spectral properties as sample covariance matrices under some assumptions, such as almost surely limit of extreme eigenvalue and the convergence of empirical eigenvalue distribution, see [3]. However, the CLTs for linear spectral statistics is different, see [4, 5].The study of the largest entries of sample correlation matrices is important for independence test. As both dimensionp and sample size n tend to infinity, for i.i.d. random variable Jiang [6] prove that the asymptotical distribution is extreme distribution of type I. Later, Li and Rosalsky [7] and Li et al. [8, 9] relax that moment assumptions. Li and Xue [10] and Li et al. [11] studied the asymptotic joint distribution of the maximum test statistic and the quadratic test statistic for one sample covariance test. Recently, Yu et al. [12] provide a new perspective to exploit the full potential of quadratic form statistics and maximum form statistics for testing high-dimensional covariance matrices. The extreme-value form statistic also can be used in Zheng et al. [13] to establish testing for correlation structures under high-dimensional setting. These studies related to the largest entries of sample correlation matrices make it more interesting to investigate the asymptotic distribution of the largest entries of sample covariance and correlation matrices. The i.i.d. assumption is usually needed in the literature. Liu and Lin [14] relax the independent assumption but require the underlying sequence to be the strictly stationary. In this paper, we remove the identically distributed condition and established limit and asymptotic distribution of the largest entries of sample covariance and correlation matrices. The rest of the paper is organized as follows. In Section 2, we give the main results; in Section 3, we first present some useful lemmas and then give the proofs of the main results.


## 2. Main Results

Suppose thatx1,x2,…,xn are n independent observations of the p-dimensional real-valued random vector x. Assume that xj=xij,1≤i≤pT denote the j-observation. Let Xn=xij be a p×n data matrix and ρ^ij be the sample correlation coefficient between the ith and jth rows of Xn. That is,(1)ρ^ij=∑k=1nxik−x¯ixjk−x¯j∑k=1nxik−x¯i2∑k=1n∑k=1nxjk−x¯j2,where x¯i=1/n∑k=1nxik. Then, Rn:=ρ^ij is the p×p sample correlation matrix generated by Xn. Let ωij=1/n∑k=1nxik−x¯ixjk−x¯j; then, the sample covariance matrix is defined as Sn=ωij. Let Tn and Ln denote the largest entries of off-diagonal elements of Sn and Rn, respectively, that is,(2)Tn=max1≤i<j≤pωij,Ln=max1≤i<j≤pρ^ij.In this paper, we mainly study the asymptotic properties ofTn,Ln. Assume that(i)
p and n both tend to infinity and p/n⟶y∈0,∞(ii)
For each1≤j≤n, the component of xj=xij,1≤i≤pT are independent(iii)
Exik=0andExik2=1, for all 1≤i≤p,1≤k≤n, and supi,kExik30+τ<∞, for some τ>0Theorem 1.
Under above assumptions (i)–(iii), we have that(3)limn⟶∞nlognTn=2a.s..Theorem 2.
Under above assumptions (i)–(iii), we have that(4)PnTn2−an≤t⟶exp−18πe−t/2,as n⟶∞, for any t∈R, where an=4logp+loglogp.The limiting distribution appearing in Theorem1 is called the extreme distribution of type I. Jiang [6] has proved that Theorems 1 and 2 hold if xik are i.i.d. random variables under some moment conditions.Theorem 3.
Under above assumptions (i)–(iii), we have that(5)limn⟶∞nlognLn=2a.s..Theorem 4.
Under above assumptions (i)–(iii), we have that(6)PnLn2−an≤t⟶exp−18πe−t/2,as n⟶∞, for any t∈R, where an=4logp+loglogp.Remark 1.
Note thatx1,x2,…,xn are n independent observations of the p-dimensional real-valued random vector x; therefore, they are independent and identically distributed random vectors.Remark 2.
Our proofs rely on Lemma1; therefore, the moment condition may be relaxed if the convergence rate in Lemma 1 can be improved; this question will be left for later.The statistics inference for ultra-high-dimensional data have been studied a lot these years because of its wide applications in many areas. In these applications, the dimensionp can be much larger than the sample size n; it may happen that p/n⟶∞. Cai and Jiang [15] investigate the limit properties of Ln for p≫n for i.i.d. random variables under more stronger moment assumption. It is interesting problem that can the identically distributed assumption be removes in the setting p/n⟶∞. We will investigate this topic in the future.


## 3. Proof of Main Results

We first introduce some lemmas that are used for the proofs of main results. Letη1,…,ηn be independent random variables such that Eηj=0,Eηj2=σj2,Eηjk<∞,j=1,2,…,n, for integer k≥3. Denote fjandγvj the characteristic function and v−order cumulant of ηj, respectively. Let Bn2=∑j=1nσj2 and Fn be the distribution function of Bn−1∑j=1nηj. Set(7)λν,n=nν−2/2Bn−ν∑j=1nγνj,ν=3,4,…,k,Qν,nx=∑′−1ν+2sΦν+2sx∏m=1ν1km!λm+2,nm+2!km,where Φx is distribution function of standard normal distribution and the summation ∑′ is taken over all nonnegative integer solutions k1,k2,…,kv of the equations:(8)k1+k2+⋯+kv=s,k1+2k2+⋯+vkv=v.For each realx, let(9)Ynj=ηjIηj≤Bn,Znjx=ηjIηj≤Bn1+x,Wnjx=ηjIηj>Bn1+x,j=1,2,…,n.Lemma 1 (see Theorem1 in [16]).
Suppose thatη1,…,ηn are independent random variables such that Eηj=0,Eηj2=σj2,Eηjk<∞,j=1,2,…,n, for integer k≥3. Let Bn2=∑j=1nσj2. Then, we have(10)Fnx−Φx−∑ν=1k−1Qν,nxn−1/2≤Ck1+x−kBn−k∑j=1nEWnxk+1+x−k−1Bn−k−1×∑j=1nEZnxk+1+1+x−k−1n1/2kk+1×supt≥δn1n∑j=1nfjt+12nn,where δn=1/12Bn2∑j=1nEYnj3−1andCk means absolute constants only depending on k.Lemma 2 (see Lemma A.1 in [6]).
Ifηn,n≥1 are i.i.d. random variables with Eη1=0, Eη1p<∞, p≥1, and Sn=∑i=1nηi, then(11)ESnp=Onp/2,ifp≥2,On,if1≤p<2.Lemma 3 (see Lemma A.2 in [6]).
Letηk,1≤k≤n be independent symmetric random variables and Sn=∑k=1nηk. Then, for each integer j≥1, there exist positive numbers Cj and Dj depending only on j such that, for all t>0,(12)PSn≥2jt≤CjPmax1≤k≤nηj≥t+DjPSn≥tj.Lemma 4 (see Lemma A.4 in [6]).
LetI be an index set and Bα,α∈I be a set of subsets of I; that is, Bα⊂I, for each α∈I. Let also ηα,α∈I be random variables. For a given t∈ℝ, set λ=∑α∈IPηα>t. Then,(13)Pmaxα∈Iηα≤t−e−λ≤1∧λ−1b1+b2+b3,where(14)b1=∑α∈I∑β∈BαPηα>tPηβ>t,b2=∑α∈I∑α≠β∈BαPηα>t,ηβ>t,b3∑α∈IEPηα>t|σηβ,β∉Bα−Pηα>t,and σηβ,β∉Bα is the σ−algebra generated by ηβ,β∉Bα. In particular, if ηα is independent of ηβ,β∉Bα for each α, then b3=0.Lemma 5 (see Theorem3 in [6]).
Letηi,1≤i≤n be independent random variables with mean zero. Assume max1≤i≤nEηiβ<∞, for some β>2. Then, for any ρ>0 and t>0,(15)P1n∑i=1nηi≥t≤Mβnρβ−1+Kne−tρ2/2M2,where(16)Ms≔1n∑i=1nEηisfors∈0,β,Kn≔expt3nρ−1/23M22e2tnρ−1/2/M2,and tρ=t−Mβnρ1−β+1/2.Now, we investigate the limit properties ofWn defined as follows:(17)Wn=1n×max1≤i<j≤p∑k=1nxikxjk.Proposition 1.
Under above assumptions (i)–(iii), we have that(18)limn⟶∞nlognWn=2a.s..Proposition 2.
Under above assumptions (i)–(iii), we have that(19)PnWn2−an≤t⟶exp−18πe−t/2,as n⟶∞, for any t∈R, where an=4logp+loglogp.Proof.
of Proposition1. We only need to prove that(20)limsupn⟶∞nlognWn≤2a.s.,(21)liminfn⟶∞nlognWn≥2a.s..
In order to prove (20), we only need to prove that, for any δ>0,(22)limsupn⟶∞nlognWn≤2+δa.s..
Letm be a integer satisfying that mδ>4; then, we have, for sufficient large integer d,(23)PdmlogdmWdm>2+δ=Pmax1≤i<j≤pdm1dm∑k=1dmxikxjk>2+δlogdm≤4yd2mmax1≤i<j≤pdmP1dm∑k=1dmxikxjk>2+δlogdm.
By Lemma1, we have that(24)max1≤i<j≤pdmP1dm∑k=1dmxikxjk>2+δlogdm=O1dm2+δ.
Thus, we have that∑d=1∞Pdm/logdmWdm>2+δ<∞. Then, by Borel–Cantelli lemma, we conclude that(25)limsupd⟶∞dmlogdmWdm≤2+δa.s..
Fordm≤n<d+1m, we have that(26)maxdm≤n<d+1mnWn−dmWdm≤maxdm≤n<d+1mmax1≤i<j≤d+1m∑k=dm+1nxikxjk.
Note that(27)Pmaxdm≤n<d+1mmax1≤i<j≤d+1m∑k=dm+1nxikxjk>δdmlogdm≤2d+12mmax1≤i<j≤d+1mPmaxdm≤n<d+1m∑k=dm+1nxikxjk>δdmlogdm.
Then, by Ottaviani’s inequality, we have that, for sufficient larged,(28)Pmaxdm≤n<d+1m∑k=dm+1nxikxjk≥δdmlogdm≤P∑k=dm+1xjkmxikxjk≥δdmlogdm/2mindm≤n<d+1mP∑k=n+1xjkmxikxjk≤δdmlogdm/2≤2P∑k=dm+1xjkmxikxjk≥δdmlogdm2.
By Lemma1, we know that, for sufficient large d,(29)P∑k=dm+1d+1mxikxjk≥δdmlogdm2=O1d4m.
Then,(30)Pmaxdm≤n<d+1mmax1≤i<j≤d+1m∑k=dm+1nxikxjk>δdmlogdm=O1d2mis summable. Noticing the fact that d+1mlogd+1m/dmlogdm⟶1, we conclude that(31)limsupd⟶∞maxdm≤n≤d+1mdmlogdmWdm−nlognWn≤δ,a.s..
Sinceδ is arbitrary, combining with (25), we know that (20) is true.
Now, we prove (21). In Lemma 4, take I=i,j;1≤i<j≤p. For α=i,j∈I, set Bα=k,l∈I;one ofkandl=iorjbutk,l≠α, ηα=∑k=1nxikxjk, t=2−δnlogn. Note that ηα is independent of ηβ,β∉Bα, and we have(32)PnWn≤t=Pmaxα∈Iηα≤t≤e−λn+b1,n+b2,n.
Here,(33)λn=∑α∈IP|ηα|>t=∑1≤i<j≤pP|∑k=1nxikxjk|>2−δnlogn,b1,n=∑α∈I∑β∈BαPηα>tPηβ>t≤2p3max1≤i<j≤pP2∑k=1nxikxjk>2−δnlogn,b2,n=∑α∈I∑α≠β∈BαPηα>t,ηβ>t≤2p3max1≤i<j≤p,1≤l≠j≤pP∑k=1nxikxjk>2−δnlogn,∑k=1nxikxlk>2−δnlogn.
Remember thatxikxjk is a sequence of independent random variables with mean 0 and variance 1. From the moment assumption (iii), by Lemma 1, we know that, for sufficiently large n,(34)P∑k=1nxikxjk>2−δnlogn−12−δ2πlogn⋅1n2−δ2/2≤1n2−δ′,where 1>δ′>δ is a constant. Combining with later Lemma 6, we obtain that, for sufficiently large n,(35)e−λn=Oe−nδ′,b1,n=O1n,b2,n=O1n.
Then, from (32) and (35), we conclude that there is a ε>0 depending on δ such that(36)PnWn≤2−δnlogn=O1nϵ.
Choose an integerm such that mε>1. Then, PdmWdm≤2−δdmlogdm=O1/dmε. Since ∑dd−mε<∞, by the Borel–Cantelli lemma, we have that(37)liminfd⟶∞dmlogdmWdm≥2−δa.s.,for any δ∈0,1. Note that, for dm≤n<d+1m,(38)infdm≤n<d+1mnWn≥dmWdm−maxdm≤n<d+1mmax1≤i<j≤pd+1m∑k=dm+1nxikxjk.
By (30) and (37) and the fact that d+1mlogd+1m/dmlogdm⟶1, we have that (21) holds.Proof.
of Proposition2. Note that(39)PnWn2−an≤t=Pmax1≤i<j≤p∑k=1nxikxjk≤nt+an.
Now, we apply Lemma4 to prove this proposition. Take I=i,j;1≤i<j≤p, for α=i,j∈I; set Xα=∑k=1nxikxjk/n and Bα=k,l∈I;one ofkandl=iorj,butk,l≠α. Define(40)λn=∑1≤i<j≤pPXα>t+an.
Note the sequencexikxjk is a sequence of independent random variables with mean 0 and variance 1. It follows from Lemma 1 that, for 1≤i,j≤p,(41)PXα>t+an=P1n∑k=1nxikxjk>t+an=e−t/2t+an2πp−2+O1p2+ϵ,for some constant ε>0. Thus, it follows that(42)λn⟶e−t/28π.
Observe thatXα is independent of Xβ;β∈I∖Bα, for any α=i,j∈I. Define that(43)b1n=∑α∈I∑β∈BαPXα>t+anPXβ>t+anb2n∑α∈I∑α≠β∈BαPXα>t+an,Xβ>t+an.
It can be seen that there at most2p terms in Bα. Thus, by (41), we have(44)b1n≤12p2−p⋅2p⋅max1≤i<j≤pP2Xα>t+an=O1p.
Similarly,(45)b2n≤pp2−p×max1≤i<j≤p,1≤l≠j≤nP∑k=1nxikxjk>nt+an,∑k=1nxikxlk>nt+an.
By Lemma6, we obtain that b2n=On−b for some constant b>0 under the moment assumption. Then, by Lemma 4, this proposition is concluded.Proof.
of Theorems1 and 2. Note that(46)∑k=1nxik−x¯ixjk−x¯j−∑k=1nxikxjk≤nx¯ix¯j.
Therefore,(47)Wn−Tn≤maxi≠jx¯ix¯j≤max1≤i≤px¯i2.
Now, we estimate the last term in the above equation:(48)Pn1/4logn1/4max1≤i≤px¯i≥ϵ≤p×max1≤i≤pPn1/4logn1/4x¯i≥ϵ.
By Lemma1, we have that, for every i,(49)max1≤i≤pPn1/4logn1/4x¯i≥ϵ=P1n∑k=1nxik≥ϵn1/4logn−1/4≤Cn−2.
Thus,(50)nlognmax1≤i≤px¯i2⟶0,a.s..
Combining with (47), we have that(51)nlognWn−Tn⟶0,a.s..
Then, Theorem1 follows. From above equation and Proposition 1, we know that Wn+Tn=Ologn1/2n−1/2. Applying (47) and (50), we obtain that(52)nWn2−Tn2=nWn−TnWn+Tn⟶0,a.s..
Theorem2 then follows.Proof.
of Theorems3 and 4. Under conditions (i)–(iii), we have from the triangle inequality that(53)Ln−Tn≤max1≤i<j≤pρ^ij−ωij≤Tnmax1≤i<j≤p1−n∑k=1nxik−x¯i2n∑k=1nxjk−x¯j2≤Tn×D1−2D22+2D2,where(54)D1=min1≤i≤p∑k=1nxik−x¯i2n,D2=max1≤i≤p∑k=1nxik−x¯i2n−1.
From the fact thatx−1≤x2−1, for any x>0, we have that(55)D2≤max1≤i≤p∑k=1nxik2n−1+max1≤i≤px¯i2.
By Lemma1 and moment condition (iii), we know that(56)Plogn×max1≤i≤p∑k=1nxik2−nn≥ϵ≤p×max1≤i≤pP1n∑k=1nxik2−n≥ϵn1/2logn−1≤n−2.
We can see that the sum above is summable. Similarly, we have that(57)Plognmax1≤i≤px¯i≥ϵ≤p×max1≤i≤pP1n∑k=1nxik≥ϵn1/2logn−1/2=On−2.
It is also summable. Thus, we conclude thatlogn×D2⟶0 almost surely. It can be seen from above discussion that D1⟶1 almost surely. Combining these facts with Theorem 1, we have that(58)nlognLn−Tn⟶0,a.s.,nLn2−Tn2=nLn−TnLn+Tn⟶0,a.s..
Theorems3 and 4 then follow.Lemma 6.
Letvn=nt+anandun=2−δnlogn be defined before. Then, we have, for some constant b>0,(59)max1≤i<j≤p,1≤l≠i≤pP∑k=1nxikxjk>vn,∑k=1nxikxlk>vn=On−3−b,max1≤i<j≤p,1≤l≠i≤pP∑k=1nxikxjk>un,∑k=1nxikxlk>un=On−7/2,as n⟶∞.Proof.
Note that, for eachi, the sequence xik is independent identically distributed random variables and xik,xik, and xlk are independent. With the moment assumption (iii), the conclusion can be obtained by modifying Lemma 2.1 in [6].



---

*Source: 9635202-2021-09-29.xml*