# Large-Deviation Results for Discriminant Statistics of Gaussian Locally Stationary Processes

Authors: Junichi Hirukawa
Journal: Advances in Decision Sciences
 (2012)

---

## Abstract

This paper discusses the large-deviation principle of discriminant statistics for Gaussian locally stationary processes. First, large-deviation theorems for quadratic forms and the log-likelihood ratio for a Gaussian locally stationary process with a mean function are proved. Their asymptotics are described by the large deviation rate functions. Second, we consider the situations where processes are misspecified to be stationary. In these misspecified cases, we formally make the log-likelihood ratio discriminant statistics and derive the large deviation theorems of them. Since they are complicated, they are evaluated and illustrated by numerical examples. We realize the misspecification of the process to be stationary seriously affecting our discrimination.


---

## Body

## 1. Introduction

Consider a sequence of random variablesS1,S2,… converging (in probability) to a real constant c. By this we mean that Pr{|ST-c|>ε}→0 as T→∞ for all ε>0. The simplest setting in which to obtain large-deviation results is that considering sums of independent identically distributed (iid) random variables on the real line. For example, we would like to consider the large excursion probabilities of sums as the sample average:
(1.1)ST=T-1∑i=1T‍Xi,
where the Xi, i=1,2,…, are i.i.d., and T approaches infinity. Suppose that E(Xi)=m exists and is finite. By the law of large numbers, we know that ST should be converging to m. Hence, c is merely the expected value of the random process. It is often the case that not only does Pr{|ST-c|>ε} go to zero, but it does so exponentially fast. That is,
(1.2)Pr{|ST-c|>ε}≈K(ε,c,T)exp⁡{-TI(ε,c)},
where K(ε,c,T) is a slowly varying function of T (relative to the exponential), and I(ε,c) is a positive quantity. Loosely, if such a relationship is satisfied, we will say that the sequence {Sn} satisfies a large-deviation principle. Large-deviation theory is concerned primarily with determining the quantities I(ε,c) and (to a lesser extent) K(ε,c,T). The reason for the nomenclature is that for a fixed ε>0 and a large index T, a large-deviation from the nominal value occurs if |ST-c|>ε. Large-deviation theory can rightly be considered as a generalization or extension of the law of large numbers. The law of large numbers says that certain probabilities converge to zero. Large-deviation theory is concerned with the rate of convergence. Bucklew [1] describes the historical statements of large-deviation in detail.There have been a few works on the large-deviation theory for time series data. Sato et al. [2] discussed the large-deviation theory of several statistics for short- and long-memory stationary processes. However, it is still hard to find the large-deviation results for nonstationary processes. Recently, Dahlhaus [3, 4] has formulated an important class of nonstationary processes with a rigorous asymptotic theory, which he calls locally stationary. A locally stationary process has a time-varying spectral density whose spectral structure changes smoothly with time. There are several papers which discuss discriminant analysis for locally stationary processes (e.g., Chandler and Polonik [5], Sakiyama and Taniguchi [6], and Hirukawa [7]). In this paper, we discuss the large-deviation theory of discriminant statistics of Gaussian locally stationary processes. In Section 2 we present the Gärtner-Ellis theorem which establishes a large-deviation principle of random variables based only upon convergence properties of the associated sequence of cumulant generating functions. Since no assumptions are made about the dependency structure of random variables, we can apply this theorem to non-stationary time series data. In Section 3, we deal with a Gaussian locally stationary process with a mean function. First, we prove the large-deviation principle for a general quadratic form of the observed stretch. We also give the large-deviation principle for the log-likelihood ratio and the misspecified log-likelihood ratio between two hypotheses. These fundamental statistics are important not only in statistical estimation and testing theory but in discriminant problems. The above asymptotics are described by the large-deviation rate functions. In our stochastic models, the rate functions are very complicated. Thus, in Section 4, we evaluate them numerically. They demonstrate that the misspecifications of non-stationary has serious effects. All the proofs of the theorems presented in Section 3 are given in the Appendix.


## 2. Gärtner-Ellis Theorem

Cramér's theorem (e.g., Bucklew [1]) is usually credited with being the first large-deviation result. It gives the large-deviation principle for sums of independent identically distributed random variables. One of the most useful and surprising generalizations of this theorem is the one due to Gärtner [8] and, more recently, Ellis [9]. These authors established a large-deviation principle of random variables based only upon convergence properties of the associated sequence of moment generating functions Φ(ω). Their methods thus allow large-deviation results to be derived for dependent random processes such as Markov chains and functionals of Gaussian random processes. Gärtner [8] assumed throughout that Φ(ω)<∞ for all ω. By extensive use of convexity theory, Ellis [9] relaxed this fairly stringent condition.Suppose that we are given an infinite sequence of random variables{YT,T∈ℕ}. No assumptions are made about the dependency structure of this sequence. Define
(2.1)ψT(ω)≡T-1log⁡E{exp⁡⁡(ωYT)}.Now let us list two assumptions.Assumption 2.1.
ψ
(
ω
)
≡
lim
T
→
∞
ψ
T
(
ω
) exists for all ω∈ℝ, where we allow ∞ both as a limit value and as an element of the sequence {ψT(ω)}.Assumption 2.2.
ψ
(
ω
) is differentiable on Dψ≡{ω:ψ(ω)<∞}.Define the large-deviation rate function by(2.2)I(x)≡sup⁡ω{ωx-ψ(ω)};
this function plays a crucial role in the development of the theory. Furthermore, define
(2.3)ψ′(Dψ)≡{ψ′(ω):ω∈Dψ},
where ψ′ indicates the derivative of ψ. Before proceeding to the main theorem, we first state some properties of this rate function.Property 1.
I
(
x
) is convex.We remark that a convex functionI(·) on the real line is continuous everywhere on DI≡{x:I(x)<∞}, the domain of I(·).Property 2.
I
(
x
) has its minimum value at m=limT→∞T-1E(YT), and I(m)=0.We now state a simple form of a general large-deviation theorem which is known as the Gärtner and Ellis theorem (e.g., Bucklew [1]).Lemma 2.3 (Gärtner-Ellis).
Let(a,b) be an interval with [a,b]∩DI≠∅. If Assumption 2.1 holds and a<b, then
(2.4)limsup⁡T→∞T-1log⁡⁡Pr{T-1YT∈[a,b]}≤-inf⁡x∈[a,b]I(x).
If Assumptions2.1 and 2.2 hold and (a,b)⊂ψ′(Dψ), then
(2.5)liminf⁡T→∞T-1log⁡⁡Pr{T-1YT∈(a,b)}≥-inf⁡x∈(a,b)I(x).Large-deviation theorems are usually expressed as two separate limit theorem: an upper bound for closed sets and a lower bound for open sets. In the case of interval subsets ofℝ, it can be guaranteed that the upper bound equals the lower bound by the continuity of I(·). For the applications that we have in mind, the interval subsets will be sufficient.


## 3. Large-Deviation Results for Locally Stationary Processes

In this section, using the Gärtner-Ellis theorem, we develop the large-deviation principle for some non-stationary time series statistics. When we deal with non-stationary processes, one of the difficult problems to solve is how to set up an adequate asymptotic theory. To overcome this problem, an important class of non-stationary process has been formulated in rigorous asymptotic framework by Dahlhaus [3, 4], called locally stationary processes. Locally stationary processes have time-varying densities, whose spectral structures smoothly change in time. We give the precise definition of locally stationary processes which is due to Dahlhaus [3, 4].Definition 3.1.
A sequence of stochastic processesXt,T(t=1,…,T;T≥1) is called locally stationary with transfer function A° and trend μ if there exists a representation:
(3.1)Xt,T=μ(tT)+∫-ππ‍exp⁡(iλt)At,T°(λ)dξ(λ),
where  (i)
ξ(λ) is a stochastic process on [-π,π] with ξ(λ)¯=ξ(-λ) and
(3.2)cum{dξ(λ1),…,dξ(λk)}=η(∑j=1k‍λj)νk(λ1,…,λk-1)dλ1⋯dλk,
where cum{…} denotes the cumulant of k-th order, ν1=0, ν2(λ)=1, |νk(λ1,…,λk-1)|≤constk for all k and η(λ)=∑j=-∞∞‍δ(λ+2πj) is the period 2π extension of the Dirac delta function. To simplify the problem, we assume in this paper that the process Xt,T is Gaussian, namely, we assume that νk(λ)=0 for all k≥3;(ii)
there exists constantK and a 2π-periodic function A:[0,1]×ℝ→ℂ with A(u,λ)=A(u,-λ)¯ and
(3.3)sup⁡t,λ|At,T°(λ)-A(tT,λ)|≤KT-1,
for all T. A(u,λ) and μ(u) are assumed to be continuous in u.The functionf(u,λ):=|A(u,λ)|2 is called the time-varying spectral density of the process. In the following, we will always denote by s and t time points in the interval [1,T], while u and v will denote time points in the rescaled interval [0,1], that is u=t/T.We discuss the asymptotics away from the expectation of some statistics used for the problem of discriminating between two Gaussian locally stationary processes with specified mean functions. Suppose that{Xt,T,t=1,…,T;T≥1} is a Gaussian locally stationary process which under the hypothesis Πj has mean function μ(j)(u) and time-varying spectral density f(j)(u,λ) for j=1,2. Let XT=(X1,T,…,XT,T)′ be a stretch of the series {Xt,T}, and let p(j)(·) be the probability density function of XT under Πj(j=1,2). The problem is to classify XT into one of two categories Π1 and Π2 in the case that we do not have any information on the prior probabilities of Π1 and Π2.SetμT(j)={μ(j)(1/T),…,μ(j)(T/T)}′ and ΣT(j)=ΣT(A(j),A(j)), where
(3.4)ΣT(A,B)={∫-ππ‍As,T°(λ)Bt,T°(-λ)exp⁡(iλ(s-t))dλ}s,t=1,…,T.Initially, we make the following assumption.Assumption 3.2.
(i) We observe a realisationX1,T,…,XT,T of a Gaussian locally stationary process with mean function μ(j) and transfer function A(j)°, under Πj, j=1,2;
(ii) theA(j)(u,λ) are uniformly bounded from above and below, and are differentiable in u and λ with uniformly continuous derivatives (∂/∂u)(∂/∂λ)A(j);
(iii) theμ(j)(u) are differentiable in u with uniformly continuous derivatives.
In time series analysis, the class of statistics which are quadratic forms ofXT is fundamental and important. This class of statistics includes the first-order terms (in the expansion with respect to T) of quasi-Gaussian maximum likelihood estimator (QMLE), tests and discriminant statistics, and so forth Assume that G° is the transfer function of a locally stationary process, where the corresponding G satisfies Assumption 3.2 (ii) and g(u) is a continuous function of u which satisfies Assumption 3.2 (iii), if we replace A(j) by G and μ(j)(u) by g(u), respectively. And set GT≡ΣT(G,G), fG(u,λ)≡|G(u,λ)|2, gT≡{g(1/T),…,g(T/T)}′ and QT≡XT′GT-1XT+gT′XT. Henceforth, E(j)(·) stands for the expectation with respect to p(j)(·). Set ST(j)(Q)≡QT-E(j)(QT) for j=1,2. We first prove the large-deviation theorem for this quadratic form QT of XT. All the proofs of the theorems are in the Appendix.Theorem 3.3.
Let Assumption3.2 hold. Then under Π1,
(3.5)lim⁡T→∞T-1log⁡⁡Pr⁡1{T-1ST(1)(Q)>x}=inf⁡ω{ψQ(ω;f(1))-ωmax⁡(x,0)},
and under Π2,
(3.6)lim⁡T→∞T-1log⁡⁡Pr2{T-1ST(2)(Q)<x}=inf⁡ω{ψQ(ω;f(2))-ωmin⁡(x,0)},
where for j=1,2, ψQ(ω;f(j)) equals
(3.7)14π∫01‍∫-ππ‍[log⁡fG(u,λ)fG(u,λ)-2ωf(j)(u,λ)-2ωf(j)(u,λ)fG(u,λ)+ω2{fG(u,0)g(u)+2μ(j)(u)}2f(j)(u,0)2π{fG(u,0)-2ωf(j)(u,0)}fG(u,0)]dλdu.Next, one considers the log-likelihood ratio statistics. It is well known that the log-likelihood ratio criterion:(3.8)ΛT≡log⁡p(2)(XT)p(1)(XT)
gives the optimal discrimination rule in the sense that it minimizes the probability of misdiscrimination (Anderson [10]). Set ST(j)(Λ)≡ΛT-E(j)(ΛT) for j=1,2. For discrimination problem one gives the large-deviation principle for ΛT.Theorem 3.4.
Let Assumption3.2 hold. Then under Π1,
(3.9)lim⁡T→∞T-1log⁡⁡Pr1{T-1ST(1)(Λ)>x}=inf⁡ω{ψL(ω;f(1),f(2))-ωmax⁡(x,0)},
where ψL(ω;f(1),f(2)) equals
(3.10)14π∫01‍∫-ππ‍[ω2{μ(1)(u)-μ(2)(u)}2f(1)(u,0)2π{(1-ω)f(2)(u,0)+ωf(1)(u,0)}f(2)(u,0)log⁡f(2)(u,λ)(1-ω)f(2)(u,λ)+ωf(1)(u,λ)+ω{f(1)(u,λ)f(2)(u,λ)-1}+ω2{μ(1)(u)-μ(2)(u)}2f(1)(u,0)2π{(1-ω)f(2)(u,0)+ωf(1)(u,0)}f(2)(u,0)]dλdu.
Similarly, under Π2,
(3.11)lim⁡T→∞T-1log⁡⁡Pr2{T-1ST(2)(Λ)<x}=inf⁡ω{ψL(-ω;f(2),f(1))-ωmin⁡(x,0)}.In practice, misspecification occurs in many statistical problems. We consider the following three situations. Although actually{Xt,T} has the time-varying mean functions μ(j)(u) and the time-varying spectral densities f(j)(u,λ), under Πj, j=1,2, respectively,  (i)
the mean functions are misspecified toμ(j)(u)≡0, j=1,2;(ii)
the spectral densities are misspecified tof(j)(u,λ)≡f(j)(0,λ), j=1,2;(iii)
the mean functions and the spectral densities are misspecified toμ(j)(u)≡0 and f(j)(u,λ)≡f(j)(0,λ), j=1,2. Namely, XT is misspecified to stationary.In each misspecified case, one can formally make the log-likelihood ratio in the form:(3.12)M1,T=12[log⁡{|Σ(1)||Σ(2)|}+XT′{ΣT(1)-1-ΣT(2)-1}XT],M2,T=12[log⁡{|Σ~(1)||Σ~(2)|}+(XT-μT(1))′Σ~T(1)-1(XT-μT(1))-(XT-μT(2))′Σ~T(2)-1(XT-μT(2))],M3,T=12[log⁡{|Σ~(1)||Σ~(2)|}+XT′{Σ~T(1)-1-Σ~T(2)-1}XT],
where
(3.13)Σ~T(j)={∫-ππ‍exp⁡(iλ(t-s))f(j)(0,λ)dλ}s,t=1,…,T.
Set ST(j)(Mk)≡Mk,T-E(j)(Mk,T) for j=1,2 and k=1,2,3. The next result is a large-deviation theorem for the misspecified log-likelihood ratios Mk,T. It is useful in investigating the effect of misspecification.Theorem 3.5.
Let Assumption3.2 hold. Then under Π1,
(3.14)lim⁡T→∞T-1log⁡⁡Pr1{T-1ST(1)(Mk)>x}=inf⁡ω{ψMk(ω;f(1),f(2),μ(1),μ(2))-ωmax⁡(x,0)},
where ψM1(ω;f(1),f(2),μ(1),μ(2)) equals
(3.15)14π∫01‍∫-ππ‍[[ωμ(1)(u){f(1)(u,0)-f(2)(u,0)}]22π{(1-ω)f(2)(u,0)+ωf(1)(u,0)}f(1)(u,0)f(2)(u,0)log⁡f(2)(u,λ)(1-ω)f(2)(u,λ)+ωf(1)(u,λ)+ω{f(1)(u,λ)f(2)(u,λ)-1}+[ωμ(1)(u){f(1)(u,0)-f(2)(u,0)}]22π{(1-ω)f(2)(u,0)+ωf(1)(u,0)}f(1)(u,0)f(2)(u,0)]dλdu,ψM2(ω;f(1),f(2),μ(1),μ(2)) equals
(3.16)14π∫01‍∫-ππ‍[log⁡f(1)(0,λ)f(2)(0,λ)f(1)(0,λ)f(2)(0,λ)-ωf(1)(u,λ){f(2)(0,λ)-f(1)(0,λ)}+ω{f(1)(u,λ)f(2)(0,λ)-f(1)(u,λ)f(1)(0,λ)}+ω2{μ(2)(u)-μ(1)(u)}2f(1)(u,0)f(1)(0,0)/f(2)(0,0)2π[f(1)(0,0)f(2)(0,0)-ωf(1)(u,0){f(2)(0,0)-f(1)(0,0)}]]dλdu
and ψM3(ω;f(1),f(2),μ(1),μ(2)) equals
(3.17)14π∫01‍∫-ππ‍[log⁡f(1)(0,λ)f(2)(0,λ)f(1)(0,λ)f(2)(0,λ)-ωf(1)(u,λ){f(2)(0,λ)-f(1)(0,λ)}+ω{f(1)(u,λ)f(2)(0,λ)-f(1)(u,λ)f(1)(0,λ)}+ω2μ(1)(u)2f(1)(u,0){f(1)(0,0)-f(2)(0,0)}2/{f(1)(0,0)f(2)(0,0)}2π[f(1)(0,0)f(2)(0,0)-ωf(1)(u,0){f(2)(0,0)-f(1)(0,0)}]]dλdu.
Similarly, under Π2,
(3.18)lim⁡T→∞T-1log⁡⁡Pr2{T-1ST(2)(Mk)<x}=inf⁡ω{ψMk(-ω;f(2),f(1),μ(2),μ(1))-ωmin⁡(x,0)}.Now, we turn to the discussion of our discriminant problem of classifyingXT into one of two categories described by two hypotheses a follows:
(3.19)Π1:μ(1)(u),f(1)(u,λ),Π2:μ(2)(u),f(2)(u,λ).
We use ΛT as the discriminant statistic for the problem (3.19), namely, if ΛT>0 we assign XT into Π2, and otherwise into Π1. Taking x=-limT→∞T-1E(1)(ΛT) in (3.9), we can evaluate the probability of misdiscrimination of XT from Π1 into Π2 as follows:
(3.20)P(2∣1)≡Pr1{ΛT>0}≈exp⁡[Tinf⁡ω{14π∫01‍∫-ππ‍[ω(ω-1){μ(1)(u)-μ(2)(u)}22π{(1-ω)f(2)(u,0)+ωf(1)(u,0)}log⁡f(1)(u,λ)ωf(2)(u,λ)1-ω(1-ω)f(2)(u,λ)+ωf(1)(u,λ)+ω(ω-1){μ(1)(u)-μ(2)(u)}22π{(1-ω)f(2)(u,0)+ωf(1)(u,0)}]dλdu}].Thus, we see that the rate functions play an important role in the discriminant problem.


## 4. Numerical Illustration for Nonstationary Processes

We illustrate the implications of Theorems3.4 and 3.5 by numerically evaluating the large-deviation probabilities of the statistics ΛT and Mk,T, k=1,2,3 for the following hypotheses:
(4.1)(Stationarywhitenoise)Π1:μ(1)(u)≡0,f(1)(u,λ)≡1,(Time-varyingAR(1))Π2:μ(2)(u)=μ(u),f(2)(u,λ)=σ(u)2|1-a(u)eiλ|2,
where μ(u)=(1/2)exp(-u2), σ(u)=(1/2)exp{-(u-1)2} and a(u)=(1/2)exp{-4(u-1/2)2}, u∈[0,1], respectively. Figure 1 plots the mean function μ(u) (the solid line), the coefficient functions σ(u) (the dashed line), and a(u) (the dotted line). The time-varying spectral density f(2)(u,λ) is plotted in Figure 2.Figure 1
The mean functionμ(u) (the solid line), the coefficient functions σ(u) (the dashed line), and a(u) (the dotted line).Figure 2
The time-varying spectral densityf(2)(u,λ).From these figures, we see that the magnitude of the mean function is large atu close to 0, while the magnitude of the time-varying spectral density is large at u close to 1.Specifically, we use the formulae in those theorems concerningΠ2 to evaluate the limits of the large-deviation probabilities:
(4.2)LDP(Λ)=lim⁡T→∞T-1log⁡Pr2{T-1ST(2)(Λ)<x},LDP(Mk)=lim⁡T→∞T-1log⁡Pr2{T-1ST(2)(Mk)<x},k=1,2,3.
Though the result is an asymptotic theory, we perform the simulation with a limited sample size. Therefore, we use some levels of x to expect fairness, that is, we take x=-0.1,-1,-10. The results are listed in Table 1.Table 1
The limits of the large-deviation probabilities ofΛT and Mk,T, k=1,2,3.
x
=
-
0.1
x
=
-
1
x
=
-
10
LDP
(
Λ
)
−0.012078
−0.562867
−9.460066
LDP
(
M
1
)
−0.009895
−0.486088
−8.857859
LDP
(
M
2
)
−0.000348
−0.026540
−0.703449
LDP
(
M
3
)
−0.000290
−0.022313
−0.629251For each valuex, the large-deviation rate of ΛT is the largest and that of M3,T is the smallest. Namely, we see that the correctly specified case is the best, and on the other hand the misspecified to stationary case is the worst. Furthermore, the large-deviation rates -LDP(M2) and -LDP(M2) are significantly small, comparing with -LDP(M1). This fact implies that the misspecification of the spectral density to be constant in the time seriously affects the large-deviation rate.Figures3, 4, 5, and 6 show the large-deviation probabilities of ΛT and Mk,T, k=1,2,3, for x=-1, at each time u and frequency λ.Figure 3
The time-frequency plot of the large-deviation probabilities ofΛT.Figure 4
The time-frequency plot of the large-deviation probabilities ofM1,T.Figure 5
The time-frequency plot of the large-deviation probabilities ofM2,T.Figure 6
The time-frequency plot of the large-deviation probabilities ofM3,T.We see that the large-deviation rate ofΛT keeps the almost constant value at all the time u and frequency λ. On the other hand, that of M1,T is small at u close to 0 and those of M2,T and M3,T are small at u close to 1 and λ close to 0. That is, the large-deviation probability of M1,T is violated by the large magnitude of the mean function, while those of M2,T and M3,T are violated by that of the time-varying spectral density. Hence, we can conclude the misspecifications seriously affect our discrimination.



---

*Source: 572919-2012-07-05.xml*