# Distributions Escaping to Infinity and the Limiting Power of the Cliff-Ord Test for Autocorrelation

Authors: Kairat T. Mynbaev
Journal: ISRN Probability and Statistics
 (2012)

---

## Abstract

We consider a family of proper random variables which converges to an improper random variable. The limit in distribution is found and applied to obtain a closed-form expression for the limiting power of the Cliff-Ord test for autocorrelation. The applications include the theory of characteristic functions of proper random variables, the theory of almost periodic functions, and the test for spatial correlation in a linear regression model.


---

## Body

## 1. Introduction

Improper random variables do not satisfy the conditionP(X∈R)=1; that is, they may take values outside the real line R. They are not used much by themselves, but there are situations when they arise as limits of proper random variables. In such cases, we say that a distribution escapes to infinity. The main problem considered in this paper is illustrated in the following in a simplified situation.Throughout the paper we denote byχA the indicator of a set A. Let g=(1/2)χ[-1,1] be a uniform distribution on the segment [-1,1]. Consider the family of densities gλ(t)=λg(λt), where λ>0. The total mass is constant: ∫R‍gλ(t)dt=1 for any λ>0. In Bayesian estimation, improper priors are obtained by letting λ→0. In this case, there are two effects at work: the support suppgλ=[-1/λ,1/λ] stretches out indefinitely and the height of the density maxgλ=λ/2 goes to zero. One might be tempted to think of the limit G of {gλ}, as λ→0, as an infinitesimally thin layer smeared over the whole real line. This notion would be wrong because
(1.1)|∫R‍gλ(t)φ(t)dt|≤λ2∫suppφ‍|φ(t)|dt⟶0,λ→0,
for any φ∈C0(R) (the set of continuous functions on R with bounded support suppφ). Thus, if the limit in distribution G of {gλ} exists, it vanishes on all elements of C0(R). By the definition of the support of a distribution [1, Chapter 1, Section 13], the support of G does not contain R. So instead of spreading the mass over R, it is more correct to say that the mass escapes to infinity.In this paper we provide a rigorous framework for treating more complex situations. To illustrate the arising complexities, let us look at the standard normal variableX with the density g(x)=(2π)-1/2exp(-x2/2). Let Xλ have the density gλ(x)=λg(λx)=λ(2π)-1/2e-(λx)2/2. Its characteristic function is
(1.2)EeitXλ=∫R‍ei(t/λ)xg(x)dx=Eei(t/λ)X=e-(t/λ)2/2.
The moment generating function of Xλ is EetXλ=e(t/λ)2/2. None of these expressions is good for characterizing the limit as λ→0. Further, let φ denote an arbitrary continuous and bounded function on R. In the expression
(1.3)∫R‍gλ(t)φ(t)dt=λ∫R‍g(λt)φ(t)dt,
the height of the density goes to zero, so the integrand converges to zero everywhere. However, the graph of the density stretches out from the origin. Therefore, the best majorant for the integrand is |g(0)φ(t)| which is generally not integrable. Thus, the dominated convergence theorem cannot be used to obtain convergence in distribution. While the case of collapsing density(λ→∞) is easily handled by the existing theory, the case of stretching out(λ→0) requires new tools which we develop here.Problem 1.
Describe the limit in distribution when the stretching-out is applied to a density along all or some variables.
This problem is solved in Theorem2.2 in Section 2 (case of all variables) and Theorem 2.6 (case of some variables). In their simplest form, those results reveal the main ideas. Let g be any summable even function on R (it may change sign and not integrate to unity). Let Mφ denote the generalized mean of φ over R:
(1.4)Mφ=limr→∞12r∫-rr‍φ(t)dt.
Then for a continuous and bounded function φ on R such that Mφ exists, one has
(1.5)∫R‍λg(λt)φ(t)dt⟶Mφ∫R‍g(t)dt,λ⟶0.Now suppose that the functiong depends on two variables and the stretching-out is applied with respect to the first of them. Let φ be continuous and bounded on R2. Denote M1φ, the result of application of (1.4) to φ with respect to the first argument,(1.6)(M1φ)(t2)=limr→∞12r∫-rr‍φ(t1,t2)dt1,
and let g1(t2)=∫R‍g(t1,t2)dt1. In cases when g is not a density, we call this function a marginal “density.”   Then, (1.5) can be used to prove
(1.7)∫R2‍λg(λt1,t2)φ(t)dt=∫R‍[∫R‍λg(λt1,t2)φ(t1,t2)dt1]dt2⟶∫R‍(M1φ)(t2)g1(t2)dt2,λ⟶0,
if g is integrable and spherically symmetric. As we show, the right sides of (1.5) and (1.7) determine distributions supported at infinity.Theorem2.2 is followed by two applications. One generalizes results from [2] on the link between jumps of a distribution function of a (proper) random variable and its characteristic function. Another application is to the fundamental theorem of H. Bohr on the Parseval identity for almost-periodic functions; see [3]. Theorem 2.6 is applied to the limiting power of the Cliff-Ord test for autocorrelation. To formulate the related problems, we need some notation.Consider a linear regression model(1.8)y=Xβ+u,Eu=0,var(u)=σ2Σ(ρ),
where X is an n×k matrix of rank k<n; a vector β∈Rk and a number σ2>0 are unknown parameters. The matrix Σ(ρ) is assumed to be a nonnegative function of the parameter ρ∈[0,a], with some a>0. ρ in applications characterizes the degree of autocorrelation. Testing for autocorrelation takes the form H0:ρ=0 versus Ha:ρ>0. The case ρ→a is of special interest for determining the limiting power of tests.Assuming thatΣ(ρ) is positive definite for ρ∈[0,a), denote ε=Σ-1/2(ρ)u and let g(ε) be the density of ε. The density of y is then given by [4, Equation (D.2)]
(1.9)fρ(y)=detΣ-1/2(ρ)g(Σ-1/2(ρ)(y-Xβ)).Problem 2.
Assuming that(1.10)rank(Σ-1(a))=n-1,
describe the limit in distribution of fρ.
This problem, as an intermediate step in the proof, was considered by Martellosio [4], and his answer is reproduced as follows. Let N(A) denote the null space of a matrix A and denote A(a)=Σ-1(a). Let Xβ+N(A(a)) be the translation by Xβ of the null space of A(a). Reference [4] states that in case (1.10)
(1.11)asρ⟶a,fρtendstoadegeneratedensitysupportedonXβ+N(A(a)),
see page 182. Equation (1.11) arises from the confusion between the stretching-out and collapsing. On page 159, rows 10 and 11, he remarks that (1.11) can be extended to the case rank(Σ-1(a))<n-1. He does not specify the mode of convergence, but, as we argue in Section 2, the convergence in distribution is the right one for the problem of the behavior of the limiting power of tests for autocorrelation. Unfortunately, (1.11) does not hold under the convergence in distribution. We prove this and solve Problem 2 in Theorem 2.7 for the general case 0<rank(Σ-1(a))<n.LetΦ be a critical region for rejecting H0 in favor of Ha. Denote
(1.12)βρ(Φ)=∫Φ‍fρ(y)dy,
the probability content of Φ under density (1.9) and define the limiting powerβ(Φ) as the limit of this probability content: β(Φ)=limρ→aβρ(Φ). Specifically, we consider critical regions that arise from the Cliff-Ord test, to be described now. With the regressor matrix X from (1.8) denote LX=X(X′X)-1X′,MX=I-LX. Under the spatial autocorrelation assumption, the regression disturbances follow
(1.13)u=ρWu+ε,Eε=0,var(ε)=σε2I,
where ε is a new disturbance and W is some known n×n matrix. The scalar ρ, which is unknown, determines the degree of correlation among the components of u. For testing the null H0:ρ=0 against the alternative Ha:ρ>0, Cliff and Ord [5] proposed a test that rejects the null if u′MXWMXu/u′MXu>c. Denote
(1.14)Φ={u∈Rn:u′MXWMXuu′MXu>c},Φ-={u∈Rn:u′MXWMXuu′MXu<c}.Problem 3.
Obtain a closed-form expression forβ(Φ).
Theorem2.7 is applied in Theorem 2.9 to solve this problem in case (1.10).Problem 4.
Describe the cases whenthe limiting power disappears, that is, when β(Φ)=0.
This problem has been the main motivation for this paper. Spatial models in general are peculiar in many respects, and the possibility of the limiting power to disappear is one of those peculiarities that has been attracting researchers' attention lately. Krämer [6] was the first to suggest that the limiting power of tests for spatial autocorrelation may vanish, for some combinations of the regressor matrix and the spatial matrix. Unfortunately, the terms in which Krämer expressed his results do not adequately reflect all the possibilities, and his proof contains an incorrect argument; see [7, Footnote 5]. Martellosio [4] was the first to suggest that the answer is better described in terms of the geometrical relationship between the eigenvectors of Σ-1(a) and the critical region Φ. However, both of his main result [4, Theorem 1] and its proof contain errors (see Remark 2.12 in Section 2). Our answer to Problem 4 is given in Theorem 2.11 and corrects [4, Theorem 1] in that where he excludes the extreme values 0 and 1 for the limiting power, we give examples showing that the extreme values are possible. In the light of our result, several statements in [4, 7] have to be reconsidered. The analysis for the Cliff-Ord test is very involved, and it is not feasible to indicate corrections for all Martellosio's main results that depend on the wrong intermediate statement or his Theorem 2.9. The complexity of our analysis necessitates a more detailed notation than that used by Martellosio. In particular, some of his verbal definitions and statements are cast in a more formal way. As a result, our citations are not word for word.
Despite the fact that the convergence in distribution is the right one for Problem4, it would be interesting to know what kind of convergence produces the limit indicated by Martellosio, as is stated next.Problem 5.
Design the mode of convergence that leads to (1.11).To this end, we introduce a new convergence concept (which, given its purpose, could be called aretrofit convergence), which may not look intuitively appealing but allows us to prove (1.11) in Theorem 2.13. Under this alternative convergence, there is no analog of (1.12). Therefore, we did not consider Problems 3 and 4 for this convergence.The previous plan will be implemented under conditions much more restrictive than suggested by Martellosio. All main results that are stated in Sections2 and 3 contains all proofs.


## 2. Main Statements

In the multidimensional version of the generalized mean (1.4) instead of averaging over segments [-r,r], we have to average over balls. The shape of those balls depends on the norm of Rn. Let ∥·∥a be an arbitrary norm in Rn. The balls are defined by Ba,n(x,r)={y∈Rn:∥x-y∥a≤r}, where the indication of the space dimension will be important when dealing with more than one space. As an example, one can think of the lp-norm defined by
(2.1)∥x∥p={(∑i=1n‍|xi|p)1/p,if1≤p<∞,max1≤i≤n|xi|,ifp=∞.
In case of the Euclidean norm ∥·∥2, we obtain usual balls; in cases p=1 and p=∞, the balls Bp,n(x,r) are cubes. Another useful example is ∥x∥A=(x′Ax)1/2, where A is a symmetric, positive definite matrix. We say that a function g on Rn is ∥·∥a-spherically symmetric if g(x)=p(∥x∥a) with some function p defined on the half-axis [0,∞). Conditions involving spherical symmetry in the following are similar to Conditions (2.1) and (2.2) from [8].Letσ denote an element of the unit sphere {σ∈Rn:∥σ∥a=1} and let (σ,ρ(σ)) be the representation of a point x∈Rn in the polar system of coordinates such that σ=x/∥x∥a and ρ(σ)=∥x∥a for x≠0. Then, the Lebesgue measure of Ba,n(x,r) is
(2.2)mesBa,n(x,r)=mesBa,n(0,r)=∫∥σ∥a=1‍∫0r(σ)‍ρn-1dρdσ=rn∫∥σ∥a=1‍∫0r(σ)/r‍ρn-1dρdσ=varn,
where va=mesBa,n(0,1) is the volume of the unit ball. The norm ∥·∥a gives rise to averages
(2.3)m(φ,Ba,n(x,r))=1mesBa,n(x,r)∫Ba,n(x,r)‍φ(t)dt,
and to the generalized mean
(2.4)Ma,nφ=limr→∞m(φ,Ba,n(0,r)).
In the one-dimensional case all balls are segments, and we write simply Mφ instead of Ma,1φ.One of the basic properties of generalized means is that they do not depend on the behavior ofφ in any fixed ball Ba,n(0,r):
(2.5)Ma,nφ=limr→∞1mesBa,n(0,r)(∫Ba,n(0,r0)‍φ(t)dt+∫Ba,n(0,r)∖Ba,n(0,r0)‍φ(t)dt)=limr→∞1mesBa,n(0,r)∫Ba,n(0,r)∖Ba,n(0,r0)‍φ(t)dt.
Other useful properties are (3.16), (3.37), and (3.42).C
L
(
R
n
) denotes the set of continuous bounded functions on Rn that satisfy the Lipschitz condition
(2.6)|φ(x)-φ(y)|≤c1∥x-y∥2,x,y∈Rn;
By [9, Theorem  3.6.1], convergence in distribution of random elements Xn→dX is equivalent to the convergence of expected values
(2.7)Eφ(Xn)⟶Eφ(X),∀φ∈CL(Rn).
[10, Theorem 2.1] asserts that here CL(Rn) can be replaced by the set of bounded uniformly continuous functions. Lp(Rn) is the space of p-summable functions on Rn provided with the norm ∥g∥Lp=(∫Rn‍|g(t)|pdt)1/p,  1≤p<∞.For our applications, in the multidimensional version of (1.5), we need to allow φ to depend on the parameter λ, as in
(2.8)limλ→0λd∫Rd‍φλ(t)g(λt)dt=Ma,dφ0∫Rd‍g(t)dt.
Here φ∈CL(Rn),  φλ(t)=φ(F(λ)t+G(λ)). F(λ),  λ≥0, are n×d matrices and G(λ),  λ≥0, are n×1 vectors such that F(λ) and G(λ) tend to F(0) and G(0), respectively, sufficiently quickly, as stipulated in the next assumption.Condition 1.
(a)
g
∈
L
1
(
R
d
)
is∥·∥a-spherically symmetric, g(x)=p(∥x∥a).
(b)
∥
F
(
λ
)
-
F
(
0
)
∥
=
o
(
λ
)
, ∥G(λ)-G(0)∥2=o(1), λ→0.Remark 2.1.
Using polar coordinates we see that item (a) imposes a certain integrability restriction onp:
(2.9)∫Rd‍|g(t)|dt=∫∥σ∥a=1‍(∫0∞‍|p(ρ)|ρd-1dρ)dσ=|S|a∫0∞‍|p(ρ)|ρd-1dρ<∞,
where |S|a stands for the surface of the unit sphere {σ:∥σ∥a=1}. The class of densities satisfying (a) includes contaminated normal distribution, multivariate t-distribution, multivariate Cauchy distribution, and see; [8]. It does not matter much which norms are used in item (b).Theorem 2.2.
Condition1 is sufficient for (2.8) to hold on φ∈CL(Rn) such that Ma,dφ0 exists.Remark 2.3.
The triangle inequality∥x+y∥a≤∥x∥a+∥y∥a is not used in the proof, and a slight generalization of Theorem 2.2 in terms of the geometry of balls is possible. The Lipschitz condition (2.6) can be omitted if F,G are constant (φ can be assumed just bounded and continuous).The first application of Theorem2.2 is to the theory of characteristic functions. Let F be the distribution function of a (proper, real-valued) random variable X. Denote j(x) the jump of F at point x and let φ(t)=EeitX be the characteristic function.Corollary 2.4.
Ifg∈L1(R) is even, then
(2.10)limλ→0∫R‍e-itxφ(t)λg(λt)dt=j(x)∫R‍g(t)dt,limλ→0∫R‍|φ(t)|2λg(λt)dt=∑k‍j2(xk)∫R‍g(t)dt,
where the sum on the right is over all jump points xk of F.Theorems 3.2.3 and 3.3.4 in [2] are a special case of this corollary with g=(1/2)χ[-1,1]. The proof of Corollary 2.4 is obtained by combining those theorems with our Theorem 2.2.The second application is to the theory of almost-periodic functions. A complex-valued continuous functionf on R is called almost-periodic if for each ε>0 there exists l(ε)>0 such that each interval (a,a+l(ε)) of length l(ε) contains at least one number τ for which supt∈R|f(t+τ)-f(t)|<ε. In the space of almost-periodic functions, the formula (φ,ψ)=Mφψ- defines a scalar product (ψ- is a complex conjugate of ψ), and the numbers α(λ)=(φ(·),eiλ) are called Fourier coefficients of φ. It is proved that for each φ at most; a countable number of these coefficients are nonzero. Denoting them α(λk),  k=1,2,…,H. Bohr's theorem states that M|φ|2=∑k‍|α(λk)|2; see [3, page 134]. This fact together with our Theorem 2.2 gives the next corollary.Corollary 2.5.
Ifg∈L1(R) is even, then for any almost-periodic function φ(2.11)limλ→0∫R‍|φ(t)|2λg(λt)dt=∑k‍|α(λk)|2∫R‍g(t)dt.
Further, if ∫R‍g(t)dt>0, the formula (φ,ψ)g=limλ→0∫R‍φ(t)ψ-(t)λg(λt)dt defines a scalar product which is equivalent to (φ,ψ).Note that the first part of this corollary applies to characteristic functions of purely discrete distribution functions because from [2, Corollary  2 of Theorem  3.2.3] any such characteristic function is almost periodic. Now, we turn to the multidimensional version of (1.7). For the density, we assume a stronger condition than Condition 1(a).Condition 2.
g
-
∈
L
1
(
R
n
) is ∥·∥2-spherically symmetric,  g-(x)=p-(∥x∥2).This assumption allows us to show that when some coordinates ofx are fixed, g-(x) as a function of the remaining coordinates satisfies Condition 1(a).Now, we provide the intuition for the next condition. The stretching-out applied in (1.7) is described by the transformation s=A(λ)t where
(2.12)A(λ)=(λo01),λ>0.gλ(t)=detA(λ)g(A(λ)t) is the analog of gλ(t)=λg(λt) because ∫R2‍gλ(s)ds=∫R2‍g(t)dt. Here, the matrix A(λ) has two eigenvalues. The limit limλ→0A(λ)=A(0) is a singular matrix because one of these eigenvalues tends to zero as λ→0. Generalizing upon this situation and also thinking of applications to invariant tests, in the n-dimensional case we consider a symmetric nonnegative matrix A(ρ) of size n×n, where the parameter ρ belongs to the segment [0,a]. Denote its eigenvalues 0≤λ1(ρ)≤⋯≤λn(ρ) and let A(ρ) be diagonalized as A(ρ)=P(ρ)Λ(ρ)P′(ρ), where Λ(ρ)=diag[λ1(ρ),…,λn(ρ)] and P(ρ) is an orthogonal matrix. A(ρ) degenerates at the right end of the segment [0,a], owing to the following assumption.Condition 3.
The matrixA(ρ) is positive definite for 0≤ρ<a.
The firstd eigenvalues tend to zero at the same rate, λj(ρ)=λ1(ρ)(1+o(1)),j=1,…,d;  λ1(ρ)→0 as ρ→a; the remaining ones have positive limits: λj(ρ)→λj(a)>0 as ρ→a,j=d+1,…,n.
The matricesP(ρ) and Λ(ρ) converge sufficiently quickly as ρ→a. Namely, with the matrix
(2.13)Λ~(ρ)=diag[λ1(ρ),…,λ1(ρ)︸dtimes,λd+1(a),…,λn(a)]
one has ∥Λ-1(ρ)Λ~(ρ)-I∥=o(λ1(ρ)),  ∥P(ρ)-P(a)∥=o(λ1(ρ)),  ρ→a.Note thatΛ-1(a) does not exist because of part (b), but part (c) allows us to set Λ-1(a)Λ~(a)=I by continuity. In line with part (b), we use the partitions
(2.14)t=(t(1)t(2)),t(1)=(t1,…,td)′,t(2)=(td+1,…,tn)′,(2.15)Λ(ρ)=(Λ1(ρ)00Λ2(ρ)),Λ1(ρ)isd×d,Λ2(ρ)is(n-d)×(n-d).
Also, partition P(a)=(P1(a),P2(a)) conformably with (2.14) (P1(a) is n×d and P2(a) is n×(n-d)). Define a transformation T by (Tφ)(t)=φ(P1(a)t(1)+P2(a)Λ2-1(a)t(2)).  (M2,dTφ)(t(2)) denotes the result of application of the generalized mean operator with respect to t(1), keeping t(2) fixed:
(2.16)(M2,dTφ)(t(2))=limr→∞m((Tφ)(·,t(2)),B2,d(0,r)),
and g-1 denotes a marginal "density":
(2.17)g-1(t(2))=∫Rd‍g-(t(1),t(2))dt(1).Theorem 2.6.
Let Conditions2 and 3 hold and let φ∈CL(Rn) be such that the limit (2.16) exists for almost all t(2)∈Rn-d. Then,
(2.18)limρ→a∫Rn‍φ(t)detA(ρ)g(A(ρ)t)dt=∫Rn-d‍(M2,dTφ)(t(2))g-1(t(2))dt(2).Next, we turn to the solution of Problem2. Denoting A(ρ)=Σ-1/2(ρ) and assuming that A(ρ) is positive definite for ρ∈[0,a), we see that condition (1.10) corresponds to the case d=1 of our Condition 3. The density (1.9) fits the framework of our Theorem 2.6 because the stretching-out is applied along one variable. Thus, in Theorem 2.7 we apply Theorem 2.6 to (a) characterize the limit in distribution of fρ in case 1≤rank(Σ-1(a))≤n-1 and (b) show that (1.11) does not hold under the weak convergence. By implication, (1.11) is wrong if any convergence stronger than the weak one is considered (e.g., uniform, almost sure, in probability and in Lp). Even though we use Theorem 2.6, the assumption on the density in part (b) of the next theorem is weaker than that in Theorem 2.6.Condition 4.
g is a density on Rn bounded by an integrable, ∥·∥2-spherically symmetric function, g(x)≤p(∥x∥2).An example of such a density isg(x)=p(∥x∥a), where p is a nonnegative and nonincreasing function on [0,∞), which decays at infinity sufficiently quickly for g to be integrable. By monotonicity of p inequality (3.14) in the following implies that g(x)=p(∥x∥a)≤p(∥x∥2/c3).Theorem 2.7.
Let Condition3 hold in which A(ρ)=Σ-1/2(ρ).(a)
DenoteQ=P2(a)P2′(a) the projector onto the subspace spanned by the last n-d eigenvectors of A(a) and let T1 be defined by
(2.19)(T1φ)(t)=φ(P1(a)t(1)+P2(a)Λ2-1(a)t(2)+QXβ).M2,dT1φ is obtained by replacing T with T1 in (2.16). If Condition 2 holds and φ∈CL(Rn) is such that (M2,dT1φ)(t(2)) exists for almost all t(2)∈Rn-d, then
(2.20)limρ→a∫Rn‍φ(t)fρ(t)dt=∫Rn-d‍(M2,dT1φ)(t(2))g-1(t(2))dt(2),
with g-1 defined in (2.17).(b)
If Condition4 holds, then (1.11) cannot be true if the convergence in distribution is understood.Remark 2.8.
Because of the identityXβ+N(A(a))=(I-Q)Xβ+QXβ+N(A(a))=QXβ+N(A(a)), (1.11) correctly captures one feature of the limit distribution: it depends on Xβ only through QXβ.Before giving the solution to Problem3, we need more notation and definitions. Obtaining a closed-form formula for β(Φ) involves a meticulous analysis of Φ based on the representation of Φ given in the following.LetImX={Xβ:β∈Rk} be the image of X. If for a given set Φ⊂Rn (a) the space Rn is represented as an orthogonal sum Rn=E1⊕E2 of two subspaces E1 and E2 and (b) there is a set B⊂E1 such that Φ=B+E2={b+e:b∈B,e∈E2}, then we say that Φ is a cylinder with the baseB and elementE2. For any set S⊂Rn, denote ΓS={γs:s∈S,γ∈R,γ≠0}. Using s=1·s, we see that S⊂ΓS. We say that S is cone-like if S=ΓS. In particular, a cone-like set with each of its element s contains its opposite -s. The next representation is a stronger statement than saying that Φ defined in (1.14) is invariant with respect to transformations y→γy+Xβ,  γ∈R,  γ≠0,  β∈Rk.Representation ofΦ
The rejection regionΦ for the Cliff-Ord test is a cylinder
(2.21)Φ=ΓS+ImX,
with a cone-like base MXΦ=ΓS, where S={s∈ImMX:∥s∥2=1,s′Ws>c}.It is convenient to call anaperture the set S in representation (2.21). The function s′Ws is continuous on the unit sphere of ImMX and the set (c,∞) is open. By the general property of continuous mappings [11, Chapter 2, § 5, Section 5, Theorem 6], the preimage of an open set under a continuous mapping is open. Thus, the aperture is an open set (in the relative topology) of the unit sphere of ImMX. We need the notations of the interior int(Φ) (defined as the set of points of Φ that belong to Φ with some neighborhood), closure cl(Φ) (the set of all limit points of Φ) and boundary bd(Φ)=cl(Φ)∖Φ.Writing (1.12) in the form
(2.22)βρ(Φ)=∫Rn‍χΦ(y)fρ(y)dy,
we see that Theorem 2.7(a) will be applicable if we manage to extend it from continuous Lipschitz functions to discontinuous functions of type χΦ. This is done in Theorem 2.9. In Theorems 2.9 and 2.11, we assume that d=1. Up to the notation, this is the same assumption as (1.10). At least within our method, generalizations of the results in the following to the case d>1 are hard to obtain.Let Condition3 hold for A(ρ)=Σ-1/2(ρ) and let d=1. Then, λ1(ρ)→0 as ρ→a and all other eigenvalues have positive limits. Denote f1,...,fn the orthonormal eigenvectors of A(a)=Σ-1/2(a) corresponding to the eigenvalues λ1(a)=0,λ2(a),…,λn(a). The partitions (2.14) become t(1)=t1, t(2)=(t2,…,tn)′, P1(a)=f1, P2(a)=(f2,…,fn). The vector z(t(2))=P2(a)Λ2-1(a)t(2)+QXβ will be called a shift because its role is to shift the line P1(a)t(1)+P2(a)Λ2-1(a)t(2)+QXβ=f1t1+z(t(2)). In the next theorem, we extend (2.20) to φ=χΦ with Φ described by (2.21). In the notation of marginal densities, the subscript will indicate the number of integrated-out variables. For example, g-1(t(2))=∫R‍g-(t1,t(2))dt1,  g-k(t)=∫Rk‍p-((t2+∥v∥22)1/2)dv.Theorem 2.9 (representation ofβ(Φ)).
Let Conditions2 and 3 hold with d=1.(a)
Iff1∈ImX and
(2.23)∫01‍g-k(t)dt<∞,
then
(2.24)β(Φ)=∫Rn-1‍(MT1χΦ)(t(2))g-1(t(2))dt(2),
where T1χΦ is defined by (T1χΦ)(t)=χΦ(f1t1+z(t(2))), and the generalized mean is applied over t1.(b)
Iff1∉ImX and
(2.25)∫01‍g-k+1(t)dt<∞,
then (2.24) is true.Remark 2.10.
(a) As can be seen from the proof, the theorem holds for any test with the critical region satisfying (2.21). (b) Conditions (2.23) and (2.25) are technical assumptions that provide integrability in the neighborhood of the origin of marginal densities that arise in the course of the proof.To avoid triviality, in the following theorem we assume that the inclusion∅⊂Φ⊂Rn is strict. This implies that in representation (2.21) the set ΓS is a nonempty proper subset of ImMX.Theorem 2.11.
Let conditions of Theorem2.9 hold, with (2.23) accompanying f1∈ImX and (2.25) accompanying f1∉ImX. Besides, we require the function p- from Condition 2 to be positive on [0,∞).(1)
Iff1∈Φ, then β(Φ)=1.(2)
Supposef1∈bd(Φ).
(2.1)
Iff1∈ImX, then β(Φ)∈(0,1).(2.2)
Iff1∉ImX, then examples can be presented such that β(Φ)=0, β(Φ)∈(0,1) or β(Φ)=1.(3)
Iff1∈Φ-, then β(Φ)=0.Remark 2.12.
Here, we compare this theorem with [4, Theorem 1].
(1.4) Our conditions on the density and critical region are much more restrictive. Our proof reveals the distinction between the cases f1∈ImX and f1∉ImX. In particular, in case (2.2), Martellosio excludes the extreme values β(Φ)=0 and β(Φ)=1, while we provide counter examples showing that they are possible. Note also that we do not impose any conditions on the structure of W. What happens to the limiting power in case of W that arises in practice needs additional investigation.
(1.5) Martellosio's proof is based on (1.11) which we disprove in Theorem 2.7(b). A series of other propositions from the same paper (see Lemmas D.2, D.3 and E.4, Corollary 1 and Propositions 1, 2 and 5), as well as from [7] (see Lemma 3.2, Theorems 3.3, 3.5 and 4.1, and Proposition 3.6), depend on (1.11) and need a revision. In particular, his claim that his results are true for any invariant critical region and any continuous density u that is unimodal at the origin is unwarranted.
(1.7) Even if statement (1.11) were right, the proof of [4, Theorem 1] would be incomplete because it incorrectly uses the n-dimensional Lebesgue measure. Its use is inappropriate because for a degenerate density even one point may carry a positive mass. In our proof, we justify the use of (n-1)-dimensional Lebesgue measure.Now, we turn to the description of the alternative approach (solution to Problem5). A mode is not a very good characteristic of a distribution when two unimodal densities with the same modes have very different spreads. A set of points where the density is close to its maximum might be a better characteristic in this case, at least for bell-shaped distributions. Let 0<ε<1 and let m=maxf(x) be the maximum of a continuous density f(x). We call Mε={x:f(x)≥(1-ε)m} an ε-maximizing set of f. The idea of this density-maximizing approach is close to the maximum likelihood principle.Suppose a decisiond is taken if the statistic x belongs to a set D. In case of a favorable decision, D is chosen in such a way that the probability P(x∈D) is high. The use of probability in this decision rule presumes that the statistic can be calculated repeatedly. However, in practice, especially in economics, the decision is based on just one value of the statistic in question. In such a case it may be preferable to choose D so that minx∈Df(x) is sufficiently close to m=maxf(x). For the ε-maximizing set, we have minx∈Mεf(x)≥(1-ε)m. Requiring ε to be close to 1 in the density-maximizing approach is similar to requiring P(x∈D) to be close to 1 in the probability-maximizing approach (although normally P(x∈Mε)→0 as ε→0).As before, we assume that the matrixA(ρ)=Σ-1/2(ρ) of size n×n is symmetric for 0≤ρ≤a and positive definite for 0≤ρ<a. The idea is to impose conditions ensuring that ε-maximizing sets are ellipsoids which in the limit give a set of the desired shape. This idea is realized through a delicate balance of the limit behavior of the eigenvalues and density contained in Conditions 5 through 7 and (2.32).Condition 5.
In the diagonal representation ofA(ρ), the orthogonal matrix P satisfies
(2.26)P(a)=limρ→aP(ρ),
the first d eigenvalues vanish as power functions
(2.27)λj(ρ)=cλ(a-ρ)αλ(1+o(1)),j=1,…,d,asλ⟶a,
with positive constants cλ,αλ; the remaining eigenvalues tend to positive constants
(2.28)λj(ρ)=λj(a)(1+o(1)),j=d+1,…,n,asλ⟶a,
where λj(a)>0.Condition 6.
The functionε(ρ) in the definition of the set Mε(ρ) vanishes as a power function
(2.29)ε(ρ)=cε(a-ρ)αε(1+o(1)),ρ⟶a,
where cε,αε are positive constants.Condition 7.
The densityg is ∥·∥2-spherically symmetric, g(x)=p(∥x∥2), where p is continuous and monotonically decreasing on [0,∞) and such that
(2.30)p(r)=m(1-cprαp)(1+o(rαε)),r⟶0,
where cp,αp>0.This assumption implies thatm=maxg(x)=p(0), that the inverse function p-1 is continuous and monotonically decreasing on (0,m], and that
(2.31)0=p-1(m)=limx→mp-1(x).Theorem 2.13.
DenoteN(A(a)) the null subspace of A(a) (spanned by the eigenvectors corresponding to its zero eigenvalues). If Conditions 5–7 hold and
(2.32)αεαp-αλ<0,
then the ε(ρ)-maximizing set Mε(ρ)={x:fρ(x)≥(1-ε(ρ))m} converges to Xβ+N(A(a)) as ρ→a.


## 3. Proofs

The idea of the proof is to approximateg with a step function h, prove the statement for h and then pass to the limit to obtain the statement for g. A step function, by definition, is a finite linear combination of indicators of measurable sets. Due to the spherical symmetry of g, these sets turn out to be balls. For the method to work, the radii of the balls should be positive and finite. Approximation of p by a continuous function in Step 1 is a trick to make sure that h takes a finite number of values.Step 1 (p can be assumed continuous on (0,∞)).
LetΩ=(0,∞) and denote Z(Ω) the space of measurable functions f on Ω such that ∥f∥Z(Ω)=∫0∞‍|f(ρ)|ρd-1dρ<∞.  Z(Ω) satisfies conditions of [12, Theorem 1]:(1)
C0∞(Ω)⊂Z(Ω)⊂L1loc(Ω).(2)
Minkowsky inequality: ifA⊂Rm is a measurable set and Φ(x,y) is a measurable on Ω×A function, then ∥∫A‍Φ(·,y)dy∥Z(Ω)≤∫A‍∥Φ(·,y)∥Z(Ω)dy.(3)
A multiplication operator by a functionφ∈C0∞(Ω) is bounded in Z(Ω): ∥fφ∥Z(Ω)≤max|φ|∥f∥Z(Ω).(4)
Any finite inΩ function f∈Z(Ω) is translation-continuous:  limh→0∥f(·+h)-f(·)∥Z(Ω)=0.
The first three properties are standard facts of the theory ofLp spaces; the last one follows from the fact that if suppf is a compact subset of Ω, then the weight ρd-1 in the definition of the norm of Z(Ω) satisfies c1≤ρd-1≤c2 on suppf and therefore for such an f the norms ∥f∥Z(Ω) and ∥f∥L1 are equivalent. Functions from L1 are known to be translation-continuous.
By Burenkov’s theorem there exists a sequence{ps}⊂C∞(Ω)∩Z(Ω) such that ∥ps-p∥Z(Ω)→0. Defining gs(x)=ps(∥x∥a), from (2.9) we have ∥gs-g∥L1=|S|a∥ps-p∥Z(Ω)→0. Hence,
(3.1)|λd∫Rd‍φλ(t)gs(λt)dt-λd∫Rd‍φλ(t)g(λt)dt|≤sup|φ|λd∫Rd‍|gs(λt)-g(λt)|dt=sup|φ|∥gs-g∥L1⟶0
uniformly in λ. Thus, if we establish
(3.2)limλ→0λd∫Rd‍φλ(t)gs(λt)dt=Ma,dφ0∫Rd‍gs(t)dt∀s,
then (2.8) will follow.Step 2 (approximatingg with a Step Function).
Take an arbitraryε>0. We can assume that p is continuous on (0,∞).(a)
Suppose∥g∥L1≥ε. We approximate g(t) by a step function h(t), which will vanish where t is large or small. By summability of g, there exist 0<N1=N1(ε)<N2=N2(ε)<∞ such that
(3.3)∫∥t∥a<N1|g(t)|dt≤ε3,∫∥t∥a>N2|g(t)|dt≤ε3g is uniformly continuous on the ring {N1≤∥t∥a≤N2}. By Condition 1(a), for any natural m we can find δ>0 and split this ring into smaller rings
(3.4)Am,l={t:N1+lδ≤∥t∥a<N1+(l+1)δ},l=0,…,L≡N2-N1δ-1,
in such a way that g in each ring is close to its value on the inner boundary:
(3.5)supt∈Am,l|g(t)-p(lm)|≤1m.
Put h(t)=0 if ∥t∥a<N1 or ∥t∥a>N2;  h(t)=p(l/m) for t∈Am,l,  l=0,…,L. Combination of (3.3), (3.4), and (3.5) leads to
(3.6)∥h-g∥L1=(∫∥t∥a<N1+∫∥t∥a>N2)|g(t)|dt+∑l=0L∫Am,l|h(t)-g(t)|dt≤2ε3+1mmesBa,d(0,N2)=2ε3+vaN2d(ε)m.
Consequently, we can fix m=m(ε) so that
(3.7)∥h-g∥L1≤ε.(b)
If∥g∥L1<ε, we just put h≡0 to get (3.7).Step 3 (replacing rings by balls in the representation ofh).
Supposeh is not identically zero. The sets Am,l in (3.4) are concentric rings with finite positive radii of the inner and outer boundaries, and by construction, h is a finite linear combination of indicators of such rings. Hence, it can be written as h=∑l=0L‍al(χBa,d(0,rl)-χBa,d(0,rl-1)), where al are some real numbers and the radii satisfy
(3.8)N1=r0<⋯<rL=N2.
Therefore, with some new constants bl, we can write h as a linear combination of indicators of balls:
(3.9)h=∑l=0L‍hl,wherehl=blχBa,d(0,rl).
Ifh≡0, we put formally r0=1,  b0=0,  L=0. Note that L,  bl and rl all depend on ε and that the constants bl may deviate from the values of h significantly.Step 4 (introducing residuals for generalized means).
Define the residualR1(r) by
(3.10)R1(r)={m(φ0,Ba,d(0,r))/Ma,dφ0-1,ifMa,dφ0≠0,m(φ0,Ba,d(0,r)),ifMa,dφ0=0.
Then,
(3.11)m(φ0,Ba,d(0,r))={Ma,dφ0(R1(r)+1),ifMa,dφ0≠0,R1(r),ifMa,dφ0=0,
and in both cases
(3.12)limr→∞R1(r)=0.
By the Lipschitz condition (2.6) and Condition 1(b),
(3.13)|φ(F(λ)t+G(λ))-φ(F(0)t+G(0))|≤c1∥(F(λ)-F(0))t+G(λ)-G(0)∥2=o(λ)∥t∥a+o(1)=o(λ)r+o(1)for∥t∥a≤r.
Here, we have used the fact that on Rd any two norms are equivalent, so
(3.14)c2∥t∥a≤∥t∥2≤c3∥t∥a,
with some c2,c3>0. Equations (2.3) and (3.13) imply that
(3.15)|m(φ(F(λ)·+G(λ))-φ0(·),Ba,d(0,r))|=o(λ)r+o(1)∀r>0,asλ→0,
where o(λ) and o(1) do not depend on r. From this bound and (3.11), it follows that
(3.16)m(φ(F(λ)·+G(λ)),Ba,d(0,r))={Ma,dφ0(R1(r)+1)+R2(λ,r),ifMa,dφ0≠0,R1(r)+R2(λ,r),ifMa,dφ0=0,
where R2(λ,r) is a new residual satisfying
(3.17)R2(λ,r)=o(λ)r+o(1)∀r>0,asλ→0.Step 5 (proving (2.8) for the approximating function).
LetMa,dφ0≠0. For one term in (3.9) by (2.2), and the first equation in (3.16) we have
(3.18)λd∫Rd‍φ(F(λ)t+G(λ))hl(λt)dt=λdbl∫Ba,d(0,rl/λ)φ(F(λ)t+G(λ))dt=varldblva(rl/λ)d∫Ba,d(0,rl/λ)φ(F(λ)t+G(λ))dt=∫Rdhl(s)ds{Ma,dφ0[R1(rlλ)+1]+R2(λ,rlλ)}.
Summation of these equations produces
(3.19)λd∫Rdφ(F(λ)t+G(λ))h(λt)dt=∑l=0L‍∫Rdhl(s)ds{Ma,dφ0[R1(rlλ)+1]+R2(λ,rlλ)}=Ma,dφ0∫Rdh(s)ds+∑l=0L‍∫Rdhl(s)ds{Ma,dφ0R1(rlλ)+R2(λ,rlλ)}.
Here, by (3.8), (3.12), and (3.17), R1(rl/λ)→0,  R2(λ,(rl/λ))=o(rl)+o(1)=o(1),  l=0,…,L, as λ→0. From these relations and (3.19), we see that for the given ε there exists λ(ε) such that
(3.20)|λd∫Rdφλ(t)h(λt)dt-Ma,dφ0∫Rdh(s)ds|≤εfor0<λ≤λ(ε).
In caseMa,dφ0=0, the only difference consists in application of the second equation in (3.16). The conclusion is (3.20) with Ma,dφ0=0.Step 6 (proving (2.8)).
Using (3.7) and (3.20), we have
(3.21)|λd∫Rdφλ(t)g(λt)dt-Ma,dφ0∫Rdg(t)dt|≤|λd∫Rdφλ(t)[g(λt)-h(λt)]dt|+|λd∫Rdφλ(t)h(λt)dt-Ma,dφ0∫Rdh(t)dt|+|Ma,dφ0||∫Rd[h(t)-g(t)]dt|≤∥φ∥C∥g-h∥L1+ε+|Ma,dφ0|∥g-h∥L1≤(∥φ∥C+1+|Ma,dφ0|)ε,0<λ≤λ(ε).
Equation (2.8) follows because ε>0 is arbitrary.


### 3.1. Proof of Theorem2.6

DenoteJρ(φ)=∫Rn‍φ(t)detA(ρ)g(A(ρ)t)dt.Step 1 (replacinglimJρ(φ) by an equivalent limit).
Using constancy ofg- on spheres (Condition 2), write Jρ(φ) as
(3.22)Jρ(φ)=∫Rn‍φ(y)detΛ(ρ)g-(P(ρ)Λ(ρ)P′(ρ)y)dy=detΛ(ρ)∫Rn‍φ(y)g-(Λ(ρ)P′(ρ)y)dy=detΛ(ρ)∫Rn‍φ(y)g-(Λ~(ρ)Λ~-1(ρ)Λ(ρ)P′(ρ)y)dy.
Denote H-1(ρ)=Λ~-1(ρ)Λ(ρ)P′(ρ) and replace H-1(ρ)y=t to get
(3.23)Jρ(φ)=detΛ(ρ)detH(ρ)∫Rn‍φ(H(ρ)t)g-(Λ~(ρ)t)dt.
By Condition 3(b,c),
(3.24)detΛ(ρ)=λ1d(ρ)detΛ2(a)(1+o(1)),detH(ρ)=det(Λ~-1(ρ)Λ(ρ))detP′(ρ)=1+o(1).
From (3.23) and (3.24) we deduce
(3.25)Jρ(φ)=λ1d(ρ)detΛ2(a)(1+o(1))∫Rn‍φ(H(ρ)t)g-(Λ~(ρ)t)dt=(1+o(1))×detΛ2(a)∫Rn-d‍[λ1d(ρ)∫Rd‍φ(H(ρ)t)g-(λ1(ρ)t(1),Λ2(a)t(2))dt(1)]dt(2).Step 2 (changing the limit variable).
Let us have a closer look at the expression in the square brackets in (3.25):
(3.26)Jρ(φ,t(2))≡λ1d(ρ)∫Rd‍φ(H(ρ)t)g-(λ1(ρ)t(1),Λ2(a)t(2))dt(1).
Since Λ-1(a)Λ~(a)=I, here H(a)=P(a), and by Condition 3(c),
(3.27)∥H(ρ)-P(a)∥=∥P(ρ)Λ-1(ρ)Λ~(ρ)-P(ρ)+P(ρ)-P(a)∥≤∥P(ρ)∥∥Λ-1(ρ)Λ~(ρ)-I∥+∥P(ρ)-P(a)∥=o(λ1(ρ)),ρ⟶a.
Considering λ=λ1 the independent variable, take ρ=ρ(λ) as its function. The limit relation ρ→a translates to λ→0, and (3.26) and (3.27) become,
(3.28)Jρ(λ)(φ,t(2))=λd∫Rd‍φ(H(ρ(λ))t)g-(λt(1),Λ2(a)t(2))dt(1),(3.29)∥H(ρ(λ))-P(a)∥=o(λ),λ⟶0,
respectively.Step 3 (proving a preliminary version of (2.18)).
Now we check that Theorem2.2 applies to (3.28) for any fixed t(2). Letting
(3.30)p(z)=p-((z2+∥Λ2(a)t(2)∥22)1/2),z≥0,g(t(1))=g-(t(1),Λ2(a)t(2)),
we see that g∈L1(Rd) is ∥·∥2-spherically symmetric:
(3.31)g(t(1))=g-(t(1),Λ2(a)t(2))=p-((∥t(1)∥22+∥Λ2(a)t(2)∥22)1/2)=p(∥t(1)∥2).
Condition 1(a) is satisfied. As for Condition 1(b), partition H=(H1,H2) conformably with (2.14) and put F(λ)=H1(ρ(λ)),  G(λ)=H2(ρ(λ))t(2),  F(0)=H1(a)=P1(a),  G(0)=P2(a)t(2). Then, H(ρ(λ))t=H1(ρ(λ))t(1)+H2(ρ(λ))t(2)=F(λ)t(1)+G(λ), and Condition 1(b) follows from (3.29).
By Theorem2.2,
(3.32)Jρ(λ)(φ,t(2))⟶Jρ(0)(φ,t(2))≡(M2,dφ-)(t(2))∫Rd‍g-(t(1),Λ2(a)t(2))dt(1),t(2)∈Rn-d,
where (M2,dφ-)(t(2)) is obtained by applying M2,d to φ-(t)≡φ(P1(a)t(1)+P2(a)t(2)) with respect to t(1). Functions (3.28) are bounded by an integrable function of t(2):
(3.33)|Jρ(λ)(φ,t(2))|≤λd∥φ∥C∫Rd‍g-(λt(1),Λ2(a)t(2))dt(1)=∥φ∥C∫Rd‍g-(t(1),Λ2(a)t(2))dt(1).
By the dominated convergence theorem, (3.32) and (3.33) imply that
(3.34)∫Rn-d‍Jρ(λ)(φ,t(2))dt(2)⟶∫Rn-d‍Jρ(0)(φ,t(2))dt(2),λ→0.
Now, we combine (3.25), (3.28), (3.32), and (3.34) to get
(3.35)limρ→aJρ(φ)=detΛ2(a)∫Rn-d‍[(M2,dφ-)(t(2))∫Rd‍g-(t(1),Λ2(a)t(2))dt(1)]dt(2)=∫Rn-d‍[(M2,dφ-)(Λ2-1(a)s(2))∫Rd‍g-(t(1),s(2))dt(1)]ds(2)=∫Rn-d‍(M2,dTφ)(s(2))g-1(s(2))ds(2)
because
(3.36)(M2,dφ-)(Λ2-1(a)s(2))=limr→∞m(φ(P1(a)·+P2(a)Λ2-1(a)s(2),B2,d(0,r)))=(M2,dTφ)(s(2)).


### 3.2. Proof of Theorem2.7

#### 3.2.1. Part (a)

Step 1 (Ma,dφ does not depend on the center of balls).
Let us make definition (2.4) more precise: (Ma,dφ)(x)=limr→∞m(φ,Ba,d(x,r)),  x∈Rd. Here, we prove that
(3.37)(Ma,dφ)(x)=(Ma,dφ)(0)foranyx∈Rd.
For r≥∥x∥a, one has implications ∥y∥a≤r-∥x∥a⇒∥y-x∥a≤r⇒∥y∥a≤∥y-x∥a+∥x∥a≤r+∥x∥a. Geometrically, they mean the inclusion Ba,d(0,r-∥x∥a)⊂Ba,d(x,r)⊂Ba,d(0,r+∥x∥a). This inclusion implies that
(3.38)(r-∥x∥ar)dm(φ,Ba,d(0,r-∥x∥a))=(r-∥x∥ar)d1va(r-∥x∥a)d∫Ba,d(0,r-∥x∥a)‍φ(y)dy≤1vard∫Ba,d(x,r)‍φ(y)dy=m(φ,Ba,d(x,r))≤(r+∥x∥ar)d1va(r+∥x∥a)d∫Ba,d(0,r+∥x∥a)‍φ(y)dy=(r+∥x∥ar)dm(φ,Ba,d(0,r+∥x∥a)).
It remains to let r→∞ to prove (3.37).Step 2 (proving (2.20)).
Ifφ∈CL(Rn), then φ(·+Xβ)∈CL(Rn). By (2.16), we need to look at
(3.39)(M2,dTφ(·+Xβ))(t(2))=limr→∞1vard∫B2,d(0,r)‍φ(P1(a)t(1)+P2(a)Λ2-1(a)t(2)+Xβ)dt(1).
Using I=P(a)P′(a)=P1(a)P1′(a)+Q, we see that
(3.40)P1(a)t(1)+Xβ=P1(a)t(1)+P1(a)P1′(a)Xβ+QXβ=P1(a)(t(1)+P1′(a)Xβ)+QXβ.
Therefore, replacing t(1)+P1′(a)Xβ=s(1), we can continue (3.39) as
(3.41)(M2,dTφ(·+Xβ))(t(2))=limr→∞1vard∫B2,d(P1′(a)Xβ,r)‍φ(P1(a)s(1)+P2(a)Λ2-1(a)t(2)+QXβ)ds(1).
Using the definition of T1 and (3.37), we see that this is the same as (M2,dT1φ)(t(2)). Hence, the existence of (M2,dTφ(·+Xβ))(t(2)) is equivalent to that of (M2,dT1φ)(t(2)). Application of Theorem 2.6 proves (2.20).


#### 3.2.2. Part (b)

Step 1 (the generalized mean vanishes onp-integrable functions).
Ifφ∈Lp(Rd) with some p∈[1,∞), then by Hölder’s inequality
(3.42)|m(φ,Ba,d(0,r))|≤1vard∫Ba,d(0,r)‍|φ(t)|dt≤1vard(mesBa,d(0,r))1-1/p(∫Ba,d(0,r)‍|φ(t)|pdt)1/p=(vard)-1/p∥φ∥Lp⟶0,r⟶∞.Step 2 (limJρ(φ) vanishes on functions with compact support).
By Condition4,
(3.43)|∫Rn‍φ(t)fρ(t)dt|≤∫Rn‍φ~(t)f~ρ(t)dt,
where φ~(t)=|φ(t)| and f~ρ is obtained from (1.9) by replacing g(x) with g~(x)=p(∥x∥2). The function φ~ satisfies the Lipschitz condition, |φ~(x)-φ~(y)|=||φ(x)|-|φ(y)||≤c1∥x-y∥2, and g~ satisfies Condition 2. Part (a) of this theorem allows us to use (2.20) with φ~ in place of φ and g~ instead of g whenever (M2,dT1φ~)(t(2)) exists for almost all t(2)∈Rn-d.
LetCL0(Rn) denote the set of functions φ∈CL(Rn) with compact support suppφ. Suppose φ∈CL0(Rn). By Step 1, (M2,dT1φ~)(t(2))≡0. Equations (2.20) and (3.43) imply that
(3.44)limρ→a∫Rn‍φ(t)fρ(t)dt=0,φ∈CL0(Rn).
Suppose (1.11) is true in the weak sense or, more generally, the family {fρ:0≤ρ<a} converges in distribution, as ρ→a, to a probabilistic measure μ supported on some set S⊂Rn. Then by (2.7) ∫Rn‍φ(t)fρ(t)dt→∫S‍φdμ,φ∈CL(Rn). This equation and (3.44) imply ∫S‍φdμ=0,φ∈CL0(Rn). Taking here φ∈CL0(Rn) such that φ≥0 and φ=1 on B2,n(0,r) we have ∫S∩B2,n(0,r)‍dμ=0 for any r>0. Thus, μ(S)=0 which is impossible.


### 3.3. Proof of Theorem2.9

The proof of Theorem2.9 will be preceded by a series of lemmas. The main difficulty consists in estimating integrals of the marginal density g-1 over cylindrical sets of an infinite Lebesgue measure.


#### 3.3.1. Preliminary Results

Lemma 3.1.
For the rejection regionΦ defined in (1.14), representation (2.21) holds.Proof.
SinceLX,  MX are complementary orthoprojectors, the images ImMX and ImLX=ImX are orthogonal to one another and Rn=ImMX⊕ImX. Representing u∈Φ as u=LXu+MXu and using MX′=MX,  MX2=MX we get
(3.45)u′MXWMXuu′MXu=(MXu)′WMXu(MXu)′MXu=v′Wvv′v,
where we have denoted v=MXu. Hence, only the component v∈MXΦ determines whether u belongs to Φ or not, while the component LXu∈ImX is arbitrary. This proves the equation Φ=MXΦ+ImX. Finally, the equation MXΦ=ΓS follows by homogeneity from (3.45).Now we review a few facts about the Moore-Penrose inverse in a form adapted to our needs. LetA be a matrix of size m×n, not necessarily square, and denote its rank r=rankA. Its Moore-Penrose inverse A+ is of size n×m.Lemma 3.2 (see, [13, pages 24-25]).
A
+ is unique and has the following properties.(a)
AA+A=A (A+ is a generalized inverse of A).(b)
A+AA+=A+ (A is a generalized inverse of A+).(c)
The matrixH=A+A is an orthoprojector, H2=H, H′=H, and rankH=rankA+=r.(d)
For a giveny∈ImA, a general solution of the equation Ax=y is given by
(3.46)x=A+y+(I-H)z,where z∈Rn is arbitrary. (Rao writes (H-I)z instead of (I-H)z, but our form is more convenient).Regarding (3.46), we can remark the following.(i)
The two terms on the right are orthogonal because by (b) and (c)(3.47)(A+y,(I-H)z)=(HA+y,(I-H)z)=(A+y,(H-H2)z)=0.
Here and in the sequel, (·,·) stands for the scalar product.(ii)
Instead of writing the second term in the form(I-H)z,  z∈Rn, we can equivalently write it as v∈Im(I-H).(iii)
A+y=0 implies that y=0. This is because if A+y=0 and y=Ax, then by (a) y=Ax=AA+(Ax)=A(A+y)=0.Recalling that a general solution of the equationAx=y is, by definition, the preimage A-1y, we state our conclusion in a lemma.Lemma 3.3.
LetY⊆ImA. The preimage A-1Y is a cylinder A+Y+Im(I-H) with the base A+Y⊆ImH and element Im(I-H). Further, A+y=0 for y∈ImA implies that y=0, that is, A+ on ImH is one-to-one.Lemma 3.4.
Supposef1∈ImX. Fix a set Y⊆ImMX and a vector τ∈ImMX. Denote A=MXP2(a)Λ2-1(a) and let A+ and H be the accompanying matrices from Lemma 3.2. Then, the set T(2) defined by T(2)={t(2)∈Rn-1:At(2)+τ∈Y} is a cylinder:
(3.48)T(2)=A+(Y-τ)+Im(I-H).
Supposef1∉ImX and denote p=MXf1/∥MXf1∥2. Then MX-pp′ is an orthoprojector with the image
(3.49)Im(MX-pp′)={x∈ImMX:x⊥p}.
In part (a) replace Y,τ,A by Y⊆Im(MX-pp′), τ∈Im(MX-pp′), and A=(MX-pp′)P2(a)Λ2-1(a), respectively. Then the set T(2) has representation (3.48).Proof.
Let us show thatf1∈ImX implies that
(3.50)ImA=ImMX.
The inclusion ImA⊂ImMX is obvious. Conversely, let x∈ImMX. Then, x=MXx. Since f1,…,fn is an orthonormal basis, we have x=∑i=1n‍(x,fi)fi. Denoting t(2)=((x,f2),…,(x,fn))′, it follows that
(3.51)x=MXx=∑i=1n‍(x,fi)MXfi=∑i=2n‍(x,fi)MXfi=(MXf2,…,MXfn)t(2)=MXP2(a)t(2)=AΛ2(a)t(2)∈ImA.
We have proved (3.50) which implies Y-τ⊂ImA. Application of Lemma 3.3 proves (3.48).
It is easy to check thatMX-pp′ is an orthoprojector and its image satisfies (3.49). From (3.49), we conclude that MX and MX-pp′ commute and
(3.52)(MX-pp′)MX=MX-pp′,
see [14, Chapter VI, Section 3, Lemma 2(a)]. Applying MX-pp′ to both sides of (3.50) and using the last equation we get
(3.53)Im[(MX-pp′)P2(a)Λ2-1(a)]=Im[(MX-pp′)MXP2(a)Λ2-1(a)]=(MX-pp′)[ImMXP2(a)Λ2-1(a)]=(MX-pp′)ImMX=Im(MX-pp′).
We have proved an analog of (3.50) for the matrix under consideration:
(3.54)ImA=Im(MX-pp′).
As previously mentioned, application of Lemma 3.3 proves (3.48).Lemma 3.5.
Letf1,τ,A,A+ and H be the same as in Lemma 3.4(a).(a)
SupposeY⊆ImMX is of the form Y=ΓΔ where Δ⊆{s∈ImMX:∥s∥2=1}. If (2.23) holds, then for the set T(2) from (3.48) one has
(3.55)α(Y,τ)≡∫T(2)‍g-1(t(2))dt(2)≤c(∥τ∥2)mes(Δ).(b)
IfY⊆{s∈ImMX:∥s∥2≤1}, then α(Y,τ)→0,as mes(Y)→0, uniformly in τ.Proof.
Part (a)
Step  1 (proving an auxiliary bound). Denote
(3.56)β(Y,τ)=∫A+(Y-τ)‍g-k(∥u∥2)du.
We intend to prove that
(3.57)β(ΓΔ,τ)≤c(∥τ∥2)mes(Δ).
By Lemma 3.2, (3.50) implies that
(3.58)r≡rankA=n-k.
By Lemma 3.3, A+ is a one-to-one mapping from ImA to ImA+, both of which are of dimension r. We use this fact twice.
Firstly, selecting two bases, one inImA and another in ImA+, we can identify A+ with its matrix representation in these bases. This matrix is square of order r and rank r, so the Jacobian c1 of the transformation u=A+v,  v∈ΓΔ-τ, is positive. Therefore,
(3.59)β(ΓΔ,τ)=c1∫ΓΔ-τ‍g-k(∥A+v∥2)dv=c1∫ΓΔ‍g-k(∥A+(w-τ)∥2)dw.
Secondly,∥A+v∥2=0 implies that A+v=0 which for v∈ImA implies that v=0. Thus, ∥A+v∥2 is a norm on ImA and by (3.14),
(3.60)∥A+(w-τ)∥2≥c2∥w-τ∥2≥max{c2(∥w∥2-∥τ∥2),0},w∈ΓΔ.
Asg- is assumed nonincreasing, g-k is also nonincreasing, and (3.59) leads to
(3.61)β(ΓΔ,τ)≤c1∫ΓΔ‍g-k(max{c2(∥w∥2-∥τ∥2),0})dw.
We bound this using polar coordinates:
(3.62)β(ΓΔ,τ)≤c1∫Δ‍[∫∥τ∥2∞‍g-k(c2(ρ-∥τ∥2))ρr-1dρ]dσ=c3mes(Δ).
Here, the quantity
(3.63)∫∥τ∥2∞‍g-k(c2(ρ-∥τ∥2))ρr-1dρ=1c2∫0∞‍g-k(ρ)(ρc2+∥τ∥2)r-1dρ
is finite because by condition (2.23), g-k is integrable in the neighborhood of zero and (ρ/c2+∥τ∥2)r-1 is of order ρr-1 at infinity.
Step  2 (proving the main bound). The matrix A from Lemma 3.4(a) is of size n×(n-1) and A+ is of size (n-1)×n. Equation (3.50) and Rn-1=ImH⊕Im(I-H) imply that
(3.64)rank(I-H)=n-1-r=k-1.
Equation (3.48) written point-wise is t(2)=u+v,  t(2)∈T(2),  u∈A+(ΓΔ-τ),  v∈Im(I-H). Since u,v here are orthogonal, we have
(3.65)g-1(t(2))=∫R‍p-((t12+∥u∥22+∥v∥22)1/2)dt1≡h(∥u∥2,∥v∥2),
where the right side defines the function h.
The transition fromt(2) to the coordinates (u,v) is described by an orthogonal matrix. This can be shown as follows. Take in ImH, an orthonormal basis and identify u with the set of its coordinates in that basis. Do the same with Im(I-H) and v. Join the two bases. The equation t(2)=u+v induces a transformation between the original coordinates of t(2) and the ones in the joint basis. Since ImH⊥Im(I-H), the Jacobian of the resulting transformation is unity. Hence, by (3.48), (3.64), (3.65), and (3.57),
(3.66)α(Y,τ)=∫A+(ΓΔ-τ)+Im(I-H)‍g-1(u+v)dudv=∫A+(ΓΔ-τ)‍[∫Rk-1‍h(∥u∥2,∥v∥2)dv]du=∫A+(ΓΔ-τ)‍g-k(∥u∥2)du=β(ΓΔ,τ)≤c(∥τ∥2)mes(Δ).
Part (b). In the bound (see (3.59) and (3.60)),
(3.67)β(Y,τ)=c1∫Y-τ‍g-k(∥A+v∥2)dv≤c1∫Y-τ‍g-k(c2∥v∥2)dv,
we have mes(Y-τ)=mes(Y)→0, so β(Y,τ)→0 uniformly in τ. As in (3.66), it follows that(3.68)α(Y,τ)=∫A+(Y-τ)+Im(I-H)‍g-1(u+v)dudv=β(Y,τ)⟶0.Lemma 3.6.
Letf1,p,τ,A,A+, and H be the same as in Lemma 3.4(b).(a)
SupposeY=ΓΔ where Δ⊆{s∈Im(MX-pp′):∥s∥2=1}. If (2.25) holds, then (3.55) is true with T(2) from Lemma 3.4(b).(b)
IfY⊆{s∈Im(MX-pp′):∥s∥2≤1}, then α(Y,τ)→0, as mes(Y)→0, uniformly in τ.Proof.
The proof essentially follows that of Lemma3.5(a). From (3.49), and (3.54) we see that instead of (3.58) we have r=n-k-1. Instead of (3.64) we have rank(I-H)=k. This leads to the definition of the marginal density used in (2.25). With these changes, (3.62), and (3.66) are true and prove part (a).
With the same changes the proof of Lemma3.5(b) goes through.In the course of the proof, we approximateχΦ by functions φε±∈CL(Rn) from below and above. Then, we show that the rate of approximation improves as ε→0 and ρ→a.Step 1 (constructing the approximating functions).
Denoteρ(x,A)=infy∈A∥x-y∥2 the distance from a point x to a set A. For a given ε>0, set Ψε+={x∈Rn:ρ(x,Φ)≤ε},  Ψε-={x∈Φ:ρ(x,bd(Φ))>ε}.Obviously,Ψε-⊂Φ⊂Ψε+. As in [10, page 12 proof of (ii)⇒(iii)], we can construct functions φε± that are uniformly continuous on Rn and satisfy
(3.69)φε+=1onΦ,φε+=0outsideΨε+,0≤φε+≤1onRn,φε-=1onΨε-,φε-=0outsideΦ,0≤φε-≤1onRn.Step 2 (reducing the convergence statement to a special case).
Denote(3.70)I(φ)=∫Rn-1(MT1φ)(t(2))g-1(t(2))dt(2).
Here, we show that
(3.71)limε→0I(φε±)=I(χΦ)
implies (2.24). From the inequality φε-≤χΦ≤φε+ we have
(3.72)Jρ(φε-)≤Jρ(χΦ)≤Jρ(φε+).
By Theorem 2.7,
(3.73)limρ→aJρ(φε±)=I(φε±),foranyε>0.
By (3.71), for any δ>0, there exists ε(δ)>0 such that
(3.74)I(φε+)-δ≤I(χΦ)≤I(φε-)+δ,0<ε≤ε(δ).
By (3.73), for the same δ, there is ρ(δ)∈(0,a) such that
(3.75)|Jρ(φε±)-I(φε±)|≤δ,ρ(δ)≤ρ<a.
Equations (3.72), (3.74), and (3.75) imply that
(3.76)I(χΦ)-2δ≤I(φε-)-δ≤Jρ(φε-)≤Jρ(χΦ)≤Jρ(φε+)≤I(φε+)+δ≤I(χΦ)+2δ,ρ(δ)≤ρ<a.
Since δ>0 is arbitrary, this proves (2.24).Step 3 (further reduction to an indicator function).
Let us callB(ε)=Ψε+∖Ψε- a band. Here, we show that (3.71) will follow if we prove that
(3.77)limε→0I(χB(ε))=0.
Note the identity
(3.78)I(χB(ε))=I(χΨε+)-I(χΨε-)=[I(χΨε+)-I(χΦ)]+[I(χΦ)-I(χΨε-)].
Since each of the square brackets on the right is nonnegative, (3.77) gives limε→0I(χΨε±)=I(χΦ). Now, it suffices to take into account the inequality I(χΨε-)≤I(φε-)≤I(χΦ)≤I(φε+)≤I(χΨε+) to see that (3.71) follows.Step 4 (geometric description of the task).
Denotel(s,i)={st+i:t∈R} a straight line with a slopes∈Rn and intercepti∈Rn and put
(3.79)bε(t(2))=(MT1χB(ε))(t(2))=limr→∞12r∫-rr‍χB(ε)(f1t1+z(t(2)))dt1.
This expression shows that we need to study the set {t1∈R:f1t1+z(t(2))∈B(ε)} of points common to the straight line l(f1,z(t(2))) and the band B(ε). The cylindrical nature of the sets considered implies that the band admits an equivalent definition as a cylinder with the base
(3.80)MXB(ε)={x∈ImMX:ρ(x,bd(MXΦ))≤ε}
and element ImX. From this definition, we see that the condition f1t1+z(t(2))∈B(ε) is equivalent to
(3.81)(MXf1)t1+MXz(t(2))∈MXB(ε).
We have to sort out points t(2)∈Rn-1 into (a) those for which the set of points t1 satisfying (3.81) is so small that (3.79) is zero and (b) that set is large enough for (3.79) to be positive.Step 5 (casef1∈ImX).
In this case,MXf1=0, and (3.81) reduces to
(3.82)MXP2(a)Λ2-1(a)t(2)+MXQXβ∈MXB(ε).
This condition does not contain t1. Therefore, if it is not satisfied, the integrand in (3.79) is identically zero and bε(t(2))=0 for such t(2). If it is satisfied, the integrand in (3.79) is identically 1 and bε(t(2))=1 for such t(2). Hence,
(3.83)I(χB(ε))=∫(3.82)istrue‍g-1(t(2))dt(2).
The set of t(2) satisfying (3.82) has an infinite Lebesgue measure which precludes us from application of the continuity property of the Lebesgue measure (for an integrable h,∫A‍h(t)dt→0 as mes(A)→0).
As one can see from Figure1, MXB(ε) is not a cone-like set. We divide it into two parts: αεin={s∈MXB(ε):∥s∥2≤1} includes the points inside the unit ball and on the unit sphere and αεout={s∈MXB(ε):∥s∥2>1} includes the points of MXB(ε) outside the unit ball. Put Δε={s∈ImMX:∥s∥2=1,ρ(s,bd(MXΦ))≤ε} and let us show that
(3.84)αεout⊂ΓΔε.
Suppose x∈αεout. Then, x∈ImMX,∥x∥2>1 and by (3.80), there exists a sequence {y(k)}⊂bd(MXΦ) such that limsup∥x-y(k)∥2≤ε. The sequence {y(k)} is bounded and therefore precompact, while bd(MXΦ)=MX(cl(Φ)∖Φ) is closed by construction. Hence, we can assume that y(k)→y∈bd(MXΦ) (passing to a subsequence, if necessary). Then, ∥x-y∥2≤ε. This implies  ∥x/∥x∥2-y/∥x∥2∥2≤ε/∥x∥2<ε. Let s=x/∥x∥2. Since bd(MXΦ) is cone-like, it follows that y/∥x∥2∈bd(MXΦ) and ρ(s,bd(MXΦ))<ε. This proves that s∈Δε,  x=∥x∥2s∈ΓΔε and (3.84) is true.
From (3.84), we conclude that
(3.85)MXB(ε)⊂αεin∪ΓΔε.
This means that the set of solutions of (3.82) is contained in the union of two sets: T1(2)={t(2)∈Rn-1:At(2)+τ∈ΓΔε} and T2(2)={t(2)∈Rn-1:At(2)+τ∈αεin} where A=MXP2(a)Λ2-1(a) and τ=MXQXβ.
By Lemma3.4(a), T1(2)=A+(ΓΔε-τ)+Im(I-H). By Lemma 3.5(a)
(3.86)∫T1(2)‍g-1(t(2))dt(2)≤c(∥τ∥2)mes(Δε)⟶0,ε→0.
By Lemma 3.4(a), T2(2)=A+(αεin-τ)+Im(I-H). By Lemma 3.5(b)
(3.87)∫T2(2)‍g-1(t(2))dt(2)⟶0asmes(αεin)⟶0.
Equations (3.86) and (3.87) prove that I(χB(ε))→0,  ε→0.Figure 1
M
X
Φ consists of two cones bounded by solid lines. MXB(ε) is bounded by dashed lines.Step 6 (casef1∉ImX).
Equation (2.5) implies that if the straight line l(MXf1,MXz(t(2))) has a bounded intersection with the band MXB(ε), then (MT1χB(ε))(t(2))=0. Therefore, we are interested in the set of slopes:
(3.88)Sl={MXf1:l(MXf1,MXz(t(2)))∩MXB(ε)isunbounded}.
Let us prove that
(3.89)MXf1∈Sl⇒MXf1/∥MXf1∥2∈bd(S).
Suppose MXf1∈Sl. Then, there exists a sequence of points p(k) which lie on the straight line, p(k)=(MXf1)t1(k)+MXz(t(2)), where |t1(k)|→∞, and which are 2ε-close to bd(MXΦ): there exists a sequence y(k)∈bd(MXΦ) such that
(3.90)∥p(k)-y(k)∥2≤2ε,∥y(k)∥2⟶∞.
Equation (2.21) implies that MXΦ=ΓS and bd(MXΦ)=Γbd(S)∪{0}. Therefore s(k)≡y(k)/∥y(k)∥2∈bd(S). Because bd(S) is compact, we can assume that s(k)→s∈bd(S). From (3.90), we have ∥(MXf1)t1(k)+MXz(t(2))-∥y(k)∥2s(k)∥2≤2ε or, dividing both sides by ∥y(k)∥2, ∥(MXf1)t1(k)/∥y(k)∥2-s(k)∥2→0. This shows that the sequence {t1(k)/∥y(k)∥2} is bounded and can be assumed convergent to a number δ. Letting k→∞, we get (MXf1)δ=s∈bd(S), and (3.89) is true.
FixMXf1∈Sl and denote p=MXf1/∥MXf1∥2∈bd(S)⊂ImMX. The matrix pp′ projects Im MX onto the straight line l(MXf1,0). The projector MX-pp′ projects Im  MX onto Im(MX-pp′). Condition (3.81) implies two equations in projections:
(3.91)pp′[(MXf1)t1+MXz(t(2))]∈pp′MXB(ε),(MX-pp′)[(MXf1)t1+MXz(t(2))]∈(MX-pp′)MXB(ε).
The first one can be safely omitted because the shift pp′MXz(t(2)) along a line does not change the property of the line to have an unbounded intersection with MXB(ε). Denoting A=(MX-pp′)P2(a)Λ2-1(a),  τ=(MX-pp′)QXβ, and using (3.52), we write the second equation as
(3.92)At(2)+τ∈(MX-pp′)B(ε).
The set of points t(2) such that (3.81) holds for an unbounded set of values t1 is clearly contained in the set T(2) of points t(2) that satisfy (3.92). For t(2)∈T(2) we use an obvious bound bε(t(2))≤1 (see (3.79)), and, therefore, I(χB(ε))≤∫(3.92)istrue‍g-1(t(2))dt(2). The rest of the proof closely follows that for the case f1∈ImX. An inclusion of type (3.85) holds with MX replaced by MX-pp′ in all definitions. The reference to Lemma 3.4(a) is replaced by Lemma 3.4(b). Lemma 3.6 is used instead of Lemma 3.5. The conclusion is that I(χB(ε))→0.We shall use several times the fact that for the densityg-1 in representation (2.24)will, the conditions ∫A‍g-1(t(2))dt(2)>0 and mes(A)>0 are equivalent. From (2.24), we see that the task is to calculate the function (3.79) with Φ in place of B(ε). Because of (2.21), the condition that f1t1+z(t(2))∈Φ is equivalent to the condition in projections:
(3.93)(MXf1)t1+MXz(t(2))∈ΓS.The magnitude ofβ(Φ) depends on the set
(3.94)A(t(2))={t1∈R:f1t1+z(t(2))∈Φ}.The properties ofA(t(2)) depend on those of mes{t(2)∈Rn-1:MXz(t(2))∈G} for certain sets G⊆ImMX, as we will see.Lemma 3.7.
SupposeImQ⊆ImX (Q is the projector from Theorem 2.7). Then, MXz(t(2))=0 for all t(2)∈Rn-1.
LetImQ⊈ImX. Then, mes{t(2):MXz(t(2))=0}=0.Proof.
The assumptionImQ⊆ImX implies that MXfi=0 for i=2,…,n, which can be written as MXP2(a)=0. Hence, MXz(t(2))=MXP2(a)[Λ2-1(a)t(2)+P2′(a)Xβ]=0 for any t(2)∈Rn-1.
IfImQ⊈ImX, then fi∉ImX and MXfi≠0 for at least one i. The matrix MXP2(a) is of size n×(n-1), and of rank at least 1. From the equation dimN(MXP2(a))+rank(MXP2(a))=n-1 we see that the null space N(MXP2(a)) is of dimension at most n-2. The set of t(2) satisfying MXz(t(2))=MXP2(a)Λ2-1(a)t(2)+MXQXβ=0 is a partial solution t0(2) plus an arbitrary element from the null space N(MXP2(a)). Thus the set of all solutions is a hyperplane in Rn-1 of dimension at most n-2 and has (n-1)-dimensional Lebesgue measure zero.Lemma 3.8.
Suppose we are given a setG⊆Rn-1 and for almost each t(2)∈G, the set A(t(2))⊆R is defined by (3.94).(a)
IfA(t(2)) is bounded for almost all t(2)∈G and mes(G)>0, then β(Φ)<1. In particular, if G=Rn-1, then β(Φ)=0.(b)
If(3.95)R∖A(t(2))isboundedforalmostallt(2)∈G
and mes(G)>0, then β(Φ)>0. In particular, if G=Rn-1, then β(Φ)=1.(c)
Suppose thatG=Rn-1 and for almost all t(2)∈G,
(3.96)thesetA(t(2))isahalf-axis
(i.e., either A(t(2))=(a(t(2)),∞) or A(t(2))=(-∞,a(t(2))) with some a(t(2))∈R). Then, β(Φ)=1/2.Proof.
IfA(t(2)) is bounded, then χΦ(f1t1+z(t(2)))=0 outside a bounded set, and by (2.5),
(3.97)MχA(t(2))=(MT1χΦ)(t(2))=0
for almost all t(2)∈G. Since 0≤MχA(t(2))≤1 outside G, we have
(3.98)β(Φ)≤∫Rn-1∖G‍g-1(t(2))dt(2)<1.
Obviously, the integral is null if G=Rn-1.
By (3.97), condition (3.95) implies that
(3.99)MχA(t(2))=M(1-χR∖A(t(2)))=1a.e.inG.
Since mes(G)>0, we have
(3.100)β(Φ)≥∫G‍g-1(t(2))dt(2)>0.
In case G=Rn-1, the last integral is 1.
Consider, for example, the case of the right half-axis:(3.101)MχA(t(2))=limr→∞12r(∫a(t(2))0‍dt+∫0r‍dt)=12.
Hence, β(Φ)=(1/2)∫Rn-1‍g-1(t(2))dt(2)=1/2.Case 1.
Letf1∈Φ. Then, f1∉ImX and MXf1≠0. Define p=MXf1/∥MXf1∥2. Obviously, MXf1∈ΓS and p∈S. In the equation
(3.102)(MXf1)t1+MXz(t(2))=∥MXf1∥2t1[p+MXz(t(2))∥MXf1∥2t1],
for all large |t1|, the term in the square brackets is close to p and, because MXΦ is open in ImMX, this term belongs to MXΦ. Since MXΦ is cone-like, the left side of (3.102) belongs to MXΦ for all large |t1|. Hence, the set A(t(2)) contains the neighborhood of infinity (more precisely, all t1 with |t1|>N where N depends on t(2)). By Lemma 3.8(b), where G=Rn-1, then β(Φ)=1.Case 2.
Supposef1∈bd(Φ).
Subcase2.1. Let f1∈ImX or, equivalently, MXf1=0. Condition (3.93) reduces to
(3.103)MXz(t(2))=MXP2(a)Λ2-1(a)t(2)+MXQXβ∈ΓS.
By Lemma 3.4(a), the set of solutions of (3.103) is T(2)=A+(ΓS-τ)+Im(I-H) where A+ is the Moore-Penrose inverse of A=MXP2(a)Λ2-1(a),τ=MXQXβ, and H is the projector from Lemma 3.2. Using Lemma 3.8(b), (3.66), and (3.59) we get
(3.104)β(Φ)=∫T(2)‍g-1(t(2))dt(2)=∫A+(ΓS-τ)‍g-k(∥u∥2)du.
The density g-k(∥u∥2) is spherically symmetric. It is positive everywhere because
(3.105)g-k(t)=∫Rk‍p-((t2+∥v∥22)1/2)dv=0
would mean p-((t2+t1)1/2)=0 for any t1≥0, that is, p-(t1)=0 for any t1≥t, which is impossible by assumption. A+ is one-to-one on ImA=ImMX, so A+(ΓS-τ) is a nonempty proper subset of ImA+. Therefore, 0<β(Φ)<1.
Subcase 2.2. Suppose f1∉ImX. As in Case  1, p is defined. Let us show that it satisfies
(3.106)p′Wp=c.
Since f1 is a limit point of Φ, there is a sequence xn∈Φ such that xn→f1. Hence, MXxn→MXf1 where ∥MXf1∥2≠0 because f1∉ImX. From (2.21), it follows that MXxn∈ΓS,zn≡MXxn/∥MXxn∥2∈S and zn′Wzn>c. Letting n→∞ and using zn→p, we get p′Wp≥c. Since f1∉Φ, we have MXf1∉MXΦ and p∉S. Thus, p′Wp>c is impossible and (3.106) is true.
In the three examples in the following we assume thatn>k+1. Then, ImQ⊈ImX, and by Lemma 3.7(b),
(3.107)MXz(t(2))≠0,foralmostallt(2)∈Rn-1.Sub-sub-case (a) (Example ofβ(Φ)=1). Let W be symmetric. Assumptions (1.13) imply that var(u)=σε2(I-ρW)-2 whenever det(I-ρW)≠0, so Σ-1(ρ)=(I-ρW)2. If μi denote eigenvalues of W, then the eigenvalues of I-ρW are νi(ρ)=1-ρμi. Let the constant c in the definition of the critical region Φ be positive. Put a=1/c and with some integer i0 and real numbers μ-,  μ+ such that 1<i0<n,  μ-<1/a<μ+ define μi by
(3.108)μ1=⋯=μi0-1=μ-<μi0=1a<μi0+1=⋯=μn=μ+.
Then, for 0<ρ≤a,
(3.109)ν1(ρ)=⋯=νi0-1(ρ)=1-ρμ->νi0(ρ)=1-ρa>νi0+1(ρ)=…=νn(ρ)=1-ρμ+,
and Σ-1/2(a) has eigenvalues
(3.110)ν1(a)=⋯=νi0-1(a)=1-aμ->νi0(a)=0<|νi0+1(a)|=⋯=|νn(a)|=|1-aμ+|.
Hence, in the notation of Condition 3,  λ1(ρ)=νi0(ρ) and λ2(ρ),…,λn(ρ) are obtained from |νi(ρ)|,  i≠i0, by reordering. For example, if 1-aμ-<|1-aμ+|, then λi(ρ)=νi-1(ρ),i=2,…,i0;  λi(ρ)=|νi(ρ)|,  i=i0+1,…,n. Assuming this ordering, denote the corresponding eigenvectors of Σ-1/2(a) by f1,f2-,…,fi0-,fi0+1+,…,fn+ where the minuses and pluses remind us that the corresponding eigenvalues depend on μ- and μ+, respectively.
Letk=i0-1 and suppose that
(3.111)f2-,…,fi0-spanImX,f1,fi0+1+,…,fn+spanImMX.
Denote
(3.112)x=(MXf1)t1+MXz(t(2))∈ImMX.
Because of (3.111), here MXf1=f1. Besides, since z(t(2)) is orthogonal to f1, the decomposition
(3.113)MXz(t(2))=∑i=i0+1n‍cifi+
does not contain f1. Hence, (3.112) becomes
(3.114)x=t1f1+∑i=i0+1n‍cifi+.
Remembering that the eigenvectors of Σ-1/2(a) are just reordered eigenvectors of W, we get
(3.115)Wx=1at1f1+μ+∑i=i0+1n‍cifi+,x′Wx=1at12+μ+∑i=i0+1n‍ci2=ct12+μ+∑i=i0+1n‍ci2,
for any x of form (3.112). From (3.107) and (3.113) we see that here
(3.116)∑i=i0+1n‍ci2≠0a.e.inRn-1.
Hence, normalizing x,  s=x/∥x∥2, and assuming that μ+>c, we get from (3.115)
(3.117)s′Ws=ct12+μ+∑i=i0+1nci2t12+∑i=i0+1nci2>c,
for all t1∈R and almost all t(2)∈Rn-1. This shows that the set (3.94) coincides with R for almost all t(2)∈Rn-1 and by Lemma 3.8(b) β(Φ)=1.
Sub-sub-case (b)Example ofβ(Φ)=0. We change (3.111) so as to obtain in (3.115) μ- in place of μ+. Let k=n-i0 and suppose that
(3.118)fi0+1+,…,fn+spanImX,f1,f2-,…,fi0-spanImMX.
Instead of (3.113), we get MXz(t(2))=∑i=2i0‍cifi- which leads to an analog of (3.114): x=t1f1+∑i=2i0‍cifi-.  Equation(3.115) is replaced by
(3.119)x′Wx=ct12+μ-∑i=2i0‍ci2,
where ∑i=2i0‍ci2≠0 a.e. in Rn-1. The result is that
(3.120)s′Ws=ct12+μ-∑i=2i0ci2t12+∑i=2i0ci2<c
for all t1∈R and almost all t(2)∈Rn-1. Hence, the set (3.94) is empty for almost all t(2)∈Rn-1 and by Lemma 3.8(a), β(Φ)=0.
Sub-sub-case
(
c
)
Example of β(Φ)=1/2]. MXz(t(2)) runs over Im(MXP2(a)). If k>1, then n-1>n-k, and we can choose the eigenvectors f2,…,fn of Σ-1/2(a) so that ImQ⊃ImMX, and then MXz(t(2)) runs over ImMX=Im(MXP2(a)). Let ΓS be a union of two cones, as in Figure 2. The line l(MXf1,0) lies on the boundary of the cones. In Figure 2, La=l(MXf1,0) and Lb=l(b(t(2)),0). For any nonzero shift MXz(t(2)), half of the shifted line is contained in MXΦ because La and Lb are not parallel. Here, “half” means “either for all large positive t1 or for all large negative t1.” By Lemma 3.8(c), β(Φ)=1/2.Figure 2
The base ofΦ contains the area between two solid lines La and  Lb. The dashed line is l(MXf1,MXz(t(2)))=La+MXz(t(2)).Case 3.
Supposef1∈Φ-. Then, as in part 1,MXf1≠0 and p is defined. It satisfies p′Wp<c. The term in the square brackets in (3.102) for all large |t1| belongs to an open set MXΦ-. Since this set is cone-like, the left side of (3.102) belongs to MXΦ- for all large |t1|. The set A(t(2)) is bounded for all t(2)∈Rn-1, and by Lemma 3.8(a), β(Φ)=0.Step 1 (theε(ρ)-maximizing set is an ellipsoid).
Fory∈Mε(ρ) by orthogonality of P and Condition 7, we have
(3.121)fρ(y)=∏j=1n‍λj(ρ)p(∥Λ(ρ)P′(ρ)(y-Xβ)∥2)≥(1-ε(ρ))∏j=1n‍λj(ρ)m.
Since p-1 is decreasing, this is equivalent to ∥Λ(ρ)P′(ρ)(y-Xβ)∥2≤p-1((1-ε(ρ))m). Denoting h(ρ)=p-1((1-ε(ρ))m),  z=P′(ρ)(y-Xβ), we rewrite the resulting inequality as
(3.122)∑j=1d‍zj2[h(ρ)/λj(ρ)]2+∑j=d+1n‍zj2[h(ρ)/λj(ρ)]2≤1.
In z coordinates, this is an ellipsoid with half-axes h(ρ)/λj(ρ).Step 2 (characterizing the rate of decay ofh(ρ)).
First we determine the rate with whichp-1(x) approaches 0 when x→m. To approximately invert p, in the equation x=p(r), we use representation (2.30): x=m(1-cprαp)(1+o(rαε)). This is equivalent to (x/m)(1+o(rαε))=1-cprαp. Solving this equation for the r involved in the right-hand side produces
(3.123)p-1(x)=r={1cp[1-xm(1+o(rαε))]}1/αp,r→0.
Since ρ→a, we can put r=a-ρ in the last equation to get
(3.124)p-1(x)={1cp[1-xm(1+o((a-ρ)αε))]}1/αp,ρ→a.
To obtainh(ρ), let x=(1-ε(ρ))m. Then, by (2.29),
(3.125)1-xm(1+o((a-ρ)αε))=1-(1-ε(ρ))(1+o((a-ρ)αε))=1-1+o((a-ρ)αε)+ε(ρ)(1+o((a-ρ)αε))=ε(ρ)(1+o(1)).
Combining this with (3.124) and (2.29), we have
(3.126)h(ρ)=p-1(x)={cεcp(a-ρ)αε(1+o(1))}1/αp=(cεcp)1/αp(a-ρ)αε/αp(1+o(1)).Step 3 (behavior of half-axes).
We intend to show that forj=d+1,…,n the half-axes tend to zero, so that in the limit
(3.127)zd+1=⋯=zn=0,
whereas for j=1,…,d, they tend to ∞, and in the limit, z1,…,zd can be any real numbers.
Forj=d+1,…,n from (2.28), we see that the condition h(ρ)/λj(ρ)→0 is equivalent to h(ρ)→0. Because of (2.31), this condition is the same as (1-ε(ρ))m→m which holds by (2.29).
For the remaining half-axes, we get by (2.27), (3.126), and (2.32)
(3.128)h(ρ)λj(ρ)=1cλ(cεcp)1/αp(a-ρ)αε/αp-αλ(1+o(1))⟶∞.
We have proved our statement on convergence of the half-axes. According to (2.26), (3.127) rewrites as (P′(a)(y-Xβ))j=0,  j=d+1,…,n. Since the rows of P′(a) are eigenvectors of A(a), the last equation means that y-Xβ is orthogonal to eigenvectors numbered j=d+1,…,n. Hence, y-Xβ belongs to the null space of A(a), and the set of all such y’s is Xβ+N(A(a)).
In this proof, the setXβ+N(A(a)) is obtained as the limit of the ridge of fρ. In general, this ridge can have any shape and the limit will not be Xβ+N(A(a)).


## 3.1. Proof of Theorem2.6

DenoteJρ(φ)=∫Rn‍φ(t)detA(ρ)g(A(ρ)t)dt.Step 1 (replacinglimJρ(φ) by an equivalent limit).
Using constancy ofg- on spheres (Condition 2), write Jρ(φ) as
(3.22)Jρ(φ)=∫Rn‍φ(y)detΛ(ρ)g-(P(ρ)Λ(ρ)P′(ρ)y)dy=detΛ(ρ)∫Rn‍φ(y)g-(Λ(ρ)P′(ρ)y)dy=detΛ(ρ)∫Rn‍φ(y)g-(Λ~(ρ)Λ~-1(ρ)Λ(ρ)P′(ρ)y)dy.
Denote H-1(ρ)=Λ~-1(ρ)Λ(ρ)P′(ρ) and replace H-1(ρ)y=t to get
(3.23)Jρ(φ)=detΛ(ρ)detH(ρ)∫Rn‍φ(H(ρ)t)g-(Λ~(ρ)t)dt.
By Condition 3(b,c),
(3.24)detΛ(ρ)=λ1d(ρ)detΛ2(a)(1+o(1)),detH(ρ)=det(Λ~-1(ρ)Λ(ρ))detP′(ρ)=1+o(1).
From (3.23) and (3.24) we deduce
(3.25)Jρ(φ)=λ1d(ρ)detΛ2(a)(1+o(1))∫Rn‍φ(H(ρ)t)g-(Λ~(ρ)t)dt=(1+o(1))×detΛ2(a)∫Rn-d‍[λ1d(ρ)∫Rd‍φ(H(ρ)t)g-(λ1(ρ)t(1),Λ2(a)t(2))dt(1)]dt(2).Step 2 (changing the limit variable).
Let us have a closer look at the expression in the square brackets in (3.25):
(3.26)Jρ(φ,t(2))≡λ1d(ρ)∫Rd‍φ(H(ρ)t)g-(λ1(ρ)t(1),Λ2(a)t(2))dt(1).
Since Λ-1(a)Λ~(a)=I, here H(a)=P(a), and by Condition 3(c),
(3.27)∥H(ρ)-P(a)∥=∥P(ρ)Λ-1(ρ)Λ~(ρ)-P(ρ)+P(ρ)-P(a)∥≤∥P(ρ)∥∥Λ-1(ρ)Λ~(ρ)-I∥+∥P(ρ)-P(a)∥=o(λ1(ρ)),ρ⟶a.
Considering λ=λ1 the independent variable, take ρ=ρ(λ) as its function. The limit relation ρ→a translates to λ→0, and (3.26) and (3.27) become,
(3.28)Jρ(λ)(φ,t(2))=λd∫Rd‍φ(H(ρ(λ))t)g-(λt(1),Λ2(a)t(2))dt(1),(3.29)∥H(ρ(λ))-P(a)∥=o(λ),λ⟶0,
respectively.Step 3 (proving a preliminary version of (2.18)).
Now we check that Theorem2.2 applies to (3.28) for any fixed t(2). Letting
(3.30)p(z)=p-((z2+∥Λ2(a)t(2)∥22)1/2),z≥0,g(t(1))=g-(t(1),Λ2(a)t(2)),
we see that g∈L1(Rd) is ∥·∥2-spherically symmetric:
(3.31)g(t(1))=g-(t(1),Λ2(a)t(2))=p-((∥t(1)∥22+∥Λ2(a)t(2)∥22)1/2)=p(∥t(1)∥2).
Condition 1(a) is satisfied. As for Condition 1(b), partition H=(H1,H2) conformably with (2.14) and put F(λ)=H1(ρ(λ)),  G(λ)=H2(ρ(λ))t(2),  F(0)=H1(a)=P1(a),  G(0)=P2(a)t(2). Then, H(ρ(λ))t=H1(ρ(λ))t(1)+H2(ρ(λ))t(2)=F(λ)t(1)+G(λ), and Condition 1(b) follows from (3.29).
By Theorem2.2,
(3.32)Jρ(λ)(φ,t(2))⟶Jρ(0)(φ,t(2))≡(M2,dφ-)(t(2))∫Rd‍g-(t(1),Λ2(a)t(2))dt(1),t(2)∈Rn-d,
where (M2,dφ-)(t(2)) is obtained by applying M2,d to φ-(t)≡φ(P1(a)t(1)+P2(a)t(2)) with respect to t(1). Functions (3.28) are bounded by an integrable function of t(2):
(3.33)|Jρ(λ)(φ,t(2))|≤λd∥φ∥C∫Rd‍g-(λt(1),Λ2(a)t(2))dt(1)=∥φ∥C∫Rd‍g-(t(1),Λ2(a)t(2))dt(1).
By the dominated convergence theorem, (3.32) and (3.33) imply that
(3.34)∫Rn-d‍Jρ(λ)(φ,t(2))dt(2)⟶∫Rn-d‍Jρ(0)(φ,t(2))dt(2),λ→0.
Now, we combine (3.25), (3.28), (3.32), and (3.34) to get
(3.35)limρ→aJρ(φ)=detΛ2(a)∫Rn-d‍[(M2,dφ-)(t(2))∫Rd‍g-(t(1),Λ2(a)t(2))dt(1)]dt(2)=∫Rn-d‍[(M2,dφ-)(Λ2-1(a)s(2))∫Rd‍g-(t(1),s(2))dt(1)]ds(2)=∫Rn-d‍(M2,dTφ)(s(2))g-1(s(2))ds(2)
because
(3.36)(M2,dφ-)(Λ2-1(a)s(2))=limr→∞m(φ(P1(a)·+P2(a)Λ2-1(a)s(2),B2,d(0,r)))=(M2,dTφ)(s(2)).


## 3.2. Proof of Theorem2.7

### 3.2.1. Part (a)

Step 1 (Ma,dφ does not depend on the center of balls).
Let us make definition (2.4) more precise: (Ma,dφ)(x)=limr→∞m(φ,Ba,d(x,r)),  x∈Rd. Here, we prove that
(3.37)(Ma,dφ)(x)=(Ma,dφ)(0)foranyx∈Rd.
For r≥∥x∥a, one has implications ∥y∥a≤r-∥x∥a⇒∥y-x∥a≤r⇒∥y∥a≤∥y-x∥a+∥x∥a≤r+∥x∥a. Geometrically, they mean the inclusion Ba,d(0,r-∥x∥a)⊂Ba,d(x,r)⊂Ba,d(0,r+∥x∥a). This inclusion implies that
(3.38)(r-∥x∥ar)dm(φ,Ba,d(0,r-∥x∥a))=(r-∥x∥ar)d1va(r-∥x∥a)d∫Ba,d(0,r-∥x∥a)‍φ(y)dy≤1vard∫Ba,d(x,r)‍φ(y)dy=m(φ,Ba,d(x,r))≤(r+∥x∥ar)d1va(r+∥x∥a)d∫Ba,d(0,r+∥x∥a)‍φ(y)dy=(r+∥x∥ar)dm(φ,Ba,d(0,r+∥x∥a)).
It remains to let r→∞ to prove (3.37).Step 2 (proving (2.20)).
Ifφ∈CL(Rn), then φ(·+Xβ)∈CL(Rn). By (2.16), we need to look at
(3.39)(M2,dTφ(·+Xβ))(t(2))=limr→∞1vard∫B2,d(0,r)‍φ(P1(a)t(1)+P2(a)Λ2-1(a)t(2)+Xβ)dt(1).
Using I=P(a)P′(a)=P1(a)P1′(a)+Q, we see that
(3.40)P1(a)t(1)+Xβ=P1(a)t(1)+P1(a)P1′(a)Xβ+QXβ=P1(a)(t(1)+P1′(a)Xβ)+QXβ.
Therefore, replacing t(1)+P1′(a)Xβ=s(1), we can continue (3.39) as
(3.41)(M2,dTφ(·+Xβ))(t(2))=limr→∞1vard∫B2,d(P1′(a)Xβ,r)‍φ(P1(a)s(1)+P2(a)Λ2-1(a)t(2)+QXβ)ds(1).
Using the definition of T1 and (3.37), we see that this is the same as (M2,dT1φ)(t(2)). Hence, the existence of (M2,dTφ(·+Xβ))(t(2)) is equivalent to that of (M2,dT1φ)(t(2)). Application of Theorem 2.6 proves (2.20).


### 3.2.2. Part (b)

Step 1 (the generalized mean vanishes onp-integrable functions).
Ifφ∈Lp(Rd) with some p∈[1,∞), then by Hölder’s inequality
(3.42)|m(φ,Ba,d(0,r))|≤1vard∫Ba,d(0,r)‍|φ(t)|dt≤1vard(mesBa,d(0,r))1-1/p(∫Ba,d(0,r)‍|φ(t)|pdt)1/p=(vard)-1/p∥φ∥Lp⟶0,r⟶∞.Step 2 (limJρ(φ) vanishes on functions with compact support).
By Condition4,
(3.43)|∫Rn‍φ(t)fρ(t)dt|≤∫Rn‍φ~(t)f~ρ(t)dt,
where φ~(t)=|φ(t)| and f~ρ is obtained from (1.9) by replacing g(x) with g~(x)=p(∥x∥2). The function φ~ satisfies the Lipschitz condition, |φ~(x)-φ~(y)|=||φ(x)|-|φ(y)||≤c1∥x-y∥2, and g~ satisfies Condition 2. Part (a) of this theorem allows us to use (2.20) with φ~ in place of φ and g~ instead of g whenever (M2,dT1φ~)(t(2)) exists for almost all t(2)∈Rn-d.
LetCL0(Rn) denote the set of functions φ∈CL(Rn) with compact support suppφ. Suppose φ∈CL0(Rn). By Step 1, (M2,dT1φ~)(t(2))≡0. Equations (2.20) and (3.43) imply that
(3.44)limρ→a∫Rn‍φ(t)fρ(t)dt=0,φ∈CL0(Rn).
Suppose (1.11) is true in the weak sense or, more generally, the family {fρ:0≤ρ<a} converges in distribution, as ρ→a, to a probabilistic measure μ supported on some set S⊂Rn. Then by (2.7) ∫Rn‍φ(t)fρ(t)dt→∫S‍φdμ,φ∈CL(Rn). This equation and (3.44) imply ∫S‍φdμ=0,φ∈CL0(Rn). Taking here φ∈CL0(Rn) such that φ≥0 and φ=1 on B2,n(0,r) we have ∫S∩B2,n(0,r)‍dμ=0 for any r>0. Thus, μ(S)=0 which is impossible.


## 3.2.1. Part (a)

Step 1 (Ma,dφ does not depend on the center of balls).
Let us make definition (2.4) more precise: (Ma,dφ)(x)=limr→∞m(φ,Ba,d(x,r)),  x∈Rd. Here, we prove that
(3.37)(Ma,dφ)(x)=(Ma,dφ)(0)foranyx∈Rd.
For r≥∥x∥a, one has implications ∥y∥a≤r-∥x∥a⇒∥y-x∥a≤r⇒∥y∥a≤∥y-x∥a+∥x∥a≤r+∥x∥a. Geometrically, they mean the inclusion Ba,d(0,r-∥x∥a)⊂Ba,d(x,r)⊂Ba,d(0,r+∥x∥a). This inclusion implies that
(3.38)(r-∥x∥ar)dm(φ,Ba,d(0,r-∥x∥a))=(r-∥x∥ar)d1va(r-∥x∥a)d∫Ba,d(0,r-∥x∥a)‍φ(y)dy≤1vard∫Ba,d(x,r)‍φ(y)dy=m(φ,Ba,d(x,r))≤(r+∥x∥ar)d1va(r+∥x∥a)d∫Ba,d(0,r+∥x∥a)‍φ(y)dy=(r+∥x∥ar)dm(φ,Ba,d(0,r+∥x∥a)).
It remains to let r→∞ to prove (3.37).Step 2 (proving (2.20)).
Ifφ∈CL(Rn), then φ(·+Xβ)∈CL(Rn). By (2.16), we need to look at
(3.39)(M2,dTφ(·+Xβ))(t(2))=limr→∞1vard∫B2,d(0,r)‍φ(P1(a)t(1)+P2(a)Λ2-1(a)t(2)+Xβ)dt(1).
Using I=P(a)P′(a)=P1(a)P1′(a)+Q, we see that
(3.40)P1(a)t(1)+Xβ=P1(a)t(1)+P1(a)P1′(a)Xβ+QXβ=P1(a)(t(1)+P1′(a)Xβ)+QXβ.
Therefore, replacing t(1)+P1′(a)Xβ=s(1), we can continue (3.39) as
(3.41)(M2,dTφ(·+Xβ))(t(2))=limr→∞1vard∫B2,d(P1′(a)Xβ,r)‍φ(P1(a)s(1)+P2(a)Λ2-1(a)t(2)+QXβ)ds(1).
Using the definition of T1 and (3.37), we see that this is the same as (M2,dT1φ)(t(2)). Hence, the existence of (M2,dTφ(·+Xβ))(t(2)) is equivalent to that of (M2,dT1φ)(t(2)). Application of Theorem 2.6 proves (2.20).


## 3.2.2. Part (b)

Step 1 (the generalized mean vanishes onp-integrable functions).
Ifφ∈Lp(Rd) with some p∈[1,∞), then by Hölder’s inequality
(3.42)|m(φ,Ba,d(0,r))|≤1vard∫Ba,d(0,r)‍|φ(t)|dt≤1vard(mesBa,d(0,r))1-1/p(∫Ba,d(0,r)‍|φ(t)|pdt)1/p=(vard)-1/p∥φ∥Lp⟶0,r⟶∞.Step 2 (limJρ(φ) vanishes on functions with compact support).
By Condition4,
(3.43)|∫Rn‍φ(t)fρ(t)dt|≤∫Rn‍φ~(t)f~ρ(t)dt,
where φ~(t)=|φ(t)| and f~ρ is obtained from (1.9) by replacing g(x) with g~(x)=p(∥x∥2). The function φ~ satisfies the Lipschitz condition, |φ~(x)-φ~(y)|=||φ(x)|-|φ(y)||≤c1∥x-y∥2, and g~ satisfies Condition 2. Part (a) of this theorem allows us to use (2.20) with φ~ in place of φ and g~ instead of g whenever (M2,dT1φ~)(t(2)) exists for almost all t(2)∈Rn-d.
LetCL0(Rn) denote the set of functions φ∈CL(Rn) with compact support suppφ. Suppose φ∈CL0(Rn). By Step 1, (M2,dT1φ~)(t(2))≡0. Equations (2.20) and (3.43) imply that
(3.44)limρ→a∫Rn‍φ(t)fρ(t)dt=0,φ∈CL0(Rn).
Suppose (1.11) is true in the weak sense or, more generally, the family {fρ:0≤ρ<a} converges in distribution, as ρ→a, to a probabilistic measure μ supported on some set S⊂Rn. Then by (2.7) ∫Rn‍φ(t)fρ(t)dt→∫S‍φdμ,φ∈CL(Rn). This equation and (3.44) imply ∫S‍φdμ=0,φ∈CL0(Rn). Taking here φ∈CL0(Rn) such that φ≥0 and φ=1 on B2,n(0,r) we have ∫S∩B2,n(0,r)‍dμ=0 for any r>0. Thus, μ(S)=0 which is impossible.


## 3.3. Proof of Theorem2.9

The proof of Theorem2.9 will be preceded by a series of lemmas. The main difficulty consists in estimating integrals of the marginal density g-1 over cylindrical sets of an infinite Lebesgue measure.


### 3.3.1. Preliminary Results

Lemma 3.1.
For the rejection regionΦ defined in (1.14), representation (2.21) holds.Proof.
SinceLX,  MX are complementary orthoprojectors, the images ImMX and ImLX=ImX are orthogonal to one another and Rn=ImMX⊕ImX. Representing u∈Φ as u=LXu+MXu and using MX′=MX,  MX2=MX we get
(3.45)u′MXWMXuu′MXu=(MXu)′WMXu(MXu)′MXu=v′Wvv′v,
where we have denoted v=MXu. Hence, only the component v∈MXΦ determines whether u belongs to Φ or not, while the component LXu∈ImX is arbitrary. This proves the equation Φ=MXΦ+ImX. Finally, the equation MXΦ=ΓS follows by homogeneity from (3.45).Now we review a few facts about the Moore-Penrose inverse in a form adapted to our needs. LetA be a matrix of size m×n, not necessarily square, and denote its rank r=rankA. Its Moore-Penrose inverse A+ is of size n×m.Lemma 3.2 (see, [13, pages 24-25]).
A
+ is unique and has the following properties.(a)
AA+A=A (A+ is a generalized inverse of A).(b)
A+AA+=A+ (A is a generalized inverse of A+).(c)
The matrixH=A+A is an orthoprojector, H2=H, H′=H, and rankH=rankA+=r.(d)
For a giveny∈ImA, a general solution of the equation Ax=y is given by
(3.46)x=A+y+(I-H)z,where z∈Rn is arbitrary. (Rao writes (H-I)z instead of (I-H)z, but our form is more convenient).Regarding (3.46), we can remark the following.(i)
The two terms on the right are orthogonal because by (b) and (c)(3.47)(A+y,(I-H)z)=(HA+y,(I-H)z)=(A+y,(H-H2)z)=0.
Here and in the sequel, (·,·) stands for the scalar product.(ii)
Instead of writing the second term in the form(I-H)z,  z∈Rn, we can equivalently write it as v∈Im(I-H).(iii)
A+y=0 implies that y=0. This is because if A+y=0 and y=Ax, then by (a) y=Ax=AA+(Ax)=A(A+y)=0.Recalling that a general solution of the equationAx=y is, by definition, the preimage A-1y, we state our conclusion in a lemma.Lemma 3.3.
LetY⊆ImA. The preimage A-1Y is a cylinder A+Y+Im(I-H) with the base A+Y⊆ImH and element Im(I-H). Further, A+y=0 for y∈ImA implies that y=0, that is, A+ on ImH is one-to-one.Lemma 3.4.
Supposef1∈ImX. Fix a set Y⊆ImMX and a vector τ∈ImMX. Denote A=MXP2(a)Λ2-1(a) and let A+ and H be the accompanying matrices from Lemma 3.2. Then, the set T(2) defined by T(2)={t(2)∈Rn-1:At(2)+τ∈Y} is a cylinder:
(3.48)T(2)=A+(Y-τ)+Im(I-H).
Supposef1∉ImX and denote p=MXf1/∥MXf1∥2. Then MX-pp′ is an orthoprojector with the image
(3.49)Im(MX-pp′)={x∈ImMX:x⊥p}.
In part (a) replace Y,τ,A by Y⊆Im(MX-pp′), τ∈Im(MX-pp′), and A=(MX-pp′)P2(a)Λ2-1(a), respectively. Then the set T(2) has representation (3.48).Proof.
Let us show thatf1∈ImX implies that
(3.50)ImA=ImMX.
The inclusion ImA⊂ImMX is obvious. Conversely, let x∈ImMX. Then, x=MXx. Since f1,…,fn is an orthonormal basis, we have x=∑i=1n‍(x,fi)fi. Denoting t(2)=((x,f2),…,(x,fn))′, it follows that
(3.51)x=MXx=∑i=1n‍(x,fi)MXfi=∑i=2n‍(x,fi)MXfi=(MXf2,…,MXfn)t(2)=MXP2(a)t(2)=AΛ2(a)t(2)∈ImA.
We have proved (3.50) which implies Y-τ⊂ImA. Application of Lemma 3.3 proves (3.48).
It is easy to check thatMX-pp′ is an orthoprojector and its image satisfies (3.49). From (3.49), we conclude that MX and MX-pp′ commute and
(3.52)(MX-pp′)MX=MX-pp′,
see [14, Chapter VI, Section 3, Lemma 2(a)]. Applying MX-pp′ to both sides of (3.50) and using the last equation we get
(3.53)Im[(MX-pp′)P2(a)Λ2-1(a)]=Im[(MX-pp′)MXP2(a)Λ2-1(a)]=(MX-pp′)[ImMXP2(a)Λ2-1(a)]=(MX-pp′)ImMX=Im(MX-pp′).
We have proved an analog of (3.50) for the matrix under consideration:
(3.54)ImA=Im(MX-pp′).
As previously mentioned, application of Lemma 3.3 proves (3.48).Lemma 3.5.
Letf1,τ,A,A+ and H be the same as in Lemma 3.4(a).(a)
SupposeY⊆ImMX is of the form Y=ΓΔ where Δ⊆{s∈ImMX:∥s∥2=1}. If (2.23) holds, then for the set T(2) from (3.48) one has
(3.55)α(Y,τ)≡∫T(2)‍g-1(t(2))dt(2)≤c(∥τ∥2)mes(Δ).(b)
IfY⊆{s∈ImMX:∥s∥2≤1}, then α(Y,τ)→0,as mes(Y)→0, uniformly in τ.Proof.
Part (a)
Step  1 (proving an auxiliary bound). Denote
(3.56)β(Y,τ)=∫A+(Y-τ)‍g-k(∥u∥2)du.
We intend to prove that
(3.57)β(ΓΔ,τ)≤c(∥τ∥2)mes(Δ).
By Lemma 3.2, (3.50) implies that
(3.58)r≡rankA=n-k.
By Lemma 3.3, A+ is a one-to-one mapping from ImA to ImA+, both of which are of dimension r. We use this fact twice.
Firstly, selecting two bases, one inImA and another in ImA+, we can identify A+ with its matrix representation in these bases. This matrix is square of order r and rank r, so the Jacobian c1 of the transformation u=A+v,  v∈ΓΔ-τ, is positive. Therefore,
(3.59)β(ΓΔ,τ)=c1∫ΓΔ-τ‍g-k(∥A+v∥2)dv=c1∫ΓΔ‍g-k(∥A+(w-τ)∥2)dw.
Secondly,∥A+v∥2=0 implies that A+v=0 which for v∈ImA implies that v=0. Thus, ∥A+v∥2 is a norm on ImA and by (3.14),
(3.60)∥A+(w-τ)∥2≥c2∥w-τ∥2≥max{c2(∥w∥2-∥τ∥2),0},w∈ΓΔ.
Asg- is assumed nonincreasing, g-k is also nonincreasing, and (3.59) leads to
(3.61)β(ΓΔ,τ)≤c1∫ΓΔ‍g-k(max{c2(∥w∥2-∥τ∥2),0})dw.
We bound this using polar coordinates:
(3.62)β(ΓΔ,τ)≤c1∫Δ‍[∫∥τ∥2∞‍g-k(c2(ρ-∥τ∥2))ρr-1dρ]dσ=c3mes(Δ).
Here, the quantity
(3.63)∫∥τ∥2∞‍g-k(c2(ρ-∥τ∥2))ρr-1dρ=1c2∫0∞‍g-k(ρ)(ρc2+∥τ∥2)r-1dρ
is finite because by condition (2.23), g-k is integrable in the neighborhood of zero and (ρ/c2+∥τ∥2)r-1 is of order ρr-1 at infinity.
Step  2 (proving the main bound). The matrix A from Lemma 3.4(a) is of size n×(n-1) and A+ is of size (n-1)×n. Equation (3.50) and Rn-1=ImH⊕Im(I-H) imply that
(3.64)rank(I-H)=n-1-r=k-1.
Equation (3.48) written point-wise is t(2)=u+v,  t(2)∈T(2),  u∈A+(ΓΔ-τ),  v∈Im(I-H). Since u,v here are orthogonal, we have
(3.65)g-1(t(2))=∫R‍p-((t12+∥u∥22+∥v∥22)1/2)dt1≡h(∥u∥2,∥v∥2),
where the right side defines the function h.
The transition fromt(2) to the coordinates (u,v) is described by an orthogonal matrix. This can be shown as follows. Take in ImH, an orthonormal basis and identify u with the set of its coordinates in that basis. Do the same with Im(I-H) and v. Join the two bases. The equation t(2)=u+v induces a transformation between the original coordinates of t(2) and the ones in the joint basis. Since ImH⊥Im(I-H), the Jacobian of the resulting transformation is unity. Hence, by (3.48), (3.64), (3.65), and (3.57),
(3.66)α(Y,τ)=∫A+(ΓΔ-τ)+Im(I-H)‍g-1(u+v)dudv=∫A+(ΓΔ-τ)‍[∫Rk-1‍h(∥u∥2,∥v∥2)dv]du=∫A+(ΓΔ-τ)‍g-k(∥u∥2)du=β(ΓΔ,τ)≤c(∥τ∥2)mes(Δ).
Part (b). In the bound (see (3.59) and (3.60)),
(3.67)β(Y,τ)=c1∫Y-τ‍g-k(∥A+v∥2)dv≤c1∫Y-τ‍g-k(c2∥v∥2)dv,
we have mes(Y-τ)=mes(Y)→0, so β(Y,τ)→0 uniformly in τ. As in (3.66), it follows that(3.68)α(Y,τ)=∫A+(Y-τ)+Im(I-H)‍g-1(u+v)dudv=β(Y,τ)⟶0.Lemma 3.6.
Letf1,p,τ,A,A+, and H be the same as in Lemma 3.4(b).(a)
SupposeY=ΓΔ where Δ⊆{s∈Im(MX-pp′):∥s∥2=1}. If (2.25) holds, then (3.55) is true with T(2) from Lemma 3.4(b).(b)
IfY⊆{s∈Im(MX-pp′):∥s∥2≤1}, then α(Y,τ)→0, as mes(Y)→0, uniformly in τ.Proof.
The proof essentially follows that of Lemma3.5(a). From (3.49), and (3.54) we see that instead of (3.58) we have r=n-k-1. Instead of (3.64) we have rank(I-H)=k. This leads to the definition of the marginal density used in (2.25). With these changes, (3.62), and (3.66) are true and prove part (a).
With the same changes the proof of Lemma3.5(b) goes through.In the course of the proof, we approximateχΦ by functions φε±∈CL(Rn) from below and above. Then, we show that the rate of approximation improves as ε→0 and ρ→a.Step 1 (constructing the approximating functions).
Denoteρ(x,A)=infy∈A∥x-y∥2 the distance from a point x to a set A. For a given ε>0, set Ψε+={x∈Rn:ρ(x,Φ)≤ε},  Ψε-={x∈Φ:ρ(x,bd(Φ))>ε}.Obviously,Ψε-⊂Φ⊂Ψε+. As in [10, page 12 proof of (ii)⇒(iii)], we can construct functions φε± that are uniformly continuous on Rn and satisfy
(3.69)φε+=1onΦ,φε+=0outsideΨε+,0≤φε+≤1onRn,φε-=1onΨε-,φε-=0outsideΦ,0≤φε-≤1onRn.Step 2 (reducing the convergence statement to a special case).
Denote(3.70)I(φ)=∫Rn-1(MT1φ)(t(2))g-1(t(2))dt(2).
Here, we show that
(3.71)limε→0I(φε±)=I(χΦ)
implies (2.24). From the inequality φε-≤χΦ≤φε+ we have
(3.72)Jρ(φε-)≤Jρ(χΦ)≤Jρ(φε+).
By Theorem 2.7,
(3.73)limρ→aJρ(φε±)=I(φε±),foranyε>0.
By (3.71), for any δ>0, there exists ε(δ)>0 such that
(3.74)I(φε+)-δ≤I(χΦ)≤I(φε-)+δ,0<ε≤ε(δ).
By (3.73), for the same δ, there is ρ(δ)∈(0,a) such that
(3.75)|Jρ(φε±)-I(φε±)|≤δ,ρ(δ)≤ρ<a.
Equations (3.72), (3.74), and (3.75) imply that
(3.76)I(χΦ)-2δ≤I(φε-)-δ≤Jρ(φε-)≤Jρ(χΦ)≤Jρ(φε+)≤I(φε+)+δ≤I(χΦ)+2δ,ρ(δ)≤ρ<a.
Since δ>0 is arbitrary, this proves (2.24).Step 3 (further reduction to an indicator function).
Let us callB(ε)=Ψε+∖Ψε- a band. Here, we show that (3.71) will follow if we prove that
(3.77)limε→0I(χB(ε))=0.
Note the identity
(3.78)I(χB(ε))=I(χΨε+)-I(χΨε-)=[I(χΨε+)-I(χΦ)]+[I(χΦ)-I(χΨε-)].
Since each of the square brackets on the right is nonnegative, (3.77) gives limε→0I(χΨε±)=I(χΦ). Now, it suffices to take into account the inequality I(χΨε-)≤I(φε-)≤I(χΦ)≤I(φε+)≤I(χΨε+) to see that (3.71) follows.Step 4 (geometric description of the task).
Denotel(s,i)={st+i:t∈R} a straight line with a slopes∈Rn and intercepti∈Rn and put
(3.79)bε(t(2))=(MT1χB(ε))(t(2))=limr→∞12r∫-rr‍χB(ε)(f1t1+z(t(2)))dt1.
This expression shows that we need to study the set {t1∈R:f1t1+z(t(2))∈B(ε)} of points common to the straight line l(f1,z(t(2))) and the band B(ε). The cylindrical nature of the sets considered implies that the band admits an equivalent definition as a cylinder with the base
(3.80)MXB(ε)={x∈ImMX:ρ(x,bd(MXΦ))≤ε}
and element ImX. From this definition, we see that the condition f1t1+z(t(2))∈B(ε) is equivalent to
(3.81)(MXf1)t1+MXz(t(2))∈MXB(ε).
We have to sort out points t(2)∈Rn-1 into (a) those for which the set of points t1 satisfying (3.81) is so small that (3.79) is zero and (b) that set is large enough for (3.79) to be positive.Step 5 (casef1∈ImX).
In this case,MXf1=0, and (3.81) reduces to
(3.82)MXP2(a)Λ2-1(a)t(2)+MXQXβ∈MXB(ε).
This condition does not contain t1. Therefore, if it is not satisfied, the integrand in (3.79) is identically zero and bε(t(2))=0 for such t(2). If it is satisfied, the integrand in (3.79) is identically 1 and bε(t(2))=1 for such t(2). Hence,
(3.83)I(χB(ε))=∫(3.82)istrue‍g-1(t(2))dt(2).
The set of t(2) satisfying (3.82) has an infinite Lebesgue measure which precludes us from application of the continuity property of the Lebesgue measure (for an integrable h,∫A‍h(t)dt→0 as mes(A)→0).
As one can see from Figure1, MXB(ε) is not a cone-like set. We divide it into two parts: αεin={s∈MXB(ε):∥s∥2≤1} includes the points inside the unit ball and on the unit sphere and αεout={s∈MXB(ε):∥s∥2>1} includes the points of MXB(ε) outside the unit ball. Put Δε={s∈ImMX:∥s∥2=1,ρ(s,bd(MXΦ))≤ε} and let us show that
(3.84)αεout⊂ΓΔε.
Suppose x∈αεout. Then, x∈ImMX,∥x∥2>1 and by (3.80), there exists a sequence {y(k)}⊂bd(MXΦ) such that limsup∥x-y(k)∥2≤ε. The sequence {y(k)} is bounded and therefore precompact, while bd(MXΦ)=MX(cl(Φ)∖Φ) is closed by construction. Hence, we can assume that y(k)→y∈bd(MXΦ) (passing to a subsequence, if necessary). Then, ∥x-y∥2≤ε. This implies  ∥x/∥x∥2-y/∥x∥2∥2≤ε/∥x∥2<ε. Let s=x/∥x∥2. Since bd(MXΦ) is cone-like, it follows that y/∥x∥2∈bd(MXΦ) and ρ(s,bd(MXΦ))<ε. This proves that s∈Δε,  x=∥x∥2s∈ΓΔε and (3.84) is true.
From (3.84), we conclude that
(3.85)MXB(ε)⊂αεin∪ΓΔε.
This means that the set of solutions of (3.82) is contained in the union of two sets: T1(2)={t(2)∈Rn-1:At(2)+τ∈ΓΔε} and T2(2)={t(2)∈Rn-1:At(2)+τ∈αεin} where A=MXP2(a)Λ2-1(a) and τ=MXQXβ.
By Lemma3.4(a), T1(2)=A+(ΓΔε-τ)+Im(I-H). By Lemma 3.5(a)
(3.86)∫T1(2)‍g-1(t(2))dt(2)≤c(∥τ∥2)mes(Δε)⟶0,ε→0.
By Lemma 3.4(a), T2(2)=A+(αεin-τ)+Im(I-H). By Lemma 3.5(b)
(3.87)∫T2(2)‍g-1(t(2))dt(2)⟶0asmes(αεin)⟶0.
Equations (3.86) and (3.87) prove that I(χB(ε))→0,  ε→0.Figure 1
M
X
Φ consists of two cones bounded by solid lines. MXB(ε) is bounded by dashed lines.Step 6 (casef1∉ImX).
Equation (2.5) implies that if the straight line l(MXf1,MXz(t(2))) has a bounded intersection with the band MXB(ε), then (MT1χB(ε))(t(2))=0. Therefore, we are interested in the set of slopes:
(3.88)Sl={MXf1:l(MXf1,MXz(t(2)))∩MXB(ε)isunbounded}.
Let us prove that
(3.89)MXf1∈Sl⇒MXf1/∥MXf1∥2∈bd(S).
Suppose MXf1∈Sl. Then, there exists a sequence of points p(k) which lie on the straight line, p(k)=(MXf1)t1(k)+MXz(t(2)), where |t1(k)|→∞, and which are 2ε-close to bd(MXΦ): there exists a sequence y(k)∈bd(MXΦ) such that
(3.90)∥p(k)-y(k)∥2≤2ε,∥y(k)∥2⟶∞.
Equation (2.21) implies that MXΦ=ΓS and bd(MXΦ)=Γbd(S)∪{0}. Therefore s(k)≡y(k)/∥y(k)∥2∈bd(S). Because bd(S) is compact, we can assume that s(k)→s∈bd(S). From (3.90), we have ∥(MXf1)t1(k)+MXz(t(2))-∥y(k)∥2s(k)∥2≤2ε or, dividing both sides by ∥y(k)∥2, ∥(MXf1)t1(k)/∥y(k)∥2-s(k)∥2→0. This shows that the sequence {t1(k)/∥y(k)∥2} is bounded and can be assumed convergent to a number δ. Letting k→∞, we get (MXf1)δ=s∈bd(S), and (3.89) is true.
FixMXf1∈Sl and denote p=MXf1/∥MXf1∥2∈bd(S)⊂ImMX. The matrix pp′ projects Im MX onto the straight line l(MXf1,0). The projector MX-pp′ projects Im  MX onto Im(MX-pp′). Condition (3.81) implies two equations in projections:
(3.91)pp′[(MXf1)t1+MXz(t(2))]∈pp′MXB(ε),(MX-pp′)[(MXf1)t1+MXz(t(2))]∈(MX-pp′)MXB(ε).
The first one can be safely omitted because the shift pp′MXz(t(2)) along a line does not change the property of the line to have an unbounded intersection with MXB(ε). Denoting A=(MX-pp′)P2(a)Λ2-1(a),  τ=(MX-pp′)QXβ, and using (3.52), we write the second equation as
(3.92)At(2)+τ∈(MX-pp′)B(ε).
The set of points t(2) such that (3.81) holds for an unbounded set of values t1 is clearly contained in the set T(2) of points t(2) that satisfy (3.92). For t(2)∈T(2) we use an obvious bound bε(t(2))≤1 (see (3.79)), and, therefore, I(χB(ε))≤∫(3.92)istrue‍g-1(t(2))dt(2). The rest of the proof closely follows that for the case f1∈ImX. An inclusion of type (3.85) holds with MX replaced by MX-pp′ in all definitions. The reference to Lemma 3.4(a) is replaced by Lemma 3.4(b). Lemma 3.6 is used instead of Lemma 3.5. The conclusion is that I(χB(ε))→0.We shall use several times the fact that for the densityg-1 in representation (2.24)will, the conditions ∫A‍g-1(t(2))dt(2)>0 and mes(A)>0 are equivalent. From (2.24), we see that the task is to calculate the function (3.79) with Φ in place of B(ε). Because of (2.21), the condition that f1t1+z(t(2))∈Φ is equivalent to the condition in projections:
(3.93)(MXf1)t1+MXz(t(2))∈ΓS.The magnitude ofβ(Φ) depends on the set
(3.94)A(t(2))={t1∈R:f1t1+z(t(2))∈Φ}.The properties ofA(t(2)) depend on those of mes{t(2)∈Rn-1:MXz(t(2))∈G} for certain sets G⊆ImMX, as we will see.Lemma 3.7.
SupposeImQ⊆ImX (Q is the projector from Theorem 2.7). Then, MXz(t(2))=0 for all t(2)∈Rn-1.
LetImQ⊈ImX. Then, mes{t(2):MXz(t(2))=0}=0.Proof.
The assumptionImQ⊆ImX implies that MXfi=0 for i=2,…,n, which can be written as MXP2(a)=0. Hence, MXz(t(2))=MXP2(a)[Λ2-1(a)t(2)+P2′(a)Xβ]=0 for any t(2)∈Rn-1.
IfImQ⊈ImX, then fi∉ImX and MXfi≠0 for at least one i. The matrix MXP2(a) is of size n×(n-1), and of rank at least 1. From the equation dimN(MXP2(a))+rank(MXP2(a))=n-1 we see that the null space N(MXP2(a)) is of dimension at most n-2. The set of t(2) satisfying MXz(t(2))=MXP2(a)Λ2-1(a)t(2)+MXQXβ=0 is a partial solution t0(2) plus an arbitrary element from the null space N(MXP2(a)). Thus the set of all solutions is a hyperplane in Rn-1 of dimension at most n-2 and has (n-1)-dimensional Lebesgue measure zero.Lemma 3.8.
Suppose we are given a setG⊆Rn-1 and for almost each t(2)∈G, the set A(t(2))⊆R is defined by (3.94).(a)
IfA(t(2)) is bounded for almost all t(2)∈G and mes(G)>0, then β(Φ)<1. In particular, if G=Rn-1, then β(Φ)=0.(b)
If(3.95)R∖A(t(2))isboundedforalmostallt(2)∈G
and mes(G)>0, then β(Φ)>0. In particular, if G=Rn-1, then β(Φ)=1.(c)
Suppose thatG=Rn-1 and for almost all t(2)∈G,
(3.96)thesetA(t(2))isahalf-axis
(i.e., either A(t(2))=(a(t(2)),∞) or A(t(2))=(-∞,a(t(2))) with some a(t(2))∈R). Then, β(Φ)=1/2.Proof.
IfA(t(2)) is bounded, then χΦ(f1t1+z(t(2)))=0 outside a bounded set, and by (2.5),
(3.97)MχA(t(2))=(MT1χΦ)(t(2))=0
for almost all t(2)∈G. Since 0≤MχA(t(2))≤1 outside G, we have
(3.98)β(Φ)≤∫Rn-1∖G‍g-1(t(2))dt(2)<1.
Obviously, the integral is null if G=Rn-1.
By (3.97), condition (3.95) implies that
(3.99)MχA(t(2))=M(1-χR∖A(t(2)))=1a.e.inG.
Since mes(G)>0, we have
(3.100)β(Φ)≥∫G‍g-1(t(2))dt(2)>0.
In case G=Rn-1, the last integral is 1.
Consider, for example, the case of the right half-axis:(3.101)MχA(t(2))=limr→∞12r(∫a(t(2))0‍dt+∫0r‍dt)=12.
Hence, β(Φ)=(1/2)∫Rn-1‍g-1(t(2))dt(2)=1/2.Case 1.
Letf1∈Φ. Then, f1∉ImX and MXf1≠0. Define p=MXf1/∥MXf1∥2. Obviously, MXf1∈ΓS and p∈S. In the equation
(3.102)(MXf1)t1+MXz(t(2))=∥MXf1∥2t1[p+MXz(t(2))∥MXf1∥2t1],
for all large |t1|, the term in the square brackets is close to p and, because MXΦ is open in ImMX, this term belongs to MXΦ. Since MXΦ is cone-like, the left side of (3.102) belongs to MXΦ for all large |t1|. Hence, the set A(t(2)) contains the neighborhood of infinity (more precisely, all t1 with |t1|>N where N depends on t(2)). By Lemma 3.8(b), where G=Rn-1, then β(Φ)=1.Case 2.
Supposef1∈bd(Φ).
Subcase2.1. Let f1∈ImX or, equivalently, MXf1=0. Condition (3.93) reduces to
(3.103)MXz(t(2))=MXP2(a)Λ2-1(a)t(2)+MXQXβ∈ΓS.
By Lemma 3.4(a), the set of solutions of (3.103) is T(2)=A+(ΓS-τ)+Im(I-H) where A+ is the Moore-Penrose inverse of A=MXP2(a)Λ2-1(a),τ=MXQXβ, and H is the projector from Lemma 3.2. Using Lemma 3.8(b), (3.66), and (3.59) we get
(3.104)β(Φ)=∫T(2)‍g-1(t(2))dt(2)=∫A+(ΓS-τ)‍g-k(∥u∥2)du.
The density g-k(∥u∥2) is spherically symmetric. It is positive everywhere because
(3.105)g-k(t)=∫Rk‍p-((t2+∥v∥22)1/2)dv=0
would mean p-((t2+t1)1/2)=0 for any t1≥0, that is, p-(t1)=0 for any t1≥t, which is impossible by assumption. A+ is one-to-one on ImA=ImMX, so A+(ΓS-τ) is a nonempty proper subset of ImA+. Therefore, 0<β(Φ)<1.
Subcase 2.2. Suppose f1∉ImX. As in Case  1, p is defined. Let us show that it satisfies
(3.106)p′Wp=c.
Since f1 is a limit point of Φ, there is a sequence xn∈Φ such that xn→f1. Hence, MXxn→MXf1 where ∥MXf1∥2≠0 because f1∉ImX. From (2.21), it follows that MXxn∈ΓS,zn≡MXxn/∥MXxn∥2∈S and zn′Wzn>c. Letting n→∞ and using zn→p, we get p′Wp≥c. Since f1∉Φ, we have MXf1∉MXΦ and p∉S. Thus, p′Wp>c is impossible and (3.106) is true.
In the three examples in the following we assume thatn>k+1. Then, ImQ⊈ImX, and by Lemma 3.7(b),
(3.107)MXz(t(2))≠0,foralmostallt(2)∈Rn-1.Sub-sub-case (a) (Example ofβ(Φ)=1). Let W be symmetric. Assumptions (1.13) imply that var(u)=σε2(I-ρW)-2 whenever det(I-ρW)≠0, so Σ-1(ρ)=(I-ρW)2. If μi denote eigenvalues of W, then the eigenvalues of I-ρW are νi(ρ)=1-ρμi. Let the constant c in the definition of the critical region Φ be positive. Put a=1/c and with some integer i0 and real numbers μ-,  μ+ such that 1<i0<n,  μ-<1/a<μ+ define μi by
(3.108)μ1=⋯=μi0-1=μ-<μi0=1a<μi0+1=⋯=μn=μ+.
Then, for 0<ρ≤a,
(3.109)ν1(ρ)=⋯=νi0-1(ρ)=1-ρμ->νi0(ρ)=1-ρa>νi0+1(ρ)=…=νn(ρ)=1-ρμ+,
and Σ-1/2(a) has eigenvalues
(3.110)ν1(a)=⋯=νi0-1(a)=1-aμ->νi0(a)=0<|νi0+1(a)|=⋯=|νn(a)|=|1-aμ+|.
Hence, in the notation of Condition 3,  λ1(ρ)=νi0(ρ) and λ2(ρ),…,λn(ρ) are obtained from |νi(ρ)|,  i≠i0, by reordering. For example, if 1-aμ-<|1-aμ+|, then λi(ρ)=νi-1(ρ),i=2,…,i0;  λi(ρ)=|νi(ρ)|,  i=i0+1,…,n. Assuming this ordering, denote the corresponding eigenvectors of Σ-1/2(a) by f1,f2-,…,fi0-,fi0+1+,…,fn+ where the minuses and pluses remind us that the corresponding eigenvalues depend on μ- and μ+, respectively.
Letk=i0-1 and suppose that
(3.111)f2-,…,fi0-spanImX,f1,fi0+1+,…,fn+spanImMX.
Denote
(3.112)x=(MXf1)t1+MXz(t(2))∈ImMX.
Because of (3.111), here MXf1=f1. Besides, since z(t(2)) is orthogonal to f1, the decomposition
(3.113)MXz(t(2))=∑i=i0+1n‍cifi+
does not contain f1. Hence, (3.112) becomes
(3.114)x=t1f1+∑i=i0+1n‍cifi+.
Remembering that the eigenvectors of Σ-1/2(a) are just reordered eigenvectors of W, we get
(3.115)Wx=1at1f1+μ+∑i=i0+1n‍cifi+,x′Wx=1at12+μ+∑i=i0+1n‍ci2=ct12+μ+∑i=i0+1n‍ci2,
for any x of form (3.112). From (3.107) and (3.113) we see that here
(3.116)∑i=i0+1n‍ci2≠0a.e.inRn-1.
Hence, normalizing x,  s=x/∥x∥2, and assuming that μ+>c, we get from (3.115)
(3.117)s′Ws=ct12+μ+∑i=i0+1nci2t12+∑i=i0+1nci2>c,
for all t1∈R and almost all t(2)∈Rn-1. This shows that the set (3.94) coincides with R for almost all t(2)∈Rn-1 and by Lemma 3.8(b) β(Φ)=1.
Sub-sub-case (b)Example ofβ(Φ)=0. We change (3.111) so as to obtain in (3.115) μ- in place of μ+. Let k=n-i0 and suppose that
(3.118)fi0+1+,…,fn+spanImX,f1,f2-,…,fi0-spanImMX.
Instead of (3.113), we get MXz(t(2))=∑i=2i0‍cifi- which leads to an analog of (3.114): x=t1f1+∑i=2i0‍cifi-.  Equation(3.115) is replaced by
(3.119)x′Wx=ct12+μ-∑i=2i0‍ci2,
where ∑i=2i0‍ci2≠0 a.e. in Rn-1. The result is that
(3.120)s′Ws=ct12+μ-∑i=2i0ci2t12+∑i=2i0ci2<c
for all t1∈R and almost all t(2)∈Rn-1. Hence, the set (3.94) is empty for almost all t(2)∈Rn-1 and by Lemma 3.8(a), β(Φ)=0.
Sub-sub-case
(
c
)
Example of β(Φ)=1/2]. MXz(t(2)) runs over Im(MXP2(a)). If k>1, then n-1>n-k, and we can choose the eigenvectors f2,…,fn of Σ-1/2(a) so that ImQ⊃ImMX, and then MXz(t(2)) runs over ImMX=Im(MXP2(a)). Let ΓS be a union of two cones, as in Figure 2. The line l(MXf1,0) lies on the boundary of the cones. In Figure 2, La=l(MXf1,0) and Lb=l(b(t(2)),0). For any nonzero shift MXz(t(2)), half of the shifted line is contained in MXΦ because La and Lb are not parallel. Here, “half” means “either for all large positive t1 or for all large negative t1.” By Lemma 3.8(c), β(Φ)=1/2.Figure 2
The base ofΦ contains the area between two solid lines La and  Lb. The dashed line is l(MXf1,MXz(t(2)))=La+MXz(t(2)).Case 3.
Supposef1∈Φ-. Then, as in part 1,MXf1≠0 and p is defined. It satisfies p′Wp<c. The term in the square brackets in (3.102) for all large |t1| belongs to an open set MXΦ-. Since this set is cone-like, the left side of (3.102) belongs to MXΦ- for all large |t1|. The set A(t(2)) is bounded for all t(2)∈Rn-1, and by Lemma 3.8(a), β(Φ)=0.Step 1 (theε(ρ)-maximizing set is an ellipsoid).
Fory∈Mε(ρ) by orthogonality of P and Condition 7, we have
(3.121)fρ(y)=∏j=1n‍λj(ρ)p(∥Λ(ρ)P′(ρ)(y-Xβ)∥2)≥(1-ε(ρ))∏j=1n‍λj(ρ)m.
Since p-1 is decreasing, this is equivalent to ∥Λ(ρ)P′(ρ)(y-Xβ)∥2≤p-1((1-ε(ρ))m). Denoting h(ρ)=p-1((1-ε(ρ))m),  z=P′(ρ)(y-Xβ), we rewrite the resulting inequality as
(3.122)∑j=1d‍zj2[h(ρ)/λj(ρ)]2+∑j=d+1n‍zj2[h(ρ)/λj(ρ)]2≤1.
In z coordinates, this is an ellipsoid with half-axes h(ρ)/λj(ρ).Step 2 (characterizing the rate of decay ofh(ρ)).
First we determine the rate with whichp-1(x) approaches 0 when x→m. To approximately invert p, in the equation x=p(r), we use representation (2.30): x=m(1-cprαp)(1+o(rαε)). This is equivalent to (x/m)(1+o(rαε))=1-cprαp. Solving this equation for the r involved in the right-hand side produces
(3.123)p-1(x)=r={1cp[1-xm(1+o(rαε))]}1/αp,r→0.
Since ρ→a, we can put r=a-ρ in the last equation to get
(3.124)p-1(x)={1cp[1-xm(1+o((a-ρ)αε))]}1/αp,ρ→a.
To obtainh(ρ), let x=(1-ε(ρ))m. Then, by (2.29),
(3.125)1-xm(1+o((a-ρ)αε))=1-(1-ε(ρ))(1+o((a-ρ)αε))=1-1+o((a-ρ)αε)+ε(ρ)(1+o((a-ρ)αε))=ε(ρ)(1+o(1)).
Combining this with (3.124) and (2.29), we have
(3.126)h(ρ)=p-1(x)={cεcp(a-ρ)αε(1+o(1))}1/αp=(cεcp)1/αp(a-ρ)αε/αp(1+o(1)).Step 3 (behavior of half-axes).
We intend to show that forj=d+1,…,n the half-axes tend to zero, so that in the limit
(3.127)zd+1=⋯=zn=0,
whereas for j=1,…,d, they tend to ∞, and in the limit, z1,…,zd can be any real numbers.
Forj=d+1,…,n from (2.28), we see that the condition h(ρ)/λj(ρ)→0 is equivalent to h(ρ)→0. Because of (2.31), this condition is the same as (1-ε(ρ))m→m which holds by (2.29).
For the remaining half-axes, we get by (2.27), (3.126), and (2.32)
(3.128)h(ρ)λj(ρ)=1cλ(cεcp)1/αp(a-ρ)αε/αp-αλ(1+o(1))⟶∞.
We have proved our statement on convergence of the half-axes. According to (2.26), (3.127) rewrites as (P′(a)(y-Xβ))j=0,  j=d+1,…,n. Since the rows of P′(a) are eigenvectors of A(a), the last equation means that y-Xβ is orthogonal to eigenvectors numbered j=d+1,…,n. Hence, y-Xβ belongs to the null space of A(a), and the set of all such y’s is Xβ+N(A(a)).
In this proof, the setXβ+N(A(a)) is obtained as the limit of the ridge of fρ. In general, this ridge can have any shape and the limit will not be Xβ+N(A(a)).


## 3.3.1. Preliminary Results

Lemma 3.1.
For the rejection regionΦ defined in (1.14), representation (2.21) holds.Proof.
SinceLX,  MX are complementary orthoprojectors, the images ImMX and ImLX=ImX are orthogonal to one another and Rn=ImMX⊕ImX. Representing u∈Φ as u=LXu+MXu and using MX′=MX,  MX2=MX we get
(3.45)u′MXWMXuu′MXu=(MXu)′WMXu(MXu)′MXu=v′Wvv′v,
where we have denoted v=MXu. Hence, only the component v∈MXΦ determines whether u belongs to Φ or not, while the component LXu∈ImX is arbitrary. This proves the equation Φ=MXΦ+ImX. Finally, the equation MXΦ=ΓS follows by homogeneity from (3.45).Now we review a few facts about the Moore-Penrose inverse in a form adapted to our needs. LetA be a matrix of size m×n, not necessarily square, and denote its rank r=rankA. Its Moore-Penrose inverse A+ is of size n×m.Lemma 3.2 (see, [13, pages 24-25]).
A
+ is unique and has the following properties.(a)
AA+A=A (A+ is a generalized inverse of A).(b)
A+AA+=A+ (A is a generalized inverse of A+).(c)
The matrixH=A+A is an orthoprojector, H2=H, H′=H, and rankH=rankA+=r.(d)
For a giveny∈ImA, a general solution of the equation Ax=y is given by
(3.46)x=A+y+(I-H)z,where z∈Rn is arbitrary. (Rao writes (H-I)z instead of (I-H)z, but our form is more convenient).Regarding (3.46), we can remark the following.(i)
The two terms on the right are orthogonal because by (b) and (c)(3.47)(A+y,(I-H)z)=(HA+y,(I-H)z)=(A+y,(H-H2)z)=0.
Here and in the sequel, (·,·) stands for the scalar product.(ii)
Instead of writing the second term in the form(I-H)z,  z∈Rn, we can equivalently write it as v∈Im(I-H).(iii)
A+y=0 implies that y=0. This is because if A+y=0 and y=Ax, then by (a) y=Ax=AA+(Ax)=A(A+y)=0.Recalling that a general solution of the equationAx=y is, by definition, the preimage A-1y, we state our conclusion in a lemma.Lemma 3.3.
LetY⊆ImA. The preimage A-1Y is a cylinder A+Y+Im(I-H) with the base A+Y⊆ImH and element Im(I-H). Further, A+y=0 for y∈ImA implies that y=0, that is, A+ on ImH is one-to-one.Lemma 3.4.
Supposef1∈ImX. Fix a set Y⊆ImMX and a vector τ∈ImMX. Denote A=MXP2(a)Λ2-1(a) and let A+ and H be the accompanying matrices from Lemma 3.2. Then, the set T(2) defined by T(2)={t(2)∈Rn-1:At(2)+τ∈Y} is a cylinder:
(3.48)T(2)=A+(Y-τ)+Im(I-H).
Supposef1∉ImX and denote p=MXf1/∥MXf1∥2. Then MX-pp′ is an orthoprojector with the image
(3.49)Im(MX-pp′)={x∈ImMX:x⊥p}.
In part (a) replace Y,τ,A by Y⊆Im(MX-pp′), τ∈Im(MX-pp′), and A=(MX-pp′)P2(a)Λ2-1(a), respectively. Then the set T(2) has representation (3.48).Proof.
Let us show thatf1∈ImX implies that
(3.50)ImA=ImMX.
The inclusion ImA⊂ImMX is obvious. Conversely, let x∈ImMX. Then, x=MXx. Since f1,…,fn is an orthonormal basis, we have x=∑i=1n‍(x,fi)fi. Denoting t(2)=((x,f2),…,(x,fn))′, it follows that
(3.51)x=MXx=∑i=1n‍(x,fi)MXfi=∑i=2n‍(x,fi)MXfi=(MXf2,…,MXfn)t(2)=MXP2(a)t(2)=AΛ2(a)t(2)∈ImA.
We have proved (3.50) which implies Y-τ⊂ImA. Application of Lemma 3.3 proves (3.48).
It is easy to check thatMX-pp′ is an orthoprojector and its image satisfies (3.49). From (3.49), we conclude that MX and MX-pp′ commute and
(3.52)(MX-pp′)MX=MX-pp′,
see [14, Chapter VI, Section 3, Lemma 2(a)]. Applying MX-pp′ to both sides of (3.50) and using the last equation we get
(3.53)Im[(MX-pp′)P2(a)Λ2-1(a)]=Im[(MX-pp′)MXP2(a)Λ2-1(a)]=(MX-pp′)[ImMXP2(a)Λ2-1(a)]=(MX-pp′)ImMX=Im(MX-pp′).
We have proved an analog of (3.50) for the matrix under consideration:
(3.54)ImA=Im(MX-pp′).
As previously mentioned, application of Lemma 3.3 proves (3.48).Lemma 3.5.
Letf1,τ,A,A+ and H be the same as in Lemma 3.4(a).(a)
SupposeY⊆ImMX is of the form Y=ΓΔ where Δ⊆{s∈ImMX:∥s∥2=1}. If (2.23) holds, then for the set T(2) from (3.48) one has
(3.55)α(Y,τ)≡∫T(2)‍g-1(t(2))dt(2)≤c(∥τ∥2)mes(Δ).(b)
IfY⊆{s∈ImMX:∥s∥2≤1}, then α(Y,τ)→0,as mes(Y)→0, uniformly in τ.Proof.
Part (a)
Step  1 (proving an auxiliary bound). Denote
(3.56)β(Y,τ)=∫A+(Y-τ)‍g-k(∥u∥2)du.
We intend to prove that
(3.57)β(ΓΔ,τ)≤c(∥τ∥2)mes(Δ).
By Lemma 3.2, (3.50) implies that
(3.58)r≡rankA=n-k.
By Lemma 3.3, A+ is a one-to-one mapping from ImA to ImA+, both of which are of dimension r. We use this fact twice.
Firstly, selecting two bases, one inImA and another in ImA+, we can identify A+ with its matrix representation in these bases. This matrix is square of order r and rank r, so the Jacobian c1 of the transformation u=A+v,  v∈ΓΔ-τ, is positive. Therefore,
(3.59)β(ΓΔ,τ)=c1∫ΓΔ-τ‍g-k(∥A+v∥2)dv=c1∫ΓΔ‍g-k(∥A+(w-τ)∥2)dw.
Secondly,∥A+v∥2=0 implies that A+v=0 which for v∈ImA implies that v=0. Thus, ∥A+v∥2 is a norm on ImA and by (3.14),
(3.60)∥A+(w-τ)∥2≥c2∥w-τ∥2≥max{c2(∥w∥2-∥τ∥2),0},w∈ΓΔ.
Asg- is assumed nonincreasing, g-k is also nonincreasing, and (3.59) leads to
(3.61)β(ΓΔ,τ)≤c1∫ΓΔ‍g-k(max{c2(∥w∥2-∥τ∥2),0})dw.
We bound this using polar coordinates:
(3.62)β(ΓΔ,τ)≤c1∫Δ‍[∫∥τ∥2∞‍g-k(c2(ρ-∥τ∥2))ρr-1dρ]dσ=c3mes(Δ).
Here, the quantity
(3.63)∫∥τ∥2∞‍g-k(c2(ρ-∥τ∥2))ρr-1dρ=1c2∫0∞‍g-k(ρ)(ρc2+∥τ∥2)r-1dρ
is finite because by condition (2.23), g-k is integrable in the neighborhood of zero and (ρ/c2+∥τ∥2)r-1 is of order ρr-1 at infinity.
Step  2 (proving the main bound). The matrix A from Lemma 3.4(a) is of size n×(n-1) and A+ is of size (n-1)×n. Equation (3.50) and Rn-1=ImH⊕Im(I-H) imply that
(3.64)rank(I-H)=n-1-r=k-1.
Equation (3.48) written point-wise is t(2)=u+v,  t(2)∈T(2),  u∈A+(ΓΔ-τ),  v∈Im(I-H). Since u,v here are orthogonal, we have
(3.65)g-1(t(2))=∫R‍p-((t12+∥u∥22+∥v∥22)1/2)dt1≡h(∥u∥2,∥v∥2),
where the right side defines the function h.
The transition fromt(2) to the coordinates (u,v) is described by an orthogonal matrix. This can be shown as follows. Take in ImH, an orthonormal basis and identify u with the set of its coordinates in that basis. Do the same with Im(I-H) and v. Join the two bases. The equation t(2)=u+v induces a transformation between the original coordinates of t(2) and the ones in the joint basis. Since ImH⊥Im(I-H), the Jacobian of the resulting transformation is unity. Hence, by (3.48), (3.64), (3.65), and (3.57),
(3.66)α(Y,τ)=∫A+(ΓΔ-τ)+Im(I-H)‍g-1(u+v)dudv=∫A+(ΓΔ-τ)‍[∫Rk-1‍h(∥u∥2,∥v∥2)dv]du=∫A+(ΓΔ-τ)‍g-k(∥u∥2)du=β(ΓΔ,τ)≤c(∥τ∥2)mes(Δ).
Part (b). In the bound (see (3.59) and (3.60)),
(3.67)β(Y,τ)=c1∫Y-τ‍g-k(∥A+v∥2)dv≤c1∫Y-τ‍g-k(c2∥v∥2)dv,
we have mes(Y-τ)=mes(Y)→0, so β(Y,τ)→0 uniformly in τ. As in (3.66), it follows that(3.68)α(Y,τ)=∫A+(Y-τ)+Im(I-H)‍g-1(u+v)dudv=β(Y,τ)⟶0.Lemma 3.6.
Letf1,p,τ,A,A+, and H be the same as in Lemma 3.4(b).(a)
SupposeY=ΓΔ where Δ⊆{s∈Im(MX-pp′):∥s∥2=1}. If (2.25) holds, then (3.55) is true with T(2) from Lemma 3.4(b).(b)
IfY⊆{s∈Im(MX-pp′):∥s∥2≤1}, then α(Y,τ)→0, as mes(Y)→0, uniformly in τ.Proof.
The proof essentially follows that of Lemma3.5(a). From (3.49), and (3.54) we see that instead of (3.58) we have r=n-k-1. Instead of (3.64) we have rank(I-H)=k. This leads to the definition of the marginal density used in (2.25). With these changes, (3.62), and (3.66) are true and prove part (a).
With the same changes the proof of Lemma3.5(b) goes through.In the course of the proof, we approximateχΦ by functions φε±∈CL(Rn) from below and above. Then, we show that the rate of approximation improves as ε→0 and ρ→a.Step 1 (constructing the approximating functions).
Denoteρ(x,A)=infy∈A∥x-y∥2 the distance from a point x to a set A. For a given ε>0, set Ψε+={x∈Rn:ρ(x,Φ)≤ε},  Ψε-={x∈Φ:ρ(x,bd(Φ))>ε}.Obviously,Ψε-⊂Φ⊂Ψε+. As in [10, page 12 proof of (ii)⇒(iii)], we can construct functions φε± that are uniformly continuous on Rn and satisfy
(3.69)φε+=1onΦ,φε+=0outsideΨε+,0≤φε+≤1onRn,φε-=1onΨε-,φε-=0outsideΦ,0≤φε-≤1onRn.Step 2 (reducing the convergence statement to a special case).
Denote(3.70)I(φ)=∫Rn-1(MT1φ)(t(2))g-1(t(2))dt(2).
Here, we show that
(3.71)limε→0I(φε±)=I(χΦ)
implies (2.24). From the inequality φε-≤χΦ≤φε+ we have
(3.72)Jρ(φε-)≤Jρ(χΦ)≤Jρ(φε+).
By Theorem 2.7,
(3.73)limρ→aJρ(φε±)=I(φε±),foranyε>0.
By (3.71), for any δ>0, there exists ε(δ)>0 such that
(3.74)I(φε+)-δ≤I(χΦ)≤I(φε-)+δ,0<ε≤ε(δ).
By (3.73), for the same δ, there is ρ(δ)∈(0,a) such that
(3.75)|Jρ(φε±)-I(φε±)|≤δ,ρ(δ)≤ρ<a.
Equations (3.72), (3.74), and (3.75) imply that
(3.76)I(χΦ)-2δ≤I(φε-)-δ≤Jρ(φε-)≤Jρ(χΦ)≤Jρ(φε+)≤I(φε+)+δ≤I(χΦ)+2δ,ρ(δ)≤ρ<a.
Since δ>0 is arbitrary, this proves (2.24).Step 3 (further reduction to an indicator function).
Let us callB(ε)=Ψε+∖Ψε- a band. Here, we show that (3.71) will follow if we prove that
(3.77)limε→0I(χB(ε))=0.
Note the identity
(3.78)I(χB(ε))=I(χΨε+)-I(χΨε-)=[I(χΨε+)-I(χΦ)]+[I(χΦ)-I(χΨε-)].
Since each of the square brackets on the right is nonnegative, (3.77) gives limε→0I(χΨε±)=I(χΦ). Now, it suffices to take into account the inequality I(χΨε-)≤I(φε-)≤I(χΦ)≤I(φε+)≤I(χΨε+) to see that (3.71) follows.Step 4 (geometric description of the task).
Denotel(s,i)={st+i:t∈R} a straight line with a slopes∈Rn and intercepti∈Rn and put
(3.79)bε(t(2))=(MT1χB(ε))(t(2))=limr→∞12r∫-rr‍χB(ε)(f1t1+z(t(2)))dt1.
This expression shows that we need to study the set {t1∈R:f1t1+z(t(2))∈B(ε)} of points common to the straight line l(f1,z(t(2))) and the band B(ε). The cylindrical nature of the sets considered implies that the band admits an equivalent definition as a cylinder with the base
(3.80)MXB(ε)={x∈ImMX:ρ(x,bd(MXΦ))≤ε}
and element ImX. From this definition, we see that the condition f1t1+z(t(2))∈B(ε) is equivalent to
(3.81)(MXf1)t1+MXz(t(2))∈MXB(ε).
We have to sort out points t(2)∈Rn-1 into (a) those for which the set of points t1 satisfying (3.81) is so small that (3.79) is zero and (b) that set is large enough for (3.79) to be positive.Step 5 (casef1∈ImX).
In this case,MXf1=0, and (3.81) reduces to
(3.82)MXP2(a)Λ2-1(a)t(2)+MXQXβ∈MXB(ε).
This condition does not contain t1. Therefore, if it is not satisfied, the integrand in (3.79) is identically zero and bε(t(2))=0 for such t(2). If it is satisfied, the integrand in (3.79) is identically 1 and bε(t(2))=1 for such t(2). Hence,
(3.83)I(χB(ε))=∫(3.82)istrue‍g-1(t(2))dt(2).
The set of t(2) satisfying (3.82) has an infinite Lebesgue measure which precludes us from application of the continuity property of the Lebesgue measure (for an integrable h,∫A‍h(t)dt→0 as mes(A)→0).
As one can see from Figure1, MXB(ε) is not a cone-like set. We divide it into two parts: αεin={s∈MXB(ε):∥s∥2≤1} includes the points inside the unit ball and on the unit sphere and αεout={s∈MXB(ε):∥s∥2>1} includes the points of MXB(ε) outside the unit ball. Put Δε={s∈ImMX:∥s∥2=1,ρ(s,bd(MXΦ))≤ε} and let us show that
(3.84)αεout⊂ΓΔε.
Suppose x∈αεout. Then, x∈ImMX,∥x∥2>1 and by (3.80), there exists a sequence {y(k)}⊂bd(MXΦ) such that limsup∥x-y(k)∥2≤ε. The sequence {y(k)} is bounded and therefore precompact, while bd(MXΦ)=MX(cl(Φ)∖Φ) is closed by construction. Hence, we can assume that y(k)→y∈bd(MXΦ) (passing to a subsequence, if necessary). Then, ∥x-y∥2≤ε. This implies  ∥x/∥x∥2-y/∥x∥2∥2≤ε/∥x∥2<ε. Let s=x/∥x∥2. Since bd(MXΦ) is cone-like, it follows that y/∥x∥2∈bd(MXΦ) and ρ(s,bd(MXΦ))<ε. This proves that s∈Δε,  x=∥x∥2s∈ΓΔε and (3.84) is true.
From (3.84), we conclude that
(3.85)MXB(ε)⊂αεin∪ΓΔε.
This means that the set of solutions of (3.82) is contained in the union of two sets: T1(2)={t(2)∈Rn-1:At(2)+τ∈ΓΔε} and T2(2)={t(2)∈Rn-1:At(2)+τ∈αεin} where A=MXP2(a)Λ2-1(a) and τ=MXQXβ.
By Lemma3.4(a), T1(2)=A+(ΓΔε-τ)+Im(I-H). By Lemma 3.5(a)
(3.86)∫T1(2)‍g-1(t(2))dt(2)≤c(∥τ∥2)mes(Δε)⟶0,ε→0.
By Lemma 3.4(a), T2(2)=A+(αεin-τ)+Im(I-H). By Lemma 3.5(b)
(3.87)∫T2(2)‍g-1(t(2))dt(2)⟶0asmes(αεin)⟶0.
Equations (3.86) and (3.87) prove that I(χB(ε))→0,  ε→0.Figure 1
M
X
Φ consists of two cones bounded by solid lines. MXB(ε) is bounded by dashed lines.Step 6 (casef1∉ImX).
Equation (2.5) implies that if the straight line l(MXf1,MXz(t(2))) has a bounded intersection with the band MXB(ε), then (MT1χB(ε))(t(2))=0. Therefore, we are interested in the set of slopes:
(3.88)Sl={MXf1:l(MXf1,MXz(t(2)))∩MXB(ε)isunbounded}.
Let us prove that
(3.89)MXf1∈Sl⇒MXf1/∥MXf1∥2∈bd(S).
Suppose MXf1∈Sl. Then, there exists a sequence of points p(k) which lie on the straight line, p(k)=(MXf1)t1(k)+MXz(t(2)), where |t1(k)|→∞, and which are 2ε-close to bd(MXΦ): there exists a sequence y(k)∈bd(MXΦ) such that
(3.90)∥p(k)-y(k)∥2≤2ε,∥y(k)∥2⟶∞.
Equation (2.21) implies that MXΦ=ΓS and bd(MXΦ)=Γbd(S)∪{0}. Therefore s(k)≡y(k)/∥y(k)∥2∈bd(S). Because bd(S) is compact, we can assume that s(k)→s∈bd(S). From (3.90), we have ∥(MXf1)t1(k)+MXz(t(2))-∥y(k)∥2s(k)∥2≤2ε or, dividing both sides by ∥y(k)∥2, ∥(MXf1)t1(k)/∥y(k)∥2-s(k)∥2→0. This shows that the sequence {t1(k)/∥y(k)∥2} is bounded and can be assumed convergent to a number δ. Letting k→∞, we get (MXf1)δ=s∈bd(S), and (3.89) is true.
FixMXf1∈Sl and denote p=MXf1/∥MXf1∥2∈bd(S)⊂ImMX. The matrix pp′ projects Im MX onto the straight line l(MXf1,0). The projector MX-pp′ projects Im  MX onto Im(MX-pp′). Condition (3.81) implies two equations in projections:
(3.91)pp′[(MXf1)t1+MXz(t(2))]∈pp′MXB(ε),(MX-pp′)[(MXf1)t1+MXz(t(2))]∈(MX-pp′)MXB(ε).
The first one can be safely omitted because the shift pp′MXz(t(2)) along a line does not change the property of the line to have an unbounded intersection with MXB(ε). Denoting A=(MX-pp′)P2(a)Λ2-1(a),  τ=(MX-pp′)QXβ, and using (3.52), we write the second equation as
(3.92)At(2)+τ∈(MX-pp′)B(ε).
The set of points t(2) such that (3.81) holds for an unbounded set of values t1 is clearly contained in the set T(2) of points t(2) that satisfy (3.92). For t(2)∈T(2) we use an obvious bound bε(t(2))≤1 (see (3.79)), and, therefore, I(χB(ε))≤∫(3.92)istrue‍g-1(t(2))dt(2). The rest of the proof closely follows that for the case f1∈ImX. An inclusion of type (3.85) holds with MX replaced by MX-pp′ in all definitions. The reference to Lemma 3.4(a) is replaced by Lemma 3.4(b). Lemma 3.6 is used instead of Lemma 3.5. The conclusion is that I(χB(ε))→0.We shall use several times the fact that for the densityg-1 in representation (2.24)will, the conditions ∫A‍g-1(t(2))dt(2)>0 and mes(A)>0 are equivalent. From (2.24), we see that the task is to calculate the function (3.79) with Φ in place of B(ε). Because of (2.21), the condition that f1t1+z(t(2))∈Φ is equivalent to the condition in projections:
(3.93)(MXf1)t1+MXz(t(2))∈ΓS.The magnitude ofβ(Φ) depends on the set
(3.94)A(t(2))={t1∈R:f1t1+z(t(2))∈Φ}.The properties ofA(t(2)) depend on those of mes{t(2)∈Rn-1:MXz(t(2))∈G} for certain sets G⊆ImMX, as we will see.Lemma 3.7.
SupposeImQ⊆ImX (Q is the projector from Theorem 2.7). Then, MXz(t(2))=0 for all t(2)∈Rn-1.
LetImQ⊈ImX. Then, mes{t(2):MXz(t(2))=0}=0.Proof.
The assumptionImQ⊆ImX implies that MXfi=0 for i=2,…,n, which can be written as MXP2(a)=0. Hence, MXz(t(2))=MXP2(a)[Λ2-1(a)t(2)+P2′(a)Xβ]=0 for any t(2)∈Rn-1.
IfImQ⊈ImX, then fi∉ImX and MXfi≠0 for at least one i. The matrix MXP2(a) is of size n×(n-1), and of rank at least 1. From the equation dimN(MXP2(a))+rank(MXP2(a))=n-1 we see that the null space N(MXP2(a)) is of dimension at most n-2. The set of t(2) satisfying MXz(t(2))=MXP2(a)Λ2-1(a)t(2)+MXQXβ=0 is a partial solution t0(2) plus an arbitrary element from the null space N(MXP2(a)). Thus the set of all solutions is a hyperplane in Rn-1 of dimension at most n-2 and has (n-1)-dimensional Lebesgue measure zero.Lemma 3.8.
Suppose we are given a setG⊆Rn-1 and for almost each t(2)∈G, the set A(t(2))⊆R is defined by (3.94).(a)
IfA(t(2)) is bounded for almost all t(2)∈G and mes(G)>0, then β(Φ)<1. In particular, if G=Rn-1, then β(Φ)=0.(b)
If(3.95)R∖A(t(2))isboundedforalmostallt(2)∈G
and mes(G)>0, then β(Φ)>0. In particular, if G=Rn-1, then β(Φ)=1.(c)
Suppose thatG=Rn-1 and for almost all t(2)∈G,
(3.96)thesetA(t(2))isahalf-axis
(i.e., either A(t(2))=(a(t(2)),∞) or A(t(2))=(-∞,a(t(2))) with some a(t(2))∈R). Then, β(Φ)=1/2.Proof.
IfA(t(2)) is bounded, then χΦ(f1t1+z(t(2)))=0 outside a bounded set, and by (2.5),
(3.97)MχA(t(2))=(MT1χΦ)(t(2))=0
for almost all t(2)∈G. Since 0≤MχA(t(2))≤1 outside G, we have
(3.98)β(Φ)≤∫Rn-1∖G‍g-1(t(2))dt(2)<1.
Obviously, the integral is null if G=Rn-1.
By (3.97), condition (3.95) implies that
(3.99)MχA(t(2))=M(1-χR∖A(t(2)))=1a.e.inG.
Since mes(G)>0, we have
(3.100)β(Φ)≥∫G‍g-1(t(2))dt(2)>0.
In case G=Rn-1, the last integral is 1.
Consider, for example, the case of the right half-axis:(3.101)MχA(t(2))=limr→∞12r(∫a(t(2))0‍dt+∫0r‍dt)=12.
Hence, β(Φ)=(1/2)∫Rn-1‍g-1(t(2))dt(2)=1/2.Case 1.
Letf1∈Φ. Then, f1∉ImX and MXf1≠0. Define p=MXf1/∥MXf1∥2. Obviously, MXf1∈ΓS and p∈S. In the equation
(3.102)(MXf1)t1+MXz(t(2))=∥MXf1∥2t1[p+MXz(t(2))∥MXf1∥2t1],
for all large |t1|, the term in the square brackets is close to p and, because MXΦ is open in ImMX, this term belongs to MXΦ. Since MXΦ is cone-like, the left side of (3.102) belongs to MXΦ for all large |t1|. Hence, the set A(t(2)) contains the neighborhood of infinity (more precisely, all t1 with |t1|>N where N depends on t(2)). By Lemma 3.8(b), where G=Rn-1, then β(Φ)=1.Case 2.
Supposef1∈bd(Φ).
Subcase2.1. Let f1∈ImX or, equivalently, MXf1=0. Condition (3.93) reduces to
(3.103)MXz(t(2))=MXP2(a)Λ2-1(a)t(2)+MXQXβ∈ΓS.
By Lemma 3.4(a), the set of solutions of (3.103) is T(2)=A+(ΓS-τ)+Im(I-H) where A+ is the Moore-Penrose inverse of A=MXP2(a)Λ2-1(a),τ=MXQXβ, and H is the projector from Lemma 3.2. Using Lemma 3.8(b), (3.66), and (3.59) we get
(3.104)β(Φ)=∫T(2)‍g-1(t(2))dt(2)=∫A+(ΓS-τ)‍g-k(∥u∥2)du.
The density g-k(∥u∥2) is spherically symmetric. It is positive everywhere because
(3.105)g-k(t)=∫Rk‍p-((t2+∥v∥22)1/2)dv=0
would mean p-((t2+t1)1/2)=0 for any t1≥0, that is, p-(t1)=0 for any t1≥t, which is impossible by assumption. A+ is one-to-one on ImA=ImMX, so A+(ΓS-τ) is a nonempty proper subset of ImA+. Therefore, 0<β(Φ)<1.
Subcase 2.2. Suppose f1∉ImX. As in Case  1, p is defined. Let us show that it satisfies
(3.106)p′Wp=c.
Since f1 is a limit point of Φ, there is a sequence xn∈Φ such that xn→f1. Hence, MXxn→MXf1 where ∥MXf1∥2≠0 because f1∉ImX. From (2.21), it follows that MXxn∈ΓS,zn≡MXxn/∥MXxn∥2∈S and zn′Wzn>c. Letting n→∞ and using zn→p, we get p′Wp≥c. Since f1∉Φ, we have MXf1∉MXΦ and p∉S. Thus, p′Wp>c is impossible and (3.106) is true.
In the three examples in the following we assume thatn>k+1. Then, ImQ⊈ImX, and by Lemma 3.7(b),
(3.107)MXz(t(2))≠0,foralmostallt(2)∈Rn-1.Sub-sub-case (a) (Example ofβ(Φ)=1). Let W be symmetric. Assumptions (1.13) imply that var(u)=σε2(I-ρW)-2 whenever det(I-ρW)≠0, so Σ-1(ρ)=(I-ρW)2. If μi denote eigenvalues of W, then the eigenvalues of I-ρW are νi(ρ)=1-ρμi. Let the constant c in the definition of the critical region Φ be positive. Put a=1/c and with some integer i0 and real numbers μ-,  μ+ such that 1<i0<n,  μ-<1/a<μ+ define μi by
(3.108)μ1=⋯=μi0-1=μ-<μi0=1a<μi0+1=⋯=μn=μ+.
Then, for 0<ρ≤a,
(3.109)ν1(ρ)=⋯=νi0-1(ρ)=1-ρμ->νi0(ρ)=1-ρa>νi0+1(ρ)=…=νn(ρ)=1-ρμ+,
and Σ-1/2(a) has eigenvalues
(3.110)ν1(a)=⋯=νi0-1(a)=1-aμ->νi0(a)=0<|νi0+1(a)|=⋯=|νn(a)|=|1-aμ+|.
Hence, in the notation of Condition 3,  λ1(ρ)=νi0(ρ) and λ2(ρ),…,λn(ρ) are obtained from |νi(ρ)|,  i≠i0, by reordering. For example, if 1-aμ-<|1-aμ+|, then λi(ρ)=νi-1(ρ),i=2,…,i0;  λi(ρ)=|νi(ρ)|,  i=i0+1,…,n. Assuming this ordering, denote the corresponding eigenvectors of Σ-1/2(a) by f1,f2-,…,fi0-,fi0+1+,…,fn+ where the minuses and pluses remind us that the corresponding eigenvalues depend on μ- and μ+, respectively.
Letk=i0-1 and suppose that
(3.111)f2-,…,fi0-spanImX,f1,fi0+1+,…,fn+spanImMX.
Denote
(3.112)x=(MXf1)t1+MXz(t(2))∈ImMX.
Because of (3.111), here MXf1=f1. Besides, since z(t(2)) is orthogonal to f1, the decomposition
(3.113)MXz(t(2))=∑i=i0+1n‍cifi+
does not contain f1. Hence, (3.112) becomes
(3.114)x=t1f1+∑i=i0+1n‍cifi+.
Remembering that the eigenvectors of Σ-1/2(a) are just reordered eigenvectors of W, we get
(3.115)Wx=1at1f1+μ+∑i=i0+1n‍cifi+,x′Wx=1at12+μ+∑i=i0+1n‍ci2=ct12+μ+∑i=i0+1n‍ci2,
for any x of form (3.112). From (3.107) and (3.113) we see that here
(3.116)∑i=i0+1n‍ci2≠0a.e.inRn-1.
Hence, normalizing x,  s=x/∥x∥2, and assuming that μ+>c, we get from (3.115)
(3.117)s′Ws=ct12+μ+∑i=i0+1nci2t12+∑i=i0+1nci2>c,
for all t1∈R and almost all t(2)∈Rn-1. This shows that the set (3.94) coincides with R for almost all t(2)∈Rn-1 and by Lemma 3.8(b) β(Φ)=1.
Sub-sub-case (b)Example ofβ(Φ)=0. We change (3.111) so as to obtain in (3.115) μ- in place of μ+. Let k=n-i0 and suppose that
(3.118)fi0+1+,…,fn+spanImX,f1,f2-,…,fi0-spanImMX.
Instead of (3.113), we get MXz(t(2))=∑i=2i0‍cifi- which leads to an analog of (3.114): x=t1f1+∑i=2i0‍cifi-.  Equation(3.115) is replaced by
(3.119)x′Wx=ct12+μ-∑i=2i0‍ci2,
where ∑i=2i0‍ci2≠0 a.e. in Rn-1. The result is that
(3.120)s′Ws=ct12+μ-∑i=2i0ci2t12+∑i=2i0ci2<c
for all t1∈R and almost all t(2)∈Rn-1. Hence, the set (3.94) is empty for almost all t(2)∈Rn-1 and by Lemma 3.8(a), β(Φ)=0.
Sub-sub-case
(
c
)
Example of β(Φ)=1/2]. MXz(t(2)) runs over Im(MXP2(a)). If k>1, then n-1>n-k, and we can choose the eigenvectors f2,…,fn of Σ-1/2(a) so that ImQ⊃ImMX, and then MXz(t(2)) runs over ImMX=Im(MXP2(a)). Let ΓS be a union of two cones, as in Figure 2. The line l(MXf1,0) lies on the boundary of the cones. In Figure 2, La=l(MXf1,0) and Lb=l(b(t(2)),0). For any nonzero shift MXz(t(2)), half of the shifted line is contained in MXΦ because La and Lb are not parallel. Here, “half” means “either for all large positive t1 or for all large negative t1.” By Lemma 3.8(c), β(Φ)=1/2.Figure 2
The base ofΦ contains the area between two solid lines La and  Lb. The dashed line is l(MXf1,MXz(t(2)))=La+MXz(t(2)).Case 3.
Supposef1∈Φ-. Then, as in part 1,MXf1≠0 and p is defined. It satisfies p′Wp<c. The term in the square brackets in (3.102) for all large |t1| belongs to an open set MXΦ-. Since this set is cone-like, the left side of (3.102) belongs to MXΦ- for all large |t1|. The set A(t(2)) is bounded for all t(2)∈Rn-1, and by Lemma 3.8(a), β(Φ)=0.Step 1 (theε(ρ)-maximizing set is an ellipsoid).
Fory∈Mε(ρ) by orthogonality of P and Condition 7, we have
(3.121)fρ(y)=∏j=1n‍λj(ρ)p(∥Λ(ρ)P′(ρ)(y-Xβ)∥2)≥(1-ε(ρ))∏j=1n‍λj(ρ)m.
Since p-1 is decreasing, this is equivalent to ∥Λ(ρ)P′(ρ)(y-Xβ)∥2≤p-1((1-ε(ρ))m). Denoting h(ρ)=p-1((1-ε(ρ))m),  z=P′(ρ)(y-Xβ), we rewrite the resulting inequality as
(3.122)∑j=1d‍zj2[h(ρ)/λj(ρ)]2+∑j=d+1n‍zj2[h(ρ)/λj(ρ)]2≤1.
In z coordinates, this is an ellipsoid with half-axes h(ρ)/λj(ρ).Step 2 (characterizing the rate of decay ofh(ρ)).
First we determine the rate with whichp-1(x) approaches 0 when x→m. To approximately invert p, in the equation x=p(r), we use representation (2.30): x=m(1-cprαp)(1+o(rαε)). This is equivalent to (x/m)(1+o(rαε))=1-cprαp. Solving this equation for the r involved in the right-hand side produces
(3.123)p-1(x)=r={1cp[1-xm(1+o(rαε))]}1/αp,r→0.
Since ρ→a, we can put r=a-ρ in the last equation to get
(3.124)p-1(x)={1cp[1-xm(1+o((a-ρ)αε))]}1/αp,ρ→a.
To obtainh(ρ), let x=(1-ε(ρ))m. Then, by (2.29),
(3.125)1-xm(1+o((a-ρ)αε))=1-(1-ε(ρ))(1+o((a-ρ)αε))=1-1+o((a-ρ)αε)+ε(ρ)(1+o((a-ρ)αε))=ε(ρ)(1+o(1)).
Combining this with (3.124) and (2.29), we have
(3.126)h(ρ)=p-1(x)={cεcp(a-ρ)αε(1+o(1))}1/αp=(cεcp)1/αp(a-ρ)αε/αp(1+o(1)).Step 3 (behavior of half-axes).
We intend to show that forj=d+1,…,n the half-axes tend to zero, so that in the limit
(3.127)zd+1=⋯=zn=0,
whereas for j=1,…,d, they tend to ∞, and in the limit, z1,…,zd can be any real numbers.
Forj=d+1,…,n from (2.28), we see that the condition h(ρ)/λj(ρ)→0 is equivalent to h(ρ)→0. Because of (2.31), this condition is the same as (1-ε(ρ))m→m which holds by (2.29).
For the remaining half-axes, we get by (2.27), (3.126), and (2.32)
(3.128)h(ρ)λj(ρ)=1cλ(cεcp)1/αp(a-ρ)αε/αp-αλ(1+o(1))⟶∞.
We have proved our statement on convergence of the half-axes. According to (2.26), (3.127) rewrites as (P′(a)(y-Xβ))j=0,  j=d+1,…,n. Since the rows of P′(a) are eigenvectors of A(a), the last equation means that y-Xβ is orthogonal to eigenvectors numbered j=d+1,…,n. Hence, y-Xβ belongs to the null space of A(a), and the set of all such y’s is Xβ+N(A(a)).
In this proof, the setXβ+N(A(a)) is obtained as the limit of the ridge of fρ. In general, this ridge can have any shape and the limit will not be Xβ+N(A(a)).


## 4. Summary

Theorem2.2 is the key result in this paper. We tried to obtain it under minimal conditions on g. It would be interesting to relax our conditions. In particular, densities that are not spherically symmetric are of interest in applications. Note that when the dependence on λ is more complex than in gλ(x)=λg(λx), one needs very different techniques and more detailed assumptions about g to obtain an analog of Theorem 2.2. Such results can be found in [15]. Another promising direction of research is to consider tests other than the Cliff-Ord one.



---

*Source: 926164-2012-12-31.xml*