Testing hypotheses about mixture distributions using not
identically distributed data
Daniel Gaigall

arXiv:1602.06383v2 [] 20 Apr 2016

Department of Mathematics, Heinrich-Heine-University Düsseldorf,
Universitätsstr. 1, 40225 Düsseldorf, Germany.

Abstract
Testing hypotheses of goodness-of-fit about mixture distributions on the basis of
independent but not necessarily identically distributed random vectors is considered.
The hypotheses are given by a specific distribution or by a family of distributions.
Moreover, testing hypotheses formulated by Hadamard differentiable functionals is discussed in this situation, in particular the hypothesis of central symmetry, homogeneity
and independence. Kolmogorov-Smirnov or Cramér-von-Mises type statistics are suggested as well as methods to determine critical values. The focus of the investigation
is on asymptotic properties of the test statistics. Further, outcomes of simulations for
finite sample sizes are given. Applications to models with not identically distributed
errors are presented. The results imply that the tests are of asymptotically exact size
and consistent.
Keywords: not identically distributed observations, mixture distribution, KolmogorovSmirnov statistic, Cramér-von-Mises statistic, empirical process, Vapnik-Čhervonenkis
class, Hadamard differentiability
2000 MSC: 62G10, 62G09

1

Introduction

Not identically distributed errors are discussed in various theoretical and practical contexts,
see, e.g., the works of Lu et al. [21], Kuljus and Zwanzig [20], Görnitz et al. [14], Eiker
[13] and Delaigle and Meister [11]. Consider here the following situation as a motivation.
Suppose independent and identically distributed data are underlying, where an independent
but not identically distributed noise is present. It is assumed that the difference in the
distribution of the noise vanishes if the number of observations increases. The interesting
statistical problem is a testing problem of goodness-of-fit formulated with the independent
and identically distributed original data.
More precisely, consider a sequence of independent and identically distributed real valued
random variables Y1 , Y2 , . . . with unknown underlying distribution LpY1 q. Suppose the user
has to treat the testing problem of goodness-of-fit
H : LpY1 q “ LpY0 q, K : LpY1 q ‰ LpY0 q,

(1.1)

where Y0 is a real valued random variable with known distribution LpY0 q. Furthermore, let
Z1 , Z2 , . . . be another sequence of independent but not necessarily identically distributed real
Email adress: Daniel.Gaigall@uni-duesseldorf.de

1

valued random variables with known distributions such that the convergence in distribution
d

Zi ÝÑ Z as i Ñ 8
holds, with a real valued random variable Z. Assume Y1 , Y2 , . . . and Z1 , Z2 , . . . are independent. In addition, a sequence of known and measurable maps ei : R ˆ R Ñ R, i P N, is
given and fulfills the convergence
@z P R : lim ei p¨, zq “ ep¨, zq uniformly on R,
iÑ8

where e : R ˆ R Ñ R is a measurable map. E.g.,
ei py, zq “ epy, zq “ y ` z, py, zq P R ˆ R, i P N.

(1.2)

Suppose the user cannot observe directly Y1 , Y2 , . . . , but observes
e1 pY1 , Z1 q, e2 pY2 , Z2 q, . . . ,
i.e., ei p¨, Zi q represents a random noise in measurement i P N. Therefore, the user has to
treat testing problem (1.1) on the basis of the independent but not necessarily identically
distributed data X1 , . . . , Xn ,
Xi :“ ei pYi , Zi q, i P N,
where n P N is a given sample size. In example (1.2),
Xi “ Yi ` Zi , i P N.
Denote by Fi the distribution function of Xi “ ei pYi , Zi q, i P N, and by F the distribution
function of epY1 , Zq. Let Hi be the distribution function of ei pY0 , Zi q, i P N, and let G be
the distribution function of epY0 , Zq. Assume that for all z P R, the map ep¨, zq is injective.
Then, testing problem (1.1) is equivalent to
H : F “ G, K : F ‰ G.
Moreover, if the hypothesis H is true, Fi “ Hi , i P N.
In order to get a permutation invariant test statistic, i.e., invariant under transformation of
the data from the set
tTn : ˆni“1 R Ñ ˆni“1 R; Tn px1 , . . . , xn q “ pxπn p1q , . . . , xπn pnq q,
(
πn : t1, . . . , nu Ñ t1, . . . , nu bijective ,

define the arithmetically averaged distribution function
Fn pxq :“

n
1 ÿ
Fi pxq, x P R,
n i“1

based on the distribution functions F1 , . . . , Fn of X1 , . . . , Xn and the arithmetically averaged
distribution function
n
1 ÿ
Gn pxq :“
Hi pxq, x P R,
n i“1

based on the distribution functions H1 , . . . , Hn , the distribution functions of X1 , . . . , Xn if
the hypothesis H is true. Under H, Fn “ Gn . Consider the empirical distribution function
based on X1 , . . . , Xn ,
n
1 ÿ
IpXi ď xq, x P R,
F̂n pxq :“
n i“1

and the Kolmogorov-Smirnov type statistic
?
KSn :“ n sup |F̂n pxq ´ Gn pxq|
xPR

2

(1.3)

as well as the Cramér-von-Mises type statistic
ż
˘2
`
F̂n pxq ´ Gn pxq Gn pdxq.
CvMn :“ n

(1.4)

Assume F is continuous. In fact, the test statistics (1.3) and (1.4) can be used to construct
asymptotically exact and consistent tests for testing problem (1.1). See the results in Section
2. Note that the distribution function Gn depends on the sample size n and that the
test statistics (1.3) and (1.4) are no common Kolmogorov-Smirnov and Cramér-von-Mises
statistics. The replacement of Gn with G in the test statistics (1.3) and (1.4), i.e., the
common Kolmogorov-Smirnov or Cramér-von-Mises statistic, yields no applicable test for
testing problem (1.1). This follows easily from the results in Section 2.
In Section 2, testing goodness-of-fit based on not identically distributed data is discussed.
In comparison to the situation above the model is more general and the hypothesis is formulated in a more general way. Therefore, generalizations of the test statistics (1.3) and
(1.4) are considered and methods to determine critical values are suggested. The tests are
applicable, e.g., in the situation presented above. It is shown that the tests are of asymptotically exact size and consistent. Section 3 extends the model in Section 2 to hypotheses
of families of distributions. Analogous results are established. Section 4 treats hypotheses
formulated by Hadamard differentiable functionals in the same setting. Similar results as in
the goodness-of-fit setting hold for a broad class of tests. Special cases are the hypothesis
of homogeneity, central symmetry and independence and applications are, e.g., situations
with not identically distributed errors. In general, the presented Kolmogorov-Smirnov and
Cramér-von-Mises type tests based on not identically distributed data and can be regarded
as generalizations of the related Kolmogorov-Smirnov and Cramér-von-Mises type tests in
the identically distributed case.
The original Kolmogorov-Smirnov and Cramér-von-Mises statistics are statistics for testing
goodness-of-fit, described by Anderson and Darling [2]. The idea of these statistics can be
adopted for testing hypothesis of homogeneity, symmetry and independence, see, e.g., Rosenblatt [27], Butler [6] and Blum et al. [5], respectively. An important generalization of the
mentioned goodness-of-fit statistics for hypotheses of families of distributions is considered
by Stute et al. [33]. Some papers treat other generalizations of Kolmogorov-Smirnov type
tests or Cramér-von-Mises type tests. Weiss [39] modified the classical Kolmogorov-Smirnov
statistic for the application to correlated data. Chicheportiche et al. [8] consider the case
of dependent and identically distributed observations and the Kolmogorov-Smirnov test as
well as the Cramér-von-Mises test. An interesting paper is [7]. The authors Chatterjee and
Sen study Kolmogorov-Smirnov type tests for testing the hypothesis of symmetry based on
independent but not necessarily identically distributed real valued random variables and
investigate the limit distributions of their test statistics with an application of the random
walk model.
Further, some works deal with goodness-of-fit tests based on independent but not necessarily
identically distributed data. Goodness-of-fit tests in this situation are discussed by Gosh
and Basu [15] in a fully parametric setting. These tests use the density power divergence.
Testing for normality in a model with independent but not necessarily identically distributed
observations is considered by Sarkadi [28]. In the discrete case, Collings et al. [9], Hüsler [17]
and Conover et al. [10] treat a goodness-of-fit test for the Poisson assumption, the binomial
test and the chi-square goodness-of-fit test, respectively, in the case of independent but not
necessarily identically distributed data.

3

2

Testing goodness-of-fit with hypotheses given by a
specific distribution

The following generalization of the model introduced in Section 1 is considered. Let X1 , X2 , . . .
be a sequence of independent but not necessarily identically distributed random vectors with
values in Rm , m P N. The random vector Xi has the distribution function Fi defined on
m
l
l
R , i P N, with R :“ r´8, 8sl , l P N. Henceforth, regard elements of R as column vectors.
m
Assume the existence of an uniformly continuous distribution function F defined on R
with
m
lim Fi “ F uniformly on R .
iÑ8

For n P N the given sample size, let Gn be another distribution function defined on R
well as G, where G is uniformly continuous and

m

as

m

lim Gn “ G uniformly on R .

nÑ8

In addition, let α1,n , . . . , αn,n be some weights, in particular real numbers, such that
αi,n ě 0, i “ 1, . . . , n,

n
ÿ

i“1

αi,n “ 1, lim

nÑ8

n
ÿ
?
n max αi,n “ 0, lim n
α2i,n “ κ P r1, 8q.
nÑ8

1ďiďn

i“1

(2.1)

Consider the mixture distribution function
Fn pxq :“

n
ÿ

i“1

m

αi,n Fi pxq, x P R ,

based on the distribution functions F1 , . . . , Fn of X1 , . . . , Xn . Suppose F1 , . . . , Fn are unknown, Gn and α1,n , . . . , αn,n are known and that the user has to verify the hypothesis of
goodness-of-fit
Hn : Fn “ Gn
(2.2)
on the basis of the observations X1 , . . . , Xn .

Define the weighted empirical distribution function
F̂n pxq :“

n
ÿ

i“1

m

αi,n IpXi ď xq, x P R ,

(2.3)

m

based on X1 , . . . , Xn . For elements a, b P R , a “ pa1 , . . . , am q1 , b “ pb1 , . . . , bm q1 , let the
inequality a ď b be equivalent to aj ď bj for j “ 1, . . . , m. Further, let Un :“ pUn pxq; x P
m
R q be the process
˘
? `
m
(2.4)
Un pxq :“ n F̂n pxq ´ Gn pxq , x P R .
Consider the Kolmogorov-Smirnov type statistic
?
KSn :“ sup |Un pxq| “ n sup |F̂n pxq ´ Gn pxq|
m

(2.5)

m

xPR

xPR

and the Cramér-von-Mises type statistic
ż
ż
˘2
`
2
F̂n pxq ´ Gn pxq Gn pdxq.
CvMn :“ Un pxqGn pdxq “ n

(2.6)

In general, the statistics (2.5) and (2.6) are not distribution free, neither in the case of Fn “
Gn . In order to approximate the distribution of the statistics KSn and CvMn if Fn “ Gn ,
a Monte-Carlo procedure is suggested. Therefor, simulate independently observations with
joint distribution function
px1 , . . . , xn q ÞÝÑ

n
ź

i“1

m

Gn pxi q, px1 , . . . , xn q P ˆni“1 R .
4

Determine a significance level α P p0, 1q and denote by cn;1´α the p1 ´ αq-quantile of the
distribution of KSn and by dn;1´α the p1 ´ αq-quantile of the distribution of CvMn if
pF1 , . . . , Fn q “ pGn , . . . , Gn q. The calculation procedure for practice is described above.
Then, testing procedure
“Reject Hn , iff KSn ě cn;1´α ” or testing procedure “Reject Hn , iff CvMn ě dn;1´α ” (2.7)
is suggested. If αi,n “ n1 , i “ 1, . . . , n, testing procedure (2.7), in particular the statistic
(2.5) or (2.6), is invariant under transformation of the data from the set
Tn :“tTn : ˆni“1 Rm Ñ ˆni“1 Rm ; Tn px1 , . . . , xn q “ pxπn p1q , . . . , xπn pnq q,
(
πn : t1, . . . , nu Ñ t1, . . . , nu bijective .

(2.8)

Remark 1. Assume another
sequence of known distribution functions pHi qiPN defined on
řn
m
R is given and Gn “ i“1 αi,n Hi .
a) Consider the model introduced in Section 1. Put αi,n “ n1 , Xi “ ei pYi , Zi q and let Hi
be the distribution function of ei pY0 , Zi q, i “ 1, . . . , n. Then, the test (2.7) is applicable
to testing problem (1.1). The test is of asymptotically exact size α and consistent for
testing problem (1.1). This follows from the results in this section.
b) The test (2.7) is applicable to the hypothesis
@i P t1, . . . , nu : Fi “ Hi .
Putting αi,n “

1
n,

i “ 1, . . . , n, the test (2.7) is also applicable to the hypothesis

Dπn : t1, . . . , nu Ñ t1, . . . , nu bijective @i P t1, . . . , nu : Fi “ Hπn piq .
The test is of asymptotically exact size α and consistent with respect to suitable alternatives to this hypotheses. See the results in this section.
c) Assume Fi “ F and Hi “ G, i P N. Putting αi,n “
(2.2) is a common hypothesis of goodness-of-fit

1
n,

i “ 1, . . . , n, the hypothesis

H : F “ G,
the test statistic (2.5) or (2.6) is a common Kolmogorov-Smirnov or Cramér-von-Mises
statistic and the test (2.7) is a common Kolmogorov-Smirnov or Cramér-von-Mises test
in a multivariate setting. In this sense, a generalization of the identically distributed
case is considered.
Remark 2. Note that the distribution function Gn depends on the sample size n and that
the test statistics (2.5) and (2.6) are no common Kolmogorov-Smirnov and Cramér-vonMises statistics, neither in the case of αi,n “ n1 , i “ 1, . . . , n. In fact, the replacement of
Gn with G in the test statistics (2.5) and (2.6), i.e., the common Kolmogorov-Smirnov or
Cramér-von-Mises statistic, yields no applicable test for testing problem (2.2). This follows
easily from the results in this section.

2.1

Limit results under the hypothesis

A possible first step to find limit distributions of statistics of type Kolmogorov-Smirnov and
Cramér-von-Mises is to show the convergence in distribution of the process in the supremum
and integral to a limit element. Let T be a non-empty set and let Wn :“ pWn ptq; t P T q be
a stochastic process on a probability space pΩ, A, P q with sample paths in
ℓ8 pT q :“ tf : T Ñ R; ||f ||T ă 8u,
where ||f ||T :“ suptPT |f ptq|, f : T Ñ R. The map pf1 , f2 q ÞÑ ||f1 ´ f2 ||T , pf1 , f2 q P
ℓ8 pT q ˆ ℓ8 pT q, defines a metric on ℓ8 pT q. Let W :“ pW ptq; t P T q be another stochastic
5

process on the probability space with sample paths in ℓ8 pT q. Then, Wn converges in
d
distribution to W, in notation Wn Ñ W as n Ñ 8, iff W is pA, Bpℓ8 pT qq-measurable and
`
˘
`
˘
@f : ℓ8 pT q Ñ R continuous and bounded : lim E˚ f pWn q “ E f pWq .
nÑ8

Bpℓ8 pT qq denotes the Borel σ-field on pℓ8 pT q, || ¨ ||T q. In addition, E˚ is the outer expectation. Details are given by van der Vaart and Wellner [35].

Let ρ be a pseudometric on T such that pT, ρq is a totally bounded pseudometric space.
Then,
Ub pT, ρq :“ tf P ℓ8 pT q; f is uniformly ρ-continuousu

is a separable subspace of ℓ8 pT q. If the process W has sample paths a.s. in Ub pT, ρq, the
process W is pA, Bpℓ8 pT qq-measurable and the distribution of W is uniquely determined by
its finite dimensional marginal distributions.

The asymptotic equicontinuity is crucial for the convergence in distribution of the process
Wn , that is
ˆ
˙
@ε ą 0 : lim lim sup P ˚
sup
|Wn psq ´ Wn ptq| ą ε “ 0,
δÓ0

nÑ8

s,tPT,ρps,tqďδ

where P ˚ is the outer probability. See [35] for details. Assume the process Wn is asymptotically equicontinuous and all finite dimensional projections of Wn converge in distribution to
finite dimensional random vectors. Then, there exists a process W with sample paths a.s. in
d
Ub pT, ρq such that Wn Ñ W as n Ñ 8. Particularly, the process W is pA, Bpℓ8 pT qqmeasurable and the distribution of W is uniquely determined by its finite dimensional
marginal distributions, the distributions of those finite dimensional random vectors. See
Theorem 1.5.4 in [35].
Now, a triangular array ξ1,n , . . . , ξn,n of row-wise independent but not necessarily identically
distributed random vectors with values in Rm on the probability space is given. Let Ki,n be
m
the distribution function of ξi,n defined on R , i “ 1, . . . , n. In the relevant applications,
Ki,n depends either only on i or only on n, i “ 1, . . . , n. Assume the existence of an
m
uniformly continuous distribution function K defined on R such that
@ε ą 0 Diε P N Dnε P N, iε ď nε , @i ą iε @n ą nε , i ď n : sup |Ki,n pxq ´ Kpxq| ď ε.
xPR

m

The interesting process is the weighted empirical process Wn given by
ˆÿ
˙
n
n
ÿ
?
m
αi,n Ipξi,n ď xq ´
αi,n P pξi,n ď xq , x P R .
Wn pxq :“ n
i“1

i“1

Such types of processes, partly special cases, are studied by Shorack [31], Shorack and
Wellner [32], Alexander [1], Pollard [23], Ziegler [40] and Kosorok [18], [19]. Let Φ be the
distribution function of the m-dimensional standard normal distribution. Define a metric
ż
m
(2.9)
ρpx, yq :“ |Ipw ď xq ´ Ipw ď yq|Φpdwq, x, y P R .
m

In fact, pR , ρq is a totally bounded metric space. With the help of the Vapnik-Čhervonenkis
class theory, it can be shown that the following result holds.
Lemma 1. The process Wn is asymptotically equicontinuous with respect to the metric space
m
pR , ρq.

6

Now, the basic results under the hypothesis will be formulated. Therefor, let U “ pU pxq; x P
m
R q be a Gaussian process with expectation function identically equal to zero, covariance
function
´ `
¯
˘
m
cpx, yq :“ κ G minpx, yq ´ GpxqGpyq , x, y P R ,
(2.10)
m

and a.s. uniformly ρ-continuous sample paths. For elements a, b P R , a “ pa1 , . . . , am q1 ,
b “ pb1 , . . . , bm q1 , put minpa, bq :“ pminpa1 , b1 q, . . . , minpam , bm qq1 .
Theorem 1. Assume Fn “ Gn for n sufficiently large. Then,
d

Un ÝÑ U as n Ñ 8.
Corollary 1. Assume Fn “ Gn for n sufficiently large. It is
ż
d
d
KSn ÝÑ sup |U pxq| and CvMn ÝÑ U 2 pxqGpdxq as n Ñ 8.
xPR

m

In order to show that testing procedure (2.7) works asymptotically, it is necessary to study
the asymptotic distribution of the statistics (2.5) and (2.6) if pF1 , . . . , Fn q “ pGn , . . . , Gn q,
pnq
pnq
too. Well, let X1 , X2 , . . . be a sequence of independent and identically distributed ranm
pnq
pnq
dom vectors with underlying distribution function Gn and let Un “ pUn pxq; x P R q be
pnq
pnq
the process (2.4) based on the random vectors X1 , . . . , Xn . Denote by KSnpnq and CvMnpnq
pnq
pnq
the statistics (2.5) and (2.6) based on the random vectors X1 , . . . , Xn , respectively.
Theorem 2. It is

d

Upnq
n ÝÑ U as n Ñ 8.
Corollary 2. It follows that
KSpnq
n

d

ÝÑ sup |U pxq| and
xPR

m

CvMpnq
n

d

ÝÑ

ż

U 2 pxqGpdxq as n Ñ 8.

The following result implies that the test (2.7) is a test of asymptotically exact size α.
Corollary 3. Suppose Fn “ Gn for n sufficiently large. Then,
lim P pKSn ě cn;1´α q “ α and lim P pCvMn ě dn;1´α q “ α.

nÑ8

2.2

nÑ8

Limit results under alternatives

Now, a general result gives the limit behavior in probability of the test statistics (2.5) and
(2.6). The result holds under the hypotheses as well as under alternatives.
Theorem 3. It is
˙
1
@ε ą 0 : P ? KSn ě sup |F pxq ´ Gpxq| ´ ε ` op p1q ÝÑ 1 as n Ñ 8,
m
n
xPR
ˆ

and
ˆ

1
@ε ą 0 : P
CvMn ě
n

ż

˙
`
˘2
F pxq ´ Gpxq Gpdxq ´ ε ` op p1q ÝÑ 1 as n Ñ 8.

The following corollary gives information about the consistency of the test (2.7).
Corollary 4. Suppose F ´ G does not vanish everywhere or G-almost everywhere. Then,
lim P pKSn ě cn;1´α q “ 1 or lim P pCvMn ě dn;1´α q “ 1.

nÑ8

nÑ8

7

3

Testing goodness-of-fit with hypotheses given by a
family of distributions

In this section, the model introduced in Section 2 will be extended by adding a parameter.
Under regularity conditions, all results still hold in this model. Regard the definitions at the
m
beginning of Section 2. For d P N and a given non-empty set Θ Ď Rd , maps Gn : R ˆΘ Ñ R
m
and G : R ˆ Θ Ñ R are given such that for all ϑ P Θ Gn p¨, ϑq and Gp¨, ϑq are distribution
functions, Gp¨, ϑq is uniformly continuous and
m

@ϑ P Θ : lim Gn p¨, ϑq “ Gp¨, ϑq uniformly on R .
nÑ8

Suppose F1 , . . . , Fn are unknown, Gn and α1,n , . . . , αn,n are known and that the user has
to verify the hypothesis of goodness-of-fit
Hn : Dϑ P Θ : Fn “ Gn p¨, ϑq

(3.1)

on the basis of the observations X1 , . . . , Xn .
m

Define a process Un pϑq :“ pUn px, ϑq; x P R q by
˘
? `
m
Un px, ϑq :“ n Fn pxq ´ Gn px, ϑq , x P R .

(3.2)

Further, let tn be a pbnj“1 Bm , BdΘ q-measurable map tn : ˆnj“1 Rm Ñ Θ and put ϑ̂n :“
tn pX1 , . . . , Xn q. Bm denotes the Borel σ-field on Rm and BdΘ denotes the Borel σ-field on
Rd restricted on Θ. Consider the Kolmogorov-Smirnov type statistic
?
KSn :“ sup |Un px, ϑ̂n q| “ n sup |Fn pxq ´ Gn px, ϑ̂n q|
(3.3)
m

m

xPR

xPR

and the Cramér-von-Mises type statistic
ż
ż
`
˘2
2
Fn pxq ´ Gn px, ϑ̂n q Gn pdx, ϑ̂n q.
CvMn :“ Un px, ϑ̂n qGn pdx, ϑ̂n q “ n

(3.4)

In general, the statistics (3.3) and (3.4) are not distribution free, neither in the case of
the existence of a ϑ P Θ with Fn “ Gn p¨, ϑq. In order to approximate the distribution of
the statistics KSn and CvMn if the existence of a ϑ P Θ with Fn “ Gn p¨, ϑq is fulfilled,
a Monte-Carlo procedure is suggested. Therefor, simulate independently observations with
joint distribution function
px1 , . . . , xn q ÞÝÑ

n
ź

i“1

m

Gn pxi , ϑ̂n q, px1 , . . . , xn q P ˆni“1 R .

Determine a significance level α P p0, 1q and denote by cn;1´α the p1 ´ αq-quantile of
the distribution of KSn and by dn;1´α the p1 ´ αq-quantile of the distribution of CvMn
if pF1 , . . . , Fn q “ pGn p¨, ϑ̂n q, . . . , Gn p¨, ϑ̂n qq. The calculation procedure for practice is described above. The testing procedure
“Reject Hn , iff KSn ě cn;1´α ” or testing procedure “Reject Hn , iff CvMn ě dn;1´α ” (3.5)
is suggested.
m

Remark 3. Assume another sequence of known maps Hi : R ˆ Θ
řnÑ R, i P N, is given
such that for all ϑ P Θ Hi p¨, ϑq is a distribution function and Gn “ i“1 αi,n Hi .

a) Extend the model introduced in Section 1 in the following way. Assume the hypothesis is
given by a family of distributions tLpY0 , ϑ1 q; ϑ1 P Θ1 u, with a non-empty set Θ1 Ă Rd1 ,
d1 P N, i.e., the user has to treat the testing problem of goodness-of-fit
H : Dϑ1 P Θ1 : LpY1 q “ LpY0 , ϑ1 q, K : @ϑ1 P Θ1 : LpY1 q ‰ LpY0 , ϑ1 q.
8

In addition, suppose the error variables pZi qiPN come from families of distributions
tpLpZi , ϑ2 qqiPN ; ϑ2 P Θ2 u, i P N, with a non-empty set Θ2 Ă Rd2 , d2 P N, and that
the true parameter ϑ2 P Θ2 is unknown. Moreover, assume the error functions pei qiPN
come from families of functions tpei p¨, ϑ3 qqiPN ; ϑ3 P Θ3 u, i P N, with a non-empty
set Θ3 Ă Rd3 , d3 P N, and that the true parameter ϑ3 P Θ3 is unknown, too. Put
d “ d1 ` d2 ` d3 , Θ “ tpϑ11 , ϑ12 , ϑ13 q1 ; pϑ1 , ϑ2 , ϑ3 q P Θ1 ˆ Θ2 ˆ Θ3 u, αi,n “ n1 , Xi “
ei ppYi , Zi q, ϑ3 q and let Hi be the distribution function of ei ppY0 , Zi , ϑ3 qq, i “ 1, . . . , n,
where ϑ3 P Θ3 is the true parameter. Then, the test (3.5) is applicable to this testing
problem. Under regularity conditions, it follows from the results in this section that
the test is of asymptotically exact size α and consistent.
b) The test (3.5) is applicable to the hypothesis
Dϑ P Θ @i P t1, . . . , nu : Fi “ Hi p¨, ϑq.
Putting αi,n “

1
n,

i “ 1, . . . , n, the test (3.5) is also applicable to the hypothesis

Dϑ P Θ Dπn : t1, . . . , nu Ñ t1, . . . , nu bijective @i P t1, . . . , nu : Fi “ Hπn piq p¨, ϑq.
Under regularity conditions, it follows from the results in this section that the test is of
asymptotically exact size α and consistent with respect to suitable alternatives to this
hypotheses.
c) Assume Fi “ F and Hi “ G, i P N. Putting αi,n “ n1 , i “ 1, . . . , n, the hypothesis
(3.1) is a common hypothesis of goodness-of-fit with families of distributions
H : Dϑ P Θ : F “ Gp¨, ϑq,
the test statistic (3.3) or (3.4) is a common Kolmogorov-Smirnov or Cramér-von-Mises
statistic with estimated parameter and the test (3.5) is a common Kolmogorov-Smirnov
or Cramér-von-Mises test with families of distributions in a multivariate setting. In
this sense, a generalization of the identically distributed case is considered.
Remark 4. Note that the distribution function Gn depends on the sample size n and that the
test statistics (3.3) and (3.4) are no common Kolmogorov-Smirnov and Cramér-von-Mises
statisticswith estimated parameter, neither in the case of αi,n “ n1 , i “ 1, . . . , n. In fact, the
replacement of Gn with G in the test statistics (3.3) and (3.4), i.e., the common KolmogorovSmirnov or Cramér-von-Mises statistic, yields no applicable test for testing problem (3.1).
This follows easily from the results in this section.

3.1

Limit results under the hypothesis

Fix a parameter ϑ P Θ. In order to establish the results of Section 2 to this model, some
technical assumptions are required. The first two are regularity conditions to the families of
distributions given by the hypothesis.
(C1) There exists an open and convex set Uϑ Ă Θ such that ϑ P Uϑ and for all x P R
Gn px, ¨q : Θ Ñ R is continuous differentiable on Uϑ .

m

m

Define gn px, ϑ̃q :“ ∇Gn px, ϑ̃q, px, ϑ̃q P R ˆ Uϑ , where ∇ :“ p Bϑ̃B , . . . , Bϑ̃B q1 .
1

m

d

m

(C2) There exists a map g : R ˆ Uϑ Ñ Rd such that limnÑ8 gn “ g uniformly on R ˆ Uϑ ,
m
m
g is uniformly continuous on R ˆ Uϑ and gp¨, ϑq : R Ñ Rd has bounded components
m
on R .
Now, a regularity condition to the parameter estimator is formulated. Particular, an asymptotic expansion is needed. The condition is fulfilled, e.g., for a weighted maximum-likelihood
estimator, see Remark 5.
9

m
d
(C3) If Fn “ Gn p¨, ϑq for n sufficiently
large, there exists
ş
ş a pB 1 , B q-measurable map
m
d
ℓn p¨, ϑq : R Ñ R such that ℓn px, ϑqF
ş i pdxq and ℓn px, ϑqℓn px, ϑqFi pdxq exist and
have finite components for i “ 1, . . . , n, ℓn px, ϑqFn pdxq “ 0 and
n
? ÿ
?
αi,n ℓn pXi , ϑq ` op p1q as n Ñ 8.
npϑ̂n ´ ϑq “ n
i“1

Put
ż ´
ż
ż
¯´
¯1
vi,n pϑq :“
ℓn px, ϑq ´ ℓn py, ϑqFi pdyq ℓn px, ϑq ´ ℓn py, ϑqFi pdyq Fi pdxq, i P N,
and
wi,n py, ϑq :“

ż

ℓn px, ϑqIpx ď yqFi pdxq ´

ż

m

ℓn px, ϑqFi pdxqFi pyq, y P R , i P N.

The following condition guarantees the convergence of the covariance function of the process
Un pϑq and that Lindeberg’s condition is fulfilled.
m

(C4) If Fn “ Gn p¨, ϑq for n sufficiently
large, there exists a vpϑq P Rdˆdř
and a wp¨, ϑq : R Ñ
řn
n
d
2
R such that limnÑ8 n i“1 αi,n vi,n pϑq “ κvpϑq and limnÑ8 n i“1 α2i,n wi,n px, ϑq “
m
κwpx, ϑq for all x P R . Well, if Fn “ Gn p¨, ϑq for n sufficiently large,
d

@a P R @t ą 0 : lim n
nÑ8

n
ÿ

i“1

α2i,n

ż
ż ˆ
´
¯
1
1 ` a ℓn px, ϑq ´ ℓn py, ϑqFi pdyq

ż
¯1 ˙
ℓn px, ϑq ´ ℓn py, ϑqFi pdyq a
ˆ
ż
´
¯
?
I p n max αj,n q2 a1 ℓn px, ϑq ´ ℓn py, ϑqFi pdyq
1ďjďn
˙
ż
´
¯1
ℓn px, ϑq ´ ℓn py, ϑqFi pdyq a ą t Fi pdxq “ 0.
´

Ignore the oP p1q term in (C3). Then, the estimator ϑ̂n is invariant under transformation
of the data from the set Tn if (C3) holds and αi,n “ n1 , i “ 1, . . . , n, and testing procedure
(3.5), in particular the statistic (3.3) or (3.4), has this invariance property, too.
The following lemma shows the weak consistency of the parameter estimator.
Lemma
? řn 2. Assume Fn ş“ Gn p¨, ϑq for n sufficiently large and (C1) - (C4). Then,
n i“1 αi,n pℓn pXi , ϑq ´ ℓn px, ϑqFi pdxqq converges in distribution to a centered d-dimensional normal distribution as n Ñ 8 and
lim ϑ̂n “ ϑ in probability.

nÑ8

Now, the basic results under the hypothesis will be formulated. Therefor, let Upϑq “
m
pU px, ϑq; x P R q be a Gaussian process with expectation function identically equal to
zero, covariance function
´ `
˘
cpx, y, ϑq :“κ G minpx, yq, ϑ ´ Gpx, ϑqGpy, ϑq
¯
(3.6)
m
´ g 1 px, ϑqwpy, ϑq ´ g 1 py, ϑqwpx, ϑq ` g 1 px, ϑqvpϑqgpy, ϑq , x, y P R ,
and a.s. uniformly ρ-continuous sample paths.

10

Theorem 4. Assume (C1) - (C4) and Fn “ Gn p¨, ϑq for n sufficiently large. Then,
d

Un pϑ̂n q ÝÑ Upϑq as n Ñ 8.
Corollary 5. Assume (C1) - (C4) and Fn “ Gn p¨, ϑq for n sufficiently large. It follows that
ż
d
d
KSn ÝÑ sup |U px, ϑq| and CvMn ÝÑ U 2 px, ϑqGpdx, ϑq as n Ñ 8.
xPR

m

In order to show that testing procedure (3.5) works asymptotically, it is necessary to study
the asymptotic distribution of the statistics (3.3) and (3.4) if pF1 , . . . , Fn q “
pGn p¨, ϑ̂n q, . . . , Gn p¨, ϑ̂n qq, too.
Well, let pϑn qnPN be an arbitrary sequence of parameters with ϑn P Θ for all n P N and
pnq
pnq
limnÑ8 ϑn “ ϑ. Further, let X1 , X2 , . . . be a sequence of independent and identically
pnq
distributed random vectors with underlying distribution function Gn p¨, ϑn q, put ϑ̂n :“
m
pnq
pnq
pnq pnq
pnq
pnq
tn pX1 , . . . , Xn q and let Un pϑ̂n q “ pUn px, ϑ̂n q; x P R q be the process (3.2) based
pnq
pnq
pnq
on the random vectors X1 , . . . , Xn . Denote by KSpnq
n and CvMn the statistics (3.3) and
pnq
pnq
(3.4) based on the random vectors X1 , . . . , Xn , respectively. In addition to the conditions
(C1) and (C2), consider the following two conditions. This conditions are modifications of
the conditions (C3) and (C4).
(C5) There exists şa map ℓn : Rm ˆ Uϑ Ñ
measurable, ℓn px, ϑ̃qGn pdx, ϑ̃q and
ş
components, ℓn px, ϑ̃qGn pdx, ϑ̃q “ 0

d
R
for all ϑ̃ P Uϑ ℓn p¨, ϑ̃q is pBm , Bd qş such that
ℓn px, ϑ̃qℓ1n px, ϑ̃qGn pdx, ϑ̃q exist and have finite
and

n
? ÿ
?
pnq
npϑ̂pnq
´
ϑ
q
“
n
αi,n ℓn pXi , ϑn q ` op p1q as n Ñ 8.
n
n
i“1

Put
vnpq pϑ̃q
and
wnpq py, ϑ̃q

:“

:“
ż

ż

ℓn px, ϑ̃qℓ1n px, ϑ̃qGn pdx, ϑ̃q, ϑ̃ P Uϑ ,
m

ℓn px, ϑ̃qIpx ď yqGn pdx, ϑ̃q, y P R , ϑ̃ P Uϑ .
pq

(C6) There exists a v pq pϑq P Rdˆd such that limnÑ8 vn pϑn q “ v pq pϑq. In addition, there
m
m
pq
exists a map wpq p¨, ϑq : R Ñ Rd such that for all x P R limnÑ8 wn px, ϑn q “
wpq px, ϑq. Well,
ż
`
˘
@a P Rd @t ą 0 : lim
1 ` a1 ℓn px, ϑn qℓ1n px, ϑn qa
nÑ8
˘
`?
I p n max αj,n q2 a1 ℓn px, ϑn qℓ1n px, ϑn qa ą t Gn pdx, ϑn q “ 0.
1ďjďn

Remark 5. Under regularity conditions, a weighted maximum-likelihood estimation function
tn fulfills the mentioned conditions. Weighted maximum-likelihood estimators are considered,
e.g., by Wang and Zidek [38], Hu and Zidek [16] and Wang, van Eeden and Zidek [37].
Assume ϑ P Θ has the property Fn “ Gn p¨, ϑq for n sufficiently large. Consider n sufficiently
large. Suppose the existence of a density hn p¨, ϑ̃q of Gn p¨, ϑ̃q for all ϑ̃ P Θ with respect
to a dominating σ-finite measure µ on pRm , Bm q and assume for all x P Rm the map
ϑ̃ ÞÑ hn px, ϑ̃q, ϑ̃ P Θ, is two times continuous differentiable. Moreover, let tn be a weighted
maximum-likelihood estimation function for ϑ with the property
sup

n
ź

ϑ̃PΘ i“1

i,n
pxi , ϑ̃q “
hα
n

n
ź

i“1

`
˘
i,n
xi , tn px1 , . . . , xn q , px1 , . . . , xn q P ˆni“1 Rm .
hα
n
11

Put Ln p¨, ϑ̃q :“ log hn p¨, ϑ̃q. Assume ty P Rm ; hn py, ϑ̃q ą 0u does not depend on ϑ̃ P Θ and
Θ is open and convex. Taylor expansion yields
n
n
ÿ
? ÿ
?
B
B
pnq
pnq
Ln pXi , ϑ̃q|ϑ̃“ϑ `
Ln pXi , ϑ̃q|ϑ̃“ϑ
αi,n ∇1
n
npϑ̂pnq
αi,n
n ´ ϑn q
n
j,n
B
ϑ̃
B
ϑ̃
j
j
i“1
i“1
n
? ÿ
B
pnq
Ln pXi , ϑ̃q| pnq , j “ 1, . . . , d,
αi,n
“ n
ϑ̃“ϑ̂n
B
ϑ̃
j
i“1
pnq

where ϑj,n is on the line between ϑ̂n and ϑn (ϑj,n can be chosen measurable, see Hilfssatz
6.7 of Müller-Funk and Witting [22]), j “ 1, . . . , d. This is equivalent to
˛
¨ř
pnq
n
1 B
α
∇
,
ϑ̃q
L
pX
i,n
n
|ϑ̃“ϑ
i
Bϑ̃1
1,n ‹
n
˚ i“1
? ÿ
‹?
˚
pnq
.
.
αi,n ∇Ln pXi , ϑ̃q|ϑ̃“ϑ ` ˚
n
‹ npϑ̂pnq
n ´ ϑn q
.
n
‚
˝
ř
i“1
pnq
n
1 B
i“1 αi,n ∇ Bϑ̃ Ln pXi , ϑ̃q|ϑ̃“ϑ
d

n
? ÿ
pnq
“ n
αi,n ∇Ln pXi , ϑ̃q|
i“1

pnq

Suppose ϑ̂n

pnq
ϑ̃“ϑ̂n

.

has the property
n
? ÿ
pnq
lim n
αi,n ∇Ln pXi , ϑ̃q|
nÑ8

d,n

i“1

pnq
ϑ̃“ϑ̂n

“ 0 in probability

and assume the existence of a matrix ipϑn q P Rdˆd such that ipϑn q is invertible and
˛
˛
¨¨ ř
pnq
n
1 B
α
∇
,
ϑ̃q
L
pX
i,n
n
|
i
ϑ̃“ϑ1,n
Bϑ̃1
‹
‹
˚˚ i“1
‹
‹
˚˚
.
.
lim ˚˚
‹ ` ipϑn q‹ “ 0 in probability.
.
nÑ8 ˝˝
‚
‚
řn
pnq
1 B
,
ϑ̃q
L
pX
α
∇
n
i,n
|
i
i“1
ϑ̃“ϑ
Bϑ̃
d

d,n

Typically, i is the Fisher information matrix with respect to the distribution function Gp¨, ϑ̃q, ϑ̃ P
m
Θ. Put ℓn px, ϑ̃q :“ i´1 pϑ̃q∇Ln px, ϑ̃q, px, ϑ̃q P R ˆ Θ, and note that
n
? ÿ
?
pnq
npϑ̂pnq
n
αi,n ℓn pXi , ϑn q ` oP p1q as n Ñ 8.
n ´ ϑn q “

ş

i“1

ş
Further, assume for all ϑ̃ P Θ ℓn px, ϑ̃qGn pdx, ϑ̃q and ℓn px, ϑ̃qℓ1n px, ϑ̃qGn pdx, ϑ̃q exist and
have finite components and
ż
ż
ℓn px, ϑ̃qGn pdx, ϑ̃q “ ℓn px, ϑ̃qhn px, ϑ̃qµpdxq “ 0 for all ϑ̃ P Θ.
(3.7)
pnq

(3.7) is a typical condition related to maximum-likelihood. Thus, the estimator ϑ̂n has the
structure required in (C5). Now, assume P pXi P ty P Rm ; hn py, ϑ̃q ą 0uq “ 1, i “ 1, . . . , n.
Analogous,
n
n
ÿ
?
? ÿ
B
B
αi,n
n
npϑ̂n ´ ϑq
αi,n ∇1
Ln pXi , ϑ̃q|ϑ̃“ϑ `
Ln pXi , ϑ̃q|
ϑ̃“ϑj,n
B ϑ̃j
B ϑ̃j
i“1
i“1
n
? ÿ
B
“ n
Ln pXi , ϑ̃q|ϑ̃“ϑ̂ , j “ 1, . . . , d,
αi,n
n
B ϑ̃j
i“1
where ϑj,n is on the line between ϑ̂n and ϑ, j “ 1, . . . , d, and
˛
¨ řn
1 B
i“1 αi,n ∇ Bϑ̃1 Ln pXi , ϑ̃q|ϑ̃“ϑ
1,n ‹
n
˚
? ÿ
‹?
˚
..
n
∇Ln pXi , ϑ̃q|ϑ̃“ϑ ` ˚
‹ npϑ̂n ´ ϑq
.
‚
˝řn
i“1
1 B
i“1 αi,n ∇ Bϑ̃ Ln pXi , ϑ̃q|
d

?
“ n

n
ÿ

i“1

αi,n ∇Ln pXi , ϑ̃q|ϑ̃“ϑ̂ .
n

12

ϑ̃“ϑd,n

Suppose
lim

nÑ8

and

n
? ÿ
n
αi,n ∇Ln pXi , ϑ̃q|ϑ̃“ϑ̂ “ 0 in probability
n

i“1

˛
αi,n ∇1 Bϑ̃B Ln pXi , ϑ̃q|
ϑ̃“ϑ1,n
1
‹
˚
‹
˚
..
lim ˚
‹ “ ´ipϑq in probability,
.
nÑ8 ˝ř
‚
n
1 B
i“1 αi,n ∇ Bϑ̃ Ln pXi , ϑ̃q|
¨ řn

i“1

ϑ̃“ϑd,n

d

too. Regard that Fn “ Gn p¨, ϑq. Consequently,

n
?
? ÿ
npϑ̂n ´ ϑq “ n
αi,n ℓn pXi , ϑq ` oP p1q as n Ñ 8.
i“1

ş

ş
Assume for all i “ 1, . . . , n, ℓn px, ϑqFi pdxq and ℓn px, ϑqℓ1n px, ϑqFi pdxq exist and have
finite components. In fact,
ż
ż
ℓn px, ϑqFn pdxq “ ℓn px, ϑqGn pdx, ϑq “ 0.
Therefore, the estimator ϑ̂n has the structure required in (C3), too.
? řn
pnq
Lemma 3. Assume (C1), (C2), (C5) and (C6). Then, n i“1 αi,n ℓn pXi , ϑn q converges
in distribution to a centered d-dimensional normal distribution as n Ñ 8 and
lim ϑ̂pnq
n “ ϑ in probability.

nÑ8

m

Let Upq pϑq “ pU pq px, ϑq; x P R q be a Gaussian process with expectation function identically
equal to zero, covariance function
´ `
˘
cpq px, y, ϑq :“κ G minpx, yq, ϑ ´ Gpx, ϑqGpy, ϑq
¯
m
´ g 1 px, ϑqwpq py, ϑq ´ g 1 py, ϑqwpq px, ϑq ` g 1 px, ϑqv pq pϑqgpy, ϑq , x, y P R ,

(3.8)

and a.s. uniformly ρ-continuous sample paths. This process is a modification of the process
Upϑq.
Theorem 5. Assume (C1), (C2), (C5) and (C6). Then,
d

pnq
pq
Upnq
n pϑ̂n q ÝÑ U pϑq as n Ñ 8.

Corollary 6. Assume (C1), (C2), (C5) and (C6). Then,
ż
` pq
˘2
d
d
U px, ϑq Gpdx, ϑq as n Ñ 8.
ÝÑ
KSnpnq ÝÑ sup |U pq px, ϑq| and CvMpnq
n
m

xPR

The following result implies that the test (3.5) is of asymptotically exact size α.
Corollary 7. Suppose the existence of a ϑ P Θ with Fn “ Gn p¨, ϑq for n sufficiently large and
(C1) - (C4) hold for this parameter. In addition, assume (C5) and (C6) hold for arbitrary
sequences pϑn qnPN with ϑn P Θ for all n P N and limnÑ8 ϑn “ ϑ and v pq pϑq “ vpϑq and
m
wpq px, ϑq “ wpx, ϑq for all x P R . Moreover, suppose the restriction of the covariance
m
m
function (3.6) to the diagonal of R ˆ R does not vanish everywhere or Gp¨, ϑq-almost
everywhere. Then,
lim P pKSn ě cn;1´α q “ α or lim P pCvMn ě dn;1´α q “ α.

nÑ8

nÑ8

13

3.2

Limit results unter alternatives

In order to establish limit results under alternatives, the weak convergence in probability of
the parameter estimator is still required. Consider the following condition.
(C7) There exists a ϑ P Θ with limnÑ8 ϑ̂n “ ϑ in probability.
Theorem 6. Assume (C7) and the conditions (C1) and (C2) hold for the parameter ϑ given
by (C7). Then,
ˆ
˙
1
@ε ą 0 : P ? KSn ě sup |F pxq ´ Gpx, ϑq| ´ ε ` op p1q ÝÑ 1 as n Ñ 8,
m
n
xPR
and
ˆ

1
@ε ą 0 : P
CvMn ě
n

ż

˙
`
˘2
F pxq ´ Gpx, ϑq Gpdx, ϑq ´ ε ` op p1q ÝÑ 1 as n Ñ 8.

The following corollary is useful for consistency considerations with respect to the test (3.5).
Corollary 8. Assume (C7) and suppose that the conditions (C1) and (C2) hold for the
parameter ϑ given by (C7). Moreover, assume for ϑ given by (C7), (C5) and (C6) hold
for arbitrary sequences pϑn qnPN with ϑn P Θ for all n P N and limnÑ8 ϑn “ ϑ. Suppose
F ´ Gp¨, ϑq does not vanish everywhere or Gp¨, ϑq-almost everywhere and the restriction of
m
m
the covariance function (3.8) to the diagonal of R ˆ R does not vanish everywhere or
Gp¨, ϑq-almost everywhere. Then,
lim P pKSn ě cn;1´α q “ 1 or lim P pCvMn ě dn;1´α q “ 1.

nÑ8

4

nÑ8

Testing hypotheses formulated by Hadamard differentiable functionals

Now, consider a general two sample situation. All results can be simply extended to more
than two samples or modified to the one sample case. Regard the definitions at the beginning
of Section 2. The second sample comes from a sequence of independent but not necessarily
identically distributed Rs -valued random vectors V1 , V2 , . . . , s P N. Assume that X1 , X2 , . . .
and V1 , V2 , . . . are independent. For i P N, let Gi be the distribution function of Vi defined
s
s
on R and let G be an uniformly continuous distribution function defined on R with
s

lim Gi “ G uniformly on R .

iÑ8

In addition, let r P N be a second sample size and assume the sample sizes increasing such
that
n
lim
“ η P p0, 8q,
(4.1)
min N Ñ8 r
where N :“ tn, ru. Furthermore, let β1,r , . . . , βr,r be some additional weights, in particular
real numbers, with
βi,r ě 0, i “ 1, . . . , r,

r
ÿ

i“1

βi,r “ 1, lim

rÑ8

r
ÿ
?
2
βi,r
“ τ P r1, 8q.
r max βi,r “ 0, lim r
rÑ8

1ďiďr

i“1

(4.2)

Define the mixture distribution
Gr pvq :“

r
ÿ

i“1

s

βi,r Gi pvq, v P R ,
14

based on the distribution functions G1 , . . . , Gr of V1 , . . . , Vr and the weighted empirical
distribution function
n
ÿ
s
βi,r IpVi ď vq, v P R ,
Ĝr pvq :“
i“1

based on V1 , . . . , Vr

The aim is to formulate a general hypothesis with the help of two functionals. Therefor,
l
l
denote by DpR q the set of all distribution functions defined on R , l P N. Consider two
m
s
u
m
s
u
maps T : DpR q ˆ DpR q Ñ DpR q, Q : DpR q ˆ DpR q Ñ DpR q, u P N, and assume this
u
m
s
maps are continuous with respect to the supremum metrics on DpR q and DpR q ˆ DpR q.
Suppose F1 , . . . , Fn and G1 , . . . , Gr are unknown, the maps T and Q as well as α1,n , . . . , αn,n
and β1,r , . . . , βr,r are known and that the user has to verify the hypothesis
HN : T pFn , Gr q “ QpFn , Gr q

(4.3)

on the basis of the observations X1 , . . . , Xn and V1 , . . . , Vr .
m

Another functional is required for the construction of the testing procedure. Let j : DpR qˆ
s
m
s
DpR q Ñ DpR q ˆ DpR q be a continuous map with respect to the supremum metric on
m
s
DpR q ˆ DpR q such that
T ˝j “Q˝j
and

m

s

jpKq “ K for all K P DpR q ˆ DpR q with T pKq “ QpKq.
m

s

The map j could be a projection on the subspace of DpR qˆ DpR q given by the hypothesis.
m
Define ĴN :“ jpF̂n , Ĝr q and write ĴN “ pĴN,m , ĴN,s q, where ĴN,m has realizations in DpR q
s
and ĴN,s has realizations in DpR q. Put J :“ jpF, Gq and write J “ pJm , Js q, where
m
s
Jm P DpR q and Js P DpR q.
u

Define a process UN :“ pUN pzq; z P R q by
˘
? `
u
UN pzq :“ 4 nr T pF̂n , Ĝr qpzq ´ QpF̂n , Ĝr qpzq , z P R ,

and consider the Kolmogorov-Smirnov type statistic
ˇ
ˇ
?
KSN :“ sup |UN pzq| “ 4 nr sup ˇT pF̂n , Ĝr qpzq ´ QpF̂n , Ĝr qpzqˇ
zPR

u

zPR

(4.4)

(4.5)

u

as well as the Cramér-von-Mises type statistic
ż
ż
2
2
CvMN : “ UN
pzqT pĴN qpdzq “ UN
pzqQpĴN qpdzq
ż
`
˘2
?
“ nr
T pF̂n , Ĝr qpzq ´ QpF̂n , Ĝr qpzq T pĴN qpdzq.

(4.6)

For simplicity, assume T pF̂n , Ĝr q, QpF̂n , Ĝr q and T pĴN q are step functions with countable
vertices. This guarantees measurability of suprema and integrals in this section.
In general, the statistics (4.5) and (4.6) are not distribution free, neither in the case of
T pFn , Gr q “ QpFn , Gr q. In order to approximate the distribution of the statistics KSN and
CvMN if T pFn , Gr q “ QpFn , Gr q, a Monte-Carlo procedure is suggested. Therefor, simulate
independently observations with joint distribution function
r
n
ź
ź
`
˘
ĴN,s pvi q,
ĴN,m pxi q
px1 , . . . , xn q, pv1 , . . . , vr q ÞÝÑ
i“1

i“1

`
˘
m
s
px1 , . . . , xn q, pv1 , . . . , vr q P ˆni“1 R ˆ ˆri“1 R .
15

Determine a significance level α P p0, 1q. Denote by cN ;1´α the p1 ´ αq-quantile of the
distribution of KSN and by dN ;1´α the p1 ´ αq-quantile of the distribution of CvMN if
pF1 , . . . , Fn q “ pĴN,m , . . . , ĴN,m q and pG1 , . . . , Gr q “ pĴN,s , . . . , ĴN,s q. The calculation procedure for practice is described above. Then, testing procedure
“Reject HN , iff KSN ě cN ;1´α ” or testing procedure “Reject HN , iff CvMN ě dN ;1´α ”
(4.7)
is suggested. In the case of αi,n “ n1 , i “ 1, . . . , n, and βi,r “ r1 , i “ 1, . . . , r, testing
procedure (4.7), in particular the statistic (4.5) or (4.6), is invariant under transformation
of the data from the set Tn,m ˆ Tr,s , where Tn,m and Tr,s are defined analogous to (2.8).

4.1

Limit results under the hypothesis

The asymptotic results in this section based on a smoothness condition on the functionals T
m
s
u
and Q. Call a map T : DpR q ˆ DpR q Ñ DpR q uniformly Hadamard differentiable in K P
m
s
m
s
u
DpR q ˆ DpR q, iff there exists a linear map dT pKq : ℓ8 pR q ˆ ℓ8 pR q Ñ ℓ8 pR q such that
u
m
s
dT pKq is continuous with respect to the supremum metrics on ℓ8 pR q and ℓ8 pR qˆℓ8 pR q
and
˘
1`
u
T pKn ` tn Ln q ´ T pKn q “ dT pKqpLq uniformly on R
lim
nÑ8 tn
m

s

for all sequences pKn qnPN with Kn P DpR q ˆ DpR q for all n P N and limnÑ8 Kn “ K
m
s
m
s
uniformly on R ˆ R , for all sequences pLn qnPN with Ln P ℓ8 pR q ˆ ℓ8 pR q for all n P N
m
s
m
s
8
8
and the existence of a L P ℓ pR q ˆ ℓ pR q with limnÑ8 Ln “ L uniformly on R ˆ R and
all sequences ptn qnPN with tn P p0, 8q for all n P N and limnÑ8 tn “ 0, where pKn ` tn Ln q P
m
s
DpR q ˆ DpR q for all n P N. dT pKq is called Hadamard derivative in K. Concepts of
differentiability of functionals and applications in statistics are discussed by v. Mises [36],
Shapiro [30], van der Vaart [34] and Ren and Sen [24], [25], [26].
In order to determine the limit distribution of the process UN under the hypothesis, let
m
Mm :“ pMm pxq; x P R q be a Gaussian process with expectation function identically equal
to zero, covariance function
´ `
¯
˘
m
cm px, yq :“ κη Jm minpx, yq ´ Jm pxqJm pyq , x, y P R ,
m

and a.s. uniformly continuous sample paths with respect to the metric ρ on R defined
s
in (2.9). In addition, let Ms :“ pMs pvq; v P R q be a Gaussian process with expectation
function identically equal to zero, covariance function
´ `
¯
˘
s
cs pv, wq :“ τ η Js minpv, wq ´ Js pvqJs pwq , v, w P R ,
s

and a.s. uniformly continuous sample paths with respect to a metric on R defined analogous
to the metric ρ in (2.9). Let Mm and Ms be independent and put M :“ pMm , Ms q and
u
M “ pMm , Ms q. Finally, let U :“ pU pzq; z P R q be a process defined by
u

U pzq :“ dT pJqpM qpzq ´ dQpJqpM qpzq, z P R ,
if T and Q are uniformly Hadamard differentiable in J with Hadamard derivative dT pJq
and dQpJq, respectively,
Theorem 7. Assume T pFn , Gr q “ QpFn , Gr q for min N sufficiently large. Suppose T and
Q are uniformly Hadamard differentiable in pF, Gq with Hadamard derivative dT pF, Gq and
dQpF, Gq, respectively. Then, J “ pF, Gq and
d

UN ÝÑ U as min N Ñ 8.

16

Corollary 9. Assume T pFn , Gr q “ QpFn , Gr q for min N sufficiently large. Suppose T and
Q are uniformly Hadamard differentiable in pF, Gq with Hadamard derivative dT pF, Gq and
dQpF, Gq, respectively. Then,
ż
d
d
KSN ÝÑ sup |U pzq| and CvMN ÝÑ U 2 pzqT pF, Gqpdzq as min N Ñ 8.
zPR

u

In order to show that testing procedure (4.7) works asymptotically, it is necessary to study
the asymptotic distribution of the statistics (4.5) and (4.6) if pF1 , . . . , Fn q “ pĴN,m , . . . , ĴN,m q
and pG1 , . . . , Gr q “ pĴN,s , . . . , ĴN,s q, too.
m

s

Well, for an arbitrary JN P DpR q ˆ DpR q with T pJN q “ QpJN q and limnÑ8 JN “ J
m
s
m
s
uniformly on DpR q ˆ DpR q, JN “ pJN,m , JN,s q, JN,m P DpR q, JN,s P DpR q, let
pN q
pN q
X1 , X2 , . . . be a sequence of independent and identically distributed random vectors
pN q
pN q
with underlying distribution function JN,m and let V1 , V2 , . . . be a sequence of independent and identically distributed random vectors with underlying distribution function
pN q
pN q
pN q
pN q
JN,s . Assume that X1 , X2 , . . . and V1 , V2 , . . . are independent. In addition, let
u
pN q
pN q
pN q
pN q
UN “ pUN pzq; z P R q be the process (4.4) based on the random vectors X1 , . . . , Xn
pN q
pN q
pN q
pN q
and V1 , . . . , Vr
and denote by KSN and CvMN the statistics (4.5) and (4.6) based
on this random vectors.
Theorem 8. Suppose T and Q are uniformly Hadamard differentiable in J with Hadamard
derivative dT pJq and dQpJq, respectively. Assume J is uniformly continuous. Then,
pN q

UN

d

ÝÑ U as min N Ñ 8.

Corollary 10. Suppose T and Q are uniformly Hadamard differentiable in J with Hadamard
derivative dT pJq and dQpJq, respectively. Assume J is uniformly continuous. Then,
ż
pN q d
pN q d
KSN ÝÑ sup |U pzq| and CvMN ÝÑ U 2 pzqT pJqpdzq as min N Ñ 8.
zPR

u

The following result implies that the test (4.7) is a test of asymptotically exact size α.
Corollary 11. Assume T pFn , Gr q “ QpFn , Gr q for min N sufficiently large. Suppose T and
Q are uniformly Hadamard differentiable in pF, Gq with Hadamard derivative
dT pF, Gq and
ş
dQpF, Gq, respectively, and the distribution function of supzPRu |U pzq| or U 2 pzqT pF, Gqpdzq
is strictly increasing on the non-negative half-line. Then,
lim

min N Ñ8

4.2

P pKSN ě cN ;1´α q “ α or

lim

min N Ñ8

P pCvMN ě dN ;1´α q “ α.

Limit results under alternatives

The limit behavior in probability of the test statistics (4.5) and (4.6) is given in the following
result. It holds under the hypotheses as well as under alternatives.
Theorem 9. It is
ˆ
˙
1
@ε ą 0 : P ?
KSN ě sup |T pF, Gqpzq ´ QpF, Gqpzq| ´ ε ` op p1q ÝÑ 1 as min N Ñ 8,
4
u
nr
zPR
and

ˆ
˙
ż
`
˘2
1
@ε ą 0 :P ? CvMN ě
T pF, Gqpzq ´ QpF, Gqpzq T pJqpdzq ´ ε ` op p1q ÝÑ 1
nr
as min N Ñ 8.

17

The following corollary is important for consistency considerations with respect to the test
(4.7).
Corollary 12. Suppose T and Q are uniformly Hadamard differentiable in J with Hadamard
derivative dT pJq and dQpJq, respectively.
ş Assume J is uniformly continuous and the distribution function of supzPRu |U pzq| or U 2 pzqT pJqpdzq is strictly increasing on the nonnegative half-line. Moreover, assume T pF, Gq ´ QpF, Gq does not vanish everywhere or
T pJq-almost everywhere. Then,
lim

min N Ñ8

4.3

P pKSN ě cN ;1´α q “ 1 or

lim

min N Ñ8

P pCvMN ě dN ;1´α q “ 1.

Testing homogeneity

Consider the two sample case with equal dimensions m “ s “ u and equal sample sizes
n “ r. Suppose F1 , . . . , Fn and G1 , . . . , Gn are unknown, α1,n , . . . , αn,n and β1,n , . . . , βn,n
are known and that the user has to treat the hypothesis of homogeneity
Hn : Fn “ Gn

(4.8)

on the basis of the observations X1 , . . . , Xn and V1 , . . . , Vn .
m

m

m

m

m

m

In this situation, the maps T : DpR qˆDpR q Ñ DpR q and Q : DpR qˆDpR q Ñ DpR q
are given by the continuous maps
m

m

T pK1 , K2 q “ K1 , QpK1 , K2 q “ K2 , K “ pK1 , K2 q P DpR q ˆ DpR q.
Let K “ pK1 , K2 q and L “ pL1 , L2 q be arbitrary objects from the definition of the uniformly
Hadamard differentiability given above. Now, the maps T and Q are uniformly Hadamard
differentiable in K with Hadamard derivative
dT pKqpLq “ L1 and dQpKqpLq “ L2 ,
m

m

m

m

respectively. Let j : DpR q ˆ DpR q Ñ DpR q ˆ DpR q be the continuous map
¯
´1
1
m
m
pK1 ` K2 q, pK1 ` K2 q , K “ pK1 , K2 q P DpR q ˆ DpR q.
jpKq “
2
2
m

m

Obviously, it is T ˝j “ Q˝j and jpKq “ pKq for all K P DpR qˆDpR q with T pKq “ QpKq.
Thus,
˘
1`
ĴN,m “ ĴN,s “ F̂n ` Ĝn ,
2
the test statistics have the expressions
?
KSn “ n sup |F̂n pxq ´ Ĝn pxq|
(4.9)
m

xPR

as well as
CvMn “ n
and testing procedure

ż

˘
˘2 1 `
`
F̂n ` Ĝn pdxq
F̂n pxq ´ Ĝn pxq
2

(4.10)

“Reject Hn , iff KSn ě cn;1´α ” or testing procedure “Reject Hn , iff CvMn ě dn;1´α ” (4.11)
is suggested, where cn;1´α and dn;1´α are calculated under pF1 , . . . , Fn q “ pG1 , . . . , Gn q “
pĴN,m , . . . , ĴN,m q. If αi,n “ βi,n , i “ 1, . . . , n, testing procedure (4.11), in particular the
statistic (4.9) or (4.10), is invariant under the transformation of the data
`
˘
`
˘
px1 , . . . , xn q, pv1 , . . . , vn q ÞÝÑ pv1 , . . . , vn q, px1 , . . . , xn q ,
`
˘
px1 , . . . , xn q, pv1 , . . . , vn q P ˆni“1 Rm ˆ ˆni“1 Rm .
18

Moreover, U is a Gaussian process with expectation function identically equal to zero. Moreover,
`
˘
`
˘
`
˘
`
˘
m
2
Var U pxq “ Var Mm pxq ` Var Ms pxq “ pκ ` τ q Jm pxq ´ Jm
pxq , x P R .

Because F and G are uniformly continuous, Jm is uniformly continuous, too, and the map
m
x ÞÑ VarpU pxqq, x P R , does not vanish everywhere şor T pJq-almost everywhere. For that
reason, the distribution function of supxPRm |U pxq| or U 2 pxqT pJqpdxq is strictly increasing
on the non-negative half-line.
Remark 6.
a) Modify the model introduced in Section 1 in the following way. Add
independent and identically distributed real valued random variables U1 , U2 , . . . with
unknown underlying distribution and independent but not necessarily identically distributed real valued random variables W1 , W2 , . . . with distribution LpWi q “ LpZi q,
i P N. Now, the distribution of the error variable LpZi q and the error function ei are
possibly unknown for all i P N. Assume the random variables Y1 , Y2 , . . . , Z1 , Z2 , . . . ,
U1 , U2 , . . . and W1 , W2 , . . . are independent. Suppose the user has to treat the testing
problem of homogeneity
H : LpY1 q “ LpU1 q, K : LpY1 q ‰ LpU1 q
on the basis of the observations e1 pY1 , Z1 q, . . . , en pYn , Zn q and e1 pU1 , W1 q, . . . ,
en pUn , Wn q. Putting αi,n “ βi,n “ n1 , Xi “ ei pYi , Zi q and Vi “ ei pUi , Wi q, i “
1, . . . , n, the test (4.11) is applicable to this testing problem. If the distribution function of epU1 , Zq is continuous, too, it follows from the results in this section that the
test is of asymptotically exact size α and consistent.
b) The test (4.11) is applicable to the hypothesis
@i P t1, . . . , nu : Fi “ Gi .
Putting αi,n “ βi,n “

1
n,

i “ 1, . . . , n, the test (4.11) is also applicable to the hypothesis

Dπn : t1, . . . , nu Ñ t1, . . . , nu bijective @i P t1, . . . , nu : Fi “ Gπn piq .
It follows from the results in this section that the test is of asymptotically exact size α
and consistent with respect to suitable alternatives to this hypotheses.
c) Assume Fi “ F and Gi “ G, i P N. Putting αi,n “ βi,n “
hypothesis (4.8) is a common hypothesis of homogeneity

1
n,

i “ 1, . . . , n, the

H : F “ G,
the test statistic (4.9) or (4.10) is a common Kolmogorov-Smirnov or Cramér-vonMises statistic for testing the hypothesis of homogeneity and the test (4.11) is a common Kolmogorov-Smirnov or Cramér-von-Mises test for testing the hypothesis of homogeneity in a multivariate setting. In this sense, a generalization of the identically
distributed case is considered.
d) If αi,n “ βi,n “ n1 , i “ 1, . . . , n, the test statistic (4.9) or (4.10) has the form of
a common Kolmogorov-Smirnov or Cramér-von-Mises type statistic for testing the
hypothesis of homogeneity and the test (4.11) hasthe form of a common KolmogorovSmirnov or Cramér-von-Mises type test for testing the hypothesis of homogeneity in
the identically distributed case. I.e., the results in this section imply that the common
Kolmogorov-Smirnov or Cramér-von-Mises type testing procedure for testing the hypothesis of homogeneity is applicable if the data come from different distributions and
the mentioned conditions are fulfilled.

19

4.4

Testing central symmetry

In order to apply the general results above to the hypothesis of central symmetry, another
sample is needed. Let U1 , U2 , . . . be a sequence of independent but not necessarily identically
distributed random vectors with values in Ru . For i P N, the random vector Ui has the
u
distribution function Ci defined on R . Assume the existence of an uniformly continuous
u
distribution function C defined on R with
u

lim Ci “ C uniformly on R .

iÑ8

u

For a given distribution function K defined on R with related probability measure µ on
u
u
u
B , i.e., Kpzq “ µpr´8u , zsq, u P R , write K ` :“ K and K ´ pzq :“ µpr´z, 8u sq, z P R .
u
u
u
B denotes the Borel σ-field on R and rz, ws :“ ˆui“1 rzi , wi s, z, w P R , z “ pz1 , . . . , zu q1 ,
w “ pw1 , . . . , wu q1 , if zi ď wi for i “ 1, . . . , u, and rz, ws :“ H else. Moreover, 8l :“
l
p8, . . . , 8q1 P R , l P N. Clearly, K ´ is a distribution function. A distribution function K `
or a probability measure µ is called centrally symmetric, iff K ` “ K ´ or µpr´8u , zsq “
u
µpr´z, 8u sq for all z P R . A random vector ξ with values in Ru has a centrally symmetric
distribution function iff ξ and ´ξ have the same distribution. Concepts of symmetry in
multivariate settings are presented by Serfling [29].
For i P N, denote by Ci` “ Ci and Ci´ the distribution functions of Ui and ´Ui , respectively.
Suppose C1 , . . . , Cn are unknown, α1,n , . . . , αn,n are known and that the user has to treat
the hypothesis
n
n
ÿ
ÿ
Hn :
αi,n Ci` “
αi,n Ci´
(4.12)
i“1

i“1

on the basis of the observations U1 , . . . , Un . Regarding
P p´Ui ď zq “ 1 ´
u

z P R , i P N,

u
ÿ

p´1qk`1

k“1

ÿ

1ďj1 ă¨¨¨ăjk ďu

P pUi,j1 ă ´zj1 , . . . , Ui,jk ă ´zjk q,

it is obvious that the hypothesis (4.12) isřequivalent to the hypothesis of central symmetry
n
about the mixture distribution Cn pzq :“ i“1 αi,n Ci pzq, z P R,
´
Hn : C`
n “ Cn .

In order to apply the general results above, consider the random vectors
˙
˙ ˆ
ˆ
U2
U1
,...
,
´U2
´U1

(4.13)

with values in R2u . Because
P pUi ď z, ´Ui ď wq
u
ÿ
“P pUi ď zq ´
p´1qk`1
k“1

u

z, w P R , i P N,

ÿ

1ďj1 ă¨¨¨ăjk ďu

P pUi,j1 ă ´wj1 , . . . , Ui,jk ă ´wjk , Ui ď zq, (4.14)

the uniformly convergence of the sequence of distribution functions pCi qiPN to the uniformly
continuous distribution function C implies that the sequence of distribution functions of
the random vectors (4.13) has those limit properties, too. For that reason, the sequence
of distribution functions of the random vectors (4.13) fulfills the model assumptions in the
general model described above.
Well, consider the one sample case with dimension m “ 2u and the sequence of random
U2
U1
q, . . . with values in Rm . Then, the hypothesis p4.12q is
q, X2 “ p ´U
vectors X1 “ p ´U
2
1
20

equivalent to the hypothesis of homogeneity of the first u-dimensional marginal distribution
and the last u-dimensional marginal distribution about the mixture distribution Fn .
m

u

m

u

In this situation, the maps T : DpR q Ñ DpR q and Q : DpR q Ñ DpR q are given by the
continuous maps
u

m

T pKqpzq “ Kp 8zu q, QpKqpzq “ Kp 8zu q, z P R , K P DpR q.
Let K and L be arbitrary objects from the definition of the uniformly Hadamard differentiability given above. Now, the maps T and Q are uniformly Hadamard differentiable in K
with Hadamard derivative
u

dT pKqpLqpzq “ Lp 8zu q and dQpKqpLqpzq “ Lp 8zu q, z P R ,
m

m

respectively. Let j : DpR q Ñ DpR q be the continuous map
jpKqp wz q “

˘
1`
u
m
Kp wz q ` Kp wz q , z, w P R , K P DpR q.
2
m

It is T ˝ j “ Q ˝ j and jpKq “ K for all K P DpR q with T pKq “ QpKq. Thus,
ĴN,m p wz q “
and

˘
1`
u
F̂n p wz q ` F̂n p wz q , z, w P R ,
2

˘
`
˘
1`
u
´
T ĴN,m pzq “ Ĉ`
n pzq ` Ĉn pzq , z P R ,
2

where
Ĉ`
n pzq :“

n
ÿ

i“1

αi,n IpUi ď zq, Ĉ´
n pzq :“

n
ÿ

i“1

The test statistics have the expressions
?
´
KSn “ n sup |C`
n pzq ´ Cn pzq|
zPR

and
CvMn “ n

ż

u

` `
˘2 1 ` `
˘
Cn pzq ´ C´
Ĉ ` Ĉ´
n pzq
n pdzq.
2 n

In the special case of m “ 1 and αi,n “
[7]. Now, testing procedure

1
n,

u

αi,n Ip´Ui ď zq, z P R .

(4.15)

(4.16)

i “ 1, . . . , n, the statistic (4.15) is considered in

“Reject Hn , iff KSn ě cn;1´α ” or testing procedure “Reject Hn , iff CvMn ě dn;1´α ” (4.17)
is suggested, where cn;1´α and dn;1´α are calculated under pF1 , . . . , Fn q “ pĴN,m, . . . , ĴN,m q.
Testing procedure (4.17), in particular the statistic (4.15) or (4.16), is invariant under the
transformation of the data
pz1 , . . . , zn q ÞÝÑ p´z1 , . . . , ´zn q, pz1 , . . . , zn q P ˆni“1 Ru .
Moreover, U is a Gaussian process with expectation function identically equal to zero and
`
˘
`
˘
`
˘
˘
`
Var U pzq “ Var Mm p 8zu q ` Var Mm p 8zu q ´ 2Cov Mm p 8zu q, Mm p 8zu q “ 2κJp 8zu q,
u

zPR .

Because F is uniformly continuous, J is uniformly continuous, too, and the map z ÞÑ
u
VarpU pzqq, z P R , does not vanish everywhereşor T pJq-almost everywhere. For that reason,
the distribution function of supzPRu |U pzq| or U 2 pzqT pJqpdzq is strictly increasing on the
non-negative half-line.

21

Remark 7.
a) Modify the model introduced in Section 1 in the following way. Assume
the error variable Zi has a centrally symmetric distribution function for all i P N.
Furthermore, the distribution LpZi q is possibly unknown for all i P N. Suppose the
user has to treat the testing problem of central symmetry
H : LpY1 q “ Lp´Y1 q, K : LpY1 q ‰ Lp´Y1 q
on the basis of the observations Y1 `Z1 , . . . , Yn `Zn . Putting αi,n “ n1 and Ui “ Yi `Zi ,
i “ 1, . . . , n, the test (4.17) is applicable to this testing problem. It follows from the
results in this section that the test is of asymptotically exact size α and consistent.
b) The test (4.17) is applicable to the hypothesis
@i P t1, . . . , nu : Ci` “ Ci´ .
Putting αi,n “

1
n,

i “ 1, . . . , n, the test (4.17) is also applicable to the hypothesis

Dπn : t1, . . . , nu Ñ t1, . . . , nu bijective @i P t1, . . . , nu : Ci` “ Cπ´n piq .
The test is of asymptotically exact size α and consistent with respect to suitable alternatives to this hypotheses. This follows from the results in this section.
c) Assume Ci “ C, i P N. Putting αi,n “
common hypothesis of central symmetry

1
n,

i “ 1, . . . , n, the hypothesis (4.12) is a

H : C ` “ C ´,
the test statistic (4.15) or (4.16) is a common Kolmogorov-Smirnov or Cramér-vonMises statistic for testing the hypothesis of central symmetry and the test (4.17) is
a common Kolmogorov-Smirnov or Cramér-von-Mises test for testing the hypothesis
of central symmetry in a multivariate setting. In this sense, a generalization of the
identically distributed case is considered.
d) If αi,n “ n1 , i “ 1, . . . , n, the test statistic (4.15) or (4.16) has the form of a common
Kolmogorov-Smirnov or Cramér-von-Mises type statistic for testing the hypothesis of
central symmetry and the test (4.17) has the form of a common Kolmogorov-Smirnov
or Cramér-von-Mises type test for testing the hypothesis of central symmetry in the
identically distributed case. I.e., the results in this section imply that the common
Kolmogorov-Smirnov or Cramér-von-Mises type testing procedure for testing the hypothesis of central symmetry is applicable if the data come from different distributions
and the mentioned conditions are fulfilled.

4.5

Testing independence

Consider the one sample case with dimension m ě 2, m “ k ` ℓ, k, ℓ P N, and that the
sequence of random vectors X1 , X2 , . . . is given by
ˆ ˙
ˆ ˙
A1
A2
X1 “
, X2 “
,...,
B1
B2
with Rk -valued random vectors A1 , A2 , . . . and Rℓ -valued random vectors B1 , B2 , . . . . For
k
ℓ
i P N, denote by FiA and FiB the distribution functions defined on R and R of Ai and Bi ,
respectively. Suppose F1 , . . . , Fn are unknown, α1,n , . . . , αn,n are known and that the user
has to verify the hypothesis of independence
k

ℓ

@pa, bq P R ˆ R : Fn p ab q “ Fn p 8aℓ qFn p 8bk q
on the basis of the observations X1 , . . . , Xn .

22

(4.18)

m

m

m

m

In this situation, the maps T : DpR q Ñ DpR q and Q : DpR q Ñ DpR q are given by the
continuous maps
k

ℓ

m

T pKqp ab q “ Kp ab q, QpKqp ab q “ Kp 8aℓ qKp 8bk q, pa, bq P R ˆ R , K P DpR q.
Let K and L be arbitrary objects from the definition of the uniformly Hadamard differentiability given above. Now, the maps T and Q are uniformly Hadamard differentiable in K
with Hadamard derivative
k

ℓ

dT pKqpLqp ab q “ Lp ab q and dQpKqpLqp ab q “ Kp 8aℓ qLp 8bk q ` Kp 8bk qLp 8aℓ q, pa, bq P R ˆ R ,
m

m

respectively. Let j : DpR q Ñ DpR q be the continuous map
k

ℓ

m

jpKqp ab q “ Kp 8aℓ qKp 8bk q, pa, bq P R ˆ R , K P DpR q.
m

Obviously, T ˝ j “ Q ˝ j and jpKq “ K for all K P DpR q with T pKq “ QpKq. Thus,
k

ℓ

B
ĴN,mp ab q “ F̂n p 8aℓ qF̂n p 8bk q “ F̂A
n paqF̂n pbq, pa, bq P R ˆ R ,

where
F̂A
n paq :“

n
ÿ

i“1

αi,n IpAi ď aq, F̂B
n pbq :“

n
ÿ

i“1

Now, the test statistics have the expressions
?
B
KSn “ n
|F̂n p ab q ´ F̂A
sup
n paqF̂n pbq|
k

pa,bqPR ˆR

and
CvMn “ n

ż ż

In addition, testing procedure

k

ℓ

αi,n IpBi ď bq, pa, bq P R ˆ R .

(4.19)

ℓ

˘2 A
`
B
B
F̂n p ab q ´ F̂A
n paqF̂n pbq F̂n pdaqF̂n pdbq.

(4.20)

“Reject Hn , iff KSn ě cn;1´α ” or testing procedure “Reject Hn , iff CvMn ě dn;1´α ” (4.21)
is suggested, where cn;1´α and dn;1´α are calculated under pF1 , . . . , Fn q “ pĴN,m, . . . , ĴN,m q.
If k “ ℓ, testing procedure (4.21), in particular the statistic (4.19) or (4.20), is invariant
under the transformation of the data
`
˘
`
˘ `
˘
pa1 , b1 q, . . . , pan , bn q ÞÝÑ pb1 , a1 q, . . . , pbn , an q , pa1 , b1 q, . . . , pan , bn q P ˆni“1 pRk ˆ Rk q.

Moreover, U is a Gaussian process with expectation function identically equal to zero. Moreover, a calculation yields
˘
˘
`
`
`
˘
`
˘
Var U p ab q “Var Mm p ab q ` J 2 p 8aℓ qVar Mm p 8bk q ` J 2 p 8bk qVar Mm p 8aℓ q
`
˘
`
˘
´ 2Jp 8aℓ qCov Mm p ab q, Mm p 8bk q ´ 2Jp 8bk qCov Mm p ab q, Mm p 8aℓ q
`
˘
` 2Jp 8aℓ qJp 8bk qCov Mm p 8bk q, Mm p 8aℓ q
`
˘
`
˘
k
ℓ
“Jp 8aℓ q 1 ´ Jp 8aℓ q Jp 8bk q 1 ´ Jp 8bk q , pa, bq P R ˆ R .
Because F is uniformly continuous, J is uniformly continuous, too, and the map p ab q ÞÑ
k
ℓ
VarpU p ab qq, pa, bq P R ˆR , does not vanish everywhere or T pJq-almost
everywhere. For that
ş
reason, the distribution function of suppa,bqPRk ˆRl |U p ab q| or U 2 p ab qT pJqpdp ab qq is strictly
increasing on the non-negative half-line.

Remark 8.
a) Modify the model introduced in Section 1 in the following way. Replace the
sequence of random variables Y1 , Y2 , . . . with a sequence of independent and identically
distributed R ˆ R-valued random vectors pY1 , W1 q, pY2 , W2 q and assume this sequence
is still independent of Z1 , Z2 , . . . . Now, the distribution of the error variable LpZi q
23

and the error function ei are possibly unknown for all i P N. Suppose the user has to
treat the testing problem of independence
H : LpY1 , W1 q “ LpY1 q b LpW1 q, K : LpY1 , W1 q ‰ LpY1 q b LpW1 q
on the basis of the observations pe1 pY1 , Z1 q, W1 q, . . . , pen pYn , Zn q, Wn q. Putting
αi,n “ n1 and Xi “ pei pYi , Zi q, Wi q, i “ 1, . . . , n, the test (4.21) is applicable to this
testing problem. If the distribution function of pei pYi , Zi q, Wi q converges uniformly on
R ˆ R to the distribution function of pepY1 , Zq, W1 q and the distribution function of
pepY1 , Zq, W1 q is uniformly continuous, it follows from the results in this section that
the test is of asymptotically exact size α and consistent.
b) Assume F1B “ ¨ ¨ ¨ “ FnB . Then, the test (4.21) is applicable to the hypothesis
k

ℓ

@i P t1, . . . , nu @pa, bq P R ˆ R : Fi p ab q “ FiA paqFiB pbq.
The test is of asymptotically exact size α and consistent with respect to suitable alternatives to this hypothesis. This follows from the results in this section.
c) Assume Fi “ F , i P N. Putting αi,n “
common hypothesis of independence
k

1
n,

i “ 1, . . . , n, the hypothesis (4.18) is a

ℓ

H : @pa, bq P R ˆ R : F p ab q “ F A paqF B pbq,
the test statistic (4.19) or (4.20) is a common Kolmogorov-Smirnov or Cramér-vonMises statistic for testing the hypothesis of independence and the test (4.21) is a common Kolmogorov-Smirnov or Cramér-von-Mises test for testing the hypothesis of independence in a multivariate setting. In this sense, a generalization of the identically
distributed case is considered.
d) If αi,n “ n1 , i “ 1, . . . , n, the test statistic (4.19) or (4.20) has the form of a common
Kolmogorov-Smirnov or Cramér-von-Mises type statistic for testing the hypothesis of
independence and the test (4.21) has the form of a common Kolmogorov-Smirnov or
Cramér-von-Mises type test for testing the hypothesis of independence in the identically
distributed case. I.e., the results in this section imply that the common KolmogorovSmirnov or Cramér-von-Mises type testing procedure for testing the hypothesis of independence is applicable if A1 , A2 , . . . come from different distributions, B1 , B2 , . . . are
identically distributed and the mentioned conditions are fulfilled.

5

Simulations

Empirical results for the probabilities of the error of the first kind and the power values
of the mentioned tests for finite sample sizes are presented. The simulation based on 1000
replications and the Monte-Carlo procedures based on 500 replications.

5.1

Goodness-of-fit with hypotheses given by a specific distribution

Assume m “ 1, αi,n “ n1 , i “ 1, . . . , n, and consider the distribution function ℓpµi , 1q of
the logistic distribution with parameters µi P R and 1, i P N. Suppose pµi qiPN is known
and there exists a µ P R with limiÑ8 µi “ µ. The upper part of Table 1 shows the
1
, 1q and Gn “
empirical error probabilities of the first kind in the case of Fi “ ℓp logpi`1q
ř
ř
n
n
1
1
1
1
1
1
?
?
ℓp logpi`1q , 1q, Fi “ ℓp i , 1q and Gn “ n i“1 ℓp i , 1q or Fi “ ℓp i , 1q and Gn “
n
ři“1
n
1
1
power values in
i“1 ℓp i , 1q, i P N. The lower part of Table 1 shows the empirical
n
řn
1
, 1q, Fi “
the case of Fi “ Lp´ 1i , 12 q (Laplace distribution) and Gn “ n1 i“1 ℓp logpi`1q
ř
n
1
1
1
1
1
, 21 q
Cp´ logpi`1q , 2 q (Cauchy distribution) and Gn “ n i“1 ℓp ?i , 1q, or Fi “ Np´ logpi`1q
ř
n
(normal distribution) and Gn “ n1 i“1 ℓp 1i , 1q, i P N.
24

Fi

řn

Gn

n “ 25
n “ 50
ř
n
“ 25
n
1
1
?
ℓp ?1i , 1q
i“1 ℓp i , 1q
n
n “ 50
n “ 25
1 řn
1
ℓp 1i , 1q
i“1 ℓp i , 1q
n
n “ 50
n “ 25
1
1 řn
1 1
Lp´ i , 2 q
i“1 ℓp logpi`1q , 1q
n
n “ 50
řn
n
“ 25
1
1
1
1
?
Cp´ logpi`1q , 2 q
i“1 ℓp i , 1q
n
n “ 50
řn
n “ 25
1
1
1
Np´ logpi`1q
, 21 q
i“1 ℓp i , 1q
n
n “ 50
1
, 1q
ℓp logpi`1q

1
n

i“1

1
, 1q
ℓp logpi`1q

α “ 0.025
KS CvM
0.033 0.033
0.026 0.026
0.032 0.032
0.026 0.026
0.030 0.030
0.023 0.023
0.510 0.510
0.640 0.640
0.780 0.780
0.732 0.732
0.805 0.805
0.750 0.750

α “ 0.05
KS CvM
0.049 0.049
0.044 0.044
0.051 0.051
0.066 0.066
0.049 0.049
0.051 0.051
0.715 0.715
0.804 0.804
0.862 0.862
0.837 0.837
0.954 0.954
0.909 0.909

α “ 0.1
KS CvM
0.097 0.097
0.096 0.096
0.105 0.105
0.106 0.106
0.102 0.102
0.109 0.109
0.879 0.879
0.928 0.928
0.931 0.931
0.913 0.913
0.991 0.991
0.991 0.991

Table 1: Simulation results for goodness-of-fit with hypotheses given by a specific distribution.

Fi

řn

Gn p¨, ϑq

n “ 25
n “ 50
n “ 25
1 řn
1
?
Expp2 ` ?1i q
i“1 Exppϑ ` i q
n
n “ 50
n “ 25
1 řn
1
Expp3 ` 1i q
i“1 Exppϑ ` i q
n
n “ 50
řn
n “ 25
1
1
, 1q n1 i“1 Exppϑ ` logpi`1q
q
Wp1 ` logp1`iq
n “ 50
řn
n “ 25
1
1
?
IGp 23 , 1 ` ?1i q
i“1 Exppϑ ` i q
n
n “ 50
řn
n “ 25
1
1
Gp 21 , 1`1 1 q
i“1 Exppϑ ` i q
n
i
n “ 50
Expp1 `

1
logpi`1q q

1
n

i“1

Exppϑ `

1
logpi`1q q

α “ 0.025
KS CvM
0.035 0.035
0.024 0.024
0.025 0.025
0.025 0.025
0.029 0.029
0.018 0.018
0.241 0.241
0.257 0.257
0.546 0.546
0.796 0.796
0.628 0.628
0.870 0.870

α “ 0.05
KS CvM
0.066 0.066
0.039 0.039
0.048 0.048
0.049 0.049
0.055 0.055
0.045 0.045
0.386 0.386
0.421 0.421
0.722 0.722
0.902 0.902
0.731 0.731
0.910 0.910

α “ 0.1
KS CvM
0.102 0.102
0.083 0.083
0.088 0.088
0.096 0.096
0.101 0.101
0.087 0.087
0.584 0.584
0.622 0.622
0.833 0.833
0.955 0.955
0.803 0.803
0.939 0.939

Table 2: Simulation results for goodness-of-fit with hypotheses given by a family of distributions.

5.2

Goodness-of-fit with hypotheses given by a family of distributions

Let m “ d “ 1, Θ “ p0, 8q, αi,n “ n1 , i “ 1, . . . , n, and consider the distribution function
Exppϑ`µi q of the exponential distribution with rate pϑ`µi q P p0, 8q, i P N. Suppose pµi qiPN
is known and there exists a µ P R with limiÑ8 µi “ µ and ϑ`µ ą 0. An estimator introduced
of
in Remark 5 is used. The upper part of Table 2 shows the empirical
řnerror probabilities
1
1
q and Gn p¨, ϑq “ n1 i“1 Exppϑ ` logpi`1q
q,
the first kind in the case of Fi “ Expp1 ` logpi`1q
řn
1
1
1
1
?
?
Fi “ Expp2 ` i q and Gn p¨, ϑq “ n i“1 Exppϑ ` i q or Fi “ Expp3 ` i q and Gn p¨, ϑq “
řn
1
1
the
i“1 Exppϑ ` i q, i P N. The lower part of Table 2 shows the empirical
n
ř power values in
1
1
, 1q (Weibull distribution) and Gn p¨, ϑq “ n1 ni“1 Exppϑ` logpi`1q
q,
case of Fi “ Wp1` logp1`iq
řn
2
1
1
1
Fi “ IGp 3 , 1 ` ?i q (inverse Gaussian distribution) and Gn p¨, ϑq “ n i“1 Exppϑ ` ?i q, or
řn
Fi “ Gp 12 , 1`1 1 q (gamma distribution) and Gn p¨, ϑq “ n1 i“1 Exppϑ ` 1i q, i P N.
i

25

Fi

Gi

n “ 25
n “ 50
n “ 25
IGp 23 , 1 ` ?1i q
IGp 32 , 1 ` ?1i q
n “ 50
n “ 25
Expp1 ` 1i q
Expp1 ` 1i q
n “ 50
n “ 25
1
1
Wp1 ` i , 1{3q
Expp1 ` i q
n “ 50
1
1
n
“ 25
1
Gp 2 , 1` ?1 q
IGp2, 1 ` ?i q
i
n “ 50
n “ 25
1
1
IGp 12 , 1 ` logpi`1q
q Wp1 ` logpi`1q
, 12 q
n “ 50
Wp1 `

1
logp1`iq , 1q

Wp1 `

1
logp1`iq , 1q

α “ 0.025
KS CvM
0.026 0.026
0.026 0.030
0.031 0.027
0.026 0.024
0.022 0.021
0.028 0.023
0.266 0.242
0.622 0.571
0.555 0.504
0.899 0.932
0.075 0.075
0.281 0.235

α “ 0.05
KS CvM
0.056 0.052
0.047 0.050
0.053 0.055
0.045 0.051
0.048 0.040
0.053 0.059
0.363 0.365
0.762 0.759
0.686 0.659
0.964 0.979
0.119 0.131
0.389 0.361

α “ 0.1
KS CvM
0.110 0.102
0.093 0.098
0.101 0.106
0.097 0.109
0.104 0.088
0.106 0.105
0.566 0.555
0.876 0.891
0.787 0.799
0.990 0.997
0.224 0.218
0.569 0.556

Table 3: Simulation results for homogeneity.

Ci
n “ 25
n “ 50
n “ 25
ℓp0, 12 ` ?1i q
n “ 50
n “ 25
Cp0, 2 ` 1i q
n “ 50
n “ 25
1
q
Np 12 , 1 ` logpi`1q
n “ 50
n “ 25
ℓp 13 , 12 ` ?1i q
n “ 50
n “ 25
Cp 32 , 2 ` 1i q
n “ 50
Np0, 1 `

1
logpi`1q q

α “ 0.025
KS CvM
0.020 0.025
0.027 0.028
0.033 0.037
0.024 0.016
0.016 0.023
0.036 0.035
0.210 0.248
0.491 0.581
0.135 0.154
0.308 0.364
0.379 0.346
0.758 0.705

α “ 0.05
KS CvM
0.038 0.045
0.054 0.053
0.055 0.063
0.045 0.044
0.035 0.046
0.059 0.059
0.320 0.374
0.620 0.699
0.201 0.230
0.421 0.468
0.493 0.443
0.848 0.800

α “ 0.1
KS CvM
0.083 0.087
0.095 0.098
0.088 0.105
0.102 0.101
0.071 0.089
0.104 0.111
0.422 0.472
0.729 0.787
0.284 0.324
0.530 0.593
0.604 0.583
0.904 0.873

Table 4: Simulation results for central symmetry.

5.3

Homogeneity

Let m “ 1 and αi,n “ βi,n “ n1 , i “ 1, . . . , n. The upper part of Table 3 shows the
1
, 1q
empirical error probabilities of the first kind in the case of Fi “ Gi “ Wp1 ` logp1`iq
2
1
(Weibull distribution), Fi “ Gi “ IGp 3 , 1 ` ?i q (inverse Gaussian distribution), or Fi “
Gi “ Expp1 ` 1i q (exponential distribution), i P N. The lower part of Table 3 shows the
empirical power values in the case of Fi “ Expp1 ` 1i q and Gi “ Wp1 ` 1i , 1{3q, Fi “
1
IGp2, 1 ` ?1i q and Gi “ Gp 12 , 1`1?1 q (gamma distribution), or Fi “ IGp 21 , 1 ` logpi`1q
q and
i

Gi “ Wp1 `

5.4

1
1
logpi`1q , 2 q,

i P N.

Central symmetry

Assume m “ 1 and αi,n “ n1 , i “ 1, . . . , n. The upper part of Table 4 shows the empirical
1
q (normal distribution),
error probabilities of the first kind in the case of Ci “ Np0, 1` logpi`1q
1
1
1
Ci “ ℓp0, 2 ` ?i q (logistic distribution), or Ci “ Cp0, 2` i q (Cauchy distribution), i P N. The
1
q,
lower part of Table 4 shows the empirical power values in the case of Ci “ Np 12 , 1 ` logpi`1q
1
1
1 1
3
?
Ci “ ℓp 3 , 2 ` i q, or Ci “ Cp 2 , 2 ` i q, i P N.

26

Fi
n “ 25
n “ 50
n
“ 25
ℓp ?1i , 1q b ℓp0, 1q
n “ 50
n “ 25
1
, 1q b Cp0, 1q
Cp logp1`iq
n “ 50
n “ 25
1 1
2
1
N2 p i p 1 q, p 1 2 qq
n “ 50
n
“ 25
1
1
2
1
t1 p ?i p 1 q, p 1 2 qq
n “ 50
n “ 25
1
1
ℓ2 p logpi`1q
, 1, logpi`1q
, 1q
n “ 50
Np 1i , 1q b Np0, 1q

α “ 0.025
KS CvM
0.047 0.027
0.035 0.024
0.040 0.023
0.034 0.018
0.052 0.028
0.034 0.016
0.487 0.548
0.725 0.851
0.664 0.678
0.985 0.987
0.965 0.990
1.000 1.000

α “ 0.05
KS CvM
0.088 0.056
0.070 0.052
0.068 0.046
0.066 0.045
0.095 0.057
0.071 0.053
0.600 0.642
0.816 0.908
0.786 0.787
0.992 0.993
0.986 0.997
1.000 1.000

α “ 0.1
KS CvM
0.170 0.115
0.137 0.105
0.134 0.083
0.119 0.104
0.160 0.110
0.130 0.100
0.720 0.730
0.895 0.945
0.871 0.870
0.998 0.997
0.997 0.999
1.000 1.000

Table 5: Simulation results for independence.

5.5

Independence

Assume m “ 2, k “ ℓ “ 1 and αi,n “ n1 , i “ 1, . . . , n. The upper part of Table 5 shows
the empirical error probabilities of the first kind in the case of Fi “ Np 1i , 1q b Np0, 1q (product of normal distributions), Fi “ ℓp ?1i , 1q b ℓp0, 1q (product of logistic distributions), or
1
, 1qbCp0, 1q (product of Cauchy distributions), i P N. The lower part of Table
Fi “ Cp logp1`iq
5 shows the empirical power values in the case of Fi “ N2 p 1i p 11 q, p 21 12 qq (bivariate normal dis1
1
, 1, logpi`1q
, 1q
tribution), Fi “ t1 p ?1i p 11 q, p 21 21 qq (bivariate t-distribution), or Fi “ ℓ2 p logpi`1q
(bivariate logistic distribution), i P N.

6

Poofs

Remark 9. Let γ1,n , . . . , γn,n be a real numbers with
γi,n ě 0, i “ 1, . . . , n, lim max γi,n “ 0, lim
nÑ8 1ďiďn

nÑ8

n
ÿ

i“1

γi,n “ γ P r1, 8q,

and let r1,n , . . . , rn,n be other real numbers such that for all n P N |ri,n | ă c P p0, 8q,
i “ 1, . . . , n, and
@ε ą 0 Diε P N Dnε P N, iε ď nε , @i ą iε @n ą nε , i ď n : |ri,n | ď ε.
Then,
lim

nÑ8

n
ÿ

i“1

γi,n ri,n “ 0.

Proof of Lemma 1. Define another triangular array η1,n , . . . , ηn,n of row-wise independent random vectors with values in X :“ Rm ˆ r0, 8q by η1,n :“ pξ1,n , nα1,n q, . . . , ηn,n :“
m
Ăn pf q; f P
Wn :“ pW
pξn,n , nαn,n q. Regard the process Wn :“ pWn pxq; x P R q as a process Ă
Vq, where
m
V :“ tf ; f : X Ñ R, f pw, uq “ uIpw ď xq, pw, uq P X , x P R u
and
Ăn pf q :“ ?1
W
n

ˆÿ
n

i“1

˙
n
ÿ
`
˘
f pηi,n q ´
E f pηi,n q , f P V.
i“1

27

It is possible to follow the argumentation in 4.2 in [40] and to check the conditions mentioned
there in order to show that the process Ă
Wn is asymptotically equicontinuous. By Dudley
[12], ttpw, uq P X ; wj ´ x ą 0u; x P Ru is a Vapnik-Čhervonenkis class, j “ 1, . . . , m. For
definition and details of Vapnik-Čhervonenkis classes, see [35]. Using Lemma 2.6.17 (i) and
m
(ii) in [35], it follows that C :“ ttpw, uq P X ; w ď xu; x P R u is a Vapnik-Čhervonenkis
class, too. Consequently, H :“ th; h : X Ñ R, hpw, uq “ Ippw, uq P Cq, pw, uq P X , C P Cu “
m
th; h : X Ñ R, hpw, uq “ Ipw ď xq, pw, uq P X , x P R u is a Vapnik-Čhervonenkis graph
class. Putting g : X Ñ R, gpw, uq :“ u, pw, uq P X , Lemma 2.16.18 (vi) in [35] implies that
g ¨ H :“ tgh; h P Hu is a Vapnik-Čhervonenkis graph class, too. Finally, V “ g ¨ H is a
Vapnik-Čhervonenkis graph class.
Let E be the distribution function of the exponential distribution with rate parameter one.
The map
ż
dpf1 , f2 q :“ |f1 pzq ´ f2 pzq|pΦ b Eqpdzq, f1 , f2 P V,
defines a metric on V and pV, dq is totally bounded. In fact,
ż
ρpx, yq “ |f1 pzq ´ f2 pzq|pΦ b Eqpdzq “ dpf1 , f2 q,

m

f1 pw, uq :“ uIpw ď xq, f2 pw, uq :“ uIpw ď yq, pw, uq P X , x, y P R .

Clearly, V is a class of functions with envelope g given above. Moreover, from Lemma 2.4
in [40], V has uniformly integrable entropy. (2.1) yields
sup
nPN

n
n
ÿ
˘
1 ÿ ` 2
E g pξi,n , nαi,n q “ sup
nα2i,n ă 8.
n i“1
nPN i“1

Remark 9 and (2.1) imply
g
f n ´
f1 ÿ
`
˘2 ¯
e
lim sup
sup
E f1 pξi,n , nαi,n q ´ f2 pξi,n , nαi,n q
n i“1
nÑ8 f1 ,f2 PV,dpf1 ,f2 qďδ
d
´
¯
`
˘
ď
sup
κ Kpxq ´ 2K minpx, yq ` Kpyq ÝÑ 0 as δ Ó 0.
m

x,yPR ,ρpx,yqďδ

Moreover,
@t ą 0 :

n
`
? ˘¯
1 ÿ ´ 2
E g pξi,n , nαi,n qI gpξi,n , nαi,n q ą nt
n i“1

ďn

n
ÿ

i“1

?
α2i,n Ip n max αj,n ą tq ÝÑ 0 as n Ñ 8.
1ďjďn

In all, it results from 4.2 in [40] that the process Ă
Wn is asymptotically equicontinuous with
respect to the metric space pV, dq, i.e.,
ˆ
˙
˚
Ă
Ă
@ε ą 0 : lim sup P
sup
|Wn pf1 q ´ Wn pf2 q| ą ε ÝÑ 0 as δ Ó 0.
nÑ8

f1 ,f2 PV,dpf1 ,f2 qďδ

It follows that

@ε ą 0 : lim sup P ˚
nÑ8

ˆ

sup

“ lim sup P ˚
nÑ8

i.e., the statement.

m

x,yPR ,ρpx,yqďδ

ˆ

|Wn pxq ´ Wn pyq| ą ε

sup
f1 ,f2 PV,dpf1 ,f2 qďδ

˙

Ăn pf1 q ´ W
Ăn pf2 q| ą ε
|W

28

˙

ÝÑ 0 as δ Ó 0,

Remark 10. Assume a sequence of distribution functions K1 , K2 , . . . defined on R
m
uniformly continuous distribution function K defined on R is given such that

m

and a

m

lim Ki “ K uniformly on R .

iÑ8

Then, Remark 9 and (2.1) imply
lim

nÑ8

n
ÿ

i“1

m

αi,n Ki “ K uniformly on R , lim n
nÑ8

as well as
lim

nÑ8

n
ÿ

i“1

m

α2i,n Ki “ κK uniformly on R ,

ˇ
ˇ n
ˇ
ˇ ÿ 2
ˇ
αi,n Ki pxqKi pyq ´ κKpxqKpyqˇˇ “ 0.
sup ˇn
m

x,yPR

i“1

Proof of Theorem 1. Consider n sufficiently large. Because Fn “ Gn ,
˘
? `
m
Un pxq “ n F̂n pxq ´ Fn pxq , x P R .

For that reason, the process Un is asymptotically equicontinuous with respect to the metric
m
m
space pR , ρq, see Lemma 1. For arbitrary p P N, x P ˆpi“1 R , x “ px1 , . . . , xp q, and
p
arbitrary a P R zt0u, put
¨ ¨
˛˛
IpXi ď x1 q ´ Fi px1 q
n
ÿ
˚ ˚
‹‹
..
s2n :“ n
α2i,n Var ˝a1 ˝
‚‚.
.
i“1

IpXi ď xp q ´ Fi pxp q

m

Remark 10 and Fn “ Gn imply limiÑ8 Fi “ G uniformly on R . Regarding Remark 10
again,
´ `
¯
ÿ
˘
aj ak κ G minpxj , xk q ´ Gpxj qGpxk q .
lim s2n “
nÑ8

1ďj,kďp

Assume without loss of generality limnÑ8 s2n ą 0. With |a|1 :“ |a1 | ` ¨ ¨ ¨ ` |ap |, it is
¨ ˇ ¨?
`
˘˛ˇ2
ˇ
nαi,n IpXi ď x1 q ´ Fi px1 q ˇˇ
n
ˇ
1 ÿ ˚
ˇ ˚
‹ˇ
..
@t ą 0 : 2
E˚ ˇa1 ˝
‚ˇ
.
sn i“1 ˝ ˇ
`
˘ ˇ
?
ˇ
nαi,n IpXi ď xp q ´ Fi pxp q ˇ
`
˘˛ˇ
˛˛
¨ˇ ¨ ?
ˇ
nαi,n IpXi ď x1 q ´ Fi px1 q ˇˇ
ˇ
˚ˇ ˚
‹ˇ a 2 ‹ ‹
..
I ˝ˇa1 ˝
‚ˇ ą sn t‚‹
.
‚
ˇ
`
˘ ˇ
?
ˇ
nαi,n IpXi ď xp q ´ Fi pxp q ˇ
n
a ¯
?
|a|2 ÿ 2 ´
ď 21 n
αi,n I |a|1 n max αj,n ą s2n t ÝÑ 0 as n Ñ 8.
1ďjďn
sn i“1

For that reason, Lindeberg’s condition is fulfilled and from Cramér-Wold device, the convergence of the finite dimensional marginal distributions of the process Un to centered multivariate normal distributions follows. Finally,
`
˘
m
lim Cov Un pxq, Un pyq “ cpx, yq, x, y P R .
nÑ8

The statement follows from Theorem 1.5.4 in [35].

Proof of Corollary 1. With Theorem 1 and the Continuous Mapping Theorem, Theorem
1.3.6 in [35]. Details are given by Baringhaus and Gaigall in the proof of Theorem 3 in
[4].

29

pnq

Proof of Theorem 2. The process Un is a process based on the triangular array
pnq
pnq
X1 , . . . , Xn of row-wise independent and identically distributed random vectors with
m
values in R . The asymptotic of this process can be treated similar to the approach related
m
to Theorem 1 without any additional conditions by using limnÑ8 Gn “ G uniformly on R .
pnq
It follows the convergence of Un to a Gaussian process with a.s. uniformly ρ-continuous
sample paths and expectation function identically equal to zero. Finally, limnÑ8 Gn “ G
m
uniformly on R yields
`
˘
m
lim Cov Unpnq pxq, Unpnq pyq “ cpx, yq, x, y P R .
nÑ8

Proof of Corollary 2. With Theorem 2 analogous to the Proof of Corollary 1.
Proof of Corollary 3. Because G is uniformly continuous, the restriction of the covarim
m
ance function (2.10) to the diagonal of R ˆ R does not vanish everywhereş or G-almost
everywhere. For that reason, the distribution function of supxPRm |U pxq| or U 2 pxqGpdxq
is strictly increasing on the non-negative half-line. For details, consider the arguments in
Remark 1 and in the Proof of Corollary 1 in [4]. Consequently, limnÑ8 cn;1´α “ c1´α ,
where c1´α is the p1 ´ αq-Quantile
of supxPRm |U pxq|, or limnÑ8 dn;1´α “ d1´α , where d1´α
ş
is the p1 ´ αq-Quantile of U 2 pxqGpdxq, see Corollary 2. Finally, the statement follows from
Corollary 1.
Proof of Theorem 3. Using triangle inequality,
1
? KSn ě sup |F pxq ´ Gpxq| ´ sup | ´ F̂n pxq ` Gn pxq ` F pxq ´ Gpxq|.
m
m
n
xPR
xPR
From Corollary 1 and Slutsky’s Theorem,
1 ?
lim sup |F̂n pxq ´ Fn pxq| “ lim ?
n sup |F̂n pxq ´ Fn pxq| “ 0 in probability.
nÑ8
nÑ8
m
m
n
xPR
xPR
For that reason,
sup | ´ F̂n pxq ` Gn pxq ` F pxq ´ Gpxq|

xPR

m

P

ď sup |F̂n pxq ´ Fn pxq| ` sup |Fn pxq ´ F pxq| ` sup |Gn pxq ´ Gpxq| ÝÑ 0 as n Ñ 8.
xPR

m

m

xPR

xPR

m

For arbitrary ε ą 0, Slutsky’s Theorem implies
ˆ
˙
1
P ? KSn ě sup |F pxq ´ Gpxq| ´ ε ` op p1q
m
n
xPR
ˆ
˙
ěP sup | ´ F̂n pxq ` Gn pxq ` F pxq ´ Gpxq| ` op p1q ď ε ÝÑ 1 as n Ñ 8.
xPR

m

This yields the first statement. To show the second statement, note that

It is

1
CvMn
n
ż
ż
ż
`
˘2
`
˘2
`
˘2
F pxq ´ Gpxq Gpdxq
F pxq ´ Gpxq Gn pdxq ´
F pxq ´ Gpxq Gpdxq `
“
ż ´
`
˘2 `
˘2 ¯
Fn pxq ´ Gn pxq ´ F pxq ´ Gpxq
`
Gn pdxq
ż ´
˘2 ¯
`
˘2 `
Fn pxq ´ Gn pxq ´ F̂n pxq ´ Gn pxq
Gn pdxq.
´
ˇż
ˇ
ż
ˇ `
ˇ
`
˘2
˘2
ˇ
lim ˇ
F pxq ´ Gpxq Gpdxqˇˇ “ 0.
F pxq ´ Gpxq Gn pdxq ´
nÑ8
30

Moreover, with Cauchy-Schwarz inequality,
ˇż ´
ˇ
ˇ
ˇ
`
˘2 `
˘2 ¯
ˇ
Fn pxq ´ Gn pxq ´ F pxq ´ Gpxq
Gn pdxqˇˇ
ˇ
dż
´`
˘ `
˘¯2
Fn pxq ´ F pxq ´ Gn pxq ´ Gpxq
Gn pdxq ÝÑ 0 as n Ñ 8.
ď2

?
Theorem 1 implies the convergence in distribution of the process defined by npFn pxq ´
m
m
analogous to the
F̂n pxqq, x P R . Gn converges uniformly on R to G. Argumentation
ş
Proof of Corollary 1 yields the convergence in distribution of n pFn pxq ´ F̂n pxqq2 Gn pdxq.
From Slutsky’s Theorem,
ż ´
ż ´
¯2
¯2
1
lim
Fn pxq ´ F̂n pxq Gn pdxq “ lim n
Fn pxq ´ F̂n pxq Gn pdxq “ 0 in probability.
nÑ8 n
nÑ8
Therefore, with Cauchy-Schwarz inequality again,
ˇż ´
ˇ
ˇ
ˇ
˘2 ¯
`
˘2 `
ˇ
Fn pxq ´ Gn pxq ´ F̂n pxq ´ Gn pxq
Gn pdxqˇˇ
ˇ
dż
´
¯2
P
ď2
Fn pxq ´ F̂n pxq Gn pdxq ÝÑ 0 as n Ñ 8.
The rest follows analogous to the proof of the first statement.
Proof of Corollary 4. It is limnÑ8 cn;1´α “ c1´α , where c1´α is the p1 ´ αq-Quantile
of
ş
supxPRm |U pxq|, or limnÑ8 dn;1´α “ d1´α , where d1´α is the p1´αq-Quantile of U 2 pxqGpdxq,
see the Proof of Corollary 3. Thus, Theorem 3 implies
lim P pKSn ě cn;1´α q
˙
ˆ
1
1
“ lim P ? KSn ě sup |F pxq ´ Gpxq| ´ sup |F pxq ´ Gpxq| ` ? cn;1´α “ 1,
nÑ8
m
m
n
n
xPR
xPR
nÑ8

or
lim P pCvMn ě dn;1´α q
ˆ
˙
ż
ż
`
˘2
`
˘2
1
1
“ lim P
F pxq ´ Gpxq Gpdxq ` dn;1´α
F pxq ´ Gpxq Gpdxq ´
CvMn ě
nÑ8
n
n
“1,
nÑ8

i.e., the statement.
Lemma 4. Assume (C1) and (C2). Then, for all sequences of parameters pϑn qnPN with
ϑn P Θ for all n P N and limnÑ8 ϑn “ ϑ,
lim sup |Gn px, ϑn q ´ Gpx, ϑq| “ 0.

nÑ8

xPR

m

Proof of Lemma 4. The statement follows with (C1), (C2), Taylor expansion and triangle
inequality.
Proof of Lemma 2. Consider n sufficiently large. The first statement will be shown firstly.
For arbitrary a P Rd zt0u, (C4) implies
lim n

nÑ8

n
ÿ

i“1

ˆ ´
ż
¯˙
α2i,n Var a1 ℓn pXi , ϑq ´ ℓn px, ϑqFi pdxq
“ a1 κvpϑqa.

31

ş
For simplicity, assume in theřfollowing calculation without loss of generality ℓn px, ϑqFi pdxq “
n
0 for all i P N. Put s2n :“ n i“1 α2i,n Varpa1 ℓn pXi , ϑqq and assume without loss of generality
limnÑ8 s2n ą 0. (C4) yields
ˆ
˙
n
´?
a ¯
1 ÿ 2
1
1
2
2t
|a
ℓ
pX
,
ϑq|
ą
n
α
E
|a
ℓ
pX
,
ϑq|
I
nα
s
i,n
n
i
n
i
i,n
n
nÑ8 s2
n i“1
ż
n
1 ÿ 2
ď lim 2 n
αi,n a1 ℓn px, ϑqℓ1n px, ϑqa
nÑ8 sn
i“1
`?
˘
I p n max αj,n q2 a1 ℓn px, ϑqℓ1n px, ϑqa ą s2n t2 Fi pdxq “ 0.

@t ą 0 : lim

1ďjďn

For
array of row-wise independent random variables
? that1 reason, the? triangular
1
nα1,n a ℓn pX?
,
ϑq,
.
.
.
,
nα
a
ℓ
pX
1
n,n
n
n , ϑq fulfills Lindeberg’s condition and from Cramérř
Wold device, n ni“1 αi,n ℓn pXi , ϑq converges in distribution to a centered d-dimensional
normal distribution as n Ñ 8. For the second statement, (C3), the first statement and
Slutsky’s theorem imply
ϑ̂n ´ ϑ “

n
ÿ

i“1

αi,n ℓn pXi , ϑq ´

ż

ℓn px, ϑqFn pdxq ` op p1q

ˆ
˙
ż
n
1 ? ÿ
P
αi,n ℓn pXi , ϑq ´ ℓn px, ϑqFi pdxq ` op p1q ÝÑ 0 as n Ñ 8,
n
“ ?
n
i“1
i.e., the statement.
Lemma 5. Put
˜
ˆ
˙¸
ż
?
1
,
Wi,n py, ϑq :“ nαi,n IpXi ď yq ´ Fi pyq ´ g py, ϑq ℓn pXi , ϑq ´ ℓn pz, ϑqFi pdzq

(6.1)

m

y P R , i P N.
Assume Fn “ Gn p¨, ϑq for n sufficiently large and (C1) - (C4). Then, for all p P N,
m
x P ˆpi“1 R , x “ px1 , . . . , xp q, and all a P Rp , a “ pa1 , . . . , ap q1 ,
ˇ ¨
˛ˇ2
ˇ
Wi,n px1 , ϑq ˇˇ
ˆ
˙
ż
ˇ
‹
ˇ
ˇ 1˚
.
2
2
2
1
..
ˇa ˝
‚ˇ ď2nαi,n |a|1 ` 2nαi,n b px, aq ℓn pXi , ϑq ´ ℓn pz, ϑqFi pdzq
ˇ
ˇ
ˇ
Wi,n pxp , ϑq ˇ
ˆ
˙1
ż
ℓn pXi , ϑq ´ ℓn pz, ϑqFi pdzq bpx, aq, i P N,

with |a|1 :“ |a1 | ` ¨ ¨ ¨ ` |ap | and

¨

˛
a1 g1 px1 , ϑq ` ¨ ¨ ¨ ` ap g1 pxp , ϑq
˚
‹
..
bpx, aq :“ ˝
‚.
.
a1 gd px1 , ϑq ` ¨ ¨ ¨ ` ap gd pxp , ϑq

Proof of Lemma 5. Define
`
˘
?
W i,n py, ϑq :“ nαi,n IpXi ď yq ´ Fi pyq ,
ˆ
˙
ż
?
m
W i,n py, ϑq :“ ´ nαi,n g 1 py, ϑq ℓn pXi , ϑq ´ ℓn pz, ϑqFi pdzq , y P R , i P N.

32

Thus, Wi,n py, ϑq “ W i,n py, ϑq ` W i,n py, ϑq. Using triangle inequality,

ˇ ¨
¨
˛ˇ2 ˇ ¨
˛
˛ˇ2
ˇ
ˇ
W i,n px1 , ϑq
Wi,n px1 , ϑq ˇˇ
W i,n px1 , ϑq ˇˇ
ˇ
ˇ
˚
‹ˇ
ˇ ˚
‹
ˇ 1˚
‹ˇ
..
..
..
ˇa ˝
‚ˇ “ ˇa1 ˝
‚ ` a1 ˝
‚ˇ
.
.
.
ˇ
ˇ
ˇ
ˇ
ˇ
ˇ
ˇ
Wi,n pxp , ϑq
W i,n pxp , ϑq
W i,n pxp , ϑq ˇ
ˇ ¨
˛ˇ˛2
¨
ˇ
W i,n px1 , ϑq ˇˇ
ˇ
‹ˇ‹
ˇ ˚
˚?
..
ď ˝ nαi,n |a|1 ` ˇa1 ˝
‚ˇ‚ .
.
ˇ
ˇ
ˇ
W i,n pxp , ϑq ˇ

Furthermore, with a simple calculation,
¨
˛
W i,n px1 , ϑq
ˆ
˙
ż
?
˚
‹
..
1
a1 ˝
“
´
nα
b
px,
aq
ℓ
pX
,
ϑq
´
ℓ
pz,
ϑqF
pdzq
, i P N.
‚
i,n
n
i
n
i
.
W i,n pxp , ϑq

Finally, the inequality py ` zq2 ď 2y 2 ` 2z 2 , y, z P R, yields the statement.

Proof of Theorem 4. Consider n sufficiently large. From Lemma 2, limnÑ8 ϑ̂n “ ϑ in
probability. Therefore, assume without loss of generality that ϑ̂n has realizations in Uϑ ,
with a set Uϑ given by (C1). Using Taylor expansion,
m

Gn px, ϑ̂n q “Gn px, ϑq ` gn1 px, ϑx,n qpϑ̂n ´ ϑq, x P R ,
where ϑx,n is on the line between ϑ̂n and ϑ. Using this and Fn “ Gn p¨, ϑq,
˘
? `
?
Un px, ϑ̂n q “ n F̂n pxq ´ Fn pxq ´ g 1 px, ϑq npϑ̂n ´ ϑq
`
˘?
m
` g 1 px, ϑq ´ gn1 px, ϑx,n q npϑ̂n ´ ϑq, x P R .
Because of (C3), this is equivalent to

ˆ
˙
ż
n
˘
? ÿ
? `
1
αi,n ℓn pXi , ϑq ´ ℓn py, ϑqFi pdyq
Un px, ϑ̂n q “ n F̂n pxq ´ Fn pxq ´ g px, ϑq n
i“1

˘?
`
m
` g 1 px, ϑq ´ gn1 px, ϑx,n q npϑ̂n ´ ϑq ´ g 1 px, ϑqop p1q, x P R .
ş
For simplicity, assume without loss of generality ℓn py, ϑqFi pdyq “ 0 for all i P N. Using
(C2), Lemma 2 and Slutsky’s Theorem,
ˇ P
ˇ`
˘?
sup ˇ g 1 px, ϑq ´ gn1 px, ϑx,n q npϑ̂n ´ ϑq ´ g 1 px, ϑqop p1qˇ ÝÑ 0 as n Ñ 8.
m

xPR

Because of Slutsky’s theorem, Example 1.4.7 in [35], it is sufficient to show the convergence
statement for the process defined by
Vn px, ϑq :“

n
˘
? `
? ÿ
m
n F̂n pxq ´ Fn pxq ´ g 1 px, ϑq n
αi,n ℓn pXi , ϑq, x P R .
i“1

?
m
The process defined by npF̂n pxq ´ Fn pxqq, x P R , is asymptotically equicontinuous with
m
respect to the metric space?pR ř, ρq, see Lemma 1. In addition, from (C2) and Lemma 2, the
m
n
process defined by g 1 px, ϑq n i“1 αi,n ℓn pXi , ϑq, x P R , is asymptotically equicontinuous
m
m
with respect to the metric space pR , ρq, too. Finally, the process pVn px, ϑq; x P R q is
m
asymptotically equicontinuous with respect to the metric space pR , ρq. Consider Wi,n p¨, ϑq
defined in (6.1), i P N. Thus,
Vn py, ϑq “

n
ÿ

i“1

m

Wi,n py, ϑq, y P R .
33

m

For arbitrary p P N, x P ˆpi“1 R , x “ px1 , . . . , xp q, and arbitrary a P Rp zt0u, put
˛˛
¨ ¨
Wi,n px1 , ϑq
n
ÿ
‹‹
˚ ˚
..
s2n :“
Var ˝a1 ˝
‚‚.
.
i“1
Wi,n pxp , ϑq

With a “ pa1 , . . . , ap q1 , (C4) yields
´ `
ÿ
˘
lim s2n “
aj ak κ G minpxj , xk q, ϑ ´ Gpxj , ϑqGpxk , ϑq
nÑ8

1ďj,kďp

¯
´ g 1 pxj , ϑqwpxk , ϑq ´ g 1 pxk , ϑqwpxj , ϑq ` g 1 pxj , ϑqvpϑqgpxk , ϑq .

Assume without loss of generality limnÑ8 s2n ą 0. With bpx, aq and |a|1 defined in Lemma
5, Lemma 5 and (C4) implies
¨ˇ ¨
˛˛
˛ˇ2 ¨ˇ ¨
˛ˇ
ˇ
ˇ
ˇ
ˇ
W
px
,
ϑq
W
px
,
ϑq
i,n
1
i,n
1
n
ˇ
ˇ
ˇ
ˇ
1 ÿ ˚
ˇ 1˚
‹ˇ ˚ˇ 1 ˚
‹ˇ a 2 ‹‹
..
..
E˚
a
@t ą 0 : 2
I
a
ˇ
ˇ
ˇ
ˇ ą sn t‚‹
˝
˝
‚
˝
‚
.
.
‚
ˇ
ˇ
ˇ
sn i“1 ˝ˇ
ˇ
ˇ
Wi,n pxp , ϑq ˇ
Wi,n pxp , ϑq ˇ
ż ˆ
n
ÿ
?
2
ď 2 |a|21 n
α2i,n I p n max αj,n q2 b1 px, aqℓn py, ϑq
1ďjďn
sn
i“1
˙
2
?
2
2
1
2t
ℓn py, ϑqbpx, aq ą sn ´ p n max αj,n q |a|1 Fi pdyq
1ďjďn
2
ż
n
2 ÿ 2
` 2n
α
b1 px, aqℓn py, ϑqℓ1n py, ϑqbpx, aq
sn i“1 i,n
ˆ
?
I p n max αj,n q2 b1 px, aqℓn py, ϑq
1ďjďn
˙
2
?
1
2t
2
2
ℓn py, ϑqbpx, aq ą sn ´ p n max αj,n q |a|1 Fi pdyq ÝÑ 0 as n Ñ 8.
1ďjďn
2
For that reason, Lindeberg’s condition is fulfilled and from Cramér-Wold device, the conm
vergence of the finite dimensional marginal distributions of the process pVn px, ϑq; x P R q
to centered multivariate normal distributions follows. Finally, (C4) yields
`
˘
m
lim Cov Vn px, ϑq, Vn py, ϑq “ cpx, y, ϑq, x, y P R ,
nÑ8

and the statement follows from Theorem 1.5.4 in [35].

Proof of Corollary 5. With Theorem 4, Lemma 2 and Lemma 4 analogous to the Proof
of Corollary 1.
Proof of Lemma 3. This follows analogous to the Proof of Lemma 2.
Lemma 6. Put

¸
˜
?
m
pnq
pnq
1
Zi,n py, ϑ̃q :“ nαi,n IpXi ď yq´Gn py, ϑ̃q´g py, ϑqℓn pXi , ϑ̃q , py, ϑ̃q P R ˆUϑ , i P N.
(6.2)
m
Assume (C1), (C2), (C5) and (C6). Then, for all p P N, x P ˆpi“1 R , x “ px1 , . . . , xp q,
and all a P Rp , a “ pa1 , . . . , ap q1 ,
ˇ ¨
˛ˇ2
ˇ
Zi,n px1 , ϑ̃q ˇˇ
ˇ
‹ˇ
ˇ 1˚
pnq
pnq
..
ˇa ˝
‚ˇ ď 2nα2i,n |a|21 ` 2nα2i,n b1 px, aqℓn pXi , ϑ̃qℓ1n pXi , ϑ̃qbpx, aq, ϑ̃ P Uϑ , i P N,
.
ˇ
ˇ
ˇ
Zi,n pxp , ϑ̃q ˇ
with bpx, aq and |a|1 defined in Lemma 5.

34

Proof of Lemma 6. The proof is analogous to the Proof of Lemma 5.
pnq

Proof of Theorem 5. Consider n sufficiently large. From Lemma 3, limnÑ8 ϑ̂n “ ϑ in
pnq
probability. Therefore, assume without loss of generality that ϑ̂n has realizations in Uϑ
and ϑn P Uϑ , with a set Uϑ given by (C1). Put
F̂pnq
n pxq :“

n
ÿ

pnq

αi,n IpXi

i“1

m

ď xq, x P R .

Using Taylor expansion,
m

1
pnq
Gn px, ϑ̂pnq
n q “Gn px, ϑn q ` gn px, ϑx,n qpϑ̂n ´ ϑn q, x P R ,
pnq

where ϑx,n is on the line between ϑ̂n and ϑn . Then,
˘
?
? ` pnq
n F̂n pxq ´ Gn px, ϑn q ´ g 1 px, ϑq npϑ̂pnq
Un px, ϑ̂pnq
n ´ ϑn q
n q “
˘?
` 1
m
pnq
1
` g px, ϑq ´ gn px, ϑx,n q npϑ̂n ´ ϑn q, x P R .
Using (C5), this is equivalent to

n
˘
? ÿ
? `
pnq
1
αi,n ℓn pXi , ϑn q
Un px, ϑ̂n q “ n F̂pnq
n pxq ´ Gn px, ϑn q ´ g px, ϑq n
i“1

`
˘?
m
1
` g 1 px, ϑq ´ gn1 px, ϑx,n q npϑ̂pnq
n ´ ϑn q ´ g px, ϑqop p1q, x P R .

(C2), Lemma 3 and Slutsky’s Theorem yield

ˇ`
ˇ P
˘?
1
ˇ
sup ˇ g 1 px, ϑq ´ gn1 px, ϑx,n q npϑ̂pnq
n ´ ϑn q ´ g px, ϑqop p1q ÝÑ 0 as n Ñ 8.

xPR

m

Because of Slutsky’s theorem, it is sufficient to show the convergence statement for the
process defined by
Vnpnq px, ϑn q :“

n
˘
? ÿ
? ` pnq
m
pnq
αi,n ℓn pXi , ϑn q, x P R .
n F̂n pxq ´ Gn px, ϑn q ´ g 1 px, ϑq n
i“1

?
m
pnq
Because limnÑ8 Gp¨, ϑn q “ Gp¨, ϑq uniformly on R , the process defined by npF̂n pxq ´
m
Gn px, ϑn qq, x P R , is asymptotically equicontinuous with respect to the metric space
m
pR , ρq, see Lemma 1. In addition, from (C2) and Lemma 3, the process defined by
? ř
m
pnq
g 1 px, ϑq n ni“1 αi,n ℓn pXi , ϑn q, x P R , is asymptotically equicontinuous with respect
m
m
pnq
to the metric space pR , ρq, too. Finally, the process pVn px, ϑn q; x P R q is asymptotm
ically equicontinuous with respect to the metric space pR , ρq. Consider Zi,n defined in
(6.2), i P N. Thus,
n
ÿ
m
Vnpnq py, ϑn q “
Zi,n py, ϑn q, y P R .
i“1

For arbitrary p P N, x P

m
ˆpi“1 R ,

x “ px1 , . . . , xp q, and arbitrary a P Rp zt0u, put
˛˛
¨ ¨
Zi,n px1 , ϑn q
n
ÿ
‹‹
˚ ˚
..
s2n :“
Var ˝a1 ˝
‚‚.
.
i“1
Zi,n pxp , ϑn q

With a “ pa1 , . . . , ap q1 , (C6) yields
´ `
ÿ
˘
lim s2n “
aj ak κ G minpxj , xk q, ϑ ´ Gpxj , ϑqGpxk , ϑq
nÑ8

1ďj,kďp

¯
´ g 1 pxj , ϑqwpq pxk , ϑq ´ g 1 pxk , ϑqwpq pxj , ϑq ` g 1 pxj , ϑqv pq pϑqgpxk , ϑq .
35

Assume without loss of generality limnÑ8 s2n ą 0. With bpx, aq and |a|1 defined in Lemma
5, Lemma 6 and (C6) yields
¨ˇ ¨
˛˛
˛ˇ2 ¨ˇ ¨
˛ˇ
ˇ
ˇ
ˇ
ˇ
Z
px
,
ϑ
q
Z
px
,
ϑ
q
i,n
1
n
i,n
1
n
n
ˇ
ˇ
ˇ
ˇ
1 ÿ ˚
ˇ 1˚
‹ˇ ˚ˇ 1 ˚
‹ˇ a 2 ‹‹
..
..
@t ą 0 : 2
E˚
a
I
a
ˇ
ˇ
ˇ
ˇ ą sn t‚‹
˝
˝
‚
˝
‚
.
.
‚
ˇ
ˇ
ˇ
sn i“1 ˝ˇ
ˇ
ˇ
Zi,n pxp , ϑn q ˇ
Zi,n pxp , ϑn q ˇ
ż ´
n
?
2 2 ÿ 2
|a|
n
α
I p n max αj,n q2 b1 px, aqℓn py, ϑn q
1
i,n
2
1ďjďn
sn
i“1
¯
2
?
t
ℓ1n py, ϑn qbpx, aq ą s2n ´ p n max αj,n q2 |a|21 Gn pdy, ϑn q
1ďjďn
2
ż
n
2 ÿ 2
α
b1 px, aqℓn py, ϑn qℓ1n py, ϑn qbpx, aq
` 2n
sn i“1 i,n
´?
I p n max αj,n q2 b1 px, aqℓn py, ϑn q
ď

1ďjďn

ℓ1n py, ϑn qbpx, aq ą s2n

¯
?
t2
´ p n max αj,n q2 |a|21 Gn pdy, ϑn q ÝÑ 0 as n Ñ 8.
1ďjďn
2

For that reason, Lindeberg’s condition is fulfilled and from Cramér-Wold device, the converm
pnq
gence of the finite dimensional marginal distributions of the process pVn px, ϑn q; x P R q
to centered multivariate normal distributions follows. Finally, Lemma 4 and (C6) yields
`
˘
m
lim Cov Vnpnq px, ϑn q, Vnpnq py, ϑn q “ cpq px, y, ϑq, x, y P R ,
nÑ8

and the statement follows from Theorem 1.5.4 in [35].

Proof of Corollary 6. With Theorem 5, Lemma 4 and Lemma 3 analogous to the Proof
of Corollary 1.
Proof of Corollary 7. Because the restriction of the covariance function (3.6) to the diagm
m
onal of R ˆ R does not vanishş everywhere or Gp¨, ϑq-almost everywhere, the distribution
function of supxPRm |U px, ϑq| or U 2 px, ϑqGpdx, ϑq is strictly increasing on the non-negative
half-line. In addition, the assumptions yield that the covariance functions (3.6) and (3.8)
coincide. For that reason, limnÑ8 cn;1´α “ c1´α in probability, where c1´α is the p1 ´ αqQuantile of supxPRmş |U px, ϑq|, or limnÑ8 dn;1´α “ d1´α in probability, where d1´α is the
p1 ´ αq-Quantile of U 2 px, ϑqGpdx, ϑq, see Corollary 6 and Lemma 2. The statement follows
from Corollary 5.
Proof of Theorem 6. This follows with arguments similar to the arguments in the Proof
of Theorem 3 by using Lemma 4 and (C7).
Proof of Corollary 8. Because the restriction of the covariance function (3.8) to the diagm
m
onal of R ˆ R does not vanish everywhere
or Gp¨, ϑq-almost everywhere, the distribution
ş
function of supxPRm |U pq px, ϑq| or pU pq px, ϑqq2 Gpdx, ϑq is strictly increasing on the nonnegative half-line. For that reason, limnÑ8 cn;1´α “ c1´α in probability, where c1´α is the
p1 ´ αq-Quantile of supxPRm |U pq px, ϑq|, or limnÑ8 dn;1´α “ d1´α in probability, where d1´α
ş
is the p1 ´ αq-Quantile of pU pq px, ϑqq2 Gpdx, ϑq, see Corollary 6 and regard (C7). With
Theorem 6, the rest follows similar to the Proof of Corollary 4.
Lemma 7. It is
lim

min N Ñ8

sup
m

px,vqPR ˆR

s

ˇˇ`
˘ `
˘ˇˇ
ˇˇ F̂n pxq, Ĝr pvq ´ F pxq, Gpvq ˇˇ “ 0 in probability.

36

Proof
of Lemma 7. Applying Corollary
1 yields the convergence in distribution of
?
?
n supxPRm |F̂n pxq ´ Fn pxq| and r supvPRs |Ĝr pvq ´ Gr pvq|. Then, from Slutsky’s TheP

P

orem, supxPRm |F̂n pxq ´ F pxq| ÝÑ 0 as n Ñ 8 as well as supvPRs |Ĝr pvq ´ Gpvq| ÝÑ 0 as
r Ñ 8.

Proof of Theorem 7. Consider n sufficiently large. Denote by MN,m :“ pMN,m pxq; x P
m
R q the process
˘
? `
m
MN,m pxq :“ 4 nr F̂n pxq ´ Fn pxq , x P R ,
s

denote by MN,s :“ pMN,s pvq; v P R q the process
˘
? `
s
MN,s pvq :“ 4 nr Ĝr pvq ´ Gr pvq , v P R ,

?
put MN :“ pMN,m , MN,s q and MN :“ pMN,m , MN,s q. Thus, MN “ 4 nrppF̂n , Ĝr q ´
pFn , Gr qq. Because the processes MN,m and MN,s are independent, Theorem 1 and (4.1)
yield the convergence in distribution
d

MN ÝÑ M as min N Ñ 8
on the related product space, see [35]. T pFn , Gr q “ QpFn , Gr q and the continuity of T and
Q imply T pF, Gq “ QpF, Gq and therefore J “ jpF, Gq “ pF, Gq. Regarding Theorem 1.10.4
m
s
in [35], one can assume a.s. limmin N Ñ8 MN “ M uniformly on R ˆ R without loss of
generality. Because T pFn , Gr q “ QpFn , Gr q,
ˆ ´
˙
¯
?
1
UN “ 4 nr T pFn , Gr q ` ?
M
´
T
pF
,
G
q
´ dT pF, GqpM q
N
n
r
4
nr
¸
˜
˙
ˆ ´
¯
?
1
4
MN ´ QpFn , Gr q ´ dQpF, GqpM q
´
nr Q pFn , Gr q ` ?
4
nr
` dT pF, GqpM q ´ dQpF, GqpM q.

u

The assumtions on T and Q yield a.s. limmin N Ñ8 UN “ U uniformly on R .
Proof of Corollary 9. The continuity of T and j and Lemma 7 imply
lim sup |T pĴN qpzq ´ T pJqpzq| “ 0 in probability.

nÑ8

zPR

u

Theorem 7 yields J “ pF, Gq. The rest follows with Theorem 7 analogous to the Proof of
Corollary 1.
Proof of Theorem 8. Put
q
F̂pN
n pxq :“

n
ÿ

q
ĜpN
r pvq :“
pN q

and ĴN

pN q

pN q

:“ jpF̂n , Ĝr

pN q

pN q

r
ÿ

βi,r IpVi

pN q

ď vq, v P R ,

pN q

pN q

i“1

s

m

q. Denote by MN,m :“ pMN,m pxq; x P R q the process

MN,mpxq :“
pN q

ď xq, x P R ,

i“1

as well as

m

pN q

αi,n IpXi

s

˘
`
?
m
q
4
nr F̂pN
n pxq ´ JN,m pxq , x P R ,

by MN,s :“ pMN,s pvq; v P R q the process
pN q

MN,s pvq :“

˘
`
?
s
q
4
nr ĜpN
r pvq ´ JN,s pvq , v P R ,
37

(6.3)

(6.4)

pN q

put MN

pN q

pN q

pN q

pN q

:“ pMN,m , MN,s q and MN

pN q
MN,s

pN q
MN,m

pN q

pN q

:“ pMN,m , MN,s q. Thus, MN

“

?
4

pN q

pN q

nrppF̂n , Ĝr

q´

based on triangular arrays of row-wise independent
and
JN q. The processes
and identically distributed random vectors. The asymptotic of this processes can be treated
similar to the approach related to Theorem 1 without any additional conditions by using
m
s
limmin N Ñ8 JN,m “ Jm uniformly on R and limmin N Ñ8 JN,s “ Js uniformly on R , the
pN q
pN q
uniformly continuity of Jm and Js and (4.1). Because the processes MN,m and MN,s are
independent, it follows that
pN q

MN

d

ÝÑ M as min N Ñ 8
pN q

on the related product space. Again, one can assume a.s. limmin N Ñ8 MN “ M uniformly
m
s
on R ˆ R without loss of generality. Because T pJN q “ QpJN q,
ˆ ´
˙
¯
?
1
pN q
pN q
4
UN “ nr T JN ` ?
M
´ T pJN q ´ dT pJqpM q
4
nr N
¸
˜
˙
ˆ ´
¯
?
1
pN q
4
M
´ QpJN q ´ dQpJqpM q
nr Q JN ` ?
´
4
nr N
` dT pJqpM q ´ dQpJqpM q.

pN q

The assumtions on T , Q and JN yield a.s. limnÑ8 UN
statement follows.
pN q

u

“ U uniformly on R and the

pN q

Proof of Corollary 10. The asymptotic of F̂n and Ĝr defined in (6.3) and (6.4) can
be treated similar to the approach related to Corollary 1 and Lemma 7 without any addim
tional conditions by using limmin N Ñ8 JN,m “ Jm uniformly on R , limmin N Ñ8 JN,s “ Js
s
uniformly on R and the uniformly continuity of Jm and Js . The continuity of T and j yield
lim

min N Ñ8

pN q

sup |T pĴN qpzq ´ T pJqpzq| “ 0 in probability,

zPR

u

pN q

where ĴN is defined in the Proof of Theorem 8. The rest follows with Theorem 8 analogous
to the Proof of Corollary 1.
Proof of Corollary 11. The proof is analogous to the Proof of Corollary 7 by applying
Corollary 9, Corollary 10, Lemma 7 and the continuity of j. Regard that J “ pF, Gq, see
Theorem 7.
Proof of Theorem 9. This follows with arguments similar to the arguments in the Proof
of Theorem 3 by using Lemma 7 and the continuity of T , Q and j.
Proof of Corollary 12. The proof is analogous to the Proof of Corollary 8 by applying
Corollary 10, Lemma 7, the continuity of j and Theorem 9.

References
[1] Alexander, K. S. (1987). The Central Limit Theorem for Weighted Empirical Processes
Indexed by Sets. J. Multivariate Anal. 22, 313-339.
[2] Anderson, T. W., Darling, D. A. (1952). Asymptotic Theory of Certain ”Goodness of
Fit” Criteria Based on Stochastic Processes. Ann. Math. Statist. 23, 193–212.
[3] Andrews, D. W. K. (1997). A Conditional Kolmogorov Test. Econometrica 65, 1097–
1128.
38

[4] Baringhaus, L., Gaigall, D. (2015). On an independence test approach to the goodnessof-fit problem. J. Multivariate Anal. 140, 193–208.
[5] Blum, J. R., Kiefer, J., Rosenblatt, M. (1961). Distribution Free Tests of Independence
Based on the Sample Distribution Function. Ann. Math. Statist. 32, 485–498.
[6] Butler, C. C. (1969). A Test for Symmetry Using the Sample Distribution Function.
Ann. Math. Statist. 40, 2209–2210. 25, 287–299.
[7] Chatterjee, S. K., Sen, P. K. (1973). On Kolmogorov-Smirnov-type tests for symmetry.
Ann. Inst. Statist. Math. 25, 287–299.
[8] Chicheportiche, R., Bouchaud, J. (2011). Goodness-of-fit tests with dependent observations. J. Stat. Mech. 9, P09003.
[9] Collings, B. J., Margolin, B. H. (1985). Testing Goodness of Fit for the Poisson Assumption When Observations Are Not Identically Distributed. J. Amer. Statist. Ass. 80,
411–418.
[10] Conover, W. J., Cox, D. D., Martz, H. F. (1997). A chi-square goodness-of-fit test
for non-identically distributed random variables: with application to empirical Bayes.
IAEA/INIS 30, LA-UR–97-4825.
[11] Delaigle, A., Meister, A. (2008). Density estimation with heteroscedastic error. Bernoulli
14, No. 2, 562–579.
[12] Dudley, R. M. (1984). A course on empirical processes. Lecture Notes in Math. 1097,
1–142. Springer, New York.
[13] Eicker, F. (1967). Limit theorems for regressions with unequal and dependent errors.
Proc. Fifth Berkeley Symp. on Math. Statist. and Prob., Vol. 1 (Univ. of Calif. Press,
1967), 59–82.
[14] Görnitz, N, Porbadnigk, A., Binder, A., Sannelli, C., Braun, M., Müller, K., Kloft,
M. (2014). Learning and evaluation in presence of non-iid label noise. AISTATS 2014,
293–302.
[15] Ghosh, A., Basu, A. (2015). Robust Bounded Influence Tests for Independent NonHomogeneous Observations. ArXiv:1502.01106.
[16] Hu, F., Zidek, J. V. (2002). The Weighted Likelihood. Can. J. of Statist. Vol. 30, 347–
371.
[17] Hüsler, J. (1987). Test Based on Independent, but Non-identically Distributed Random
Variables. Biometrical J. 29, 423–428.
[18] Kosorok, M. C. A. (2003). Bootstraps of sums of independent but not identically distributed stochastic processes. J. Multivariate Anal. 84, 299–318.
[19] Kosorok, M. C. A. (2008). Introduction to Empirical Processes and Semiparametric
Inference. Springer, New York.
[20] Kuljus, K., Zwanzig, S. (2013). Asymptotic properties of a rank estimate in linear
regression with symmetric non-identically distributed errors. Statistics: A Journal of
Theoretical and Applied Statistics 47, 1160–1183.
[21] Lu, J., Yang, W., Wang, J., Li, B., Dou, W. (2013). An independent but not identically distributed bit error model for heavy-tailed wireless channels. Journal of Zhejiang
University SCIENCE C 14, 42–29.
[22] Müller-Funk, U., Witting, H. (1995). Mathematische Statistik II - Asymptotische Statistik: Parametrische Modelle und nicht-parametrische Funktionale. Teubner, Leipzig.

39

[23] Pollard, D. (1990). Empirical Processes: Theory and Applications. NSF-CBMS Regional Conference Series in Probability and Statistics 2.
[24] Ren, J., Sen, P. K. (1991). On hadamard differentiability of extended statistical functional. J. Multivariate Anal. 39, 30–43.
[25] Ren, J., Sen, P. K. (1995). Hadamard Differentiability on Dr0, 1sp. J. Multivariate Anal.
55, 14–28.
[26] Ren, J., Sen, P. K. (2001). Second Order Hadamard Differentiability in Statistical
Applications. J. Multivariate Anal. 77, 187–228.
[27] Rosenblatt, M. (1952). Limit Theorems Associated with Variants of the Von Mises
Statistic. Ann. Math. Statist. 23, 617–623.
[28] Sarkadi, K. (1967). On testing for normality. Berkeley Symp. on Math. Statist. and
Prob. Proc. Fifth Berkeley Symp. on Math. Statist. and Prob. 1 (Univ. of Calif. Press),
373–387.
[29] Serfling, R. (2006). Multivariate symmetry and asymmetry. In Encyclopedia of Statistical Sciences, 2nd Ed. (Kotz, Balakrishnan, Read and Vidakovic eds.). Wiley, New York,
5338–5345.
[30] Shapiro, A. (1990). On concepts of directional differentiability. J. of Optimization Theory and Appl. 66, 477–487.
[31] Shorack, G. R. (1979). The weighted empirical process of row independent random
variables with arbitrary distribution functions. Statistica Neerlandica 33, 169–229.
[32] Shorack, G. R., Wellner, J. A. (2009). Empirical Processes with Applications to Statistics. Society for Industrial and Applied Mathematics.
[33] Stute, W.,Manteiga, W. G., Quindimil, M. P. (1993). Bootstrap based goodness-of-fittests. Metrika 40, 243–256.
[34] van der Vaart (1991). Efficiency and Hadamard Differentiability. Scand. J. of Statist.
18, 63–75.
[35] van der Vaart, A., Wellner, J. A. (1996). Weak convergence and empirical processes.
Springer, New York.
[36] v. Mises, R. (1947). On the Asymptotic Distribution of Differentiable Statistical Functions. Ann. Math. Statist. 18, 309–348.
[37] Wang, X., van Eeden, C., Zidek, J. V. (2004). Asymptotic properties of maximum
weighted likelihood estimators. J. of Statist. Planning and Inference 119, 37 -54.
[38] Wang, X., Zidek, J. V. (2005). Selecting likelihood weights by cross-validation. Ann.
Statist. Vol. 33, 463–500.
[39] Weiss, M. S. (1978). Modification of the Kolmogorov-Smirnov Statistic for Use with
Correlated Data. J. Amer. Statist. Ass. 73, 872–875.
[40] Ziegler, K. (1997). Functional Central Limit Theorems for Triangular Arrays of
Function-Indexed Processes under Uniformly Integrable Entropy Conditions. J. Multivariate Anal. 62, 233–272.

40

