Computation of Power Loss in Likelihood
Ratio Tests for Probability Densities
Extended by Lehmann Alternatives
Lucas Gallindo Martins Soares
Departamento de Estatı́stica e Informática
Universidade Federal Rural de Pernambuco, Brasil
lucasgallindo@gmail.com
Abstract
We compute the loss of power in likelihood ratio tests when we test
the original parameter of a probability density extended by the first
Lehmann alternative.
1 Distributions Generated by Lehmann Alter-
natives
In the context of parametric models for lifetime data, [Gupta et alii 1998]
disseminated the study of distributions generated by Lehmann alterna-
tives, cumulative distributions that take one of the following forms:
G1 (x, λ) = [F (x)]
λ or G2 (x, λ) = 1− [1− F (x)]
λ (1)
where F (x) is any cumulative distribution and λ > 0. In the present note,
we are going to call both G distributions generated distributions or extended
distributions. It is easy to see that for integer values of λ, G1 and G2 are,
respectively, the distribution of the maximum and the minimum of a sample
of size λ, the support of the two distribution is the same of F , and that the
associated density functions are
g1 (x, λ) = λf(x) [F (x)]
λ−1 and g2 (x, λ) = λf(x) [1− F (x)]
λ−1 (2)
where f(x) is the density function associated with F . Suppose that we
generate a distribution G(x|λ) based on the distribution F (x), and want to
generate another distribution G′(x|λ, λ′) repeating the process; It is easy
to see that the distribution G′ will be the same as G, for the new param-
eter of the distribution, λλ′ may be summarized as a single one. This has
the interesting side effect that the standard uniparametric exponential dis-
tribution may be seen as a distribution generated by the second Lehmann
alternative from the distribution F (x) = 1− e−x.
To compute the moments of distribution generated by Lehmann alter-
natives, we use the change of variables u = F (x) in the expression
xkλf(x) [F (x)]λ−1 dx (3)
yielding
λQk(u)uλ−1du = EBeta(λ,1) [Q(u)] (4)
where Q(u) = F−1(u) is the quantile function. This integral is equivalent to
the expectancy of Q(u) with respect to a Beta distribution with parameters
α = λ, β = 1. The same reasoning can be used to show that, for the second
Lehmann alternative, E
= EBeta(1,λ) [Q(u)].
Using the log-likelihood functions
G1 (x, λ) = n ln (λ) +
ln f (xj) + (λ− 1)
lnF (xj) (5)
G2 (x, λ) = n ln (λ) +
ln f (xj) + (λ− 1)
ln [1− F (xj)] (6)
we see that the maximum likelihood estimators to the parameter λ have
the forms
λ̂ = −
j=1 lnF (xj)
and λ̂ = −
j=1 ln [1− F (xj)]
The existing literature about distributions generated by Lehmann al-
ternatives concerns mostly distributions defined on the interval (0,∞) or in
the real line, with the paper by [Nadarajah and Kotz 2006] being the more
complete review of progresses and the paper [Nadarajah 2006] being an in-
teresting application of the concepts developed outside the original proposal
by [Gupta et alii 1998], which was to analyze lifetime data. In the present
paper, we are concerned with some information theoretical quantities of the
first extension. These are not the only papers dealing with the subject, but
a complete list with comments would be a paper on its own.
2 Kullback-Leibler Divergence
Given two probability density functions, the quantity defined as
DKL (f |g) =
f(x) ln
dx (8)
is called Kullback-Leibler Divergence (abbreviated DKL) after the authors
of the classical paper [Kullback and Leibler 1951]. Very often, this quantity
is used as a measure of distance between two probability density functions,
even though it is not a metric; This divergence measure clearly is greater
or equal than zero, with zero occurring only and only if f = g, but it is
not symmetric, so DKL (f |g) 6= DKL (g|f), and it does not obey the triangle
inequality also.
Rewriting equation (8), we get∫
f(x) ln
f(x) ln(f(x))− f(x) ln(g(x))dx (9)
= Ef [ln(f(X))]− Ef [ln(g(X))] (10)
where Ef [h(X)] is the expectation of the random variable h(X) with respect
to the probability density f . Since DKL (f |g) is greater than zero, we have
Ef [ln(f(X))] > Ef [ln(g(X))] (11)
We will now show that maximizing the likelihood is equivalent to mini-
mize DKL (f |e), where e is the empirical distribution function. Calculating
DKL (f |e) we arrive at
DKL (f |e) = Ef [ln(f(X))]−
ln (f(xj , θ)) (12)
where the rightmost term is the empirical log-likelihood multiplied by a
constant. So, maximizing the rightmost term we minimize the whole diver-
gence; Then the process of maximizing the likelihood is equivalent to mi-
nimizing the divergence between the empirical density and the parametric
model. This result is very common in the related literature, and is shown in
full detail on sources like [Eguchi and Copas 1998], which gives an accessi-
ble but rather compact deduction of properties of methods based on Like-
lihood Functions using DKL. In the next (and last) section we draw freely
from a result shown in the [Eguchi and Copas 1998] paper that states that
DKL might be used to measure the loss of power in likelihood ratio tests
when the distribution under the alternative hypothesis is mis-specified.
3 Wrong Specification of Reference Distribu-
tion and Loss of Power in Likelihood Ratio
Tests
Suppose we have data from a probability distribution H(x|θ, λ), and want
to test the hypothesis that (θ = θ0, λ = λ0). The usual log-likelihood ratio is
expressed as
Λ(λ0, θ0) =
`(λ̂, θ̂)
`(λ0, θ0)
where the notation ξ̂ is used for the unrestricted maximum likelihood es-
timative of the parameter ξ. Suppose we are not willing to (or not able
to) compute `(λ̂, θ̂) because the estimative of the parameter λ is trouble-
some and decide to approximate the likelihood ratio statistic using `(λ1, θ̃)
instead of the likelihood under the alternative hypothesis, where θ̃ is the
maximum likelihood estimator of θ given that λ = λ1. We have then the
relation
Λ(x) ≈
`(λ1, θ̃)
`(λ0, θ0)
A result by [Eguchi and Copas 1998], section 3, states that the test statistic
generated this way is less powerful than the usual one, with the loss in the
power equal to
∆Power = DKL
f(x|λ̂, θ̂), f(x|λ1, θ̃)
In the present paper, we are concerned with the case where the data
follows a distribution extended with the first Lehmann alternative, where
the original distribution is such that F = F (x|θ) for a parameter θ. The null
hypothesis will be of the form
H0 : θ = θ0, λ = 1 (16)
against a alternative hypothesis
HA : θ 6= θ0, λ 6= 1 (17)
If we erroneously consider that the data doesn’t come from a extended dis-
tribution G(x|λ, θ), but from a population that follows the original F (x|θ)
distribution, we can say that we are approximating the log-likelihood un-
der the alternative hypothesis like in the previous discussion. In this case,
the log-likelihood will be taken under the hypothesis
HA′ : θ 6= θ0, λ = 1 (18)
which generates the following expression for the log-likelihood:
Λ(x) ≈
`(1, θ̃)
`(1, θ0)
Then we have that the test has less power than the one using the full G
distribution; The difference on the power of the tests is given by
∆Power = DKL
g(x|λ̂, θ̂)|g(x|1, θ̃)
The main point in the above discussion is that for testing hypotheses about
the ”original” parameter ξ, the tests using the extended version of distribu-
tions are always more powerful, with a considerable difference in the error
type II rate.
Expanding the equation (20) we have that
∆P = DKL
g(x|λ̂, θ̂)|g(x|1, θ̃)
g(x|λ̂, θ̂) ln
g(x|λ̂, θ̂)
g(x|1, θ̃)
dx (22)
λf(x|λ̂, θ̂)Fλ−1(x|λ̂, θ̂) ln
λf(x|λ̂, θ̂)Fλ−1(x|λ̂, θ̂)
f(x|1, θ̃)
dx (23)
λf(x|λ̂, θ̂)Fλ−1(x|λ̂, θ̂) ln
λFλ−1(x|λ̂, θ̂)
dx (24)
= lnλ+
λ(λ− 1)f(x|λ̂, θ̂)Fλ−1(x|λ̂, θ̂) ln
F (x|λ̂, θ̂)
dx (25)
Integrating by parts, we get
∆Power = lnλ+
The graphic of this function is the loss of power that we have on our test
when we the distribution of our data is one extended by the first Lehmann
alternative and we fail to notice that, and is depicted in Figure 1 for values
of λ bigger than one.
References
[Eguchi and Copas 1998] EGUCHI, S. AND COPAS, J. (2006). Interpreting
Kullback-Leibler divergence with the Neyman-Pearson lemma. Journal
of Multivariate Analysis, vol. 97, Issue 9, pages 2034-2040.
[Gupta et alii 1998] GUPTA, R. C., GUPTA, P. L. AND GUPTA, R. D. (1998).
Modeling failure time data by Lehman alternatives. Communication in
Statistics: Theory and Methods, vol. 27, pages 887-904.
[Kullback and Leibler 1951] KULLBACK, S. AND LEIBLER, R. A. (1951).
On information and sufficiency. The Annals of Mathematical Statistics,
vol. 22, Number 1, pages 79-86.
[Nadarajah and Kotz 2006] NADARAJAH, S., KOTZ, S. (2006). The Expo-
nentiated Type Distributions. Acta Applicandae Mathematicae, vol. 92,
pages 97-111.
[Nadarajah 2006] NADARAJAH, S. 2006. The exponentiated Gumbel dis-
tribution with climate application. Environmetrics, vol. 17, Number 1,
pages 13-23.
Figure 1: Loss of Power as a Function of λ, for λ > 1.
	Distributions Generated by Lehmann Alternatives
	Kullback-Leibler Divergence
	Wrong Specification of Reference Distribution and Loss of Power in Likelihood Ratio Tests

  We compute the loss of power in likelihood ratio tests when we test the
original parameter of a probability density extended by the first Lehmann
alternative.

Computation of Power Loss in Likelihood
Ratio Tests for Probability Densities
Extended by Lehmann Alternatives
Lucas Gallindo Martins Soares
Departamento de Estatı́stica e Informática
Universidade Federal Rural de Pernambuco, Brasil
lucasgallindo@gmail.com
Abstract
We compute the loss of power in likelihood ratio tests when we test
the original parameter of a probability density extended by the first
Lehmann alternative.
1 Distributions Generated by Lehmann Alter-
natives
In the context of parametric models for lifetime data, [Gupta et alii 1998]
disseminated the study of distributions generated by Lehmann alterna-
tives, cumulative distributions that take one of the following forms:
G1 (x, λ) = [F (x)]
λ or G2 (x, λ) = 1− [1− F (x)]
λ (1)
where F (x) is any cumulative distribution and λ > 0. In the present note,
we are going to call both G distributions generated distributions or extended
distributions. It is easy to see that for integer values of λ, G1 and G2 are,
respectively, the distribution of the maximum and the minimum of a sample
of size λ, the support of the two distribution is the same of F , and that the
associated density functions are
g1 (x, λ) = λf(x) [F (x)]
λ−1 and g2 (x, λ) = λf(x) [1− F (x)]
λ−1 (2)
where f(x) is the density function associated with F . Suppose that we
generate a distribution G(x|λ) based on the distribution F (x), and want to
generate another distribution G′(x|λ, λ′) repeating the process; It is easy
to see that the distribution G′ will be the same as G, for the new param-
eter of the distribution, λλ′ may be summarized as a single one. This has
the interesting side effect that the standard uniparametric exponential dis-
tribution may be seen as a distribution generated by the second Lehmann
alternative from the distribution F (x) = 1− e−x.
To compute the moments of distribution generated by Lehmann alter-
natives, we use the change of variables u = F (x) in the expression
xkλf(x) [F (x)]λ−1 dx (3)
yielding
λQk(u)uλ−1du = EBeta(λ,1) [Q(u)] (4)
where Q(u) = F−1(u) is the quantile function. This integral is equivalent to
the expectancy of Q(u) with respect to a Beta distribution with parameters
α = λ, β = 1. The same reasoning can be used to show that, for the second
Lehmann alternative, E
= EBeta(1,λ) [Q(u)].
Using the log-likelihood functions
G1 (x, λ) = n ln (λ) +
ln f (xj) + (λ− 1)
lnF (xj) (5)
G2 (x, λ) = n ln (λ) +
ln f (xj) + (λ− 1)
ln [1− F (xj)] (6)
we see that the maximum likelihood estimators to the parameter λ have
the forms
λ̂ = −
j=1 lnF (xj)
and λ̂ = −
j=1 ln [1− F (xj)]
The existing literature about distributions generated by Lehmann al-
ternatives concerns mostly distributions defined on the interval (0,∞) or in
the real line, with the paper by [Nadarajah and Kotz 2006] being the more
complete review of progresses and the paper [Nadarajah 2006] being an in-
teresting application of the concepts developed outside the original proposal
by [Gupta et alii 1998], which was to analyze lifetime data. In the present
paper, we are concerned with some information theoretical quantities of the
first extension. These are not the only papers dealing with the subject, but
a complete list with comments would be a paper on its own.
2 Kullback-Leibler Divergence
Given two probability density functions, the quantity defined as
DKL (f |g) =
f(x) ln
dx (8)
is called Kullback-Leibler Divergence (abbreviated DKL) after the authors
of the classical paper [Kullback and Leibler 1951]. Very often, this quantity
is used as a measure of distance between two probability density functions,
even though it is not a metric; This divergence measure clearly is greater
or equal than zero, with zero occurring only and only if f = g, but it is
not symmetric, so DKL (f |g) 6= DKL (g|f), and it does not obey the triangle
inequality also.
Rewriting equation (8), we get∫
f(x) ln
f(x) ln(f(x))− f(x) ln(g(x))dx (9)
= Ef [ln(f(X))]− Ef [ln(g(X))] (10)
where Ef [h(X)] is the expectation of the random variable h(X) with respect
to the probability density f . Since DKL (f |g) is greater than zero, we have
Ef [ln(f(X))] > Ef [ln(g(X))] (11)
We will now show that maximizing the likelihood is equivalent to mini-
mize DKL (f |e), where e is the empirical distribution function. Calculating
DKL (f |e) we arrive at
DKL (f |e) = Ef [ln(f(X))]−
ln (f(xj , θ)) (12)
where the rightmost term is the empirical log-likelihood multiplied by a
constant. So, maximizing the rightmost term we minimize the whole diver-
gence; Then the process of maximizing the likelihood is equivalent to mi-
nimizing the divergence between the empirical density and the parametric
model. This result is very common in the related literature, and is shown in
full detail on sources like [Eguchi and Copas 1998], which gives an accessi-
ble but rather compact deduction of properties of methods based on Like-
lihood Functions using DKL. In the next (and last) section we draw freely
from a result shown in the [Eguchi and Copas 1998] paper that states that
DKL might be used to measure the loss of power in likelihood ratio tests
when the distribution under the alternative hypothesis is mis-specified.
3 Wrong Specification of Reference Distribu-
tion and Loss of Power in Likelihood Ratio
Tests
Suppose we have data from a probability distribution H(x|θ, λ), and want
to test the hypothesis that (θ = θ0, λ = λ0). The usual log-likelihood ratio is
expressed as
Λ(λ0, θ0) =
`(λ̂, θ̂)
`(λ0, θ0)
where the notation ξ̂ is used for the unrestricted maximum likelihood es-
timative of the parameter ξ. Suppose we are not willing to (or not able
to) compute `(λ̂, θ̂) because the estimative of the parameter λ is trouble-
some and decide to approximate the likelihood ratio statistic using `(λ1, θ̃)
instead of the likelihood under the alternative hypothesis, where θ̃ is the
maximum likelihood estimator of θ given that λ = λ1. We have then the
relation
Λ(x) ≈
`(λ1, θ̃)
`(λ0, θ0)
A result by [Eguchi and Copas 1998], section 3, states that the test statistic
generated this way is less powerful than the usual one, with the loss in the
power equal to
∆Power = DKL
f(x|λ̂, θ̂), f(x|λ1, θ̃)
In the present paper, we are concerned with the case where the data
follows a distribution extended with the first Lehmann alternative, where
the original distribution is such that F = F (x|θ) for a parameter θ. The null
hypothesis will be of the form
H0 : θ = θ0, λ = 1 (16)
against a alternative hypothesis
HA : θ 6= θ0, λ 6= 1 (17)
If we erroneously consider that the data doesn’t come from a extended dis-
tribution G(x|λ, θ), but from a population that follows the original F (x|θ)
distribution, we can say that we are approximating the log-likelihood un-
der the alternative hypothesis like in the previous discussion. In this case,
the log-likelihood will be taken under the hypothesis
HA′ : θ 6= θ0, λ = 1 (18)
which generates the following expression for the log-likelihood:
Λ(x) ≈
`(1, θ̃)
`(1, θ0)
Then we have that the test has less power than the one using the full G
distribution; The difference on the power of the tests is given by
∆Power = DKL
g(x|λ̂, θ̂)|g(x|1, θ̃)
The main point in the above discussion is that for testing hypotheses about
the ”original” parameter ξ, the tests using the extended version of distribu-
tions are always more powerful, with a considerable difference in the error
type II rate.
Expanding the equation (20) we have that
∆P = DKL
g(x|λ̂, θ̂)|g(x|1, θ̃)
g(x|λ̂, θ̂) ln
g(x|λ̂, θ̂)
g(x|1, θ̃)
dx (22)
λf(x|λ̂, θ̂)Fλ−1(x|λ̂, θ̂) ln
λf(x|λ̂, θ̂)Fλ−1(x|λ̂, θ̂)
f(x|1, θ̃)
dx (23)
λf(x|λ̂, θ̂)Fλ−1(x|λ̂, θ̂) ln
λFλ−1(x|λ̂, θ̂)
dx (24)
= lnλ+
λ(λ− 1)f(x|λ̂, θ̂)Fλ−1(x|λ̂, θ̂) ln
F (x|λ̂, θ̂)
dx (25)
Integrating by parts, we get
∆Power = lnλ+
The graphic of this function is the loss of power that we have on our test
when we the distribution of our data is one extended by the first Lehmann
alternative and we fail to notice that, and is depicted in Figure 1 for values
of λ bigger than one.
References
[Eguchi and Copas 1998] EGUCHI, S. AND COPAS, J. (2006). Interpreting
Kullback-Leibler divergence with the Neyman-Pearson lemma. Journal
of Multivariate Analysis, vol. 97, Issue 9, pages 2034-2040.
[Gupta et alii 1998] GUPTA, R. C., GUPTA, P. L. AND GUPTA, R. D. (1998).
Modeling failure time data by Lehman alternatives. Communication in
Statistics: Theory and Methods, vol. 27, pages 887-904.
[Kullback and Leibler 1951] KULLBACK, S. AND LEIBLER, R. A. (1951).
On information and sufficiency. The Annals of Mathematical Statistics,
vol. 22, Number 1, pages 79-86.
[Nadarajah and Kotz 2006] NADARAJAH, S., KOTZ, S. (2006). The Expo-
nentiated Type Distributions. Acta Applicandae Mathematicae, vol. 92,
pages 97-111.
[Nadarajah 2006] NADARAJAH, S. 2006. The exponentiated Gumbel dis-
tribution with climate application. Environmetrics, vol. 17, Number 1,
pages 13-23.
Figure 1: Loss of Power as a Function of λ, for λ > 1.
	Distributions Generated by Lehmann Alternatives
	Kullback-Leibler Divergence
	Wrong Specification of Reference Distribution and Loss of Power in Likelihood Ratio Tests
