ESTIMATION OF THE KULLBACK-LEIBLER
DIVERGENCE

M Broniatowski

► To cite this version:

M Broniatowski. ESTIMATION OF THE KULLBACK-LEIBLER DIVERGENCE. Mathematical Methods of Statistics, 2003, 12 (4), pp.391-409. hal-04363067

HAL Id: hal-04363067

https://hal.science/hal-04363067v1

Submitted on 23 Dec 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>
# ESTIMATION OF THE KULLBACK-LEIBLER DIVERGENCE

M. BRONIATOWSKI

Univ. Reims, Dept. Math., Campus du Moulin de la Housse, 51100 Reims, France
and LSTA Univ. Paris 6, 175 Rue du Chevaleret, 75013 Paris, France
E-mail: mbr@ccr.jussieu.fr

The Kullback-Leibler (KL) divergence $K(\Omega, P)$ between a set $\Omega$ of probability measures (PMs) on $\mathbb{R}^d$ and some PM $P$ cannot be estimated by $K(\Omega, P_n)$ when $\Omega$ contains PM whose support is not included in the support of the empirical measure $P_n$. We propose an estimation procedure which avoids any smoothing of $P_n$. Also we provide an estimate of the derivative of the projected measure with respect to $P$. We make use of the representation of the KL divergence in terms of the Fenchel-Legendre transform of the moment generating function. As an application we introduce some new technique in parametric estimation, and also apply our results to updating problems.

Key words: Kullback-Leibler divergence; Fenchel transform; duality

2000 Mathematics Subject Classification: Primary 62G99; secondary 62B10, 62F30.

## 1. Introduction and Notation

Let $M_1$ be the set of all PMs on $\mathbb{R}^d$. For any PMs $P$ and $Q$ in $M_1$ the Kullback-Leibler divergence (or relative entropy) between $Q$ and $P$ is defined by

$$ (1.1) \qquad K(Q, P) := \begin{cases} Q \log \frac{dQ}{dP} & \text{if } Q \text{ is absolutely continuous (a.c.)} \\ \infty & \text{with respect to } P, \\ \end{cases} $$

Here and in the sequel we write $Qf$ instead of $\int f(x)dQ(x)$ whenever well-defined.

Let $\Omega$ be some subset of PMs on $\mathbb{R}^d$. We are interested in the estimation of

$$ (1.2) \qquad K(\Omega, P) := \inf_{Q \in \Omega} K(Q, P) $$
M. Broniatowski

when $\Omega$ satisfies some regularity condition with respect to $P$, and also in the estimation of the probability measure achieving (1.2).

The role of Kullback-Leibler divergence in the probability theory is well known, as it appears in a natural way in the Sanov bound for the large deviation rate of the empirical measure $P_n$, see [1]. In statistics a deep insight into the KL projection has been proposed in [2] in connection with the Gibbs Principle; it also appears in the calculation of Bahadur efficiency of tests (see [3]) and has mainly been used as a statistical criterion in the case of PM $P$ with finite support. Indeed, in this case, for large $n$, $P$ is necessarily a.c. w.r.t. $P_n$, the empirical probability measure pertaining to an i.i.d. sample $X := (X_1, \dots, X_n)$, and the divergence $K(\Omega, P)$ between $\Omega$ and $P$ may be estimated by $K(\Omega, P_n)$ when $\Omega$ is a set of PMs with the same support as $P$. In this vein a statistical decision theory has been developed, stemming from the book [4], see also [5] for estimation techniques based on KL divergence in parametric models defined on a finite set; applications in information theory and others can be found in [6]. However the constraint on the support of $P$ is a major restriction of the Kullback-Leibler divergence approach, since it seems that a projection of the empirical measure $P_n$ on some set of measures $\Omega$ can only be realized either in the finite-support case or by some preliminary smoothing of $P_n$ as suggested in [7].

In the present paper we will surmount this difficulty without any smoothing; indeed we will replace the above definition of $K(Q, P)$ for fixed $P$ by its so-called dual counterpart. This formula, developed in Section 2, is equivalent to (1.1). It is quite customary in the large deviation theory and is closely connected to the convexity properties of the KL divergence. Obviously, it determines the same values as the more classical formula (1.1). However we will see that a slight modification in this dual form of the divergence permits us to keep $K(Q, P)$ unaltered when $Q$ is a.c. w.r.t. $P$ while making it finite otherwise, at least in cases encountered in statistics for which (1.1) cannot be used, that is when $P$ is substituted by $P_n$ and $Q$ has a density with respect to the Lebesgue measure. Also as will be noticed in Section 4, the case when both $P$ and $Q$ belong to the same parametric set of PMs demonstrates the advantage of this dual form for parametric estimation. Indeed, in this case it yields quite simple $M$-estimates of the unknown parameter.

We are also interested in estimation of the KL projection defined by (1.2), namely

$$ (1.3) \qquad Q^* := \arg \inf_{Q \in \Omega} K(Q,P) $$

and

$$ (1.4) \qquad f^* := \log \frac{dQ^*}{dP} $$

based on the empirical measure $P_n$, and we will provide a number of statistical results related to the fact that the available information comes from the random sample $X$. Such problems have been addressed in [8], where $\Omega$ is convex and $P$ lies outside $\Omega$. Deterministic projection schemes with known $P$ based on both moment constraints and regularity assumptions, such as $dQ^*/dx$ lying in some class of regular functions, have been proposed in [9]. An extension of the recovery problem when projecting the uniform measure on some interval on sets of PMs defined by empirical moments has been proposed in [10].
Estimation of the Kullback-Leibler Divergence

The set $\Omega$ is generally defined through global and/or local constraints. A global constraint can be thought of as a moment constraint, for example. On the contrary, a local constraint describes the regularity we wish for the density $dQ^*/dP$. Although $P$ is unknown, we will see that in many cases we can describe a class of functions $\mathcal{F}$ that contains $\log dQ^*/dP$, under mild assumptions on $P$. Since the aim of the present paper is to introduce some general approach to estimation and testing in various contexts, we present two generic situations, which we denote (A) and (B) in the sequel. Moreover, in order to get explicit results, we will sometimes consider some restrictive kind of conditions on the various tools to be handled (typically on $\mathcal{F}$). As appears in the sequel those restrictive conditions can be much relaxed in specific settings. In [11] the present technique is extended to general $\phi$-divergences in a parametric setting in order to recover classical results of the parametric inference and to get improved confidence areas. In [12] some semi-parametric procedures are presented. Also, [13] adapts the present approach to the two-sample problem.

### 1.1. CASE (A). The projection of $P$ on sets of measures $\Lambda$ defined by moment conditions (global constraint)

$$Q\varphi = a$$

for some $\mathbb{R}^k$-valued function $\varphi$ and some $a$ in $\mathbb{R}^k$ has a peculiar structure, which we describe referring to [14] and [2] for the mathematical derivation.

Let $t_a$ be the solution of the equation

$$\mathrm{grad} \log P e^{(t, \varphi(\cdot))} = a.$$

Then

$$\log \frac{dQ^*}{dP}(x) = \langle t_a, \varphi(x) \rangle - \log P e^{(t_a, \varphi(\cdot))}$$

indicating that $Q^*$ belongs to the so-called exponential family of distributions associated with $P$.

Assume that the support of $P$ is $\mathbb{R}^d$. For $S \subset \mathbb{R}$ and $T \subset \mathbb{R}^k$, we introduce the family of functions

$$\mathcal{F} := \{ x \mapsto \langle t, \varphi(x) \rangle + s \text{ for some } t \in T \text{ and } s \in S \\ \text{such that } t_a \in T \text{ and } -\log P e^{(t_a, \varphi(\cdot))} \in S \},$$

where all functions in $\mathcal{F}$ are defined on $\mathbb{R}^d$. The class $\mathcal{F}$ is the local constraint. The infimum in (1.2) can be searched inside a smaller set than $\Lambda$, namely in the set $\mathcal{G}$ of PMs $Q$ such that $\log \frac{dQ^*}{dP} \in \mathcal{F}$ and $Q \in \Lambda$. Define

$$\Omega := \Lambda \cap \mathcal{G}.$$

Necessarily $\log \frac{dQ^*}{dP} \in \mathcal{F}$. Therefore with this definition of $\Omega$ we must estimate $\log \frac{dQ^*}{dP}$ in $\mathcal{F}$, a class of functions completely specified above.

### 1.2. CASE (B). The global constraint $\Lambda$ is defined without any assumption on the local behavior of $Q^*$. Define the local constraint assuming that $Q^*$ should
M. Broniatowski

be a.c. w.r.t. the Lebesgue measure $\lambda$ on $\mathbb{R}^d$, with density $q^*$ in some class $Q$
with prescribed regularity. Assume that $P$ is also a.c. w.r.t. $\lambda$ and has density
$p$ in some class of functions $P$. Call $\mathcal{F} := \log Q - \log P := \{\log q - \log p, q \in$
$Q, p \in P\}$ and $\mathcal{G} := \{Q \in M_1 : \log \frac{dQ}{dP} \in \mathcal{F}\}$. The constraint $\Omega$
writes also here $\Omega := \Lambda \cap \mathcal{G}$. Examples to which our method applies are: $\Lambda$ is the set of PMs on
$\mathbb{R}^+$ with decreasing failure rate (DFR) or with unimodal distribution with known
mode, and $Q$ is the class of log-convex densities on $\mathbb{R}^+$. In these examples the
constraint $\Omega$ is highly nonlinear.

The most important case is when both $p$ and $q$ belong to the same parametric
model $\{P_\theta, \theta \in \Theta\}$; let $dP = p_\alpha dx$ for some $\alpha$ in $\Theta$. Consider a subset $\Theta_0 \subset \Theta$ and
define $\Omega := \{p_\theta, \theta \in \Theta_0\}$. The class of functions $\mathcal{F}$ is then $\{\log p_\theta - \log p_\theta', \text{ for } \theta \in$
$\Theta_0$ and $\theta' \in \Theta\}$. An estimate of $K(\Omega, p_\alpha)$ yields a test statistic for the composite
null hypothesis $\alpha \in \Theta_0$. In this case $\Omega$ is not defined by linear constraints and is
by no means a convex set. Therefore the method which is developed here yields
also estimation and test procedures in the context of parametric statistics. We will
briefly sketch this in Section 4 and refer to [11] for a complete treatment.

Most results are stated in terms of the properties of the class $\mathcal{F}$. Additional remarks make clear that these properties can be stated on the classes $Q$ and $P$ when dealing with case (B).

This paper is organized as follows. Section 2 states the estimator. We first provide a brief account of the representation of the Kullback-Leibler divergence as the Fenchel-Legendre transform of some extended moment generating function. Next we explore existence conditions for the solution of problem (1.3) in the spirit of [2], along with a basic minimax result. The role of the class $\mathcal{F}$ is then enlightened, giving rise to the formal definition of the estimate. Section 3 states convergence results. In Section 4 we present statistical applications, namely, a new approach to parametric estimation and projection on $M$-type constraints. Proofs are postponed to Section 5.

For brevity, the results are stated in the case where the r.v.'s $X_i$'s are indepen-
dent. Since the methodology that we propose does not depend upon this assump-
tion, all convergence results can be adapted to classical dependent cases for which
the technical basic limit theorems exist, namely, Glivenko-Cantelli and central limit
theorems.

2. Definition of the Estimator

2.1. GLOBAL AND LOCAL CONSTRAINTS. EXISTENCE OF THE PROJECTION.
Existence results for the KL projection on $\Omega = \Lambda \cap \mathcal{G}$ can be derived under various
regularity conditions, according to the context. The following topologies on sets
of measures will be used. Apart from the weak topology on $M_1$, the $\tau$-topology
(see [15]) and the var-topology (see, e.g., [14]), we will make use of the $\tau_{\Phi}$-topology
defined as follows (see [16]). Let $\Phi$ be a set of $\mathbb{R}^d$-measurable functions $\phi : \mathbb{R}^d \to \mathbb{R}$
containing $B_b(\mathbb{R}^d)$, the set of measurable bounded functions defined on $\mathbb{R}^d$. Define

$$
\mathcal{M}_{\Phi}(\mathbb{R}^d) := \{ Q \in M_1 : \int |\phi| dQ < \infty \text{ for every } \phi \in \Phi \}.
$$

Let $\tau_{\Phi}$ denote the coarsest topology on $\mathcal{M}_{\Phi}(\mathbb{R}^d)$ such that the map $\mathcal{M}_{\Phi}(\mathbb{R}^d) \ni Q \mapsto \int \phi \, dQ$ is continuous for every $\phi \in \Phi$. If $\Phi = B_b(\mathbb{R}^d)$ then $\mathcal{M}_{\Phi}(\mathbb{R}^d) = M_1$ and
$\tau_{\Phi}$ coincides with the usual $\tau$-topology. We say that $Q$ satisfies the $\Phi$-exponential condition if for every $\phi$ in $\Phi$ and every $\alpha > 0$

$$ (2.1) \qquad Q \exp(\alpha |\phi|) < \infty. $$

Let us start with case (B). Let $\mathcal{G}$ be the set of PMs on $\mathbb{R}^d$ with density $q$ in $Q$, a subset of $L_1(\mathbb{R}^d, \lambda)$. We have

**Proposition 2.1.** Let $\Lambda$ be closed in the var-topology (resp., var-compact) in $M_1$ and let $Q$ be strongly compact (resp., strongly closed) in $L_1(\mathbb{R}^d, \lambda)$. Whenever $K(\Omega, P)$ is finite, (1.3) has a solution in $\Omega$.

**Remark 2.1.** Examples of var-closed subsets are provided by linear constraints. Most global constraints can be written in terms of some $\mathbb{R}^m$-valued function $T$ defined on $M_1$. Let $F$ be some closed subset of $\mathbb{R}^m$, then

$$ \Lambda := \{Q: T(Q) \in F\}. $$

Many functions $T$ are var-continuous and therefore induce var-closed subsets in $M_1$, see [15], Section 6, for statistically relevant examples of such functions $T$ weakly or $\tau$-continuous, and therefore var-continuous.

Using Theorem 2.1 in Csiszar [14] we also can obtain a similar result relaxing the compactness assumptions.

**Proposition 2.2.** Assume that $\Lambda$ is var-closed in $M_1$ and $Q$ is strongly closed in $L_1(\mathbb{R}^d, \lambda)$. If $\Omega$ is convex and $K(\Omega, P)$ is finite then (1.3) has a solution in $\Omega$.

The set of unimodal PMs with known mode is weakly closed in $M_1$. For such cases we state

**Proposition 2.3.** Let $\Lambda$ be a weakly closed set of PMs and let $\mathcal{G}$ be the set of PMs on $\mathbb{R}^d$ with density $q$ in $Q$, a compact subset of $L_1(\mathbb{R}^d, \lambda)$. If $K(\Omega, P)$ is finite then (1.3) has a solution in $\Omega$.

The assumption on the local constraint can be weakened through some restriction on the global constraint $\Lambda$. This is achieved under some conditions regarding the derivatives of the PMs in $\Omega$ with respect to the unknown PM $P$, which can be assumed, for example, in case (A) or in the parametric context. We formulate this result in the $\tau_{\Phi}$-topology.

**Proposition 2.4.** Let $\mathcal{R}$ be a closed set of densities $\frac{dQ}{dP}$ in the weak topology on $L_1(\mathbb{R}^d, P)$. Let $\mathcal{G} := \{Q \in M_1: \frac{dQ}{dP} \in \mathcal{R}\}$ and let $\Lambda$ be a $\tau_{\Phi}$-closed subset of $M_{\Phi}(\mathbb{R}^d)$. Assume that (2.1) holds for all $Q$ in $\Lambda$. Then if $K(\Omega, P)$ is finite, (1.3) has a solution in $\Omega$.

### 2.2. KULLBACK-LEIBLER DIVERGENCE AND MOMENT GENERATING FUNCTION.
We recall some known facts which set the framework for the definition of the estimate. In order to apply general convexity arguments we first extend the Kullback-Leibler divergence function onto the whole space $M_{\Phi}(\mathbb{R}^d)$ of signed finite measures on $\mathbb{R}^d$ such that $|Q| |\varphi| < \infty$ for all $\varphi$ in $\Phi$, setting $K(Q, P) = \infty$ whenever $Q$ is not in $M_1$.
M. Broniatowski

The class $\Phi$ contains $B_b(\mathbb{R}^d)$; denote $\mathcal{X}$ the linear span of $\Phi$. Then $\mathcal{X}$ is a total vector space and, when equipped with the $\tau_{\mathcal{X}}$-topology, $M_{\mathcal{X}}(\mathbb{R}^d) = M_{\Phi}(\mathbb{R}^d)$ is a locally convex Hausdorff topological space with topological dual space $\mathcal{X}$, see [17], pp. 429 and ff.

On $\mathcal{X}$ define the Fenchel-Legendre transform of $K(\cdot, P)$, namely,

$$ \Lambda(f) := \sup_{Q \in \mathcal{M}_{\Phi}(\mathbb{R}^d)} Qf - K(Q, P) = Pe^{f-1} $$

for all $f$ in $\mathcal{X}$, as seen through usual optimality conditions for Gateaux differentiable convex functions on convex domains, see Theorem 31, p. 75, in [18].

By Lemma 1.8 in [16], under (2.1) $K(\cdot, P)$ is convex and lower semi-continuous (l.s.c.) in the $\tau_{\mathcal{X}}$-topology on $M_{\Phi}(\mathbb{R}^d)$. It follows from standard arguments (extend Lemma 4.5.8 in [1]) that the Kullback-Leibler divergence admits the basic representation $K(Q, P) = \sup_{f \in \mathcal{X}} Qf - \Lambda(f)$. When $Q$ is a probability measure it is easily checked that the function $f \to Qf - \Lambda(f)$ has the same supremum on $\mathcal{X}$ as $f \to Qf - \log Pe^f$, a quite familiar expression in the large deviation literature. The sup in the last expression is reached at $f^* := \log \frac{dQ}{dP}$, which we assume to be in $\Phi$. We thus have, extending the classical duality formula for the Kullback-Leibler divergence,

$$ (2.2) \qquad K(Q, P) = \sup_{f \in \mathcal{X}} Qf - \Lambda(f) = \sup_{f \in \mathcal{X}} Qf - \log Pe^f. $$

The mapping $f \mapsto \log Pe^f$ is the (extended) *moment generating function*. Furthermore, the sup in the RHS of (2.2) is reached at

$$ (2.3) \qquad f^* = \log \frac{dQ}{dP}, $$

which is in $\mathcal{X}$.

**Example 2.1.** Suppose that $P$ and $Q$ are two normal PMs on $\mathbb{R}$. Then $\log \frac{dQ}{dP}$ is a polynomial of degree 2, therefore unbounded. With $\Phi$ containing those functions, restricting $K(\cdot, P)$ to the class of PMs $Q$ in $M_{\Phi}(\mathbb{R})$, the sup in (2.2) exists in $\Phi$, although not in $B_b(\mathbb{R})$. This is precisely the reason for introducing the set of PMs $M_{\Phi}(\mathbb{R})$.

Let us consider the causes of unboundedness of $K(Q, P_n)$ when $P_n$ is the empirical measure pertaining to the sample $X := (X_1, \dots, X_n)$. Assume $Q$ in $M_{\Phi}(\mathbb{R}^d)$ absolutely continuous w.r.t. $\lambda$, the Lebesgue measure on $\mathbb{R}^d$. Since $\mathcal{X}$ contains $B_b(\mathbb{R}^d)$, it holds

$$ K(Q, P_n) := \sup_{f \in \mathcal{X}} Qf - \log P_n e^f \ge Q(-M_1 X(\cdot)) - \log P_n \exp(-M_1 X(\cdot)) = M $$

for any $M > 0$, proving unboundedness. In order to define an estimate for $K(Q, P)$ it is thus sufficient to remove such functions from $\mathcal{X}$. The parametric case provides a typical example: since both $Q$ and $P$ belong to the model, we know that the sup in (2.2) is reached for some function in $\mathcal{F} := \{ \log p_{\theta} - \log p_{\theta'} \text{ for } \theta, \theta' \in \Theta \}$. Clearly,
Estimation of the Kullback-Leibler Divergence

when the sup in the above display is considered in $\mathcal{F}$ instead of $\mathcal{X}$ then unboundedness vanishes. Also the definition of $K(Q, P)$ remains the same, since $K(Q, P) = \sup_{f \in \mathcal{F}} Qf - \log Pe^f = \sup_{f \in \mathcal{F}} Qf - \log Pe^f$. Consider now the case when we estimate $K(\Omega, P)$ for $\Omega$ in $M_\Phi(\mathbb{R}^d)$. Assume that $Q^* := \arg\inf_{Q \in \Omega} K(Q, P)$ exists in $\Omega$ with $f^* := \log \frac{dQ^*}{dP^*}$ in a class $\mathcal{F}$ of functions contained in $\Phi$. We then have

$$K(\Omega, P) := \inf_{Q \in \Omega} \sup_{f \in \mathcal{X}} Qf - \log Pe^f = \inf_{Q \in \Omega} \sup_{f \in \mathcal{F}} Qf - \log Pe^f.$$

We are led to define

$$(2.4) \qquad \tilde{K}(\Omega, P_n) := \inf_{Q \in \Omega} \sup_{f \in \mathcal{F}} Qf - \log P_n e^f,$$

which may be finite, even though $Q$ not a.c. w.r.t. $P_n$. Case (A) in the Introduction contains the definition of the ad-hoc class $\mathcal{F}$, and $\tilde{K}(\Omega, P_n)$ is finite for all $n$. Also logistic models yield well-defined classes $\mathcal{F}$.

### 2.3. A MINIMAX ARGUMENT AND THE DEFINITION OF THE ESTIMATES.
Formula (2.4) is not suitable when studying asymptotic properties. We develop a minimax argument in order to interchange the inf and sup operations.

We make the crucial assumption

(H0) There exists a unique solution to problem (1.2) in $\Omega$.

Call this solution $Q^*$ and let $f^* := \log \frac{dQ^*}{dP^*}$. Therefore

$$K(\Omega, P) = K(Q^*, P).$$

We examine the behavior of $K(Q, P)$ close to $K(Q^*, P)$. Let $C$ be any positive number and define the subset $\Omega_C$ of $\Omega$ by

$$\Omega_C := \Omega \cap \{Q \in M_1 : K(Q, P) \le K(\Omega, P) + C\}.$$

Let (2.1) hold for all $Q$ in $\Omega$. Then $\Omega_C$ is a $\tau_\Phi$-compact subset of $\Omega$; see [16], Lemma 1.8. There is no restriction in substituting the infimum on $\Omega$ by an infimum on $\Omega_C$, as $K(\Omega, P) = K(\Omega_C, P)$.

Also it is convenient to introduce some simple geometrical description of the set of PMs "on the other side of $Q^*$ with respect to $P$": Let

$$(2.5) \qquad H := \left\{ R \in M_1 : R \log \frac{dQ^*}{dP} - K(Q^*, P) \ge 0 \right\}.$$

Denote

(H1) For some $C > 0$ the set $\Omega_C$ is contained in $H$.

**Example 2.2.** If $\Omega$ is convex and (H0) holds then $\Omega$ is contained in $H$. This happens in case (A) with $\varphi \in \Phi$. Other important and less classical cases deserve attention. For example, let $\Lambda$ denote the set of unimodal distributions on $\mathbb{R}^d$ with known mode, which is convex and weakly closed in $M_1$ (see [19]), and therefore var-closed. The same properties hold for $\Lambda$ the class of DFR distributions. Let
M. Broniatowski

Q be the class of log-convex densities on $\mathbb{R}^+$, a subclass of all DFR distributions. The set Q is convex by Hölder's inequality. Also Q is closed in $L^1(\mathbb{R}^1, \lambda)$ as can be deduced following the same arguments as in Lemma 2.2 in [20]. By Proposition 2.3 it follows that the projection of P on $\Omega$ exists for any P such that $K(\Omega, P) < \infty$. By Theorem 2.2 in [14] condition (H1) is fulfilled in this highly nonlinear case.

We now state the minimax result.

**Theorem 2.5.** Assume that (H0) and (H1) hold. Then

$$ (2.6) \qquad \inf_{Q \in \Omega} \sup_{f \in \mathcal{F}} Qf - \log Pe^f = \sup_{f \in \mathcal{F}} \inf_{Q \in \Omega} Qf - \log Pe^f. $$

**Remark.** Theorem 2.5 also holds under linear constraints, as a consequence of the Minimum Discrimination Information Theorem (see [4]): If $\Omega$ is a set of PMs and $Q^* \in \Omega$ is such that $\frac{dQ^*}{dP}(x) = c \exp g(x)$ and if $Qg = a < \infty$ for all $Q \in \Omega$ then $K(Q, P) = K(Q, Q^*) + K(Q^*, P)$ and $K(Q^*, P) = K(\Omega, P)$. This is precisely what is required in the proof of Theorem 2.5. Therefore a sufficient condition for Theorem 2.5 under linear constraints of the form $\Lambda = \{Q: Qg = a\}$ with $\log \frac{dQ}{dP} \in \mathcal{F}$ is the existence of $Q^* \in \Omega$ such that $Q^*g = a$ and $f^* := \log \frac{dQ^*}{dP}(x) = \log c + g(x)$ for some $c > 0$.

We simply plug in the empirical PM $P_n$ in (2.6) and define the estimate of the Kullback-Leibler divergence between $\Omega$ and $P$ by

$$ (2.7) \qquad K_n := K_n(\Omega, P) := \sup_{f \in \mathcal{F}} \inf_{Q \in \Omega} Qf - \log P_n e^f. $$

Motivated by (2.7) we define for any PM $R$ in $M_1$

$$ (2.8) \qquad \Phi_R(f) := \inf_{Q \in \Omega} Qf - \log Re^f. $$

This function might seem cumbersome. However the optimization step may be easy, depending on the set $\Omega$. For example, in case (A) it reduces to a parametric optimization. The same situation may occur when $\Omega$ is included in some parametric model, or when $\Omega$ is convex and can be described through its extremal points. Moreover, as appears in the proof of our results, the asymptotic behavior of the processes $f \to \Phi_{P_n}(f)$ as $n \to \infty$ is simple.

We now provide an estimate of $f^* = \log \frac{dQ^*}{dP}$. In the following lemma we state that if Theorem 2.5 holds then $f^*$ can be written as the supremum of $\Phi_P(f)$. This is the key result to the estimation procedure. We first state conditions entailing existence of the supremum of $\Phi_P(f)$. Sometimes no condition is needed but some mild regularity condition on the class $\mathcal{F}$, as occurs for parametric models or for moment constraints (case (A)). In more involved situations we may require more restrictive conditions. The scope of the coming developments is to present some sufficient conditions for the asymptotic study of the estimates. However specific situations may lead to much weaker conditions, as in [11] and [12]. Let $\mathcal{F}$ be equipped with the sup norm.
Denote

(H2) $\mathcal{F}$ is compact,

which is a classical assumption when the supports of all PMs in $\Omega$ are in the same compact subset of $\mathbb{R}^d$ and all densities (of $Q$ and $P$ w.r.t. $\lambda$) are bounded from above and below, see [10] and the references therein.

**Lemma 2.6.** When H2 holds then (i) the mapping $f \mapsto \Phi_Q(f)$ is continuous in the sup-norm for any PM $Q$ in $M_\Phi$. (ii) $\Phi_P$ has a unique maximum in $\mathcal{F}$ and $\arg\sup_{f \in \mathcal{F}} \Phi_P(f) = f^*$.

As a consequence of Lemma 2.6 we define

$$ (2.9) \qquad f_n := \arg\sup_{\mathcal{F}} \Phi_{P_n}(f). $$

The existence of $f_n$ in $\mathcal{F}$ follows from the compactness of $\mathcal{F}$ and Lemma 2.6. Uniqueness may not be achieved. In the sequel we denote by $f_n$ any of the maximizers of $\Phi_{P_n}(f)$.

**Example 2.3.** The role of the class $\mathcal{F}$ is enlightened in this example. Consider the case of a constraint on the mean, and $d=1$. Let $\varphi$ be some bounded real valued function defined on $\mathbb{R}$ and let $\Lambda := \{Q \in M_1: Q\varphi = a\}$, which is convex and variation closed. Let $\Lambda_1 := \Lambda \cap \{Q \in M_1: K(Q, P) \le c\}$ for some constant $c$ large enough, so that $\Lambda_1$ is non-empty. The set $\Lambda_1$ is convex and $\tau$-compact (see [15]). The infimum of the KL divergence between $\Lambda_1$ and $P_n$ is reached at $Q_n$ a.c.-$P_n$ with $\log \frac{dQ_n}{dP_n}(x) = f_n(x) = m\varphi(x) + p$, where $m$ satisfies $\frac{\phi'_n}{\phi_n}(m) = a$ and $p = -\log (\phi_n^{-1}(\frac{\phi'_n}{\phi_n})^{-1}(a))$, and $\phi_n(t) := P_n e^{t\varphi}$.

Let $\Phi := \{f: \mathbb{R} \to \mathbb{R}, f(x) = m\varphi(x) + p; (m, p) \in \mathbb{R}^2\}$. Let $\mathcal{F}$ be the subset of $\Phi$ such that $m$ and $p$ belong to some compact set whose interior contains $((\frac{\phi'}{\phi})^{-1}(a), -\log(\phi^{-1}(\frac{\phi'}{\phi})^{-1}(a)))$, with $\phi(t) := P e^{t\varphi}$. In this example, (H2) does not hold, but still the conclusion of Lemma 2.6 holds. Let $\Omega = \Lambda_1 \cap \mathcal{G}$ as in case (A), i.e., $\mathcal{G} := \{Q \in M_1: \log \frac{dQ}{dP} \in \mathcal{F}\}$. The optimization in $\sup_{\mathcal{F}} \inf_{\Omega} Qf - \log P_n e^f$ yields now: For any $f = \alpha\varphi + \beta \in \mathcal{F}$, $\inf_{\Omega} Qf - \log P_n e^f = \alpha a - \log P_n e^{\alpha\varphi}$ which in turn implies that $\arg\sup_{\mathcal{F}} \inf_{\Omega} Qf - \log P_n e^f = g_n(x) = m\varphi(x) + p$, with the same values of $m$ and $p$ as before. However, the solution is now in $\mathcal{F}$. Notice that $f_n$ does not belong to $\Phi$. We see that we have extended the projected measure from the support of $P_n$ to the support of $P$.

### 3. Convergence Results

All results in this section rely upon classical functional theorems in the empirical processes literature, namely, Glivenko-Cantelli and central limit theorems. In this way, many classes $\mathcal{F}$ can be considered. Also independence of the $X_i$'s is not required, depending upon the available limit theorems. We treat the i.i.d. case for the sake of clarity.

3.1. ALMOST SURE RESULTS. The following sketch of the proof of a.s. convergence of $K_n$ to $K(\Omega, P)$ illustrates the kind of condition that is needed, as well as the basic role of the minimax Theorem 2.5 in the definition of the estimate.
M. Broniatowski

By (2.7) $K_n := \sup_{f \in \mathcal{F}} \Phi_{P_n}(f)$, the supremum on $\mathcal{F}$ of the concave function $\Phi_{P_n}$.

By Theorem 2.5

$$|\Phi_{P_n}(f) - \Phi_P(f)| \leq \left| \log \left( 1 + (P_n - P) \frac{e^f}{Pe^f} \right) \right|$$

If $\overline{\mathcal{F}} := \{\frac{e^f}{Pe^f}, f \in \mathcal{F}\}$ is a Glivenko-Cantelli (GC) class then

$$\lim_{n \to \infty} \sup_{f \in \mathcal{F}} |\Phi_{P_n}(f) - \Phi_P(f)| = 0 \quad \text{a.s.,}$$

which yields

$$\lim_{n \to \infty} \sup_{f \in \mathcal{F}} \Phi_{P_n}(f) = \sup_{f \in \mathcal{F}} \Phi_P(f) \quad \text{a.s.,}$$

i.e.,

$$(3.1) \qquad \lim_{n \to \infty} K_n = K(\Omega, P) \quad \text{a.s.}$$

We are led to consider conditions on $\mathcal{F}$ which imply that $\overline{\mathcal{F}}$ is GC.

Suppose that

(H3) $\mathcal{F}$ has a lower envelope $L$ and an upper envelope $U$, i.e.,

$$L(x) \leq f(x) \leq U(x) \quad \text{for all } x \text{ and all } f \text{ in } \mathcal{F} \text{ and } 0 < Pe^L \leq Pe^U < \infty.$$

We handle the GC property through covering numbers. Let $D(\epsilon, \mathcal{F}, \|\cdot\|)$ denote the minimal number of $\|\cdot\|$-spheres with radius $\epsilon$ necessary to cover $\mathcal{F}$. By (H3), $\overline{\mathcal{F}}$ has a $P$-integrable envelope. Recall (see, e.g., [21], Theorem 24, Chapter 2) that $\overline{\mathcal{F}}$ is GC if

$$(3.2) \qquad \log D(C\epsilon, \overline{\mathcal{F}}, \|\cdot\|_{L^1(P_n)}) = o_p(n) \quad \text{for any } \epsilon > 0.$$

**Theorem 3.1.** If (H0), (H1), and (H3) hold together with (3.2) for any positive $\epsilon$ then

$$\lim_{n \to \infty} K_n = K(\Omega, P) \quad \text{a.s.}$$

The following result may be useful in order to check condition (3.2).

**Lemma 3.2.** Assume that (H3) holds. There exists a finite constant $C > 0$ such that for any $\epsilon > 0$ and for large $n$, $D(C\epsilon, \overline{\mathcal{F}}, \|\cdot\|_{L^1(P_n)}) \leq D(\epsilon, \mathcal{F}, \|\cdot\|_{\text{sup}})$ with probability 1.

Proposition 11.1.4 in [22] states that if $D(\epsilon, \mathcal{F}/U, \|\cdot\|_{\text{sup}})$ is finite for any $\epsilon > 0$ then $D(\epsilon, \mathcal{F}, \|\cdot\|_{\text{sup}}) < \infty$, which implies (3.2). See also Theorem 11.1.6 in [22] for the Glivenko-Cantelli Theorem under such hypothesis.

**Remark 3.1.** When $\mathcal{F}$ is defined as in case (B), (3.2) holds whenever both $D(\epsilon, \log Q, \|\cdot\|_{L^1(P_n)})$ and $D(\epsilon, \log P, \|\cdot\|_{L^1(P_n)})$ are finite, since $\mathcal{F}$ is covered by $D(\epsilon, \log Q, \|\cdot\|_{L^1(P_n)}) \cdot D(\epsilon, \log P, \|\cdot\|_{L^1(P_n)})$ spheres.
Together with Theorem 3.1 we have

**Corollary 3.3.** *When (H0)-(H3) and (3.2) hold then*

$$ \lim_{n \to \infty} \sup_x |f_n(x) - f^*(x)| = 0 \quad \text{a.s.} $$

Let us turn to the properties of $Q_n$ defined by

$$ (3.3) \qquad Q_n := \arg \inf_{\Omega} Q f_n - \log P_n e^{f_n}. $$

When (H2) holds then $Q_n$ is defined, although it may not be unique. Indeed, when $\Omega$ is $\tau_\Phi$-closed we may replace $\Omega$ by $\Omega_C$ in (3.3), and $\Omega_C$ is $\tau_\Phi$-compact. Obviously we make the same change in the definition (2.7) of $K_n$ and in (2.8). All convergence results for $f_n$ and $K_n$ remain true with this minor change. However we have gained $\tau_\Phi$-compactness of $\Omega_C$. The linear mapping $Q \mapsto Qf_n$ is continuous in the $\tau_\Phi$-topology since $f_n$ is in $\mathcal{F}$. Therefore $Q_n$ is well defined. Of course, $f_n$ defined in (2.9) is not equal to $\log \frac{dQ_n}{dP}$, but still we can obtain quite good limit properties of any of the $(Q_n, f_n)$.

We see that $(Q_n, f_n)$ has similar properties to $(Q^*, f^*)$. It is easily seen using formula (2.2) that $K(\Omega, P) = Q^*f^*$. Therefore a natural estimate for $K(\Omega, P)$ is $Q_n f_n$, which converges a.s. to $K(\Omega, P)$ when (H2) holds. In this case we also may define

$$ (3.4) \qquad d\bar{Q}_n := \frac{e^{f_n} dP_n}{P_n e^{f_n}}, $$

which converges weakly to $Q^*$ with probability 1.

### 3.2. CONVERGENCE IN DISTRIBUTION OF THE ESTIMATE OF THE ENTROPY. We now assume

(P) $P$ does not belong to $\Omega$.

We keep the notation of the previous section and assume that (H0) and (H1) hold. The limiting behavior of $K_n$ under (P) is a consequence of the central limit theorem applied to the processes $f \mapsto \Phi_{P_n}(f)$ and of some control of the approximating Gaussian processes in a neighborhood of $f^*$. We assume that $\mathcal{F}$ is compact in sup-norm, which allows for quite explicit conditions for the convergence in distribution of the estimates. We briefly recall some useful facts from Gaussian approximations (see, e.g., [23]). We will assume that

$$ (3.5) \qquad \int_0^1 (\log D(\epsilon, \mathcal{F}, \| \cdot \|_{\text{sup}}))^{1/2} d\epsilon < \infty $$

and

(H4) $Pe^{2U} < \infty$,

which together with (H3) implies that $\mathcal{F}$ has an envelope in $L^2(P)$.

The following result parallels Lemma 3.2.
M. Broniatowski

**Lemma 3.4.** Whenever (H3) and (H4) hold,

$$D(C\varepsilon, \bar{\mathcal{F}}, \|\cdot\|_{L^2(P_n)}) \leq D(\varepsilon, \mathcal{F}, \|\cdot\|_{\sup})$$

for some constant C and all large n with probability 1.

**Remark 3.2.** The same comment as in Remark 3.1 holds for the above condition.

Lemma 3.4 is a direct consequence of Lemma 3.2. By Lemma 3.4 when (H3) and (H4) hold and $D(\varepsilon, \mathcal{F}, \|\cdot\|_{\sup})$ satisfies (3.5), $\bar{\mathcal{F}}$ is also a Donsker class, as follows, for example, from Lemma 15 and Theorem 21 in [21], Chapter 7.

Let $(A, \mathcal{A}, P)$ be a probability space. Denote by $G_P$ a P-Brownian bridge, i.e., a centered Gaussian process indexed by $L^2(A, \mathcal{A}, P)$, with covariance

$$\operatorname{cov}(G_P(u), G_P(v)) = Puv - PuPv.$$

All trajectories $G_P(\cdot)(\omega)$ are bounded and uniformly continuous on $\bar{\mathcal{F}} \subset L^2(A, \mathcal{A}, P)$ equipped with the metric $\rho_P^2(u, v) = E(G_P(u) - G_P(v))^2$. The Weak Invariance Principle states that on $\bar{\mathcal{F}}$ there exists a sequence of P-Brownian bridges $X^{(1)}, X^{(2)}, \dots$ defined on the same probability space as the $X_i$'s such that for any $\epsilon > 0$

$$\lim_{n \to \infty} P\left(\sup_{t \in \bar{\mathcal{F}}} \left|\sqrt{n}(P_n - P)(t) - X^{(n)}(t)\right| > \epsilon\right) = 0.$$

Consider the processes $u \mapsto \sqrt{n}(\Phi_{P_n}(u) - \Phi_P(u))$ defined on $\mathcal{F}$.

**Lemma 3.5.** If (H3) and (H4) hold and $\mathcal{F}$ satisfies (3.5) then

$$\lim_{n \to \infty} P\left(\sup_{u \in \mathcal{F}} \left| \sqrt{n}(\Phi_{P_n}(u) - \Phi_P(u)) - X^{(n)}\left(\frac{e^u}{Pe^u}\right) \right| > \epsilon\right) = 0$$

for all $\epsilon > 0$, with $X^{(n)}$ defined as above.

Denote

$$\bar{f} := \frac{e^{f^*}}{Pe^{f^*}}.$$

The limiting distribution of $K_n$ is captured in the following result.

**Theorem 3.6.** Assume (P), (H0)-(H4) and let $\mathcal{F}$ satisfy (3.5). Then

$$n^{1/2}(K_n - K(\Omega, P)) \to_d G_P(\bar{f}).$$

When $P$ belongs to $\Omega$, we have $f^* = 0$ and $\bar{f} = 1$. Then the variance of $G_P(\bar{f})$ is equal to 0, $\sqrt{n}K_n$ tends to 0 almost surely, and therefore the limiting distribution of $\sqrt{n}K_n$ is degenerate. The asymptotic distribution of the statistics $nK_n$ can be studied in specific cases. This will not be discussed here.
4. Statistical Applications

4.1. PARAMETRIC ESTIMATION AND DUALITY. Consider a parametric model {$P_\theta, \theta \in \Theta$} with $\Theta$ an open subset of $\mathbb{R}$, say. We assume that any $P_\theta$ has density $p_\theta$ w.r.t. the Lebesgue measure. Suppose that this model is suitably parametrized, i.e., that $K(P_\theta, P_{\theta'}) = 0$ implies $\theta = \theta'$. Let $X_1, \dots, X_n$ be realizations of $n$ i.i.d. r.v.'s with common distribution $P_{\theta_0}$, with $\theta_0$ unknown in $\Theta$. In this section we propose a new estimate for $\theta_0$. Recall that parametric estimation using divergence criterions has only been considered for distributions with discrete support, see [5]. Duality considerations as exposed in the present paper allow for a complete development of divergence optimization techniques in the field of parametric statistics, see [11]. We briefly sketch here the kind of result that can be obtained for the Kullback-Leibler divergence.

Let $\alpha$ in $\Theta$ be such that $\int \log \frac{p_\alpha}{p_\theta} p_\alpha dx < \infty$ for any $\theta$ in some convex neighborhood $V(\theta_0)$ contained in $\Theta$.

Define $\mathcal{F} := \{x \to \log \frac{p_\alpha}{p_\theta}(x), \theta \in V(\theta_0)\}$, a class of functions depending upon $\alpha$ and $V(\theta_0)$, and let $K_{\theta_0}(\theta) := \int \log \frac{p_\alpha}{p_\theta} p_\alpha dx - \log \int \frac{p_\alpha}{p_\theta} p_{\theta_0} dx$.

**Lemma 4.1.** (i) $\sup_{\theta \in V(\theta_0)} K_{\theta_0}(\theta) = K_{\theta_0}(\theta_0) = K(P_\alpha, P_{\theta_0})$.

(ii) $\theta_0$ is the unique optimizer of $K_{\theta_0}(\theta)$ in $V(\theta_0)$.

Indeed, let $\mathcal{G}$ denote the class of all functions such that $\int g p_\alpha dx < \infty$. Then $K(P_\alpha, P_{\theta_0}) = \sup_{g \in \mathcal{G}} \int g p_\alpha dx - \log \int e^g p_{\theta_0} dx$. But the supremum is achieved at $g = \log \frac{p_\alpha}{p_{\theta_0}}$, which belongs to $\mathcal{F}$, hence (i) holds. The model is suitably parametrized, which implies (ii).

Let us introduce the estimate of $\theta_0$. We plug in the empirical measure pertaining to the $X'_i$s and define

$$ \alpha_n := \arg \sup_{\theta \in V(\theta_0)} K_{P_n}(\theta) $$

with $K_{P_n}(\theta) := \int \log \frac{p_\alpha}{p_\theta} p_\alpha dx - \log \frac{1}{n} \sum_{i=1}^n \frac{p_\alpha}{p_\theta}(X_i)$. Note that this M-estimate depends upon some a priori knowledge on $\theta_0$, through the choice of $\alpha$ and $V(\theta_0)$.

**Proposition 4.2.** When the class of functions {$p_\alpha/p_\theta, \theta \in V(\theta_0)$} is Glivenko-Cantelli and $p_\alpha/p_\theta$ is bounded away from 0 for $\theta \in V(\theta_0)$ then $\alpha_n$ is a strongly consistent estimate of $\theta_0$.

The proof of this result relies on the fact that the function $\theta \to K_{\theta_0}(\theta)$ has a unique maximum at $\theta_0$ and upon uniform convergence of $K_{P_n}(\theta)$ to $K_{\theta_0}(\theta)$, which follows from the hypotheses in the proposition.

The weak behavior of this estimate is captured in the following result. Let

$$ V := \frac{\int \left( \frac{d}{d\theta} p_{\theta} \right)^2_{\theta=\theta_0} dx}{\left( \int \frac{p_{\alpha}}{p_{\theta_0}} \left( \frac{d}{d\theta} p_{\theta} \right)_{\theta=\theta_0} dx \right)^2} $$

and assume that $V$ is finite. Then it can be proved via the usual asymptotic expansions that

**Proposition 4.3.** When $\alpha$ is not equal to $\theta_0$ then $V^{-1/2}(\alpha_n - \theta_0)$ has limiting $N(0, 1)$ distribution.
M. Broniatowski

4.2. UPDATING UNDER AN M-TYPE CONSTRAINT. Consider an M-func-
tional, i.e., a real-valued function T on M₁ defined by the equation

$$
\int \psi(x, T(Q)) dQ(x) = 0,
$$

where $\psi$ is a function defined on $[0, 1] \times \mathbb{R}$. Let $\lambda_Q(t) := \int \psi(x,t) dQ(x)$. It is known that whenever $\psi(x,t)$ is monotone in $t$ and the equation $\lambda_Q(t) = 0$ has an isolated root $t_0$ for any PM $Q$ in $M_1$ then $T(Q)$ is well defined and is equal to $t_0$. See, e.g., [24] for definitions and properties, as well as other sufficient conditions on $\psi$ for existence of the M-functional. We consider the constraint

$$
\Lambda := \{Q \in M_1 : T(Q) \leq a\}
$$

for some $a$ in $\mathbb{R}$. Such case includes moment or quantile constraints, as well as con-
straints on maximum likelihood functionals, see [24] for examples. It is convenient
to introduce the following conditions.

(M0) The equation λ<sub>Q</sub>(t) = 0 has an isolated root t<sub>0</sub> for any PM Q in M<sub>1</sub>;

(M1) $\psi(x, t)$ is monotone in $t$;

(M1') $\psi(x, t)$ is continuous in $t$ and bounded;

(M2) $\psi$ is twice differentiable in $t$ and $\frac{\left(\int \frac{\partial^2 \psi}{\partial t^2}(x, T(Q)) dQ(x)\right)}{\left(\int \frac{\partial \psi}{\partial t}(x, t(Q)) dQ(x)\right)^3} \le 0$ for all $Q \in M_1$.

If $d=1$ and $t$ is a location parameter, conditions (M0) and (M1) are sufficient for $T$ to be weakly continuous when $\psi(x,t) = x-t$ (constraint on the mean), or $\psi(x-t) = -1$ for $x-t < 0, = 0$ for $x=t$, and $=1$ for $x-t > 0$ (constraint on the median), or for Winsorized means. Conditions (M0) and (M1') imply weak continuity of $T$ for the cases when $\psi(x-t)$ returns to 0 for large values of $|x-t|$, which may be useful if the constraint should not be sensitive to extreme values. See [25].

Condition (M2) is sufficient for convexity of *T* at *Q*. Indeed, the second Gâteaux-differential of *T* at *Q* in direction *R* writes

$$
T''(R-Q, R-Q) = \frac{\left( \int \frac{\partial^2 \psi}{\partial t^2}(x, T(Q)) \, dQ(x) \right)}{\left( \int \frac{\partial \psi}{\partial t}(x, t(Q)) \, dQ(x) \right)^3} (\lambda_R - Q(T(Q)))^2.
$$

Conditions (M1) or (M1') together with (M0) and (M2) thus imply that Λ is
weakly closed and convex, which in turn implies the existence and uniqueness of
the KL projection of P onto Λ.

$$
\text{Let } \Lambda_r := \{Q \in M_1 : T(Q) = r\} = \{Q \in M_1 : \int \psi(x,r) dQ(x) = 0\}. \text{ Then}
$$

$$
\Lambda = \bigcup_{r \le a} \Lambda_r
$$

and therefore

$$
\begin{align*}
K(\Lambda, P) &= \inf_{r \le a} K(\Lambda_r, P) = \inf_{r \le a} \inf_{Q \in \Lambda_r} K(Q, P) \\
&= \inf_{r \le a} \inf_{Q \in \Lambda_r} \sup_{f \in \mathcal{B}_b([0,1])} Qf - \log Pe^f.
\end{align*}
$$
Note that the minimax Theorem 2.5 holds on $\Lambda_r$ (see Remark 2.2). Thus

$$K(\Lambda, P) = \inf_{r \le a} \sup_{f \in B_b([0,1])} \inf_{Q \in \Lambda_r} Qf - \log Pe^f.$$

The classical result of KL projection under linear constraints (see Section 3) indicates that $K(\Lambda_r, P) = -\log \int e^{\theta_r \psi(x,r)} dP(x)$, where $\theta_r$ satisfies $\int \psi(x,r) dQ_r^*(x) = 0$ with $\log \frac{dQ_r^*}{dP^*}(x) = \theta_r \psi(x,r) - C_r$ and $C_r = \log \int e^{\theta_r \psi(x,r)} dP(x)$.

In order to update $P_n$ under the constraint $\Lambda$ write $K_n := \inf_{r \le a} K_{n,r}$ with $K_{n,r} := \sup_{f \in \mathcal{F}_r} \inf_{Q \in \Lambda_r} Qf - \log P_n e^f$, where the class $\mathcal{F}_r$ contains $\log \frac{dQ_r^*}{dP^*}$. Since both $\theta_r$ and $C_r$ are unknown, we let $\mathcal{F}_r := \{x \mapsto m\psi(x,r) + p\}$, where $m$ and $p$ belong to closed intervals which we assume to contain $\theta_r$ and $C_r$. This class of functions is a VC class, and therefore all results in Section 3 hold. We obtain $K_{n,r} = \sup_m (-\log P_n e^{m\psi(\cdot,r)})$. The sup in the last formula is attained at $m = m_{n,r}$ which satisfies

$$(4.1) \qquad \frac{P_n \psi(\cdot, r) e^{m_{n,r} \psi(\cdot, r)}}{P_n e^{m_{n,r} \psi(\cdot, r)}} = 0,$$

and the estimate of $\log \frac{dQ_r^*}{dP^*}(\cdot)$ is $f_{n,r}(\cdot) = m_{n,r}\psi(\cdot, r) + p_{n,r}$ with $p_{n,r} = K_{n,r}$; see Example 2.3.

Usually the optimization step in the variable $r$ has to be handled numerically. The following classical easy case is worth noting. It copies the case when $\psi(x,t) = t - x$. Assume that $\psi(x,r)$ is non-decreasing in $r$ for all $x$. It follows that $K_n = K_{n,a} = \sup_{f \in \mathcal{F}_a} \inf_{Q \in \Lambda_a} Qf - \log P_n e^f$ and hence $f_n(x) = f_{n,a}(x) := m_{n,a}\psi(x,a) - \log \int e^{m_{n,a}\psi(x,a)} dP_n(x)$.

## 5. Proofs

Here we present most missing technical arguments in the results and comments of Sections 2 and 3.

5.1. PROOF OF PROPOSITION 2.1. Since $Q$ is strongly compact, $G$ is var-compact (Scheffé's Lemma). Therefore $\Lambda \cap G$ is var-compact. The mapping $Q \to K(Q, P)$ is l.s.c. on $M_1$ equipped with the var-topology, and the proof is complete. $\square$

5.2. PROOF OF PROPOSITION 2.2. By Scheffé's Lemma, if $Q$ is strongly closed then $G$ is var-closed. Apply Theorem 2.1 in Csiszar [14] to conclude. $\square$

5.3. PROOF OF PROPOSITION 2.4. For any $c > 0$ let $L_c := \{Q \in M_1 : K(Q, P) \le c\}$, which is a $\tau_\Phi$-compact subset of $M_\Phi$, see [16], Lemma 1.8.

Let $L_c := \{\frac{dQ}{dP} \text{ with } Q \in M_1 : K(Q, P) \le c\}$. The mapping $\nu: Q \to \frac{dQ}{dP}$ is a homeomorphism from $L_c$ equipped with the $\tau_\Phi$-topology to $L_c$ equipped with the weak topology on $L_1(\mathbb{R}^d, P)$ (the same argument as in Dembo and Zeitouni [1], Section 6.2). Now $\mathcal{R} \cap L_c$ is a weakly compact subset of $L_1(\mathbb{R}^d, P)$. Thus $G \cap L_c$ is $\tau_\Phi$-compact. Since $\Lambda$ is $\tau_\Phi$-closed, $\Omega$ is $\tau_\Phi$-compact.

Therefore $K_c := \{Q \in \Omega : K(Q, P) \le c\}$ is a $\tau_\Phi$-compact subset of $M_\Phi$. Since the mapping $Q \to K(Q, P)$ is l.s.c. in the $\tau_\Phi$-topology, the proposition is proved. $\square$
5.4. PROOF OF THEOREM 2.5. We prove Theorem 2.5 by contradiction. On $M(\mathbb{R}^d) \times B_b(\mathbb{R}^d)$ define $T(Q, f) := Qf - \log Pe^f$. We have

$$ \sup_{f \in \mathcal{F}} \inf_{Q \in \Omega_C} T(Q, f) \le \inf_{Q \in \Omega_C} \sup_{f \in \mathcal{F}} T(Q, f). $$

By assumption there exists $(Q^*, f^*) \in \Omega_C \times \mathcal{F}$ such that $\inf_{Q \in \Omega_C} \sup_{f \in \mathcal{F}} T(Q, f) = T(Q^*, f^*) = K(\Omega, P)$. Suppose that

$$ \sup_{f \in \mathcal{F}} \inf_{Q \in \Omega_C} T(Q, f) < \inf_{Q \in \Omega_C} \sup_{f \in \mathcal{F}} T(Q, f). $$

We then have $Qf^* - \log Pe^f < K(\Omega, P)$ for some $Q \in \Omega_C$, $Q \ne Q^*$ a.s.-P. Since $f^* = \log \frac{dQ^*}{dP}$ and $Qf^* = K(Q, P) - K(Q, Q^*)$, this would imply $K(Q, P) - K(Q, Q^*) - K(\Omega, P) < 0$. Now for any PM $Q$, $\int \log \frac{dQ^*}{dP} dQ - K(Q^*, P) = K(Q, P) - K(Q, Q^*) - K(\Omega_C, P)$. Therefore whenever $\Omega_C$ lies entirely in $H$ for some $C > 0$, the inequality cannot hold. Note that this by no means implies that $\Omega$ is convex. Indeed, $\{Q \in M_{\Phi}(\mathbb{R}^d): \log \frac{dQ}{dP} \in \mathcal{F}\}$ may be a non-convex set, which could make $\Omega$ not convex. $\square$

5.5. PROOF OF LEMMA 2.6. (i) Let $\epsilon > 0$, and let $f$ and $g$ be two functions in $\mathcal{F}$ such that $\sup_x |f(x) - g(x)| \le \epsilon$. Then $|\Phi_P(f) - \Phi_P(g)| \le |\inf_{Q \in \Omega} Qf - \inf_{Q \in \Omega} Qg| + |\log Pe^f - \log Pe^g| =: A+B$. We provide an upper bound for $A$. There exists $Q_\epsilon \in \Omega$ with $\inf_{Q \in \Omega} Qg \le Q_\epsilon g \le \inf_{Q \in \Omega} Qg + \epsilon$. Further, $\inf_{Q \in \Omega} Qf \le Q_\epsilon f$. Combining these two inequalities yields $|\inf_{Q \in \Omega} Qf - \inf_{Q \in \Omega} Qg| \le Q_\epsilon |f - g| + \epsilon$, implying $|\inf_{Q \in \Omega} Qf - \inf_{Q \in \Omega} Qg| \le \sup_x |f(x) - g(x)|$. Turn to the upper bound of $B$. Use $|\log(1+x)| \le C|x|$ for small $x$ and some constant $C$ in $|\log Pe^f - \log Pe^g| = |\log(1+\frac{P(e^f - e^g)}{Pe^g})|$, together with $|\frac{P(e^f - e^g)}{Pe^g}| \le C \sup_x |f(x) - g(x)| \le C\epsilon$, to deduce $|\log Pe^f - \log Pe^g| \le C \sup_x |f(x) - g(x)|$. The constant may depend upon $(f,g)$. Those two upper bounds yield $|\Phi_P(f) - \Phi_P(g)| \le (1+C)\sup_x |f(x) - g(x)|$, which concludes the proof of (i).

(ii) Compactness of $\mathcal{F}$ and (i) imply the existence of a set $F$ of maximizers for $\Phi_P(\cdot)$. Let us first prove that $f^*$ belongs to $F$. By (H1) for any $Q$ in $\Omega_C$, $Q \log \frac{dQ^*}{dP} - K(Q^*, P) \ge 0$ and hence $\inf_{\Omega_C} Q \log \frac{dQ^*}{dP} \ge K(Q^*, P)$. Indeed, this proves that $\inf_{\Omega_C} Q \log \frac{dQ^*}{dP} = K(Q^*, P)$. If $f^*$ does not belong to $F$ then the two above formulas imply $\sup_{\mathcal{F}} \inf_{\Omega_C} Qf - \log Pe^f > K(Q^*, P) = \inf_{\Omega_C} \sup_{\mathcal{F}} Qf - \log Pe^f$, a contradiction with Theorem 2.5. Therefore $f^*$ belongs to $F$.

Let us prove that $f^*$ is the only function in $F$. Let $g$ belong to $F$, $g$ different from $f^*$. Since $g$ is a proper maximizer of $\Phi_P$ on $\mathcal{F}$ it holds $\inf_{\Omega_C} Qg - \log Pe^g = K(Q^*, P)$. Hence $K(Q^*, P) = \inf_{\Omega_C} Qg - \log Pe^g \le Q^*g - \log Pe^g \le \sup_{\mathcal{F}} Q^*f - \log Pe^f = K(Q^*, P)$, which yields $Q^*g - \log Pe^g = K(Q^*, P)$, implying $g = f^*$. P-a.e. by (H1). This concludes the proof. $\square$

5.6. PROOF OF LEMMA 3.2. We have for any $(f,g)$

$$ P_n \left| \frac{e^f}{Pe^f} - \frac{e^g}{Pe^g} \right| \le \frac{1}{Pe^f} P_n |e^f - e^g| + \left| \frac{1}{Pe^f} - \frac{1}{Pe^g} \right| P_n e^g $$
Estimation of the Kullback-Leibler Divergence

$$
\begin{align*}
&\le C \frac{P_n e^U}{P_n e^L} \sup_x |f(x) - g(x)| + \frac{P_n e^U}{(P_n e^L)^2} |Pe^f - Pe^g| \\
&\le C' \sup_x |f(x) - g(x)|,
\end{align*}
$$

whence

$$
(5.1) \qquad D(C'\epsilon, \bar{\mathcal{F}}, \|\cdot\|_{L^1(P_n)}) \le D(\epsilon, \mathcal{F}, \|\cdot\|_{\text{sup}}).
$$

By the strong law of large numbers $P_n$ converges weakly almost surely to $P$. Therefore $P_n e^L$ and $P_n e^U$ converge to their respective limits and the constant $C'$ in (5.1) can be made deterministic and independent on $n$. $\square$

5.7. PROOF OF COROLLARY 3.3. Let $\tilde{f}$ be a limit point of some converging
subsequence $(f_{n'})_{n'}$ with $\{n'\} \subseteq \{n\}$. Recall that $\mathcal{F}$ is compact. It holds

$$
|\Phi_P(\tilde{f}) - \Phi_P(f^*)| \le |\Phi_P(\tilde{f}) - \Phi_P(f_{n'})| + |\Phi_P(f_{n'}) - \Phi_{P_{n'}}(f_{n'})| \\
\phantom{|\Phi_P(\tilde{f}) - \Phi_P(f^*)| \le } + |\Phi_{P_{n'}}(f_{n'}) - \Phi_P(f^*)| := A + B + C.
$$

Now $A \to 0$ since $\Phi_P$ is continuous, $B \to 0$ by the Glivenko-Cantelli Theorem, and $C \to 0$ by Theorem 3.1. Therefore $\tilde{f} = f^*$, since $\Phi_P$ has a unique maximum on $\mathcal{F}$. $\square$

5.8. PROOF OF THEOREM 3.6. We first prove Lemma 3.5. We have $\Phi_{P_n}(u) - \Phi_P(u) = \log\left(1 + \frac{(P_n-P)e^u}{P_n e^u}\right)$, whence $n^{1/2}(\Phi_{P_n}(u) - \Phi_P(u)) = E_n(g) + O_P(1/\sqrt{n})$ as $n \to \infty$, where $E_n(g) := \sqrt{n}(P_n - P)g$ and $g = \frac{e^u}{P_n e^u} \in \bar{\mathcal{F}}$, which is also a Donsker class. The proof ends through the central limit and Slutsky's Theorems.

We turn to the proof of Theorem 3.6. This proof has common features with the
ones in [26] and [27].

With the above notation and setting $K := K(\Omega, P)$, we have

$$
\begin{align*}
X^{(n)}(\bar{f}) - n^{1/2}(K_n - K) &= X^{(n)}(\bar{f}) - n^{1/2} \left( \sup_{u \in \mathcal{F}} \Phi_{P_n}(u) - \Phi_P(f^*) \right) \\
&\le X^{(n)}(\bar{f}) - n^{1/2} \left( \Phi_{P_n}(f^*) - \Phi_P(f^*) \right).
\end{align*}
$$

Therefore $\lim_{n \to \infty} P(X^{(n)}(\bar{f}) - n^{1/2}(K_n - K) > \epsilon) = 0$ for all $\epsilon > 0$. We have used Lemma 3.5.

For the converse inequality note that

$$
(5.2) \quad n^{1/2}(K_n - K) - X^{(n)}(\bar{f}) \le \left\{ n^{1/2} \left( \sup_{u \in \mathcal{F}_n} \Phi_{P_n}(u) - K \right) - X^{(n)}(\bar{f}) \right\} \\
\qquad \vee \left\{ n^{1/2} \left( \sup_{u \in \mathcal{F}_n^c} \Phi_{P_n}(u) - K \right) - X^{(n)}(\bar{f}) \right\},
$$

where $\mathcal{F}_n := \{u \in \mathcal{F} : K - n^{-1/4} \le \Phi_P(u) \le K\}$ and $\mathcal{F}_n^c := \{u \in \mathcal{F} : \Phi_P(u) < K - n^{-1/4}\}$. The second term in the right-hand side of the above inequality is less than $n^{1/2}(\Phi_{P_n}(f_0) - \Phi_P(f_0)) + |X^{(n)}(\bar{f})| - n^{1/4}$, where $f_0 \in \mathcal{F}$ stands
M. Broniatowski

for $\arg\sup_{u \in \mathcal{F}_n} \Phi_{P_n}(u)$. This expression is less than $[n^{1/2}(\Phi_{P_n}(f_0) - \Phi_P(f_0)) - X^{(n)}(\bar{f}_0)] + X^{(n)}(\bar{f}_0) + |X^{(n)}(\bar{f})| - n^{1/4}$, which tends to $-\infty$ as $n \to \infty$, since $X^{(n)}(\cdot)$ is bounded.

The first term in the RHS of (5.2) is less than $[n^{1/2}(\Phi_{P_n}(g_0) - \Phi_P(g_0)) - X^{(n)}(\overline{g_0})] + X^{(n)}(\overline{g_0}) - X^{(n)}(\bar{f})$ with $g_0 = g_0(n) := \arg\sup_{u \in \mathcal{F}_n} \Phi_{P_n}(u)$ and $\overline{g_0} = \overline{g_0}(n) := \frac{e^{g_0}}{P_{e,g_0}}$.

Continuity of $\Phi_P(\cdot)$ and uniform continuity of the processes $X^{(n)}(\cdot)$, as well as the a.s. weak convergence of $P_n$ to $P$ imply that for any $\eta > 0$, $X^{(n)}(\overline{g_0}) - X^{(n)}(\bar{f}) \le \eta$ a.s. for large $n$. Note that $\bar{f}$ belongs to $\mathcal{F}_n$ for all $n$, $g_0$ belongs to $\mathcal{F}_n$ since $\mathcal{F}_n$ is closed, and $\lim_{n \to \infty} \mathcal{F}_n = \{\bar{f}\}$. Therefore, by Lemma 3.5, the first term in the RHS of (5.2) converges in probability to 0 as $n \to \infty$. This closes the proof of Theorem 3.6. $\square$

**Acknowledgement.** The author thanks a referee for having drawn his attention to the $\tau_\Phi$-topology, which led to an improved version from a previous draft.

## References

[1] A. Dembo and O. Zeitouni, *Large Deviation Techniques and Applications*, Jones and Bartlett Publishers, 1993.

[2] I. Csiszar, *Sanov property, generalized I-projection and a conditional limit theorem*, Ann. Probab., 12 (1984), 768-793.

[3] Ya. Nikitin, *Asymptotic Efficiency of Nonparametric Tests*, Cambridge University Press, Cambridge, 1995.

[4] S. Kullback, *Information Theory and Statistics*, Wiley, New York, 1959.

[5] D. Morales, L. Pardo, and I. Vajda, *Asymptotic divergence of estimates of discrete distributions*, J. Statist. Plann. Inference, 38 (1995), 347-369.

[6] T. M. Cover and J. A. Thomas, *Information Theory*, Wiley Series in Telecommunica-
tions, 1991.

[7] A. Basu and B. G. Lindsay, *Minimum disparity estimation for continuous models: Efficiency, distributions and robustness*, Ann. Inst. Statist. Math., 46 (1994), 683-705, 1994.

[8] V. Patilea, *Convex models, MLE and misspecification*, Ann. Statist., 29 (2001), 94-123.

[9] D. Dacunha-Castelle and F. Gamboa, *Maximum d' entropie et problème des moments*, Ann. Inst. H. Poincaré, Sér. 1, 26 (1990), 567-596.

[10] A. R. Barron and C. H. Sheu, *Approximation of density functions by sequences of exponential families*, Ann. Statist., 18 (1989), 1347-1369.

[11] M. Broniatowski and A. Keziou, *Parametric estimation and tests through divergences*, submitted, 2003.

[12] M. Broniatowski and A. Keziou, *Empirical divergences and duality*, 2003.

[13] A. Keziou and S. Leoni, *The two-samples problem from a divergence-based approach*, 2003.

[14] I. Csiszar, *I-divergence of probability distributions geometry and minimization problems*, Ann. Probab., 3 (1975), 146-158.

[15] P. Groeneboom, J. Oosterhooff, and F. H. Ruymgaart, *Large deviation theorem for empirical probability measures*, Ann. Probab., 7 (1979), 553-586.

[16] P. Eichelsbacher and U. Schmock, *Large deviations of products of empirical measures and u-empirical measures in strong topologies*, Technical report, Bielefeld University, 1997.

[17] N. Dunford and J. T. Schwartz, *Linear Operators*, Interscience Publishers, 1958.

[18] D. Aze, *Elements D' Analyse Convexe et Variationnelle*, Ellipses, 1997.

[19] S. Dharmadhikari and K. Joag-Dev, *Unimodality, Convexity, and Applications*, Academic Press, 1988.
Estimation of the Kullback-Leibler Divergence

[20] P. Feigin and E. Yashchin, *On a strong tauberian result*, Z. Wahrsch. verw. Gebiete, 65 (1983), 35-48.

[21] D. Pollard, *Convergence of Stochastic Processes*, Springer, 1984.

[22] R. M. Dudley, *A course in Empirical Processes*, Ecole D' Eté Saint Flour 1982, Lecture Notes in Math., Springer, 1984.

[23] A. van der Vaart and J. Wellner, *Weak Convergence of Empirical Processes*, Springer, 1996.

[24] R. J. Serfling, *Approximation Theorems of Mathematical Statistics*, Wiley, 1980.

[25] F. R. Hampel, *Robust Statistics*, Wiley, 1974.

[26] J. H. J. Einmahl and D. Mason, *Generalized quantile processes*, Ann. Statist., 20 (1992), 1062-1078.

[27] S. Wey, *Test du nombre de modes. Un estimateur du minimum d'entropie sous contrainte non lineaire. Un theoreme limite local*, PhD thesis, Université Paris 6, 1998.