287
Statistica Neerlandica (1997) Vol. 51, nr. 3, pp. 287±317

Conditioning as disintegration
J. T. Chang and D. Pollard*
Statistics Department, Yale University, Box 208290 Yale Station,
New Haven, CT 06520, USA
Conditional probability distributions seem to have a bad reputation
when it comes to rigorous treatment of conditioning. Technical arguments are published as manipulations of Radon±Nikodym derivatives,
although we all secretly perform heuristic calculations using elementaryde®nitions of conditional probabilities. In print, measurability andaveraging properties substitute for intuitive ideas about randomvariables behaving like constants given particular conditioning information.
One way to engage in rigorous, guilt-free manipulation of conditional
distributions is to treat them as disintegrating measuresÐfamilies ofprobability measures concentrating on the level sets of a conditioning
statistic. In this paper we present a little theory and a range ofexamplesÐfrom EM algorithms and the Neyman factorization, through
Bayes theory and marginalization paradoxesÐto suggest that disintegrations have both intuitive appeal and the rigor needed for many
problems in mathematical statistics.
Key Words & Phrases: Conditional probability distributions, disintegrations, EM algorithm, suciency, Bayes theory, admissibility, marginalization paradoxes, Basu's theorem, exchangeability.

1

Introduction

In elementary probability courses one learns to calculate conditional probabilities bytaking ratios, sometimes on little intervals that shrink to a point at the end of a proof.
Conditional probability distributions are used and enjoyed freely, in restrictedsettings. In more advanced courses, where conditioning is placed on a rigorous
measure-theoretic basis, one learns that real probabilists use Radon±Nikodym
derivatives. One is warned that only in special cases can the conditional expectation
HX t  P X j T  t be treated rigorously as the expectation of the random variable
X with respect to a probability measure P  j T  t that concentrates on the set
fT  tg. Instead, in the abstract Kolmogorov approach, HX  is characterized up toalmost-sure equivalence as the measurable function for which
PfT 2 BgHX T   PfT 2 BgX 

1chang@stat.yale.edu, pollard@stat.yale.edu, world-wide web URL http://www.stat.yale.edu* Supported by NSF Grants DMS-9102286 and DMS-9404180# VVS, 1997. Published by Blackwell Publishers, 108 Cowley Road, Oxford OX4 1JF, UK and 350 Main Street, Malden, MA 02148, USA.

288

J. T. Chang and D. Pollard

for all measurable B. (Note that we are using linear functional notation for
expectations, as explained at the end of this section.) The abstract approach has thevirtue of making P X j T  t well de®ned (up to an almost sure equivalence) as afunction of t whenever X is integrable. It has the disadvantage of sacri®cing intuition
to rigor.
Conditional probability distributions are clearly missed in some advanced work.
Probabilists and statisticians often really do think in terms of conditional distributions, returning to them for private side calculations performed to get initial understanding of a problem. One ®rst guesses the form of HX t, perhaps with the help of
an unjusti®ed manipulation of the nonexistent probability measure P  j T  t, or bya hand-waving reduction to the discrete case. Then the proof reduces to a mechanicalchecking of the necessary measurability and averaging properties. Moreover,attempts to construct rigorous arguments using only elementary methods ofconditioning can lead to the imposition of extraneous regularity conditions. Suchattempts also lead to contortions, such as the introduction of unnecessary randomvariables and maps that transform the problem to a setting in which conditional
densities may be calculated as ratios of joint to marginal densities on Euclidean
spaces.
In this paper we discuss an approach to conditioning that combines the advantagesof both the elementary and the abstract Kolmogorov approaches. We advocate theuse of disintegrations, which are regular conditional distributions P  j T  t that
also satisfy natural concentration requirements of the form PfT 6 t j T  tg  0.
We borrow the term ``disintegration'' from the French to emphasize the extra
concentration property. The level of generality achievable by the disintegration
approach to conditioning is much higher than with elementary methods. The extra
requirements do sacri®ce slight generality compared with the abstract Kolmogorov
approach, but in the problems that we consider the generality is not missed. As
compensation, arguments using disintegrations tend to look and feel much closer tothe elementary arguments; by aiming for slightly less generality, we get to makestronger statements that come closer to the way that we tend to think intuitively
about conditioning.
Consider a typical example.
Example 1. The intuitive de®nition of suciency says that a statistic T is sucient fora family of probability measures P  fP :  2 Yg if the conditional distributions
given T do not depend on . The elementary approach is based on conditional
distributions, which work ®ne in the simplest discrete and absolutely continuous
settings, but are typically abandoned in rigorous treatments that aim for any moregenerality. As LEHMANN (1959, page 18) noted, there are some ``diculties concerning
the behavior of conditional probabilities'' that make a precise analysis delicate.
As an example, suppose P is the uniform distribution on the square 0; 2 , for an
unknown positive . The coordinate maps X and Y are independent Uniform 0; under P . The maximum, M, of X and Y is a sucient statistic. Given M  m, the# VVS, 1997

Conditioning as disintegration289

conditional distribution P  j M  m is uniformly distributed around two edges
where one of X or Y equals m and the other is smaller.
One could argue informally, by conditioning on fm  M  m  g and thenletting  tend to zero, to get the form of the conditional distribution. It is also easy tocheck the Radon±Nikodym property by direct calculation of probabilities, but wefeel that it is helpful to be able to think of the conditional distribution concentrated
around the two edges where M  m.
Frequently one sees suciency for this particular example demonstrated by anappeal to a factorization theorem for the joint density of X and Y. A diligent student
might be dismayed to learn that the form of that theorem needed in the present simple
case is beyond the scope of most textsÐtypically one is oered the proof for thesimple, discrete version of the theorem, with a suggestion to read about the generalcase in the thorough text of LEHMANN (1959). Even there, one might suspect that thesimple proof (page 19) for smooth continuous distributions might have small
problems with the non-dierentiability of the maximum function. To be reallyrigorous one seems forced to skip forward several sections (to page 46) to ®ndLehmann's treatment of the HALMOS and SAVAGE (1949) approach, based on Radon±
Nikodym derivatives.
Is it really that complicated? See Example 6.
&
It has long bothered us (and other authors, such as TJUR, 1974 and WINTER, 1979)
that there should be such a wide gap between intuition and rigor in conditioningarguments. We feel that, in many statistical problems, manipulation of theconditional probability distribution is the most intuitive way to proceed. However,
we mathematical statisticians are trained to treat such conditional distributions withgreat caution, being aware of the menagerie of nasty counterexamplesÐsuch as theBorel paradoxÐthat warn one away from conditional distributions. Apparently such
examples have left conditional distributions with a bad name. As KOLMOGOROV (1930,
page 51) put it, ``the concept of a conditional probability with regard to an isolated
hypothesis whose probability equals 0 is inadmissible.'' There is a technical diculty,
but it does not require us to abandon the notion of a conditional distribution. We feel
our profession may have overreacted to the diculties of placing conditioning on asound basis and, in so doing, given up too much of the power of intuition.
By way of a small amount of theory and a collection of illustrative examples, in thispaper we present a case that disintegrations are easy to manipulate and that they
recapture some of the intuition lost by the more abstract approach, allowing guiltfree manipulation of conditional distributions. Most of our mathematics is wellknown and well used in certain areas of probability theory, such as Markov process
theory. The disintegration property is essentially the assertion of the Decomposition
Theorem in Section 29.2 of LOEÁ VE (1978), or of Theorem 6 in Section 2.5 of LEHMANN(1959). (For further references see Section 5.) Nevertheless, it seems to us that theideas are not as widely known or used as they should be, which is our reason forcollecting together some of the facts we might easily have learned in graduate school,# VVS, 1997

290

J. T. Chang and D. Pollard

but didn't. We suggest that the concept of disintegration should be part of theeducation of every young probabilist and mathematical statistician.
In Section 2 we outline some theory for disintegrations, which we apply to acollection of conditioning examples in Section 3. We would suggest that the reader
might contemplate how one usually attacks these problems, before looking at our
explanations. We were all too often surprised and embarrassed by how muchdiculty we were having using traditional methods as a ®rst pass on the problems
during the drafting of the paper.
With some slight trepidationÐwe fear some readers might take fright at theabsence of integral signsÐwe have chosen to use notation that we have found mostconvenient and most helpful to our understanding.
 We adopt linear functional
notation for integrals, writing l f instead of fdl or  f xl dx. We also identify setswith their indicator functions: l fA instead of f x1fx 2 Agl dx. When wewant to identify explicitly the dummy variable of integrationÐfor example, when
integrating a function of more than one variableÐwe
do so by attaching a super
script to the measure: ly f x; y is the same as f x; yl dy.
We also adopt a slightly unusual notation for image measures. If T is a measurable
function from X ; A into T ; B, and if l is a measure on X ; A, we denote theimage measure of l under the map T by T l, or simply Tl. It is de®ned byTl g  l g  T 
for nonnegative measurable functions g on T ; B. If l is a point mass at x0 then Tl isa point mass at Tx0 . If g is the indicator function of a set B then g  T is the indicatorfunction of the inverse image T ÿ1 B, and TlB  l T ÿ1 B. That is, our Tl is thesame as the measure sometimes denoted by lT ÿ1 . If l is a probability measure, Tl isalso called the distribution of T under l.
2What is a disintegration?

In the elementary approach to conditioning, there are two ways to calculateconditional distributions. In the discrete case everything reduces to ratios of probabilities. For continuous distributions on Euclidean spaces (that is, distributions
absolutely continuous with respect to Lebesgue measure), with conditioning on the
projection onto a coordinate space, one calculates conditional densities by dividing
marginal densities into joint densities. Conditioning on other random variables(or vectors) presents some diculty when contemplated in any generality. Specialtransformations under extra smoothness assumptions are needed to reduce thecalculations to the special case.
In this section we will describe a method that covers both discrete and continuous
cases with equal ease. The same formulae appear in all cases. But ®rst let us consider
the elementary discrete case in more detail.
For discrete random variables conditioning is straightforward, as long as we heed
the admonition not to try to condition on events of probability zero. Suppose P is a
# VVS, 1997

291

Conditioning as disintegration

probability measure on X ; A. Suppose T takes values only in a ®nite subset R of T ,with PfT  tg > 0 for all t in R. The elementary de®nition has
P A j T  t 

PAfT  tg
for A  A and t  R
PfT  tg

Following KOLMOGOROV (1933), we will also use the more compact Pt(A) for
P A j T  t. The elementary de®nition enjoys the following pleasant properties.
(a)(b)Pt(.) is a probability measure on X ; A for all t 2 R.
The measure Pt concentrates on the set fT  tg:
Pt fT 6 tg (c)

PfT 6 tgfT  tg
0
PfT  tg

For A 2 A;
P A 

X

PfT  tgPt A

t2R

The decomposition in property (c) writes P as a weighted sum of the conditional
probability measures Pt for t in R, where the measure Pt concentrates on the level set
fT  tg. Notice that PfT  tg is the mass placed by the image measure T P at the
point t. In our notation, the averaging property in (c) is writtentPA  T P Pt A
We would like with equal assurance to be able to talk about and work withconditional probabilities of the form P A j T  t for more general spaces X ; A andmaps T. The standard Kolmogorov de®nition of conditional expectation has anaccounting problem: for each A 2 A the measurable function P A j T  t is free tobe de®ned arbitrarily on any set of probability zero, and as there are in general many
events A 2 A, those sets of probability zero could accumulate into a nonnegligible
set. Worse yet, however many events there may be, there are still more sequences of
events. For each such disjoint sequence A1 , A2 , . . . , we have relations likeP [An j T  t   P An j T  t

2

holding, at least almost surely, in the sense that there exists a set N  T for which
PfT 2 Ng  0 and for which (2) holds if t 62 N. The set N depends on the particularsequence {Ai}. Thus, the unpleasant prospect arises that there might be no t for which(2) holds simultaneously for all sequences A1 , A2 , . . . of disjoint events.
These considerations are the familiar motivation for introducing the concept of aregular conditional distribution. Under stronger assumptions than required for the
existence of Kolmogorov's conditional expectations, one can choose appropriate
versions of each P A j T  t, as a function of t, to make P  j T  t a probability
measure for (almost) all t. The slightly stronger notation of a disintegration also
requires P  j T  t to concentrate on the set fT  tg.
# VVS, 1997

292

J. T. Chang and D. Pollard

For the general de®nition of a disintegration we will consider not just probability
measures, but also measures (such as Lebesgue measure) that have in®nite total mass.
Let T be a measurable map from X ; A into T ; B. Let l be a sigma-®nite measure
on A and  be a sigma-®nite measure on B. Here l is the measure to be disintegrated
and  is often the image measure Tl, although, as we will see below, it is usefulto admit other possibilities for m, especially to cover cases where Tl is not sigma®nite.
Definition 1. We say that l has a disintegration flt g with respect to T and m, or aT; -disintegration, if:(i)lt is a sigma-®nite measure on A concentrated on fT  tg, that is,
lt fT 6 tg  0, for -almost all t;and, for each nonnegative measurable f on X :
(ii)
(iii)

t 7! lt f is measurable;l f  t lt f .

We will refer to the flt g as the disintegrating measures and to  as the mixing measure.
We will also write l  j T  t for lt  on occasion.
Requirement (i) is analogous to property (b) in the discrete case; requirement (iii) isthe analog of (c) generalized to functions. As de®ned by DELLACHERIE and MEYER(1978, page 78) the disintegrating measures flt g are required to be probability
measures, analogously to (a). However we ®nd that it is better to hold that property
in reserve, and allow more general disintegrating measures. (Purist disintegrators
might prefer us to invent yet another name.) As we will soon demonstrate, the lt canbe taken as probability measures if and only if the image measure Tl is sigma-®nite
and we take  to be that image measure. In that case we will speak of aT-disintegration, omitting explicit mention of .
When l and (almost) all the lt are probability measures we will also refer to thedisintegrating measures as (regular) conditional distributions or (regular) conditional
probabilities; we will usually write P and Pt , instead of l and lt , in this case. If X is a
P-integrable random variable, its expecation with respect to Pt is then a version of theconditional expectation P X j T  t. As shown in Section 6, the concentration
property (i) for conditional probabilities is a simple consequence of (1) when thesigma-®nite B contains all singleton sets and is countably generated. Thus a disintegration of a probability measures may be thought of as resulting from a carefulselection of versions of the conditional expectations (in Kolmogorov's sense), in a
way that eliminates awkward complications caused by uncountable families ofnegligible sets. Not surprisingly, as with many stochastic process problems involving
uncountable families of random variables, we need some extra (topological) assumptions about the underlying spaces and maps to ensure existence of the disintegration.
It might appear that, as a proper probabilist or mathematical statistician, one
should be interested only in the case where the disintegrating measures are# VVS, 1997

Conditioning as disintegration

293

probabilities. However, then one could not recover as a special case of a generaldisintegration result the elementary formula for calculating a conditional density(with respect to Lebesgue measure) as a ratio of a joint density to a marginal density.
It would also hamper the improper urges of Bayesians with their priors (seeExample 9).
Example 2. Suppose l is a product of two sigma-®nite measures, l   , on a
product space S T . Let T be the map that projects onto the T coordinate space.
For example, l might be Lebesgue measure on R2 and m might be Lebesgue measureon the x-axis.
Think of S ftg as a copy of S imbedded into the product space, and let lt be 
living on that copy. With a mild abuse of notation we will write lt  . (Moreformally, let lt be the image of l under the map s 7! s; t, for t ®xed.) Then Fubini's
theorem implies that flt g is a T;  disintegration of l. As in the case of Lebesgue
measure on R2 , the image measure Tl is not sigma-®nite unless  is a ®nite measure.
So it is handy that the de®nition of a T; -disintegration does not require m to be theimage measure Tl. Moreover, there is no way to get a disintegration with almost all&
lt probability measures if  is not ®nite.
One grudge held against disintegrations concerns existence. The abstract
Kolmogorov approach to conditioning requires only pure measure theory; disintegrations, in general, are tainted by topological requirements, but they deliver more interms of natural and useful properties. There is the usual trade-o: stronger
requirements give stronger properties. We believe that the extra generality sacri®cedby restricting to situations in which disintegrations exist will not be missed in manystatistical applications.
We have found the following version of the existence theorem quite adequate, eventhough it is not the most general possible. We require that l be a Radon measure(also known as a tight measure) on a metric space. That is, l is a Borel measure forwhich lK < 1 for each compact K and lB  supK  B lK, the supremum being takenover compact sets, for each Borel set B. For example, a ®nite Borel measure on acomplete, separable metric space is RadonÐsee Theorem 1.4 of BILLINGSLEY (1968).
Theorem 1. (Existence Theorem) Let l be a sigma-®nite Radon measure on a metricspace X and let T be a measurable map from X into T ; B. Let  be a sigma-®nite
measure on B that dominates the image measure Tl. If B is countably generated andcontains all the singleton sets {t}, then l has a T; -disintegration. The lt measures are
uniquely determined up to an almost sure equivalence: if flt g is another T; disintegration then ft 2 T : lt 6 lt g  0:
Notice that the uniqueness assertion is much stronger than the almost sureuniqueness of P X j T  t for each integrable X in the Kolmogorov approach toconditioning. It requires existence of a single -negligible set N such that lt A  lt A
for all t 2= N and all Borel sets A.
# VVS, 1997

294

J. T. Chang and D. Pollard

The proof of existence is just dicult enough to intimidate the typical graduate
student, even though versions of it appear in many texts. We sketch a proof in the
Appendix, to make the point that, with the possible exception of one topological/
measure-theoretic fact, the argument is within the reach of most graduate probability
courses.
On occasion, one works only with conditional probabilities of events involving
another measurable map S into a space S; C. In such a case one needs thedisintegrating measures de®ned only on the sigma-®eld A0 on X generated by the
map C  S; T  into S T . If the image measure C l has a disintegration ft g withrespect to the coordinate projection onto T and the measure m, and if the complement
C X c of the range of C has zero outer C l measure, then the disintegrating
measures can be pulled back to A0 using the de®nition t  C lt . Compare withLOEÁ VE (1978, Section 30.2). It is easy to see that C X c necessarily has zero innermeasure. If it is not in the product sigma-®eld there might be some diculty inarguing for zero outer measure. If l were a Radon measure the set would have zero
outer measure, but in that case why would one want to settle for less than the full
disintegration for l?
A few simple facts about disintegrations make them easy to work with. First let us
be precise about when the disintegrating measures are probabilities. In essence, to get
conditional probabilities one has only to standardize the disintegrating measures. The
only subtlety is that standardization cannot work on a set of in®nite or zero measure.
Theorem 2. Let l have a T; -disintegration flt g, with l and  each sigma-®nite.
(i)
(ii)
(iii)
(iv)

The image measure Tl is absolutely continuous with respect to , with density
lt X .
The measures flt g are ®nite for -almost all t if and only if Tl is sigma-®nite.
The measures flt g are probabilities for -almost all t if and only if   Tl.
If Tl is sigma-®nite then Tlflt X  0g  0 and Tlflt X  1g  0: For
Tl-almost all t, the measureslt ~
f0 < lt X < 1glt  lt X
are probabilities that give a T-disintegration of l.

Proof. We abbreviate ``for -almost all t'' to ``mod '', and write ` t for the total
mass, lt X , of lt . For nonnegative measurable g,
x

t x

t

Tlg  l g Tx   lt g Tx   g t` t

3

As a service to readers who may still be getting used to our notation, we could write
the last equalities asg t Tl dt  g Txl dx # VVS, 1997

g Txlt dx dt  g t` t dt295

Conditioning as disintegration

The simpli®cation in the last equality occurs because g Tx  g t for lt-almost all x,
and that g(t) can be brought outside the innermost integral as a constantÐexactly
what intuition says conditional distributions should allow.
For (i): If g  0 and g  0 then g t  0 mod , whence g t` t  0 mod . In
particular, every -negligible set is also Tl-negligible. Equation (3) is the formal
statement that ` t is the density.
For (ii): Sigma-®niteness of a measure is equivalent to the existence of a strictly
positive real-valued function with a ®nite integral. In particular, there exists an h > 0
for which h < 1. If ` t < 1 mod , the function g t  h t= 1  ` t is strictly
positive mod  and Tlg  h < 1, which makes Tl sigma-®nite. Conversely, if
Tlk < 1 for some strictly positive k then k t` t < 1 mod , by (i), which gives®niteness of ` t mod .
For (iii): If ` t  1 mod  then equation (3) shows that Tlg  g. By assumption,  is always sigma-®nite. For the converse, let h be strictly positive with h < 1
as in the previous paragraph. Choosing g t  h tf` t < 1g in (3) and using theassumption that Tl   givestt

1 >  h tf` t < 1g   h tf` t < 1g` twhich implies f` < 1g  0. A similar argument shows that f` > 1g  0.
For (iv): From (ii) we have fl  1g  0, so that (i) gives Tlfl  1g  0. Takeg t  f` t  0g in (3) to show that Tlfl  0g  0. For nonnegative measurable f,
we then havetl f   lt f

lt f   f` t  0glt f    f` t  1glt f   ` t~t~
 Tl l f  0  0
t

t

t

t

The second term is zero because lt is the zero measure when ` t  0. The third term
is zero because f`  1g  0:&
Caution! The result in part (i) can be most misleading when the image measure Tl
is not sigma-®nite. For example, if T projects Lebesgue measure l on R2 onto acoordinate axis, the image measure is not sigma-®nite; it gives in®nite measure to
every set of nonzero one-dimensional Lebesgue measure. In one sense the function
lt R2  1 is the correct Radon±Nikodym density, but the integration theory for suchan extremely in®nite measure is delicate and of little use; every set has image measure
either zero or in®nity. It would perhaps be better to insist that a density be ®nite
almost everywhere, to avoid bad measures of this type. Only when Tl is sigma-®nite
can it sensibly be used as the mixing measure . (The reader should exercise similar
caution when interpreting part (ii) of the next Theorem.)
Notice that the construction for part (iv) can be applied more generally. If m isdominated by a measure  with a ®nite density d=d  m t, then l has a (T, )disintegration {Lt} given by Lt f  m tlt f , because l f  t lt f   t m tlt f .
# VVS, 1997

296

J. T. Chang and D. Pollard

Much of the convenience of working with disintegrations comes from the way they®t nicely with image measures and densities. Most of the following results are easy
consequences of the special case treated by Theorem 2. We state them in full for
future reference. See HOFFMANN-JéRGENSEN (1994, Section 10.11) for similar
assertions for probability measures, proved in more traditional notation.
Theorem 3. Let l have a T; -disintegration {lt }, and let  be absolutely continuouswith respect to l with a ®nite density r x, with each of l; , and  sigma-®nite.
(i)
(ii)
(iii)
(iv)
(v)

The measure  has a T; m-disintegration {t} where each t is dominated by
the corresponding lt , with density r x.
The image measure T is absolutely continuous with respect to , with density
ltr.
The measures {t } are ®nite for  almost all t if and only if T is sigma-®nite.
The measures {t } are probabilities for  almost all t if and only if   T.
If T is sigma-®nite then Tflt r  0g  0 and Tflt r  1g  0. For Talmost all t, the measures de®ned by
~t f  lt fr
f0 < lt r < 1g
lt r

4

are probabilities that give a T-disintegration of .
Proof. For (i) note that f  l rf   t lt rf . The other assertions follow from
Theorem 2 via the equality t X  lt r:&
The ~t measures in part (v) are just the t of part (i) standardized to be probability
measures, on the set f0 < lt r < 1g where standardization is possible. The complement of that set has zero T measure, so it wouldn't matter if we changed thede®nition of ~t there. The disintegrating measures can be changed arbitrarily on aT-negligible set without disturbing the disintegration.
The simple formula (4) is the general version of the familiar method for calculating
conditional densities as a ratio of joint density to marginal density. It is more usefulthan the familiar formula because it does not require the conditioning variable to be acoordinate projection on a Euclidean space with Lebesgue measure playing the roleof l. LEHMANN (1959, Chapter 2, Lemma 6) used a special case of (4) in his treatmentof exponential families.
Example 3. Suppose P is a probability on Rk Rnÿk with density p(x, y) with respect
to Lebesgue measure. Disintegrate the dominating Lebesgue measure l on Rn as in
Example 2. Writing X for the projection onto Rk , we have disintegrating probability
measures Px with (conditional) densities
p x; y=lyx p x; y  

p x; yp x; y 0 dy 0

with respect to Lebesgue measure on Rk , as taught in undergraduate classes.
# VVS, 1997

Conditioning as disintegration

297

Much theory in the statistical literature is based on this special case. One supposesthat each member of a family fP :  2 Yg is a probability on Rn and that T maps Rnsmoothly into a lower-dimensional space Rk . One assumes existence of anothersmooth map S from Rn into Rn7k such that  x  T x; S x is smoothly invertible. Inferences on the family fP :  2 Yg should then be equivalent to inferences on
the family of image measures fP :  2 Yg, for which there are densities with respect
to Lebesgue measure on Rn in its role as the range space,q s; t;   p ÿ1s; t; j s; t5Here j(s, t) involves the Jacobian of the transformation . The conditioning variable
is now one of the coordinate projections, for which the conditional density can becalculated as the ratio of joint to marginal densities,

q s; t; q s0 ; t; ds0

We have three qualms about this method for continuous distributions. First, itapplies only to densities on Euclidean spaces. Second, it requires invention of an
auxiliary map S that need be of no particular interest except that it builds theinteresting T into a one-to-one transform of the dataÐone needs to force theconditioning variable to be a coordinate projection on a Euclidean space. Third, it
requires extraneous smoothness assumptions about the conditioning map T, in orderthat the image measure might be absolutely continuous with respect to Lebesgue
measure. As Theorem 3 shows, one needs none of these restrictive assumptions inorder to derive a conditional density analogous to the ratio of joint to marginaldensities. It is merely a matter of making a proper choice for the measure to use whencalculating the ``marginal'' density.
&
Many facts about abstract conditional expectations have analogs for disintegrations that make slightly stronger assertions under slightly more restrictive circumstances. We present just one example.
Conditional expectations given sigma-®elds have the nesting property
P P X j F 1  j F 0   P X j F 0  when F 0  F 1
There is an analogous formula for disintegrations, which corresponds to the idea oftaking conditional expectations over the variables that are discarded in pulling backto the coarser sigma-®eld.
Example 4. Suppose l is a sigma-®nite measure on X ; A with a (T, )-disintegration {lt}, for a sigma-®nite  on T ; B, which in turn has a (S, )-disintegration{s } for a sigma-®nite  on S; C. Here T is a measurable map from X into T and S isa measurable map from T into S. Their composition S  T is a measurable map from
X into S.
# VVS, 1997

298

J. T. Chang and D. Pollard

The measure l has an S  T; -disintegration {gs} given symbolically by gs  ts lt .
One averages the lt disintegrations over all level sets that S maps onto s. That gs has
the right averaging property follows fromtstlf   lt f    s lt f 

6

That it concentrates on the right level sets follows from the concentration propertiesof the other two disintegrations:
s x

s t

x

 gs fS Tx 6 sg   s fSt  sglt fTx  tgfS Tx 6 sg  s ts lxt fSt  s; Tx  t; S Tx 6 sg
0because the region of integration is an empty subset of X T S.
Sigma-®niteness of l implies existence of a strictly positive f for which lf is ®nite.
Equality (6) then gives ®niteness of ts lt f for -almost all s; the measure gs is sigma®nite for  almost all s.
&
3

ExamplesIn this section we present a small collection of examples that shows some of thebene®ts of treating conditioning as a matter of disintegration.
We start (Example 5) with the EM-algorithm, where it seems that one has to work
explicitly with the conditional probability measure for a particular realization of astatistic. We set aside worries that the realization might fall in the negligible set wherea meddling probabilist might decide to change the disintegrating measure. Once theconditional measure is ®xed, the conditioning interpretation plays no further role in
the analysisÐour ®rst example of conditioning has very little to do withconditioning.
We next turn to the Factorization Theorem for sucient statistics (Example 6), a
topic that ®rst got us seriously interested in a more satisfactory way to work withconditioning. Most textbooks make it clear that the general version of the theorem ismuch too hard for general discussion. We feel the diculty diminishes when one
thinks of conditioning as disintegration.
The third example (Example 7) shows how the disintegrating measures can inherit
invariance properties under a group of transformations.
The fourth example (Example 8) proves the converse of Basu's theorem aboutancillary statistics. The proof is easy. It helped us to understand the need forsomething beyond independence from the sucient statistic when we saw that the
distributions concentrate on level sets de®ned by the disintegration.
The ®fth example (Example 9) should be common knowledge to Bayesians, who
know that posterior distributions are probability measures and not just collections ofmeasurable functions that almost hang together in the right way. Their posteriors are
disintegrating measures. To make life more interesting, we allow improper priors,with a reminder that even nonBayesians make use of Bayes estimators that guarantee# VVS, 1997

Conditioning as disintegration

299

admissibility. One has only to be careful about in®nite expectations at awkward
moments.
The sixth example (Example 10) is an elementary Bayesian problem concerning theposterior distribution for a probability concentrated on two lines. We ®rst present a
non-rigorous, elementary method of solution, which we suspect would be theinstinctive approach of most mathematical statisticians. (It was certainly how weinitially solved the problem.) We then show how an even more general problem
almost solves itself when properly framed: a small disappointment for anyone bent on
demonstrating superiority of disintegrations, perhaps, but a genuine example of amethod of solution that hadn't occurred to us before we started writing this paper.
We recommend that our readers provide their own complete, rigorous solutions
before looking at what we come up with.
Examples 11 and 12 come as a pair. They describe a marginalization paradox of
STONE and DAWID (1972) that can aict Bayesians with improper priors. We end up
agreeing with Hartigan (1983, page 29), who pointed out the dangers in calculating
marginal distributions by integration over unwanted variables. One must be carefulwhen interpreting independence when probabilities are not ®nite.
In Example 13 we present a disintegration interpretation of the Gibbs sampler.
We could cite many stochastic process examples where the disintegration approachsheds light on complicated conditioning arguments. In an initial version of this paper
we included one such applicationÐthe proof of continuity for the sample paths ofmartingales adapted to a Brownian ®ltrationÐand a referee pointed out otherapplications (interpretation of the strong Markov property; re¯ection principle forBrownian motion). For the sake of brevity, we decided to omit those examples fromthe ®nal version, after realizing that stochastic process experts are unlikely to needfurther reminder of the advantages of working with regular conditional distributions
or disintegrations.
Example 5. The EM algorithm is often presented as a technique of maximum
likelihood estimation for problems with missing data. For example, LITTLE andRUBIN (1987, page 127) describe it in the following way:
Suppose as before that we have a model for the complete data Y, withassociated density f Y j  indexed by an unknown parameter . We
write Y  Yobs ; Ymis  where Yobs represents the observed part of Y andYmis denotes the missing values. In this chapter we assume for simplicity
that the data are [missing at random] and that the objective is tomaximize the likelihood
L  j Yobs   f Yobs ; Ymis j dYmis
with respect to .
The dYmis here has presumably the symbolic meaning of whatever averaging isnecessary to obtain the marginal density of Yobs . The measure corresponding to# VVS, 1997

300

J. T. Chang and D. Pollard

dYmis would be Lebesgue measure if f Yobs ; Ymis j  were interpreted as a densitywith respect to Lebesgue measure on a product of Euclidean spaces. (A similarinterpretation is needed for the dx at the top of page 96 of the WU (1983) paper.)In situations where the observed data are given as some arbitrary function of Y,
one must concoct a Ymis so that the pair (Yobs , Ymis) becomes a one-to-one functionof Y. The density for Y then transforms into a joint density for (Yobs , Ymis), in muchthe same way as in Example 3, and then the problem ®ts into a framework whereconditioning can be handled by elementary means.
We would argue that in problems where data are naturally modelled as a function
T(x) on a probability space X ; A; P  it is an unnecessary arti®ce to invent a missing
function merely to accommodate EM theory to elementary methods of conditioning.
One should instead start from a family of probability densities fp x;  :  2 Yg withrespect to a sigma-®nite measure l, which has a disintegration flt g with respect to(T, ). The image measure has densityx t;   lt p x; with respect to . For a given t, the maximum likelihood method seeks a  tomaximize  t; .
More generally, one could consider the problem of maximizing a functionx

G   g g x; 
where fg x;  :  2 Yg is a family of positive functions, and g is a sigma-®nite
measure for which 0 < G  < 1 for each .
The generalized EM algorithm consists of repeated application of two steps that
improve upon an initial guess 0 for the value maximizing G. Let Q be the probability
measure with density
q x;   g x; =G with respect to g. In the E-step one calculates the expectationx

L0   Q log g x; 0In the M-step one maximizes L0 , or at least ®nds a 1 for which
L0 1  > L0 0 
The two steps are guaranteed to giveG  1  > G 0 
because0 < L0 1  ÿ L0 0 q x; 1 G 1 
 Qx log0q x; 0 G 0 
q x; 0 
G 1 
x
ÿ g q x; 0 log
 log
G 0 
q x; 1 # VVS, 1997

Conditioning as disintegration

301

The last term is the Kullback±Leibler distance between Q and Q , which is positive0
1
by Jensen's inequality.
&
One then repeats the two steps, with 1 taking over the role of 0 . And so on.
The next example is the perfect illustration of how a disintegration proof can bebuilt by analogy with simple arguments for a discrete case. The exposition is slightly
more complicated than we would have liked, because we chose not to ignore some
subtleties concerning division by zero. (The reader might care to ponder how thesesubtleties are usually taken care of in textbook proofs for the discrete case.) We havebeen told that a proof similar to ours appears in a book of Borovkov, but we have notyet been able to ®nd that book.
Example 6. The intuitive de®nition of suciency says that a statistic T is sucient
for a family of probability measures P  fP :  2 Yg if the conditional distributions
given T do not depend on . We avoid technical ``diculties concerning the behavior
of conditional probabilities'' by interpreting the de®nition to mean existence of ashared disintegration {Pt}. That is, Pt  should serve as a conditional distributionP  j T  t for every .
Most often one checks for suciency by means of a factorization criterion, whose
general proof has a forbidding reputation. LEHMANN (1959) approached the proof ina most sensible manner, by discussing ®rst the discrete version, then a special case of
the continuous version (using methods based on the transformation idea described in
Example 3), and ®nally presenting (Section 2.6) the full-blown Radon±Nikodym
approach of HALMOS and SAVAGE (1949) only after careful measure-theoretic
preparation. We were struck by the dierences between the proofs in the various
cases. The disintegration interpretation allows us to use the same idea for all cases.
Consider ®rst the proof that factorization implies suciency in the discrete case.
Here each P is de®ned on a ®nite set X with probabilities that factorize as
p x;   P fxg  g Tx; h x
for some statistic T. The conditional expectations are then obtained as simple ratios.
For ®xed t,
P f j T  t Tx  t g Tx; h xf x Tx  t h xf xTx  t g Tx; h x
Tx  t h xThe factors involving g in the numerator and denominator have cancelled out,
leaving a ratio that does not depend on . (Might there be any problem with 0/0 here?)
The last formula has a simple disintegration interpretation. Let us regard p(x, ) as
the density of P with respect to counting measure l on X. With  as counting
measure on T , the T; -disintegration of l has lt equal to counting measure onfT  tg. The last displayed ratio is just the expectation
Pt f  # VVS, 1997

lt f hf0 < lt h < 1g
lt h

302

J. T. Chang and D. Pollard

We have included the explicit indicator function to avoid one 0/0 problem. The upperbound on lth is automatic for the case of a ®nite set X , but is needed already forcountably in®nite sets.
Now consider the general case, where P has density g Tx; h x with respect to a
general sigma-®nite measure l. Suppose l has a (T; )-disintegration {lt} for somesigma-®nite . For a ®xed , Theorem 3 shows that P has a T-disintegration fP;t g,
whereP;t f  

lxt g Tx; h xf x
x
f0 < lt g Tx; h x < 1g
lxt g Tx; h x

By parts (ii) and (v) of the same Theorem and the concentration property of the {lt},x0 < lt g Tx; h x  g t; lt h < 1 for T P -almost all t
We are therefore almost everywhere justi®ed in cancelling out a g(t, ) factor fromnumerator and denominator to getP;t   Pt  for T P -almost all twhere Pt  is de®ned by (7), just as in the discrete case except for the changed
meaning of lt . The disintegration property is unaected if we change the disintegrating measures for a T P -negligible set of t. The {Pt} also de®ne a T-disintegration
for P , as required for suciency.
For the converse it is useful to replace l by a dominating probability measure of theform P  i 2ÿi P , for some countable subfamily fP g of P. (A device due toii
HALMOS and SAVAGE, 1949Ðsee Theorem 2 in the Appendix of LEHMANN, 1959).
What matters is that the common disintegrating probabilities {Pt} for each P also
provide a T-disintegration for P, becauset

T P Pt f X ÿit2 T P  Pt f  Pfii

If we write g(t, ) for the density of T P with respect to T P, we havetdefinition of fPt g

 T P g t; Pt fx
 P g Tx; f x

definition of g t; P f  T P  Pt ftthe last equality holding because {Pt} is also a disintegration for P. Thus P has
density g(Tx, ) with respect to P, and density g Tx; d P=dl with respect to l:&Sometimes the disintegration can be identi®ed by an appeal to symmetry, or to an
invariance argument, with the uniqueness of disintegrations simplifying the formal
proof.
# VVS, 1997

303

Conditioning as disintegration

Example 7. Let P be a probability measure on a space X . Suppose a probability
measure P is invariant under a group G of transformations on X . That is, gP  P for
all g in G. Suppose also that the sets fT  tg are invariant under G. Does it follow thatthe conditional distributions are also invariant under G ?
For example, the standard normal distribution on R2 is invariant under rotations
about the origin. The statistic T x1 ; x2   x21  x22 is constant on circles centered at
the origin, sets that are invariant under rotations. The conditional distributions are
uniform around the circles, a fact that it usually demonstrated by means of acalculation with Jacobians. In higher dimensions the argument becomes quite messy.
How much easier it would be if we could deduce the form the conditional distributions directly from invariance considerations.
The argument succeeds if G can be replaced by a countable subclass G0 . Suppose
measures invariant under G0 are necessarily also invariant under the whole of G. Thenif P is invariant under G, the conditional distributions Pt must also be invariant underG, for TP-almost all t. The proof depends on the uniqueness of disintegrations.
For each bounded measurable f on X , and each g in G,
Pf  gPf

invariance of P

 P f  gde®nition of image measure gPt TP Pt f  g

disintegrationt TP gPt  fde®nition of image measure gPt

When Pt concentrates on the set fT  tg, so does gPt . It follows that fgPt : t 2 T g isanother disintegration for P. By uniqueness of disintegrations, there exists a TPc
negligible set N g such that gPt  Pt for all t in N g . Cast out a sequence of negligiblesetsÐone for each member of G0 Ðto deduce that, for TP-almost all t, the&probability measure Pt is invariant under G0 , and hence invariant under G:
The Borel paradox (KOLMOGOROV, 1993, page 50) is the classic example of an
unjusti®able appeal to invariance for the construction of conditional distributions.
POLLARD (1996, Chapter 5) has explained the source of the diculty, using thelanguage of disintegration.
Example 8. Suppose T is sucient for P  fP :  2 Yg on O, with disintegrating
measures {Pt}. Let f and g be bounded real functions on R, and let S be anotherstatistic. De®ne G t  Pt g S. Thent

P f T g S  T P  f tPt g S  P f T G T 

8

In particular, g(S) and G(T) have the same P expectation, C().
If S is ancillary, then the expected value C() is equal to a constant C. If T isboundedly complete, then the assertion P G T  C implies that P concentrates on
fG T  Cg for each , whenceP f T G T   P f T   C  P f T P g S# VVS, 1997

304

J. T. Chang and D. Pollard

It follows that S is independent of T. That is the BASU (1955, 1958) theorem.
Conversely, if S is independent of T under each P , then by choosing f  G in (8)we get2

P G T   P G T 2

so that P concentrates on the level set O   fG T   C g. If there were 0 and 1
for which C 0  6 C 1 , we would have a partition of Y into two nonempty
subfamilies,
Y0  f : P concentrated on O 0 gc
Y1  f : P concentrated on O 0  gwith the corresponding families of probability measures supported by disjoint subsets
of O. If such a partition of Y is assumed impossible then C() must be a constant, that
is, S must be ancillary. That is the converse to Basu's theorem.
&
Basu's results can be proved without the use of disintegrations. For us, the advantage of the proof with disintegrations is clean de®nition of the two sets Y0 and Y1 .
Bayesians work with conditional distributions, by choice. Decision theorists often
apply Bayesian arguments. The next Example broadens slightly the scope of avenerable admissibility argument, as used by EATON (1992), for example, by removing
unnecessary sigma-®niteness assumptions. Our approach is based on an idea
explained to us by John Hartigan.
Example 9. Let fPt : t 2 T g be a family of probability measures on X . If the map
t 7! Pt f is measurable for nonnegative measurable f on X , and if  is a probability(a prior distribution) on T , then a probability measure Q can be de®ned on X T by
Qg  t Pxt g x; t
The coordinate maps X and T have joint distribution Q. The fPt g have theinterpretation of a T-disintegration of Q, that is, the conditional distribution of X
given T  t is Pt . The X-disintegration of Q de®nes the Bayesian posterior distribution Qx   Q  j X  x.
If each Pt has a density p x; t with respect to a sigma-®nite  on X, thent x

Qg    p x; tg x; tThat is, Q has density p(x, t) with respect to the product measure 
measure has the trivial disintegration. The productx  ;if we abuse notation as in Example 2. It follows from Example 3 that Qx has densitytp x; t= p x; twith respect to .
# VVS, 1997

9

305

Conditioning as disintegration

Given a nonnegative loss function L t; d  on T T , a Bayes estimator (x) can bede®ned by the value of that minimizes the posterior expected loss Q tx L t; ,tt

Qx L t;  x  inf Qx L t;  for each x

10

Even nonBayesians are interested in such estimators because they enjoy a number of
nice decision theoretic properties. For example, suppose  has ®nite Bayes risk
t

x

x

t

QL t;  x   Pt L t;  x   Qx L t;  x < 1
where  stands for the marginal distribution of X under Q. Suppose also that  (x) is
another estimator with smaller expected loss,
Pxt L t;  x  Pxt L t;  x < 1 for -almost all t

11

Then strict inequality can hold only on a -negligible set, for otherwisetx

0 >  Pt L t;  x ÿ L t;  x12

  x Qtx L t;  x ÿ L t;  xThe de®ning property of (x) requires the last integrand to be everywhere nonnegative, which gives a contradiction to the inequality (12).
The preceding argument has little to do with  or any of the disintegrating {Qx}being probability measures, nor with  being the marginal X distribution. It is valid
for any (X, ) disintegration and any sigma-®nite  (an improper prior), provided theBayes estimator de®ned by equality (10) has ®nite Bayes risk QL(t, (x)).
For example, if Pt is the Bin(n, t) distribution and  is the improper prior withdensity tÿ1 1 ÿ tÿ1 with respect to Lebesgue measure l on (0, 1), then Q has density
 
n xÿ1
1 ÿ tnÿxÿ1p x; t 
txwith respect to l , where m is counting measure on {0, 1, . . . , n}. The marginal
measure   XQ is not sigma-®nite; it puts in®nite mass at 0 and at n. Nevertheless, Q
has an X; -disintegration with Qx having density p(x, t) with respect to l. Noticethat Qx is a ®nite measure for 1  x  n ÿ 1, and both Q0 and Qn are in®nite (butsigma-®nite).
Let L t; d   t ÿ d 2 . For 1  x  n ÿ 1 the usual argument shows that theestimator  x  x=n minimizes the posterior expected loss. It also minimizes
1t
Q0 L t;2 ÿ1tÿ  t

nÿ1

1 ÿ t

dt

0

for the trivial reason that the integral is ®nite only when
# VVS, 1997

equals zero. Similar trivial

306

J. T. Chang and D. Pollard

reasoning applies to Qn . The estimator  is Bayes for the improper prior , with a®nite Bayes risk,
1 
n
X
n xÿ1
nÿxÿ1
2
t
1 ÿ t
t ÿ x=n dt < 1xx00

The inequality corresponding to (11) could hold only on a -negligible set. As bothsides of that inequality would be polynomials in t, the negligible set would have to be
empty: a contradiction. The Bayes estimator is admissible for quadratic loss.
&
Example 10. Suppose a distribution P on R2 concentrates on two straight lines, L1and L2 , neither of them orthogonal to the x-axis. Suppose the total mass pi that P
assigns to Li is distributed according to a density gi with respect to Lebesgue measure
along the line. An observation (X; Y ) is taken from P giving a point with X  x0 .
What is the conditional probability that the point lies on the line L1?
The elementary method approximates fX  x0 g by fx0  X  x0  g, for a small
positive , then argues that
P X; Y 2 L1 ; x0  X  x0  
P x0  X  x0  
p1 1 g1 x0 

p1 1 g1 x0   p2 2 g2 x0 

P X; Y  2 L1 j X  x0  

where 1/ i is the absolute value of the cosine of the angle between Li and the x-axis.
The small  factors cancel out, leaving an equality
P X; Y  2 L1 j X  x0  

p1 1 g1 x0 
p1 1 g1 x0   p2 2 g2 x0 

in the limit.
Write H1(x0) for the last ratio. If one wants a totally rigorous derivation using theKolmogorov approach, one can easily check the de®ning property analogous to (1),
Pf X; Y  2 L1 gfX 2 Bg  PH1 X fX 2 Bg
for all Borel sets B. Alternatively, one might appeal to some sort of abstract
dierentiation theorem to guarantee existence of the limiting ratio and justify its
interpretation as a conditional probability.
Both rigorous derivations would obscure the simple form of the conditional
probability distribution Pf j X  x0 g, which puts mass H1(x0) at the point where L1
intersects the line where X  x0 , and mass 1 ÿ H1 x0  at the corresponding point on
L2 . Provided (X; Y ) does not land at the intersection of the two lines, this conditional
probability distribution gives the asserted mass to line L1 . We prefer an argumentthat identi®es the conditional distribution directly, rather than have it emerge
indirectly from a calculation of uncertain rigor.
# VVS, 1997

307

Conditioning as disintegration

To determine the X-disintegration of P we need ®rst to be precise about what wemean by a distribution with a density with respect to Lebesgue measure along a line in
R2 . Bring everything back to the x-axis X , by regarding Lebesgue measure along Li as
the image of i times Lebesgue measure  along X . The geometry of the lines enters
only through the i factors. The measure with density pi gi  with respect to Lebesgue
measure on Li is just the image of the measure i with density hi x  i pi gi x withrespect to . Now we can forget all about lines and Lebesgue measure, and solve amore general problem.
Suppose  is a sigma-®nite measure on X and that h1 , . . . , hk are nonnegative
integrable functions on X. Let 1 , . . . , k be measurable maps into another space Y.
Let i be the ®nite measure with density hi with respect to . Let Qi be the image of iunder the map i that takes x onto x; i x. That is, Qi is the result of sliding i tolive on the graph of i in the product space X Y. De®ne P to be the sum of the Qi .
What is the conditional distribution P  j X  x?
Formally,
Pf 

k
X
i 1

i i  f  

k
X
i 1

x

 hi xf x;

i x

13

Taking the  outside the last sum we immediately get a representation of P as an
X; -disintegration,x

Pf   Px fwhere Px is the measure that puts mass hi(x) at the points (x, i(x)) for i  1; . . . ; k.
Notice that Px is not a probability, but it does live on the set fX  xg. To make thedisintegrating measures probabilities, we need to standardize as prescribed by
Theorem 2. For i  1; . . . ; k the conditional probability measure P  j X  x) (that is,
the X-disintegrating measure) puts mass hi x= h1 x      hk x, except at thenegligible set of x values where the denominator is zero.
&
The result from the previous Example is a solution to a Bayesian problem posed to
us by John Hartigan. LE CAM (1986, page 477) has used an analogous disintegration
to establish a bound on Hellinger anities for convex hulls. With reference to thisresult, DONOHO and LIU (1991, page 644) remarked that ``Le Cam has established afact which seems, at ®rst, quite similar to . . . but is in fact far deeper''. The case of®nite convex combinations is a simple consequence of an identity like (13); the generalcase is a consequence of a general disintegration.
For a measure l on a product space X Y it is traditional to use the nameX -marginal for the image of l under the map X that projects onto the X coordinate
space. If l happens to be a product of probability measures, P , the X -marginal
equals P. One can safely refer to both Pfx 2 X : x 2 Ag and P f x; y 2
X Y : x 2 Ag as ``the probability that X lies in the set A''. However, if  is not a
probability measure, the X-marginal of l does not equal P. At worst,  might not# VVS, 1997

308

J. T. Chang and D. Pollard

even be a ®nite measure, in which case the image measure assigns mass 1 to every A
with PA > 0. In this situation there is real danger in thinking of the X - andY-coordinates as being independent, or even in thinking of P as the distribution of X.
Bayesians with a penchant for improper priors should be particularly aware of thisproblem.
Example 11. Suppose (X; Y ) has strictly positive probability density f(x, y) withrespect to Lebesgue measure on the unit square0; 1
0; 1. Then, in traditional
notation, X has marginal density fX x  10 f x; y dy and the conditional distribution of Y given X  x has conditional density fYjX y j x  f x; y=fX x. Given
X  x and Y  y, let the Z distribution be the constant multiple 1/f(x, y) times
Lebesgue measure on R. The joint (improper) distribution of (X, Y, Z) is equal tothree-dimensional Lebesgue measure l on 0; 1
0; 1 R.
With l expressed as a product of Lebesgue measure on each coordinate space, wemight be tempted to think of X, Y, and Z as independent, each uniformly distributed.
Indeed, for a product of proper probability measures, the coordinate maps are
independent random variables with those probabilities as their marginal distributions. However, our Example involves products of improper distributions: the Z
marginal is Lebesgue measureÐthe improper uniform distribution on the real lineÐ
and conditional on Z  z, the pair X; Y  is uniformly distributed on the square.
(That is, we have a Z-disintegration of l with Lebesgue measure on the unit square as
disintegrating measure.) Since the last conditional distribution does not depend on z,
we might conclude that (X; Y ) is uniform on the square, so that X and Y are
independent and uniform on (0, 1). Or should we use the (X; Y )-marginal measure,which is very in®nite, as the joint distribution? Or should we stick with the original
f x; y density?
Is there any paradox in X; Y  appearing to have several dierent joint
distributions? We think not.
The confusion arises because Lebesgue measure on 0; 1
0; 1 R can bedisintegrated in many dierent ways. The X; Y -image measure is not sigma-®nite; itcannot be used as the mixing measure in an X; Y -disintegration. With the image
measure no longer a candidate, there are many equally plausible mixing measures anddisintegrations, giving many dierent plausible answers for the joint distribution of X
and Y and for the conditional distribution of Y given X.
When one works only with probability measures, all arguments lead back to thesame joint (marginal) distributions. With in®nite measures, dierent derivations canlead to dierent measures. One should exercise some care in bestowing the title ofjoint distribution.
&
The rather obvious sort of distinction in the last Example can become much morepuzzling when buried within more complicated collections of marginal and conditional distributions, as in the following marginalization paradox of STONE and DAWID(1972). When their constructions are expressed as explicit assertions about disintegrations, the ¯aw behind the paradox is quickly revealed.
# VVS, 1997

309

Conditioning as disintegration

Example 12. Consider a measure de®ned on R 4 , speci®ed in the traditional way bymeans of distributions of random variables as coordinate projections. LetF  L  Lebesgue measure on R ;Y j F    probability density   with respect to L;
X j Y  ; F    exponential; with mean 1= ;Z j X  x; Y  ; F    exponential; with mean 1= x:The random vector (F, Y, X, Z) has a joint (improper) distribution l with density2f ; ; x; z    xeÿ   zx

14That is, we have de®ned a sigma-®nite measure on (R 4 with density f with respect tothe product L4 of Lebesgue measures on the coordinate spaces.
The (F, Y, Z)-marginal distribution is sigma-®nite with density f ; ; z   z2and the (F, Z) marginal is sigma-®nite with density
!
 

f ; z  I z  L
  z2
Notice that neither density depends on ; both marginal measures are products
having the measure L on the F-axis as a factor.
Disintegration with respect to the (F, Z) marginal measure gives the (Yj F, Z)
conditional probability densityf  j ; z    z2 I z

15

Noting that the last conditional distribution does not depend on , we might betempted to conclude that the Y j Z  conditional density is?
f  j z    z2 I z

16

Example 4 would justify such a conclusion if the (F, Z)-marginal had a disintegration
with the Z-marginal as mixing measure. Unfortunately the Z-marginal is not sigma®nite. Assertion (16) is based on a false analogy with the result for averaging overdisintegrating probability measures.
Something has gone wrong already, but now let us repeat the same sort ofreasoning along a dierent path. The (Y, X, Z)-marginal is sigma-®nite with densityf ; x; z 
# VVS, 19972 
x2   z3

310

J. T. Chang and D. Pollard

and the (X, Z)-marginal is sigma-®nite with density 2J(z)/x2 , where
!
 

J z  L  z3
The marginal distribution of Y; X; Z  has a
Y j X; Z  conditional probability densityf  j x; z    z3 J zX; Z -disintegration with the17

Again we might be tempted to interpret the lack of dependence on one variable, x, as
meaning that the Y j Z  conditional density is?
f  j z    z3 J z18

And again we would be misled by the analogy with Example 4. The Z-marginal is still
not sigma-®nite, no matter how it is calculated.
Formulae (16) and (18) appear contradictoryÐa paradox. We would explain theparadox by pointing out that neither formula represents a disintegration of the(Y, Z)-marginal with the Z-marginal as mixing measure. There is no such disintegration. The assertions (15) and (17) are ®ne; each statement gives a disintegration withrespect to a sigma-®nite measure. In both cases, the trouble comes when we are
tempted to throw away one of the conditioning variables, leaving just the variable Z,
whose image measure is not sigma-®nite. We must live with the fact that distributions
conditional on Z are not determined uniquely. So there is no such thing as the Y j Z conditional density. Indeed, in this case, much as in Example 11, the (Y, Z ) image
measure is not sigma-®nite. So in constructing a (Y, Z )-disintegration for the joint
distribution of (F, Y, X, Z ), we are left with a rather arbitrary choice of what
measure to use as the (Y, Z )-mixing measure. If we then regard our choice of (Y, Z )mixing measures as the ``joint distribution'' for that pair of variables, then clearly wecan arrive at many dierent ``conditional distributions'' for Y j Z .
More concisely, by integrating out variables in dierent orders we have constructed
two distinct sequences of disintegrations for calculating L L L L fg:
"
"
##
 
 z

x
2 2 ÿ zx
L x   z  eg ; ; z; x
L L I zL  z2 I z
"
"
"
 2
###
 
x 1
z

 
3 3 ÿ zxL  z x e
L
L 2J zLg ; ; z; x2x2
  z3 J zAll integrals correspond to probability measures, except for the ®rst ``L   '' andthe second ``Lx (  )'' and ``Lz (  )''. If L  = were ®nite, then we could also have
# VVS, 1997

Conditioning as disintegration

311

standardized J to be a probability density. It is futile to try to interpret the probability
measures as the conditional distributions.
&
Example 13. Terms like ``Markov chain Monte Carlo'' and ``Markov sampling''
refer to methods for generating random samples from given distributions by running
Markov chains. Although such methods have quite a long history, they have become
the subject of renewed interest in the last decade, particularly with the introductionof the ``Gibbs sampler'' by GEMAN and GEMAN (1984), who used the method in a
Bayesian approach to image reconstruction. The Gibbs sampler itself has enjoyed arecent surge of intense interest within statistics community, spurred by GELFAND
and SMITH (1990), who applied the Gibbs sampler to a wide variety of inference
problems.
Recall that a distribution P being stationary for a Markov chain X0 , X1 , . . . means
that, if X0  P, then Xn  P for all n. The theoretical foundation of Markov sampling
methods is the convergence in distribution of a Markov chain to its stationary
distribution: If a Markov chain X0 , X1 , . . . has stationary distribution P, then underquite general conditions (involving irreducibility and aperiodicity), the distribution of
Xn for large n is close to P. Thus, in order to generate an observation from a desired
distribution P on X , we ®nd a Markov chain X0 , X1 , . . . on X that has P as its
stationary distribution. The theory then suggests that running or simulating the chain
until a large time n will produce a random variable Xn whose distribution is close to
the desired P. By taking n large enough, in principle we obtain a value that may forpractical purposes be considered a random draw from the distribution P.
The Gibbs sampler is a way of constructing a Markov chain having a desired
stationary distribution. To illustrate the idea, consider a product space X  S T
with coordinate maps S and T. The problem is to generate an observation from agiven probability measure P on X . We assume that both S- and T-disintegrations of P
exist, giving conditional probability distributions that we will denote as P  j S  
and P  j T  : To perform a Gibbs sampler, start with any initial point (S0 , T0).
Then generate S1 from the conditional distribution P  j T  T0 , and generate T1
from the conditional distribution P  j S  S1 . Continue on in this way, generating
S2 from the conditional distribution P  j T  T1  and T2 from the conditional
distribution P  j S  S2 , and so on. Then the distribution P is stationary for theMarkov chain f Sn ; Tn  : n  0; 1; . . .g. To see this, suppose S0 ; T0   P. In
particular, T0 is distributed according to the T-marginal of P, so that, since S1 isdrawn from the conditional distribution of S given T  T0 , we have S1 ; T0   P.
Now we use the same reasoning again: S1 is distributed according to the S-marginalSP, so that S1 ; T1   P.
Here is a general formulation of the Gibbs sampler in terms of disintegrations.
Suppose we wish to simulate an observation from a probability measure P on a
space X . The Gibbs sampler consists of a sequence of ``moves'' that tell us how tochoose a new point Xn1 , given a current point Xn . For each map T for which a
T-disintegration {Pt} of P exists, there is a corresponding ``T-move'', which is# VVS, 1997

312

J. T. Chang and D. Pollard

performed as follows: Given the current point Xn 2 X , draw the next point Xn  1
according to the distribution PT X  : A T-move leaves the measure P invariant, that is,nx

P PT x f  Pf
In fact, this is just a restatement of the averaging property required in the de®nition ofdisintegration: de®ning g t  Pt f , we have
Pf  TPt Pt f  TPt g t  Px g T x  Px PT x fThus, for any map T, the Markov chain X0 , X1 , . . . produced by a succession ofT-moves has the desired distribution P as a stationary distribution. However, such a
chain would stay on the same level set of the map T forever, that is, we would haveXn 2 fx : T x  T X0 g for all n. To have convergence in distribution to P startingfrom an arbitrary initial distribution, we must perform moves using more than one
disintegration. That is the Gibbs sampler: given a sequence of disintegrations, theGibbs sampler is a performance of the corresponding moves. For example, given two
maps S and T, we could alternate making S-moves and T-moves. Or we could ¯ip acoin at each iteration to decide whether to make an S-move or a T-move. There is noneed to restrict to product spaces and coordinate maps as in the illustrative simple
setting above.
&
4

Other notions of conditioning

We hope we have convinced you that the existence of a disintegration is very
convenient in many statistical problems. However we do not wish to give the impression that we never feel the need to condition on sigma-®elds. After all, the expectationof an integrable X with respect to the disintegrating Pt is just a version of the abstract
Kolmogorov conditional expectation. More precisely, if we de®ne G t  Pt X then,at least for bounded measurable functions H,
t

PH T X  T P Pt H T X 
 T Pt H tPt X

because Pt concentrates on fT  tg

19

 PH T G T That is, Y  G T  is the (almost surely) unique random variable that is measurablewith respect to the sigma-®eld G generated by T for which
PWX  PWY

all bounded G-measurable W

20

As the proof in the Appendix shows, the fact that Pt concentrates on fT  tg is
actually equivalent to equality (19) for a suitable countable collection of H functions.
No topology is needed there. It is in the interpretation of P  j G  as an expectation
with respect to a probability measure that topology intervenes, as a way of sorting out
problems with uncountable collections of negligible sets. If we are concerned with the# VVS, 1997

313

Conditioning as disintegration

conditional expectations of only countably many random variablesÐas in the theoryof discrete-time martingales, for exampleÐthen there is no need to bring in
topological tools to manage the almost sure equivalences. However, in statisticalproblems many surprises can lie hidden in the formulations using conditioning on
sigma-®elds.
Consider, for example, the concept of suciency. One could call a sub-sigma-®eld
G sucient for a family P of probability measures on (O, F ) if, for each bounded
F -measurable random variable X there exists a single G-measurable Y for whichequality (20) holds for every P in P. That is the standard rigorous de®nition. As
BURKHOLDER (1961) showed, the de®nition allows some disturbing consequences,such as the possibility of a sucient G being contained in a ®ner sigma-®eld G that is
not sucient for P. If the intuition behind suciency says that G contains all theinformation about which P in P we are sampling from, then how can G be telling us
something extra? Apparently, the abstract de®nition has let a few nonintuitive beasts
through the gate. In the case of a dominated family no such problem can existÐ
compare with the factorization theorem of Example 6.
In some situations even the abstract de®nition is too concrete; the interpretation of
Y  P X j G as a random variable (or as an equivalence class of random variables)
becomes super¯uous. We can identify Y with a transition operator g, mapping
L1 P; F  into L1 (P, G ), identi®ed by the analog of equality (20),hgX; W i  hX; W i all W in L

1

P; G

And then we can dispense with Y altogether and express conditioning properties
purely in terms of a transition operator. DAWID (1980) chose something similar as thebest way to deal with the general form of conditional independence.
Finally, one can dispense with the interpretation of the domains of probability
measures as families of random variables on a speci®c O set, and treat conditioning asa transition map between abstract spaces, as in HARTIGAN's (1983) development ofBayes theory, or LE CAM's (1986) theory for convergence of experiments.
By stripping away assumptions unnecessary for the development of a particularstatistical or probabilistic idea, one gains in generality and sometimes even in insight.
We would claim that disintegrations oer more insight into something like thefactorization criterion for suciency than a collection of more elementary calculations for speci®c cases, sometimes involving unnecessary technical assumptions toaccommodate the details of a particular method. In the same way, disintegrations
could be regarded as overly restrictive, involving unnecessary topological assumptions in many abstract conditioning arguments using Radon±Nikodym derivatives.
And so on.
Conditioning is one of the most important ideas of probability and statistics. It isneeded at many dierent levels of understanding. We see great value in there also
being many ways of formalizing its mathematical description, each suited to adierent purpose.
# VVS, 1997

314
5

J. T. Chang and D. Pollard

History

The concepts of conditioning have a long history, which we cannot claim to haveresearched carefully. The best we can do is oer some references that might help those
who wish to pursue the topic further.
LOEÁ VE (1978, Section 30.2) mentioned that the problem of existence of regularconditional probabilities was ``investigated principally by Doob'', but he cited no
speci®c reference. DOOB (1953, page 624) cited a counterexample to the unrestrictedexistence, which also appears in the exercises to Section 48 of the 1969 printing of
HALMOS (1950). Doob's remarks suggest that the original edition of the Halmos bookcontained a slightly weaker form of the counterexample. Doob also noted that thecounterexample destroyed a claim made in (DOOB, 1938), an error pointed out by
DieudonneÂ (no citation) and Andersen and Jessen (no citation)Ðperhaps in their(1946) paper?
BLACKWELL (1956) cited DIEUDONNEÂ (1948) as the source of a counterexample forunrestricted existence of a regular conditional probabilities. Blackwell also proved
existence of regular conditional distributions for (what are now known as) Blackwell
spaces. The proof given by DELLACHERIE and MEYER (1978) uses the same sort ofregularity properties on the underlying space.
HOFFMANN-JéRGENSEN (1994, page 162) asserted that KOLMOGOROV (1933) was the®rst to establish existence of regular conditional distributions (for ``ordinary random
vectors''). We could not ®nd this result in Kolmogorov's book; indeed he stressed(page 50) that the conditional probability Pu(B) was determined only up to an almost
sure equivalence. Chapter 10 of the Homan-Jùrgensen book contains an expositionof the best disintegration theorem available, a result due to PACHL (1978). Pachl cited
a number of earlier papers on disintegrations.
PARTHASARATHY (1967, Sections V.7 and V.8) cited notes of Varadarajan for his
existence proof for a disintegration.
A mention of the names Doob and Kuratowski by WILLIAMS (1979, page 100) was
drawn to our attention by a referee, but we were unable to trace furtherÐWilliams
cited no works of those two authors. Probably DOOB (1953) was intended, but we canonly guess about Kuratowski. (Maybe the Topology book?)
The key idea in all proofs of existence of regular conditional distributions is that ofcompact approximationÐexistence of a class of approximating sets with properties
analogous to the class of compact sets in a metric spaceÐas a means for deducing
countably additivity from ®nite additivity. PFANZAGL and PIERLO (1969) developed asystematic theory of compact approximation. They were cited in the Note Historique
by BOURBAKI (1969), who also gave credit to Ryll-Nardzewski for disintegration(no citation), perhaps in some point-process context. In point process theory disintegrations appear as Palm distributionsÐconditional distributions given a point of
the process at a particular position (KALLENBERG, 1969).
PFANZAGL (1979) gave a condition under which a regular conditional distribution
can be obtained by means of the elementary ``limit of ratio of probabilities''.
# VVS, 1997

Conditioning as disintegration

315

The BARNDORFF-NIELSEN, BLAESILD and ERIKSEN (1989) book contains muchmaterial on the invariance properties of conditional distributions, which we have notyet studied in detail.
6

Appendix: Existence of disintegrations

Here is a condensed proof of the Existence Theorem 1, based on ideas fromDELLACHERIE and MEYER (1978, page 78). We agree with them that ``The theorem on
disintegration of measures has a bad reputation, and probabilists often try to avoid
the use of conditional distributions . . . But it really is simple and easy to prove.''
The assumptions let us reduce the proofs of both existence and uniqueness to thecase where X is compact and both l and  are ®nite measures. (Partition T into
countably many disjoint sets Bi , each of ®nite  measure. Partition each T ÿ1 Bi into
sets Ni ; Ki1 ; Ki2 ; . . . ; with lNi  0 and each Kij compact. For existence, construct®nite disintegrating measures for the restriction of l to Kij and  restricted to Bi , thenpiece together the restrictions. Notice that each disintegrating measure will be sigma®nite, being constructed from countably many ®nite measures concentrated on
disjoint sets. For uniqueness, combine the trivial result for the restriction of l to anegligible set with the result for compact sets.)
De®ne a ®nite measure  (the image of l under the map that takes x onto (x, Tx))
on A B by h x; t  lh x; Tx: It lives on the graph of T, in the sense that
f x; t : Tx 6 tg  0

21This assertion follows from the countable generation property, and the fact that B
contains all the singleton sets. (Let B0 be the countable subclass that generates B. For
each t 2 T , the singleton ftg is equal to ftg  \fB 2 B0 : t 2 Bg, which implies that
= B; t 2 Bg. For ®xed B in B0 ,f x; t : Tx 6 tg  [B 2 B fTx 20

fTx 2 B; t 2= Bg  lfTx 2 B; Tx 2
= Bg  0
The set fTx 6 tg is a countable union of -negligible sets.)
Now we use compactness to avoid the problem, mentioned at the start of Section 2,
with uncountable families of negligible sets. The trick is to reduce countable
additivity to a condition involving only countably many assertions about conditional
expectations. On the real line one can determine measures from the values taken by
their distribution functions at a countable dense set. On more general spaces, thefollowing consequence of the Riesz representation theorem, and of the fact that there
exists a sequence of functions dense in the space of all continuous real functions on X
(under the uniform metric), suces.
If X is a compact metric space then there exists a countable family C0 of nonnegative, continuous functions on X such that(i)(ii)
# VVS, 1997

C0 is closed under addition
for each additive functional ` : C0 ! R there exists a unique Borel measure L
such that ` f   L f for each f in C0 .

316

J. T. Chang and D. Pollard

For ®xed f in C0 , the map g 7!  f xg t de®nes a measure on B, which isdominated by  because j  f xg t j  Cl j g Tx j C Tl j g j for someconstant C that bounds f. Write lt f for a density of this measure with respect to :t

 f xg t   g tlt f (As a function of t, the lt f integral corresponds to the Kolmogorov conditional
expectation of f.) For almost all t, the map f 7! lt f is nonnegative and additive, andhence corresponds to a measure on A. Invoke a generating-class argument to deducethat t 7! lxt h x; t is measurable, for bounded measurable h, and h  t lxt h x; t. In
particular, l f  t lxt f x for each bounded, A-measurable f. Put h x; t 
f x; t : Tx 6 tg to deduce from property (21) that t lt fx : Tx 6 tg  0. Consequently, lt fx : Tx 6 tg  0 for  almost all t.
For uniqueness, suppose we have two disintegrations, {lt} and flt g, of a ®nite
Radon measure l on a compact metric space. Consider an f in C0 . De®ne
Bf  ft 2 T : lt f < lt f g. The two disintegrations of l givett ft 2 Bf glt f  lfT 2 Bf gf   ft 2 Bf glt f
Deduce that Bf  0, that is, lt f  lt f for almost all t. Argue analogously to get thereverse inequality. Cast out countably many negligible sets as f ranges over C0 , todeduce that lt and lt can be dierent measures only for a -negligible set of t values.
7

Acknowledgements

We thank John Hartigan for suggesting several troublesome examples. We are also
grateful for the comments of two careful referees and the editor, which helped us inour ®nal revision.
References
ANDERSEN, E. S. and B. JESSEN (1946), Some limit theorems on integrals in an abstract set,
Det Kongelige Danske Videnskabernes Selskab, Matematisk-Fysiske Meddelelser, Bind 22,
no. 14.
BARNDORFF-NIELSEN, O. E., P. BLAESILD and P. S. ERIKSEN (1989), Decomposition andinvariance of measures, and statistical transformation models, Vol. 58 of Springer Lecture
Notes in Statistics, Springer-Verlag, New York.
a 15, 377±
BASU, D. (1955), On statistics independent of a complete sucient statistic, Sankhy
380.
a 20, 223±226.
BASU, D. (1958), On statistics independent of a sucient statistic, Sankhy
BLACKWELL, D. (1956), On a class of probability spaces, in: J. NEYMAN (ed.) Proceedings of theThird Berkeley Symposium on Mathematical Statistics and Probability, Vol. I, University ofCalifornia Press, Berkeley, pp. 1±6.
BOURBAKI, N. (1969), InteÂgration, Vol. IX of EÂleÂments de matheÂmatique, Hermann, Paris.
(Fascicule XXXV).
DELLACHERIE, C. and P. A. MEYER (1978), Probabilities and potential, North-Holland,
Amsterdam.
# VVS, 1997

Conditioning as disintegration

317

DIEUDONNEÂ , J. (1948), Sur le theÂoreÁme de Lebesgue-Nikodym, III, Annales de l'Institut Fourier(Grenoble) 23, 25±53.
DONOHO, D. L. and R. C. LIU (1991), Geometrizing rates of convergence, II, Annals of Statistics
19, 633±667.
DOOB, J. L. (1938), Stochastic processes with integral-valued parameter, Transactions of theAmerican Mathematical Society 44, 87±150.
DOOB, J. L. (1953), Stochastic processes, Wiley, New York.
EATON, M. (1992), A statistical diptych: admissible inferences-recurrence of symmetric Markov
chains, Annals of Statistics 20, 1147±1179.
GELFAND, A. E. and A. F. M. SMITH (1990), Sampling based approaches to calculating
marginal densities, Journal of the American Statistical Association 85, 398±409.
GEMAN, S. and D. GEMAN (1984), Stochastic relaxation, Gibbs distributions, and the Bayesian
restoration of images, IEEE Transactions on Pattern Analysis and Machine Intelligence 6,
721±741.
HALMOS, P. R. (1950), Measure theory, Van Nostrand, New York, NY. (July 1969 reprinting).
HALMOS, P. R. and L. J. SAVAGE (1949), Application of the Radon-Nikodym theorem to thetheory of sucient statistics, Annals of Mathematical Statistics 20, 225±241.
HARTIGAN, J. A. (1983), Bayes theory, Springer, New York.
HOFFMANN-JéRGENSEN, J. (1994), Probability with a view toward statistics, Vol. 2, Chapman andHall, New York.
KALLENBERG, O. (1969), Random measures, Akademie-Verlag, Berlin. (US publisher: Academic
Press).
KOLMOGOROV, A. N. (1933), Foundations of probability, Chelsea, New York, NY. Second
English Edition 1950.
LE CAM, L. (1986), Asymptotic methods in statistical decision theory, Springer, New York.
LEHMANN, E. L. (1959), Testing statistical hypotheses, Wiley, New York. Later edition
published by Chapman and Hall.
LITTLE, R. J. A. and D. B. RUBIN (1987), Statistical analysis with missing data, Wiley, NewYork.
LOEÁ VE (1978), Probability theory, Springer, New York. Fourth Edition, Part II.
PACHL, J. (1978), Disintegration and compact measures, Mathematica Scandinavica 43, 157±
168.
PARTHASARATHY, K. R. (1967), Probability measures on metric spaces, Academic, New York.
PFANZAGL, J. (1979), Conditional distributions as derivatives, Annals of Probability 7, 1046±
1050.
PFANZAGL, J. and N. PIERLO (1969), Compact systems of sets, Vol. 16 of Springer Lecture Notes
in Mathematics, Springer-Verlag, New York.
POLLARD, D. (1996), Probability explained (Unpublished book manuscript).
STONE, M. and A. P. DAWID (1972), Un-Bayesian implications of improper Bayes inference inroutine statistical problems, Biometrika 59, 369±375.
TJUR, T. (1974), Conditional probability distributions, Vol. 2 of Lecture Notes, Institute ofMathematical Statistics, University of Copenhagen.
WILLIAMS, D. (1979), Diusions, Markov processes, and martingales, Vol. 1, Wiley, New York.
WINTER, B. B. (1979), An alternate development of conditioning, Statistica Neerlandica 33,
197±212.
WU, C. F. J. (1983), On the convergence properties of the EM algorithm, Annals of Statistics
10, 95±103.
Received: September 1994. Revised: April 1996.

# VVS, 1997