will have a high-school degree. Let X be the number who respond and have a high-school degree, and let Y be the total number who respond. Note that X Y. Find E[ Y X = x ]. The trick clearly comes in getting the joint law pX,Y (x, y). We do know that Y ~ Poisson() and X Y = y ~ Bin(y, ). Thus pX,Y (x, y) = pY (y) pXY (x | y) = I(0 x y) Using this, we can sum out y to get the marginal probability law for X . pX(x) = = = = = = [change index to k = y - x ] = = = Thus, we see that (marginally) X ~ Poisson(). Thus, we can complete the derivation of the conditional law of Y, given X = x. = = = = = In this form, it appears that, given X = x, it happens that Y x is Poisson with parameter (1 ). This can be made completely explicit by writing Z = Y x. The probability law for Z (given x) will be pZ(x) = Since E Z = (1 ), we have E[ Y | X = x ] = x + (1 ).
Here’s another one. Let’s suppose that X and Y are independent binomial random variables. Say that X ~ Bin(m, p) and Y ~ Bin(n, p). Observe that the p’s are the same. Let random variable T be the total; T = X + Y. What is the probability law of X, given that T = t ? This asks us to find P[ X = x | T = t ]. Certainly T ~ Bin(m+n, p). Then P[ X = x | T = t ] = = = = = = It is interesting that the symbol p completely disappeared! Later, we’ll identify this as a case in which T is a sufficient statistic for p. Let’s suppose that random variables X and Y have the joint probability law fXY(x, y). This could be a joint probability function for two discrete random variables, it could be a joint density function for two continuous random variables, it could be a mixed law. We know all about the following (Yes…..is this review?) fX(x) = marginal density of X = The integral is written from ∞ to +∞, but the actual work will depend on what set of values Y takes. fY(y) = marginal density of Y = X = mean of X = expected value of X = = Let’s look at the last form. It really means . The order of integration can usually be reversed (assuming it’s a convergent integral) as = . The last form has in { } the density fX(x), so this all makes sense. = = = Y and are defined in an analogous way.
fX | Y (x | y) = conditional density of X, given (Y = y) = E( X | Y = y ) = = This is a function of the given value y. Since we can think of y as a carrier (or dummy) symbol for Y, the expression E( X | Y ) should be regarded as a random variable which is a function of Y. Since E( X | Y ) is a random variable which is a function of Y, we can get its expectation by integrating relative to the density of Y. This work gives E { E( X | Y ) } = = = The previous step reversed the order of integration. In problems in which X and Y are distributed over a twodimensional set which is not a rectangle, this can be very tricky. The inner integral in { } is fX(x). = = E( X ) This is a very important finding. This works as well for any function of X, say g(X), as g(X) is just another random variable. Thus it happens that E { E( g(X) | Y ) } = E( g(X) ). Var( X | Y = y ) = E[ X2 | Y = y ] { E( X | Y = y ) }2 This is a function of the given value y. Since we can think of y as a carrier (or dummy) symbol for Y, the expression Var( X | Y ) should be regarded as a random variable which is a function of Y. Examine now E{ Var (X | Y ) }. Of course, E(X | Y ) is some function of Y, and it may help to give it a temporary name. Let it be h(Y). E{ Var (X | Y ) } = E{ E[ X2 | Y ] ( E( X | Y ) ) 2 }
= E{ E[ X2 | Y ] h2 (Y) } = E{ X2 } E { h2 (Y) } This thinks of X2 in the context of g(X) above. Next examine Var{ E( X | Y ) }. Var{ E( X | Y ) } = Var{ h(Y) } = E { h2 (Y) } [ E( h(Y) ) ]2 Of course E( h(Y) ) = E( E(X | Y ) ) = E(X). = E { h2 (Y) } [ E(X) ]2 = E { h2 (Y) } Interesting. Now add the last two results: E{ Var (X | Y ) } + Var{ E( X | Y ) } = E{ X2 } E { h2 (Y) } + E { h2 (Y) } = E{ X2 } = Var(X) This proves the incredibly insightful result that Var(X) = E{ Var (X | Y ) } + Var{ E( X | Y ) } As a particular useful consequence Var(X) ≥ Var{ E( X | Y ) }. This is the essence of the Rao-Blackwell theorem. Let’s next consider the definition of the gamma function. For any positive value q we define the gamma function of q as the integral (q) = As a aside, let’s note that Rice, page 53, defines this as (x) = In this form, there are two dummy variables. First of all, u is the dummy variable used to define the integral, and it’s used only in the integral. This could be replaced by any other letter (assuming no conflicts with other letters) without changing the meaning. Secondly, x is the dummy variable used to define the function. The letter x appears on both sides of the equals sign. It could also be replaced with any other (nonconflicting) letter. Thus, the two definitions of the gamma function just provided are identical.