97
9. Week 9
Remark 9.1 (Conditional Distribution for discrete random vectors). Let X = (X1, X2, · · · , Xp+q)
be a discrete random vector with support SX and joint p.m.f. fX. Let Y = (X1, X2, · · · , Xp) and
Z = (Xp+1, Xp+2, · · · , Xp+q). Then Y and Z both are discrete random vectors. Let fY and SY
denote the joint p.m.f. and support of Y , respectively. Let fZ and SZ denote the joint p.m.f. and
support of Z, respectively. For z ∈ SZ, consider the set
Tz := {y ∈ R
p
: (y, z) ∈ SX}.
The conditional p.m.f. of Y given Z = z ∈ SZ is defined by
fY |Z(y | z) := P(Y = y | Z = z) = P(Y = y, Z = z)
P(Z = z)
=



fX(y,z)
fZ (z)
, if y ∈ Tz
0, otherwise.
By definition, fY |Z(y | z) ≥ 0, ∀y ∈ R
p and P
y∈Rp fY |Z(y | z) = P
y∈Tz
fY |Z(y | z) = 1. Therefore,
for every z ∈ SZ, the function y ∈ R
p
7→ fY |Z(y | z) is a joint p.m.f. with support Tz. We refer to
the probability law/distribution described by this p.m.f. as the conditional distribution of Y given
Z = z ∈ SZ. The conditional DF of Y given Z = z ∈ SZ is given by
FY |Z(y | z) := P(Y ≤ y | Z = z) = P(Y ≤ y, Z = z)
P(Z = z)
=
X
t≤y
t∈Tz
fX(t, z)
fZ(z)
=
X
t≤y
t∈Tz
fY |Z(t | z),
where t ≤ y refers to component-wise inequalities tj ≤ yj
for all j = 1, 2, · · · , p.
Note 9.2. For notational convenience, we have discussed the conditional distribution of first p
component RVs with respect to the final q component RVs. However, as long as the (p + q)-
dimensional joint distribution is known, we can discuss the conditional distribution of any of the
k-component RVs with respect to the other (p + q − k)-component RVs.
Note 9.3. When values for some of the components RVs are given, the conditional distribution
provides an updated probability distribution for the rest of the component RVs.
98
Note 9.4. Let (X, Y ) be a 2-dimensional discrete random vector such that X and Y are independent. Then fX,Y (x, y) = fX(x)fY (y), ∀x, y ∈ R. Then
fY |X(y | x) = fY (y), ∀x ∈ SX, y ∈ SY .
This statement can be generalized to higher dimensions with appropriate changes in the notation.
Example 9.5. In Example 8.51, we have, for fixed x ∈ {1, 2, 3, 4},
fY |X(y | x) =



fX,Y (x,y)
fX(x)
, if y ∈ {1, 2, 3, 4}
0, otherwise.
=



x+y
2(2x+5) , if y ∈ {1, 2, 3, 4}
0, otherwise.
Definition 9.6 (Continuous Random Vector and its Joint Probability Density Function (Joint
p.d.f.)). A random vector X = (X1, X2, · · · , Xp) is said to be a continuous random vector if there
exists an integrable function f : R
p → [0,∞) such that
FX(x) = P(X1 ≤ x1, X2 ≤ x2, · · · , Xp ≤ xp)
=
Z x1
t1=−∞
Z x2
t2=−∞
· · · Z xp
tp=−∞
f(t1, t2, · · · , tp) dtpdtp−1 · · · dt2dt1, ∀x = (x1, x2, · · · , xp) ∈ R
p
.
The function f is called the joint probability density function (joint p.d.f.) of X.
Remark 9.7. Let X be a continuous random vector with joint DF FX and joint p.d.f. fX. Then
we have the following observations.
(a) FX is jointly continuous in all co-ordinates.
(b) P(X = x) = 0, ∀x ∈ R
p
. More generally, if A ⊂ R
p
is finite or countably infinite, then by
the finite/countable additivity of PX, we have
P(X ∈ A) = PX(A) = X
x∈A
PX({x}) = X
x∈A
P(X = x) = 0.
(c) By definition, we have fX(x) ≥ 0, ∀x ∈ R
p and
1 = lim xj→∞ ∀j
FX(x1, x2, · · · , xp)
= lim xj→∞ ∀j
Z x1
t1=−∞
Z x2
t2=−∞
· · · Z xp
tp=−∞
f(t1, t2, · · · , tp) dtpdtp−1 · · · dt2dt1
99
=
Z ∞
−∞
Z ∞
−∞
· · · Z ∞
−∞
fX(t1, t2, · · · , tp) dtpdtp−1 · · · dt2dt1.
(d) Suppose that the joint p.d.f. fX of a p-dimensional random vector X is piecewise continuous. Then by the Fundamental Theorem of Calculus (specifically multivariable Calculus),
we have
fX(x1, x2, · · · , xp) = ∂
p
∂x1∂x2 · · · ∂xp
FX(x1, x2, · · · , xp),
wherever the partial derivative on the right hand side exists.
(e) If X is a p-dimensional random vector such that its joint DF FX is continuous on R
p
and such that the partial derivative ∂
p
∂x1∂x2···∂xp
FX exists everywhere except possibly on a
countable number of curves on R
p
. Let A ⊂ R
p denote the set of all points on such curves.
Then X is a continuous random vector with the joint p.d.f.
fX(x) =



∂
p
∂x1∂x2···∂xp
FX(x), if x = (x1, x2, · · · , xp) ∈ Ac
,
0, if x = (x1, x2, · · · , xp) ∈ A.
(f) The joint p.d.f. of a continuous random vector is not unique. As in the case of continuous
RVs, the joint p.d.f. is determined uniquely upto sets of ‘volume 0’. Here, we also get
versions of the joint p.d.f..
(g) For A ⊂ R
p
, we have
P(X ∈ A) = ZZZ
A
fX(t1, t2, · · · , tp) dtpdtp−1 · · · dt2dt1
=
ZZZ
Rp
fX(t1, t2, · · · , tp)1A(t1, t2, · · · , tp) dtpdtp−1 · · · dt2dt1,
provided the integral can be defined. We do not prove this statement in this course.
(h) For any j ∈ {1, 2, · · · , p}, for xj ∈ R
FXj
(xj ) = P(Xj ∈ (−∞, xj
])
= P(X1 ∈ R, · · · , Xj−1 ∈ R, Xj ∈ (−∞, xj
], Xj+1 ∈ R, · · · , Xp ∈ R)
= P(X ∈ R × · · · × R × (−∞, xj
] × R × · · · × R)
100
=
Z ∞
t1=−∞
· · · Z ∞
tj−1=−∞
Z xj
tj=−∞
Z ∞
tj+1=−∞
· · · Z ∞
tp=−∞
fX(t1, t2, · · · , tp) dtpdtp−1 · · · dt2dt1.
Consider gj
: R → R defined by
gj (tj ) := Z ∞
t1=−∞
· · · Z ∞
tj−1=−∞
Z ∞
tj+1=−∞
· · · Z ∞
tp=−∞
fX(t1, t2, · · · , tp) dtp · · · dtj−1dtj+1 · · · dt2dt1.
It is immediate that gj satisfies the properties of a p.d.f. and FXj
(xj ) = R xj
tj=−∞ gj (tj ) dtj
.
Therefore, Xj
is a continuous RV with p.d.f. gj
. More generally, all marginal distributions of
X are also continuous and can be obtained by integrating out the unnecessary co-ordinates.
The function gj
is usually referred to as the marginal p.d.f. of Xj
.
Remark 9.8. Let f : R
p → [0,∞) be an integrable function with
Z ∞
−∞
Z ∞
−∞
· · · Z ∞
−∞
fX(t1, t2, · · · , tp) dtpdtp−1 · · · dt2dt1 = 1.
Then f is the joint p.d.f. of some p-dimensional continuous random vector X. We are not going
to discuss the proof of this statement in this course.
We can identify the independence of the component RVs for a continuous random vector via the
joint p.d.f.. The proof is similar to Theorem 8.48 and is skipped for brevity.
Theorem 9.9. Let X = (X1, X2, · · · , Xp) be a continuous random vector with joint DF FX, joint
p.d.f. fX. Let fXj
denote the marginal p.d.f. of Xj
. Then X1, X2, · · · , Xp are independent if and
only if
fX1,X2,··· ,Xp
(x1, x2, · · · , xp) = Y
p
j=1
fXj
(xj ), ∀x1, x2, · · · , xp ∈ R.
Example 9.10. Given p.d.f.s f1, f2, · · · , fp : R → [0,∞), consider the function f : R
p → [0,∞)
defined by
f(x) := Y
p
j=1
fj (xj ), ∀x = (x1, x2, · · · , xp) ∈ R
p
.
Then
Z ∞
−∞
Z ∞
−∞
· · · Z ∞
−∞
fX(t1, t2, · · · , tp) dtpdtp−1 · · · dt2dt1 = 1.
101
By Remark 9.8, we have that f is the joint p.d.f. of a p-dimensional continuous random vector such
that the component RVs are independent, by Theorem 9.9. Using this method, we can construct
many examples of continuous random vectors.
Remark 9.11. Let X = (X1, X2, · · · , Xp) be a continuous random vector with joint p.d.f. fX. Then
X1, X2, · · · , Xp are independent if and only if
fX1,X2,··· ,Xp
(x1, x2, · · · , xp) = Y
p
j=1
gj (xj ), ∀x1, x2, · · · , xp ∈ R
for some integrable functions g1, g2, · · · , gp : R → [0,∞). In this case, the marginal p.d.fs fXj have
the form cjgj
, where the number cj can be determined from the relation cj =
R ∞
−∞ gj (x) dx−1
.
Example 9.12. Let Z = (X, Y ) be a 2-dimensional continuous random vector with the joint p.d.f.
of the form
fZ(x, y) =



αxy, if 0 < x < y < 1
0, otherwise
for some constant α ∈ R. For fZ to take non-negative values, we must have α > 0. Now,
Z ∞
−∞
Z ∞
−∞
fZ(x, y) dxdy =
Z 1
y=0
Z y
x=0
αxy dxdy =
Z 1
y=0
α
y
3
2
dy =
α
8
.
For fZ to be a joint p.d.f., we need R ∞
−∞
R ∞
−∞ fZ(x, y) dxdy = 1 and hence α = 8 > 0. Also note
that for this value of α, fZ takes non-negative values. The marginal p.d.f. fX of X can now be
computed as follows.
fX(x) = Z ∞
−∞
fZ(x, y) dy =



R 1
y=x
8xy dy, if x ∈ (0, 1)
0, otherwise
=



4x[1 − x
2
], if x ∈ (0, 1)
0, otherwise
.
The marginal p.d.f. fY of Y follows by a similar computation.
fY (y) = Z ∞
−∞
fZ(x, y) dx =



R y
x=0 8xy dx, if y ∈ (0, 1)
0, otherwise
=



4y
3
, if y ∈ (0, 1)
0, otherwise
.
Observe that fZ(
1
2
,
1
2
) = 0 and fX(
1
2
)fY (
1
2
) = 3
2 ×
1
2 =
3
4
. Hence X and Y are not independent.
102
Example 9.13. Let U = (X, Y, Z) be a 3-dimensional continuous random vector with the joint
p.d.f. of the form
fU (x, y, z) =



αxyz, if x, y, z ∈ (0, 1)
0, otherwise
for some constant α ∈ R. For fZ to take non-negative values, we must have α > 0. Now,
Z ∞
−∞
Z ∞
−∞
Z ∞
−∞
fU (x, y, z) dxdydz =
Z 1
x=0
Z 1
y=0
Z 1
z=0
αxyz dxdydz =
α
8
.
For fU to be a joint p.d.f., we need R ∞
−∞
R ∞
−∞
R ∞
−∞ fU (x, y, z) dxdydz = 1 and hence α = 8 > 0. Also
note that for this value of α, fU takes non-negative values. The marginal p.d.f. fX of X can now
be computed as follows.
fX(x) = Z ∞
−∞
Z ∞
−∞
fU (x, y, z) dydz =



R 1
z=0
R 1
y=0 8xyz dydz, if x ∈ (0, 1)
0, otherwise
=



2x, if x ∈ (0, 1)
0, otherwise
.
By the symmetry of fU (x, y, z) in the variables x, y and z, we conclude that X
d= Y
d= Z. Observe
that fX,Y,Z(x, y, z) = fX(x)fY (y)fZ(z), ∀x, y, z and hence the RVs X, Y, Z are independent.
Note 9.14. There are random vectors which are neither discrete nor continuous. We do not
discuss such examples in this course.
Remark 9.15 (Conditional Distribution for continuous random vectors). We now discuss an analogue of conditional distributions as discussed in Remark 9.1 for discrete random vectors. To avoid
notational complexity, we work in dimension 2. Let (X, Y ) be a 2-dimensional continuous random
vector with joint DF FX,Y and joint p.d.f. fX,Y . Let fX and fY denote the marginal p.d.fs of X
and Y respectively. Since P(X = x) = 0, ∀x ∈ R, expressions of the form P(Y ∈ A | X = x)
are not defined for A ⊂ R. We consider x ∈ R such that fX(x) > 0 and look at the following
computation. For y ∈ R,
lim
h↓0
P(Y ≤ y | x − h < X ≤ x) = limh↓0
P(Y ≤ y, x − h < X ≤ x)
P(x − h < X ≤ x)
103
= lim
h↓0
R x
x−h
R y
−∞ fX,Y (t, s) dsdt
R x
x−h
fX(t) dt
= lim
h↓0
1
h
R x
x−h
R y
−∞ fX,Y (t, s) dsdt
1
h
R x
x−h
fX(t) dt
=
R y
−∞ fX,Y (x, s) ds
fX(x)
=
Z y
−∞
fX,Y (x, s)
fX(x)
ds
Here, we have assumed continuity of the p.d.fs. Motivated by the above computation, we define
the conditional DF of Y given X = x (provided fX(x) > 0) by
FY |X(y | x) := limh↓0
P(Y ≤ y | x − h < X ≤ x), y ∈ R
and the conditional p.d.f. of Y given X = x (provided fX(x) > 0) by
fY |X(y | x) := fX,Y (x, y)
fX(x)
, y ∈ R.
These calculations generalizes to the higher dimensions as follows. Let X = (X1, X2, · · · , Xp+q)
be a continuous random vector with joint p.d.f. fX. Let Y = (X1, X2, · · · , Xp) and Z =
(Xp+1, Xp+2, · · · , Xp+q). If z ∈ R
q be such that fZ(z) > 0, then we define the conditional DF
of Y given Z = z by
FY |Z(y | z) := limhj↓0
j=p+1,p+2,··· ,p+q
P(X1 ≤ y1, · · · , Xp ≤ yp | xj − hj < Xj ≤ xj
, ∀j), y ∈ R
p
and the conditional p.d.f. of Y given Z = z by
fY |Z(y | z) := fY,Z(y, z)
fZ(z)
, y ∈ R
p
.
Note 9.16. For notational convenience, we have discussed the conditional distribution of first
p component RVs with respect to the final q component RVs. However, as long as the (p + q)-
dimensional joint distribution is known, we can discuss the conditional distribution of any of the
k-component RVs with respect to the other (p + q − k)-component RVs.
104
Note 9.17. Let (X, Y ) be a 2-dimensional continuous random vector such that X and Y are
independent. Then fX,Y (x, y) = fX(x)fY (y), ∀x, y ∈ R. Then
fY |X(y | x) = fY (y), ∀y ∈ R,
provided fX(x) > 0. This statement can be generalized to higher dimensions with appropriate
changes in the notation.
Example 9.18. In Example 9.12, we have, for fixed x ∈ (0, 1),
fY |X(y | x) = fX,Y (x, y)
fX(x)
=



2xy
x(1−x2)
, if y ∈ (x, 1)
0, otherwise.
Earlier, we have discussed about the distribution of functions of RVs. We now generalize the
same concept for random vectors.
Remark 9.19. Let X = (X1, . . . , Xp) be a p-dimensional discrete/continuous random vector with
joint p.m.f./p.d.f. fX. We are interested in the distribution of Y = h(X) for functions h : R
p →
R
q
. Here, Y = (Y1, . . . , Yq) is a q-dimensional random vector with Yj = hj (X1, . . . , Xp), where
hj
: R
p → R, j = 1, 2, · · · , q denotes the component functions of h. The distribution of Y is
uniquely determined as soon as we are able to compute the joint DF FY of Y . Note that
FY (y1, · · · , yq) = P(Y1 ≤ y1, · · · , Yq ≤ yq) = P(h1(X) ≤ y1, · · · , hq(X) ≤ yq), ∀(y1, · · · , yq) ∈ R
q
.
Once the joint DF FY is known, the joint p.m.f./p.d.f. of Y can then be deduced by standard
techniques.
Example 9.20. Let X1 ∼ Uniform(0, 1) and X2 ∼ Uniform(0, 1) be independent RVs. Suppose
we are interested in the distribution of Y = X1 + X2. By independence of X1 and X2, the joint
p.d.f. (X1, X2) is given by
fX1,X2
(x1, x2) = fX1
(x1) fX2
(x2)
=



1, if x1, x2 ∈ (0, 1)
0, otherwise.
105
Consider the function h : R
2 → R defined by h(x1, x2) := x1 + x2, ∀(x1, x2) ∈ R
2
. Then Y =
h(X1, X2). Now, for y ∈ R
FY (y) = P(Y ≤ y)
= P(h(X1, X2) ≤ y)
=
Z ∞
−∞
Z ∞
−∞
1(−∞,y](h(x1, x2))fX1,X2
(x1, x2) dx1dx2
=
Z 1
0
Z 1
0
1(−∞,y](x1 + x2) dx1dx2
=



0, if y < 0,
R y
x1=0
R y−x1
x2=0 dx2dx1, if 0 ≤ y < 1,
1 −
1
2 × (2 − y) × (2 − y), if 1 ≤ y < 2,
1, if y ≥ 2
=



0, if y < 0,
y
2
2
, if 0 ≤ y < 1,
4y−y
2−2
2
, if 1 ≤ y < 2,
1, if y ≥ 2
Here, FY is differentiable everywhere except possibly at the points 0, 1, 2 and
F
′
Y
(y) =



y, if y ∈ (0, 1),
2 − y, if y ∈ (1, 2),
0, otherwise.
Observe that R ∞
−∞ F
′
Y
(y) dy = 1 and the derivative is non-negative. Hence, Y is a continuous RV
with the p.d.f. given by F
′
Y
.
As done in the case of RVs, in the setting of Remark 9.19, we consider the computation of the
joint p.m.f./p.d.f. of Y directly, instead of computing the joint DF FY first. The next result is a
direct generalization of Theorem 5.21 and we skip the proof for brevity.
106
Theorem 9.21 (Change of Variables for Discrete random vectors). Let X = (X1, . . . , Xp) be a
p-dimensional discrete random vector with joint p.m.f. fX and support SX. Let h = (h1, · · · , hq) :
R
p → R
q
be a function and let Y = (Y1, · · · , Yq) = h(X) = (h1(X), · · · , hq(X)). Then Y is a
discrete random vector with support
SY = h(SX) = {h(x) : x ∈ SX},
joint p.m.f.
fY (y) =



X
x∈SX
h(x)=y
fX(x), if y ∈ SY ,
0, otherwise
and joint DF
FY (y) = X
x∈SX
h(x)≤y
fX(x), ∀y ∈ R
q
.
Example 9.22. Fix p ∈ (0, 1) and let n1, · · · , nq be positive integers. Let X1, · · · , Xq be independent RVs with Xi ∼ Binomial(ni
, p), i = 1, · · · , q. Here, the with p.m.f.s are given by
fXi
(xi) =




ni
x

p
x
(1 − p)
ni−x
, ∀x ∈ {0, 1, · · · , ni},
0, otherwise
for i = 1, · · · , q. Using independence, the joint p.m.f. is given by
fX(x1, · · · , xq) =



Y
q
i=1
ni
xi
!
p
Pq
i=1 xi
(1 − p)
n−
Pq
i=1 xi
, ∀(x1, · · · , xq) ∈
Y
q
i=1
{0, 1, · · · , ni},
0, otherwise
where n = n1 + · · · + nq. Consider Y = X1 + · · · + Xq. Now, if y /∈ {0, 1, · · · , n}, fY (y) =
P(X1 + · · · + Xq = y) = 0 and if y ∈ {0, 1, · · · , n}, then
fY (y) = P(X1 + · · · + Xq = y)
=
X
(x1,··· ,xq)∈
Qq
i=1{0,1,··· ,ni}
x1+···+xq=y
fX(x1, · · · , xq)
107
= p
y
(1 − p)
n−y X
(x1,··· ,xq)∈
Qq
i=1{0,1,··· ,ni}
x1+···+xq=y
Y
q
i=1
ni
xi
!
=

n
y
!
p
y
(1 − p)
n−y
.
Therefore, Y = X1 + · · · + Xq ∼ Binomial(n, p) with n = n1 + · · · + nq.
Remark 9.23. We had earlier mentioned that Bernoulli(p) distribution is the same as Binomial(1, p)
distribution. Using the above computation, we can identify a Binomial(n, p) RV as a sum of n
independent RVs each having distribution Bernoulli(p). We shall come back to this observation
in later lectures.
For continuous random vectors, we have the following generalization of Theorem 6.1. Proof of
this result is being skipped.
Theorem 9.24. Let X = (X1, . . . , Xp) be a p-dimensional continuous random vector with joint
p.d.f. fX. Suppose that {x ∈ R
p
: fX(x) > 0} can be written as a disjoint union ∪
k
i=1Si of open
sets in R
p
.
Let h
j
: R
p → R, j = 1, · · · , p be functions such that h = (h
1
, · · · , hp
) : Si → R
p
is one-to-one
with inverse h
−1
i = ((h
1
i
)
−1
, · · · ,(h
p
i
)
−1
) for each i = 1, · · · , k. Moreover, assume that (h
j
i
)
−1
, i =
1, 2, · · · , k; j = 1, · · · , p have continuous partial derivatives and the Jacobian determinant of the
transformation
Ji
:=










∂(h
1
i
)−1
∂y1
(t) · · ·
∂(h
1
i
)−1
∂yp
(y)
.
.
.
.
.
.
.
.
.
∂(h
p
i
)−1
∂y1
(y) · · ·
∂(h
p
i
)−1
∂yp
(y)










̸= 0, ∀i = 1, · · · , k.
Then the p-dimensional random vector Y = (Y1, · · · , Yp) = h(X) = (h
1
(X), · · · , hp
(X)) is a
continuous with joint p.d.f.
fY (y) = X
k
i=1
fX((h
1
i
)
−1
(y), · · · ,(h
p
i
)
−1
(y))|Ji
| 1h(Si)(y).
Example 9.25. Fix λ > 0. Let X1 ∼ Exponential(λ) and X2 ∼ Exponential(λ) be independent
RVs defined on the same probability space. The joint distribution of (X1, X2) is given by the joint
108
p.d.f.
fX1,X2
(x1, x2) = fX1
(x1)fX2
(x2) =



1
λ2 exp(−
x1+x2
λ
), if x1 > 0, x2 > 0
0, otherwise.
Consider the function
h(x1, x2) =



(x1 + x2,
x1
x1+x2
), ∀x1 > 0, x2 > 0,
0, otherwise.
Here, {(x1, x2) ∈ R
2
: fX1,X2
(x1, x2) > 0} = {(x1, x2) ∈ R
2
: x1 > 0, x2 > 0} and h : {(x1, x2) ∈
R
2
: x1 > 0, x2 > 0} → R
2
is one-to-one with range (0,∞) × (0, 1). The inverse function is
h
−1
(y1, y2) = (y1y2, y1(1 − y2)) for (y1, y2) ∈ (0,∞) × (0, 1) with Jacobian determinant given by
J(y1, y2) =






y2 y1
1 − y2 −y1






= −y1.
Now, Y = (Y1, Y2) = h(X1, X2) = (X1 + X2,
X1
X1+X2
) has the joint p.d.f. given by
fY1,Y2
(y1, y2) =



fX1,X2
(y1y2, y1(1 − y2))|J(y1, y2)|, if y1 > 0, y2 ∈ (0, 1)
0, otherwise.
=



1
λ2 y1 exp 
−
y1
λ

, if y1 > 0, y2 ∈ (0, 1)
0, otherwise.
Now, we compute the marginal distributions. The marginal p.d.f. fY1
is given by
fY1
(y1) = Z ∞
−∞
fY1,Y2
(y1, y2) dy2 =



1
λ2 y1 exp 
−
y1
λ

, if y1 > 0
0, otherwise
and the marginal p.d.f. fY2
is given by
fY2
(y2) = Z ∞
−∞
fY1,Y2
(y1, y2) dy1 =



1, if y2 ∈ (0, 1)
0, otherwise
.
109
Therefore Y1 = X1 + X2 ∼ Gamma(2, λ) and Y2 =
X1
X1+X2
∼ Uniform(0, 1). Moreover,
fY1,Y2
(y1, y2) = fY1
(y1)fY2
(y2), ∀(y1, y2) ∈ R
2
and hence Y1 and Y2 are independent.
Remark 9.26. We had earlier mentioned that Exponential(λ) distribution is the same as Gamma(1, λ)
distribution. Using the above computation, we can identify a Gamma(2, λ) RV as a sum of two
independent RVs each having distribution Gamma(1, λ). A more general property in this regard
is mentioned in practice problem set 8.
We now consider expectations for random vectors and for functions of random vectors. The
concepts are same as discussed in the case of RVs.
Definition 9.27 (Expectation/Mean/Expected Value for functions of Random Vectors). Let X =
(X1, X2, · · · , Xp) be a p-dimensional discrete/continuous random vector with joint p.m.f./p.d.f. fX.
Let h : R
p → R be a function. Then h(X) is an one-dimensional random vectors, i.e. an RV. We
say that the expectation of h(X), denoted by Eh(X), is defined as the quantity
Eh(X) :=



P
x∈SX
h(x)fX(x), if P
x∈SX
|h(x)|fX(x) < ∞ for discrete X,
R ∞
−∞
R ∞
−∞ · · · R ∞
−∞ h(x)fX(x) dx, if R ∞
−∞
R ∞
−∞ · · · R ∞
−∞ |h(x)|fX(x) dx < ∞ for continuous X.
In the discrete case, SX denotes the support of X.
Remark 9.28. If the sum or the integral above converges absolutely, we say that the expectation
Eh(X) exists or equivalently, Eh(X) is finite. Otherwise, we shall say that the expectation Eh(X)
does not exist.
The following results is a generalization of Proposition 6.19. We skip the proof for brevity.
Proposition 9.29. (a) Let X = (X1, X2, · · · , Xp) be a discrete random vector with joint p.m.f.
fX and support SX and let h : R
p → R be a function. Consider the discrete RV Y := h(X)
with p.m.f. fY and support SY . Then EY exists if and only if P
y∈SY
|y|fY (y) < ∞ and in
110
this case,
EY = Eh(X) = X
x∈SX
h(x)fX(x) = X
y∈SY
yfY (y).
(b) Let X = (X1, X2, · · · , Xp) be a continuous random vector with joint p.d.f. fX. Let h :
R
p → R be a function such that the RV Y := h(X) is continuous with p.d.f. fY . Then EY
exists if and only if R ∞
−∞ |y|fY (y) dy < ∞ and in this case,
EY = Eh(X) = Z ∞
−∞
Z ∞
−∞
· · · Z ∞
−∞
h(x)fX(x) dx =
Z ∞
−∞
yfY (y) dy.
Note 9.30. As considered for the case of RVs, by choosing different functions h : R
p → R, we
obtain several quantities of interest of the form Eh(X) for a p-dimensional random vector X.
Definition 9.31 (Some special expectations for Random Vectors). Let X = (X1, X2, · · · , Xp) be
a p-dimensional discrete/continuous random vector.
(a) (Joint Moments) For non-negative integers k1, . . . , kp, let h(x) := x
k1
1
· · · x
kp
p
, ∀x ∈ R
p
.
Then,
µ
′
k1,...,kp
:= E

X
k1
1
· · · X
kp
p

is called a joint moment of order k1 + · · · + kp of X, provided it exists.
(b) (Joint Central Moments) For non-negative integers k1, . . . , kp, let
h(x) := (x1 − E (X1))k1
· · ·(xp − E (Xp))kp
, ∀x ∈ R
p
.
Then
µk1,...,kp
:= E

(X1 − E (X1))k1
· · ·(Xp − E (Xp))kp

is called a joint central moment of order k1 + · · · + kp of X, provided it exists.
(c) (Covariance) Fix i, j = 1, . . . , p. Let h(x) := (xi − E (Xi)) (xj − E (Xj )), ∀x = (x1, x2, · · · , xp) ∈
R
p
. Then, E [(Xi − E (Xi)) (Xj − E (Xj ))] is called the covariance between Xi and Xj
, provided it exists. We shall denote this quantity by Cov(Xi
, Xj ).
(d) (Joint Moment Generating Function, or simply, Joint MGF) We define
A := n
t = (t1, t2, . . . , tp) ∈ R
p
: E

e
Pp
i=1 tiXi

< ∞
o
,
111
and consider the function MX : A → R defined by
MX(t) = E

e
Pp
i=1 tiXi

, ∀t = (t1, t2, . . . , tp) ∈ A.
If (−a1, a1) × (−a2, a2) × · · · × (−ap, ap) ⊆ A for some a1, a2, · · · , ap > 0, then the function
MX is called the joint moment generating function (joint MGF) of the random vector X.
Note that t = (0, 0, · · · , 0) ∈ R
p yields MX(t) = 1 and hence (0, 0, · · · , 0) ∈ A.
(e) (Joint Characteristic Function) Define ΦX : R
p → C by
ΦX(t) = E

e
Pp
j=1 i tjXj

= E exp (i t.X) = E cos(t.X) + i E sin(t.X), ∀t = (t1, t2, . . . , tp) ∈ R
p
,
where i denotes the complex number √
−1 and t.X =
Pp
j=1 tjXj
is the standard dot product
in R
p
. This function exists for all t ∈ R
p
.
Remark 9.32. We now list some properties of the above quantities. The properties are being
stated under the assumption that the expectations involved exist. Let X = (X1, X2, · · · , Xp) be a
p-dimensional discrete/continuous random vector.
(a) Let a1, . . . , ap be real constants. Then, E
 X
p
i=1
aiXi
!
=
X
p
i=1
aiEXi
. To see this for discrete
X, observe that
E
 X
p
i=1
aiXi
!
=
X
x∈SX
X
p
i=1
aixifX(x) = X
p
i=1
X
x∈SX
aixifX(x) = X
p
i=1
aiEXi
.
The interchange of the order of summation is allowed due to absolute convergence of the
series involved. The proof for continuous X is similar.
(b) Cov(Xi
, Xj ) = Cov(Xj
, Xi), for all i, j = 1, . . . , p.
(c) Cov(Xi
, Xi) = V ar(Xi), for all i = 1, . . . , p.
(d) For all i, j = 1, . . . , p, we have
Cov(Xi
, Xj ) = E [XiXj − Xi(EXj ) − Xj (EXi) + (EXi)(EXj )]
= E (XiXj ) − E (Xi) E (Xj )
112
(e) Let X1, X2, . . . , Xp, Y1, Y2, . . . , Yq be RVs, and let a1, . . . , ap, b1, . . . , bq be real constants.
Then,
Cov


X
p
i=1
aiXi
,
X
q
j=1
bjYj

 =
X
p
i=1
X
q
j=1
aibjCov (Xi
, Yj ).
In particular,
V ar X
p
i=1
aiXi
!
=
X
p
i=1
a
2
i V ar (Xi) + X
p
i=1
X
p
j=1
j̸=i
aiajCov (Xi
, Xj )
=
X
p
i=1
a
2
i V ar (Xi) + 2 X
1≤i<j≤p
aiajCov (Xi
, Xj ).
(f) Let X1, X2, · · · , Xp be independent and let h1, h2, · · · , hp : R → R be functions. Then
E
 Y
p
i=1
hi(Xi)
!
=
Y
p
i=1
Ehi(Xi).
For simplicity, we discuss the proof when p = 2 and X = (X1, X2) is continuous with joint
p.d.f. fX. Recall from Theorem 9.9 that fX(x1, x2) = fX1
(x1)fX2
(x2), ∀x1, x2, ∈ R. Then,
E
 Y
2
i=1
hi(Xi)
!
=
Z ∞
−∞
Z ∞
−∞
h1(x1)h2(x2)fX(x1, x2) dx1dx2
=
Z ∞
−∞
Z ∞
−∞
h1(x1)h2(x2)fX1
(x1)fX2
(x2) dx1dx2
=
Z ∞
−∞
h1(x1)fX1
(x1) dx1
 Z ∞
−∞
h2(x2)fX2
(x2) dx2

=
Y
2
i=1
Ehi(Xi).
(g) This is a special case of statement (f). Let A1, A2, · · · , Ap ⊆ R. Consider the functions
hi(xi) :=



1, if x ∈ Ai
0, otherwise.
= 1Ai
(xi), ∀xi ∈ R, i = 1, 2, · · · , p.
Note that Ehi(Xi) = R ∞
−∞
R ∞
−∞ · · · R ∞
−∞ 1Ai
(xi)fX1
(x1)fX2
(x2)· · · fXp
(xp) dx1dx2 · · · dxp =
R ∞
−∞ 1Ai
(xi)fXi
(xi) dxi = P(Xi ∈ Ai), when X is continuous. The same equality is also
113
true when X is discrete. Now, consider the function h : R
p → R defined by h(x) =
Qp
i=1 hi(xi), ∀x ∈ R
p
. Using (f), we have
P(X1 ∈ A1, X2 ∈ A2, · · · , Xp ∈ Ap) = Y
p
i=1
P(Xi ∈ Ai).
(h) Continue with the assumptions of statement (f). For fixed y1, y2, · · · , yp ∈ R, consider the
functions g1, g2, · · · , gp : R → R defined by
gi(xi) :=



1, if hi(xi) ≤ yi
0, otherwise.
∀xi ∈ R, i = 1, 2, · · · , p.
Note that Egi(Xi) = P(hi(Xi) ≤ yi) = Fhi(Xi)(yi), ∀i and
Fh1(X1),h2(X2),··· ,hp(Xp)(y1, y2, · · · , yp) = P(h1(X1) ≤ y1, h2(X2) ≤ y2, · · · , hp(Xp) ≤ yp)
=
Y
p
i=1
P(hi(Xi) ≤ yi)
=
Y
p
i=1
Fh(Xi)(yi).
Hence, the RVs h1(X1), h2(X2), · · · , hp(Xp) are independent.
(i) Let X1, X2 be independent RVs. Then E(X1X2) = (EX1)(EX2) and hence, using (d),
Cov(X1, X2) = 0.
Further, if X1, X2, · · · , Xp are independent, then using (e),
V ar X
p
i=1
aiXi
!
=
X
p
i=1
a
2
i V ar (Xi)
for all real constants a1, a2, · · · , ap.
(j) Recall that MX : A → R is given by
MX(t) = E

e
Pp
i=1 tiXi

, ∀t = (t1, t2, . . . , tp) ∈ A,
114
with
A := n
t = (t1, t2, . . . , tp) ∈ R
p
: E

e
Pp
i=1 tiXi

< ∞
o
.
Taking t = (0, 0, · · · , 0) ∈ R
p yields MX(0, 0, · · · , 0) = 1 and hence (0, 0, · · · , 0) ∈ A. In
particular, A ̸= ∅. Also, MX(t) > 0, ∀t ∈ A.
(k) If t = (0, · · · , 0, ti
, 0, · · · , 0) ∈ A, then MX(t) = E

e
Pp
k=1 tkXk

= E

e
tiXi

= MXi
(ti).
Similarly, if t = (0, · · · , 0, ti
, 0, · · · , 0, tj
, 0, · · · , 0) ∈ A, then MX(t) = E

e
Pp
k=1 tkXk

=
E

e
tiXi+tjXj

= MXi,Xj
(ti
, tj ).
(l) This result is being stated without proof. If (−a1, a1) × (−a2, a2) × · · · × (−ap, ap) ⊆ A for
some a1, a2, · · · , ap > 0, then MX possesses partial derivatives of all orders in (−a1, a1) ×
(−a2, a2) × · · · × (−ap, ap). Furthermore, for non-negative integers k1, . . . , kp
E

X
k1
1 X
k2
2
· · · X
kp
p

=
"
∂
k1+k2+k3+···+kp
∂tk1
1
· · · ∂tkp
p
MX(t)
#
(t1,t2...tp)=(0,...,0)
.
For i ̸= j with i, j ∈ {1, . . . , p}, we have
Cov (Xi
, Xj )
= E (XiXj ) − E (Xi) E (Xj )
=
"
∂
2
∂ti∂tj
MX(t)
#
(t1,t2...tp)=(0,...,0)
−
"
∂
∂ti
MX(t)
#
(t1,t2...tp)=(0,...,0)
"
∂
∂tj
MX(t)
#
(t1,t2...tp)=(0,...,0)
=
"
∂
2
∂ti∂tj
ΨX(t)
#
,
where ΨX(t) := ln MX(t), t ∈ A. Compare this with the one-dimensional case in Proposition 6.45.
(m) If X1, X2, · · · , Xp are independent, then for all t ∈ A,
MX(t) = E

e
Pp
i=1 tiXi

= E
 Y
p
i=1
e
tiXi
!
=
Y
p
i=1
E

e
tiXi

=
Y
p
i=1
MXi
(ti).
(n) If (−a1, a1) × (−a2, a2) × · · · × (−ap, ap) ⊆ A for some a1, a2, · · · , ap > 0 and MX(t) =
Qp
i=1 MXi
(ti), ∀t ∈ A, then it can be shown that X1, X2, · · · , Xp are independent. We do
not discuss the proof of this result in this course.
115
(o) As discussed for the case of random variables, the joint Characteristic function for random vectors has nice properties similar to the joint MGF. If some joint moment for the
random vectors exists, then it can be recovered from the partial derivatives of the joint
Characteristic function etc.. We do not discuss these properties in detail in this course. summarise the whole pdf  AND GIVE ITS latex code in summarised form