Chapter 1
Probability Concepts
This chapter reviews basic probability concepts that are necessary for themodeling and statistical analysis of financial data.

1.1

Random Variables

We start with the basic definition of a random variable:
Definition 1 A Random variable X is a variable that can take on a given
set of values, called the sample space and denoted SX , where the likelihood of the values in SX is determined by X’s probability distributionfunction (pdf).
Example 2 Future price of Microsoft stock
Consider the price of Microsoft stock next month. Since the price of Microsoft
stock next month is not known with certainty today, we can consider it arandom variable. The price next month must be positive and realisticallyit can’t get too large. Therefore the sample space is the set of positive real
numbers bounded above by some large number: SP = {P : P ∈ [0, M],
M > 0}. It is an open question as to what is the best characterization of theprobability distribution of stock prices. The log-normal distribution is one
possibility1 . ¥1

If P is a positive random variable such that ln P is normally distributed the P has a
log-normal distribution.

1

2

CHAPTER 1 PROBABILITY CONCEPTS

Example 3 Return on Microsoft stock
Consider a one-month investment in Microsoft stock. That is, we buy one
share of Microsoft stock at the end of month t − 1 (today) and plan to sell itat the end of month t. The return on this investment, Rt = (Pt − Pt−1 )/Pt ,is a random variable because we do not know what the price will be at theend of the month. In contrast to prices, returns can be positive or negative
and are bounded from below by -100%. We can express the sample spaceas SRt = {Rt : Rt ∈ [−1, M], M > 0}. The normal distribution is often
a good approximation to the distribution of simple monthly returns, andis a better approximation to the distribution of continuously compoundedmonthly returns. ¥Example 4 Up-down indicator variableAs a final example, consider a variable X defined to be equal to one if themonthly price change on Microsoft stock, Pt −Pt−1 , is positive, and is equal tozero if the price change is zero or negative. Here, the sample space is the set
SX = {0, 1}. If it is equally likely that the monthly price change is positiveor negative (including zero) then the probability that X = 1 or X = 0 is 0.5.
This is an example of a bernoulli random variable. ¥The next sub-sections define discrete and continuous random variables.

1.1.1

Discrete Random Variables

Consider a random variable generically denoted X and its set of possible
values or sample space denoted SX .
Definition 5 A discrete random variable X is one that can take on a finite
number of n diﬀerent values SX = {x1 , x2 , . . . , xn } or, at most, a countably
infinite number of diﬀerent values SX = {x1 , x2 , . . . .}.
Definition 6 The pdf of a discrete random variable, denoted p(x), is a function such that p(x) = Pr(X = x). The pdf must
P satisfy (i) p(x) ≥ 0 for allx ∈ SX ; (ii) p(x) = 0 for all x ∈/ SX ; and (iii) x∈SX p(x) = 1.

Example 7 Annual return on Microsoft stock3

1.1 RANDOM VARIABLES
State of Economy SX = Sample Space p(x) = Pr(X = x)
Depression

-0.30

0.05

Recession

0.0

0.20

Normal0.10

0.50

Mild Boom

0.20

0.20

Major Boom

0.50

0.05

Table 1.1: Probability distribution for the annual return on Microsoft
Let X denote the annual return on Microsoft stock over the next year. We
might hypothesize that the annual return will be influenced by the generalstate of the economy. Consider five possible states of the economy: depression, recession, normal, mild boom and major boom. A stock analyst
might forecast diﬀerent values of the return for each possible state. Hence,
X is a discrete random variable that can take on five diﬀerent values. Table 1.1describes such a probability distribution of the return and a graphicalrepresentation of the probability distribution is presented in Figure 1.1.
¥
The Bernoulli Distribution
Let X = 1 if the price next month of Microsoft stock goes up and X = 0 if
the price goes down (assuming it cannot stay the same). Then X is clearlya discrete random variable with sample space SX = {0, 1}. If the probabilityof the stock price going up or down is the same then p(0) = p(1) = 1/2 andp(0) + p(1) = 1.
The probability distribution described above can be given an exact mathematical representation known as the Bernoulli distribution. Consider two
mutually exclusive events generically called “success” and “failure”. For example, a success could be a stock price going up or a coin landing heads
and a failure could be a stock price going down or a coin landing tails. In
general, let X = 1 if success occurs and let X = 0 if failure occurs. Let
Pr(X = 1) = π, where 0 < π < 1, denote the probability of success. Then
Pr(X = 0) = 1 − π is the probability of failure. A mathematical model

4

CHAPTER 1 PROBABILITY CONCEPTS

0.0

0.1

0.2

0.3

0.4

0.5

Annual Return on Microsoft

-0.3

0

0.1

0.2

0.5

returnFigure 1.1: Discrete distribution for Microsoft stock.
describing this distribution isp(x) = Pr(X = x) = π x (1 − π)1−x , x = 0, 1.

(1.1)

When x = 0, p(0) = π 0 (1 − π)1−0 = 1 − π and when x = 1, p(1) = π 1 (1 −π)1−1 = π.
The Binomial Distribution
To be completed.

1.1.2

Continuous Random Variables

Definition 8 A continuous random variable X is one that can take on any
real value. That is, SX = {x : x ∈ R}.
Definition 9 The probability density function (pdf) of a continuous randomvariable X is a nonnegative function f, defined on the real line, such that for
any interval A
Z
Pr(X ∈ A) =f (x)dx.
A

5

0.2
0.0

0.1

pdf

0.3

0.4

1.1 RANDOM VARIABLES

-4

-2

0

2

4

x

Figure 1.2: Pr(−2 ≤ X ≤ 1) is represented by the area under the probability
curve.
That is, Pr(X ∈ A) is the “area under the probabilityRcurve over the interval∞
A”. The pdf f (x) must satisfy (i) f (x) ≥ 0; and (ii) −∞ f (x)dx = 1.

A typical “bell-shaped” pdf is displayed in Figure 1.2 and the area under
the curve between −2 and 1 represents Pr(−2 ≤ X < 1). For a continuous
random variable, f (x) 6= Pr(X = x) but rather gives the height of theprobability curve at x. In fact, Pr(X = x) = 0 for all values of x. That is,probabilities are not defined over single points. They are only defined overintervals. As a result, for a continuous random variable X we havePr(a ≤ X ≤ b) = Pr(a < X ≤ b) = Pr(a < X < b) = Pr(a ≤ X < b).
The Uniform Distribution on an Interval
Let X denote the annual return on Microsoft stock and let a and b be tworeal numbers such that a < b. Suppose that the annual return on Microsoft
stock can take on any value between a and b. That is, the sample space isrestricted to the interval SX = {x ∈ R : a ≤ x ≤ b}. Further suppose thatthe probability that X will belong to any subinterval of SX is proportional to6

CHAPTER 1 PROBABILITY CONCEPTS

the length of the interval. In this case, we say that X is uniformly distributedon the interval [a, b]. The pdf of X has the very simple mathematical form:⎧⎨ 1
for a ≤ x ≤ b,
f (x) = b−a⎩ 0otherwise,
and is presented graphically in Figure xxx. Notice that the area under the
curve over the interval [a, b] (area of rectangle) integrates to one:
Z b

1
1
dx =
b−a
a b−a

Z b

dx =

a

1
1
[x]ba =[b − a] = 1.
b−ab−a[Insert figure here]
Example 10 Uniform distribution on [−1, 1]
Let a = −1 and b = 1, so that b − a = 2. Consider computing the probability
that the return will be between -50% and 50%.We solve
Z 0.5
1
1
1
1
Pr(−50% < X < 50%) =dx = [x]0.5[0.5 − (−0.5)] = .
−0.5 =
2
2
2−0.5 2
Next, consider computing the probability that the return will fall in theinterval [0, δ] where δ is some small number less than b = 1 :1
Pr(0 ≤ X ≤ δ) =
2

Z δ

1
1
dx = [x]δ0 = δ.
220As δ → 0, Pr(0 ≤ X ≤ δ) → Pr(X = 0). Using the above result we see that1
lim Pr(0 ≤ X ≤ δ) = Pr(X = 0) = lim δ = 0.
δ→0
δ→0 2
Hence, probabilities are defined on intervals but not at distinct points. ¥The Standard Normal Distribution
The normal or Gaussian distribution is perhaps the most famous and mostuseful continuous distribution in all of statistics. The shape of the normal

7

0.2
0.0

0.1

pdf

0.3

0.4

1.1 RANDOM VARIABLES

-4

-2

0

2

4

x

Figure 1.3: Standard normal density.
distribution is the familiar “bell curve”. As we shall see, it can be used to describe the probabilistic behavior of stock returns although other distributions
may be more appropriate.
If a random variable X follows a standard normal distribution then we
often write X ∼ N(0, 1) as short-hand notation. This distribution is centeredat zero and has inflection points at ±1. The pdf of a normal random variable
is given by
1 2
1f (x) = √ · e− 2 x − ∞ ≤ x ≤ ∞.
2π
It can be shown via the change of variables formula in calculus that the area
under the standard normal curve is one:
Z ∞
1 2
1
√ · e− 2 x dx = 1.
2π−∞
The standard normal distribution is illustrated in Figure 1.3. Notice thatthe distribution is symmetric about zero; i.e., the distribution has exactly
the same form to the left and right of zero.

The normal distribution has the annoying feature that the area under the8

CHAPTER 1 PROBABILITY CONCEPTS

normal curve cannot be evaluated analytically. That is
Pr(a ≤ X ≤ b) =

Z b

1 2
1
√ · e− 2 x dx,
2πadoes not have a closed form solution. The above integral must be computedby numerical approximation. Areas under the normal curve, in one form or
another, are given in tables in almost every introductory statistics book andstandard statistical software can be used to find these areas. Some usefulapproximate results are
Pr(−1 ≤ X ≤ 1) ≈ 0.67,
Pr(−2 ≤ X ≤ 2) ≈ 0.95,
Pr(−3 ≤ X ≤ 3) ≈ 0.99.

1.1.3

The Cumulative Distribution Function

Definition 11 The cumulative distribution function (cdf) of a random variable X (discrete or continuous), denoted FX , is the probability that X ≤ x :FX (x) = Pr(X ≤ x), − ∞ ≤ x ≤ ∞.
¥The cdf has the following properties:(i) If x1 < x2 then FX (x1 ) ≤ FX (x2 )(ii) FX (−∞) = 0 and FX (∞) = 1(iii) Pr(X > x) = 1 − FX (x)(iv) Pr(x1 < X ≤ x2 ) = FX (x2 ) − FX (x1 )dFX (x) = f (x) if X is a continuous random variable and(v) FX0 (x) = dx
FX (x) is continuous and diﬀerentiable.

Example 12 FX (x) for a discrete random variable1.1 RANDOM VARIABLES

9The cdf for the discrete distribution of Microsoft is given by⎧⎪⎪x < −0.3
0,
⎪
⎪
⎪
⎪
⎪
⎪
0.05, −0.3 ≤ x < 0⎪⎪⎪⎪⎨ 0.25, 0 ≤ x < 0.1
FX (x) =⎪⎪0.75 0.1 ≤ x < 0.2
⎪
⎪
⎪
⎪
⎪
⎪
0.95 0.2 ≤ x < 0.5
⎪
⎪
⎪
⎪
⎩ 1
x > 0.5and is graphed Figure xxx. Notice that the cdf in this case is a discontinuous
step function with jumps at the four return values.
Insert figure here¥
Example 13 F (x) for a uniform random variable

The cdf for the uniform distribution over [a, b] can be determined analytically:Z x
FX (x) = Pr(X < x) =f (t)dt
−∞
Z x
1
x−a
1
[t]xa =
.
dt =
=
b−a a
b−a
b−a
We can determine the pdf of X directly from the cdf viad1
FX (x) =.
f (x) = FX0 (x) =dxb−a¥
Example 14 FX (x) for a standard normal random variable
The cdf of standard normal random variable X is used so often in statistics
that it is given its own special symbol:
Z x
1 2
1
√ e− 2 z dz.
FX (x) = Φ(x) =
(1.2)
2π
−∞

The cdf Φ(x), however, does not have an analytic representation like thecdf of the uniform distribution and must be approximated using numerical
techniques. A graphical representation of Φ(x) is given in Figure 1.4.

10

0.0

0.2

0.4

CDF

0.6

0.8

1.0

CHAPTER 1 PROBABILITY CONCEPTS

-4

-2

0

2

4

x

Figure 1.4: Standard normal cdf Φ(x).

1.1.4

Quantiles of the Distribution of a Random Variable

Consider a random variable X with cdf FX (x) = Pr(X ≤ x). For 0 ≤ α ≤ 1,
the 100 · α% quantile of the distribution for X is the value qα that satisfies
FX (qα ) = Pr(X ≤ qα ) = α.
For example, the 5% quantile of X, q0.05 , satisfies
FX (q0.05 ) = Pr(X ≤ q0.05 ) = 0.05.
The median of the distribution is 50% quantile. That is, the median, q0.5 ,satisfies
FX (q0.5 ) = Pr(X ≤ q0.5 ) = 0.5.
If FX is invertible2 then qα may be determined analytically asqα = FX−1 (α)
where FX−1 denotes the inverse function of FX . Hence, the 5% quantile andthe median may be determined fromq0.05 = FX−1 (.05), q0.5 = FX−1 (.5).
2

The inverse of F (x) will exist if F is strictly increasing and is continuous.

11

1.1 RANDOM VARIABLES
Example 15 Quantiles from a uniform distribution
Let X ∼ U[a, b] where b > a. Recall, the cdf of X is given by
FX (x) =x−a, a ≤ x ≤ b.
b−aGiven α ∈ [0, 1] such that FX (x) = α, solving for x gives the inverse cdf:
x = FX−1 (α) = α(b − a) + a.

(1.3)

Using (1.3), the 5% quantile and median, for example, are given by
q0.05 = FX−1 (.05) = .05(b − a) + a = .05b + .95a,
q0.5 = FX−1 (.5) = .5(b − a) + a = .5(a + b).
If a = 0 and b = 1, then q0.05 = 0.05 and q0.5 = 0.5.
Example 16 Quantiles from a standard normal distribution
Let X ∼ N(0, 1). The quantiles of the standard normal distribution aredetermined by solving
qα = Φ−1 (α),
where Φ−1 denotes the inverse of the cdf Φ. This inverse function must be
approximated numerically and is available in most spreadsheets and statistical software. Using the numerical approximation to the inverse function, the
1%, 2.5%, 5%, 10% quantiles and median are given by
q0.01 = Φ−1 (.05) = −2.33, q0.025 = Φ−1 (.05) = −1.96,
q0.05 = Φ−1 (.05) = −1.645, q0.10 = Φ−1 (.05) = −1.28,
q.05 = Φ−1 (.5) = 0.

1.1.5

R Functions for Discrete and Continuous Distributions

R has built-in functions for a number of discrete and continuous distributions.
These are summarized in Table 1.2. For each distribution, there are fourfunctions starting with d, p, q and r that compute density (pdf) values,
cumulative probabilities (cdf), quantiles (inverse cdf) and random draws,

12

CHAPTER 1 PROBABILITY CONCEPTS

respectively. Consider, for example, the functions associated with the normal
distribution. The functions dnorm(), pnorm() and qnorm() evaluate thestandard normal density (), the cdf (), and the inverse cdf (), respectively,
with the default values mean=1 and sd = 0. The function rnorm() returns
a specified number of simulated values from the normal distribution.
Finding Areas Under the Normal Curve Most spreadsheet and statistical software packages have functions for finding areas under the normal curve. Let X denote a standard normal random variable. Some tables
and functions give Pr(0 ≤ X ≤ z) for various values of z > 0, some give
Pr(X ≥ z) and some give Pr(X ≤ z). Given that the total area under
the normal curve is one and the distribution is symmetric about zero thefollowing results hold:• Pr(X ≤ z) = 1 − Pr(X ≥ z) and Pr(X ≥ z) = 1 − Pr(X ≤ z)
• Pr(X ≥ z) = Pr(X ≤ −z)• Pr(X ≥ 0) = Pr(X ≤ 0) = 0.5
The following examples show how to compute various probabilities.
Example 17 Finding areas under the normal curve using R
First, consider finding Pr(X ≥ 2). By the symmetry of the normal distribution, Pr(X ≥ 2) = Pr(X ≤ −2) = Φ(−2). In R use> pnorm(-2)[1] 0.0228
Next, consider finding Pr(−1 ≤ X ≤ 2). Using the cdf, we compute Pr(−1 ≤X ≤ 2) = Pr(X ≤ 2) − Pr(X ≤ −1) = Φ(2) − Φ(−1). In R use> pnorm(2) - pnorm(-1)[1] 0.8186
Finally, using R the exact values for Pr(−1 ≤ X ≤ 1), Pr(−2 ≤ X ≤ 2) andPr(−3 ≤ X ≤ 3) are13

1.1 RANDOM VARIABLES

Distribution

Function (root) Parameters

Defaults

beta

beta

shape1, shape2_, _binomial

binom

size, prob_, _Cauchycauchylocation, scale

0, 1chi-squaredchisq

df, ncp

_, 1

F

f

df1, df2

_, _

gamma

gamma

shape, rate, scale _, 1, 1/rate

geometric

geom

prob

_

hyper-geometric

hyper

m, n, k_, _, _

log-normal

lnorm

meanlog, sdlog

0, 1logistic

logis

location, scale

0, 1

negative binomial nbinom

size, prob, mu_, _, _

normalnorm

mean, sd0, 1Poissonpois

Lambda1Student’s ttdf, ncp_, 1

uniform

unifmin, max

0, 1

Weibull

weibullshape, scale_, 1

Wilcoxon

wilcoxon

m, n

_, _

Table 1.2: Probability distributions in R

14

CHAPTER 1 PROBABILITY CONCEPTS> pnorm(1) - pnorm(-1)[1] 0.6827> pnorm(2) - pnorm(-2)[1] 0.9545> pnorm(3) - pnorm(-3)[1] 0.9973
Plotting Distributions
When working with a probability distribution, it is a good idea to make plotsof the pdf or cdf to reveal important characteristics. The following examples
illustrate plotting distributions using R.
Example 18 Plotting the standard normal curve
The graphs of the standard normal pdf and cdf in Figures 1.3 and 1.4 werecreated using the following R code:# plot pdf> x.vals = seq(-4, 4, length=150)> plot(x.vals, dnorm(x.vals), type="l", lwd=2, col="blue",+
xlab="x", ylab="pdf")# plot cdf> plot(x.vals, pnorm(x.vals), type="l", lwd=2, col="blue",+
xlab="x", ylab="CDF")¥
Example 19 Shading a region under the standard normal curve
Figure 1.2 showing Pr(−2 ≤ X ≤ 1) as a red shaded area is created with thefollowing code
> lb = -2> ub = 1> x.vals = seq(-4, 4, length=150)> d.vals = dnorm(x.vals)# create plot layout but do not plot anything> plot(x.vals, d.vals, type="n", xlab="x", ylab="pdf")> i = x.vals >= lb & x.vals <= ub

15

1.1 RANDOM VARIABLES# add normal curve> lines(x.vals, d.vals)# add shaded region between -2 and 1> polygon(c(lb, x.vals[i], ub), c(0, d.vals[i], 0), col="red")
¥

1.1.6

Shape Characteristics of Probability Distributions

Very often we would like to know certain shape characteristics of a probability
distribution. We might want to know where the distribution is centered, andhow spread out the distribution is about the central value. We might want toknow if the distribution is symmetric about the center or if the distribution
has a long left or right tail. For stock returns we might want to know about
the likelihood of observing extreme values for returns representing market
crashes. This means that we would like to know about the amount of probability in the extreme tails of the distribution. In this section we discuss four
important shape characteristics of a probability distribution:
1. expected value (mean): measures the center of mass of a distribution
2. variance and standard deviation: measures the spread about the mean
3. skewness: measures symmetry about the mean
4. kurtosis: measures “tail thickness”
Expected Value
The expected value of a random variable X, denoted E[X] or μX , measuresthe center of mass of the pdf. For a discrete random variable X with sample
space SX , the expected value is defined as
X
x · Pr(X = x).
(1.4)μX = E[X] =x∈SX

Eq. (1.4) shows that E[X] is a probability weighted average of the possible
values of X.
Example 20 Expected value of discrete random variable

16

CHAPTER 1 PROBABILITY CONCEPTS

Using the discrete distribution for the return on Microsoft stock in Table 1.1,
the expected return is computed as:
E[X] = (−0.3) · (0.05) + (0.0) · (0.20) + (0.1) · (0.5) + (0.2) · (0.2) + (0.5) · (0.05)= 0.10.
¥
Example 21 Expected value of Bernoulli random variable
Let X be a Bernoulli random variable with success probability π. Then
E[X] = 0 · (1 − π) + 1 · π = πThat is, the expected value of a Bernoulli random variable is its probabilityof success. ¥For a continuous random variable X with pdf f (x), the expected value isdefined asZ ∞x · f (x)dx.
(1.5)
μX = E[X] =−∞Example 22 Expected value of a uniform random variable
Suppose X has a uniform distribution over the interval [a, b]. Then
¸b
∙
Z b
1
1
1 2
E[X] =
x
xdx =
b−a a
b−a 2
a
£ 2
¤
1
2
=
b −a
2(b − a)b+a(b − a)(b + a)=.
=2(b − a)2
If b = −1 and a = 1, then E[X] = 0.
Example 23 Expected value of a standard normal random variable
Let X ∼ N(0, 1). Then it can be shown that
Z ∞
1 2
1
x · √ e− 2 x dx = 0.
E[X] =
2π−∞
Hence, the standard normal distribution is centered at zero.

17

1.1 RANDOM VARIABLES
Expectation of a Function of a Random Variable

The other shape characteristics of the distribution of a random variable X
are based on expectations of certain functions of X. Let g(X) denote somefunction of the random variable X. If X is a discrete random variable withsample space SX thenXg(x) · Pr(X = x),
E[g(X)] =x∈SXand if X is a continuous random variable with pdf f thenZ ∞E[g(X)] =g(x) · f (x)dx.
−∞

Variance and Standard Deviation
The variance of a random variable X, denoted var(X) or σ 2X , measures the
spread of the distribution about the mean using the function g(X) = (X −μX )2 . If most values of X are close to μX then on average (X − μX )2 will besmall. In contrast, if many values of X are far below and/or far above μX
then on average (X − μX )2 will be large. Squaring the deviations about μX
guarantees a positive value. The variance of X is defined as
X(x − μX )2 · Pr(X = x),σ 2X = var(X) = E[(X − μX )2 ] =x∈SXfor X discrete, andσ 2X =Z ∞−∞(x − μX )2 f (x)dx

for X continuous.
Because σ2X represents an average squared deviation, it is not in the sameunits as X. The standard deviation of X, denoted SD(X) or σX , is thesquare root of the variance and is in the same units as X. For “bell-shaped”
distributions, σ X measures the typical size of a deviation from the meanvalue.
Example 24 Variance and standard deviation for a discrete random variable

18

CHAPTER 1 PROBABILITY CONCEPTS

Using the discrete distribution for the return on Microsoft stock in Table 1.1
and the result that μX = 0.1, we havevar(X) = (−0.3 − 0.1)2 · (0.05) + (0.0 − 0.1)2 · (0.20) + (0.1 − 0.1)2 · (0.5)+(0.2 − 0.1)2 · (0.2) + (0.5 − 0.1)2 · (0.05)= 0.020√
SD(X) = σX = 0.020 = 0.141.
Given that the distribution is fairly bell-shaped we can say that typical values
deviate from the mean value of 10% by about 14.1%. ¥Example 25 Variance and standard deviation of a Bernoulli random variable
Let X be a Bernoulli random variable with success probability π. Given that
μX = π it follows that¥var(X) = (0 − π)2 · (1 − π) + (1 − π)2 · π= π 2 (1 − π) + (1 − π 2 )π
= π(1 − π) [π + (1 − π)]= π(1 − π),p
SD(X) = π(1 − π).

Example 26 Variance and standard deviation of a uniform random variable
To be completed
Example 27 Variance and standard deviation of a standard normal randomvariable
Let X ∼ N(0, 1).Here, μX = 0 and it can be shown that
Z ∞
1 2
1
2
x2 · √ e− 2 x dx = 1.
σX =
2π
−∞
It follows that SD(X) = 1.¥

1.1 RANDOM VARIABLES

19

The General Normal Distribution
Recall, if X has a standard normal distribution then E[X] = 0, var(X) = 1.
A general normal random variable X has E[X] = μX and var(X) = σ2X andis denoted X ∼ N(μX , σ 2X ). Its pdf is given by
¾
½
1
1
2
f (x) = p
exp − 2 (x − μX ) , − ∞ ≤ x ≤ ∞.
2σ X
2πσ 2X

Showing that E[X] = μX and var(X) = σ 2X is a bit of work and is good
calculus practice. As with the standard normal distribution, areas under
the general normal curve cannot be computed analytically. Using numericalapproximations, it can be shown that
Pr(μX − σ X < X < μX + σ X ) ≈ 0.67,
Pr(μX − 2σX < X < μX + 2σ X ) ≈ 0.95,
Pr(μX − 3σX < X < μX + 3σ X ) ≈ 0.99.
Hence, for a general normal random variable about 95% of the time we expect
to see values within ± 2 standard deviations from its mean. Observations
more than three standard deviations from the mean are very unlikely.
Example 28 Normal distribution for monthly returns

Let R denote the monthly return on an investment in Microsoft stock, andassume that it is normally distributed with mean μR = 0.01 and standard
deviation σ R = 0.10. That is, R ∼ N(0.01, (0.10)2 ). Notice that σ 2R = 0.01
and is not in units of return per month. Figure 1.5 illustrates the distribution.
Notice that essentially all of the probability lies between −0.4 and 0.4. Using
the R function pnorm(), we can easily compute the probabilities Pr(R <−0.5), Pr(R < 0), Pr(R > 0.5) and Pr(R > 1):> pnorm(-0.5, mean=0.01, sd=0.1)[1] 1.698e-07> pnorm(0, mean=0.01, sd=0.1)[1] 0.4602> 1 - pnorm(0.5, mean=0.01, sd=0.1)[1] 4.792e-07> 1 - pnorm(1, mean=0.01, sd=0.1)
[1] 0

20

2
0

1

pdf

3

4

CHAPTER 1 PROBABILITY CONCEPTS

-0.4

-0.2

0.0

0.2

0.4

x

Figure 1.5: Normal distribution for the monthly returns on Microsoft: R ∼N (0.01, (0.10)2 ).
Using the R function qnorm(), we can find the quantiles q0.01 , q0.05 , q0.95 andq0.99 :
> a.vals = c(0.01, 0.05, 0.95, 0.99)> qnorm(a.vals, mean=0.01, sd=0.10)[1] -0.2226 -0.1545 0.1745 0.2426
Hence, over the next month, there are 1% and 5% chances of losing more than
22.2% and 15.5%, respectively. In addition, there are 1% and 5% chances ofgaining more than 17.5% and 24.3%, respectively.
¥
Example 29 Why the normal distribution may not be appropriate for simple
returns
Let Rt denote the simple annual return on an asset, and suppose that Rt ∼N (0.05, (0.50)2 ). Because asset prices must be non-negative, Rt must always be larger than −1. However, based on the assumed normal distribution
Pr(Rt < −1) = 0.018. That is, there is a 1.8% chance that Rt is smaller than21

1.1 RANDOM VARIABLES−1. This implies that there is a 1.8% chance that the asset price at the endof the year will be negative! This is why the normal distribution may not
appropriate for simple returns. ¥Example 30 The normal distribution is more appropriate for continuously
compounded returns
Let rt = ln(1 + Rt ) denote the continuously compounded annual return on
an asset, and suppose that rt ∼ N(0.05, (0.50)2 ). Unlike the simple return,the continuously compounded return can take on values less than −1. For
example, suppose rt = −2. This implies a simple return of Rt = e−2 − 1 =−0.865. Then Pr(rt ≤ −2) = Pr(Rt ≤ −0.865) = 0.00002. Although thenormal distribution allows for values of rt smaller than −1, the implied simple
return Rt will always be greater than −1.¥
The Log-Normal distribution
Let X ∼ N (μX , σ 2X ), which is defined for −∞ < X < ∞. The log-normally
distributed random variable Y is determined from the normally distributed
random variable X using the transformation Y = eX . In this case, we saythat Y is log-normally distributed and write
Y ∼ ln N(μX , σ 2X ), 0 < Y < ∞.
Due to the exponential transformation, Y is only defined for non-negativevalues. It can be shown that2μY = E[Y ] = eμX +σX /2 ,σ 2Y

2μX +σ2X= var(Y ) = e

(1.6)
σ2X

(e

− 1).

Example 31 Log-normal distribution for simple returns
Let rt = ln(Pt /Pt−1 ) denote the continuously compounded monthly return on
an asset and assume that rt ∼ N (0.05, (0.50)2 ). That is, μr = 0.05 and σ r =t−1
denote the simple monthly return. The relationship0.50. Let Rt = Pt −P
Pt
between rt and Rt is given by rt = ln(1 + Rt ) and 1 + Rt = ert . Since rt
is normally distributed 1 + Rt is log-normally distributed. Notice that the
distribution of 1 + Rt is only defined for positive values of 1 + Rt . This is22

CHAPTER 1 PROBABILITY CONCEPTS

0.4
0.0

0.2

density

0.6

0.8

Normal distribution for r(t)=ln(1+R(t))

-1

0

1

2

r(t)

0.4
0.0

0.2

density0.6

0.8

Log-normal distribution for R(t)=exp(r(t))

0

2

4

6

R(t)

Figure 1.6: Normal distribution for rt and log-normal distribution for Rt =ert .
appropriate since the smallest value that Rt can take on is −1. Using (1.6),
the mean and variance for 1 + Rt are given by2

μ1+R = e0.05+(0.5) /2 = 1.191
2

2

σ 21+R = e2(0.05)+(0.5) (e(0.5) − 1) = 0.563
The pdfs for rt and Rt are shown in figure 1.6.
Using standard deviation as a measure of risk
Consider the following investment problem. We can invest in two nondividend paying stocks, Amazon and Boeing, over the next month. Let RA

1.1 RANDOM VARIABLES

23

denote the monthly return on Amazon and RB denote the monthly returnon Boeing. These returns are to be treated as random variables since thereturns will not be realized until the end of the month. We assume thatRA ∼ N (μA , σ 2A ) and RB ∼ N(μB , σ 2B ). Hence, μi gives the expected return,
E[Ri ], on asset i = A, B and σ i gives the typical size of the deviation of the
return on asset i from its expected value. Figure xxx shows the pdfs for thetwo returns. Notice that μA > μB but also that σ A > σ B . The return we
expect on asset A is bigger than the return we expect on asset B but thevariability of the return on asset A is also greater than the variability on
asset B. The high return variability of asset A reflects the risk associatedwith investing in asset A. In contrast, if we invest in asset B we get a lower
expected return but we also get less return variability or risk. This example illustrates the fundamental “no free lunch” principle of economics andfinance: you can’t get something for nothing. In general, to get a higher
return you must take on extra risk.
insert figure here
Skewness
The skewness of a random variable X, denoted skew(X), measures the symmetry of a distribution about its mean value using the function g(X) =(X − μX )3 /σ 3X , where σ 3X is just SD(X) raised to the third power. For a
discrete random variable X with sample space SX
P
3
E[(X − μX )3 ]x∈SX (x − μX ) · Pr(X = x)skew(X) =
=
.
σ 3X
σ 3X
For a continuous random variable X with pdf p(x)R∞(x − μX )3 · p(x)dx
E[(X − μX )3 ]−∞skew(X) =
=
.
σ 3X
σ 3X
If X has a symmetric distribution then skew(X) = 0 since positive andnegative values in the formula for skewness cancel out. If skew(X) > 0
then the distribution of X has a “long right tail” and if skew(X) < 0 the
distribution of X has a “long left tail”. These cases are illustrated in Figure
6.
insert figure 6 here

24

CHAPTER 1 PROBABILITY CONCEPTS

Example 32 Skewness for a discrete random variable
Using the discrete distribution for the return on Microsoft stock in Table 1,
the results that μX = 0.1 and σ X = 0.141, we haveskew(X) = [(−0.3 − 0.1)3 · (0.05) + (0.0 − 0.1)3 · (0.20) + (0.1 − 0.1)3 · (0.5)+(0.2 − 0.1)3 · (0.2) + (0.5 − 0.1)3 · (0.05)]/(0.141)3= 0.0
Example 33 Skewness for a normal random variable
Suppose X has a general normal distribution with mean μX and varianceσ 2X . Then it can be shown thatZ ∞− 12 (x−μX )2(x − μX )3
1
2σ
X
√
·
e
dx = 0.
skew(X) =
σ 3X
2πσ 2
−∞
This result is expected since the normal distribution is symmetric about it’smean value μX .
Example 34 Skewness for a log-Normal random variable
Let Y = eX , where X ∼ N(μX , σ 2X ), be a log-normally distributed randomvariable with parameters μX and σ 2X . Then it can be shown that
´p
³ 2
2
σX
eσX − 1 > 0
skew(Y ) = e + 2
Notice that skew(Y ) is always positive, indicating that the distribution of Y
has a long right tail, and that it is an increasing function of σ 2X .

Kurtosis
The kurtosis of a random variable X, denoted kurt(X), measures the thickness in the tails of a distribution and is based on g(X) = (X − μX )4 /σ 4X . Fora discrete random variable X with sample space SX
P
4E[(X − μX )4 ]x∈SX (x − μX ) · Pr(X = x)kurt(X) =
=
σ 4X
σ 4X

1.1 RANDOM VARIABLES

25

where σ 4X is just SD(X) raised to the fourth power. For a continuous randomvariable X with pdf p(x)R∞(x − μX )4 · p(x)dx
E[(X − μX )4 ]−∞kurt(X) =
=
.
σ 4X
σ 4X
Since kurtosis is based on deviations from the mean raised to the fourth
power, large deviations get lots of weight. Hence, distributions with large
kurtosis values are ones where there is the possibility of extreme values. In
contrast, if the kurtosis is small then most of the observations are tightly
clustered around the mean and there is very little probability of observing
extreme values.
Example 35 Kurtosis for a discrete random variable
Using the discrete distribution for the return on Microsoft stock in Table 1,
the results that μX = 0.1 and σ X = 0.141, we havekurt(X) = [(−0.3 − 0.1)4 · (0.05) + (0.0 − 0.1)4 · (0.20) + (0.1 − 0.1)4 · (0.5)+(0.2 − 0.1)4 · (0.2) + (0.5 − 0.1)4 · (0.05)]/(0.141)4= 6.5
Example 36 Kurtosis for a normal random variable
Suppose X has a general normal distribution mean μX and variance σ 2X .
Then it can be shown thatZ ∞(x − μX )4
1− 12 (x−μX )2
kurt(X) =·pedx = 3.
4σX
2πσ 2X−∞

Hence a kurtosis of 3 is a benchmark value for tail thickness of bell-shaped
distributions. If a distribution has a kurtosis greater than 3 then the distribution has thicker tails than the normal distribution and if a distribution has
kurtosis less than 3 then the distribution has thinner tails than the normal.
Sometimes the kurtosis of a random variable is described relative to thekurtosis of a normal random variable. This relative value of kurtosis is referred to as excess kurtosis and is defined as
excess kurt(X) = kurt(X) − 3
If excess the excess kurtosis of a random variable is equal to zero then the
random variable has the same kurtosis as a normal random variable. If excess
kurtosis is greater than zero, then kurtosis is larger than that for a normal; ifexcess kurtosis is less than zero, then kurtosis is less than that for a normal.

26

CHAPTER 1 PROBABILITY CONCEPTSThe Student-t Distribution To be completed

1.1.7

Linear Functions of a Random Variable

Let X be a random variable either discrete or continuous with E[X] = μX ,var(X) = σ 2X and let a and b be known constants. Define a new randomvariable Y via the linear function of X
Y = g(X) = aX + b.
Then the following results hold:• E[Y ] = aE[X] + b or μY = aμX + b.
• var(Y ) = a2 var(X) or σ 2Y = a2 σ 2X .
The first result shows that expectation is a linear operation. That is,
E[aX + b] = aE[X] + b.
In the second result notice that adding a constant to X does not aﬀect its
variance and that the eﬀect of multiplying X by the constant a increases thevariance of X by the square of a. These results will be used often enoughthat it useful to go through the derivations, at least for the case that X is adiscrete random variable.
Proof. Consider the first result. By the definition of E[g(X)] withg(X) = b + aX we have
E[Y ] =

X

x∈SX

= a

(ax + b) · Pr(X = x)

X

x∈SX

x · Pr(X = x) + b= aE[X] + b · 1= aμX + b
= μY .

Xx∈SX

Pr(X = x)1.1 RANDOM VARIABLES

27

Next consider the second result. Since μY = aμX + b we havevar(Y ) = E[(Y − μy )2 ]= E[(aX + b − (aμX + b))2 ]= E[(a(X − μX ) + (b − b))2 ]= E[a2 (X − μX )2 ]= a2 E[(X − μX )2 ] (by the linearity of E[·])= a2 var(X)a2 σ 2X .
Notice that our proof of the second result works for discrete and continuous
random variables.
A normal random variable has the special property that a linear functionof it is also a normal random variable. The following proposition establishes
the result.
Proposition 37 Let X ∼ N(μX , σ 2X ) and let a and b be constants. Let
Y = aX + b. Then Y ∼ N(aμX + b, a2 σ 2X ).
The above property is special to the normal distribution and may or may
not hold for a random variable with a distribution that is not normal.
Standardizing a Random Variable
Let X be a random variable with E[X] = μX and var(X) = σ 2X . Define a
new random variable Z as
Z=1μ
X − μX
=
X− X
σX
σX
σX

which is a linear function aX + b where a = σ1X and b = − σμXX . This transformation is called “standardizing” the random variable X since, using theresults of the previous section,μ1μ1
E[X] − X =μX − X = 0
σX
σX
σX
σX
µ ¶2
1
σ2
var(Z) =var(X) = X
= 1.
σX
σ 2X
E[Z] =

28

CHAPTER 1 PROBABILITY CONCEPTS

Hence, standardization creates a new random variable with mean zero andvariance 1. In addition, if X is normally distributed then so is Z.
Let X ∼ N(2, 4) and suppose we want to find Pr(X > 5). Since X is notstandard normal we can’t use the standard normal tables to evaluate Pr(X >5) directly. We solve the problem by standardizing X as follows:
µ
¶
X −2
5−2
√ > √
Pr (X > 5) = Pr
4
4
¶
µ
3
= Pr Z >2¡¢
where Z ∼ N (0, 1) is the standardized value of X. Pr Z > 32 can be founddirectly from the standard normal tables.
Standardizing a random variable is often done in the construction of test
statistics. For example, the so-called “t-statistic” or “t-ratio” used for testing
simple hypotheses on coeﬃcients in the linear regression model is constructedby the above standardization process.
A non-standard random variable X with mean μX and variance σ 2X canbe created from a standard random variable via the linear transformation
X = μX + σ X Z.
This result is useful for modeling purposes. For example, in Chapter 3 wewill consider the Constant Expected Return (CER) model of asset returns.
Let R denote the monthly continuously compounded return on an asset andlet μ = E[R] and σ2 =var(R). A simplified version of the CER model is
R=μ+σ·ε
where ε is a random variable with mean zero and variance 1. The randomvariable ε is often interpreted as representing the random news arriving in agiven month that makes the observed return diﬀer from the expected value
μ. The fact that ε has mean zero means that new, on average, is neutral.
The value of σ represents the typical size of a news shock.

1.1.8

Value at Risk: An Introduction

To illustrate the concept of Value-at-Risk (VaR), consider an investment of$10,000 in Microsoft stock over the next month. Let R denote the monthly

1.1 RANDOM VARIABLES

29

simple return on Microsoft stock and assume that R ~N(0.05, (0.10)2 ). That
is, E[R] = μ = 0.05 and var(R) = σ2 = (0.10)2 . Let W0 denote the investment value at the beginning of the month and W1 denote the investment
value at the end of the month. In this example, W0 = $10, 000. Consider thefollowing questions:• What is the probability distribution of end of month wealth, W1 ?
• What is the probability that end of month wealth is less than $9, 000
and what must the return on Microsoft be for this to happen?
• What is the monthly VaR on the $10, 000 investment in Microsoft stockwith 5% probability? That is, what is the loss that would occur if the
return on Microsoft stock is equal to its 5% quantile, q.05 ?
To answer the first question, note that end of month wealth W1 is related
to initial wealth W0 and the return on Microsoft stock R via the linear
function
W1 = W0 (1 + R) = W0 + W0 R
= $10, 000 + $10, 000 · R.
Using the properties of linear functions of a random variable we haveE[W1 ] = W0 + W0 E[R]= $10, 000 + $10, 000(0.05) = $10, 500
andvar(W1 ) = (W0 )2 var(R)= ($10, 000)2 (0.10)2 ,SD(W1 ) = ($10, 000)(0.10) = $1, 000.
Further, since R is assumed to be normally distributed we have
W1 ∼ N($10, 500, ($1, 000)2 )To answer the second question, we use the above normal distribution for
W1 to getPr(W1 < $9, 000) = 0.067

30

CHAPTER 1 PROBABILITY CONCEPTS

To find the return that produces end of month wealth of $9, 000 or a loss of$10, 000 − $9, 000 = $1, 000 we solveR∗ =$9, 000 − $10, 000= −0.10.
$10, 000

In other words, if the monthly return on Microsoft is −10% or less then endof month wealth will be $9, 000 or less. Notice that −0.10 is the 6.7% quantileof the distribution of R :Pr(R < −0.10) = 0.067
The third question can be answered in two equivalent ways. First, use
R ∼ N(0.05, (0.10)2 ) and solve for the the 5% quantile of Microsoft Stock:
R
R
Pr(R < q.05
) = 0.05 ⇒ q.05
= −0.114.

That is, with 5% probability the return on Microsoft stock is −11.4% or less.
Now, if the return on Microsoft stock is −11.4% the loss in investment value
is $10, 000 · (0.114) = $1, 144. Hence, $1, 144 is the 5% VaR over the next
month on the $10, 000 investment in Microsoft stock.
For the second method, use W1 ~N($10, 500, ($1, 000)2 ) and solve for the
5% quantile of end of month wealth:W1
W1) = 0.05 ⇒ q.05
= $8, 856
Pr(W1 < q.05

This corresponds to a loss of investment value of $10, 000 − $8, 856 = $1, 144.
W1
Hence, if W0 represents the initial wealth and q.05
is the 5% quantile of the
distribution of W1 then the 5% VaR is
W1
5% VaR = W0 − q.05.

In general, if W0 represents the initial wealth in dollars and qαR is theα × 100% quantile of distribution of the simple return R then the α × 100%
VaR may be computed using
VaRα = |W0 · qαR |.
In words, VaRα represents the dollar loss that could occur with probability
α. By convention, it is reported as a positive number (hence the use of theabsolute value function).

31

1.1 RANDOM VARIABLES

Value-at-Risk Calculations for Continuously Compounded Returns
The above calculations illustrate how to calculate value-at-risk using the normal distribution for simple returns. However, as argued in Example xxx, thenormal distribution may not be appropriate for characterizing the distribution of simple returns and is more appropriate for characterizing continuously compounded returns. Let R denote the simple monthly return, let
r = ln(1 + R) denote the continuously compounded return and assume thatr ∼ N(μr , σ 2r )The α × 100% monthly VaR on an investment of $W0 may be computed asfollows:• Compute the α · 100% quantile, qαr , from the Normal distribution forthe continuously compounded return rqα = μr + σ r zα
where zα is the α · 100% quantile of the standard normal distribution.
• Convert the continuously compounded return quantile, qαr , to a simple
return quantile using the transformationrqαR = eqα − 1• Compute VaR using the simple return quantileR
VaRα = |W0 · q.05
|.

1.1.9

Log-Normal Distribution and Jensen’s Inequality(discuss Jensen’s inequality: E[g(X)] < g(E[X]) for a convex function. Use
this to illustrate the diﬀerence between E[W0 exp(R)] and W0 exp(E[R])
where R is a continuously compounded return.) Note, this is where thelog-normal distribution will come in handy.