acceptance-rejection method

acceptance-rejection method

The

acceptance-rejection method

is an algorithm for generating random samples from
an arbitrary probability distribution, given
as ingredients random samples from a related

distribution

and the

uniform distribution

.

The acceptance-rejection method’s chief advantage over the

inverse

CDF
method of generating

random numbers

is that it requires neither the

cumulative distribution function

nor its inverse to be computed. So in many cases it can run faster.

Set-up

•

Let

X

be a

random variable

with some other probability distribution
that we know how to draw samples from — that is, generate on a computer.

•

Let

U

be a random variable

uniformly distributed

on the interval

[

0

,

1

]

.

•

Let

Y

be the random variable that we want to be able to generate.
Assume

Y

has a probability distribution that
is

absolutely continuous

to the probability distribution for

X

,
with

density

ρ

.

•

Further assume that the density

ρ

is bounded above by
a

constant

c

.
So

ρ

⁢

(

x

)

≤

c

for all

x

in the range of

X

;
and necessarily

c

≥

1

.

In most applications, both

X

and

Y

will be continuous random variables
with densities

g

and

f

respectively.
In that case we have

ρ

⁢

(

x

)

=

f

⁢

(

x

)

/

g

⁢

(

x

)

,
and we require

f

⁢

(

x

)

≤

c

⁢

g

⁢

(

x

)

.

(If

g

⁢

(

x

)

=

0

, then set

ρ

⁢

(

x

)

=

0

.
Note that we cannot have

f

⁢

(

x

)

>

0

and

g

⁢

(

x

)

=

0

simultaneously
on a set of

positive measure

, since

Y

has a distribution
absolutely continuous with respect to that of

X

.)

The random variables

X

and

Y

can be multi-variate.

Algorithm

The procedure to generate a value for

Y

is:

1.

Generate a value for

X

.

2.

Generate a value for

U

.

3.

If

U

≤

ρ

⁢

(

X

)

/

c

, then set

Y

=

X

(“

accept

”).

4.

Otherwise, go back to step 1 (“

reject

”),
repeating until we obtain a value of

Y

in step 3.

Intuitive explanation

When we generate

X

and

U

as prescribed in the algorithm,
we are in fact picking the point

(

X

,

c

⁢

U

)

in the rectangular box
below. And the test

U

≤

ρ

⁢

(

X

)

/

c

determines
that point lies below the graph of

ρ

.
It seems plausible that if we keep only the points that
fall under the graph of the density

ρ

, and ignore the points above,
then the distribution of the abscissa
should have density

ρ

.

Figure 1:

Acceptance and rejection regions for a density

The acceptance-rejection method works more efficiently
as the distribution of

X

and

Y

become

similar

enough
— that is,

ρ

⁢

(

x

)

and its

upper bound

c

are close to one.
This makes the rejection region smaller, and so the algorithm
is likely to go through fewer repetitions discarding the rejects.

Justification

We now prove that the acceptance-rejection method works.

Let

X

n

, for

n

∈

ℕ

, be

independent

random variables representing the samples, all with law

G

.
Let

U

n

, for

n

∈

ℕ

,
be independent random variables, with the uniform distribution over

[

0

,

1

]

,
and independent from

X

n

.

Let

N

=

inf

⁡

{

n

∈

ℕ

:

U

n

≤

ρ

⁢

(

X

n

)

c

}

be the number of draws (for

X

n

and

U

n

) taken by the algorithm
before acceptance.
Then we must show that

Y

=

X

N

has the correct distribution:
it should be distributed with density

ρ

⁢

(

x

)

with respect to

d

⁢

G

⁢

(

x

)

.

We have, by independence,

ℙ

(

N

≥

n

)

=

ℙ

(

⋂

k

=

1

n

-

1

{

U

k

>

ρ

⁢

(

X

k

)

c

}

)

=

ℙ

(

U

1

>

ρ

⁢

(

X

1

)

c

)

n

-

1

.

We can calculate the last

probability

explicitly.
Letting

H

be the law of

Z

1

=

ρ

⁢

(

X

1

)

/

c

,
and using the independence of

U

1

from

Z

1

, we find:

ℙ

(

U

1

>

Z

1

)

=

∫

0

1

∫

z

1

𝑑

u

⁢

𝑑

H

⁢

(

z

)

=

∫

0

1

(

1

-

z

)

⁢

𝑑

H

⁢

(

z

)

=

1

-

𝔼

⁢

Z

1

=

1

-

∫

ρ

⁢

(

x

)

c

⁢

𝑑

G

⁢

(

x

)

=

1

-

1

c

.

From the

equation

N

=

∑

n

=

1

∞

𝟏

{

N

≥

n

}

,
we take

expectations

of both sides, evaluating
the resulting geometric series:

𝔼

N

=

∑

n

=

1

∞

ℙ

(

N

≥

n

)

=

∑

n

=

1

∞

(

1

-

1

c

)

n

-

1

=

c

.

Thus

N

<

∞

almost surely,
and the algorithm terminates, on

average

, after drawing

c

samples.

Finally, for all

measurable sets

B

,
we have

ℙ

(

Y

∈

B

)

=

∑

n

=

1

∞

ℙ

(

{

Y

∈

B

}

∩

{

N

=

n

}

)

=

∑

n

=

1

∞

ℙ

(

X

n

∈

B

,

U

n

≤

ρ

⁢

(

X

n

)

c

,

N

≥

n

)

=

∑

n

=

1

∞

ℙ

(

N

≥

n

)

ℙ

(

X

n

∈

B

,

U

n

≤

ρ

⁢

(

X

n

)

c

)

=

∑

n

=

1

∞

ℙ

(

N

≥

n

)

∫

B

∫

0

ρ

⁢

(

x

)

/

c

d

u

d

G

(

x

)

=

(

∫

B

ρ

⁢

(

x

)

c

d

G

(

x

)

)

∑

n

=

1

∞

ℙ

(

N

≥

n

)

=

∫

B

ρ

⁢

(

x

)

⁢

𝑑

G

⁢

(

x

)

,

as is to be shown.

References

1

James E. Gentle.

Random Number Generation and

Monte Carlo Methods

,
second edition. Springer, 2003.

Title

acceptance-rejection method

Canonical name

AcceptancerejectionMethod

Date of creation

2013-03-22 17:19:20

Last modified on

2013-03-22 17:19:20

Owner

stevecheng (10074)

Last modified by

stevecheng (10074)

Numerical id

13

Author

stevecheng (10074)

Entry type

Algorithm

Classification

msc 65C10

Synonym

acceptance-rejection

Synonym

accept-reject algorithm

Related topic

MonteCarloSimulation

Generated on Sat Feb 10 12:35:09 2018 by

LaTeXML