Monte Carlo Integration and Sampling Methods | by Mengsay Loem | Towards Data Science

Integration is a critical calculation used frequently in problem solving. With a probability task, an expectation value of a continuous random variable x is defined by the following integration where p(x) is a probability density function of x.
However, to calculate this value is not an easy process with a computer. To effectively define this integration, numerical approximation methods are used. Here, I will introduce a simple approximation method, Monte Carlo Integration
Monte Carlo Integration is a numerical integration calculation method that uses random numbers to approximate the integration value. Consider the following calculation of the expectation value of f(x). Here, p(x) is a probability density function of x.
In this method, we choose n samples {x_i} (i=1,2,...,n) independent and identically distributed (i.i.d) from the probability identity function p(x). The integration value is approximated by the mean of f(x) for all sample x.
From the law of large numbers, a limitation of the Monte Carlo integration when n →∞ converges to the expectation value of f(x).
Let’s calculate an approximate value of π with the Monte Carlo integration method. Consider a square with side-length 1 and a unit circle as shown in the following figure.
The area of the circle is π. Now let f(x,y) be a function which outputs 1 when (x,y) lies inside the circle, and otherwise 0. Let p(x,y) be a uniform distribution on [-1,1]2. In this case, the area of the unit circle π can be written as
With Monte Carlo integration, this integration (π) can be approximate by using i.i.d samples from p(x,y). n_(in circle) is the number of samples that lie in the unit circle.
The following figure shows the convergence of Monte Carlo integration when n is large enough.
With the Monte Carlo integration, we can simply compute the integration value with random numbers. However, the problem is “how to generate random numbers?” Of course, if the probability distribution is a well-known one, uniform distribution, Gaussian distribution, for example, we can easily implement the random number generator with some library. What if we have to generate random numbers from a distribution density function that is not implemented in any library? To solve this problem, sampling methods are used.
A frequently used method is Importance Sampling. In this method, a proxy distribution is introduced to sample random numbers from any distribution. In most cases, we choose a well-known distribution such as Gaussian distribution, uniform distribution as a proxy distribution. The main concept of this method can be simply written in the following form, where q(x) is a proxy distribution.
With Importance Sampling, instead of generating random numbers from p(x), we choose i.i.d samples {x’_i} (i=1,2,...,n) from a proxy distribution q(x) and approximate the integration value with the following calculation. Here, p(x)/q(x) is called importance of the sampling.
Now, let’s use a Laplace distribution’ variance calculation as an example. Consider f(x)=x2 and a probability density function p(x)=1/2 Exp(-|x|). The distribution with density function like p(x) is called Laplace distribution. If we choose a uniform distribution as a proxy distribution, the variance of the Laplace distribution can be approximately calculated by
With paper and a pencil, we can easily calculate the Var[x]. The value of this calculation is 2. Now, let’s confirm the result of the Importance Sampling method.
In Importance Sampling, we use random numbers (samples) generated from a proxy distribution to approximate the integration value. To directly generated samples from a distribution with any probability density function, I introduce the following sampling method.
In Inverse Transform Sampling method, we use a random number u generated from a 1-dimensional uniform distribution to generate a random number x of any 1-dimensional probability density function p(x). In this case, we use the inverse function of the cumulative distribution function of p(x). If the cumulative distribution function of p(x) is P(x), then the inverse function of u=P(x) is x =P^-1 (u). Now, x = P^-1 (u) has p(x) as probability density function on [0,1]. Therefore, with n samples from uniform distribution on [0,1] {u_i} (i=1,2,...,n), we can generate n sample of p(x) distribution {x_i} (i=1,2,...,n) by calculate x_i = P^-1(u_i).
Again, let's consider the Laplace distribution’s variance calculation as an example. This time we directly generate random numbers (samples) from the Laplace distribution’s probability density function using the Inverse Transform Sampling method. With these random numbers, we will again recalculate the approximated value of Var[x].
Let’s check the result of this method.
The following figure shows the approximated value of Laplace distribution variance V[x] using Inverse Transform Sampling method.
From this result, it is straight forward that using directly generated random numbers from a distribution offers a better approximation compared to a case of proxy distribution. However, with the Inverse Transform Sampling method, it is impossible to directly generate random numbers from only 2 or higher dimension distribution. To do this, Rejection Sampling method is used.
The idea in Rejection Sampling is to use a proxy distribution (Gaussian or uniform distribution, etc.) called q(x) to generate a random number and use another uniform distribution to evaluate the generated sample whether or not to accept it as a sample generated from p(x). With this method, we can also generate random numbers from a higher dimensional distribution.
As preparation in generating random numbers with this method, we need to know a finite value of L where max[p(x)/q(x)] < L. Here, q(x) is a proxy distribution.
First, we generate a random number x’ from a proxy distribution q(x). This x’ is called a proposal point.
Next, generate a random number v from a uniform distribution on [0, L]. This v will be used to evaluate the proposal point, whether to be fine considering generated from p(x).
If v ≤ p(x’)/q(x’), then x’ is accepted as a random number generated by p(x), else, x’ is rejected.
The algorithm in generating n random numbers with Rejection Sampling is
Result on random number generating and Laplace distribution variance calculation (Proxy distribution: Gaussian distribution)
In the Rejection sampling method, it is impossible to generate random numbers when the upper boundary L is not known. MCMC method is an effective solution to this problem. MCMC method uses the concept of a stochastic process (Markov chain in this case). In this case, the generation of i-th sample x_i depends on the previous sample x_(i-1). x1, x2, ...,xn with this concept is called a Markov chain. Here, I introduce one of the MCMC methods, the Metropolis-Hastings method.
The process in this method is similar to Rejection sampling. But here, a proxy distribution density function is represented by a conditional probability q(x|x_i), and the evaluation index v is generated from a uniform distribution on [0,1].
First, we generate a random number x’ from a proxy distribution q(x|x_i). This x’ is called a proposal point.
Next, generate a random number v from a uniform distribution on [0, 1]. This v will be used to evaluate the proposal point, whether to be fine considering generated from p(x).
If v ≤ p(x’)q(x_i|x’)/(p(x_i)q(x’|x_i)), then x’ is accepted as a random number generated by p(x), else, x’ is rejected.
The algorithm in generating n random numbers with Rejection Sampling is
Result on random number generating and Laplace distribution variance calculation (Proxy distribution: Gaussian distribution)
import randomimport numpy as npfrom scipy.stats import norm
def f(x,y):    if (x*x+y*y<=1):        return 1    else:        return 0n = 10N = []P = []while n <= 100000:    s = np.random.uniform(-1,1,(n,2))    nn = 0    for x in s:        nn+=f(x[0],x[1])            pi = 4*nn/n    N.append(n)    P.append(pi)    n = n*2
def f(x):    return x*xdef p_pdf(x):    return 0.5*np.exp(-abs(x))def imp_pdf(x):    return norm.pdf(x)n = 10N = []P = []while n <= 100000:    s = np.random.normal(0,1,n)    nn = 0    for x in s:        nn+=f(x)*p_pdf(x)/imp_pdf(x)            p = nn/n    N.append(n)    P.append(p)    n = n*2
def sgn(x):    if x>0:        return 1    elif x==0:        return 0    else:        return -1    def p_laplace(x):    return 0.5*np.exp(-abs(x))#def P_laplace_cum(x):    #return 0.5*((1+sgn(x))(1-np.exp(-abs(x))))def P_laplace_inv(u):    k = 1-2*abs(u-0.5)    return -1*sgn(u-0.5)*np.log(k)# random number generatordef reverse_generator(X):    res = []    for u in X:        res.append(P_laplace_inv(u))    return resn_sample = 50s = np.random.uniform(0,1,n_sample)theta = reverse_generator(s)   print("n_sample = ",n_sample)gen_xx =random.sample(theta,n_sample)plt.scatter(gen_xx,[0]*n_sample,marker='x')xx=np.linspace(-15,15,1000)plt.plot(xx,[p_laplace(i) for i in xx],c='red')plt.legend(['Laplace Distribution','generated number'])plt.title("Random Numbers directly generated from p(x)")plt.show()# Laplace distribution variance calculationN = []P = []n= 10while n <= 100000:    s = np.random.uniform(0,1,n)    theta = reverse_generator(s)    p = 0    for x in theta:        p += x*x    N.append(n)    P.append(p/n)    n = n*2    P_r = [2]*len(N)plt.plot(N,P)plt.plot(N,P_r,c='red')plt.legend(["Inverse Transform Sampling value","true value(2)"])plt.xlabel("number of samples (n) ")plt.show()
def p_laplace(x):    return 0.5*np.exp(-abs(x))def p_proxy(x):    return norm.pdf(x)# random number generatordef rejection_generator(n):    xx = np.linspace(0,1,100)    K = max(p_laplace(xx)/p_proxy(xx))+1    random_samples = []    while len(random_samples) < n:        proposal = np.random.randn()        v = np.random.uniform(0,K)        if v <= (p_laplace(proposal)/p_proxy(proposal)):            random_samples.append(proposal)    return random_samplesprint("generating...")random_samples = rejection_generator(n=50)print("n_sample = ",len(random_samples))plt.scatter(random_samples,[0]*len(random_samples),marker='x')xx=np.linspace(-15,15,1000)plt.plot(xx,[p_laplace(i) for i in xx],c='red')plt.legend(['true','gen'])plt.show()# Laplace distribution variance calculationN = []P = []n= 10while n <= 100000:    theta = rejection_generator(n)    p = 0    for x in theta:        p += x*x    N.append(n)    P.append(p/n)    n = n*2    P_r = [2]*len(N)plt.plot(N,P)plt.plot(N,P_r,c='red')plt.legend(["Rejection Sampling value","true value(2)"])plt.xlabel("number of samples (n) ")plt.show()
def p_laplace(x):    return 0.5*np.exp(-abs(x))def p_proxy(x,theta,sigma):    return norm.pdf(x,theta,sigma)def MCMC_generator(n):    random_samples = []    theta_i = 0    while len(random_samples) < n:        proposal = np.random.normal(loc=theta_i,scale=1)        v = np.random.uniform(0,1)        a = p_laplace(proposal)*p_proxy(theta_i,proposal,1)        b = p_laplace(theta_i)*p_proxy(proposal,theta_i,1)        if v <= (a/b):            random_samples.append(proposal)            theta_i = proposal    return random_samplesn = 50random_samples = MCMC_generator(n)print("n_sample = ",n)plt.scatter(random_samples,[0]*n,marker='x')xx=np.linspace(-15,15,1000)plt.plot(xx,[p_laplace(i) for i in xx],c='red')plt.legend(['true','gen'])plt.show()N = []P = []n= 10while n <= 10000:    theta = MCMC_generator(n=n)    p = 0    for x in theta:        p += x*x    N.append(n)    P.append(p/n)    n = n*2    P_r = [2]*len(N)plt.plot(N,P)plt.plot(N,P_r,c='red')plt.legend(["Rejection Sampling value","true value(2)"])plt.xlabel("number of samples (n) ")plt.show()
M.Yamasugi. Statistical Machine Learning-generative model-based pattern recognition-. Ohmsha, Japan, 2019
M.Yamasugi. Statistical Machine Learning-generative model-based pattern recognition-. Ohmsha, Japan, 2019