Gamma Function — Intuition, Derivation, and Examples | by Aerin Kim | Towards Data Science

Why should I care?
Many probability distributions are defined by using the gamma function — such as Gamma distribution, Beta distribution, Dirichlet distribution, Chi-squared distribution, and Student’s t-distribution, etc.For data scientists, machine learning engineers, researchers, the Gamma function is probably one of the most widely used functions because it is employed in many distributions. These distributions are then used for Bayesian inference, stochastic processes (such as queueing models), generative statistical models (such as Latent Dirichlet Allocation), and variational inference. Therefore, if you understand the Gamma function well, you will have a better understanding of a lot of applications in which it appears!
Because we want to generalize the factorial!
The factorial function is defined only for discrete points (for positive integers — black dots in the graph above), but we wanted to connect the black dots. We want to extend the factorial function to all complex numbers. The simple formula for the factorial, x! = 1 * 2 * ... * x, cannot be used directly for fractional values because it is only valid when x is a whole number.
So mathematicians had been searching for...
“What kind of functions will connect these dots smoothly and give us factorials of all real values?”
However, they couldn’t find *finite* combinations of sums, products, powers, exponential, or logarithms that could express x! for real numbers until...
The formula above is used to find the value of the Gamma function for any real value of z.
Let’s say you want to calculate Γ(4.8). How would you solve the integration above? Can you calculate Γ(4.8) by hand? Maybe using the integral by parts?
Try it and let me know if you find an interesting way to do so! For me (and many others so far), there is no quick and easy way to evaluate the Gamma function of fractions manually. (If you are interested in solving it by hand, here is a good starting point.)
Ok, then, forget about doing it analytically. Can you implement this integral from 0 to infinity — adding the term infinite times programmatically?
You can implement this in a few ways. Two of the most often used implementations are Stirling’s approximation and Lanczos approximation.
For implementation addicts: the codes of Gamma function (mostly Lanczos approximation) in 60+ different language - C, C++, C#, python, java, etc.
Let’s calculate Γ(4.8) using a calculator that is implemented already.
We got 17.837.
17.837 falls between 3!(= Γ(4) = 6) and 4!(= Γ(5) = 24) — as we expected.
(When z is a natural number, Γ(z) =(z-1)! We are going to prove this shortly.)
Unlike the factorial, which takes only the positive integers, we can input any real/complex number into z, including negative numbers. The Gamma function connects the black dots and draws the curve nicely.
Confusion-buster: We are integrating over x (NOT z) from 0 to infinity. • x is a helper variable that is being integrated out.• We are NOT plugging 4.8 into x. We are plugging 4.8 into z.
If you take a look at the Gamma function, you will notice two things.
First, it is definitely an increasing function, with respect to z.
Second, when z is a natural number, Γ(z+1) = z! (I promise we’re going to prove this soon!)
Therefore, we can expect the Gamma function to connect the factorial.
How did the Gamma function end up with current terms x^z and e^-x?
I don’t know exactly what Euler’s thought process was, but he is the one who discovered the natural number e, so he must have experimented a lot with multiplying e with other functions to find the current form.
As x goes to infinity ∞, the first term (x^z) also goes to infinity ∞, but the second term (e^-x) goes to zero.
Then, will the Gamma function converge to finite values?
We can rigorously show that it converges using L’Hôpital’s rule. But we can also see its convergence in an effortless way. If you think about it, we are integrating a product of x^z — a polynomially increasing function — and e^-x — an exponentially decreasing function. Because the value of e^-x decreases much more quickly than that of x^z, the Gamma function is pretty likely to converge and have finite values.
Let’s plot each graph, since seeing is believing.
Let’s look at the case of Γ(4.8).
Python code is used to generate the beautiful plots above. Plot it yourself and see how z changes the shape of the Gamma function!
######################### f(x) = exp(-x) graph #########################import matplotlib.pyplot as pltimport numpy as np# Create x and yx = np.linspace(-2, 20, 100)y = np.exp(-x)# Create the plotfig, ax = plt.subplots()plt.plot(x, y, label='f(x) = exp(-x)', linewidth=3, color='palegreen')# Make the x=0, y=0 thickerax.set_aspect('equal')ax.grid(True, which='both')ax.axhline(y=0, color='k')ax.axvline(x=0, color='k')# Add a titleplt.title('f(x) = exp(-x)', fontsize=20)# Add X and y Labelplt.xlabel('x', fontsize=16)plt.ylabel('f(x)', fontsize=16)# Add a gridplt.grid(alpha=.4, linestyle='--')# Show the plotplt.show()##################### f(x) = x^z graph #####################import matplotlib.pyplot as pltimport numpy as np# Create x and yx = np.linspace(0, 2, 100)y1 = x**1.3y2 = x**2.5 y3 = x**3.8# Create the plotfig, ax = plt.subplots()plt.plot(x, y1, label='f(x) = x^1.3', linewidth=3, color='palegreen')plt.plot(x, y2, label='f(x) = x^2.5', linewidth=3, color='yellowgreen')plt.plot(x, y3, label='f(x) = x^3.8', linewidth=3, color='olivedrab')# Make the x=0, y=0 thickerax.set_aspect('equal')ax.grid(True, which='both')ax.axhline(y=0, color='k')ax.axvline(x=0, color='k')# Add a titleplt.title('f(x) = x^z', fontsize=20)# Add X and y Labelplt.xlabel('x', fontsize=16)plt.ylabel('f(x)', fontsize=16)# Add a gridplt.grid(alpha=.4, linestyle='--')# Add a Legendplt.legend(bbox_to_anchor=(1, 1), loc='best', borderaxespad=1, fontsize=12)# Show the plotplt.show()################################ f(x) = x^(3.8)*e^(-x) graph ################################import matplotlib.pyplot as pltimport numpy as np# Create x and yx = np.linspace(0, 20, 100)y = x**3.8 * np.exp(-x)# Create the plotfig, ax = plt.subplots()plt.plot(x, y, label='f(x) = x^(3.8) * np.exp(-x)', linewidth=3, color='palegreen')ax.fill_between(x, 0, y, color='yellowgreen')# Make the x=0, y=0 thickerax.set_aspect('equal')ax.grid(True, which='both')ax.axhline(y=0, color='k')ax.axvline(x=0, color='k')# Add a titleplt.title('f(x) =  x^(3.8)*e^(-x) ', fontsize=20)# Add X and y Labelplt.xlabel('x', fontsize=16)plt.ylabel('f(x)' ,fontsize=16)# Add a gridplt.grid(alpha=.4, linestyle='--')# Add a Legendplt.legend(bbox_to_anchor=(1, 1), loc='upper right', borderaxespad=1, fontsize=12)# Show the plotplt.show()
The code in ipynb: https://github.com/aerinkim/TowardsDataScience/blob/master/Gamma%20Function.ipynb
If you take one thing away from this post, it should be this section.
Property 1.given z > 1Γ(z)   = (z-1) * Γ(z-1)or you can write it asΓ(z+1) =  z    * Γ(z)
Let’s prove it using integration by parts and the definition of Gamma function.
Beautifully proved!
Property 2. If n is a positive integerΓ(n) = (n-1)!
Let’s prove it using the property 1.
What is Γ(1)?
Therefore, Γ(n) = (n-1)!
You might have also seen the expression Γ(n+1) = n! instead of Γ(n) = (n-1)!.This is just to make the right hand side n!, instead of (n-1)! All we did was to shift n by 1.
A quick recap about the Gamma “distribution” (not the Gamma “function”!): Gamma Distribution Intuition and Derivation.
Here goes the proof:
For the proof addicts: Let’s prove the red arrow above.
We’ll use integration by substitution.
Beautifully proved again!
A few things to note:
How old is the Gamma function?
How old is the Gamma function?
Pretty old. About 300 yrs. (Are you working on something today that will be used 300 years later?;)
An interesting side note: Euler became blind at age 64 however he produced almost half of his total works after losing his sight.
2. Some interesting values at points:
Γ(1/2) = sqrt(π)Many interesting ways to show this:https://math.stackexchange.com/questions/215352/why-is-gamma-left-frac12-right-sqrt-piΓ(-1/2) = -2 * sqrt(π)Γ(-1) = Γ(-2) = Γ(-3) = infinity ∞
Can you prove these?
3. Here is a quick look at the graph of the Gamma function in real numbers.
The Gamma function, Γ(z) in blue, plotted along with Γ(z) + sin(πz) in green. (Notice the intersection at positive integers because sin(πz) is zero!) Both are valid analytic continuations of the factorials to the non-integers.
4. Gamma function also appears in the general formula for the volume of an n-sphere.