Build Intuition for the Fourier Transform
By: Ravi Charan
Tags: Computer Vision, Mathematics, Data Science, Chemistry, Machine Learning

Build Intuition for the Fourier Transform

A Magical Algorithm for Convolution and Signal Processing

The Fourier Transform and its cousins (the Fourier Series, the Discrete Fourier Transform, and the Spherical Harmonics) are powerful tools that we use in computing and to understand the world around us. The Discrete Fourier Transform (DFT) is used in the convolution operation underlying computer vision and (with modifications) in audio-signal processing while the Spherical Harmonics give the structure of the Periodic Table of the Elements.

While it’s easy enough to write down a formula, there is a broader view revealing that each arises from simple considerations about symmetry. We will pursue that view here.

As always, I will strive to avoid unnecessary technical jargon complications. If you’re interested in them, try the footnotes. There will, however, be some complex numbers: you’ve been warned.

The Fourier Series

The Fourier Series is the oldest of the bunch and was originally studied by a Frenchman, Joseph Fourier. Fourier, a friend of Napoleon’s, wrote about it in the 1820s while studying heat flow, shortly before he discovered the Greenhouse Effect we spend so much time worrying about today.

The natural setup for the Fourier Series is complex-valued functions on a circle.¹ As you will recall, a function is something that takes and input and gives an output.

In this case, the output will be a complex number (we denote the complex numbers as ℂ)

The input will be a point on the circle. Note that we aren’t including the interior of the circle, just its boundary. We denote the circle as S¹. We will think about this circle as having radius 1 if it comes up.

We will also require our functions to be “nice.” They won’t try to do mean things like be jump around (be discontinuous), go to infinity, be undefined, or call you rude names.

Symmetry

Symmetry is one of those mystical things people love to talk about. For example, research has consistently shown that if your face is symmetric, people find you more attractive. It also shows up in physics: the best theory of the world we currently have can cryptically be described as the symmetry SU(3)×SU(2)×U(1), whatever that means. But when we talk about symmetry, we mean something laughably simple: if you rotate a function on a circle, you get a function on a circle.

Now, it’s a little tricky to draw a function on a circle. So please recall that if you take a piece of string and connect the two ends, you can make a circle. With this in mind, we can graph a function on a circle by graphing it on a line segment; the fact that the left and right ends have to be connected you have to visualize in your head.

The figure below shows a simple function on a circle (left). To rotate it, we just “shift” with the proviso that whatever goes off the right end comes back on the left. In this case, the rotation is by ᴨ/2 radians (which is 90°). The input to the function is just the angle around the circle (going clockwise starting from the point all the way on the right, say).

A Function on a Circle (left) and its rotation by 90° (Right)

Another way to think about this is, if you have a function on a circle, you can just rotate the circle (because the circle is, tautologically, circularly symmetric).²

Hopefully you are convinced so far that there is no particular wizardry going on yet. All we said was that if you rotate the function (or the circle), you get another perfectly good function.

Representation Theory

The study of the fact that you can rotate a function on a circle to get another function is called representation theory by mathematicians and it involves one extra step that opens up a whole world of understanding.

The extra step isn’t too complicated in itself. In the previous setup, when you wanted to rotate the circle, you just rotated it. Now we will add an intermediary. I want you to imagine that you are going to give instructions to rotate the circle and I am going to interpret them and do the rotation. For example, you might say “rotate the circle 73°” and I will rotate the circle 73°.

Now, I never said I have to do exactly what you said. There are some rules that apply to what I do, and we’re going to explore them by example.

If you say “rotate the circle 360°” I am not going to do anything. Similarly, if you said “rotate the circle 370°”, I will only rotate it 10. Why? Well, hopefully you see that if I rotate 360°, we are just going to get back to where we started. You could object and say “no no I really wanted to see it spin around, it’s fun” but actually you should be picturing all of these rotations as happening instantaneously. We are not interested in the process of rotating; we’re interested in the result. Now imagine that when you say “rotate the circle 10°” I instead rotate it 15. Likewise, if you say 20°, I will rotate it 30! In general I rotate by 1.5 times what you ask. This may seem all well and good (I never did make any promises) but there is a problem. If you say “rotate the circle 360°” you are, in fact, entitled to the assumption that nothing will change. So if I were to do this “150% effort overachiever” strategy, I would be breaking the rules because I would rotate 180° (the same as doing 360×1.5 = 540 = 360 + 180). It is a rule that, whatever number of degrees you say, I have to do some multiple. So what multiples are allowed? 1 is obviously fine. So is 2! If you ask for 360°, you’ll get no rotation (720°), exactly what you should expected. So is 3, 4, 5, etc. Also 0, -1, -2, etc. These are in fact the only options.

It would be quite a diversion to go deep into the details of where these rules come from. But very briefly, the “rotations of a circle” come with a certain mathematical structure (called a group). Roughly, the structure is that obvious fact that rotating by 1° and then again by 1° is the same as rotating by 2°. Plus the fact that 360° is the same as 0°, which does nothing. The rules:

I have to rotate by a constant multiple of what you say

If you say 360°, I have to end up doing nothing

follow from the requirement that we preserve this structure and in turn imply that I can only multiply by an integer. This consequence is called quantization and is precisely what is meant by the “quantum” in quantum mechanics. In fact, it turns out that the reason that electric charges come in discrete (quantized) amounts follows directly from the argument we have just made. For this reason, the integer multiple I use is sometimes called charge!

To recap, recall that we have:

You, asking for a certain number of degrees of rotation. Let’s call the collection of possible rotations G (for “Group”).

Me, interpreting those instructions. In a sense, I am a function that takes an input (what you tell me) and gives an output (what to do). We’ll call this ρ.

Functions, which are being acted on. We’ll call the collection of all the nice-enough functions V (for “Vector” because functions are vectors: they can be added, multiplied by a constant, and there is a 0 function).

Taken all together, these three mathematical objects form a representation.³ I imagine it is so called because I am “representing” your abstract instructions with a concrete action.

Irreducible Representations

This is where the magic happens. In many contexts (like the one here), the thing being acted on (the functions) can be decomposed into constituent parts.⁴ Roughly what’s going to happen is:

A function could be really complicated Imagine we have a bunch of buckets labeled …-3, -2, -1, 0, 1, 2, 3, 4, etc. and inside each bucket is a very simple function like cos(2θ). Every function can be made by adding up components, 1 from each bucket.

If we have done this, we have broken down a complicated thing into the sum of a number of simple parts.⁵ This is good. If we do this, we win.

Now, proving all of this is going to work is beyond the scope of this article. If you are interested, it’s often the first thing covered in any textbook on representation theory and it isn’t crazy hard or anything. Instead, we are just going to introduce a sleight of hand and then describe the buckets.

Rotation is Multiplication

Consider the function f below, and look at what happens when we rotate it by an angle a. As in the previous diagram, rotation is the same as shifting the argument to the function by a. i is the imaginary unit, the square root of -1 (don’t worry about it for now). Everything else follows from the exponent rules you learned in math class.

Rotation is Multiplication

It turns out (by Schur’s lemma) that the functions in our buckets have to satisfy the property that rotating them is the same as multiplying them. And the functions above, where n is any integer, are exactly the functions that do satisfy this property. Therefore, we can write any function on the circle as a sum of such functions:

Decomposition of functions on the circle

where the coefficients a just tell us how much of each function to take. This decomposition is called the Fourier Series. If you don’t like complex numbers as much, you may be more comfortable using the fact that

which means that we can write every function f on the circle in the form

Trigonometric version of the Fourier Series

and the complex numbers are hiding in the coefficients a and b if they are necessary. (Note: we combined terms like n=2 and n=-2 so we only sum over non-negative n. And for n=0, we only really need one of these terms.)

Finally, note that we might need infinitely many terms whereas in practice we only get finitely many. The animation below shows what happens as we take more and more terms. The blue lines are a “sawtooth wave” which you can think about as a discontinuous function on the circle (it jumps every 360°). The red curve is the Fourier Series approximation with 1, 2, 3, etc. terms.

The Cousins

Let’s recap what we did. We realized that if you rotate a circle you get a circle back (the symmetry). We expanded this so that we worked with functions on the circle and allowed our rotation instructions to be modified (the representation). We learned that the functions can be decomposed into simpler parts (the irreducible representations) and that the simpler parts had to satisfy a simple property that rotation is multiplication (Schur’s lemma). With this we could write down those simple parts.

To recover everything else, all we need to do is consider functions on X where X is something other than the circle.

The Fourier Transform

Okay, let’s make X into ℝ, the real numbers. They symmetry is that you can translate the real numbers (add 5, say) and end up with what you started with. Or stated with functions, a regular function that takes as input a real number and has as output a real number can be translated and you get back another perfectly good function.⁶

Formally, this is similar to what we did on a circle where rotation also looked like a shift. Unlike before, however, the result will not be quantized. Instead of having discrete buckets, we will have on bucket for every real number. To add up a continuous set of things, we need to use an integral instead of a summation; but conceptually this is the same as with the circle.

Change the variable from θ to x

Change the sum to an integral

Change the bucket index from n to p (p stands for “momentum”)

Change the coefficients a so that there is one for every bucket-index p

The Fourier Transform decomposes a function

Note this is not the formula you will see on Wikipedia for a couple reasons. First, the Fourier transform is usually written down with a slightly different normalization (we need a 2ᴨ term somewhere to make some later formulas nice). Second, it is the inverse transform: the transform computes the coefficients a(p), and the inverse transform is how we write down the decomposition of the function f.

The Discrete Fourier Transform

Now let’s change the underlying space to be a “discrete” version of the circle. Let’s say there are N points equally spaced around a circle. The symmetry is that you can rotate by 360/N degrees or any integer multiple of it. A “function” on the “discrete circle” is just a list of N numbers, 1 for each point. Rotating the function is just shifting the list of N numbers (and moving anything that goes off the edge back to the other side).

Now there will only be N buckets. The smallest rotation is by an angle of 2ᴨ/N radians. The variable changes from x to k and k is an integer, one of 0,1,2,…,N-1.

The Discrete Fourier Decomposition

Again, slightly different than what you will see on Wikipedia for the same reasons.

The Discrete-Time Fourier Transform

Now let’s change the underlying space to be a discrete version of the line (the real numbers). So we get the integers. The symmetry is shifting by any integer. A “function” is an infinite list of numbers, one for every integer.

Discrete Time Fourier Decomposition

Again, we aren’t being careful with our 2ᴨ normalizations.

The Spherical Harmonics

What if the underlying space is a sphere? By sphere, I mean the boundary and exclude the interior. This space is denoted S². The symmetry is any rotation of 3 dimensional space. Because this symmetry group is a little more complicated, the number of functions in each “bucket” is not always one.

The result is called the spherical harmonics and you learned about them in chemistry class. The size of these buckets also gives the structure of the periodic table. This is pretty amazing: just by trying to do a Fourier Transform on a sphere, you can basically figure out chemistry. (There are some more details: everything gets doubled because electrons can be spin up or spin down).

I will briefly try to describe the buckets and their relation to the periodic table.

The first bucket is called “s” has only one function in it. Double that and you get 2, the number of elements in the first row of the periodic table (the 1s orbital). The second bucket is called “p” and has three functions in it. Double that and you get 6 electrons that can go in the 2p orbital (alongside the 2 in the 2s orbital). The third bucket is called “d” and has 5 functions in it. Double that and you get 10, the number of electrons that can fit in the 4d orbital (Sc–Zn, the blue elements in the middle below). The fourth bucket is called “f” and has 7 functions in it. Double that and you get 14, the number of electrons that can go in the 6f orbital. (pink La–Yb below and excluding the last one Lu which should properly be colored blue). The fifth bucket is called “g” and has 9 functions in it. Double that and you get 18, the number of electrons that could go in a g-orbital if the atom was stable enough to exist.

And so on (1,3,5,7,9,11, etc.).

Periodic Table (E. Generalic, periodni)

To spell it out, the quantum state of an atom is like a function on a sphere (the radial part factors out and accounts for the 1s, 2s 2p, 3s 3p, 4s 4p 4d part of the structure). If you rotate that state you get another perfectly good state (you’d better!). So the representation theory applies and we can do our “Fourier” analysis.

Conclusion

We’ve seen that the Fourier transform and its cousins can be viewed as doing the same process to a function, with slightly different results depending on the data that make up the function. The table at left captures the 5 versions we described. And of course you can do it on any shape that has a symmetry. For example, you could work out Fourier analysis on a function that is defined on a plane (2-dimensional real numbers) or for an image (which is a finite 2d array of numbers that we can take to loop back on itself, forming a discrete torus).

We’ve also seen just how powerful this tool is. It gives us the periodic table of the elements (essentially our whole world). In computer vision, it turns out that it’s faster to do a (discrete) Fourier transform, multiply the results, and then inverse transform than just naively multiplying numbers. In audio signal processing, the Fourier transform and its variants (like the wavelet transform) is a powerful tool for feature extraction. Tune in next time for applications.