The Counterintuitive Walk
Tags: Statistics, Mathematics, Data Science, Probability, Science

Probability theory is full of surprising results. Often times, these results go against our nature yet reveal key aspects of how the world works. My personal favorite, first introduced to me by the Feynman Lectures, is the random walk. The setup is incredibly simple, yet it yields an interesting outcome that is key for understanding uncertainty.

Let’s set the scene. Imagine you are standing in a line and can choose to take one step either forward or backward. You pick randomly with equal probability of each and continue to do so for a total of n times. Assuming that each step is the same distance (we will refer to this distance as 1 and ignore units) you will be some distance D from the original line. See Figure 1 for some examples.

Figure 1: Two Random Walks

It does not make sense to say that D is a function of n, because this process is not deterministic. D could be drastically different for two walks, even at the same n. Instead, we can talk about what we expect to happen. How far away are we on average after n steps? To specify this value, we will use <D(n)>. The symbols “<>” specify an expectation value. That is, what we get when we average over a very large number of random walks. Since this is no longer random, we can now think of it as a function of n, the number of steps taken.

There’s one more thing to consider before we starting calculating a value. It is meaningless to differentiate between getting far away in the negative direction and positive direction. We could choose to solve for the expectation value of the absolute value of D (wow that’s a mouthful) which is <|D(n)|>. Instead, we’ll use the square of D and solve for <D(n)²>. Squaring the distance will also satisfy our goal of ignoring the difference between positive and negative values. The reasons for taking the square instead of the absolute value will become clear later.

We can actually explicitly define this value for n = 1. Since it’s the first step, our walk either took us 1 step forward (+1) or one step backward (-1). Both give the same number when squared, so <D(1)²> = 1. But what about the next step?

Let’s take a break from expectation values and think about what specifically is happening here. Assume that we know the value of D(n-1). Then, D(n) is either equal to D(n-1)+1, or D(n-1)-1 depending on which way we randomly walked. Squaring this gives us the following two options. Either

D(n)² = D(n-1)²-2*D(n-1)+1 or D(n-1)²+2*D(n-1)+1.

Both outcomes are equally likely, so the expectation value of D(n)² is just the average of the two. This gives a surprisingly simple result of

<D(n)²> = <D(n-1)²>+1.

Hopefully, now you see why we squared <D(n)> instead of taking the absolute value. This process would be much more difficult with the latter! Using this relationship and the fact that <D(1)²> = 1, induction tells us that <D(n)²> = n. Note that I just skipped over some mathematical formalities. If you want to learn about induction and how it would apply to this situation, check out this article.

We’re going to simplify these results by taking the root mean square to get <D(n)> = sqrt(n). Cool! But what does this tell us? As an example, say we have taken 16 steps in total using the random walk as described earlier. Then we expect to be 4 steps away from our starting point. This is not deterministic! If our process is truly random, then there is no way to anticipate our location after 16 steps. We can, however, get a rough idea of where we should be.

I find this outcome pretty interesting. Our average distance will still be 0, since walking forward or backward is equally likely. However, as we continue to take more steps, we expect to have a greater and greater spread of distances from this center.