How to Detect Random Walk and White Noise in Time Series Forecasting | by Bex T. | Towards Data Science

No matter how powerful, machine learning cannot predict everything. A well-known area where it can become pretty helpless is related to time series forecasting.
Despite the availability of a large suite of autoregressive models and many other algorithms for time series, you cannot predict the target distribution if it is white noise or follows a random walk.
So, you must detect such distributions before you make further efforts.
In this article, you will learn what white noise and random walk are and explore proven statistical techniques to detect them.
This is my third article on the time series forecasting series (you can check out the whole series from this list, a new Medium feature).
While the first one was about every single Pandas function to manipulate TS data, the second was about time series decomposition and autocorrelation.
To get the most out of this post, you need to understand at least what autocorrelation is. Here, I will give a brief explanation, but check out my last article if you want to go deeper.
Autocorrelation involves finding the correlation between a time series and a lagged version of itself. Consider this distribution:
Lagging a time series means shifting it 1 or more periods backward:
The Autocorrelation Function (ACF) finds the correlation coefficient between a time series and its lagged version at each lag k. You can plot it using the plot_acf function from statsmodels. Here is what it looks like:
The XAxis is the lag k, and the YAxis is the Pearson’s correlation coefficient at each lag. The red shaded region is a confidence interval. If the height of the bars is outside this region, it means the correlation is statistically significant.
In short, white noise distribution is any distribution that has:
Zero mean
A constant variance/standard deviation (does not change over time)
Zero autocorrelation at all lags
Essentially, it is a series of random numbers, and by definition, no algorithm can reasonably model its behavior.
There are special types of white noise. If the noise is normal (follows a normal distribution), it is called Gaussian white noise. Let’s see an example of this visually:
Even though there are occasional spikes, there are no discernible patterns visible, i.e., the distribution is completely random.
The best way you can validate this is to create the ACF plot:
There are also “strict” white noise distributions — these have strictly 0 serial correlation. This is different from brown/pink noise or other natural random phenomena where there is a weak serial correlation but still remain memory-free.
Even though white noise distributions are considered dead ends, they can be quite useful in other contexts.
For example, in time series forecasting, if the differences between predictions and actual values represent a white noise distribution, you can pat yourself on the back for a job well done.
When the residual errors show any pattern, whether seasonal or trending or have a non-zero mean, this suggests there is still room for improvement. In contrast, if the residuals are purely white noise, you maxed out the abilities of the chosen model.
In other words, the algorithm managed to capture all the important signals and properties of the target. What’s left are the random fluctuations and inconsistent data points that could not be attributed to anything.
For example, we will predict the amount of carbon monoxide in the air using the July Kaggle playground competition. We will leave the inputs “as-is” — we won’t perform any feature engineering, and we will choose a baseline model with default parameters:
There is some pattern in the ACF plot, but they are within the confidence interval. These two plots suggest that Random Forests could capture almost all the important signals from the training data even with default parameters.
A more challenging but equally unpredictable distribution in time series forecasting is a random walk. Unlike white noise, it has non-zero mean, non-constant std/variance, and when plotted, looks a lot like a regular distribution:
Random walk series are always cleverly disguised in this manner, but still, they are unpredictable as ever. The best guess for today’s value is yesterday’s.
A common confusion among beginners is thinking of a random walk as a simple sequence of random numbers. This is not the case because, in a random walk, each step is dependent on the previous step.
For this reason, the Autocorrelation function of random walks does return non-zero correlations.
The formula of a random walk is simple:
Whatever the previous data point is, add some random value to it and continue for as long as you like. Let’s generate this in Python with a starting value of, let’s say, 99:
Let’s also plot the ACF:
As you can see, the first ~40 lags yield statistically significant correlations.
So, how do we detect a random walk when a visualization is not an option?
Because of how they are created, differencing the time series should isolate the random addition of each step. Taking the first-order difference is done by lagging the series by 1 and subtracting it from the original. Pandas has a convenient diff function to do this:
If you plot the first-order difference of a time series and the result is white noise, then it is a random walk.
A slight modification to regular random walks is adding a constant value called a drift at random step:
Drift is often denoted with μ, and in terms of values changing over time, drift means gradually changing into something.
For example, even though stocks fluctuate constantly, they might have a positive drift, i.e., gain an overall gradual increase over time.
Now, let’s see how to simulate this in Python. We will first create the regular random walk with a start value of 25:
From the above formula, we see that we need to add the desired drift at each step. Let’s add a drift of 5 and look at the plot:
Despite the wild fluctuations, the series has a discernible upward drift. If we perform differencing, we will see that the series is still a random walk:
drifty_walk.diff().plot();
You might ask if there are better methods of identifying random walks than just “eyeballing” them from plots.
As an answer, there is a hypothesis test outlined in 1979 by Dicker D. A. and Fuller W. A., and it is called the augmented Dickey-Fuller test.
Essentially, it tries to test the null hypothesis that a series follows a random walk. Under the hood, it regresses the difference in prices on the lagged price.
If the found slope (β) is equal to 0, the series is a random walk. If the slope is significantly different from 0, we reject the null hypothesis that the series follows a random walk.
Fortunately, you don’t have to worry about the math because the test is already implemented in Python.
We import the adfuller function from statsmodels and use it on the drifty random walk created in the last section:
We look at the p-value, which is ~0.26. Since 0.05 is the significance threshold, we fail to reject the null hypothesis that drifty_walk is a random walk, i.e., it is a random walk.
Let’s perform another test on a distribution we know isn’t a random walk. We will use the carbon monoxide target from the TPS July Kaggle playground competition:
The p-value is extremely small, suggesting we can easily reject the null hypothesis that target_carbon_monoxide follows a random walk.
We only finished the third part of this Time Series “series,” and you already know a ton.
From here on, things are only going to get more and more interesting as we draw closer to the actual “forecasting” part in the series. There are some interesting articles planned on key time series topics such as stationarity and time-series cross-validation.
Besides, I will dedicate a post solely on feature engineer specific to time series — this is something to be excited about! Stay tuned!
Every Pandas Function You Can (Should) Use to Manipulate Time Series
Advanced Time Series Analysis in Python: Decomposition, Autocorrelation
Matplotlib vs. Plotly: Let’s Decide Once and for All