Introduction to Time Series Analysis | Smoothing | Part 1
Tags: Time Series Forecasting, Time Series Analysis, Machine Learning

By helping us identify hidden trends and make future predictions, time series can help us deal with problems like a sales forecast, inventory management, and budget analysis. Now there are multiple ways to model and forecast time series and over the course of this series, I will try to touch on as many of them as possible. The goal is to improve my own understanding along the way and have a better grasp of the underlying mathematics.

It is quite natural for most real-world data collections to have at least some sort of randomness; and to study underlying characteristics (seasonality, cyclicalness, etc), we need to reduce its effect. Thankfully, smoothing techniques like averaging and exponential smoothing allow us to achieve that.

Let’s say we have the past data of how many items were sold on our website every day, and we want to use to it to take some business decisions. (I am hoping this ambiguous statement hides my ignorance well)

Averaging

The simplest way to make some sense of this data would be to smooth it so that we can find average numbers of items sold daily. Maybe, we can use it replenish our inventory.

We just sum all the values and then divide them by the number of days.

items_sold = [7,6,1,8,10,9,8,11]

average = sum(items)/len(items)

It seems we are selling 7.5 items every day, and it is time to make our first data-driven decision by order 750 items for the next 10 days.

Now let’s see how well we did by evaluating our result using mean squared error.

Delta of actual versus average values

Square of this difference

Sum of this difference

And finally, a mean of this sum

error = [(actual — predicted)**2 for actual in item_sale]

[0.25, 2.25, 42.25, 0.25, 6.25, 2.25, 0.25, 12.25]

And on taking a mean of the sum of these errors, we get:

8.25

So that’s our error.

Now, it has been a good start but there is a very obvious problem with just using the average. Say Jake Peralta and Captain Holt are seen ordering items from our online store, and, in no time, we are the hippest place in the town. However, our sales don’t shoot up, but they do increase steadily.

Suddenly, our one-trick business intelligence falls on its face. Our mean squared error increases and we are clueless why items are out of stock even though we ordered 170 of them for the next 10 days.

Well, the problem is apparent; our averaging technique gives equal weight to all past observations and is thus not quite effective when it is dealing with trends.

Single Moving Average

Single Moving Average is an averaging technique — which instead of taking into account all the previous observations — uses the mean of only the recent items. If the size of our moving window is 3, it will use the average of the last 3 items.

The results we get are more encouraging, and our MSE(Mean Squared Error) falls to 41 from 139. This is expected since this new technique is able to forget our dark, pre-fame days.

An insightful observation here would be why not place the mean of the three items in the middle of the averaging set, i.e. the first average should be added against Day 2 instead of Day 3. This, in fact, would be a more sensible approach and is known as Centered Moving Average.

But what about the times when our window length (M) is even? The middle of the set would then be 1.5,2.5,3.5….

The solution is to apply moving average on already averaged values with a window size of M = 2.

Unfortunately, these averaging techniques are not suitable for forecasting when our data shows a notable trend. In this case, a more effective approach would be to use double moving averages, i.e. get a second moving from the first moving average using the same window length. This is known as Double Moving Averages for a Linear Trend Process and allows us to make future predictions.

Exponential Smoothing Techniques

In real-life, one sure way to stagnate is by becoming a complete slave to history, and it needs extra care to avoid that. We need a similar kind of caution when it comes to time-series where we give more importance to recent examples. One popular way to do is by through Exponential Smoothing Techniques. Here, as we move to previous observations, we exponentially reduce the weight given to them using different smoothing parameters.

In the following sections, we would go over Single, Double and Triple Exponential Smoothing.

Single Exponential Smoothing

In this type of smoothing, we need a time zero smoothed value and a rate of decrease. This first value can either be the actual initial value, or maybe an average of the first few values, or through some other method. One thing to remember is that smaller the value of the exponential rate, the more significant is the effect of this initial value.

Some Mathematics

This is what the basic equation of exponential smoothing looks lie:

`St = a*yt_1 + (1- a)*St_1`

and the rate at which the weight of previous observations decreases is determined by the smoothing factor (a) which can have a value between 1 and 0.

Say a is .7, this means the value at time t would be equal to a sum of percentage of the actual (a*yt_1) and the smoothed ( (1- a)*St_1) value at t - 1.

Now, if we expand St_1 further, we see the

St = a yt-1 + (1 — a) [ayt-2 + (1 — a)St-2]

If we continue expanding it, we would see something like this for say t = 4

a*[(1-a)⁰*y3 + (1-a)¹ *y2] + (1-a)³*S2

Where S2 is the initial value we started with. We can note that the weight of the previous samples decrease exponentially (1-a)⁰, (1-a)¹, (1-a)²…..

The general form we get is:

Example

Say, we collect the following values over a period of 10 days.

6.4, 5.6, 7.8, 8.8, 11, 11.6, 16.7, 15.3, 21.6, 22.4

Using our small exponential smoothing snippet

We get the following results

Single Exponential Smoothing

While this is much better than equal averaging, we can note that this method is really bad at predicting future values in case of a trend. See how after the 10th day the graph flattens.

Now to fix that we might have to upgrade our basic equation so that it takes into account the delta in previous values by using Double Exponential Smoothing Technique

Double Exponential Smoothing Technique

In this technique, we use another γ constant along with the smoothing factor constant α.

What for Single Exponential Smoothing Technique was

St = a*yt_1 + (1 — a)*St_1

Now becomes:

St = a*yt_1 + (1 — a)*(St_1 + bt_1)

Where we add the previous trend — bt_1 — to our smoothed value.

But how do we find the trend at a time period? It definitely would include the delta between previous values to see the average historical difference. And we see the same in the trend equation

b_t = γ*(St - St_1)+(1−γ)*bt_1 St — St_1 : Difference between the current and previous smoothed values bt_1 : Trend at t-1

It seems that this would perform better on our last example. However, before we proceed we need to find the initial value of the trend factor b. While there are various powerful methods to find an optimal starting point, we can use the first difference, or average of the past differences.

Example

Applying this technique on our previous dataset using the following code:

Applying Double Exponential Smoothing

we get a much better result:

And our mean squared error falls to 3.17 from 8.88 we got with single exponential smoothing.

Forecast

Up till now we have looked at ways to smoothing our data so that it is easier for us to interpret. But what if we want to see into the future?

For double exponential smoothing, we can sum the current smoothed value and the trend value to predict the next value. In general, the forecast can be found using:

Forecast at time t+M = Smoothed Value at time t + M*Trend Factor at time t

In case of Single Exponential Smoothing, future predictions is a horizontal line since it gives us the last prediction for all future values. This, as shown previously, means it is not a great technique when trends are involved.

The following graph shows a comparison of future forecast for the two techniques

The winner is pretty obvious here.

Woohoo!!!!

Now that the initial euphoria is over, we realize that our predictions are going to show a drastic increase. Now what if we had not only a trend but also seasonality.

References

Github Repo: https://github.com/Bazarovay/timeseries