What is time series decomposition and how does it work?
By: Sachin Date
Tags: Decomposition, Programming, Python, Data Science, Time Series Analysis

A step-by-step procedure for decomposing a time series into trend, seasonal and noise components using Python

There are many decomposition methods available ranging from simple moving average based methods to powerful ones such as STL.

In Python, the statsmodels library has a seasonal_decompose() method that lets you decompose a time series into trend, seasonality and noise in one line of code.

In my articles, we like to get into the weeds. So before we use seasonal_decompose(), let’s do a deep dive into a simple, yet powerful time series decomposition technique.

Let’s understand how decomposition really works under the covers.

We’ll hand-crank out the decomposition of a time series into its trend, seasonal and noise components using a simple procedure based on moving averages using the following steps:

STEP 1: Identify the length of the seasonal period

STEP 2: Isolate the trend

STEP 3: Isolate the seasonality+noise

STEP 4: Isolate the seasonality

STEP 5: Isolate the noise

We’ll use as an example, the following time series of retail sales of user cars dealers in the US:

Retail Used Car Sales. Data source: US FRED (Image by Author)

Let’s load the data into a pandas DataFrame and plot the time series:

import pandas as pd

import numpy as np

import math

from matplotlib import pyplot as plt mydateparser = lambda x: pd.datetime.strptime(x, '%d-%m-%y') df = pd.read_csv('retail_sales_used_car_dealers_us_1992_2020.csv', header=0, index_col=0, parse_dates=['DATE'], date_parser=mydateparser) fig = plt.figure() fig.suptitle('Retail sales of used car dealers in the US in millions of dollars') df['Retail_Sales'].plot() plt.show()

Now let’s begin the step by step decomposition of this time series.

STEP 1: Try to guess the duration of the seasonal component in your data. In the above example, we’ll guess it to be 12 months.

STEP 2: Now run a 12 month centered moving average on the data. This moving average is spread across a total of 13 months. i.e. 6 months each on the left and right side of the center month. The 12 month centered MA is an average of two moving averages that are shifted from each other by 1 month, effectively making it a weighted moving average.

Here is an illustration of how this centered MA can be calculated in Microsoft Excel:

Illustration of a 2 x 12 centered moving average (Image by Author)

This MA will smooth out seasonality and noise and bring out the trend.

Continuing with our Python example, here is how we can calculate the centered moving average in Python:

#Add an empty column to store the 2x12 centered MA values

df['2 x 12 CMA (TREND)'] = np.nan #Fill it up with the 2x12 centered MA values

for i in range(6,df['Retail_Sales'].size-6):

df['2 x 12 CMA (TREND)'][i] = np.round(

df['Retail_Sales'][i - 6] * 1.0 / 24 +

(

df['Retail_Sales'][i - 5] +

df['Retail_Sales'][i - 4] +

df['Retail_Sales'][i - 3] +

df['Retail_Sales'][i - 2] +

df['Retail_Sales'][i - 1] +

df['Retail_Sales'][i] +

df['Retail_Sales'][i + 1] +

df['Retail_Sales']i + 2] +

df['Retail_Sales'][i + 3] +

df['Retail_Sales'][i + 4] +

df['Retail_Sales'][i + 5]

) * 1.0 / 12 +

df['Retail_Sales'][i + 6] * 1.0 / 24

Notice how the values at indices [i-6] and [i+6] are weighted by 1.0/24 while the rest of the values are each weighted by 1.0/12.

Let’s plot the resulting time series that is contained in column ‘2 x 12 CMA (TREND)’:

#plot the trend component

fig = plt.figure() fig.suptitle('TREND component of Retail sales of used car dealers in the US in millions of dollars') df['2 x 12 CMA (TREND)'].plot() plt.show()

As you can see, our moving average transformation has highlighted the trend component of the retail sales time series:

(Image by Author)

STEP 3: Now we have a decision to make. Depending on whether the composition is multiplicative or additive, we’ll need to divide or subtract the trend component from the original time series to retrieve the seasonal and noise components. If we inspect the original car sales time series, we can see that the seasonal swings are increasing in proportion to the current value of the time series. Hence we’ll assume that the seasonality is multiplicative. We’ll also take a small leap of faith to assume that the noise is multiplicative.

Thus the retail used car sales time series is assumed to have the following multiplicative decomposition model:

Time series value = trend component * seasonal component * noise component

Therefore:

seasonal component * noise component = Time series value / trend component

We’ll add a new column into our data frame and fill it with the product of the seasonal and noise components using the above formula.

df['SEASONALITY AND NOISE'] = df['Retail_Sales']/df['2 x 12 CMA (TREND)']

Let’s plot the new column. This time, we will see the seasonality and noise showing through:

fig = plt.figure() fig.suptitle('SEASONALITY and NOISE components') plt.ylim(0, 1.3) df['SEASONALITY AND NOISE'].plot() plt.show()

(Image by Author)

STEP 4: Next, we will get the ‘pure’ seasonal component out of the mixture of seasonality and noise, by calculating the average value of the seasonal component for all January months, all February months, all March months and so on.

#first add a month column

df['MONTH'] = df.index.strftime('%m').astype(np.int) #initialize the month based dictionaries to store the running total of the month wise seasonal sums and counts

average_seasonal_values = {1:0, 2:0, 3:0, 4:0, 5:0, 6:0, 7:0, 8:0, 9:0, 10:0, 11:0, 12:0} average_seasonal_value_counts = {1:0, 2:0, 3:0, 4:0, 5:0, 6:0, 7:0, 8:0, 9:0, 10:0, 11:0, 12:0} #calculate the sums and counts

for i in range(0, df['SEASONALITY AND NOISE'].size):

if math.isnan(df['SEASONALITY AND NOISE'][i]) is False:

average_seasonal_values[df['MONTH'][i]] =

average_seasonal_values[df['MONTH'][i]] +

df['SEASONALITY AND NOISE'][i] average_seasonal_value_counts[df['MONTH'][i]] =

average_seasonal_value_counts[df['MONTH'][i]] + 1 #calculate the average seasonal component for each month

for i in range(1, 13):

average_seasonal_values[i] = average_seasonal_values[i] / average_seasonal_value_counts[i] #create a new column in the data frame and fill it with the value of the average seasonal component for the corresponding month df['SEASONALITY'] = np.nan for i in range(0, df['SEASONALITY AND NOISE'].size):

if math.isnan(df['SEASONALITY AND NOISE'][i]) is False:

df['SEASONALITY'][i] =

average_seasonal_values[df['MONTH'][i]]

Let's plot this pure seasonal component:

#plot the seasonal component

fig = plt.figure() fig.suptitle('The \'pure\' SEASONAL component') plt.ylim(0, 1.3) df['SEASONALITY'].plot() plt.show()

(Image by Author)

STEP 5: Finally, we will divide the noisy seasonal value that we had isolated earlier with the averaged out seasonal value to yield just the noise component for each month.

noise component = noisy seasonal component / averaged out seasonal component

df['NOISE'] = df['SEASONALITY AND NOISE']/df['SEASONALITY'] #plot the seasonal component fig = plt.figure() fig.suptitle('The NOISE component') plt.ylim(0, 1.3) df['NOISE'].plot() plt.show()

(Image by Author)

So there you have it! We just hand cranked out the procedure for decomposing a time series into its trend, seasonal and noise components.

Here is a collage of the time series and its constituent components: